Latest version available here.
†Department of Economics, Boston University, email: celi@bu.edu
‡Microsoft Research, New England and the School of Engineering and Applied Sciences, Harvard University, email: tlin@g.harvard.edu
Introduction
An information designer (the designer; she) sends signals about an unknown state to steer the action of a receiver (he) in her favor. Designing the optimal signaling strategy requires the designer to know the receiver’s prior belief over states. Canonical economic models [e.g. 21 ] assume the parties share a common prior. In many applications, however, the designer has limited knowledge of the receiver’s belief. We study how she can learn that belief through repeated interactions and design approximately optimal signaling strategies when the receiver acts strategically.
For example, consider a large language model (LLM) API platform (the designer) that provides a business user (the receiver) with access to LLMs through Application Programming Interfaces (APIs). The LLM API platform charges the user based on the amount of tokens consumed to handle the user’s request. After the user submits a request, the platform has an internal evaluation about how likely the request will be successfully handled, for example, high confidence and low confidence (the state). Based on that internal assessment of the success probability, the platform then decides how to signal to the user, e.g., recommending the user to proceed for more details or admitting its inability to provide a reliable solution. Upon receiving the signal, the user updates his belief about the success probability of his request, and then takes an action to maximize his payoff, such as obeying the platform’s recommendation to proceed or stopping the interaction. The platform always prefers that the user proceed to gain payment from the user for the token consumption, regardless of the state. On the user’s side, his payoff is the highest when he pays for tokens that truly solve his request and the lowest when he pays for tokens that generate unwanted content, such as hallucinations.
The user has a subjective prior belief about the states, but the platform may not know that belief. As a result, the limited knowledge of the user’s belief fundamentally constrains the platform from designing the optimal signaling scheme. As implied by our previous work (Li and Lin, 2025), the platform may learn the user’s belief in the case where the user’s belief equals the true distribution through the user’s actions. However, the user may choose deceptive actions in some periods on purpose to manipulate the platform’s learning of his prior belief and further the platform’s design of the signaling scheme in favor of the user himself.
Motivated by such scenarios, we study an information design problem where the subjective prior belief of the receiver is unknown to the designer and the receiver is strategic.
Standard economic models, when tackling the problem of an unknown belief, would assume that the designer has a distribution over possible prior beliefs of the receiver [e.g. 1 , 22 ] . However, such an approach suffers from several concerns, such as computational hardness. A detailed discussion of the drawbacks of such an approach is presented in the literature review in detail.
Are there near-optimal signaling strategies for the designer when she does not know the subjective prior belief of a strategic receiver ? Our main contribution is to construct approximately optimal signaling schemes which enable the designer to learn that subjective prior from repeated interactions with a strategic receiver. Moreover, our designer achieves such approximate optimality at efficient speeds. In addition, the designer’s learning is robust to the receiver’s strategic manipulation. Finally, such fast learning speeds are achieved against both a static benchmark and a dynamic benchmark.
Specifically, we study a learning model where an information designer (e.g., the LLM platform) learns to design signaling schemes over T T T periods based on historical interactions with a long-lived receiver (e.g., a business user). The designer does not know the belief of the receiver. In each period, the designer chooses a signaling scheme, which maps a state i.i.d. sampled from a true distribution over the states to a randomized signal. Upon receiving the signal, the receiver updates his belief about the state and takes an action. The payoffs of the two players are then realized. We design learning algorithms for the designer to improve the signaling schemes over time, such that (1) the designer’s time-averaged payoff converges to optimum benchmarks and that (2) the convergence is fast , while (3) being robust to strategic manipulation by the receiver.
The rate at which the designer’s payoff converges to the optimal objective depends on the learning algorithm she employs. We measure the performance of the learning algorithm by a general notion of regret : the difference between the designer’s time-cumulative actual payoffs and an optimality benchmark, based on what the designer could achieve if she knew the receiver’s belief. We first characterize different regret benchmarks for the problem of an unknown prior belief in a learning model. The static benchmark is T T T times the single-period optimum for the designer had the designer known the receiver’s belief. The static regret is defined relative to the static benchmark. This allows us to relate our results to the results of the canonical information design models in Kamenica and Gentzkow [21 ] .
In addition, we consider a dynamic benchmark that measures the designer’s global dynamic optimality under a known belief, which is stronger than the static benchmark. The dynamic regret is then defined relative to the dynamic benchmark. A learning algorithm is called no-regret if its regret grows sublinearly with time (alternatively, its time-averaged regret converges to zero). The rate at which the time-averaged regret declines reflects the learning algorithm’s efficiency. We aim to design learning algorithms that not only achieve no regret against the two benchmarks, but also yield fast convergence to optimality under the unknown belief while being robust to the receiver’s strategic manipulation.
Our first main result is an O ( log 2 T ) O(\log^{2} T) O ( log 2 T ) upper bound on the static regret . Our second main result is also an O ( log 2 T ) O(\log^{2} T) O ( log 2 T ) regret upper bound on the dynamic regret . We design two learning algorithms to ensure that, for any problem instance, the designer’s regrets for T T T periods, relative to the static benchmark and the dynamic benchmark , respectively, are both no more than a constant times log 2 T \log^2 T log 2 T for all T T T . Both regret upper bounds imply convergence rates of O ( log 2 T T ) O\left(\frac{\log^2 T}{T}\right) O ( T l o g 2 T ) for the designer’s regret to converge to zero, which are significantly faster than other approaches in the literature, as discussed later.
Both of our two learning algorithms contain two phases. The first phase is the learning of the receiver’s belief, which has the same learning logic under the two regret benchmarks. The learning of the belief is based on binary search . The key idea is to learn the unknown belief from the receivers’ actions. To see the intuition, consider two states and two actions, where the receiver prefers different actions in the two states. For the moment, suppose the receiver is myopic in the sense that he does not consider the future in choosing his action. Then, the designer can compute a cutoff belief such that the receiver is indifferent between the two actions. The actual action chosen by the receiver then reveals whether the receiver’s belief is below or above the cutoff. By observing the receiver’s action, the designer gradually narrows down the range of possible values for the prior. That process does not stop until the designer obtains a sufficiently accurate estimate of the prior belief. Our formal analysis shows how to generalize that intuition to multiple states and multiple actions.
How can the designer learn the receiver’s belief when the receiver is not myopic? The algorithm limits the strategic manipulation from the receiver by delaying the effect of the receiver’s action on future signaling schemes. In particular, in each iteration, the algorithm records the receiver’s action when the designated signal first occurs. It then waits a fixed number of additional periods before using that recorded action to update the signaling scheme, ignoring actions taken during the waiting periods. Thus, the recorded action can influence future information only after a delay. Since the receiver discounts future payoffs, a sufficiently long delay makes the potential gain from manipulation small relative to any substantial immediate payoff loss. The receiver may still deviate from a myopic best response when the immediate loss is small, but the resulting error in the belief estimate is small. By choosing the delay appropriately, the designer can therefore learn the receiver’s belief to the desired accuracy despite the receiver’s strategic manipulation.
The second phase, where our learning algorithms for the two benchmarks differ, is robustification . The signaling scheme that is optimal for the designer under the belief estimate is converted into another signaling scheme that is approximately optimal under the receiver’s belief. The motivation is that even if two prior beliefs are very close, the signaling scheme optimal under one belief may perform arbitrarily badly under the other, leaving no guarantee on the designer’s payoff. However, the robustification varies across benchmarks. Our first learning algorithm uses static robustification, where the robustification process is performed on the signaling scheme optimal for the belief estimate once to obtain a robustified signaling scheme for being repeatedly used for future periods.
Our second learning algorithm applies a dynamic robustification to the designer’s globally optimal strategy, where the signaling schemes in different periods are chosen so that the receiver at any period is still incentivized to participate in the future. The challenge is that the static robustification may backfire: the static robustification only increases the persuasiveness of the signaling scheme, namely, increasing the positive gap of the expected payoff from the recommended action relative to that from any other actions. However, the static robustification may lower the receiver’s ex ante (before a recommendation is sent) expected payoff from obeying the recommendations in the future. Thus, after an action is recommended by a statically robustified signaling scheme, the receiver’s incentive to obey that current recommendation becomes stronger, but his ex ante payoff from obeying recommendations in future periods may get worse, potentially violating the receiver’s participation constraint.
To overcome this, we propose payoff-improving robustification , which weakly increases the receiver’s ex ante payoff from being obedient. We then take the designer’s optimal dynamic signaling strategy, given her belief estimate, and apply the payoff-improving robustification procedure to each period’s signaling scheme in that dynamic strategy. We show that the resulting dynamic strategy is persuasive for the receiver and approximately optimal for the designer.
The efficacy of both of our learning algorithms is reflected by the logarithmic upper bounds on their regrets: O ( log 2 T ) O(\log^{2} T) O ( log 2 T ) . That means that the designer’s average payoff converges to the optimality benchmarks at a fast speed, in the order of O ( log 2 T T ) O\left(\frac{\log^{2} T}{T}\right) O ( T l o g 2 T ) . In contrast, an intuitive method is to use an empirical estimation : using the empirical frequency to estimate the state distribution. Empirical estimation does not work in our problem as the receiver’s belief is subjective and may not equal the true distribution over states. Even if empirical estimation could be applied, it would suffer a regret of at least O ( T ) O(\sqrt{T}) O ( T ) due to sampling error, which is exponentially worse than the O ( log 2 T ) O(\log^{2} T) O ( log 2 T ) regret bound of our learning algorithms. Our algorithms circumvent the limitation of empirical estimation and achieve an exponential improvement by actively learning the receiver’s belief from his actions.
Related Work
First, this paper contributes to the literature on robust information design under uncertainty, especially when the designer lacks knowledge of the receiver’s prior belief. Classical models such as Bayesian persuasion [21 , 5 ] and cheap talk [10 ] typically assume a common prior shared by the designer and the receiver. Different from the canonical models, Alonso and Câmara [1 ] study persuasion with heterogeneous priors. Recent work relaxes this by introducing uncertainty over the prior. One approach assumes the designer holds a belief over the unknown prior (e.g., Alonso and Câmara [1 ] ), but computing the optimal signaling scheme is generally intractable [19 ] . Another approach, rooted in robust mechanism design, has the designer optimize against a worst-case prior in a set of possible priors. Kosterina [23 ] studies a designer who chooses a signaling scheme to maximize her payoff under the least favorable receiver prior in a given set. Hu and Weng (2021) and Dworczak and Pavan [11 ] consider uncertainty about the receiver’s private information or additional information sources. These papers typically study a single-shot game, limiting the designer’s ability to learn over time. Of course, when the designer has the opportunity to learn, one expects she may achieve a much higher payoff. In contrast, we design learning algorithms that infer the prior, achieving near optimality quickly while avoiding computational hardness.
Departing from the standard approach for addressing the robustness, Li and Lin (2025) study how to learn the belief of a sequence of short-lived receivers from their actions. However, if a receiver interacts with the platform repeatedly, thus becoming long-lived, then he will be incentivized to be strategic. Thus, he may take actions to convey deceptive information about his belief, manipulating the platform’s learning in a direction favorable to them but unfavorable to the platform.
Our work also relates to dynamic information provision. Ball [4 ] consider a dynamic information design problem in which the beliefs of the designer and the receiver are different. However, their model uses persistent states while we consider i.i.d. states. More importantly, they consider both players to be equally patient, while we consider a patient designer and an impatient receiver. The different patience levels in our paper have crucial influence on their dynamic strategies and the designer’s regret benchmarks. Besides, they assume more structure on the dynamic process and the players’ payoffs, while our work keep the dynamic process and the payoff functions general without further structure.
Our work also contributes to the literature on online Bayesian persuasion, which studies learning when some game parameters are unknown. Castiglioni et al. [8 ] , Castiglioni et al. [9 ] , and Feng, Tang, and Xu [13 ] consider unknown receiver utilities; Zu, Iyer, and Xu (2021) and Wu et al. (2022) focus on unknown priors; and Bacchiocchi et al. [3 ] allow both utilities and priors to be unknown. Previous approaches to unknown priors rely on empirical estimation and incur Ω ( T ) \Omega(\sqrt{T}) Ω ( T ) regret due to sampling error. This option is not available here as the designer needs to learn the receiver’s belief, not the true distribution. We bypass this by using best responses to infer the prior more efficiently, achieving O ( log 2 T ) O(\log^{2} T) O ( log 2 T ) regret. Camara, Hartline, and Johnsen [7 ] also study information design with learning agents but drop distributional assumptions and allow adversarial state sequences; their regret notion therefore differs from ours, which assumes i.i.d. states from an unknown objective prior.
Our work also connects to the line of works on misspecified learning (Ba, 2023; [6 ] ; [14 ] ; [12 ] ; Fudenberg, Lanzani, and Strack, 2021; Fudenberg, Romanyuk, and Strack, 2017) and on the long-run outcomes of misspecified Bayesian agents who cannot obtain the true distribution over states (e.g., [27 , 26 , 15 ] (), and He and Libgober 2021)). Those works study belief convergence and actions in general learning environments. One can view our model as studying a receiver whose belief is the limiting outcome of his misspecified learning. We do not analyze the process by which this belief is formed, focusing instead on how the designer interacts with a receiver whose belief has already converged.
Models and Prelimaries
Models
Single-period game. We first define a single-period game of information design (Bayesian persuasion) with subjective prior beliefs. There is a finite set of states of the world Ω \Omega Ω . The information designer and the receiver have subjective prior beliefs μ D , μ R ∈ Δ ( Ω ) \mu_D, \mu_R \in\Delta(\Omega) μ D , μ R ∈ Δ ( Ω ) over the states, respectively, where Δ ( Ω ) \Delta(\Omega) Δ ( Ω ) denotes the set of probability distributions over Ω \Omega Ω . We think of μ D \mu_D μ D as the true distribution over the states and μ R \mu_R μ R as a subjective prior belief over the states, which may not be the same as the true distribution. The designer is better informed about the true distribution from access to sufficient state observation data, so the designer is assumed to know the true distribution. The receiver’s prior μ R \mu_R μ R can be different from the true distribution, which may be the convergence outcome of the learning process of a misspecified belief (e.g., [17 ] ).
The receiver has a finite set of actions A A A . The designer and the receiver have payoff functions u , v : A × Ω → [ 0 , 1 ] u, v : A \times\Omega\to[0,1] u , v : A × Ω → [ 0 , 1 ] , respectively. This single-period game is summarized by an instance I = ( Ω , A , u , v , μ D , μ R ) \mathcal{I} = (\Omega, A, u, v, \mu_D, \mu_R) I = ( Ω , A , u , v , μ D , μ R ) .
At the beginning of the game, the designer announces a signaling scheme π : Ω → Δ ( S ) \pi: \Omega\to\Delta(S) π : Ω → Δ ( S ) , which maps each state to a probability distribution over a finite set of signals S S S . Let Π = Ω → Δ ( S ) \Pi= \Omega\to\Delta(S) Π = Ω → Δ ( S ) be the set of signaling schemes. The receiver believes that the state is drawn according to the distribution μ R \mu_R μ R . Thus, after receiving a signal s ∼ π ( ⋅ ∣ ω ) s \sim\pi(\cdot|\omega) s ∼ π ( ⋅ ∣ ω ) sampled according to state ω \omega ω and the signaling scheme π \pi π , the receiver obtains his posterior belief μ R ( ω ) π ( s ∣ ω ) ∑ ω ′ ∈ Ω μ R ( ω ′ ) π ( s ∣ ω ′ ) \frac{\mu_R(\omega)\pi(s|\omega)}{\sum_{\omega'\in\Omega}\mu_R(\omega')\pi(s|\omega')} ∑ ω ′ ∈ Ω μ R ( ω ′ ) π ( s ∣ ω ′ ) μ R ( ω ) π ( s ∣ ω ) , ∀ ω ∈ Ω \forall\omega\in\Omega ∀ ω ∈ Ω , and takes an action in response. Let ρ : S → Δ ( A ) \rho: S \to\Delta(A) ρ : S → Δ ( A ) be a possibly randomized single-period strategy of the receiver. Under a pair of strategies ( π , ρ ) (\pi,\rho) ( π , ρ ) , the expected payoffs of the designer and the receiver in in the single-period game are, respectively,
U ( π , ρ ; μ D ) = ∑ ω ∈ Ω μ D ( ω ) ∑ s ∈ S π ( s ∣ ω ) ∑ a ∈ A ρ ( a ∣ s ) u ( a , ω ) . (1) U(\pi,\rho;\mu_D)=\sum_{\omega\in\Omega}\mu_D(\omega)\sum_{s\in S}\pi(s\mid\omega)\sum_{a\in A}\rho(a\mid s)u(a,\omega). \tag*{(1)} U ( π , ρ ; μ D ) = ω ∈ Ω ∑ μ D ( ω ) s ∈ S ∑ π ( s ∣ ω ) a ∈ A ∑ ρ ( a ∣ s ) u ( a , ω ) . (1)
V ( π , ρ ; μ R ) = ∑ ω ∈ Ω μ R ( ω ) ∑ s ∈ S π ( s ∣ ω ) ∑ a ∈ A ρ ( a ∣ s ) v ( a , ω ) . (2) V(\pi,\rho;\mu_R)=\sum_{\omega\in\Omega}\mu_R(\omega)\sum_{s\in S}\pi(s\mid\omega)\sum_{a\in A}\rho(a\mid s)v(a,\omega). \tag*{(2)} V ( π , ρ ; μ R ) = ω ∈ Ω ∑ μ R ( ω ) s ∈ S ∑ π ( s ∣ ω ) a ∈ A ∑ ρ ( a ∣ s ) v ( a , ω ) . (2)
We say the receiver myopically best responds 2 if he maximizes his single-period payoff by choosing a strategy
ρ ∗ ∈ arg max ρ : S → Δ ( A ) V ( π , ρ ) \rho^* \in\underset{\rho:S\to\Delta(A)}{\arg\max}\ V(\pi,\rho) ρ ∗ ∈ ρ : S → Δ ( A ) arg max V ( π , ρ )
or, equivalently, an action ρ ∗ ( s ) ∈ arg max a ∈ A μ R ( ω ) π ( s ∣ ω ) v ( a , ω ) \rho^*(s)\in\arg\max_{a\in A}\mu_R(\omega)\pi(s\mid\omega)v(a,\omega) ρ ∗ ( s ) ∈ arg max a ∈ A μ R ( ω ) π ( s ∣ ω ) v ( a , ω ) given each signal s ∈ S s\in S s ∈ S based on his posterior belief. With a myopically best-responding receiver, the optimal utility of the designer is
U B P ∗ ( I ) : = max π ∈ Π max ρ ∈ arg max s → Δ ( A ) V ( π , ρ ; μ R ) U ( π , ρ ; μ D ) . (3) U^*_{\mathrm{BP}}(\mathcal{I}) := \max_{\pi\in\Pi}\ \max_{\rho\in\arg\max_{s\to\Delta(A)} V(\pi,\rho;\mu_R)} U(\pi,\rho;\mu_D). \tag*{(3)} U BP ∗ ( I ) := π ∈ Π max ρ ∈ a r g m a x s → Δ ( A ) V ( π , ρ ; μ R ) max U ( π , ρ ; μ D ) . (3)
We call U B P ∗ ( I ) U^*_{\mathrm{BP}}(\mathcal{I}) U BP ∗ ( I ) the Bayesian persuasion benchmark .
T T T -period learning process. We consider the T T T -period repetitions of the above single-period game with an instance I = ( Ω , A , u , v , μ D , μ R ) \mathcal{I}=(\Omega,A,u,v,\mu_D,\mu_R) I = ( Ω , A , u , v , μ D , μ R ) . Importantly, the designer now does not know the receiver’s prior belief μ R \mu_R μ R but she knows other parameters u , v , μ D u,v,\mu_D u , v , μ D . In particular, the designer learns to design signaling schemes using an algorithm G \mathcal{G} G which, at every period t t t , maps the history H ( t − 1 ) ∈ Π ( t − 1 ) × S ( t − 1 ) × A ( t − 1 ) H^{(t-1)}\in\Pi^{(t-1)}\times S^{(t-1)}\times A^{(t-1)} H ( t − 1 ) ∈ Π ( t − 1 ) × S ( t − 1 ) × A ( t − 1 ) to a signaling scheme π ( t ) ∈ Π \pi^{(t)}\in\Pi π ( t ) ∈ Π for the current period. In response to the learning algorithm G \mathcal{G} G , the receiver chooses a T T T -period strategy ϕ = ( ϕ ( t ) ) t = 1 T \phi=(\phi^{(t)})_{t=1}^{T} ϕ = ( ϕ ( t ) ) t = 1 T where each ϕ ( t ) \phi^{(t)} ϕ ( t ) maps the history H ( t − 1 ) ∈ Π ( t − 1 ) × S ( t − 1 ) × A ( t − 1 ) H^{(t-1)}\in\Pi^{(t-1)}\times S^{(t-1)}\times A^{(t-1)} H ( t − 1 ) ∈ Π ( t − 1 ) × S ( t − 1 ) × A ( t − 1 ) and the current period’s signaling scheme π ( t ) ∈ Π \pi^{(t)}\in\Pi π ( t ) ∈ Π to a strategy ρ ( t ) : S → Δ ( A ) \rho^{(t)}:S\to\Delta(A) ρ ( t ) : S → Δ ( A ) that specifies a randomized action for each possible signal. Then, the pair of strategies ( G , ϕ ) (\mathcal{G},\phi) ( G , ϕ ) will generate a stochastic sequence of ( π ( t ) , ρ ( t ) ) t = 1 T (\pi^{(t)},\rho^{(t)})_{t=1}^{T} ( π ( t ) , ρ ( t ) ) t = 1 T of the designer’s signaling schemes and the receiver’s single-period strategies. We assume that the receiver is impatient in the sense that he has a discount factor γ ∈ ( 0 , 1 ) \gamma\in(0,1) γ ∈ ( 0 , 1 ) , so his total discounted expected payoff is
V T ( G , ϕ ; μ R ) = E ( π ( t ) , ρ ( t ) ) t = 1 T ∼ ( G , ϕ ) [ ∑ t = 1 T γ t ⋅ V ( π ( t ) , ρ ( t ) ; μ R ) ] . (4) V_T(\mathcal{G},\phi;\mu_R)=\mathbb{E}_{(\pi^{(t)},\rho^{(t)})_{t=1}^{T}\sim(\mathcal{G},\phi)}\left[\sum_{t=1}^{T}\gamma^t\cdot V(\pi^{(t)},\rho^{(t)};\mu_R)\right]. \tag*{(4)} V T ( G , ϕ ; μ R ) = E ( π ( t ) , ρ ( t ) ) t = 1 T ∼ ( G , ϕ ) [ t = 1 ∑ T γ t ⋅ V ( π ( t ) , ρ ( t ) ; μ R ) ] . (4)
The designer, on the other hand, is patient , that is, having a discount factor of one. The designer aims to maximize her total expected payoff
U T ( G , ϕ ; μ D ) = E ( π ( t ) , ρ ( t ) ) t = 1 T ∼ ( G , ϕ ) [ ∑ t = 1 T U ( π ( t ) , ρ ( t ) ; μ D ) ] . (5) U_T(\mathcal{G}, \phi; \mu_D) = \mathbb{E}_{(\pi^{(t)},\rho^{(t)})_{t=1}^{T}\sim(\mathcal{G},\phi)}\left[\sum_{t=1}^{T} U(\pi^{(t)},\rho^{(t)};\mu_D)\right]. \tag*{(5)} U T ( G , ϕ ; μ D ) = E ( π ( t ) , ρ ( t ) ) t = 1 T ∼ ( G , ϕ ) [ t = 1 ∑ T U ( π ( t ) , ρ ( t ) ; μ D ) ] . (5)
We consider a strategic (or long-lived) receiver who best responds to the designer’s learning algorithm G \mathcal{G} G . Formally, given the algorithm G \mathcal{G} G and the belief μ R \mu_R μ R , the receiver chooses a globally optimal strategy
ϕ ∗ ( G , μ R ) = arg max ϕ V T ( G , ϕ ; μ R ) . \phi^*(\mathcal{G},\mu_R) = \underset{\phi}{\arg\max}\ V_T(\mathcal{G},\phi;\mu_R). ϕ ∗ ( G , μ R ) = ϕ arg max V T ( G , ϕ ; μ R ) .
Then, the designer obtains her total expected payoff
U T ( G , ϕ ∗ ( G , μ R ) ; μ D ) . U_T(\mathcal{G},\phi^*(\mathcal{G},\mu_R);\mu_D). U T ( G , ϕ ∗ ( G , μ R ) ; μ D ) .
Benchmarks and regrets. The regret of the designer under instance I = { Ω , A , u , v , μ D , μ R } \mathcal{I} = \{\Omega, A, u, v, \mu_D, \mu_R\} I = { Ω , A , u , v , μ D , μ R } is the difference between a benchmark U T ∗ ( I ) U_T^*(\mathcal{I}) U T ∗ ( I ) and the actual payoff U T U_T U T that the designer obtains by using the algorithm G \mathcal{G} G over the T T T periods. In particular, we will consider two benchmarks for defining regret.
The first benchmark is the static benchmark denoted by U B P ∗ ( I ) U_{\mathrm{BP}}^*(\mathcal{I}) U BP ∗ ( I ) , which is T T T times the single-period Bayesian persuasion benchmark at a problem instance I \mathcal{I} I .
U T ∗ ( I ) : = T ⋅ U B P ∗ ( I ) . U_T^*(\mathcal{I}) := T \cdot U_{\mathrm{BP}}^*(\mathcal{I}). U T ∗ ( I ) := T ⋅ U BP ∗ ( I ) .
The static benchmark is a natural benchmark to consider, especially for measuring how good the performance of our learning algorithm is compared to the Bayesian persuasion optimality in the canonical work [21 ] , in which the designer knows the receiver’s prior μ R \mu_R μ R .
We call the regret against the static benchmark static regret :
Reg s t a t i c ( T ; G , I ) = U T ∗ ( I ) − U T ( G , ϕ ∗ ( G , μ R ) ; μ D ) . (6) \operatorname{Reg}^{\mathrm{static}}(T;\mathcal{G},\mathcal{I}) = U_T^*(\mathcal{I}) - U_T(\mathcal{G},\phi^*(\mathcal{G},\mu_R);\mu_D). \tag*{(6)} Reg static ( T ; G , I ) = U T ∗ ( I ) − U T ( G , ϕ ∗ ( G , μ R ) ; μ D ) . (6)
Regularity Assumptions
The second benchmark, the dynamic benchmark , is the maximal T T T -period payoff the designer could obtain had the designer known the receiver’s belief μ R \mu_R μ R and the receiver best responded over the T T T periods. We denote this by U T ∗ ∗ ( I ) U_T^{**}(\mathcal{I}) U T ∗∗ ( I ) at a problem instance I \mathcal{I} I .
U T ∗ ∗ ( I ) : = max T -period strategy σ U T ( σ , ϕ ∗ ( σ , μ R ) ; μ D ) , U_T^{**}({\mathcal{I}}) := \max_{\substack{T\text{-period strategy }\sigma}} U_T(\sigma,\phi^*(\sigma,\mu_R);\mu_D), U T ∗∗ ( I ) := T -period strategy σ max U T ( σ , ϕ ∗ ( σ , μ R ) ; μ D ) ,
where σ \sigma σ is a T T T -period strategy of the designer that designs signaling schemes based on historical information and the receiver’s belief μ R \mu_R μ R .3
We call the regret against the dynamic benchmark dynamic regret :
Reg d y n a m i c ( T ; G , I ) = U T ∗ ∗ ( I ) − U T ( G , ϕ ∗ ( G , μ R ) ; μ D ) . (7) \operatorname{Reg}^{\mathrm{dynamic}}(T;\mathcal{G},\mathcal{I}) = U_T^{**}(\mathcal{I}) - U_T(\mathcal{G},\phi^*(\mathcal{G},\mu_R);\mu_D). \tag*{(7)} Reg dynamic ( T ; G , I ) = U T ∗∗ ( I ) − U T ( G , ϕ ∗ ( G , μ R ) ; μ D ) . (7)
Not surprisingly, as we will show in § 5 , the dynamic benchmark U T ∗ ∗ ( I ) U_T^{**}(\mathcal{I}) U T ∗∗ ( I ) is always weakly larger than the static benchmark U T ∗ ( I ) U_T^*(\mathcal{I}) U T ∗ ( I ) and can be strictly larger. Consequently, any upper bound on dynamic regret also bounds static regret. We remark that our results show the designer can approach this stronger benchmark at the same asymptotic rate. In particular, we construct algorithms that achieve O ( log 2 T ) O(\log^2 T) O ( log 2 T ) regret against the static and dynamic benchmarks, respectively. Thus, in both cases, the designer’s average expected payoff falls below the corresponding per-period benchmark by at most O ( log 2 T / T ) O(\log^2 T/T) O ( log 2 T / T ) . Learning to attain the stronger dynamic benchmark therefore does not require a slower rate of convergence.
A regret upper bound is a performance guarantee of an algorithm G \mathcal{G} G in the following sense: for any problem instance I \mathcal{I} I , that is, for any unknown belief μ R \mu_R μ R , the regret incurred by the algorithm G \mathcal{G} G grows no faster than some function of T T T , i.e.,
Reg ( T ; G , I ) ≤ C I ⋅ f ( T ) = O ( f ( T ) ) , ∀ T , ∀ I , \operatorname{Reg}(T;\mathcal{G},\mathcal{I}) \le C_{\mathcal{I}}\cdot f(T) = O(f(T)), \qquad\forall T,\ \forall\mathcal{I}, Reg ( T ; G , I ) ≤ C I ⋅ f ( T ) = O ( f ( T )) , ∀ T , ∀ I ,
where C I C_{\mathcal{I}} C I is a constant depending on I \mathcal{I} I but not on T T T . Here, f ( T ) f(T) f ( T ) represents the worst-case rate at which the regret can grow. An algorithm G \mathcal{G} G with regret upper bound O ( f ( T ) ) O(f(T)) O ( f ( T )) is no-regret if f ( T ) f(T) f ( T ) grows at a rate that is sublinear with T T T , that is,
lim T → ∞ f ( T ) T → 0. \lim_{T\to\infty}\frac{f(T)}{T}\to0. T → ∞ lim T f ( T ) → 0.
For example, the worst-case rate f ( T ) f(T) f ( T ) can be f ( T ) = T f(T)=\sqrt{T} f ( T ) = T or f ( T ) = log T f(T)=\log T f ( T ) = log T .
Regularity Assumptions
We present two regularity assumptions used throughout the paper. First, we assume that the beliefs μ D , μ R \mu_D, \mu_R μ D , μ R have full support and the designer knows the minimum probability p 0 > 0 p_0 > 0 p 0 > 0 of any state. This probability p 0 p_0 p 0 will be used by the designer for designing the algorithms later.
Assumption 1 (Known lower bound on prior). Both players’ prior beliefs μ D , μ R ∈ Δ ( Ω ) \mu_D, \mu_R \in\Delta(\Omega) μ D , μ R ∈ Δ ( Ω ) have full support and there exists p 0 ∈ ( 0 , 1 ) p_0 \in(0,1) p 0 ∈ ( 0 , 1 ) such that the designer knows min ω ∈ Ω μ R ( ω ) ≥ p 0 > 0 \min_{\omega\in\Omega} \mu_R(\omega) \ge p_0 > 0 min ω ∈ Ω μ R ( ω ) ≥ p 0 > 0 .
Assumption 2 (Regularity of receiver’s utility). There is no weakly dominated action for the receiver.
Assumption 2 implies that there exists a positive number D > 0 D > 0 D > 0 such that, for every action a ∈ A a \in A a ∈ A of the receiver, there exists a belief η a ∈ Δ ( Ω ) \eta_a \in\Delta(\Omega) η a ∈ Δ ( Ω ) on which action a a a is strictly better than any other action by a margin of D D D :
E ω ∼ η a [ v ( a , ω ) ] ≥ E ω ∼ η a [ v ( a ′ , ω ) ] + D , ∀ a ′ ∈ A ∖ { a } . (8) \mathbb{E}_{\omega\sim\eta_a}[v(a,\omega)] \ge\mathbb{E}_{\omega\sim\eta_a}[v(a',\omega)] + D,\quad\forall a' \in A \setminus\{a\}. \tag*{(8)} E ω ∼ η a [ v ( a , ω )] ≥ E ω ∼ η a [ v ( a ′ , ω )] + D , ∀ a ′ ∈ A ∖ { a } . (8)
The designer knows D D D since she knows the receiver’s utility function v v v .
Denote by a ω ∗ = arg max a ∈ A v ( a , ω ) a^*_\omega= \arg\max_{a \in A} v(a,\omega) a ω ∗ = arg max a ∈ A v ( a , ω ) the receiver’s optimal action in state ω ∈ Ω \omega\in\Omega ω ∈ Ω . We impose two further assumptions on the receiver’s utility function.
Assumption 3 (Unique optimal action). The receiver has a unique optimal action a ω ∗ a^*_\omega a ω ∗ in each state ω ∈ Ω \omega\in\Omega ω ∈ Ω .
Since the state and action sets are finite, Assumption 3 immediately implies that there exists a constant G > 0 G > 0 G > 0 such that
∀ ω ∈ Ω , ∀ a ∈ A ∖ { a ω ∗ } , v ( a ω ∗ , ω ) − v ( a , ω ) > G > 0. \forall\omega\in\Omega,\ \forall a \in A \setminus\{a^*_\omega\},\ v(a^*_\omega,\omega) - v(a,\omega) > G > 0. ∀ ω ∈ Ω , ∀ a ∈ A ∖ { a ω ∗ } , v ( a ω ∗ , ω ) − v ( a , ω ) > G > 0.
Since the designer knows the utility functions, she knows this constant G G G .
We call two states distinguishable if their receiver-optimal actions differ.
Assumption 4 (Distinguishable states). There exist at least one pair of states ω i , ω j ∈ Ω \omega_i,\omega_j \in\Omega ω i , ω j ∈ Ω whose corresponding receiver-optimal actions are different: a ω i ∗ ≠ a ω j ∗ a^*_{\omega_i} \ne a^*_{\omega_j} a ω i ∗ = a ω j ∗ .
Assumption 4 makes the designer’s learning problem non-trivial. If Assumption 4 does not hold, then the receiver would have the same optimal action a ∗ a^* a ∗ across all states regardless of the signal sent. The designer then cannot influence the receiver’s action through signals. The expected payoff of the designer would be a constant, leaving the designer zero regret trivially from not knowing the receiver’s prior.
Learning to Achieve No Regret Against the Static Benchmark
We will design a learning algorithm for the designer to achieve a regret upper bounded by O ( log 2 T ) O(\log^2 T) O ( log 2 T ) against the static benchmark. The learning algorithm contains two parts. The first is the learning. The second is a static robustification process for obtaining a near-optimal signaling scheme from the learned belief.
Efficient Learning of the Strategic Receiver’s Belief
The designer learns the receiver’s prior belief μ R \mu_R μ R by observing his actions under suitably chosen signaling schemes. Our approach is to learn the prior belief ratio μ R ( ω i ) / μ R ( ω j ) \mu_R(\omega_i)/\mu_R(\omega_j) μ R ( ω i ) / μ R ( ω j ) for any pair of states to recover the receiver’s prior belief μ R \mu_R μ R . We first consider pairs of distinguishable states, for which the receiver’s optimal actions differ, and develop a binary search procedure for learning their ratios under the prior belief. We show that how delaying the effect of the receiver’s actions on future signaling schemes limits his incentive to manipulate the designer’s learning of his prior. We then extend the procedure to arbitrary pairs of states and construct an estimate of the receiver’s prior belief μ R \mu_R μ R .
Algorithm 1 shows how to estimate the receiver’s prior belief ratio μ R ( ω i ) μ R ( ω j ) \frac{\mu_R(\omega_i)}{\mu_R(\omega_j)} μ R ( ω j ) μ R ( ω i ) for any pair of distinguishable states ω 1 \omega_1 ω 1 and ω 2 \omega_2 ω 2 based on binary search . The designer uses a signaling scheme π \pi π that sends a certain signal s 0 s_0 s 0 only under states ω i \omega_i ω i and ω j \omega_j ω j , that is, π ( s 0 ∣ ω ) > 0 \pi(s_0 \mid\omega)>0 π ( s 0 ∣ ω ) > 0 for ω = ω i , ω j \omega=\omega_i,\omega_j ω = ω i , ω j and π ( s 0 ∣ ω ) = 0 \pi(s_0 \mid\omega)=0 π ( s 0 ∣ ω ) = 0 for ω ≠ ω i , ω j \omega\ne\omega_i,\omega_j ω = ω i , ω j . Then the receiver will believe that the state must be either ω i \omega_i ω i or ω j \omega_j ω j upon receiving signal s 0 s_0 s 0 . In particular, the ratio of the receiver’s posterior belief about those two states is
μ ( ω j ∣ s 0 ) μ ( ω i ∣ s 0 ) = μ R ( ω j ) μ R ( ω i ) π ( s 0 ∣ ω j ) π ( s 0 ∣ ω i ) , (9) \frac{\mu(\omega_j\mid s_0)}{\mu(\omega_i\mid s_0)}
=
\frac{\mu_R(\omega_j)}{\mu_R(\omega_i)}
\frac{\pi(s_0\mid\omega_j)}{\pi(s_0\mid\omega_i)}, \tag*{(9)} μ ( ω i ∣ s 0 ) μ ( ω j ∣ s 0 ) = μ R ( ω i ) μ R ( ω j ) π ( s 0 ∣ ω i ) π ( s 0 ∣ ω j ) , (9)
which depends on the receiver’s prior belief μ R \mu_R μ R and the signaling scheme π \pi π used by the designer. The receiver takes an action based on this posterior belief to maximize his total payoff from the current and future periods. Based on the receiver’s action, the designer then updates the signaling scheme for the next period. That process iterates until the designer obtains an estimate of the receiver’s prior belief ratio μ R ( ω i ) μ R ( ω j ) \frac{\mu_R(\omega_i)}{\mu_R(\omega_j)} μ R ( ω j ) μ R ( ω i ) with a desired accuracy.
To learn the belief ratio of a strategic receiver, it is useful to first understand how to learn that ratio if the receiver best responds to the signaling scheme π \pi π myopically , that is, if the receiver maximizes his payoff only in the current period. If the receiver’s prior belief ratio μ R ( ω j ) μ R ( ω i ) \frac{\mu_R(\omega_j)}{\mu_R(\omega_i)} μ R ( ω i ) μ R ( ω j ) is sufficiently small, then his posterior belief ratio (9) will also be small. Thus, the receiver will believe that the state is ω i \omega_i ω i with high probability and take the corresponding optimal action a ω i ∗ a^*_{\omega_i} a ω i ∗ . On the other hand, if the receiver’s prior belief ratio μ R ( ω j ) μ R ( ω i ) \frac{\mu_R(\omega_j)}{\mu_R(\omega_i)} μ R ( ω i ) μ R ( ω j ) is very large, then he will believe that the state is ω j \omega_j ω j with high probability and take the corresponding optimal action a ω j ∗ a_{\omega_j}^* a ω j ∗ . In between, there is a range for the ratio μ R ( ω j ) μ R ( ω i ) \frac{\mu_R(\omega_j)}{\mu_R(\omega_i)} μ R ( ω i ) μ R ( ω j ) , where the receiver might take neither action but a completely different action.
Therefore, given the signaling scheme π \pi π and a pair of distinguishable states, there exists a cutoff c ( π , ω i , ω j ) c(\pi,\omega_i,\omega_j) c ( π , ω i , ω j ) on the receiver’s prior belief ratio such that the receiver is indifferent in taking action a ω i ∗ a_{\omega_i}^* a ω i ∗ versus some other action a ~ ω j , ω i \tilde{a}_{\omega_j,\omega_i} a ~ ω j , ω i , which might or might not be a ω j ∗ a_{\omega_j}^* a ω j ∗ . Thus, the receiver takes action a ~ ω j , ω i \tilde{a}_{\omega_j,\omega_i} a ~ ω j , ω i when his prior belief ratio μ R ( ω j ) μ R ( ω i ) \frac{\mu_R(\omega_j)}{\mu_R(\omega_i)} μ R ( ω i ) μ R ( ω j ) is above the cutoff c ( π , ω i , ω j ) c(\pi,\omega_i,\omega_j) c ( π , ω i , ω j ) . As explained above, a ~ ω j , ω i \tilde{a}_{\omega_j,\omega_i} a ~ ω j , ω i is not necessarily a ω j ∗ a_{\omega_j}^* a ω j ∗ . We call such an action a ~ ω j , ω i \tilde{a}_{\omega_j,\omega_i} a ~ ω j , ω i the first indifferent action for the receiver under the signaling scheme π \pi π and a pair of distinguishable states ω i , ω j \omega_i,\omega_j ω i , ω j . The cutoff c ( π , ω i , ω j ) c(\pi,\omega_i,\omega_j) c ( π , ω i , ω j ) is the prior belief ratio μ ( ω j ) μ ( ω i ) \frac{\mu(\omega_j)}{\mu(\omega_i)} μ ( ω i ) μ ( ω j ) that solves the following indifference equation under the experimented signaling scheme π \pi π :
μ ( ω j ) π ( s 0 ∣ ω j ) ( v ( a ω i ∗ , ω j ) − v ( a ~ ω j , ω i , ω j ) ) + μ ( ω i ) π ( s 0 ∣ ω i ) ( v ( a ω i ∗ , ω i ) − v ( a ~ ω j , ω i , ω i ) ) = 0 \mu(\omega_j)\pi(s_0\mid\omega_j)\left(v(a_{\omega_i}^*,\omega_j)-v(\tilde{a}_{\omega_j,\omega_i},\omega_j)\right)+\mu(\omega_i)\pi(s_0\mid\omega_i)\left(v(a_{\omega_i}^*,\omega_i)-v(\tilde{a}_{\omega_j,\omega_i},\omega_i)\right)=0 μ ( ω j ) π ( s 0 ∣ ω j ) ( v ( a ω i ∗ , ω j ) − v ( a ~ ω j , ω i , ω j ) ) + μ ( ω i ) π ( s 0 ∣ ω i ) ( v ( a ω i ∗ , ω i ) − v ( a ~ ω j , ω i , ω i ) ) = 0
⟹ c ( π , ω i , ω j ) = μ ( ω j ) μ ( ω i ) = π ( s 0 ∣ ω i ) π ( s 0 ∣ ω j ) v ( a ω i ∗ , ω i ) − v ( a ~ ω j , ω i , ω i ) v ( a ~ ω j , ω i , ω j ) − v ( a ω i ∗ , ω j ) . \Longrightarrow\quad c(\pi,\omega_i,\omega_j)=\frac{\mu(\omega_j)}{\mu(\omega_i)}=\frac{\pi(s_0\mid\omega_i)}{\pi(s_0\mid\omega_j)}\frac{v(a_{\omega_i}^*,\omega_i)-v(\tilde{a}_{\omega_j,\omega_i},\omega_i)}{v(\tilde{a}_{\omega_j,\omega_i},\omega_j)-v(a_{\omega_i}^*,\omega_j)}. ⟹ c ( π , ω i , ω j ) = μ ( ω i ) μ ( ω j ) = π ( s 0 ∣ ω j ) π ( s 0 ∣ ω i ) v ( a ~ ω j , ω i , ω j ) − v ( a ω i ∗ , ω j ) v ( a ω i ∗ , ω i ) − v ( a ~ ω j , ω i , ω i ) .
The first indifferent action a ~ ω j , ω i \tilde{a}_{\omega_j,\omega_i} a ~ ω j , ω i is formally defined as follows. Define by ζ ω j , ω i a : = v ( a ω i ∗ , ω i ) − v ( a , ω i ) v ( a , ω j ) − v ( a ω i ∗ , ω j ) \zeta_{\omega_j,\omega_i}^{a}:=\frac{v(a_{\omega_i}^*,\omega_i)-v(a,\omega_i)}{v(a,\omega_j)-v(a_{\omega_i}^*,\omega_j)} ζ ω j , ω i a := v ( a , ω j ) − v ( a ω i ∗ , ω j ) v ( a ω i ∗ , ω i ) − v ( a , ω i ) the value for the ratio μ ( ω j ) π ( s 0 ∣ ω j ) μ ( ω i ) π ( s 0 ∣ ω i ) \frac{\mu(\omega_j)\pi(s_0\mid\omega_j)}{\mu(\omega_i)\pi(s_0\mid\omega_i)} μ ( ω i ) π ( s 0 ∣ ω i ) μ ( ω j ) π ( s 0 ∣ ω j ) under which the receiver, when best responding myopically, is indifferent in taking actions a a a and a ω i ∗ a_{\omega_i}^* a ω i ∗ .
Definition 1 (First indifferent action). For a pair of distinguishable states ω i , ω j \omega_i,\omega_j ω i , ω j , for each action a ∈ A ∖ { a ω i ∗ } a\in A\setminus\{a_{\omega_i}^*\} a ∈ A ∖ { a ω i ∗ } , the first indifferent action a ~ ω j , ω i \tilde{a}_{\omega_j,\omega_i} a ~ ω j , ω i for the receiver is
a ~ ω j , ω i : = arg min { ζ ω j , ω i a ∣ a ∈ A s.t. v ( a , ω j ) > v ( a ω i ∗ , ω j ) } . \tilde{a}_{\omega_j,\omega_i}:=\arg\min\{\zeta_{\omega_j,\omega_i}^{a}\mid a\in A\text{ s.t. }v(a,\omega_j)>v(a_{\omega_i}^*,\omega_j)\}. a ~ ω j , ω i := arg min { ζ ω j , ω i a ∣ a ∈ A s.t. v ( a , ω j ) > v ( a ω i ∗ , ω j )} .
If the arg min set is not a singleton, choose a ~ ω j , ω i \tilde{a}_{\omega_j,\omega_i} a ~ ω j , ω i to be the action in the set that maximizes v ( a , ω j ) v(a,\omega_j) v ( a , ω j ) .
The designer knows the cutoff c ( ⋅ , ⋅ , ⋅ ) c(\cdot,\cdot,\cdot) c ( ⋅ , ⋅ , ⋅ ) for the prior belief ratio μ ( ω j ) μ ( ω i ) \frac{\mu(\omega_j)}{\mu(\omega_i)} μ ( ω i ) μ ( ω j ) based on her knowledge about the receiver’s payoff function v v v . That enables the designer to use a binary search algorithm to identify a precise range containing the receiver’s prior belief ratio μ R ( ω j ) μ R ( ω i ) \frac{\mu_R(\omega_j)}{\mu_R(\omega_i)} μ R ( ω i ) μ R ( ω j ) . In particular, by observing whether the receiver takes action a ω i ∗ a_{\omega_i}^* a ω i ∗ or not, the designer is able to tell whether the receiver’s actual prior belief ratio μ R ( ω j ) μ R ( ω i ) \frac{\mu_R(\omega_j)}{\mu_R(\omega_i)} μ R ( ω i ) μ R ( ω j ) is below or above the cutoff c ( π , ω i , ω j ) c(\pi,\omega_i,\omega_j) c ( π , ω i , ω j ) . That enables the designer to adjust the signaling scheme and the corresponding cutoff threshold through iterations over time. By multiple iterations, the designer narrows down the range of μ R ( ω j ) μ R ( ω i ) \frac{\mu_R(\omega_j)}{\mu_R(\omega_i)} μ R ( ω i ) μ R ( ω j ) to length ε \varepsilon ε in O ( log 1 ε ) O(\log\frac{1}{\varepsilon}) O ( log ε 1 ) steps.
We now turn to the case when the receiver is strategic. A long-lived receiver may sacrifice his current payoff to take deceptive actions (i.e., deviating from a myopic best response), in order to manipulate the designer’s learning of his belief μ R \mu_R μ R to obtain more favorable information in future periods. Our learning algorithm makes the designer’s learning robust to such strategic manipulation. lation. First, the algorithm delays the influence of the receiver’s current action on the update of the signaling scheme. In particular, in each iteration k k k , after recording the action when signal s 0 s_0 s 0 first occurs, the algorithm keeps the signaling scheme π ( k ) \pi^{(k)} π ( k ) fixed for an additional m m m periods before updating the signaling scheme for the next iteration k + 1 k+1 k + 1 . Thus, the recorded action cannot affect the signaling scheme during these m m m periods. With discounting, the receiver’s potential gain from a manipulation is at most γ m + 1 / ( 1 − γ ) \gamma^{m+1}/(1-\gamma) γ m + 1 / ( 1 − γ ) . So any manipulation that incurs an immediate loss larger than that gain is not attractive to the receiver.
Second, the algorithm uses only the receiver’s action when signal s 0 s_0 s 0 first occurs in each iteration and ignores subsequent actions during the additional m m m periods. If the algorithm also took subsequent actions into account, actions taken near the end of the delay could affect the signaling scheme updating almost immediately, giving the receiver a stronger incentive to manipulate the update. Together, these two features make the receiver’s actions relevant for future signaling scheme updating only at sufficiently separated periods. The algorithm need not induce exact myopic best responses when the receiver is nearly indifferent between manipulation and obedience (i.e., when deviating from the myopic best response incurs a small immediate loss), but the resulting error in belief estimate can be bounded and made small by a delay length m m m chosen to be sufficiently large.
Algorithm 1, based on binary search, formally shows how to learn the receiver’s prior belief ratio μ R ( ω i ) μ R ( ω j ) \frac{\mu_R(\omega_i)}{\mu_R(\omega_j)} μ R ( ω j ) μ R ( ω i ) for any pair of distinguishable states ω i , ω j \omega_i,\omega_j ω i , ω j when the receiver acts strategically.
Figure 1.
The performance of Algorithm 1 is characterized by Lemma 1, which shows that a good estimate of μ R ( ω i ) μ R ( ω j ) \frac{\mu_R(\omega_i)}{\mu_R(\omega_j)} μ R ( ω j ) μ R ( ω i ) can be obtained within a very short amount of time: Algorithm 1 requires at most O ( log T ) O(\log T) O ( log T ) periods to reach a precision τ = O ( 1 T ) \tau= O\left(\frac{1}{T}\right) τ = O ( T 1 ) .
Lemma 1. The output of Algorithm 1, ρ ^ \hat{\rho} ρ ^ , satisfies ∣ ρ ^ − μ R ( ω i ) μ R ( ω j ) ∣ ≤ 1 G ( τ + γ m ( 1 − γ ) G 3 p 0 3 ) ≤ 1 G p 0 ( τ + γ m ( 1 − γ ) G 3 p 0 3 ) μ R ( ω i ) μ R ( ω j ) \left|\hat{\rho}-\frac{\mu_R(\omega_i)}{\mu_R(\omega_j)}\right| \leq\frac{1}{G}\left(\tau+\frac{\gamma^m}{(1-\gamma)G_3p_0^3}\right) \leq\frac{1}{Gp_0}\left(\tau+\frac{\gamma^m}{(1-\gamma)G_3p_0^3}\right)\frac{\mu_R(\omega_i)}{\mu_R(\omega_j)} ρ ^ − μ R ( ω j ) μ R ( ω i ) ≤ G 1 ( τ + ( 1 − γ ) G 3 p 0 3 γ m ) ≤ G p 0 1 ( τ + ( 1 − γ ) G 3 p 0 3 γ m ) μ R ( ω j ) μ R ( ω i ) , and Algorithm 1 terminates in at most ( 1 p 0 + m ) log 2 1 G p 0 τ \left(\frac{1}{p_0}+m\right)\log_2\frac{1}{Gp_0\tau} ( p 0 1 + m ) log 2 G p 0 τ 1 periods in expectation.
Proof. See Appendix B.1.
A key part in the proof of Lemma 1 is the following Lemma 2, which shows that the strategic receiver is incentivized to perform consistently with single-period best responses if the signaling scheme ratio estimate π ( k ) ( s 0 ∣ ω j ) π ( k ) ( s 0 ∣ ω i ) \frac{\pi^{(k)}(s_0|\omega_j)}{\pi^{(k)}(s_0|\omega_i)} π ( k ) ( s 0 ∣ ω i ) π ( k ) ( s 0 ∣ ω j ) differs from the targeted ratio π ∗ ( s 0 ∣ ω j ) π ∗ ( s 0 ∣ ω i ) \frac{\pi^*(s_0|\omega_j)}{\pi^*(s_0|\omega_i)} π ∗ ( s 0 ∣ ω i ) π ∗ ( s 0 ∣ ω j ) by a positive distance ι = \iota= ι = O ( γ m ) O(\gamma^m) O ( γ m ) , where the targeted ratio π ∗ ( s 0 ∣ ω j ) π ∗ ( s 0 ∣ ω i ) \frac{\pi^*(s_0\mid\omega_j)}{\pi^*(s_0\mid\omega_i)} π ∗ ( s 0 ∣ ω i ) π ∗ ( s 0 ∣ ω j ) is the solution to
μ R ( ω j ) μ R ( ω i ) = π ∗ ( s 0 ∣ ω i ) π ∗ ( s 0 ∣ ω j ) ⋅ v ( a ω i ∗ , ω i ) − v ( a ~ ω j , ω i , ω i ) v ( a ~ ω j , ω i , ω j ) − v ( a ω i ∗ , ω j ) . \frac{\mu_R(\omega_j)}{\mu_R(\omega_i)}=\frac{\pi^*(s_0\mid\omega_i)}{\pi^*(s_0\mid\omega_j)}\cdot\frac{v(a^*_{\omega_i},\omega_i)-v(\tilde a_{\omega_j,\omega_i},\omega_i)}{v(\tilde a_{\omega_j,\omega_i},\omega_j)-v(a^*_{\omega_i},\omega_j)}. μ R ( ω i ) μ R ( ω j ) = π ∗ ( s 0 ∣ ω j ) π ∗ ( s 0 ∣ ω i ) ⋅ v ( a ~ ω j , ω i , ω j ) − v ( a ω i ∗ , ω j ) v ( a ω i ∗ , ω i ) − v ( a ~ ω j , ω i , ω i ) .
Lemma 2. Choose ι = γ m ( 1 − γ ) G 2 p 0 2 \iota=\frac{\gamma^m}{(1-\gamma)G^2p_0^2} ι = ( 1 − γ ) G 2 p 0 2 γ m . In Line 6 of Algorithm 1, when signal s 0 s_0 s 0 is sent,
if π ( k ) ( s 0 ∣ ω j ) π ( k ) ( s 0 ∣ ω i ) < ( 1 − ι ) π ∗ ( s 0 ∣ ω j ) π ∗ ( s 0 ∣ ω i ) \frac{\pi^{(k)}(s_0\mid\omega_j)}{\pi^{(k)}(s_0\mid\omega_i)}<(1-\iota)\frac{\pi^*(s_0\mid\omega_j)}{\pi^*(s_0\mid\omega_i)} π ( k ) ( s 0 ∣ ω i ) π ( k ) ( s 0 ∣ ω j ) < ( 1 − ι ) π ∗ ( s 0 ∣ ω i ) π ∗ ( s 0 ∣ ω j ) , then the receiver will take action a ω i ∗ a^*_{\omega_i} a ω i ∗ ;
if π ( k ) ( s 0 ∣ ω j ) π ( k ) ( s 0 ∣ ω i ) > ( 1 + ι ) π ∗ ( s 0 ∣ ω j ) π ∗ ( s 0 ∣ ω i ) \frac{\pi^{(k)}(s_0\mid\omega_j)}{\pi^{(k)}(s_0\mid\omega_i)}>(1+\iota)\frac{\pi^*(s_0\mid\omega_j)}{\pi^*(s_0\mid\omega_i)} π ( k ) ( s 0 ∣ ω i ) π ( k ) ( s 0 ∣ ω j ) > ( 1 + ι ) π ∗ ( s 0 ∣ ω i ) π ∗ ( s 0 ∣ ω j ) , then the receiver will not take action a ω i ∗ a^*_{\omega_i} a ω i ∗ .
To see the intuition of Lemma 2, note that when recommended by a signal s 0 s_0 s 0 according to Algorithm 1, the strategic receiver is incentivized to take actions truthfully because his loss from deviation is larger than his gain from deviation. Regarding the gain, the receiver’s accumulated gain from deviating is upper bounded by a positive number due to his discounted factor. Regarding the loss, the receiver’s loss is lower bounded by a positive number that depends on how close our ratio estimate π ( k ) ( s 0 ∣ ω j ) π ( k ) ( s 0 ∣ ω i ) \frac{\pi^{(k)}(s_0\mid\omega_j)}{\pi^{(k)}(s_0\mid\omega_i)} π ( k ) ( s 0 ∣ ω i ) π ( k ) ( s 0 ∣ ω j ) is from the targeted ratio π ∗ ( s 0 ∣ ω j ) π ∗ ( s 0 ∣ ω i ) \frac{\pi^*(s_0\mid\omega_j)}{\pi^*(s_0\mid\omega_i)} π ∗ ( s 0 ∣ ω i ) π ∗ ( s 0 ∣ ω j ) , as well as the positive margin G G G by the unique optimal action assumption (Assumption 3). Then, at a controlled distance of ratio estimate π ( k ) ( s 0 ∣ ω j ) π ( k ) ( s 0 ∣ ω i ) \frac{\pi^{(k)}(s_0\mid\omega_j)}{\pi^{(k)}(s_0\mid\omega_i)} π ( k ) ( s 0 ∣ ω i ) π ( k ) ( s 0 ∣ ω j ) from ratio π ∗ ( s 0 ∣ ω j ) π ∗ ( s 0 ∣ ω i ) \frac{\pi^*(s_0\mid\omega_j)}{\pi^*(s_0\mid\omega_i)} π ∗ ( s 0 ∣ ω i ) π ∗ ( s 0 ∣ ω j ) , the receiver’s loss lower bound is strictly larger than his gain upper bound, making him incentivized not to deviate from the recommendation.
As an illustration, Figure 4 a shows the case of π ( k ) ( s 0 ∣ ω j ) π ( k ) ( s 0 ∣ ω i ) < ( 1 − ι ) π ∗ ( s 0 ∣ ω j ) π ∗ ( s 0 ∣ ω i ) \frac{\pi^{(k)}(s_0\mid\omega_j)}{\pi^{(k)}(s_0\mid\omega_i)}<(1-\iota)\frac{\pi^*(s_0\mid\omega_j)}{\pi^*(s_0\mid\omega_i)} π ( k ) ( s 0 ∣ ω i ) π ( k ) ( s 0 ∣ ω j ) < ( 1 − ι ) π ∗ ( s 0 ∣ ω i ) π ∗ ( s 0 ∣ ω j ) , in which the strategic receiver is incentivized to take action a ω i ∗ a^*_{\omega_i} a ω i ∗ . The payoff brought by action a ω i ∗ a^*_{\omega_i} a ω i ∗ is the highest for that case. Figure 4 b shows the case of π ( k ) ( s 0 ∣ ω j ) π ( k ) ( s 0 ∣ ω i ) > ( 1 − ι ) π ∗ ( s 0 ∣ ω j ) π ∗ ( s 0 ∣ ω i ) \frac{\pi^{(k)}(s_0\mid\omega_j)}{\pi^{(k)}(s_0\mid\omega_i)}>(1-\iota)\frac{\pi^*(s_0\mid\omega_j)}{\pi^*(s_0\mid\omega_i)} π ( k ) ( s 0 ∣ ω i ) π ( k ) ( s 0 ∣ ω j ) > ( 1 − ι ) π ∗ ( s 0 ∣ ω i ) π ∗ ( s 0 ∣ ω j ) , in which the strategic receiver is incentivized not to take action a ω i ∗ a^*_{\omega_i} a ω i ∗ . The payoff brought by action a ω i ∗ a^*_{\omega_i} a ω i ∗ is no longer the highest.
Figure 4. Normalized utility of the receiver for different actions. The x x x -coordinate is the signaling ratio π ( s 0 ∣ ω j ) π ( s 0 ∣ ω i ) \frac{\pi(s_0|\omega_j)}{\pi(s_0|\omega_i)} π ( s 0 ∣ ω i ) π ( s 0 ∣ ω j ) . Each line corresponds to an action a a a , with equation y a ( x ) = v ( a , ω i ) + μ R ( ω j ) μ R ( ω i ) ⋅ x ⋅ v ( a , ω j ) y_a(x)=v(a,\omega_i)+\frac{\mu_R(\omega_j)}{\mu_R(\omega_i)}\cdot x\cdot v(a,\omega_j) y a ( x ) = v ( a , ω i ) + μ R ( ω i ) μ R ( ω j ) ⋅ x ⋅ v ( a , ω j ) .
Proof. See Appendix § 10 .§ 2 .
What if the receiver-optimal actions of a pair of states are the same? Algorithm 2 deals with such a case, allowing the information designer to estimate the prior ratio between any pair of states. The idea of Algorithm 2 is that, if states ω i \omega_i ω i and ω j \omega_j ω j have the same receiver-optimal actions, then both of them must be distinguishable from one state in a distinguishable pair of states, say ω k ∈ Ω \omega_k\in\Omega ω k ∈ Ω , by Assumption 4. Thus, we obtain the estimate of μ R ( ω i ) μ R ( ω j ) \frac{\mu_R(\omega_i)}{\mu_R(\omega_j)} μ R ( ω j ) μ R ( ω i ) by estimating the ratios μ R ( ω i ) μ R ( ω k ) \frac{\mu_R(\omega_i)}{\mu_R(\omega_k)} μ R ( ω k ) μ R ( ω i ) and μ R ( ω j ) μ R ( ω k ) \frac{\mu_R(\omega_j)}{\mu_R(\omega_k)} μ R ( ω k ) μ R ( ω j ) separately.
Algorithm 2: Estimating Belief Ratio for any Pair of States ( ω i , ω j ) (\omega_i,\omega_j) ( ω i , ω j )
Parameter: τ > 0 \tau>0 τ > 0 and m ≥ 1 m\ge1 m ≥ 1
If ω i \omega_i ω i and ω j \omega_j ω j are distinguishable, i.e., a ω i ∗ ≠ a ω j ∗ a^*_{\omega_i}\ne a^*_{\omega_j} a ω i ∗ = a ω j ∗ , then run Algorithm 1 on ω i \omega_i ω i and ω j \omega_j ω j .
Otherwise, i.e., a ω i ∗ = a ω j ∗ a^*_{\omega_i}=a^*_{\omega_j} a ω i ∗ = a ω j ∗ , find ω k ∈ Ω \omega_k\in\Omega ω k ∈ Ω such that a ω k ∗ ≠ a ω i ∗ = a ω j ∗ a^*_{\omega_k}\ne a^*_{\omega_i}=a^*_{\omega_j} a ω k ∗ = a ω i ∗ = a ω j ∗ . Run Algorithm 1 to obtain an estimate ρ ^ i k \hat\rho_{ik} ρ ^ ik for μ R ( ω i ) μ R ( ω k ) \frac{\mu_R(\omega_i)}{\mu_R(\omega_k)} μ R ( ω k ) μ R ( ω i ) and an estimate ρ ^ j k \hat\rho_{jk} ρ ^ j k for μ R ( ω j ) μ R ( ω k ) \frac{\mu_R(\omega_j)}{\mu_R(\omega_k)} μ R ( ω k ) μ R ( ω j ) . Return ρ ^ i j = ρ ^ i k ρ ^ j k \hat\rho_{ij}=\frac{\hat\rho_{ik}}{\hat\rho_{jk}} ρ ^ ij = ρ ^ j k ρ ^ ik .
Figure 1 (PDF p. 19). Normalized utility of the receiver for different actions. The x x x -coordinate is the signaling ratio π ( s 0 ∣ ω j ) π ( s 0 ∣ ω i ) \frac{\pi(s_0 \mid\omega_j)}{\pi(s_0 \mid\omega_i)} π ( s 0 ∣ ω i ) π ( s 0 ∣ ω j ) . Each line corresponds to an action a a a , with equation y a ( x ) = v ( a , ω i ) + μ R ( ω j ) μ R ( ω i ) ⋅ x ⋅ v ( a , ω j ) y_a(x)=v(a,\omega_i)+\frac{\mu_R(\omega_j)}{\mu_R(\omega_i)}\cdot x\cdot v(a,\omega_j) y a ( x ) = v ( a , ω i ) + μ R ( ω i ) μ R ( ω j ) ⋅ x ⋅ v ( a , ω j ) .
Full algorithm for learning the belief μ R \mu_R μ R . After showing how to learn the belief ratio for any pair of states, we now present the full algorithm in Algorithm 3 for the designer to learn the receiver’s belief μ R \mu_R μ R . Algorithm 3 first uses Algorithm 2 to learn the ratio μ R ( ω i ) μ R ( ω 1 ) \frac{\mu_R(\omega_i)}{\mu_R(\omega_1)} μ R ( ω 1 ) μ R ( ω i ) for each state ω i \omega_i ω i , i = 2 , … , ∣ Ω ∣ i=2,\ldots,|\Omega| i = 2 , … , ∣Ω∣ and state ω 1 \omega_1 ω 1 , and then reconstructs the estimate of the unknown belief from those ratio estimates.
Algorithm 3: Learning the Belief of the Strategic Receiver
Parameter: desired estimation accuracy ε > 0 \varepsilon>0 ε > 0
Let τ = G p 3 ε 24 ∣ Ω ∣ \tau=\frac{G_p^3\varepsilon}{24|\Omega|} τ = 24∣Ω∣ G p 3 ε , m = ⌈ 1 ln ( 1 / γ ) ln ( 24 ∣ Ω ∣ ( 1 − γ ) G 4 p 0 6 ε ) ⌉ m=\left\lceil\frac{1}{\ln(1/\gamma)}\ln\left(\frac{24|\Omega|}{(1-\gamma)G^4p_0^6\varepsilon}\right)\right\rceil m = ⌈ l n ( 1/ γ ) 1 ln ( ( 1 − γ ) G 4 p 0 6 ε 24∣Ω∣ ) ⌉ .
For every state ω i ∈ Ω \omega_i\in\Omega ω i ∈ Ω , i = 2 , … , ∣ Ω ∣ i=2,\ldots,|\Omega| i = 2 , … , ∣Ω∣ , apply Algorithm 2 to state pair ( ω i , ω 1 ) (\omega_i,\omega_1) ( ω i , ω 1 ) with parameters τ \tau τ and m m m to learn the ratio μ R ( ω i ) μ R ( ω 1 ) \frac{\mu_R(\omega_i)}{\mu_R(\omega_1)} μ R ( ω 1 ) μ R ( ω i ) , obtaining ratio estimate ρ ^ i 1 \hat{\rho}_{i1} ρ ^ i 1 .
Return belief estimate μ ^ ∈ Δ ( Ω ) \hat{\mu}\in\Delta(\Omega) μ ^ ∈ Δ ( Ω ) by
μ ^ ( ω 1 ) = 1 / ( 1 + ∑ i = 2 ∣ Ω ∣ ρ ^ i 1 ) ; μ ^ ( ω i ) = ρ ^ i 1 μ ^ ( ω 1 ) , for i = 2 , … , ∣ Ω ∣ . \begin{aligned}
\hat{\mu}(\omega_1)&=1/\left(1+\sum_{i=2}^{|\Omega|}\hat{\rho}_{i1}\right);\\
\hat{\mu}(\omega_i)&=\hat{\rho}_{i1}\hat{\mu}(\omega_1),\quad\text{for }i=2,\ldots,|\Omega|.
\end{aligned} μ ^ ( ω 1 ) μ ^ ( ω i ) = 1/ 1 + i = 2 ∑ ∣Ω∣ ρ ^ i 1 ; = ρ ^ i 1 μ ^ ( ω 1 ) , for i = 2 , … , ∣Ω∣.
Algorithm 3 gives a belief estimate μ ^ \hat{\mu} μ ^ with an ℓ 1 \ell_1 ℓ 1 distance precision ε \varepsilon ε at ∥ μ ^ − μ R ∥ 1 ≤ ε \lVert\hat{\mu}-\mu_R\rVert_1\leq\varepsilon ∥ μ ^ − μ R ∥ 1 ≤ ε within a short running time of O ( log 2 1 ε ) O\left(\log^2\frac{1}{\varepsilon}\right) O ( log 2 ε 1 ) periods.
Lemma 3. Given 0 < ε ≤ 1 0<\varepsilon\leq1 0 < ε ≤ 1 ,
periods.
Proof. See Appendix § 10 .§ 3 .
Static Robustification of Signaling Schemes
Computing the signaling scheme optimal for a belief. In general, given any prior belief μ 0 ∈ Δ ( Ω ) \mu_0 \in\Delta(\Omega) μ 0 ∈ Δ ( Ω ) of the receiver, assumed to be known to the designer, the optimal signaling scheme in the static Bayesian persuasion problem can be computed easily.
In particular, given any prior belief μ 0 ∈ Δ ( Ω ) \mu_0 \in\Delta(\Omega) μ 0 ∈ Δ ( Ω ) known to be the receiver’s belief, there exists a signaling scheme π 0 \pi_0 π 0 that is direct and persuasive ([21 ] ) under μ 0 \mu_0 μ 0 : a signaling scheme π : Ω → Δ ( A ) \pi: \Omega\to\Delta(A) π : Ω → Δ ( A ) is direct if it maps each state to a probability distribution over actions ; a direct signaling scheme π 0 \pi_0 π 0 is persuasive if all actions recommended by π 0 \pi_0 π 0 are optimal for the receiver under prior μ 0 \mu_0 μ 0 : ∑ ω ∈ Ω μ 0 ( ω ) π 0 ( a ∣ ω ) [ v ( a , ω ) − v ( a ′ , ω ) ] ≥ 0 \sum_{\omega\in\Omega} \mu_0(\omega)\pi_0(a \mid\omega)[v(a,\omega)-v(a',\omega)] \ge0 ∑ ω ∈ Ω μ 0 ( ω ) π 0 ( a ∣ ω ) [ v ( a , ω ) − v ( a ′ , ω )] ≥ 0 , ∀ a , a ′ ∈ A \forall a,a' \in A ∀ a , a ′ ∈ A . Denote by Pers ( μ 0 ) \operatorname{Pers}(\mu_0) Pers ( μ 0 ) the set of persuasive signaling schemes given the receiver’s belief μ 0 ∈ Δ ( Ω ) \mu_0 \in\Delta(\Omega) μ 0 ∈ Δ ( Ω ) :
Pers ( μ 0 ) : = { π : Ω → Δ ( A ) ∣ ∑ ω ∈ Ω μ 0 ( ω ) π ( a ∣ ω ) [ v ( a , ω ) − v ( a ′ , ω ) ] ≥ 0 , ∀ a , a ′ ∈ A } , (10) \operatorname{Pers}(\mu_0) := \left\{ \pi: \Omega\to\Delta(A) \mathrel{\big|} \sum_{\omega\in\Omega} \mu_0(\omega)\pi(a \mid\omega)[v(a,\omega)-v(a',\omega)] \ge0,\ \forall a,a' \in A \right\}, \tag*{(10)} Pers ( μ 0 ) := { π : Ω → Δ ( A ) ω ∈ Ω ∑ μ 0 ( ω ) π ( a ∣ ω ) [ v ( a , ω ) − v ( a ′ , ω )] ≥ 0 , ∀ a , a ′ ∈ A } , (10)
where
∑ ω ∈ Ω μ 0 ( ω ) π 0 ( a ∣ ω ) [ v ( a , ω ) − v ( a ′ , ω ) ] ∀ a , a ′ ∈ A \sum_{\omega\in\Omega} \mu_0(\omega)\pi_0(a \mid\omega)[v(a,\omega)-v(a',\omega)]\ \forall a,a' \in A ω ∈ Ω ∑ μ 0 ( ω ) π 0 ( a ∣ ω ) [ v ( a , ω ) − v ( a ′ , ω )] ∀ a , a ′ ∈ A
is the persuasiveness of the signaling scheme π \pi π under the receiver’s belief μ 0 \mu_0 μ 0 .
The signaling scheme optimal for the designer with μ D \mu_D μ D under her knowledge of the receiver’s belief μ 0 \mu_0 μ 0 is solved tractably from the following linear program
π ∗ ( μ 0 ) = arg max π ∈ Pers ( μ 0 ) ∑ ω ∈ Ω μ D ( ω ) ∑ a ∈ A π ( s ∣ ω ) u ( a , ω ) . (11) \pi^*(\mu_0) = \underset{\pi\in\operatorname{Pers}(\mu_0)}{\arg\max}\ \sum_{\omega\in\Omega} \mu_D(\omega)\sum_{a \in A}\pi(s \mid\omega)u(a,\omega). \tag*{(11)} π ∗ ( μ 0 ) = π ∈ Pers ( μ 0 ) arg max ω ∈ Ω ∑ μ D ( ω ) a ∈ A ∑ π ( s ∣ ω ) u ( a , ω ) . (11)
Thus, if the designer knows the receiver’s belief μ R \mu_R μ R , then the designer will obtain her optimal signaling scheme π ∗ \pi^* π ∗ for μ R \mu_R μ R exactly by the linear program 11. However, after the learning phase, the designer has a learned belief μ ^ \hat{\mu} μ ^ , which is only an approximation for the receiver’s prior belief μ R \mu_R μ R .
Static robustification. After the first phase of learning, we have obtained an estimate μ ^ \hat{\mu} μ ^ of the receiver’s prior belief μ R \mu_R μ R . However, even when the estimate μ ^ \hat{\mu} μ ^ is very close to the prior belief μ R \mu_R μ R , the signaling scheme π ^ \hat{\pi} π ^ optimal for the designer under the estimate μ ^ \hat{\mu} μ ^ may not be persuasive or even perform arbitrarily badly under μ R \mu_R μ R . The reason is that even though a signal may induce two close posterior beliefs from two close prior beliefs μ ^ \hat{\mu} μ ^ and μ R \mu_R μ R , respectively, the optimal actions for the receiver under those two posterior beliefs may be different, which may lead to an arbitrarily low payoff for the designer under the receiver’s prior belief μ R \mu_R μ R .
We propose designing a signaling scheme π \pi π using static robustification . In general, we convert any direct signaling scheme π \pi π under μ ^ \hat{\mu} μ ^ to another signaling scheme π ~ \tilde{\pi} π ~ , such that π ~ \tilde{\pi} π ~ , when applied to a targeted prior belief (e.g., the receiver’s prior μ R \mu_R μ R ), has a weakly higher persuasiveness compared to the case when that typical signaling scheme π \pi π is applied to the belief estimate μ ^ \hat{\mu} μ ^ . Recall that the persuasiveness of a direct signaling scheme under a prior belief measures how any recommended action from that signaling scheme is weakly better than any other unrecommended action. That is, for any a , a ′ ∈ A a, a' \in A a , a ′ ∈ A ,
∑ ω μ R ( ω ) π ~ ( a ∣ ω ) [ v ( a , ω ) − v ( a ′ , ω ) ] ≥ ∑ ω μ ^ ( ω ) π ( a ∣ ω ) [ v ( a , ω ) − v ( a ′ , ω ) ] . \sum_{\omega} \mu_R(\omega)\tilde{\pi}(a \mid\omega)[v(a,\omega)-v(a',\omega)] \ge\sum_{\omega} \hat{\mu}(\omega)\pi(a \mid\omega)[v(a,\omega)-v(a',\omega)]. ω ∑ μ R ( ω ) π ~ ( a ∣ ω ) [ v ( a , ω ) − v ( a ′ , ω )] ≥ ω ∑ μ ^ ( ω ) π ( a ∣ ω ) [ v ( a , ω ) − v ( a ′ , ω )] .
Moreover, static robustification causes only a small loss in the designer’s payoff. In particular, applying static robustification to π ^ \hat{\pi} π ^ yields a signaling scheme that is persuasive under the receiver’s actual belief μ R \mu_R μ R and approximately optimal for the designer. By being static, the robustification process is applied to the signaling scheme π ^ \hat{\pi} π ^ once, and the same statically robustified signaling scheme π ~ \tilde{\pi} π ~ is used repeatedly in each period for the robustification phase till the end.
Lemma 4 summarizes the effects of static robustification. We briefly introduce the idea of static robustification. We first construct a posterior whose persuasiveness is strengthened compared to any direct signaling scheme π \pi π under belief estimate μ ^ \hat{\mu} μ ^ . To construct that posterior, we first consider the posterior μ ^ a \hat{\mu}_a μ ^ a induced by π \pi π under belief estimate μ ^ \hat{\mu} μ ^ for each action a a a . Then we consider a buffer belief η a \eta_a η a for each action a a a , under which action a a a gives the receiver an expected payoff at least D D D higher than every other action by Assumption 2. We construct posterior ξ a \xi_a ξ a for each action a a a by
ξ a = ( 1 − δ ) μ ^ a + δ η a , (12) \xi_a = (1-\delta)\hat{\mu}_a + \delta\eta_a, \tag*{(12)} ξ a = ( 1 − δ ) μ ^ a + δ η a , (12)
for a robustification strength δ ∈ ( 0 , p 0 2 ] \delta\in(0,\frac{p_0}{2}] δ ∈ ( 0 , 2 p 0 ] . Thus, if π \pi π is persuasive under belief μ ^ \hat{\mu} μ ^ , then the receiver prefers action a a a to every alternative by at least δ D \delta D δ D under the constructed posterior ξ a \xi_a ξ a . This positive margin allows persuasiveness to be remained when the receiver’s prior μ R \mu_R μ R is close to belief μ ^ \hat{\mu} μ ^ . Then, we restore Bayesian plausibility by choosing a residual belief χ \chi χ on the boundary of the belief simplex Δ ( Ω ) \Delta(\Omega) Δ ( Ω ) and a weight y ∈ [ 0 , δ p 0 ] y \in[0,\frac{\delta}{p_0}] y ∈ [ 0 , p 0 δ ] such that
μ ^ = ( 1 − y ) ∑ a : q a > 0 q a ξ a + y χ , \hat{\mu} = (1-y)\sum_{a:q_a>0}q_a\xi_a + y\chi, μ ^ = ( 1 − y ) a : q a > 0 ∑ q a ξ a + y χ ,
where q a = ∑ ω ∈ Ω μ ^ ( ω ) π ( a ∣ ω ) q_a=\sum_{\omega\in\Omega}\hat{\mu}(\omega)\pi(a\mid\omega) q a = ∑ ω ∈ Ω μ ^ ( ω ) π ( a ∣ ω ) is the probability that π \pi π recommends action a a a under belief μ ^ \hat{\mu} μ ^ . Thus, we obtain a signaling scheme π ~ \tilde{\pi} π ~ that induces posterior ξ a \xi_a ξ a with probability ( 1 − y ) q a (1-y)q_a ( 1 − y ) q a and fully reveals state ω \omega ω with probability y χ ( ω ) y\chi(\omega) y χ ( ω ) under belief μ ^ \hat{\mu} μ ^ . Finally, we replace each state-revealing signal for full revelation with a receiver-optimal action recommendation to ensure static robustification gives a direct signaling scheme π ~ \tilde{\pi} π ~ .
Lemma 4 (Static Robustification). Assume Assumptions 1 and 2. Choose ϵ ≤ p 0 2 D 4 \epsilon\le\frac{p_0^2D}{4} ϵ ≤ 4 p 0 2 D . Given the belief estimate μ ^ \hat{\mu} μ ^ and the robustification strength δ ∈ [ 2 ϵ p 0 D , p 0 2 ] \delta\in\left[\frac{2\epsilon}{p_0D},\frac{p_0}{2}\right] δ ∈ [ p 0 D 2 ϵ , 2 p 0 ] , any direct signaling scheme π \pi π can be converted into another direct signaling scheme π ~ \tilde{\pi} π ~ such that
(improved persuasiveness for the receiver) for any prior beliefs μ \mu μ and μ ^ \hat{\mu} μ ^ with ∥ μ − μ ^ ∥ 1 ≤ ϵ \lVert\mu-\hat{\mu}\rVert_1 \le\epsilon ∥ μ − μ ^ ∥ 1 ≤ ϵ , for any recommendation a ∈ A a \in A a ∈ A from π ~ \tilde{\pi} π ~ , the persuasiveness of π ~ \tilde{\pi} π ~ and π \pi π under the resulted posterior beliefs μ ( ⋅ ∣ a , π ~ ) \mu(\cdot|a,\tilde{\pi}) μ ( ⋅ ∣ a , π ~ ) , μ ^ ( ⋅ ∣ a , π ~ ) \hat{\mu}(\cdot|a,\tilde{\pi}) μ ^ ( ⋅ ∣ a , π ~ ) are close:
E ω ∼ μ ( ⋅ ∣ a , π ~ ) [ v ( a , ω ) − v ( a ′ , ω ) ] ≥ min { E ω ∼ μ ^ ( ⋅ ∣ a , π ) [ v ( a , ω ) − v ( a ′ , ω ) ] + δ D − 2 ϵ p 0 , 0 } , ∀ a ′ ∈ A ∖ { a } , \mathbb{E}_{\omega\sim\mu(\cdot|a,\tilde{\pi})}\left[v(a,\omega)-v(a',\omega)\right]\ge\min\left\{\mathbb{E}_{\omega\sim\hat{\mu}(\cdot|a,\pi)}\left[v(a,\omega)-v(a',\omega)\right]+\delta D-\frac{2\epsilon}{p_0},0\right\},\ \forall a'\in A\setminus\{a\}, E ω ∼ μ ( ⋅ ∣ a , π ~ ) [ v ( a , ω ) − v ( a ′ , ω ) ] ≥ min { E ω ∼ μ ^ ( ⋅ ∣ a , π ) [ v ( a , ω ) − v ( a ′ , ω ) ] + δ D − p 0 2 ϵ , 0 } , ∀ a ′ ∈ A ∖ { a } , and, in particular, if the direct signaling scheme π \pi π is persuasive for μ ^ \hat{\mu} μ ^ , then its robustification result π ~ \tilde{\pi} π ~ is persuasive for μ \mu μ ;
(tiny influence for the designer) when the receiver obeys the recommendation, the designer’s payoff from the robustified signaling scheme π ~ \tilde{\pi} π ~ , compared to that from the direct signaling scheme π \pi π , satisfies
U ( π ~ , ρ o b ; μ D ) ≥ U ( π , ρ o b ; μ D ) − 3 δ p 0 , U(\tilde{\pi},\rho_{\mathrm{ob}};\mu_D)\ge U(\pi,\rho_{\mathrm{ob}};\mu_D)-\frac{3\delta}{p_0}, U ( π ~ , ρ ob ; μ D ) ≥ U ( π , ρ ob ; μ D ) − p 0 3 δ , where ρ o b \rho_{\mathrm{ob}} ρ ob is the receiver’s obedience strategy that puts weight one on the recommended action;
∥ π ~ ( ⋅ ∣ ω ) − π ( ⋅ ∣ ω ) ∥ 1 ≤ 4 δ p 0 2 , ∀ ω ∈ Ω . \lVert\tilde{\pi}(\cdot|\omega)-\pi(\cdot|\omega)\rVert_1\le\frac{4\delta}{p_0^2},\ \forall\omega\in\Omega. ∥ π ~ ( ⋅ ∣ ω ) − π ( ⋅ ∣ ω ) ∥ 1 ≤ p 0 2 4 δ , ∀ ω ∈ Ω.
To see the intuition, recall that a signaling scheme obtained from static robustification is constructed based on the posterior ξ a \xi_a ξ a for each action a a a as in Eq. (12) . First, μ ^ a \hat{\mu}_a μ ^ a is one component of ξ a \xi_a ξ a , but that’s under the belief estimated μ ^ \hat{\mu} μ ^ , so the receiver under his prior μ R \mu_R μ R may still deviate from any recommendation and obtain a gain at most 2 ϵ / p 0 2\epsilon/p_0 2 ϵ / p 0 . However, the buffer belief η a \eta_a η a with weight δ \delta δ , as another component of ξ a \xi_a ξ a , guarantees the receiver a payoff at least δ D \delta D δ D under μ ^ \hat{\mu} μ ^ . Thus, before the signaling scheme is finally made direct, the receiver gains at least
δ D − 2 ϵ p 0 ≥ 0 \delta D-\frac{2\epsilon}{p_0}\ge0 δ D − p 0 2 ϵ ≥ 0
at δ = 2 ϵ / ( p 0 D ) \delta=2\epsilon/(p_0D) δ = 2 ϵ / ( p 0 D ) . The final step of static robustification, i.e., turning the signaling scheme into a direct one, requires sending the receiver his optimal action in the corresponding state, which also preserves the receiver’s obedience incentives. Thus, the direct signaling scheme π ~ \tilde{\pi} π ~ from static robustification is persuasive under μ R \mu_R μ R .
In contrast to previous works that apply a robustifying process to the signaling scheme optimal for the belief estimate {(citep}(Zu, Iyer, and Xu, 2021)} or to signaling schemes persuasive for the belief estimate {(citep}(Li and Lin, 2025)}, our static robustification is applied to any direct signaling schemes, thus being the most general. If we apply Lemma 4 to the designer’s optimal signaling scheme π ^ \hat{\pi} π ^ under the belief estimate μ ^ \hat{\mu} μ ^ with the robustification strength δ = 2 ε p 0 2 D \delta= \frac{2\varepsilon}{p_0^2D} δ = p 0 2 D 2 ε , then we will obtain an approximately optimal signaling scheme for the actual prior μ R \mu_R μ R of the receiver. This specific application is summarized in Corollary 1.
Corollary 1 (Robustification for the optimal signaling scheme). With ∥ μ ^ − μ R ∥ 1 ≤ ε ≤ p 0 2 D 4 \lVert\hat{\mu}-\mu_R\rVert_1 \le\varepsilon\le\frac{p_0^2D}{4} ∥ μ ^ − μ R ∥ 1 ≤ ε ≤ 4 p 0 2 D , the designer’s optimal signaling scheme π ^ \hat{\pi} π ^ under belief μ ^ \hat{\mu} μ ^ can be converted into another signaling scheme π ~ \tilde{\pi} π ~ that is persuasive under belief μ R \mu_R μ R and 6 ε p 0 2 D \frac{6\varepsilon}{p_0^2D} p 0 2 D 6 ε -optimal for the designer.
See details of Lemma 4 in Appendix § 10 .§ 5 .
Full Learning Algorithm with O ( log 2 T ) O(\log^2 T) O ( log 2 T ) Regret
Finally, we combine the prior-learning algorithm and the static robustification to obtain a signaling scheme π ~ \tilde{\pi} π ~ that is O ( ε ) O(\varepsilon) O ( ε ) -approximately optimal for the designer and persuasive for the receiver under belief μ R \mu_R μ R . Using the robustified signaling scheme π ~ \tilde{\pi} π ~ in each of the remaining periods thus incurs a small regret. The total regret of Algorithm 4 is formally characterized in Theorem 1.
Algorithm 4: Learning Algorithm for the Static Benchmark
Parameter: ε > 0 \varepsilon> 0 ε > 0
Run Algorithm 3 with parameter ε \varepsilon ε to obtain an estimate μ ^ \hat{\mu} μ ^ of receiver’s belief. Let T 0 T_0 T 0 be the number of periods taken.
Apply Corollary 1 to μ ^ \hat{\mu} μ ^ with parameter ε \varepsilon ε to obtain a signaling scheme π ~ \tilde{\pi} π ~ . Use π ~ \tilde{\pi} π ~ for the remaining T − T 0 T-T_0 T − T 0 periods.
Theorem 1. Assume Assumptions 1, 2, 3, 4. Choose ε = p 0 2 D 4 T \varepsilon= \frac{p_0^2D}{4T} ε = 4 T p 0 2 D . The static regret of Algorithm 4 is at most
Reg s t a t i c ( T ; Algorithm 4 , I ) ≤ O ( 1 1 − γ log 2 T ) . (13) \operatorname{Reg}^{\mathrm{static}}(T;\operatorname{Algorithm} 4,\mathcal{I}) \le O\left(\frac{1}{1-\gamma}\log^2 T\right). \tag*{(13)} Reg static ( T ; Algorithm 4 , I ) ≤ O ( 1 − γ 1 log 2 T ) . (13)
Proof. By Lemma 3, we have ∥ μ ^ − μ R ∥ 1 ≤ ε \lVert\hat{\mu}-\mu_R\rVert_1 \le\varepsilon ∥ μ ^ − μ R ∥ 1 ≤ ε and E [ T 0 ] ≤ 2 ∣ Ω ∣ ( 1 p 0 + 1 1 − γ ln ( 24 ∣ Ω ∣ ( 1 − γ ) G 4 p 0 6 ε ) ) log 2 24 ∣ Ω ∣ G 2 p 0 4 ε \mathbb{E}[T_0] \le2|\Omega|\left(\frac{1}{p_0}+\frac{1}{1-\gamma}\ln\left(\frac{24|\Omega|}{(1-\gamma)G^4p_0^6\varepsilon}\right)\right)\log_2\frac{24|\Omega|}{G^2p_0^4\varepsilon} E [ T 0 ] ≤ 2∣Ω∣ ( p 0 1 + 1 − γ 1 ln ( ( 1 − γ ) G 4 p 0 6 ε 24∣Ω∣ ) ) log 2 G 2 p 0 4 ε 24∣Ω∣ . According to Corollary 1, the constructed signaling scheme π ~ \tilde{\pi} π ~ is persuasive for μ R \mu_R μ R and is 6 ε p 0 2 D \frac{6\varepsilon}{p_0^2D} p 0 2 D 6 ε approximately optimal for the designer. Thus, the regret of Algorithm 4 satisfies
Reg s t a t i c ( T ; G , I ) ≤ E [ T 0 ⋅ 1 ] ⏟ regret in the first T 0 periods + ( T − E [ T 0 ] ) 6 ε p 0 2 D ⏟ regret in the remaining periods \operatorname{Reg}^{\mathrm{static}}(T;\mathcal{G},\mathcal{I}) \le\underbrace{\mathbb{E}[T_0\cdot1]}_{\text{regret in the first }T_0\text{ periods}} + \underbrace{(T-\mathbb{E}[T_0])\frac{6\varepsilon}{p_0^2D}}_{\text{regret in the remaining periods}} Reg static ( T ; G , I ) ≤ regret in the first T 0 periods E [ T 0 ⋅ 1 ] + regret in the remaining periods ( T − E [ T 0 ]) p 0 2 D 6 ε
≤ 2 ∣ Ω ∣ ( 1 p 0 + 1 1 − γ ln ( 24 ∣ Ω ∣ ( 1 − γ ) G 4 p 0 6 ε ) ) log 2 24 ∣ Ω ∣ G 2 p 0 4 ε + T 6 ε p 0 2 D . \le2|\Omega|\left(\frac{1}{p_0}+\frac{1}{1-\gamma}\ln\left(\frac{24|\Omega|}{(1-\gamma)G^4p_0^6\varepsilon}\right)\right)\log_2\frac{24|\Omega|}{G^2p_0^4\varepsilon}+T\frac{6\varepsilon}{p_0^2D}. ≤ 2∣Ω∣ ( p 0 1 + 1 − γ 1 ln ( ( 1 − γ ) G 4 p 0 6 ε 24∣Ω∣ ) ) log 2 G 2 p 0 4 ε 24∣Ω∣ + T p 0 2 D 6 ε .
Choosing ε = p 0 2 D 4 T \varepsilon=\frac{p_0^2D}{4T} ε = 4 T p 0 2 D (which satisfies the condition in Corollary 1), we obtain
Reg s t a t i c ( T ; G , I ) ≤ 2 ∣ Ω ∣ ( 1 p 0 + 1 1 − γ ln ( 96 ∣ Ω ∣ T ( 1 − γ ) G 4 p 0 8 D ) ) log 2 96 ∣ Ω ∣ T G 2 p 0 6 D + 3 2 = O ( 1 1 − γ log 2 T ) . \begin{aligned}
\operatorname{Reg}^{\mathrm{static}}(T;\mathcal{G},\mathcal{I})\le2|\Omega|\left(\frac{1}{p_0}+\frac{1}{1-\gamma}\ln\left(\frac{96|\Omega|T}{(1-\gamma)G^4p_0^8D}\right)\right)\log_2\frac{96|\Omega|T}{G^2p_0^6D}+\frac{3}{2} \\
&=O\left(\frac{1}{1-\gamma}\log^2 T\right).
\end{aligned} Reg static ( T ; G , I ) ≤ 2∣Ω∣ ( p 0 1 + 1 − γ 1 ln ( ( 1 − γ ) G 4 p 0 8 D 96∣Ω∣ T ) ) log 2 G 2 p 0 6 D 96∣Ω∣ T + 2 3 = O ( 1 − γ 1 log 2 T ) .
Characterization of the Dynamic Benchmark
To see the intuition of Theorem 1, recall the algorithm has two phases: (i) learning the receiver’s belief μ R \mu_R μ R for an accuracy ε \varepsilon ε , and (ii) robustification, which repeatedly uses a robustified signaling scheme that is guaranteed persuasive for the receiver’s belief μ R \mu_R μ R and approximately optimal for the designer. Phase (i) runs binary search algorithms to keep updating the experimented signaling scheme, which takes at most O ( log ( 1 ε ) ) O(\log(\frac{1}{\varepsilon})) O ( log ( ε 1 )) distinct experimented signaling schemes. Further, for each distinct experimented signaling scheme, making the obedience incentive strong enough for a strategic receiver requires an additional repetition sub-phase of length m = O ( log ( 1 ε ) 1 − γ ) m=O(\frac{\log(\frac{1}{\varepsilon})}{1-\gamma}) m = O ( 1 − γ l o g ( ε 1 ) ) , contributing another log ( 1 ε ) \log(\frac{1}{\varepsilon}) log ( ε 1 ) . Thus, we get the O ( 1 1 − γ log 2 ( 1 ε ) ) O(\frac{1}{1-\gamma}\log^2(\frac{1}{\varepsilon})) O ( 1 − γ 1 log 2 ( ε 1 )) exploration cost formalized in Lemma 3. Phase (ii) robustifies the designer’s optimal signaling scheme under the estimate μ ^ \hat{\mu} μ ^ . That preserves persuasiveness for all beliefs within a distance of ε \varepsilon ε of μ ^ \hat{\mu} μ ^ , thus also for the receiver’s belief μ R \mu_R μ R , while degrading the designer’s payoff by only O ( ε p 0 2 D ) O(\frac{\varepsilon}{p_0^2D}) O ( p 0 2 D ε ) per period (Corollary 1). Choosing ε = p 0 2 D 4 T \varepsilon=\frac{p_0^2D}{4T} ε = 4 T p 0 2 D results in a loss of T ⋅ O ( ε p 0 2 D ) = O ( 1 ) T\cdot O(\frac{\varepsilon}{p_0^2D})=O(1) T ⋅ O ( p 0 2 D ε ) = O ( 1 ) for phase (ii) (exploitation), so the regret is dominated by phase (i) (exploitation), yielding Reg s t a t i c ( T ) = O ( 1 1 − γ log 2 T ) \operatorname{Reg}^{\mathrm{static}}(T)=O(\frac{1}{1-\gamma}\log^2 T) Reg static ( T ) = O ( 1 − γ 1 log 2 T ) as claimed in Theorem 1.
We add a final brief remark on Algorithm 4 and Theorem 1. In the first phase, the designed signaling scheme used in binary searches may be not persuasive . The designer’s primary goal is to gather informative responses from the receiver’s actions to accurately estimate the prior through binary search during that phase. Thus, achieving persuasiveness and optimality at each step is secondary to this learning objective.
Characterization of the Dynamic Benchmark
This section characterizes the dynamic benchmark U T ∗ ∗ ( I ) U_T^{**}(\mathcal{I}) U T ∗∗ ( I ) , which is the optimal payoff obtainable by the designer if she knows the receiver’s belief μ R \mu_R μ R . That is, it is the designer’s optimal payoff in a T T T -period repeated Bayesian persuasion problem with full commitment power. In § 5.1 , we show that a certain class of grim-trigger-style strategies are optimal for the designer, that the dynamic benchmark is weakly larger than the static benchmark, and that the designer’s optimal strategy can be obtained through dynamic programming. In § 5.2 , we present examples to illustrate the dynamic benchmark and the optimal strategy.
Optimal Dynamically Direct + Punishment (DDP) Strategy
Optimal DDP strategy: dynamic benchmark characterization We show that the designer’s dynamic benchmark U T ∗ ∗ ( I ) U_T^{**}(\mathcal{I}) U T ∗∗ ( I ) can be achieved by a class of T T T -period strategies with a grim-trigger property: the designer recommends actions to the receiver until the receiver deviates from the recommendation, and provides no information if the receiver deviates. We call such a class of T T T -period strategies “dynamically direct + punishment (DDP)” strategies.
Definition 2 (DDP strategy). A T T T -period strategy σ \sigma σ of the designer is a dynamically direct + punishment (DDP) strategy if σ \sigma σ
(dynamically direct) uses a direct signaling scheme if the receiver has never deviated from past recommendations,
(punishment) and reveals no information (i.e., always sending one signal regardless of the state) if the receiver did not follow the recommendation at some point in the past.
The punishment depends only on whether the receiver’s observed action matches the recommendation he received. The designer does not observe how he would have responded to other recommendations.
We now introduce two important properties of DDP strategies. The first property is history independence , which means that the signaling scheme in each period does not depend on what happened in the past, as long as the receiver has been obedient since the beginning.
Definition 3 (History Independence). A DDP strategy σ \sigma σ is history-independent if the direct signaling scheme π σ ( t ) \pi_{\sigma}^{(t)} π σ ( t ) for each period t t t depends on the period number t t t but not on history, as long as the receiver followed all past recommendations.
Definition 4 (Dynamic Persuasiveness). A DDP strategy σ = ( π σ ( t ) ) t = 1 T \sigma= \left(\pi_{\sigma}^{(t)}\right)_{t=1}^{T} σ = ( π σ ( t ) ) t = 1 T is dynamically persuasive for the receiver if the receiver’s T T T -period obedient strategy
ϕ o b = obey the action recommendation in every period t ∈ { 1 , … , T } given any history \phi_{\mathrm{ob}} = \text{obey the action recommendation in every period } t \in\{1,\ldots,T\} \text{ given any history} ϕ ob = obey the action recommendation in every period t ∈ { 1 , … , T } given any history is a best response to σ \sigma σ .
If the direct signaling scheme in each period of a DDP strategy is all persuasive in the per-period Bayesian persuasion game, then obedience is clearly a best response for the receiver, and the DDP strategy is dynamically persuasive. However, a dynamically persuasive DDP strategy may include some direct signaling schemes that are not persuasive in the static game. We will provide such examples in § 5.2 .
The first result of this subsection shows that the designer’s dynamic benchmark U T ∗ ∗ ( I ) U_T^{**}(\mathcal{I}) U T ∗∗ ( I ) can be achieved by a history-independent and dynamically persuasive DDP strategy σ ∗ \sigma^* σ ∗ :
Proposition 1 (Optimality of DDP Strategy). The dynamic benchmark U T ∗ ∗ ( I ) U_T^{**}(\mathcal{I}) U T ∗∗ ( I ) can be achieved by a pair of strategies ( σ ∗ , ϕ o b ) (\sigma^*,\phi_{\mathrm{ob}}) ( σ ∗ , ϕ ob ) , where σ ∗ = ( π σ ∗ ( t ) ) t = 1 T \sigma^* = \left(\pi_{\sigma^*}^{(t)}\right)_{t=1}^{T} σ ∗ = ( π σ ∗ ( t ) ) t = 1 T is a history-independent and dynamically persuasive DDP strategy of the designer, and ϕ o b \phi_{\mathrm{ob}} ϕ ob is the T T T -period obedient strategy of the receiver.
Proof. The proof of Proposition 1 has two steps. First, we provide a dynamic revelation principle (in a similar spirit to Myerson, [25 ] ) in the following Lemma 5, which shows that the designer’s T T T -period (global) optimal utility U T ∗ ∗ ( I ) U_T^{**}(\mathcal{I}) U T ∗∗ ( I ) can always be achieved by a history-dependent dynamically persuasive DDP strategy. See Appendix § 11 .§ 1 for the proof of Lemma 5.
Lemma 5 (Dynamic revelation principle). There exists an optimal strategy σ ∗ = ( π σ ∗ ( t ) ) t = 1 T \sigma^* = \left(\pi_{\sigma^*}^{(t)}\right)_{t=1}^{T} σ ∗ = ( π σ ∗ ( t ) ) t = 1 T for the designer that is a history-dependent dynamically persuasive DDP strategy, that is, U T ∗ ∗ ( I ) = U T ( σ ∗ , ϕ o b ; μ D ) U_T^{**}(\mathcal{I}) = U_T(\sigma^*,\phi_{\mathrm{ob}};\mu_D) U T ∗∗ ( I ) = U T ( σ ∗ , ϕ ob ; μ D ) and ϕ o b = ϕ ∗ ( σ ∗ , μ R ) \phi_{\mathrm{ob}} = \phi^*(\sigma^*,\mu_R) ϕ ob = ϕ ∗ ( σ ∗ , μ R ) .
Next, we show in Lemma 6 that history-dependence is not needed. At a high level, this is because the Bayesian persuasion problems at different periods are independent, as the states are i.i.d. sampled. As long as the receiver has not deviated in the past, his actions will not contain information that is valuable for the designer. Thus, the designer does not need to adjust signaling schemes based on the irrelevant history. The formal proof of Lemma 6 is in Appendix § 11 .§ 2 .
Lemma 6. There exists an optimal dynamically persuasive DDP strategy that is history-independent.
The above two lemmas together prove Proposition 1.
Computing the optimal DDP strategy (dynamic benchmark) Proposition 1 significantly simplifies the designer’s optimization problem. We can write the dynamic benchmark U T ∗ ∗ ( I ) U_T^{**}(\mathcal{I}) U T ∗∗ ( I ) as an optimization problem over history-independent DDP strategies subject to the dynamic persuasiveness constraint.
We now study the dynamic persuasiveness constraint in detail. For a history-independent DDP strategy σ = ( π σ ( t ) ) t = 1 T \sigma= (\pi_\sigma^{(t)})_{t=1}^{T} σ = ( π σ ( t ) ) t = 1 T , dynamic persuasiveness requires that for each period t t t , the sequence of ongoing (period t t t included) direct signaling schemes ( π σ ( t ) , … , π σ ( T ) ) (\pi_\sigma^{(t)},\ldots,\pi_\sigma^{(T)}) ( π σ ( t ) , … , π σ ( T ) ) in σ = ( π σ ( t ) ) t = 1 T \sigma= (\pi_\sigma^{(t)})_{t=1}^{T} σ = ( π σ ( t ) ) t = 1 T should satisfy the following: for any action a a a recommended by π σ ( t ) \pi_\sigma^{(t)} π σ ( t ) with positive probability P μ R , π σ ( t ) ( a ) = ∑ ω ∈ Ω μ R ( ω ) π σ ( t ) ( a ∣ ω ) > 0 P_{\mu_R,\pi_\sigma^{(t)}}(a)=\sum_{\omega\in\Omega}\mu_R(\omega)\pi_\sigma^{(t)}(a\mid\omega)>0 P μ R , π σ ( t ) ( a ) = ∑ ω ∈ Ω μ R ( ω ) π σ ( t ) ( a ∣ ω ) > 0 , for any a ′ ∈ A a'\in A a ′ ∈ A ,
γ t ∑ ω ∈ Ω μ R ( ω ) π σ ( t ) ( a ∣ ω ) P μ R , π σ ( t ) ( a ) v ( a , ω ) + ∑ t ′ = t + 1 T γ t ′ V ( π σ ( t ′ ) , ρ o b ; μ R ) ≥ γ t ∑ ω ∈ Ω μ R ( ω ) π σ ( t ) ( a ∣ ω ) P μ R , π σ ( t ) ( a ) v ( a ′ , ω ) + ∑ t ′ = t + 1 T γ t ′ V u n i n f o r m e d ( μ R ) , (14) \gamma^t\sum_{\omega\in\Omega}\frac{\mu_R(\omega)\pi_\sigma^{(t)}(a\mid\omega)}{P_{\mu_R,\pi_\sigma^{(t)}}(a)}v(a,\omega)+\sum_{t'=t+1}^{T}\gamma^{t'}V(\pi_\sigma^{(t')},\rho_{\mathrm{ob}};\mu_R)
\geq\gamma^t\sum_{\omega\in\Omega}\frac{\mu_R(\omega)\pi_\sigma^{(t)}(a\mid\omega)}{P_{\mu_R,\pi_\sigma^{(t)}}(a)}v(a',\omega)+\sum_{t'=t+1}^{T}\gamma^{t'}V_{\mathrm{uninformed}}(\mu_R), \tag*{(14)} γ t ω ∈ Ω ∑ P μ R , π σ ( t ) ( a ) μ R ( ω ) π σ ( t ) ( a ∣ ω ) v ( a , ω ) + t ′ = t + 1 ∑ T γ t ′ V ( π σ ( t ′ ) , ρ ob ; μ R ) ≥ γ t ω ∈ Ω ∑ P μ R , π σ ( t ) ( a ) μ R ( ω ) π σ ( t ) ( a ∣ ω ) v ( a ′ , ω ) + t ′ = t + 1 ∑ T γ t ′ V uninformed ( μ R ) , (14)
where V u n i n f o r m e d ( μ R ) = max a ∈ A ∑ ω ∈ Ω μ R ( ω ) v ( a , ω ) V_{\mathrm{uninformed}}(\mu_R)=\max_{a\in A}\sum_{\omega\in\Omega}\mu_R(\omega)v(a,\omega) V uninformed ( μ R ) = max a ∈ A ∑ ω ∈ Ω μ R ( ω ) v ( a , ω ) is the receiver’s payoff from being punished. The left-hand side of Eq. (14) is the receiver’s payoff of obedience from period t t t to T T T given the action a a a recommended at period t t t , while the second line is the receiver’s payoff from period t t t to T T T by deviating to a ′ a' a ′ at period t t t and then being punished in the future.
An equivalent way to write the dynamic persuasiveness constraint uses the receiver’s posterior belief. In particular, denote by μ R ( ω ∣ a , π σ ( t ) ) = μ R ( ω ) π σ ( t ) ( a ∣ ω ) P μ R , π σ ( t ) ( a ) \mu_R(\omega\mid a,\pi_\sigma^{(t)})=\frac{\mu_R(\omega)\pi_\sigma^{(t)}(a\mid\omega)}{P_{\mu_R,\pi_\sigma^{(t)}}(a)} μ R ( ω ∣ a , π σ ( t ) ) = P μ R , π σ ( t ) ( a ) μ R ( ω ) π σ ( t ) ( a ∣ ω ) the posterior belief of state ω \omega ω given action a a a sent by signaling scheme π ( t ) \pi^{(t)} π ( t ) under belief μ R \mu_R μ R . Then, Eq. (14) can be rewritten as follows. For each period t t t , ∀ a , a ′ ∈ A \forall a,a'\in A ∀ a , a ′ ∈ A ,
γ t E ω ∼ μ R ( ⋅ ∣ a , π σ ( t ) ) [ v ( a , ω ) − v ( a ′ , ω ) ] + ∑ t ′ = t + 1 T γ t ′ ( V ( π σ ( t ′ ) , ρ o b ; μ R ) − V u n i n f o r m e d ( μ R ) ) ≥ 0. (15) \gamma^t\mathbb{E}_{\omega\sim\mu_R(\cdot\mid a,\pi_\sigma^{(t)})}[v(a,\omega)-v(a',\omega)]+\sum_{t'=t+1}^{T}\gamma^{t'}\left(V(\pi_\sigma^{(t')},\rho_{\mathrm{ob}};\mu_R)-V_{\mathrm{uninformed}}(\mu_R)\right)\geq0. \tag*{(15)} γ t E ω ∼ μ R ( ⋅ ∣ a , π σ ( t ) ) [ v ( a , ω ) − v ( a ′ , ω )] + t ′ = t + 1 ∑ T γ t ′ ( V ( π σ ( t ′ ) , ρ ob ; μ R ) − V uninformed ( μ R ) ) ≥ 0. (15)
Formally, the optimization problem for the dynamic benchmark U T ∗ ∗ ( I ) U_T^{**}(\mathcal{I}) U T ∗∗ ( I ) is
U T ∗ ∗ ( I ) = max history-independent DDP strategy σ = ( π σ ( t ) ) t = 1 T U T ( σ ; ϕ o b , μ D ) = ∑ t = 1 T U ( π σ ( t ) , ρ o b ; μ D ) (16) U_T^{**}(\mathcal{I})=
\max_{\text{history-independent DDP strategy }\sigma=(\pi_\sigma^{(t)})_{t=1}^{T}}
U_T(\sigma;\phi_{\mathrm{ob}},\mu_D)
=\sum_{t=1}^{T}U(\pi_\sigma^{(t)},\rho_{\mathrm{ob}};\mu_D) \tag*{(16)} U T ∗∗ ( I ) = history-independent DDP strategy σ = ( π σ ( t ) ) t = 1 T max U T ( σ ; ϕ ob , μ D ) = t = 1 ∑ T U ( π σ ( t ) , ρ ob ; μ D ) (16)
subject to σ \sigma σ satisfies (14) or (15).
Lemma 7 shows that the dynamic benchmark is always weakly larger than the static benchmark. Moreover, the dynamic benchmark is strictly larger than the static benchmark in some instances, as we will show in § 5.2 .
Lemma 7. The dynamic benchmark is weakly larger than the static benchmark: U T ∗ ∗ ( I ) ≥ U T ∗ ( I ) U_T^{**}(\mathcal{I}) \ge U_T^*(\mathcal{I}) U T ∗∗ ( I ) ≥ U T ∗ ( I ) .
Proof. Let π B P ∗ \pi_{\mathrm{BP}}^* π BP ∗ be an optimal signaling scheme in the static Bayesian persuasion problem. From § 3.2 , π B P ∗ \pi_{\mathrm{BP}}^* π BP ∗ is persuasive, where the obedient strategy ρ o b \rho_{\mathrm{ob}} ρ ob of the receiver is a single-period best response. Thus, a T T T times repetition of π B P ∗ \pi_{\mathrm{BP}}^* π BP ∗ , denoted by σ r e p = ( π B P ∗ ( 1 ) , … , π B P ∗ ( T ) ) \sigma^{\mathrm{rep}} = (\pi_{\mathrm{BP}}^{*(1)}, \ldots, \pi_{\mathrm{BP}}^{*(T)}) σ rep = ( π BP ∗ ( 1 ) , … , π BP ∗ ( T ) ) , is a feasible strategy for the T T T -period game. The receiver’s global best response to σ r e p \sigma^{\mathrm{rep}} σ rep consists of obedience in each period. Thus, the designer’s T T T -period payoff is T ⋅ U B P ∗ ( I ) T \cdot U_{\mathrm{BP}}^*(\mathcal{I}) T ⋅ U BP ∗ ( I ) . By the optimality of the dynamic benchmark, we immediately have
U T ∗ ∗ ( I ) ≥ U T ( σ r e p , ϕ o b ; μ D ) = T ⋅ U B P ∗ ( I ) = U T ∗ ( I ) . U_T^{**}(\mathcal{I}) \ge U_T(\sigma^{\mathrm{rep}}, \phi_{\mathrm{ob}}; \mu_D) = T \cdot U_{\mathrm{BP}}^*(\mathcal{I}) = U_T^*(\mathcal{I}). U T ∗∗ ( I ) ≥ U T ( σ rep , ϕ ob ; μ D ) = T ⋅ U BP ∗ ( I ) = U T ∗ ( I ) .
A dynamic program to solve for U T ∗ ∗ ( I ) U_T^{**}(\mathcal{I}) U T ∗∗ ( I ) . Having shown that the designer’s dynamic benchmark U T ∗ ∗ ( I ) U_T^{**}(\mathcal{I}) U T ∗∗ ( I ) can be achieved by a history-independent dynamically persuasive DDP strategy, we present a dynamic programming algorithm for obtaining such an optimal DDP strategy.
Define a value function F ( t , g ) F(t,g) F ( t , g ) as the designer’s optimal payoff from period t t t to period T T T under a history-independent dynamically persuasive DDP strategy, such that the receiver’s continuation payoff from being obedient (from period t t t to T T T ) is weakly larger than the continuation payoff from being punished by receiving no information plus a non-negative margin of g ≥ 0 g \ge0 g ≥ 0 . Then the designer’s optimization problem is, for each period t = { T , T − 1 , … , 1 } t = \{T,T-1,\ldots,1\} t = { T , T − 1 , … , 1 } ,
F ( t , g ) = max π ( t ) , … , π ( T ) ∑ t ′ = t T U ( π ( t ′ ) , ρ o b ; μ D ) (17) F(t,g) = \max_{\pi^{(t)},\ldots,\pi^{(T)}} \sum_{t'=t}^{T} U(\pi^{(t')},\rho_{\mathrm{ob}};\mu_D) \tag*{(17)} F ( t , g ) = π ( t ) , … , π ( T ) max t ′ = t ∑ T U ( π ( t ′ ) , ρ ob ; μ D ) (17)
s.t. ∑ t ′ = t T γ t ′ − t ⋅ V ( π ( t ′ ) , ρ o b ; μ R ) ≥ ∑ t ′ = t T γ t ′ − t ⋅ V u n i n f o r m e d ( μ R ) + g \text{s.t.}\quad\sum_{t'=t}^{T} \gamma^{t'-t} \cdot V(\pi^{(t')},\rho_{\mathrm{ob}};\mu_R) \ge\sum_{t'=t}^{T} \gamma^{t'-t} \cdot V_{\mathrm{uninformed}}(\mu_R) + g s.t. t ′ = t ∑ T γ t ′ − t ⋅ V ( π ( t ′ ) , ρ ob ; μ R ) ≥ t ′ = t ∑ T γ t ′ − t ⋅ V uninformed ( μ R ) + g
( π ( t ) , … , π ( T ) ) (\pi^{(t)},\ldots,\pi^{(T)}) ( π ( t ) , … , π ( T ) ) is dynamically persuasive.
Essentially, the designer promises the receiver that he will obtain a payoff of at least ∑ t ′ = t T γ t ′ − t ⋅ V u n i n f o r m e d ( μ R ) + g \sum_{t'=t}^{T}\gamma^{t'-t}\cdot V_{\mathrm{uninformed}}(\mu_R)+g ∑ t ′ = t T γ t ′ − t ⋅ V uninformed ( μ R ) + g if he keeps being obedient from period t t t to T T T . Set F ( t , g ) = − ∞ F(t,g)=-\infty F ( t , g ) = − ∞ if the optimization problem (17) is infeasible, which leads to the dynamic benchmark obtained at U T ∗ ∗ ( I ) = F ( 1 , 0 ) U_T^{**}(\mathcal{I})=F(1,0) U T ∗∗ ( I ) = F ( 1 , 0 ) .
We compute the value function F ( t , g ) F(t,g) F ( t , g ) recursively from the last period. That is, we first solve the optimization (18) for the last period t = T t=T t = T by solving
F ( T , g ) = max π ( T ) U ( π ( T ) , ρ o b ; μ D ) (18) F(T,g)=\max_{\pi^{(T)}} U(\pi^{(T)},\rho_{\mathrm{ob}};\mu_D) \tag*{(18)} F ( T , g ) = π ( T ) max U ( π ( T ) , ρ ob ; μ D ) (18)
s . t . V ( π ( T ) , ρ o b ; μ R ) − V u n i n f o r m e d ( μ R ) ≥ g , \mathrm{s.t.}\quad V(\pi^{(T)},\rho_{\mathrm{ob}};\mu_R)-V_{\mathrm{uninformed}}(\mu_R)\ge g, s.t. V ( π ( T ) , ρ ob ; μ R ) − V uninformed ( μ R ) ≥ g ,
π ( T ) \pi^{(T)} π ( T ) is persuasive.
Then, for each t ∈ { T − 1 , T − 2 , … , 1 } t\in\{T-1,T-2,\ldots,1\} t ∈ { T − 1 , T − 2 , … , 1 } , g ≥ 0 g\ge0 g ≥ 0 , solve the following optimization problem, which makes use of the value function F ( t + 1 , g ′ ) F(t+1,g') F ( t + 1 , g ′ ) for the next period t + 1 t+1 t + 1 :
F ( t , g ) = max π ( t ) , g ′ ≥ 0 U ( π ( t ) , ρ o b ; μ D ) + F ( t + 1 , g ′ ) (19) F(t,g)=\max_{\pi^{(t)},g'\ge0} U(\pi^{(t)},\rho_{\mathrm{ob}};\mu_D)+F(t+1,g') \tag*{(19)} F ( t , g ) = π ( t ) , g ′ ≥ 0 max U ( π ( t ) , ρ ob ; μ D ) + F ( t + 1 , g ′ ) (19)
s . t . V ( π ( t ) , ρ o b ; μ R ) − V u n i n f o r m e d ( μ R ) + γ ⋅ g ′ ≥ g \mathrm{s.t.}\quad V(\pi^{(t)},\rho_{\mathrm{ob}};\mu_R)-V_{\mathrm{uninformed}}(\mu_R)+\gamma\cdot g'\ge g s.t. V ( π ( t ) , ρ ob ; μ R ) − V uninformed ( μ R ) + γ ⋅ g ′ ≥ g
∑ ω ∈ Ω μ R ( ω ) π ( t ) ( a ∣ ω ) ( v ( a , ω ) − v ( a ′ , ω ) + γ ⋅ g ′ ) ≥ 0 ∀ a , a ′ ∈ A . \sum_{\omega\in\Omega}\mu_R(\omega)\pi^{(t)}(a\mid\omega)\left(v(a,\omega)-v(a',\omega)+\gamma\cdot g'\right)\ge0\quad\forall a,a'\in A. ω ∈ Ω ∑ μ R ( ω ) π ( t ) ( a ∣ ω ) ( v ( a , ω ) − v ( a ′ , ω ) + γ ⋅ g ′ ) ≥ 0 ∀ a , a ′ ∈ A .
To see the intuition, we note that F ( t , g ) F(t,g) F ( t , g ) is the designer’s best total payoff from period t t t onward, while promising the receiver that his obedient payoff will be larger than his deviating payoff by at least a margin of g ≥ 0 g\ge0 g ≥ 0 , starting from the current period t t t to the end . At the current period t t t , the designer chooses the current signaling scheme π ( t ) \pi^{(t)} π ( t ) and the next promised payoff g ′ g' g ′ that the designer must deliver starting from the next period t + 1 t+1 t + 1 . The current signaling scheme π ( t ) \pi^{(t)} π ( t ) and the next promised payoff g ′ g' g ′ should satisfy the first constraints in the above optimization problems, which ensure that the receiver’s obedient payoff is larger than the deviating payoff by a margin of g g g starting from the beginning of period t t t . The choices of π ( t ) \pi^{(t)} π ( t ) and g ′ g' g ′ should also satisfy the second constraints above, such that the receiver, after receiving the current recommendation a a a , will indeed find the recommendation persuasive, considering the gain from deviating from a a a to a ′ a' a ′ and the loss of promised future utility g ′ g' g ′ after deviation.
The following Lemma 8 shows that the dynamic program above computes the dynamic benchmark correctly, with its full proof in Appendix § 11 .§ 3 .
Lemma 8 (Dynamic program for dynamic benchmark). The dynamic program (18)-(19) computes an optimal solution ( σ ∗ , ϕ o b ) (\sigma^*,\phi_{\mathrm{ob}}) ( σ ∗ , ϕ ob ) that reaches the dynamic benchmark U T ∗ ( I ) U_T^*(\mathcal{I}) U T ∗ ( I ) , where σ ∗ = ( π σ ∗ ( t ) ) t = 1 T \sigma^*=(\pi_{\sigma^*}^{(t)})_{t=1}^T σ ∗ = ( π σ ∗ ( t ) ) t = 1 T is an optimal DDP strategy that is history-independent and dynamically persuasive.
Examples of Optimal DDP Strategies
Here we illustrate the structure of the optimal history-independent dynamically persuasive DDP strategy that attains the dynamic benchmark U T ∗ U_T^* U T ∗ . We consider an example with three states Ω = \Omega= Ω =
{ ω 1 , ω 2 , ω 3 } \{\omega_1,\omega_2,\omega_3\} { ω 1 , ω 2 , ω 3 } and three actions A = { a 1 , a 2 , a 3 } A=\{a_1,a_2,a_3\} A = { a 1 , a 2 , a 3 } . The payoffs of the designer and receiver are
Table 1.
In words, the designer prefers a 1 a_1 a 1 the most, a 2 a_2 a 2 the second, a 3 a_3 a 3 the least, regardless of the state, while the receiver wants to match the state. The designer and the receiver have a common prior μ D = μ R = ( 0.2 , 0.3 , 0.5 ) \mu_D=\mu_R=(0.2,0.3,0.5) μ D = μ R = ( 0.2 , 0.3 , 0.5 ) . We will compare the optimal DDP strategy against the optimal signaling scheme π B P ∗ \pi^*_{\mathrm{BP}} π BP ∗ in the static Bayesian persuasion problem
Table 2.
which provides the designer and the receiver with expected payoffs
U B P ∗ = U ( π B P ∗ , ρ o b ; μ D ) = 0.7 , V ( π B P ∗ , ρ o b ; μ R ) = 0.5 = V u n i n f o r m e d ( μ R ) . (20) U^*_{\mathrm{BP}}=U(\pi^*_{\mathrm{BP}},\rho_{\mathrm{ob}};\mu_D)=0.7,\qquad V(\pi^*_{\mathrm{BP}},\rho_{\mathrm{ob}};\mu_R)=0.5=V_{\mathrm{uninformed}}(\mu_R). \tag*{(20)} U BP ∗ = U ( π BP ∗ , ρ ob ; μ D ) = 0.7 , V ( π BP ∗ , ρ ob ; μ R ) = 0.5 = V uninformed ( μ R ) . (20)
For different choices of T T T and γ \gamma γ , we compute an optimal DDP strategy σ ∗ = ( π σ ∗ ( t ) ) t = 1 T \sigma^*=(\pi_{\sigma^*}^{(t)})_{t=1}^{T} σ ∗ = ( π σ ∗ ( t ) ) t = 1 T for the designer using the dynamic programming in Lemma 8 and present the results below.
Optimal DDP strategy for T = 2 T=2 T = 2 periods
We start with the simple case where the total number of periods is T = 2 T=2 T = 2 and the receiver is patient (having discount factor γ = 1 \gamma=1 γ = 1 ). In this case, the optimal DDP strategy σ ∗ = ( π σ ∗ ( 1 ) , π σ ∗ ( 2 ) ) \sigma^*=(\pi_{\sigma^*}^{(1)},\pi_{\sigma^*}^{(2)}) σ ∗ = ( π σ ∗ ( 1 ) , π σ ∗ ( 2 ) ) for the designer is the following:
σ ∗ = ( π σ ∗ ( 1 ) : ω 1 ω 2 ω 3 a 1 1 1 1 a 2 0 0 0 a 3 0 0 0 , π σ ∗ ( 2 ) : ω 1 ω 2 ω 3 a 1 1 0 2 / 5 a 2 0 1 0 a 3 0 0 3 / 5 ) . \sigma^*=\left(
\begin{array}{c|ccc}
\pi_{\sigma^*}^{(1)}: & \omega_1 & \omega_2 & \omega_3\\
a_1 & 1 & 1 & 1\\
a_2 & 0 & 0 & 0\\
a_3 & 0 & 0 & 0
\end{array},
\begin{array}{c|ccc}
\pi_{\sigma^*}^{(2)}: & \omega_1 & \omega_2 & \omega_3\\
a_1 & 1 & 0 & 2/5\\
a_2 & 0 & 1 & 0\\
a_3 & 0 & 0 & 3/5
\end{array}
\right). σ ∗ = π σ ∗ ( 1 ) : a 1 a 2 a 3 ω 1 1 0 0 ω 2 1 0 0 ω 3 1 0 0 , π σ ∗ ( 2 ) : a 1 a 2 a 3 ω 1 1 0 0 ω 2 0 1 0 ω 3 2/5 0 3/5 .
With an obedient receiver, the designer’s expected payoffs in the two periods are U ( π σ ∗ ( 1 ) , ρ o b ; μ D ) = 1 U(\pi_{\sigma^*}^{(1)},\rho_{\mathrm{ob}};\mu_D)=1 U ( π σ ∗ ( 1 ) , ρ ob ; μ D ) = 1 and U ( π σ ∗ ( 2 ) , ρ o b ; μ D ) = 0.55 U(\pi_{\sigma^*}^{(2)},\rho_{\mathrm{ob}};\mu_D)=0.55 U ( π σ ∗ ( 2 ) , ρ ob ; μ D ) = 0.55 . The (average) dynamic benchmark is
1 T U T ∗ ∗ = 1 T U T ( σ ∗ , ϕ o b ; μ D ) = 1 2 ( U ( π σ ∗ ( 1 ) , ρ o b ; μ D ) + U ( π σ ∗ ( 2 ) , ρ o b ; μ D ) ) = 0.775 , \frac{1}{T}U_T^{**}=\frac{1}{T}U_T(\sigma^*,\phi_{\mathrm{ob}};\mu_D)=\frac{1}{2}\left(U(\pi_{\sigma^*}^{(1)},\rho_{\mathrm{ob}};\mu_D)+U(\pi_{\sigma^*}^{(2)},\rho_{\mathrm{ob}};\mu_D)\right)=0.775, T 1 U T ∗∗ = T 1 U T ( σ ∗ , ϕ ob ; μ D ) = 2 1 ( U ( π σ ∗ ( 1 ) , ρ ob ; μ D ) + U ( π σ ∗ ( 2 ) , ρ ob ; μ D ) ) = 0.775 ,
which is larger than the static BP benchmark (20). The obedient receiver’s expected payoffs are V ( π σ ∗ ( 1 ) , ρ o b ; μ R ) = 0.2 V(\pi_{\sigma^*}^{(1)},\rho_{\mathrm{ob}};\mu_R)=0.2 V ( π σ ∗ ( 1 ) , ρ ob ; μ R ) = 0.2 and V ( π σ ∗ ( 2 ) , ρ o b ; μ D ) = 0.8 V(\pi_{\sigma^*}^{(2)},\rho_{\mathrm{ob}};\mu_D)=0.8 V ( π σ ∗ ( 2 ) , ρ ob ; μ D ) = 0.8 , with the undiscounted average
1 T V T ( σ ∗ , ϕ o b ; μ D ) = 0.5 = V u n i n f o r m e d ( μ R ) . \frac{1}{T}V_T(\sigma^*,\phi_{\mathrm{ob}};\mu_D)=0.5=V_{\mathrm{uninformed}}(\mu_R). T 1 V T ( σ ∗ , ϕ ob ; μ D ) = 0.5 = V uninformed ( μ R ) .
Here are some interesting observations about the optimal DDP strategy:
The signaling scheme π σ ∗ ( 1 ) \pi_{\sigma^*}^{(1)} π σ ∗ ( 1 ) in the first period is not persuasive : it always recommends action a 1 a_1 a 1 , which is in favor of the designer but not the receiver. An obedient receiver’s payoff under π σ ∗ ( 1 ) \pi_{\sigma^*}^{(1)} π σ ∗ ( 1 ) , 0.2 0.2 0.2 , is even lower than the uninformed payoff V u n i n f o r m e d ( μ R ) = 0.5 V_{\mathrm{uninformed}}(\mu_R)=0.5 V uninformed ( μ R ) = 0.5 .
The signaling scheme π σ ∗ ( 2 ) \pi_{\sigma^*}^{(2)} π σ ∗ ( 2 ) in the second period is persuasive and provides the receiver with a high ex-ante payoff of V ( π σ ∗ ( 2 ) , ρ o b ; μ R ) = 0.8 V(\pi_{\sigma^*}^{(2)},\rho_{\mathrm{ob}};\mu_R)=0.8 V ( π σ ∗ ( 2 ) , ρ ob ; μ R ) = 0.8 , which is significantly larger than the receiver’s uninformed payoff V u n i n f o r m e d ( μ R ) = 0.5 V_{\mathrm{uninformed}}(\mu_R)=0.5 V uninformed ( μ R ) = 0.5 .
Due to the high ex-ante payoff promised by π σ ∗ ( 2 ) \pi_{\sigma^*}^{(2)} π σ ∗ ( 2 ) , the receiver is willing to follow the non-persuasive recommendation from π σ ∗ ( 1 ) \pi_{\sigma^*}^{(1)} π σ ∗ ( 1 ) in the first period, because otherwise the receiver will suffer a large loss in the second period. Therefore, the designer’s two-period strategy σ ∗ = ( π σ ∗ ( 1 ) , π σ ∗ ( 2 ) ) \sigma^*=(\pi_{\sigma^*}^{(1)},\pi_{\sigma^*}^{(2)}) σ ∗ = ( π σ ∗ ( 1 ) , π σ ∗ ( 2 ) ) satisfies dynamic persuasiveness.
In summary, the optimal two-period DDP strategy σ ∗ = ( π σ ∗ ( 1 ) , π σ ∗ ( 2 ) ) \sigma^*=(\pi_{\sigma^*}^{(1)},\pi_{\sigma^*}^{(2)}) σ ∗ = ( π σ ∗ ( 1 ) , π σ ∗ ( 2 ) ) exhibits a first-exploit-then-compensate structure. By exploiting the receiver early and compensating him later, the designer attains a higher average payoff than the static benchmark. This strategy requires a sufficiently forward-looking receiver: if the discount factor γ \gamma γ is too small, the receiver will not accept later compensation for early exploitation. The DDP strategy σ ∗ \sigma^* σ ∗ is dynamically persuasive even though the individual signaling schemes are not necessarily persuasive in the static game.
Optimal DDP strategy for T = 20 T=20 T = 20 periods
We next consider the ( T = 20 ) (T=20) ( T = 20 ) -period case with the receiver’s discount factor γ = 0.7 \gamma=0.7 γ = 0.7 . In Figure 2 , we plot the designer’s and receiver’s per-period payoffs U ( π σ ∗ ( t ) , ρ o b ; μ D ) U(\pi_{\sigma^*}^{(t)},\rho_{\mathrm{ob}};\mu_D) U ( π σ ∗ ( t ) , ρ ob ; μ D ) and V ( π σ ∗ ( t ) , ρ o b ; μ R ) V(\pi_{\sigma^*}^{(t)},\rho_{\mathrm{ob}};\mu_R) V ( π σ ∗ ( t ) , ρ ob ; μ R ) under the direct signaling schemes of the optimal DDP strategy σ ∗ = ( π σ ∗ ( t ) ) t = 1 T \sigma^*=(\pi_{\sigma^*}^{(t)})_{t=1}^{T} σ ∗ = ( π σ ∗ ( t ) ) t = 1 T , depicted in red and blue solid curves, respectively. The average payoff of the designer under the optimal DDP strategy is 1 T U T ∗ ∗ = 1 T ∑ t = 1 T U ( π σ ∗ ( t ) , ρ o b ; μ D ) ≈ 0.7279 \frac{1}{T}U_T^{**}=\frac{1}{T}\sum_{t=1}^{T}U(\pi_{\sigma^*}^{(t)},\rho_{\mathrm{ob}};\mu_D)\approx0.7279 T 1 U T ∗∗ = T 1 ∑ t = 1 T U ( π σ ∗ ( t ) , ρ ob ; μ D ) ≈ 0.7279 , which is larger than the static BP benchmark U B P ∗ = 0.7 U_{\mathrm{BP}}^*=0.7 U BP ∗ = 0.7 .
Figure 2. Per-period payoffs of designer and receiver under the optimal DDP strategy in § 5.2.2 .
We observe oscillations in both players’ payoffs. The oscillations can be explained by the first-exploit-then-compensate structure and by the fact that the designer is more patient than the receiver. To see one oscillation, as in the 2-period case, the designer’s optimal strategy is to exploit first to obtain a desired payoff and then to compensate the receiver. The repeated oscillations arise because the receiver is impatient while the designer is patient. If compensation is postponed by T ′ T' T ′ periods, the designer must promise a payoff in the order of γ − T ′ \gamma^{-T'} γ − T ′ for it to be persuasive for the receiver, which becomes prohibitively costly for the designer. Thus, the designer alternates short exploitation with timely compensation, keeping the receiver’s continuation payoff near the participation threshold (V u n i n f o r m e d V_{\mathrm{uninformed}} V uninformed ). This generates the periodic pattern in both players’ payoffs.
Finally, we remark that the oscillation occurs only for an impatient but sufficiently forward-looking receiver. If the receiver is impatient with a very small γ \gamma γ , then the designer’s optimal strategy will be to repeat her optimal strategy in the single-period Bayesian persuasion game, without causing oscillations, as demonstrated in the next subsection.
Effect of γ \gamma γ on Dynamic Benchmark
We illustrate the effect of the receiver’s discount factor γ \gamma γ on the designer’s dynamic benchmark U T ∗ ∗ U_T^{**} U T ∗∗ . Figure 3 plots the designer’s average payoff 1 T U T ∗ ∗ = 1 T ∑ t = 1 T U ( π σ ∗ ( t ) , ρ o b ; μ D ) \frac{1}{T}U_T^{**}=\frac{1}{T}\sum_{t=1}^{T}U(\pi_{\sigma^*}^{(t)},\rho_{ob};\mu_D) T 1 U T ∗∗ = T 1 ∑ t = 1 T U ( π σ ∗ ( t ) , ρ o b ; μ D ) and the receiver’s normalized payoff 1 ∑ t = 1 T γ t ∑ t = 1 T γ t V ( π σ ∗ ( t ) , ρ o b ; μ R ) \frac{1}{\sum_{t=1}^{T}\gamma^t}\sum_{t=1}^{T}\gamma^t V(\pi_{\sigma^*}^{(t)},\rho_{ob};\mu_R) ∑ t = 1 T γ t 1 ∑ t = 1 T γ t V ( π σ ∗ ( t ) , ρ o b ; μ R ) under the optimal DDP strategy as a function of γ \gamma γ . We observe that the designer’s payoff is increasing with γ \gamma γ , which implies that a more patient receiver leads to a higher dynamic benchmark for the designer. The receiver’s normalized payoff, on the other hand, is not monotone with respect to γ \gamma γ . When γ \gamma γ is small (smaller than 0.5 in this example), the dynamic benchmark 1 T U T ∗ ∗ \frac{1}{T}U_T^{**} T 1 U T ∗∗ coincides with the static Bayesian persuasion benchmark U B P ∗ U_{\mathrm{BP}}^* U BP ∗ .
Figure 3. Designer’s and receiver’s payoffs under the optimal DDP strategy, as a function of receiver’s discount factor γ \gamma γ .
More generally, the designer’s dynamic benchmark is weakly increasing in the receiver’s discount factor γ \gamma γ for any fixed instance I \mathcal{I} I and horizon T T T , as shown in Proposition 4. To see why, consider an optimal history-independent, dynamically persuasive DDP strategy σ γ ∗ \sigma_{\gamma}^* σ γ ∗ . A deviation gives the receiver a possible current payoff gain but causes him to lose access to information in all subsequent periods. The current payoff gain does not depend on his discount factor. To compare the future loss, note that dynamic persuasiveness ensures that, at the beginning of each period before punishment, continued obedience gives the receiver an expected discounted payoff at least as high as receiving no further information. Backward induction shows that increasing the discount factor weakly increases the discounted payoff loss from punishment. Thus, any deviation that was not attractive under γ \gamma γ remains unattractive under γ ′ > γ \gamma' > \gamma γ ′ > γ , so σ γ ∗ \sigma_{\gamma}^{*} σ γ ∗ remains dynamically persuasive. The designer can therefore use σ γ ∗ \sigma_{\gamma}^{*} σ γ ∗ with the more patient receiver and obtain the same payoff as before. An optimal strategy for γ ′ \gamma' γ ′ gives the designer at least as much payoff as continuing to use σ γ ∗ \sigma_{\gamma}^{*} σ γ ∗ . Thus, the dynamic benchmark is weakly increasing in γ \gamma γ .
Proposition 2. For any instance I \mathcal{I} I and any T ≥ 1 T \ge1 T ≥ 1 , the designer’s dynamic benchmark U T ∗ ∗ ( I ) U_{T}^{**}(\mathcal{I}) U T ∗∗ ( I ) is weakly increasing with the receiver’s discount factor γ \gamma γ .
Proof. The full proof is given in Appendix C.4.
Though the dynamic benchmark U T ∗ ∗ ( I ) U_{T}^{**}(\mathcal{I}) U T ∗∗ ( I ) is increasing in γ \gamma γ , the designer is not always better off in the learning problem with an unknown receiver’s belief μ R \mu_R μ R . The learning cost of O ( 1 1 − γ log 2 1 ε ) O\left(\frac{1}{1-\gamma}\log^2\frac{1}{\varepsilon}\right) O ( 1 − γ 1 log 2 ε 1 ) shown in Lemma 3 is also increasing in γ \gamma γ , as a more patient receiver has a larger incentive to manipulate the learning process of the designer. In fact, we will show in § 7 that the designer has to suffer a linear regret of Ω ( T ) \Omega(T) Ω ( T ) when the receiver is fully patient ( γ = 1 ) (\gamma= 1) ( γ = 1 ) .
The designer benefits in the dynamic benchmark compared to that in the static benchmark as T T T goes to infinity. To see that, in Figure 3 a, the x-axis intersects the designer’s average payoff at her single-period Bayesian persuasion optimality, while the highest horizontal line intersects the designer’s average payoff at her single-period optimality under IR. When T → ∞ T \to\infty T → ∞ , when the receiver converges to being fully patient γ → 1 \gamma\to1 γ → 1 , then the time-averaged dynamic benchmark converges to the designer’s single-period optimality under the receiver’s individual rationality constraint (for simplicity, it is named as the designer’s optimality under IR), while the designer’s optimality under IR is weakly larger than the designer’s single-period Bayesian persuasion optimality (i.e., the designer’s optimality under the constraint of the signaling scheme being persuasive for the receiver). To see why, the solution to the designer’s single-period Bayesian persuasion problem is a feasible solution to her single-period optimization under the receiver’s individual rationality constraint. When the receiver is not patient with γ → 0 \gamma\to0 γ → 0 , the time-averaged dynamic benchmark converges to the designer’s single-period Bayesian persuasion optimality. The idea is formally characterized in Proposition 3.
Proposition 3. With Assumptions 1 and 3, the designer’s dynamic benchmark satisfies
lim T → ∞ lim γ → 1 1 T U T ∗ ∗ ( I ; γ ) = U I R ∗ , \lim_{T \to\infty} \lim_{\gamma\to1} \frac{1}{T} U_T^{**}(\mathcal{I};\gamma) = U_{\mathrm{IR}}^*, T → ∞ lim γ → 1 lim T 1 U T ∗∗ ( I ; γ ) = U IR ∗ , where U I R ∗ U_{\mathrm{IR}}^* U IR ∗ is the designer’s single-period optimality under the constraint of the receiver’s individual rationality:
max π : Ω → Δ ( A ) U ( π , ρ o b ; μ D ) s.t. V ( π , ρ o b ; μ R ) ≥ V u n i n f o r m e d ( μ R ) . (21) \max_{\pi:\Omega\to\Delta(A)} U(\pi,\rho_{\mathrm{ob}};\mu_D) \quad\text{s.t.} \quad V(\pi,\rho_{\mathrm{ob}};\mu_R) \ge V_{\mathrm{uninformed}}(\mu_R). \tag*{(21)} π : Ω → Δ ( A ) max U ( π , ρ ob ; μ D ) s.t. V ( π , ρ ob ; μ R ) ≥ V uninformed ( μ R ) . (21)
Proof. See Appendix C.5.
Learning to Achieve No Regret Against the Dynamic Benchmark
For the dynamic benchmark, the learning algorithm again consists of two phases. The first is an exploration phase for learning the receiver’s prior belief, which is based on the same algorithm as for the static benchmark but with different parameters. For the second phase, we propose a new robustification process, dynamic robustification , to robustify a designer-optimal strategy computed from the learned prior, which ensures approximate optimality for the designer under the receiver’s belief μ R \mu_R μ R . In this section, we first study the dynamic robustification based on the estimated belief. Then, we show the full algorithm to achieve no regret for the dynamic benchmark.
Dynamic Robustification for DDP Strategy
Denote by σ ^ = ( π σ ^ ( 1 ) , … , π σ ^ ( T ) ) \hat{\sigma} = (\pi_{\hat{\sigma}}^{(1)}, \ldots, \pi_{\hat{\sigma}}^{(T)}) σ ^ = ( π σ ^ ( 1 ) , … , π σ ^ ( T ) ) the designer’s optimal DDP strategy, which achieves the dynamic benchmark U T ∗ ∗ ( Ω , A , u , v , μ D , μ ^ ) U_T^{**}(\Omega, A, u, v, \mu_D, \hat{\mu}) U T ∗∗ ( Ω , A , u , v , μ D , μ ^ ) under belief μ ^ \hat{\mu} μ ^ . Proposition 1 implies that the strategy σ ^ \hat{\sigma} σ ^ is history-independent and dynamically persuasive for μ ^ \hat{\mu} μ ^ . We show that σ ^ \hat{\sigma} σ ^ can be dynamically robustified into another DDP strategy σ ~ = ( π ~ σ ^ ( 1 ) , … , π ~ σ ^ ( T ) ) \tilde{\sigma}=(\tilde{\pi}_{\hat{\sigma}}^{(1)},\ldots,\tilde{\pi}_{\hat{\sigma}}^{(T)}) σ ~ = ( π ~ σ ^ ( 1 ) , … , π ~ σ ^ ( T ) ) that is history-independent and dynamically persuasive for a strategic receiver with any belief μ R \mu_R μ R satisfying ∥ μ R − μ ^ ∥ 1 ≤ ε \lVert\mu_R-\hat{\mu}\rVert_1\le\varepsilon ∥ μ R − μ ^ ∥ 1 ≤ ε .
We first show how to convert any signaling scheme π σ ^ ( t ) \pi_{\hat{\sigma}}^{(t)} π σ ^ ( t ) for any period t t t from the designer’s optimal DDP strategy σ ^ \hat{\sigma} σ ^ under belief μ ^ \hat{\mu} μ ^ to a corresponding signaling scheme π σ ~ ( t ) \pi_{\tilde{\sigma}}^{(t)} π σ ~ ( t ) .
A natural idea is to apply static robustification (Lemma 4) to the signaling scheme π σ ^ ( t ) \pi_{\hat{\sigma}}^{(t)} π σ ^ ( t ) in each period t t t in the DDP strategy σ ^ \hat{\sigma} σ ^ . Recall that static robustification mixes over signaling schemes and thereby increases the persuasiveness of the statically robustified signaling scheme under the receiver’s prior belief μ R \mu_R μ R , i.e.,
E ω ∼ μ R ( ⋅ ∣ a , π ~ ( t ) ) [ v ( a , ω ) − v ( a ′ , ω ) ] ≥ min { E ω ∼ μ ^ ( ⋅ ∣ a , π σ ^ ( t ) ) [ v ( a , ω ) − v ( a ′ , ω ) ] + O ( δ ) , 0 } , ∀ a ′ ∈ A ∖ { a } . \mathbb{E}_{\omega\sim\mu_R(\cdot\mid a,\tilde{\pi}^{(t)})}\left[v(a,\omega)-v(a',\omega)\right]\ge\min\left\{\mathbb{E}_{\omega\sim\hat{\mu}(\cdot\mid a,\pi_{\hat{\sigma}}^{(t)})}\left[v(a,\omega)-v(a',\omega)\right]+O(\delta),0\right\},\quad\forall a'\in A\setminus\{a\}. E ω ∼ μ R ( ⋅ ∣ a , π ~ ( t ) ) [ v ( a , ω ) − v ( a ′ , ω ) ] ≥ min { E ω ∼ μ ^ ( ⋅ ∣ a , π σ ^ ( t ) ) [ v ( a , ω ) − v ( a ′ , ω ) ] + O ( δ ) , 0 } , ∀ a ′ ∈ A ∖ { a } .
where π ~ ( t ) \tilde{\pi}^{(t)} π ~ ( t ) is the signaling scheme statically robustified from π σ ^ ( t ) \pi_{\hat{\sigma}}^{(t)} π σ ^ ( t ) . Thus, the persuasiveness of the statically robustified signaling scheme improves only after the receiver receives a recommendation. Before the receiver receives a recommendation, however, static robustification may lower the receiver’s ex ante payoff from obedience, i.e., V ( π ~ ( t ′ ) , ρ o b ; μ R ) < V ( π σ ^ ( t ′ ) , ρ o b ; μ R ) V(\tilde{\pi}^{(t')},\rho_{\mathrm{ob}};\mu_R)<V(\pi_{\hat{\sigma}}^{(t')},\rho_{\mathrm{ob}};\mu_R) V ( π ~ ( t ′ ) , ρ ob ; μ R ) < V ( π σ ^ ( t ′ ) , ρ ob ; μ R ) for any t ′ > t t'>t t ′ > t . In period t t t , after a recommendation is sent by the statically robustified signaling scheme, the receiver’s incentive to obey the recommendation in the current period may become stronger. But the receiver’s ex ante payoff from obeying the recommendations in future periods onward may get worse. Thus, on balance, he may refuse to obey the current recommendation and leave the game. In that way, a DDP strategy obtained by statically robustifying the signaling scheme in each period is not guaranteed to be dynamically persuasive.
Hence, we propose a new type of robustification, payoff-improving robustification , which not only increases the persuasiveness of an unrobustified signaling scheme but also weakly increases the receiver’s ex ante payoff from being obedient.
Given a typical signaling scheme π σ ^ \pi_{\hat{\sigma}} π σ ^ in a DDP strategy σ ^ \hat{\sigma} σ ^ , the main idea of the payoff-improving robustification is as follows. We first compute the signaling scheme π σ ~ \pi_{\tilde{\sigma}} π σ ~ that is obtained from static robustification (Lemma 4) on π σ ^ \pi_{\hat{\sigma}} π σ ^ . Then, we mix π σ ~ \pi_{\tilde{\sigma}} π σ ~ with the full-revelation signaling scheme π f u l l \pi_{\mathrm{full}} π full , which always recommends the optimal action for the receiver at each state. In particular, if π σ ~ \pi_{\tilde{\sigma}} π σ ~ is already close to π f u l l \pi_{\mathrm{full}} π full (with distance less than O ( δ ) O(\sqrt{\delta}) O ( δ ) ), we directly change π σ ~ \pi_{\tilde{\sigma}} π σ ~ to π f u l l \pi_{\mathrm{full}} π full . If π σ ~ \pi_{\tilde{\sigma}} π σ ~ is far from π f u l l \pi_{\mathrm{full}} π full (with distance more than O ( δ ) O(\sqrt{\delta}) O ( δ ) ), we take a mixture ( 1 − O ( δ ) ) π σ ~ + O ( δ ) π f u l l (1-O(\sqrt{\delta}))\pi_{\tilde{\sigma}}+O(\sqrt{\delta})\pi_{\mathrm{full}} ( 1 − O ( δ )) π σ ~ + O ( δ ) π full . Such a mixture weakly improves the receiver’s utility (compared to π σ ~ \pi_{\tilde{\sigma}} π σ ~ ), improves the persuasiveness, while hurting the designer’s utility by at most O ( δ ) O(\sqrt{\delta}) O ( δ ) .
Lemma 9 shows the result of any signaling scheme from the dynamic robustification in a DDP strategy σ ^ \hat{\sigma} σ ^ . We omit the time index of the signaling scheme for simplicity. See Appendix § 12 .§ 1 for the full proof of Lemma 9.4
Lemma 9 (Payoff-improving robustification). Assume Assumptions 1, 2, and 3. For any signaling scheme π σ ^ \pi_{\hat{\sigma}} π σ ^ of the designer’s optimal DDP strategy under belief μ ^ \hat{\mu} μ ^ , with any belief μ \mu μ satisfying ∥ μ − μ ^ ∥ 1 ≤ ε ≤ p 0 4 D G 16 \lVert\mu-\hat{\mu}\rVert_1 \le\varepsilon\le\frac{p_0^4DG}{16} ∥ μ − μ ^ ∥ 1 ≤ ε ≤ 16 p 0 4 D G , with a dynamic robustification parameter δ ∈ [ 2 ε p 0 D , p 0 3 G 8 ] \delta\in\left[\frac{2\varepsilon}{p_0D},\frac{p_0^3G}{8}\right] δ ∈ [ p 0 D 2 ε , 8 p 0 3 G ] , the signaling scheme π σ ^ \pi_{\hat{\sigma}} π σ ^ can be converted into another signaling scheme π σ ~ \pi_{\tilde{\sigma}} π σ ~ such that:
(improved persuasiveness) the persuasiveness of π σ ~ \pi_{\tilde{\sigma}} π σ ~ and π σ ^ \pi_{\hat{\sigma}} π σ ^ for the receiver under beliefs μ \mu μ and μ ^ \hat{\mu} μ ^ , respectively, are close:*
E ω ∼ μ ( ⋅ ∣ a , π σ ~ ) [ v ( a , ω ) − v ( a ′ , ω ) ] ≥ min { E ω ∼ μ ^ ( ⋅ ∣ a , π σ ^ ) [ v ( a , ω ) − v ( a ′ , ω ) ] + δ D − 2 ε p 0 , 0 } , \mathbb{E}_{\omega\sim\mu(\cdot\mid a,\pi_{\tilde{\sigma}})}[v(a,\omega)-v(a',\omega)] \ge\min\left\{\mathbb{E}_{\omega\sim\hat{\mu}(\cdot\mid a,\pi_{\hat{\sigma}})}[v(a,\omega)-v(a',\omega)]+\delta D-\frac{2\varepsilon}{p_0},0\right\}, E ω ∼ μ ( ⋅ ∣ a , π σ ~ ) [ v ( a , ω ) − v ( a ′ , ω )] ≥ min { E ω ∼ μ ^ ( ⋅ ∣ a , π σ ^ ) [ v ( a , ω ) − v ( a ′ , ω )] + δ D − p 0 2 ε , 0 } , for any action a ∈ A a\in A a ∈ A recommended by π σ ~ \pi_{\tilde{\sigma}} π σ ~ and a ′ ∈ A ∖ { a } a'\in A\setminus\{a\} a ′ ∈ A ∖ { a } ;
(weakly improving the payoff of the receiver) the receiver’s payoff from obedience is weakly increased: V ( π σ ~ , ρ o b ; μ ) ≥ V ( π σ ^ , ρ o b ; μ ) V(\pi_{\tilde{\sigma}},\rho_{\mathrm{ob}};\mu)\ge V(\pi_{\hat{\sigma}},\rho_{\mathrm{ob}};\mu) V ( π σ ~ , ρ ob ; μ ) ≥ V ( π σ ^ , ρ ob ; μ ) ;
(small influence on the payoff of the designer) with an obedient receiver, the designer’s payoff has a small decrease: U ( π σ ~ , ρ o b ; μ D ) ≥ U ( π σ ^ , ρ o b ; μ D ) − 4 δ p 0 3 G U(\pi_{\tilde{\sigma}},\rho_{\mathrm{ob}};\mu_D)\ge U(\pi_{\hat{\sigma}},\rho_{\mathrm{ob}};\mu_D)-4\sqrt{\frac{\delta}{p_0^3G}} U ( π σ ~ , ρ ob ; μ D ) ≥ U ( π σ ^ , ρ ob ; μ D ) − 4 p 0 3 G δ .
We then apply the above payoff-improving robustification procedure to every signaling scheme π σ ^ ( t ) \pi_{\hat{\sigma}}^{(t)} π σ ^ ( t ) in the designer-optimal DDP strategy σ ^ \hat{\sigma} σ ^ for belief μ ^ \hat{\mu} μ ^ , a process we call dynamic robustification . The result of the complete dynamic robustification is given in Theorem 2.
Theorem 2 (Dynamic robustification). Assume Assumptions 1, 2, and 3. Given the designer’s optimal DDP strategy σ ^ \hat{\sigma} σ ^ under belief μ ^ \hat{\mu} μ ^ , with any belief μ R \mu_R μ R satisfying ∥ μ R − μ ^ ∥ 1 ≤ ε \lVert\mu_R-\hat{\mu}\rVert_1\le\varepsilon ∥ μ R − μ ^ ∥ 1 ≤ ε for ε ≤ p 0 4 1 + p 0 γ / ( 1 − γ ) D G 16 \varepsilon\le\frac{p_0^4}{1+p_0\gamma/(1-\gamma)}\frac{DG}{16} ε ≤ 1 + p 0 γ / ( 1 − γ ) p 0 4 16 D G , there exists another history-independent DDP strategy σ ~ = ( π σ ~ ( 1 ) , … , π σ ~ ( T ) ) \tilde{\sigma}=(\pi_{\tilde{\sigma}}^{(1)},\ldots,\pi_{\tilde{\sigma}}^{(T)}) σ ~ = ( π σ ~ ( 1 ) , … , π σ ~ ( T ) ) such that
We now give the intuition for Theorem 2. The high-level idea of our dynamic robustification procedure is to robustify the signaling scheme of each period t t t in the DDP strategy σ ^ \hat{\sigma} σ ^ backward from T T T . In particular, for each period t = T , T − 1 , … , 1 t = T, T-1, \ldots, 1 t = T , T − 1 , … , 1 , we convert π σ ~ ( t ) \pi_{\tilde{\sigma}}^{(t)} π σ ~ ( t ) to another signaling scheme π σ ~ ( t ) \pi_{\tilde{\sigma}}^{(t)} π σ ~ ( t ) by Lemma 9 while treating the dynamically robustified tail ( π σ ~ ( t + 1 ) , … , π σ ~ ( T ) ) (\pi_{\tilde{\sigma}}^{(t+1)}, \ldots, \pi_{\tilde{\sigma}}^{(T)}) ( π σ ~ ( t + 1 ) , … , π σ ~ ( T ) ) as fixed.
Given a recommended action a a a at time t t t and any deviation a ′ a' a ′ , Lemma 9 ensures the persuasiveness of π σ ~ ( t ) \pi_{\tilde{\sigma}}^{(t)} π σ ~ ( t ) under the belief μ R \mu_R μ R after robustification compared to the persuasiveness of π σ ~ ( t ) \pi_{\tilde{\sigma}}^{(t)} π σ ~ ( t ) under the estimate μ ^ \hat{\mu} μ ^ by
E ω ∼ μ R ( ⋅ ∣ a , π σ ~ ( t ) ) [ v ( a , ω ) − v ( a ′ , ω ) ] ≥ min { E ω ∼ μ ^ ( ⋅ ∣ a , π σ ~ ( t ) ) [ v ( a , ω ) − v ( a ′ , ω ) ] + δ t D − 2 ε p 0 , 0 } . \mathbb{E}_{\omega\sim\mu_R(\cdot\mid a,\pi_{\tilde{\sigma}}^{(t)})}\left[v(a,\omega)-v(a',\omega)\right] \ge\min\left\{\mathbb{E}_{\omega\sim\hat{\mu}(\cdot\mid a,\pi_{\tilde{\sigma}}^{(t)})}\left[v(a,\omega)-v(a',\omega)\right]+\delta_t D-\frac{2\varepsilon}{p_0},0\right\}. E ω ∼ μ R ( ⋅ ∣ a , π σ ~ ( t ) ) [ v ( a , ω ) − v ( a ′ , ω ) ] ≥ min { E ω ∼ μ ^ ( ⋅ ∣ a , π σ ~ ( t ) ) [ v ( a , ω ) − v ( a ′ , ω ) ] + δ t D − p 0 2 ε , 0 } .
where a buffer δ t D \delta_t D δ t D and a loss from the belief estimate 2 ε p 0 \frac{2\varepsilon}{p_0} p 0 2 ε are exerted on the persuasiveness of π σ ~ ( t ) \pi_{\tilde{\sigma}}^{(t)} π σ ~ ( t ) under the estimate μ ^ \hat{\mu} μ ^ .
But that is not enough. Then, we consider the dynamic persuasiveness constraint to secure the receiver’s obedience in period t t t . The dynamic persuasiveness in period t t t is
γ t E ω ∼ μ R ( ⋅ ∣ a , π σ ~ ( t ) ) [ v ( a , ω ) − v ( a ′ , ω ) ] ⏟ today’s incentive + ∑ t ′ = t + 1 T γ t ′ ( V ( π σ ~ ( t ′ ) , ρ o b ; μ R ) − V u n i n f o r m e d ( μ R ) ) ⏟ discounted future payoffs from obedience ≥ 0. (DP t ) (22) \underbrace{\gamma^t\mathbb{E}_{\omega\sim\mu_R(\cdot\mid a,\pi_{\tilde{\sigma}}^{(t)})}\left[v(a,\omega)-v(a',\omega)\right]}_{\text{today's incentive}}+\underbrace{\sum_{t'=t+1}^{T}\gamma^{t'}\left(V(\pi_{\tilde{\sigma}}^{(t')},\rho_{\mathrm{ob}};\mu_R)-V_{\mathrm{uninformed}}(\mu_R)\right)}_{\text{discounted future payoffs from obedience}}\ge0.\quad\text{(DP}_t\text{)} \tag*{(22)} today’s incentive γ t E ω ∼ μ R ( ⋅ ∣ a , π σ ~ ( t ) ) [ v ( a , ω ) − v ( a ′ , ω ) ] + discounted future payoffs from obedience t ′ = t + 1 ∑ T γ t ′ ( V ( π σ ~ ( t ′ ) , ρ ob ; μ R ) − V uninformed ( μ R ) ) ≥ 0. (DP t ) (22)
There are two intuitive cases. First, when the persuasiveness of π σ ~ ( t ) \pi_{\tilde{\sigma}}^{(t)} π σ ~ ( t ) under μ R \mu_R μ R is already non-negative, then the (already robustified) tail is dynamically persuasive, so the future accumulated payoffs from obedience is non-negative as well. Thus, the dynamic persuasiveness constraint (DP_t) holds immediately.
The second case is that the persuasiveness of π σ ~ ( t ) \pi_{\tilde{\sigma}}^{(t)} π σ ~ ( t ) under μ R \mu_R μ R is already negative. Then we consider the help from the already robustified tail, which comes from two sources: (i) robustifying the tail cannot hurt the receiver (Lemma 9), and (ii) V ( ⋅ , ⋅ ; ⋅ ) V(\cdot,\cdot;\cdot) V ( ⋅ , ⋅ ; ⋅ ) is 1-Lipschitz in the belief (Lemma 14 ), so the gap from μ ^ \hat{\mu} μ ^ to μ R \mu_R μ R costs at most 2 ε 2\varepsilon 2 ε per period. Putting the contribution from those two sources of the tail, the left-hand side of the dynamic persuasiveness constraint (DP_t) is at least
γ t ( δ t D − 2 ε p 0 ) − 2 ε γ t + 1 1 − γ . \gamma^t\left(\delta_tD-\frac{2\varepsilon}{p_0}\right)-\frac{2\varepsilon\gamma^{t+1}}{1-\gamma}. γ t ( δ t D − p 0 2 ε ) − 1 − γ 2 ε γ t + 1 .
Choosing the dynamic robustification parameter δ t = ( 1 p 0 + γ 1 − γ ) 2 ε D \delta_t=\left(\frac{1}{p_0}+\frac{\gamma}{1-\gamma}\right)\frac{2\varepsilon}{D} δ t = ( p 0 1 + 1 − γ γ ) D 2 ε , we obtain a non-negative right-hand side of (DP_t), so the dynamic persuasiveness constraint (DP_t) holds.
Starting from t = T t=T t = T and moving backward, the same argument secures (DP_t) for each period t t t . Thus the completely dynamically robustified strategy σ ~ \tilde{\sigma} σ ~ is dynamically persuasive for every μ R \mu_R μ R with ∥ μ R − μ ^ ∥ 1 ≤ ε \lVert\mu_R-\hat{\mu}\rVert_1\le\varepsilon ∥ μ R − μ ^ ∥ 1 ≤ ε .
By the upper bound of the cost from each robustification at each period t t t on the designer, 4 δ t p 0 3 G 4\sqrt{\frac{\delta_t}{p_0^3G}} 4 p 0 3 G δ t (Lemma 9), as well as our choice of δ t \delta_t δ t above, we have
U T ( σ ~ , ϕ o b ; μ D ) ≥ U T ( σ ^ , ϕ o b ; μ D ) − 4 T ( 1 p 0 + γ 1 − γ ) 2 ε p 0 3 G D . U_T(\widetilde{\sigma},\phi_{\mathrm{ob}};\mu_D)\ge U_T(\widehat{\sigma},\phi_{\mathrm{ob}};\mu_D)-4T\sqrt{\left(\frac{1}{p_0}+\frac{\gamma}{1-\gamma}\right)\frac{2\varepsilon}{p_0^3GD}}. U T ( σ , ϕ ob ; μ D ) ≥ U T ( σ , ϕ ob ; μ D ) − 4 T ( p 0 1 + 1 − γ γ ) p 0 3 G D 2 ε .
See the full proof in Appendix § 12 .§ 2 .
Full Learning Algorithm with O ( log 2 T ) O(\log^2 T) O ( log 2 T ) Regret
The full algorithm for the learning performance relative to the dynamic benchmark is shown in Algorithm 5. Algorithm 5 has two phases: a short learn-the-prior phase and a long exploitation phase. The first phase is learning the belief μ R \mu_R μ R , which is the same as in Algorithm 4, but with different choices of parameters. The second phase is the dynamic robustification.
Algorithm 5: Learning Algorithm for the Dynamic Benchmark
Parameter: ε > 0 \varepsilon> 0 ε > 0
Run Algorithm 3 with parameter ε \varepsilon ε to obtain an estimate μ ^ \widehat{\mu} μ of the receiver’s belief. Let T 0 T_0 T 0 be the number of periods taken.
Compute the optimal DDP strategy σ ^ \widehat{\sigma} σ for the ( T − T 0 ) (T-T_0) ( T − T 0 ) -period game assuming that the receiver has a prior belief μ ^ \widehat{\mu} μ (by using, e.g., the dynamic program in Lemma 8).
Apply dynamic robustification (Theorem 2) to σ ^ \widehat{\sigma} σ to obtain DDP strategy σ ~ \widetilde{\sigma} σ . Use σ ~ \widetilde{\sigma} σ for the remaining T − T 0 T-T_0 T − T 0 periods.
Theorem 3 shows the regret incurred by Algorithm 5 is at most in the order of O ( log 2 T ) O(\log^2 T) O ( log 2 T ) . We first provide a brief intuition for proof here. Algorithm 5 has two phases: a short learn-the-prior phase and a long exploitation phase. In the learning phase we run the estimation routine until the receiver’s belief is identified up to ℓ 1 \ell_1 ℓ 1 –error ε \varepsilon ε , which (even against a strategic receiver) takes
T 0 = O ( 1 1 − γ log 2 1 ε ) . T_0=O\left(\frac{1}{1-\gamma}\log^2\frac{1}{\varepsilon}\right). T 0 = O ( 1 − γ 1 log 2 ε 1 ) .
rounds. In the exploitation phase we compute the optimal DDP plan for μ ^ \widehat{\mu} μ and then apply dynamic robustification so that obedience remains optimal for any belief within ε \varepsilon ε of μ ^ \widehat{\mu} μ . Dynamic robustification preserves persuasiveness while incurring at most a T ⋅ O ( ε ) T\cdot O(\sqrt{\varepsilon}) T ⋅ O ( ε ) loss in the designer’s value. Choosing ε \varepsilon ε in the order of T − 2 T^{-2} T − 2 makes this second-phase loss negligible relative to T 0 T_0 T 0 , so the total dynamic regret is driven by the learning time, yielding
O ( 1 1 − γ log 2 T ) . O\left(\frac{1}{1-\gamma}\log^2 T\right). O ( 1 − γ 1 log 2 T ) .
Theorem 3. *Assume Assumptions 1, 2, 3, 4. Choose ε = p 0 4 1 + p 0 γ / ( 1 − γ ) D G 16 T 2 \varepsilon=\frac{p_0^4}{1+p_0\gamma/(1-\gamma)}\frac{DG}{16T^2} ε = 1 + p 0 γ / ( 1 − γ ) p 0 4 16 T 2 D G with γ ∈ ( 0 , 1 ) \gamma\in(0,1) γ ∈ ( 0 , 1 ) . The
dynamic regret of Algorithm 5 is at most
Reg d y n a m i c ( T ; A l g o r i t h m 5 , I ) ≤ O ( 1 1 − γ log 2 T ) . (23) \operatorname{Reg}^{\mathrm{dynamic}}(T; \mathit{Algorithm\ 5}, \mathcal{I}) \le O\left(\frac{1}{1-\gamma}\log^2 T\right). \tag*{(23)} Reg dynamic ( T ; Algorithm 5 , I ) ≤ O ( 1 − γ 1 log 2 T ) . (23)
Proof. The proof will use the following Lemma 10 (with proof given in Appendix § 12 .§ 3 ) to compare the dynamic benchmarks for T T T -period game and ( T − T 0 ) (T-T_0) ( T − T 0 ) -period game.
Lemma 10. For any integers T ≥ T 0 ≥ 0 T \ge T_0 \ge0 T ≥ T 0 ≥ 0 and beliefs μ D , μ ^ \mu_D, \hat{\mu} μ D , μ ^ ,
U T − T 0 ∗ ∗ ( μ D , μ ^ ) ≥ U T ∗ ∗ ( μ D , μ ^ ) − T 0 . \mathcal{U}^{**}_{T-T_0}(\mu_D,\hat{\mu}) \ge\mathcal{U}^{**}_T(\mu_D,\hat{\mu})-T_0. U T − T 0 ∗∗ ( μ D , μ ^ ) ≥ U T ∗∗ ( μ D , μ ^ ) − T 0 .
By the dynamic robustification result (Theorem 2), σ ~ \tilde{\sigma} σ ~ is dynamically persuasive for a receiver with prior μ R \mu_R μ R and approximately optimal for the designer in the ( T − T 0 ) (T-T_0) ( T − T 0 ) -period game under a prior μ ^ \hat{\mu} μ ^ :
U T − T 0 ( σ ~ , ϕ o b ; μ D ) ≥ U T − T 0 ( σ ^ , ϕ o b ; μ D ) − 4 ( T − T 0 ) ( 1 p 0 + γ 1 − γ ) 2 ε p 0 3 G D = U T − T 0 ∗ ∗ ( μ D , μ ^ ) − 4 ( T − T 0 ) ( 1 p 0 + γ 1 − γ ) 2 ε p 0 3 G D . \begin{aligned}
\mathcal{U}_{T-T_0}(\tilde{\sigma},\phi_{\mathrm{ob}};\mu_D) \ge\mathcal{U}_{T-T_0}(\hat{\sigma},\phi_{\mathrm{ob}};\mu_D)-4(T-T_0)\sqrt{\left(\frac{1}{p_0}+\frac{\gamma}{1-\gamma}\right)\frac{2\varepsilon}{p_0^3\mathcal{G}D}} \\
&= \mathcal{U}^{**}_{T-T_0}(\mu_D,\hat{\mu})-4(T-T_0)\sqrt{\left(\frac{1}{p_0}+\frac{\gamma}{1-\gamma}\right)\frac{2\varepsilon}{p_0^3\mathcal{G}D}}.
\end{aligned} U T − T 0 ( σ ~ , ϕ ob ; μ D ) ≥ U T − T 0 ( σ ^ , ϕ ob ; μ D ) − 4 ( T − T 0 ) ( p 0 1 + 1 − γ γ ) p 0 3 G D 2 ε = U T − T 0 ∗∗ ( μ D , μ ^ ) − 4 ( T − T 0 ) ( p 0 1 + 1 − γ γ ) p 0 3 G D 2 ε .
where the equality is because σ ^ \hat{\sigma} σ ^ is optimal for the ( T − T 0 ) (T-T_0) ( T − T 0 ) -period game under receiver prior μ ^ \hat{\mu} μ ^ . Using Lemma 10, the total utility of the designer in T T T periods is at least
U T ≥ 0 + U T − T 0 ( σ ~ , ϕ o b ; μ D ) ≥ U T ∗ ∗ ( μ D , μ ^ ) − T 0 − 4 ( T − T 0 ) ( 1 p 0 + γ 1 − γ ) 2 ε p 0 3 G D . \mathcal{U}_T \ge0+\mathcal{U}_{T-T_0}(\tilde{\sigma},\phi_{\mathrm{ob}};\mu_D) \ge\mathcal{U}^{**}_T(\mu_D,\hat{\mu})-T_0-4(T-T_0)\sqrt{\left(\frac{1}{p_0}+\frac{\gamma}{1-\gamma}\right)\frac{2\varepsilon}{p_0^3\mathcal{G}D}}. U T ≥ 0 + U T − T 0 ( σ ~ , ϕ ob ; μ D ) ≥ U T ∗∗ ( μ D , μ ^ ) − T 0 − 4 ( T − T 0 ) ( p 0 1 + 1 − γ γ ) p 0 3 G D 2 ε .
Applying dynamic robustification (Theorem 2) to the optimal DDP strategy under receiver prior μ R \mu_R μ R , we obtain an approximately optimal DDP strategy under receiver prior μ ^ \hat{\mu} μ ^ , so U T ∗ ∗ ( μ D , μ ^ ) ≥ U T ∗ ∗ ( μ D , μ R ) − 4 T ( 1 p 0 + γ 1 − γ ) 2 ε p 0 3 G D \mathcal{U}^{**}_T(\mu_D,\hat{\mu}) \ge\mathcal{U}^{**}_T(\mu_D,\mu_R)-4T\sqrt{\left(\frac{1}{p_0}+\frac{\gamma}{1-\gamma}\right)\frac{2\varepsilon}{p_0^3\mathcal{G}D}} U T ∗∗ ( μ D , μ ^ ) ≥ U T ∗∗ ( μ D , μ R ) − 4 T ( p 0 1 + 1 − γ γ ) p 0 3 G D 2 ε . Thus, we have
U T ≥ U T ∗ ∗ ( μ D , μ R ) − 4 T ( 1 p 0 + γ 1 − γ ) 2 ε p 0 3 G D − T 0 − 4 ( T − T 0 ) ( 1 p 0 + γ 1 − γ ) 2 ε p 0 3 G D ≥ U T ∗ ∗ ( μ D , μ R ) − T 0 − 8 T ( 1 p 0 + γ 1 − γ ) 2 ε p 0 3 G D . \begin{aligned}
\mathcal{U}_T \ge\mathcal{U}^{**}_T(\mu_D,\mu_R)-4T\sqrt{\left(\frac{1}{p_0}+\frac{\gamma}{1-\gamma}\right)\frac{2\varepsilon}{p_0^3\mathcal{G}D}}-T_0-4(T-T_0)\sqrt{\left(\frac{1}{p_0}+\frac{\gamma}{1-\gamma}\right)\frac{2\varepsilon}{p_0^3\mathcal{G}D}} \\
&\ge\mathcal{U}^{**}_T(\mu_D,\mu_R)-T_0-8T\sqrt{\left(\frac{1}{p_0}+\frac{\gamma}{1-\gamma}\right)\frac{2\varepsilon}{p_0^3\mathcal{G}D}}.
\end{aligned} U T ≥ U T ∗∗ ( μ D , μ R ) − 4 T ( p 0 1 + 1 − γ γ ) p 0 3 G D 2 ε − T 0 − 4 ( T − T 0 ) ( p 0 1 + 1 − γ γ ) p 0 3 G D 2 ε ≥ U T ∗∗ ( μ D , μ R ) − T 0 − 8 T ( p 0 1 + 1 − γ γ ) p 0 3 G D 2 ε .
Choose ε = p 0 4 1 + p 0 γ / ( 1 − γ ) D G 16 T 2 \varepsilon=\frac{p_0^4}{1+p_0\gamma/(1-\gamma)}\frac{D G}{16T^2} ε = 1 + p 0 γ / ( 1 − γ ) p 0 4 16 T 2 D G , which satisfies the requirement for ε \varepsilon ε in Theorem 2. By Lemma 3,
T 0 ≤ 2 ∣ Ω ∣ ( 1 p 0 + 1 1 − γ ln ( 24 ∣ Ω ∣ ( 1 − γ ) G 4 p 0 6 ε ) ) log 2 24 ∣ Ω ∣ G 2 p 0 ε . T_0 \le2|\Omega|\left(\frac{1}{p_0}+\frac{1}{1-\gamma}\ln\left(\frac{24|\Omega|}{(1-\gamma)G^4p_0^6\varepsilon}\right)\right)\log_2\frac{24|\Omega|}{G^2p_0\varepsilon}. T 0 ≤ 2∣Ω∣ ( p 0 1 + 1 − γ 1 ln ( ( 1 − γ ) G 4 p 0 6 ε 24∣Ω∣ ) ) log 2 G 2 p 0 ε 24∣Ω∣ .
Reg d y n a m i c ( T ; A l g o r i t h m 5 , I ) = T 0 + 8 T ( 1 p 0 + γ 1 − γ ) 2 ε p 0 3 G D ≤ 2 ∣ Ω ∣ ( 1 p 0 + 1 1 − γ ln ( 24 ∣ Ω ∣ ( 1 − γ ) G 4 p 0 6 ε ) ) log 2 24 ∣ Ω ∣ G 2 p 0 ε + 8 T ( 1 p 0 + γ 1 − γ ) 2 ε p 0 3 G D = O ( 1 1 − γ log 2 T ) . \begin{aligned}
\operatorname{Reg}^{\mathrm{dynamic}}(T;\mathrm{Algorithm}\ 5,\mathcal{I}) &= T_0+8T\sqrt{\left(\frac{1}{p_0}+\frac{\gamma}{1-\gamma}\right)\frac{2\varepsilon}{p_0^3GD}}\\
&\le2|\Omega|\left(\frac{1}{p_0}+\frac{1}{1-\gamma}\ln\left(\frac{24|\Omega|}{(1-\gamma)G^4p_0^6\varepsilon}\right)\right)\log_2\frac{24|\Omega|}{G^2p_0\varepsilon}+8T\sqrt{\left(\frac{1}{p_0}+\frac{\gamma}{1-\gamma}\right)\frac{2\varepsilon}{p_0^3GD}}\\
&=O\left(\frac{1}{1-\gamma}\log^2 T\right).
\end{aligned} Reg dynamic ( T ; Algorithm 5 , I ) = T 0 + 8 T ( p 0 1 + 1 − γ γ ) p 0 3 G D 2 ε ≤ 2∣Ω∣ ( p 0 1 + 1 − γ 1 ln ( ( 1 − γ ) G 4 p 0 6 ε 24∣Ω∣ ) ) log 2 G 2 p 0 ε 24∣Ω∣ + 8 T ( p 0 1 + 1 − γ γ ) p 0 3 G D 2 ε = O ( 1 − γ 1 log 2 T ) .
Interestingly, the regret bounds of our two learning algorithms, Algorithms 4 and 5 (against the static benchmark and the dynamic benchmark, respectively) are both logarithmic in time horizon: O ( log 2 T ) O(\log^2 T) O ( log 2 T ) . While empirical estimation by observing the frequency distribution of states is not relevant here, as the receiver’s belief is subjective, we note that our algorithm gives faster learning. Learning a distribution via empirical estimation leads to regret at most O ( T ) O(\sqrt{T}) O ( T ) , which is exponentially worse than our regret of O ( log 2 T ) O(\log^2 T) O ( log 2 T ) .
Necessity of Receiver’s Discount: Lower Bound on Regrets
In this section, we prove that the assumption that the receiver has a discount factor of γ < 1 \gamma<1 γ < 1 is necessary for the designer to achieve no regret asymptotically (or, sub-linear regret). In particular, if the receiver has a discount factor γ = 1 \gamma=1 γ = 1 , then no matter what learning algorithm is used by the designer, there always exists an information design instance such that the designer’s regret is at least a constant times T T T .
We now introduce the notion of regret lower bound . A regret lower bound means that for any algorithm G \mathcal{G} G , there exists some instance I \mathcal{I} I , i.e., for some unknown prior μ R \mu_R μ R , such that
Reg ( T ; G , I ) ≥ c I ⋅ g ( T ) = Ω ( g ( T ) ) , ∀ T , \operatorname{Reg}(T;\mathcal{G},\mathcal{I})\ge c_{\mathcal{I}}\cdot g(T)=\Omega(g(T)),\qquad\forall T, Reg ( T ; G , I ) ≥ c I ⋅ g ( T ) = Ω ( g ( T )) , ∀ T ,
for some instance-dependent constant c I > 0 c_{\mathcal{I}}>0 c I > 0 . The function g ( T ) g(T) g ( T ) captures the unavoidable cost of not knowing μ R \mu_R μ R no matter what learning algorithm is used, which fundamentally shows how hard the problem of information design with an unknown prior is.
Denote by T γ = ∑ t = 1 T γ t T_\gamma=\sum_{t=1}^{T}\gamma^t T γ = ∑ t = 1 T γ t the receiver’s effective horizon . Theorem 4 shows that, with respect to both static benchmark and dynamic benchmark, the regret of any learning algorithm G \mathcal{G} G for the designer is lower bounded by a linear function of the receiver’s effective horizon .
Theorem 4 (Regret lower bound for both benchmarks). For any learning algorithm G \mathcal{G} G , there exists an instance I \mathcal{I} I with equal static and dynamic benchmarks U T ∗ ( I ) = U T ∗ ∗ ( I ) U_T^{*}(\mathcal{I})=U_T^{**}(\mathcal{I}) U T ∗ ( I ) = U T ∗∗ ( I ) such that the designer’s regret with respect to both benchmarks is at least Ω ( T γ ) \Omega(T_\gamma) Ω ( T γ ) . In particular,
for γ ∈ [ 0 , 1 − 1 T ) \gamma\in[0,1-\frac{1}{T}) γ ∈ [ 0 , 1 − T 1 ) , the designer’s regret is at least Ω ( 1 1 − γ ) \Omega\left(\frac{1}{1-\gamma}\right) Ω ( 1 − γ 1 ) .
for γ ∈ [ 1 − 1 T , 1 ] \gamma\in[1-\frac{1}{T},1] γ ∈ [ 1 − T 1 , 1 ] , the designer’s regret is at least Ω ( T ) \Omega(T) Ω ( T ) .
We remark that the instance above satisfies the regularity assumptions in Theorem 1, which implies that the designer’s regrets come from the receiver’s strategic decision-making instead of irregularities. Theorem 4 implies that the designer cannot obtain sub-linear regret against a patient receiver (γ = 1 \gamma=1 γ = 1 ). This shows the necessity of an impatient receiver in our model.
Proof of the Regret Lower Bound
The rest of this section proves Theorem 4. Our proof uses the idea of reduction . We will construct a family of new problems, which is a family of single-period information design games with menu , where the designer (not knowing the receiver’s prior belief) offers a menu of signaling schemes for the receiver to choose. In this single-period game, the designer’s regret is defined to be the difference between the optimal payoffs the designer can obtain with and without knowing the receiver’s prior. We then show that, if there exists a learning algorithm for the T T T -period information design game with regret smaller than c 0 ⋅ T γ c_0 \cdot T^\gamma c 0 ⋅ T γ , then there will be a menu for the single-period game with regret smaller than c 0 c_0 c 0 . However, because the designer does not know the receiver’s prior, we can show that no menu has regret smaller than some constant c 0 > 0 c_0>0 c 0 > 0 . Therefore, no learning algorithm has regret better than c 0 ⋅ T γ c_0 \cdot T^\gamma c 0 ⋅ T γ , thus proving a regret lower bound of Ω ( T γ ) \Omega(T^\gamma) Ω ( T γ ) .
Single-period information design problem with menu To prove Theorem 4, we start by constructing a family of single-period information design problems with menus . At a high level, these problems capture single-buyer single-seller auctions. Suppose there are two states Ω = { ω 0 , ω 1 } \Omega=\{\omega_0,\omega_1\} Ω = { ω 0 , ω 1 } and two actions A = { a 0 , a 1 } A=\{a_0,a_1\} A = { a 0 , a 1 } . The designer always prefers the receiver to take a 1 a_1 a 1 regardless of the state, i.e.,
u ( a 1 , ω ) = 1 , u ( a 0 , ω ) = 0 , ∀ ω ∈ Ω . (24) u(a_1,\omega)=1,\quad u(a_0,\omega)=0,\quad\forall\omega\in\Omega. \tag*{(24)} u ( a 1 , ω ) = 1 , u ( a 0 , ω ) = 0 , ∀ ω ∈ Ω. (24)
The receiver wants take an action to match the state, i.e.,
v ( a 0 , ω 0 ) = 0 , v ( a 0 , ω 1 ) = 0 , v ( a 1 , ω 0 ) = − 1 , v ( a 1 , ω 1 ) = 1. (25) v(a_0,\omega_0)=0,\quad v(a_0,\omega_1)=0,\quad v(a_1,\omega_0)=-1,\quad v(a_1,\omega_1)=1. \tag*{(25)} v ( a 0 , ω 0 ) = 0 , v ( a 0 , ω 1 ) = 0 , v ( a 1 , ω 0 ) = − 1 , v ( a 1 , ω 1 ) = 1. (25)
The designer’s prior belief μ D \mu_D μ D is
μ D ( ω 0 ) = 1 − p 0 , μ D ( ω 1 ) = p 0 , with 0 < p 0 < 1 / 2. (26) \mu_D(\omega_0)=1-p_0,\quad\mu_D(\omega_1)=p_0,\quad\text{with }0<p_0<1/2. \tag*{(26)} μ D ( ω 0 ) = 1 − p 0 , μ D ( ω 1 ) = p 0 , with 0 < p 0 < 1/2. (26)
The receiver’s prior belief μ R \mu_R μ R is parameterized by a number μ ~ R ∈ [ 0 , 1 ] \tilde{\mu}_R \in[0,1] μ ~ R ∈ [ 0 , 1 ] that is unknown to the designer. In particular,
μ R ( ω 0 ) = 1 1 + μ ~ R , μ R ( ω 1 ) = μ ~ R 1 + μ ~ R . (27) \mu_R(\omega_0) = \frac{1}{1+\tilde{\mu}_R}, \qquad\mu_R(\omega_1) = \frac{\tilde{\mu}_R}{1+\tilde{\mu}_R}. \tag*{(27)} μ R ( ω 0 ) = 1 + μ ~ R 1 , μ R ( ω 1 ) = 1 + μ ~ R μ ~ R . (27)
Consider a single-period game where the designer first elicits the receiver’s belief and then uses a direct signaling scheme to recommend an action to the receiver. Formally, the designer designs a menu , which is a family of direct signaling schemes M = { π μ } \mathcal{M} = \{\pi_\mu\} M = { π μ } . If the receiver reports belief μ \mu μ and obeys the recommendation from the signaling scheme π μ \pi_\mu π μ , then he gets the following single-period payoff
V s p ( M , μ R , μ ) = ∑ ω ∈ Ω μ R ( ω ) ∑ a ∈ A π μ ( a ∣ ω ) v ( a , ω ) = − μ R ( ω 0 ) π μ ( a 1 ∣ ω 0 ) + μ R ( ω 1 ) π μ ( a 1 ∣ ω 1 ) = 1 1 + μ ~ R ( μ ~ R π μ ( a 1 ∣ ω 1 ) − π μ ( a 1 ∣ ω 0 ) ) . (28) \begin{aligned}
V^{\mathrm{sp}}(\mathcal{M},\mu_R,\mu) = \sum_{\omega\in\Omega}\mu_R(\omega)\sum_{a\in A}\pi_\mu(a\mid\omega)v(a,\omega) \\
&= -\mu_R(\omega_0)\pi_\mu(a_1\mid\omega_0)+\mu_R(\omega_1)\pi_\mu(a_1\mid\omega_1) \\
&= \frac{1}{1+\tilde{\mu}_R}\left(\tilde{\mu}_R\pi_\mu(a_1\mid\omega_1)-\pi_\mu(a_1\mid\omega_0)\right). \tag*{(28)}
\end{aligned} V sp ( M , μ R , μ ) = ω ∈ Ω ∑ μ R ( ω ) a ∈ A ∑ π μ ( a ∣ ω ) v ( a , ω ) = − μ R ( ω 0 ) π μ ( a 1 ∣ ω 0 ) + μ R ( ω 1 ) π μ ( a 1 ∣ ω 1 ) = 1 + μ ~ R 1 ( μ ~ R π μ ( a 1 ∣ ω 1 ) − π μ ( a 1 ∣ ω 0 ) ) . (28)
Eq. (28) captures the payoff of a buyer in an auction. Specifically, the receiver’s prior belief μ ~ R \tilde{\mu}_R μ ~ R corresponds to the buyer’s private value of an item; π μ ( a 1 ∣ ω 1 ) \pi_\mu(a_1\mid\omega_1) π μ ( a 1 ∣ ω 1 ) captures the probability that the receiver gets the item; π μ ( a 1 ∣ ω 0 ) \pi_\mu(a_1\mid\omega_0) π μ ( a 1 ∣ ω 0 ) is the receiver’s payment. We say the menu M = { π μ } \mathcal{M} = \{\pi_\mu\} M = { π μ } is
incentive compatible if the receiver is willing to report his prior belief μ R \mu_R μ R (his private value) truthfully: ∀ μ R , ∀ μ , V s p ( M , μ R , μ R ) ≥ V s p ( M , μ R , μ ) \forall\mu_R,\ \forall\mu,\ V^{\mathrm{sp}}(\mathcal{M},\mu_R,\mu_R) \ge V^{\mathrm{sp}}(\mathcal{M},\mu_R,\mu) ∀ μ R , ∀ μ , V sp ( M , μ R , μ R ) ≥ V sp ( M , μ R , μ ) ;
individually rational if the receiver is weakly better off with the menu compared to receiving no information: ∀ μ R , V s p ( M , μ R , μ R ) ≥ V u n i n f o r m e d ( μ R ) = max a ∈ A ∑ ω ∈ Ω μ R ( ω ) v ( a , ω ) \forall\mu_R,\ V^{\mathrm{sp}}(\mathcal{M},\mu_R,\mu_R) \ge V^{\mathrm{uninformed}}(\mu_R) = \max_{a\in A}\sum_{\omega\in\Omega}\mu_R(\omega)v(a,\omega) ∀ μ R , V sp ( M , μ R , μ R ) ≥ V uninformed ( μ R ) = max a ∈ A ∑ ω ∈ Ω μ R ( ω ) v ( a , ω ) .
Benchmark for the single-period game with menu If the designer knows the receiver’s prior belief μ R \mu_R μ R , then the designer does not need to offer a menu of multiple signaling schemes; she can just use the following optimal single-period signaling scheme:
π μ R ∗ ( a 1 ∣ ω 1 ) = 1 , π μ R ∗ ( a 0 ∣ ω 1 ) = 0 , π μ R ∗ ( a 1 ∣ ω 0 ) = μ ~ R , π μ R ∗ ( a 0 ∣ ω 0 ) = 1 − μ ~ R , (29) \pi^*_{\mu_R}(a_1\mid\omega_1)=1,\qquad\pi^*_{\mu_R}(a_0\mid\omega_1)=0,\qquad\pi^*_{\mu_R}(a_1\mid\omega_0)=\tilde{\mu}_R,\qquad\pi^*_{\mu_R}(a_0\mid\omega_0)=1-\tilde{\mu}_R, \tag*{(29)} π μ R ∗ ( a 1 ∣ ω 1 ) = 1 , π μ R ∗ ( a 0 ∣ ω 1 ) = 0 , π μ R ∗ ( a 1 ∣ ω 0 ) = μ ~ R , π μ R ∗ ( a 0 ∣ ω 0 ) = 1 − μ ~ R , (29)
where π μ R ∗ ( a 1 ∣ ω 1 ) = 1 \pi^*_{\mu_R}(a_1\mid\omega_1)=1 π μ R ∗ ( a 1 ∣ ω 1 ) = 1 means always allocating the item to the buyer, and π μ R ∗ ( a 1 ∣ ω 0 ) = μ ~ R \pi^*_{\mu_R}(a_1\mid\omega_0)=\tilde{\mu}_R π μ R ∗ ( a 1 ∣ ω 0 ) = μ ~ R means setting the payment equal to the buyer’s value μ ~ R \tilde{\mu}_R μ ~ R . See Appendix § 13 .§ 2 for a proof of the optimality of π μ R ∗ \pi^*_{\mu_R} π μ R ∗ . The optimal signaling scheme gives the designer an expected payoff of
U s p ∗ ( μ R ) = μ D ( ω 0 ) π μ R ∗ ( a 1 ∣ ω 0 ) + μ D ( ω 1 ) π μ R ∗ ( a 1 ∣ ω 1 ) = ( 1 − p 0 ) μ ~ R + p 0 . (30) U^*_{\mathrm{sp}}(\mu_R)=\mu_D(\omega_0)\pi^*_{\mu_R}(a_1\mid\omega_0)+\mu_D(\omega_1)\pi^*_{\mu_R}(a_1\mid\omega_1)=(1-p_0)\tilde{\mu}_R+p_0. \tag*{(30)} U sp ∗ ( μ R ) = μ D ( ω 0 ) π μ R ∗ ( a 1 ∣ ω 0 ) + μ D ( ω 1 ) π μ R ∗ ( a 1 ∣ ω 1 ) = ( 1 − p 0 ) μ ~ R + p 0 . (30)
Here U s p ∗ ( μ R ) U^*_{\mathrm{sp}}(\mu_R) U sp ∗ ( μ R ) is the benchmark for the single-period game with menu, against which we define the single-period regret.
Single-period regret When the designer does not know the receiver’s prior belief μ R \mu_R μ R and uses a certain incentive compatible and individually rational menu M \mathcal{M} M , she gets a single-period payoff of
U s p ( M , μ R ) = ( 1 − p 0 ) π μ R ( a 1 ∣ ω 0 ) + p 0 π μ R ( a 1 ∣ ω 1 ) . (31) U^{\mathrm{sp}}(\mathcal{M},\mu_R)=(1-p_0)\pi_{\mu_R}(a_1\mid\omega_0)+p_0\pi_{\mu_R}(a_1\mid\omega_1). \tag*{(31)} U sp ( M , μ R ) = ( 1 − p 0 ) π μ R ( a 1 ∣ ω 0 ) + p 0 π μ R ( a 1 ∣ ω 1 ) . (31)
Compared to the optimal payoff (30) , the designer’s single-period regret is
Reg s p ( M , μ R ) = ( 1 − p 0 ) μ ~ R + p 0 − U s p ( M , μ R ) . (32) \operatorname{Reg}^{\mathrm{sp}}(\mathcal{M},\mu_R)=(1-p_0)\widetilde{\mu}_R+p_0-U^{\mathrm{sp}}(\mathcal{M},\mu_R). \tag*{(32)} Reg sp ( M , μ R ) = ( 1 − p 0 ) μ R + p 0 − U sp ( M , μ R ) . (32)
We show that the single-period regret of any menu is at least 1 / 16 1/16 1/16 in the worst case.
Lemma 11. For any incentive compatible and individually rational menu M \mathcal{M} M , there exists an instance with p 0 = 1 8 p_0=\frac{1}{8} p 0 = 8 1 and μ ~ R ∈ [ 3 8 , 1 ] \widetilde{\mu}_R\in[\frac{3}{8},1] μ R ∈ [ 8 3 , 1 ] , which corresponds to a prior belief μ R \mu_R μ R satisfying the regularity Assumption 1, such that Reg s p ( M , μ R ) ≥ 1 16 \operatorname{Reg}^{\mathrm{sp}}(\mathcal{M},\mu_R)\ge\frac{1}{16} Reg sp ( M , μ R ) ≥ 16 1 .
Proof. See Appendix § 13 .§ 1 .
Reduction from T T T -period game to single-period game We then consider the T T T -period information design game that repeats the above single-period instance I \mathcal{I} I for T T T periods but without the menu. We will show that any learning algorithm for the T T T -period game with a small regret can be converted into a single-period menu with a small regret, which will then imply a lower bound for the former regret. Before presenting the conversion, we show that the static benchmark and dynamic benchmark for the T T T -period game (without menu) are the same and equal to T T T times the benchmark U s p ∗ ( μ R ) U_{\mathrm{sp}}^*(\mu_R) U sp ∗ ( μ R ) for the single-period game with menu:
Lemma 12. The benchmarks satisfy U T ∗ ( I ) = U T ∗ ∗ ( I ) = T ⋅ U s p ∗ ( μ R ) U_T^*(\mathcal{I})=U_T^{**}(\mathcal{I})=T\cdot U_{\mathrm{sp}}^*(\mu_R) U T ∗ ( I ) = U T ∗∗ ( I ) = T ⋅ U sp ∗ ( μ R ) .
Proof. See Appendix § 13 .§ 2 .
We now convert any learning algorithm G \mathcal{G} G of the designer for the T T T -period game (without menu) to a menu M \mathcal{M} M for the single-period game. Given learning algorithm G \mathcal{G} G , a strategic receiver with prior μ R \mu_R μ R best responds by a T T T -period strategy ϕ ∗ ( G , μ R ) = ( ϕ ( t ) ) t = 1 T \phi^*(\mathcal{G},\mu_R)=(\phi^{(t)})_{t=1}^T ϕ ∗ ( G , μ R ) = ( ϕ ( t ) ) t = 1 T . The pair ( G , ϕ ∗ ( G , μ R ) ) (\mathcal{G},\phi^*(\mathcal{G},\mu_R)) ( G , ϕ ∗ ( G , μ R )) generates a stochastic process ( π ( t ) , ρ ( t ) ) t = 1 T (\pi^{(t)},\rho^{(t)})_{t=1}^T ( π ( t ) , ρ ( t ) ) t = 1 T of the designer’s signaling schemes and the receiver’s single-period strategies. We consider the average probability σ μ R ( a ∣ ω ) \sigma_{\mu_R}(a\mid\omega) σ μ R ( a ∣ ω ) that the receiver takes action a a a conditioning on state ω \omega ω when the receiver uses ϕ ∗ ( G , μ R ) \phi^*(\mathcal{G},\mu_R) ϕ ∗ ( G , μ R ) in response to the designer’s algorithm G \mathcal{G} G , weighted by the receiver’s discount factor:
σ μ R ( a ∣ ω ) : = E ( π ( t ) , ρ ( t ) ) ∼ G , ϕ ∗ ( G , μ R ) [ 1 T γ ∑ t = 1 T γ t ∑ s ∈ S π ( t ) ( s ∣ ω ) ρ ( t ) ( a ∣ s ) ] . (33) \sigma_{\mu_R}(a\mid\omega):=\mathbb{E}_{(\pi^{(t)},\rho^{(t)})\sim\mathcal{G},\phi^*(\mathcal{G},\mu_R)}\left[\frac{1}{T\gamma}\sum_{t=1}^{T}\gamma^t\sum_{s\in S}\pi^{(t)}(s\mid\omega)\rho^{(t)}(a\mid s)\right]. \tag*{(33)} σ μ R ( a ∣ ω ) := E ( π ( t ) , ρ ( t ) ) ∼ G , ϕ ∗ ( G , μ R ) [ T γ 1 t = 1 ∑ T γ t s ∈ S ∑ π ( t ) ( s ∣ ω ) ρ ( t ) ( a ∣ s ) ] . (33)
Since σ μ R ( ⋅ ∣ ω ) \sigma_{\mu_R}(\cdot\mid\omega) σ μ R ( ⋅ ∣ ω ) is a distribution over the set of actions A A A , σ μ R \sigma_{\mu_R} σ μ R can be regarded as a direct signaling scheme. Consider the menu M = { σ μ R } μ R ∈ Δ ( Ω ) \mathcal{M} = \{\sigma_{\mu_R}\}_{\mu_R\in\Delta(\Omega)} M = { σ μ R } μ R ∈ Δ ( Ω ) of the direct signaling scheme σ μ R \sigma_{\mu_R} σ μ R associated with prior belief μ R \mu_R μ R . Because the receiver is best-responding, the menu M \mathcal{M} M constructed in this way must be incentive-compatible and individually rational for the receiver; this is reminiscent to a “revelation principle”, which we formalize below (with proof given in Appendix § 13 .§ 3 ):
Claim 1. The menu M = { σ μ R } μ R ∈ Δ ( Ω ) \mathcal{M} = \{\sigma_{\mu_R}\}_{\mu_R\in\Delta(\Omega)} M = { σ μ R } μ R ∈ Δ ( Ω ) is incentive compatible and individually rational.
Then, we prove that the designer’s T T T -period regret is at least her single-period regret times the receiver’s effective horizon T γ T_\gamma T γ , formalized in Lemma 13 . Intuitively, this is because the designer suffers a single-period regret of Reg s p ( M , μ R ) \operatorname{Reg}^{\mathrm{sp}}(\mathcal{M}, \mu_R) Reg sp ( M , μ R ) in every period where she has not fully learned the receiver’s prior belief, due to the receiver’s manipulation, and the receiver’s manipulation lasts for T γ T_\gamma T γ periods. See details in Appendix § 13 .§ 5 .
Lemma 13. The designer’s regret satisfies Reg ( T ; G , I ) ≥ T γ ⋅ Reg s p ( M , μ R ) \operatorname{Reg}(T; \mathcal{G}, \mathcal{I}) \ge T_\gamma\cdot\operatorname{Reg}^{\mathrm{sp}}(\mathcal{M}, \mu_R) Reg ( T ; G , I ) ≥ T γ ⋅ Reg sp ( M , μ R ) .
According to Lemma 11 , for any incentive compatible and individually rational menu M \mathcal{M} M , there exists an instance such that the designer’s single-period regret Reg s p ( M , μ R ) \operatorname{Reg}^{\mathrm{sp}}(\mathcal{M}, \mu_R) Reg sp ( M , μ R ) is at least 1 16 \frac{1}{16} 16 1 . That implies Reg ( T ; G , I ) ≥ 1 16 T γ \operatorname{Reg}(T; \mathcal{G}, \mathcal{I}) \ge\frac{1}{16}T_\gamma Reg ( T ; G , I ) ≥ 16 1 T γ , which proves Theorem 4.
Discussion
We study the information design problem when the information designer does not know the belief of a strategic receiver. We design learning algorithms so that the designer can learn the receiver’s belief through repeated interactions. Our learning algorithms are robust to the receiver’s strategic manipulation of the learning process of the designer.
We define regret relative to two benchmarks to measure the performance of the learning algorithms. The static benchmark is T T T times the single-period optimum for the designer for the known belief. The dynamic benchmark is the global optimality for the designer for the known belief. Our learning algorithms achieve no regret against both benchmarks at fast speeds of O ( log 2 T ) O(\log^2 T) O ( log 2 T ) . Our work thus provides a robust learning foundation for the information design problems with unknown subjective beliefs of strategic receivers.
We offer a brief discussion of several natural extensions. First, while we consider the receiver’s belief as a convergence outcome of his misspecified learning, it would be interesting to extend the analysis to consider a receiver who is learning about the distribution. Will the two players’ learning converge to the true distribution of the states? Or will their learning converge to a belief that is neither the true distribution of the states nor the receiver’s belief (μ R \mu_R μ R in the paper) formed by his own misspecified learning? Will any factors in the game, such as their patience comparisons, determine which learning outcome they will reach? Second, it is interesting to extend our results to consider a designer with a discount factor close to but strictly less than 1. Our results require the designer to be more patient than the receiver. But what if the designer is close to being patient, i.e., γ D \gamma_D γ D close to 1? And what if the designer is only more patient than the receiver but is still impatient, i.e., γ < γ D < 1 \gamma< \gamma_D < 1 γ < γ D < 1 ? Such questions may have interesting implications for real-world problems. For example, the online platform may have limited patience with a certain user because interactions require time, but time is valuable.
Third, our learning algorithms indeed enable the designer to achieve no regret at fast speeds. But our learning algorithms rely on using the receiver’s actions, which may contain the receiver’s privacy and security information. Can the designer still learn the receiver’s belief at a fast speed without learning the receiver’s privacy? And if learning the privacy cannot be avoided, can the designer use algorithms that unlearn the receiver’s privacy to make the learning reliable and trustworthy for the receiver?
Finally, we are also interested in whether our contributions, including the learning of a strategic receiver’s belief as well as the dynamic incentives of a strategic receiver, to the information design setting can be generalized or extended to other possible settings. We leave those as open questions for future work.
Acknowledgment Declaration of financial support or competing interests: None.
Useful Lemmas
Lemma 14 (Lipschitz continuity of utility). The designer’s and receiver’s expected utility functions, U ( π , ρ ; μ D ) U(\pi,\rho;\mu_D) U ( π , ρ ; μ D ) , V ( π , ρ ; μ R ) V(\pi,\rho;\mu_R) V ( π , ρ ; μ R ) , are:
∣ U ( π , ρ ; μ 1 ) − U ( π , ρ ; μ 2 ) ∣ ≤ ∥ μ 1 − μ 2 ∥ 1 . \left|U(\pi,\rho;\mu_1)-U(\pi,\rho;\mu_2)\right| \le\|\mu_1-\mu_2\|_1. ∣ U ( π , ρ ; μ 1 ) − U ( π , ρ ; μ 2 ) ∣ ≤ ∥ μ 1 − μ 2 ∥ 1 . ∣ U ( π 1 , ρ ; μ D ) − U ( π 2 , ρ ; μ D ) ∣ ≤ max ω ∈ Ω ∑ s ∈ S ∣ π 1 ( s ∣ ω ) − π 2 ( s ∣ ω ) ∣ . \left|U(\pi_1,\rho;\mu_D)-U(\pi_2,\rho;\mu_D)\right| \le\max_{\omega\in\Omega}\sum_{s\in S}\left|\pi_1(s\mid\omega)-\pi_2(s\mid\omega)\right|. ∣ U ( π 1 , ρ ; μ D ) − U ( π 2 , ρ ; μ D ) ∣ ≤ ω ∈ Ω max s ∈ S ∑ ∣ π 1 ( s ∣ ω ) − π 2 ( s ∣ ω ) ∣ . Analogous inequalities hold for V ( ⋅ , ρ ; ⋅ ) V(\cdot,\rho;\cdot) V ( ⋅ , ρ ; ⋅ ) .
Proof. Write
U ( π , ρ ; μ D ) = ∑ ω μ D ( ω ) ∑ s π ( s ∣ ω ) ∑ a ρ ( a ∣ s ) u ( a , ω ) . U(\pi,\rho;\mu_D)=\sum_{\omega}\mu_D(\omega)\sum_s\pi(s\mid\omega)\sum_a\rho(a\mid s)u(a,\omega). U ( π , ρ ; μ D ) = ω ∑ μ D ( ω ) s ∑ π ( s ∣ ω ) a ∑ ρ ( a ∣ s ) u ( a , ω ) . (Lipschitz in the prior). For fixed ( π , ρ ) (\pi,\rho) ( π , ρ ) , the inner expectation lies in [ 0 , 1 ] [0,1] [ 0 , 1 ] , hence
∣ U ( π , ρ ; μ 1 ) − U ( π , ρ ; μ 2 ) ∣ = ∣ ∑ ω ( μ 1 − μ 2 ) ( ω ) ∑ s , a π ( s ∣ ω ) ρ ( a ∣ s ) u ( a , ω ) ⏟ ∈ [ 0 , 1 ] ∣ ≤ ∑ ω ∣ μ 1 − μ 2 ∣ ( ω ) = ∥ μ 1 − μ 2 ∥ 1 . \left|U(\pi,\rho;\mu_1)-U(\pi,\rho;\mu_2)\right|
=\left|\sum_{\omega}(\mu_1-\mu_2)(\omega)\underbrace{\sum_{s,a}\pi(s\mid\omega)\rho(a\mid s)u(a,\omega)}_{\in[0,1]}\right|
\le\sum_{\omega}|\mu_1-\mu_2|(\omega)=\|\mu_1-\mu_2\|_1. ∣ U ( π , ρ ; μ 1 ) − U ( π , ρ ; μ 2 ) ∣ = ω ∑ ( μ 1 − μ 2 ) ( ω ) ∈ [ 0 , 1 ] s , a ∑ π ( s ∣ ω ) ρ ( a ∣ s ) u ( a , ω ) ≤ ω ∑ ∣ μ 1 − μ 2 ∣ ( ω ) = ∥ μ 1 − μ 2 ∥ 1 . (Lipschitz in the signaling scheme). For fixed ( ρ , μ D ) (\rho,\mu_D) ( ρ , μ D ) , set u ˉ s ( ω ) : = ∑ a ρ ( a ∣ s ) u ( a , ω ) ∈ [ 0 , 1 ] \bar{u}_s(\omega):=\sum_a\rho(a\mid s)u(a,\omega)\in[0,1] u ˉ s ( ω ) := ∑ a ρ ( a ∣ s ) u ( a , ω ) ∈ [ 0 , 1 ] . Then
U ( π 1 , ρ ; μ D ) − U ( π 2 , ρ ; μ D ) = ∑ ω μ D ( ω ) ∑ s ( π 1 − π 2 ) ( s ∣ ω ) u ˉ s ( ω ) . U(\pi_1,\rho;\mu_D)-U(\pi_2,\rho;\mu_D)=\sum_{\omega}\mu_D(\omega)\sum_s(\pi_1-\pi_2)(s\mid\omega)\bar{u}_s(\omega). U ( π 1 , ρ ; μ D ) − U ( π 2 , ρ ; μ D ) = ω ∑ μ D ( ω ) s ∑ ( π 1 − π 2 ) ( s ∣ ω ) u ˉ s ( ω ) . For each ω \omega ω , the inner term is the difference of expectations of a [ 0 , 1 ] [0,1] [ 0 , 1 ] -valued function under two distributions on S S S , so by the dual characterization of total variation,
∣ ∑ s ( π 1 − π 2 ) ( s ∣ ω ) u ˉ s ( ω ) ∣ ≤ TV ( π 1 ( ⋅ ∣ ω ) , π 2 ( ⋅ ∣ ω ) ) . \left|\sum_s(\pi_1-\pi_2)(s\mid\omega)\bar{u}_s(\omega)\right|\le\operatorname{TV}(\pi_1(\cdot\mid\omega),\pi_2(\cdot\mid\omega)). s ∑ ( π 1 − π 2 ) ( s ∣ ω ) u ˉ s ( ω ) ≤ TV ( π 1 ( ⋅ ∣ ω ) , π 2 ( ⋅ ∣ ω )) . Averaging over ω \omega ω (since μ D \mu_D μ D is a probability measure) gives the stated bound. Finally, TV ( p , q ) = 1 2 ∑ s ∣ p ( s ) − q ( s ) ∣ ≤ ∣ S ∣ 2 max s ∣ p ( s ) − q ( s ) ∣ \operatorname{TV}(p,q)=\frac{1}{2}\sum_s|p(s)-q(s)|\le\frac{|S|}{2}\max_s|p(s)-q(s)| TV ( p , q ) = 2 1 ∑ s ∣ p ( s ) − q ( s ) ∣ ≤ 2 ∣ S ∣ max s ∣ p ( s ) − q ( s ) ∣ yields the matrix ℓ 1 \ell_1 ℓ 1 -norm version. The proof for V V V is identical with u u u replaced by v v v and μ D \mu_D μ D by μ R \mu_R μ R .
Lemma 15 (Lipschitz continuity of posterior). Let π : Ω → Δ ( S ) \pi:\Omega\to\Delta(S) π : Ω → Δ ( S ) be any signaling scheme. Let μ , μ ′ ∈ Δ ( Ω ) \mu,\mu'\in\Delta(\Omega) μ , μ ′ ∈ Δ ( Ω ) be two priors. Let μ s , μ s ′ \mu_s,\mu'_s μ s , μ s ′ be the posterior belief induced by signal s s s under π \pi π and prior μ , μ ′ \mu,\mu' μ , μ ′ respectively. Suppose min ω ∈ Ω μ ( ω ) ≥ p 0 > 0 \min_{\omega\in\Omega}\mu(\omega)\ge p_0>0 min ω ∈ Ω μ ( ω ) ≥ p 0 > 0 . Then ∥ μ s − μ s ′ ∥ 1 ≤ 2 p 0 ∥ μ − μ ′ ∥ 1 \|\mu_s-\mu'_s\|_1\le\frac{2}{p_0}\|\mu-\mu'\|_1 ∥ μ s − μ s ′ ∥ 1 ≤ p 0 2 ∥ μ − μ ′ ∥ 1 .
Proof. Let π ( s ) = ∑ ω ∈ Ω μ ( ω ) π ( s ∣ ω ) \pi(s)=\sum_{\omega\in\Omega}\mu(\omega)\pi(s\mid\omega) π ( s ) = ∑ ω ∈ Ω μ ( ω ) π ( s ∣ ω ) and π ′ ( s ) = ∑ ω ∈ Ω μ ′ ( ω ) π ( s ∣ ω ) \pi'(s)=\sum_{\omega\in\Omega}\mu'(\omega)\pi(s\mid\omega) π ′ ( s ) = ∑ ω ∈ Ω μ ′ ( ω ) π ( s ∣ ω ) be the probability of signal s s s under prior μ \mu μ and μ ′ \mu' μ ′ respectively. By the definition of μ s , μ s ′ \mu_s,\mu'_s μ s , μ s ′ and by triangle inequality,
∥ μ s − μ s ′ ∥ 1 = ∑ ω ∈ Ω ∣ μ ( ω ) π ( s ∣ ω ) π ( s ) − μ ′ ( ω ) π ( s ∣ ω ) π ′ ( s ) ∣ ≤ ∑ ω ∈ Ω ∣ μ ( ω ) π ( s ∣ ω ) π ( s ) − μ ′ ( ω ) π ( s ∣ ω ) π ( s ) ∣ + ∑ ω ∈ Ω ∣ μ ′ ( ω ) π ( s ∣ ω ) π ( s ) − μ ′ ( ω ) π ( s ∣ ω ) π ′ ( s ) ∣ . \begin{aligned}
\lVert\mu_s-\mu'_s\rVert_1
&=\sum_{\omega\in\Omega}\left|\frac{\mu(\omega)\pi(s\mid\omega)}{\pi(s)}-\frac{\mu'(\omega)\pi(s\mid\omega)}{\pi'(s)}\right|\\
&\leq\sum_{\omega\in\Omega}\left|\frac{\mu(\omega)\pi(s\mid\omega)}{\pi(s)}-\frac{\mu'(\omega)\pi(s\mid\omega)}{\pi(s)}\right|+\sum_{\omega\in\Omega}\left|\frac{\mu'(\omega)\pi(s\mid\omega)}{\pi(s)}-\frac{\mu'(\omega)\pi(s\mid\omega)}{\pi'(s)}\right|.
\end{aligned} ∥ μ s − μ s ′ ∥ 1 = ω ∈ Ω ∑ π ( s ) μ ( ω ) π ( s ∣ ω ) − π ′ ( s ) μ ′ ( ω ) π ( s ∣ ω ) ≤ ω ∈ Ω ∑ π ( s ) μ ( ω ) π ( s ∣ ω ) − π ( s ) μ ′ ( ω ) π ( s ∣ ω ) + ω ∈ Ω ∑ π ( s ) μ ′ ( ω ) π ( s ∣ ω ) − π ′ ( s ) μ ′ ( ω ) π ( s ∣ ω ) .
For the first term above, ∑ ω ∈ Ω ∣ μ ( ω ) π ( s ∣ ω ) π ( s ) − μ ′ ( ω ) π ( s ∣ ω ) π ( s ) ∣ = ∑ ω ∈ Ω π ( s ∣ ω ) π ( s ) ∣ μ ( ω ) − μ ′ ( ω ) ∣ \sum_{\omega\in\Omega}\left|\frac{\mu(\omega)\pi(s\mid\omega)}{\pi(s)}-\frac{\mu'(\omega)\pi(s\mid\omega)}{\pi(s)}\right|=\sum_{\omega\in\Omega}\frac{\pi(s\mid\omega)}{\pi(s)}|\mu(\omega)-\mu'(\omega)| ∑ ω ∈ Ω π ( s ) μ ( ω ) π ( s ∣ ω ) − π ( s ) μ ′ ( ω ) π ( s ∣ ω ) = ∑ ω ∈ Ω π ( s ) π ( s ∣ ω ) ∣ μ ( ω ) − μ ′ ( ω ) ∣ . We note that, ∀ ω ∈ Ω \forall\omega\in\Omega ∀ ω ∈ Ω ,
π ( s ∣ ω ) π ( s ) = π ( s ∣ ω ) ∑ ω ′ ∈ Ω μ ( ω ′ ) π ( s ∣ ω ′ ) ≤ π ( s ∣ ω ) p 0 ∑ ω ′ ∈ Ω π ( s ∣ ω ′ ) ≤ 1 p 0 . (34) \frac{\pi(s\mid\omega)}{\pi(s)}
=\frac{\pi(s\mid\omega)}{\sum_{\omega'\in\Omega}\mu(\omega')\pi(s\mid\omega')}
\leq\frac{\pi(s\mid\omega)}{p_0\sum_{\omega'\in\Omega}\pi(s\mid\omega')}
\leq\frac{1}{p_0}. \tag*{(34)} π ( s ) π ( s ∣ ω ) = ∑ ω ′ ∈ Ω μ ( ω ′ ) π ( s ∣ ω ′ ) π ( s ∣ ω ) ≤ p 0 ∑ ω ′ ∈ Ω π ( s ∣ ω ′ ) π ( s ∣ ω ) ≤ p 0 1 . (34)
Thus, ∑ ω ∈ Ω ∣ μ ( ω ) π ( s ∣ ω ) π ( s ) − μ ′ ( ω ) π ( s ∣ ω ) π ( s ) ∣ ≤ ∑ ω ∈ Ω 1 p 0 ∣ μ ( ω ) − μ ′ ( ω ) ∣ = 1 p 0 ∥ μ − μ ′ ∥ 1 \sum_{\omega\in\Omega}\left|\frac{\mu(\omega)\pi(s\mid\omega)}{\pi(s)}-\frac{\mu'(\omega)\pi(s\mid\omega)}{\pi(s)}\right|\leq\sum_{\omega\in\Omega}\frac{1}{p_0}|\mu(\omega)-\mu'(\omega)|=\frac{1}{p_0}\lVert\mu-\mu'\rVert_1 ∑ ω ∈ Ω π ( s ) μ ( ω ) π ( s ∣ ω ) − π ( s ) μ ′ ( ω ) π ( s ∣ ω ) ≤ ∑ ω ∈ Ω p 0 1 ∣ μ ( ω ) − μ ′ ( ω ) ∣ = p 0 1 ∥ μ − μ ′ ∥ 1 .
For the second term,
∑ ω ∈ Ω ∣ μ ′ ( ω ) π ( s ∣ ω ) π ( s ) − μ ′ ( ω ) π ( s ∣ ω ) π ′ ( s ) ∣ = ∑ ω ∈ Ω μ ′ ( ω ) π ( s ∣ ω ) ∣ π ′ ( s ) − π ( s ) π ( s ) π ′ ( s ) ∣ = ∑ ω ∈ Ω μ ′ ( ω ) π ( s ∣ ω ) ∣ ∑ ω ′ ∈ Ω ( μ ′ ( ω ′ ) − μ ( ω ′ ) ) π ( s ∣ ω ′ ) π ( s ) π ′ ( s ) ∣ ≤ ∑ ω ∈ Ω μ ′ ( ω ) π ( s ∣ ω ) ∑ ω ′ ∈ Ω ∣ μ ′ ( ω ′ ) − μ ( ω ′ ) ∣ ⋅ max ω ′ ∈ Ω π ( s ∣ ω ′ ) π ( s ) π ′ ( s ) = ∥ μ ′ − μ ∥ 1 ∑ ω ∈ Ω μ ′ ( ω ) π ( s ∣ ω ) π ′ ( s ) max ω ′ ∈ Ω π ( s ∣ ω ′ ) π ( s ) ≤ by x r e f ‘ ( 33 ) ‘ ∥ μ ′ − μ ∥ 1 ∑ ω ∈ Ω μ ′ ( ω ) π ( s ∣ ω ) π ′ ( s ) 1 p 0 = 1 p 0 ∥ μ ′ − μ ∥ 1 . \begin{aligned}
\sum_{\omega\in\Omega}\left|\frac{\mu'(\omega)\pi(s\mid\omega)}{\pi(s)}-\frac{\mu'(\omega)\pi(s\mid\omega)}{\pi'(s)}\right|
&=\sum_{\omega\in\Omega}\mu'(\omega)\pi(s\mid\omega)\left|\frac{\pi'(s)-\pi(s)}{\pi(s)\pi'(s)}\right|\\
&=\sum_{\omega\in\Omega}\mu'(\omega)\pi(s\mid\omega)\left|\frac{\sum_{\omega'\in\Omega}(\mu'(\omega')-\mu(\omega'))\pi(s\mid\omega')}{\pi(s)\pi'(s)}\right|\\
&\leq\sum_{\omega\in\Omega}\mu'(\omega)\pi(s\mid\omega)\frac{\sum_{\omega'\in\Omega}|\mu'(\omega')-\mu(\omega')|\cdot\max_{\omega'\in\Omega}\pi(s\mid\omega')}{\pi(s)\pi'(s)}\\
&=\lVert\mu'-\mu\rVert_1\sum_{\omega\in\Omega}\frac{\mu'(\omega)\pi(s\mid\omega)}{\pi'(s)}\frac{\max_{\omega'\in\Omega}\pi(s\mid\omega')}{\pi(s)}\\
&\mathrel{\overset{\text{by }{xref}`(33)`}{\leq}}\lVert\mu'-\mu\rVert_1\sum_{\omega\in\Omega}\frac{\mu'(\omega)\pi(s\mid\omega)}{\pi'(s)}\frac{1}{p_0}\\
&=\frac{1}{p_0}\lVert\mu'-\mu\rVert_1.
\end{aligned} ω ∈ Ω ∑ π ( s ) μ ′ ( ω ) π ( s ∣ ω ) − π ′ ( s ) μ ′ ( ω ) π ( s ∣ ω ) = ω ∈ Ω ∑ μ ′ ( ω ) π ( s ∣ ω ) π ( s ) π ′ ( s ) π ′ ( s ) − π ( s ) = ω ∈ Ω ∑ μ ′ ( ω ) π ( s ∣ ω ) π ( s ) π ′ ( s ) ∑ ω ′ ∈ Ω ( μ ′ ( ω ′ ) − μ ( ω ′ )) π ( s ∣ ω ′ ) ≤ ω ∈ Ω ∑ μ ′ ( ω ) π ( s ∣ ω ) π ( s ) π ′ ( s ) ∑ ω ′ ∈ Ω ∣ μ ′ ( ω ′ ) − μ ( ω ′ ) ∣ ⋅ max ω ′ ∈ Ω π ( s ∣ ω ′ ) = ∥ μ ′ − μ ∥ 1 ω ∈ Ω ∑ π ′ ( s ) μ ′ ( ω ) π ( s ∣ ω ) π ( s ) max ω ′ ∈ Ω π ( s ∣ ω ′ ) ≤ by x r e f ‘ ( 33 ) ‘ ∥ μ ′ − μ ∥ 1 ω ∈ Ω ∑ π ′ ( s ) μ ′ ( ω ) π ( s ∣ ω ) p 0 1 = p 0 1 ∥ μ ′ − μ ∥ 1 .
Therefore, we obtain ∥ μ s − μ s ′ ∥ 1 ≤ 2 p 0 ∥ μ ′ − μ ∥ 1 \lVert\mu_s-\mu'_s\rVert_1\leq\frac{2}{p_0}\lVert\mu'-\mu\rVert_1 ∥ μ s − μ s ′ ∥ 1 ≤ p 0 2 ∥ μ ′ − μ ∥ 1 .
Useful Lemmas
Lemma 16 (coalescing signals). Let s 1 , s 2 ∈ S s_1,s_2\in S s 1 , s 2 ∈ S be two signals from a signaling scheme π \pi π , to which the receiver responds by the same action ρ ( s 1 ) = ρ ( s 2 ) ∈ A \rho(s_1)=\rho(s_2)\in A ρ ( s 1 ) = ρ ( s 2 ) ∈ A . Then, after coalescing signals s 1 s_1 s 1 and s 2 s_2 s 2 into s s s by letting π c ( s ∣ ω ) = π ( s 1 ∣ ω ) + π ( s 2 ∣ ω ) \pi^{c}(s\mid\omega)=\pi(s_1\mid\omega)+\pi(s_2\mid\omega) π c ( s ∣ ω ) = π ( s 1 ∣ ω ) + π ( s 2 ∣ ω ) , we have:
For any actions a , a ′ ∈ A a,a'\in A a , a ′ ∈ A , if the receiver’s posterior μ s 1 \mu_{s_1} μ s 1 under s 1 s_1 s 1 satisfies E ω ∼ μ s 1 [ v ( a , ω ) − v ( a ′ , ω ) ] ≥ X 1 \mathbb{E}_{\omega\sim\mu_{s_1}}[v(a,\omega)-v(a',\omega)]\geq X_1 E ω ∼ μ s 1 [ v ( a , ω ) − v ( a ′ , ω )] ≥ X 1 and the posterior μ s 2 \mu_{s_2} μ s 2 under s 2 s_2 s 2 satisfies E ω ∼ μ s 2 [ v ( a , ω ) − v ( a ′ , ω ) ] ≥ X 2 \mathbb{E}_{\omega\sim\mu_{s_2}}[v(a,\omega)-v(a',\omega)]\geq X_2 E ω ∼ μ s 2 [ v ( a , ω ) − v ( a ′ , ω )] ≥ X 2 ,
then the receiver’s posterior under s s s satisfies
E ω ∼ μ s [ v ( a , ω ) − v ( a ′ , ω ) ] ≥ min { X 1 , X 2 } . \mathbb{E}_{\omega\sim\mu_s}\left[v(a,\omega)-v(a',\omega)\right]\ge\min\{X_1,X_2\}. E ω ∼ μ s [ v ( a , ω ) − v ( a ′ , ω ) ] ≥ min { X 1 , X 2 } .
Proof. The expected utilities of the two players clearly do not change after coalescing. We then verify the second claim. By definition, the receiver’s posterior μ s \mu_s μ s satisfies
E ω ∼ μ s [ v ( a , ω ) − v ( a ′ , ω ) ] = ∑ ω ∈ Ω μ ^ ( ω ) π c ( s ∣ ω ) ∑ ω ′ ∈ Ω μ ^ ( ω ′ ) π c ( s ∣ ω ′ ) [ v ( a , ω ) − v ( a ′ , ω ) ] = ∑ ω ∈ Ω μ ^ ( ω ) π ( s 1 ∣ ω ) ∑ ω ′ ∈ Ω μ ^ ( ω ′ ) π c ( s ∣ ω ′ ) [ v ( a , ω ) − v ( a ′ , ω ) ] + ∑ ω ∈ Ω μ ^ ( ω ) π ( s 2 ∣ ω ) ∑ ω ′ ∈ Ω μ ^ ( ω ′ ) π c ( s ∣ ω ′ ) [ v ( a , ω ) − v ( a ′ , ω ) ] = ∑ ω ′ ∈ Ω μ ^ ( ω ′ ) π ( s 1 ∣ ω ′ ) ∑ ω ′ ∈ Ω μ ^ ( ω ′ ) π c ( s ∣ ω ′ ) ∑ ω ∈ Ω μ ^ ( ω ) π ( s 1 ∣ ω ) ∑ ω ′ ∈ Ω μ ^ ( ω ′ ) π ( s 1 ∣ ω ′ ) [ v ( a , ω ) − v ( a ′ , ω ) ] + ∑ ω ′ ∈ Ω μ ^ ( ω ′ ) π ( s 2 ∣ ω ′ ) ∑ ω ′ ∈ Ω μ ^ ( ω ′ ) π c ( s ∣ ω ′ ) ∑ ω ∈ Ω μ ^ ( ω ) π ( s 2 ∣ ω ) ∑ ω ′ ∈ Ω μ ^ ( ω ′ ) π ( s 2 ∣ ω ′ ) [ v ( a , ω ) − v ( a ′ , ω ) ] = λ ⋅ E ω ∼ μ s 1 [ v ( a , ω ) − v ( a ′ , ω ) ] + ( 1 − λ ) ⋅ E ω ∼ μ s 2 [ v ( a , ω ) − v ( a ′ , ω ) ] ≥ min { X 1 , X 2 } , \begin{aligned}
\mathbb{E}_{\omega\sim\mu_s}\left[v(a,\omega)-v(a',\omega)\right]
&=\sum_{\omega\in\Omega}\frac{\hat{\mu}(\omega)\pi^c(s\mid\omega)}{\sum_{\omega'\in\Omega}\hat{\mu}(\omega')\pi^c(s\mid\omega')}\left[v(a,\omega)-v(a',\omega)\right]\\
&=\sum_{\omega\in\Omega}\frac{\hat{\mu}(\omega)\pi(s_1\mid\omega)}{\sum_{\omega'\in\Omega}\hat{\mu}(\omega')\pi^c(s\mid\omega')}\left[v(a,\omega)-v(a',\omega)\right]\\
&\quad+\sum_{\omega\in\Omega}\frac{\hat{\mu}(\omega)\pi(s_2\mid\omega)}{\sum_{\omega'\in\Omega}\hat{\mu}(\omega')\pi^c(s\mid\omega')}\left[v(a,\omega)-v(a',\omega)\right]\\
&=\frac{\sum_{\omega'\in\Omega}\hat{\mu}(\omega')\pi(s_1\mid\omega')}{\sum_{\omega'\in\Omega}\hat{\mu}(\omega')\pi^c(s\mid\omega')}\sum_{\omega\in\Omega}\frac{\hat{\mu}(\omega)\pi(s_1\mid\omega)}{\sum_{\omega'\in\Omega}\hat{\mu}(\omega')\pi(s_1\mid\omega')}\left[v(a,\omega)-v(a',\omega)\right]\\
&\quad+\frac{\sum_{\omega'\in\Omega}\hat{\mu}(\omega')\pi(s_2\mid\omega')}{\sum_{\omega'\in\Omega}\hat{\mu}(\omega')\pi^c(s\mid\omega')}\sum_{\omega\in\Omega}\frac{\hat{\mu}(\omega)\pi(s_2\mid\omega)}{\sum_{\omega'\in\Omega}\hat{\mu}(\omega')\pi(s_2\mid\omega')}\left[v(a,\omega)-v(a',\omega)\right]\\
&=\lambda\cdot\mathbb{E}_{\omega\sim\mu_{s_1}}\left[v(a,\omega)-v(a',\omega)\right]+(1-\lambda)\cdot\mathbb{E}_{\omega\sim\mu_{s_2}}\left[v(a,\omega)-v(a',\omega)\right]\\
&\ge\min\{X_1,X_2\},
\end{aligned} E ω ∼ μ s [ v ( a , ω ) − v ( a ′ , ω ) ] = ω ∈ Ω ∑ ∑ ω ′ ∈ Ω μ ^ ( ω ′ ) π c ( s ∣ ω ′ ) μ ^ ( ω ) π c ( s ∣ ω ) [ v ( a , ω ) − v ( a ′ , ω ) ] = ω ∈ Ω ∑ ∑ ω ′ ∈ Ω μ ^ ( ω ′ ) π c ( s ∣ ω ′ ) μ ^ ( ω ) π ( s 1 ∣ ω ) [ v ( a , ω ) − v ( a ′ , ω ) ] + ω ∈ Ω ∑ ∑ ω ′ ∈ Ω μ ^ ( ω ′ ) π c ( s ∣ ω ′ ) μ ^ ( ω ) π ( s 2 ∣ ω ) [ v ( a , ω ) − v ( a ′ , ω ) ] = ∑ ω ′ ∈ Ω μ ^ ( ω ′ ) π c ( s ∣ ω ′ ) ∑ ω ′ ∈ Ω μ ^ ( ω ′ ) π ( s 1 ∣ ω ′ ) ω ∈ Ω ∑ ∑ ω ′ ∈ Ω μ ^ ( ω ′ ) π ( s 1 ∣ ω ′ ) μ ^ ( ω ) π ( s 1 ∣ ω ) [ v ( a , ω ) − v ( a ′ , ω ) ] + ∑ ω ′ ∈ Ω μ ^ ( ω ′ ) π c ( s ∣ ω ′ ) ∑ ω ′ ∈ Ω μ ^ ( ω ′ ) π ( s 2 ∣ ω ′ ) ω ∈ Ω ∑ ∑ ω ′ ∈ Ω μ ^ ( ω ′ ) π ( s 2 ∣ ω ′ ) μ ^ ( ω ) π ( s 2 ∣ ω ) [ v ( a , ω ) − v ( a ′ , ω ) ] = λ ⋅ E ω ∼ μ s 1 [ v ( a , ω ) − v ( a ′ , ω ) ] + ( 1 − λ ) ⋅ E ω ∼ μ s 2 [ v ( a , ω ) − v ( a ′ , ω ) ] ≥ min { X 1 , X 2 } ,
where λ = ∑ ω ′ ∈ Ω μ ^ ( ω ′ ) π ( s 1 ∣ ω ′ ) ∑ ω ′ ∈ Ω μ ^ ( ω ′ ) π c ( s ∣ ω ′ ) ∈ [ 0 , 1 ] \lambda=\frac{\sum_{\omega'\in\Omega}\hat{\mu}(\omega')\pi(s_1\mid\omega')}{\sum_{\omega'\in\Omega}\hat{\mu}(\omega')\pi^c(s\mid\omega')}\in[0,1] λ = ∑ ω ′ ∈ Ω μ ^ ( ω ′ ) π c ( s ∣ ω ′ ) ∑ ω ′ ∈ Ω μ ^ ( ω ′ ) π ( s 1 ∣ ω ′ ) ∈ [ 0 , 1 ] .
Omitted Proofs in Section 3
Proof of Lemma 1
Claim 2. During the binary search algorithm (Algorithm 1), the left endpoint always satisfies ℓ ( k ) ≤ ( 1 + ι ) π ∗ ( s 0 ∣ ω j ) π ∗ ( s 0 ∣ ω i ) \ell^{(k)}\le(1+\iota)\frac{\pi^*(s_0\mid\omega_j)}{\pi^*(s_0\mid\omega_i)} ℓ ( k ) ≤ ( 1 + ι ) π ∗ ( s 0 ∣ ω i ) π ∗ ( s 0 ∣ ω j ) , and the right endpoint always satisfies r ( k ) ≥ ( 1 − ι ) π ∗ ( s 0 ∣ ω j ) π ∗ ( s 0 ∣ ω i ) r^{(k)}\ge(1-\iota)\frac{\pi^*(s_0\mid\omega_j)}{\pi^*(s_0\mid\omega_i)} r ( k ) ≥ ( 1 − ι ) π ∗ ( s 0 ∣ ω i ) π ∗ ( s 0 ∣ ω j ) . Additionally, ℓ ( k ) ≤ r ( k ) \ell^{(k)}\le r^{(k)} ℓ ( k ) ≤ r ( k ) . (This claim includes the initial step where k = 0 k=0 k = 0 .)
Proof. First, we verify that the claim holds at the initial step k = 0 k=0 k = 0 . By definition, we have ℓ ( 0 ) = 0 ≤ ( 1 + ι ) π ∗ ( s 0 ∣ ω j ) π ∗ ( s 0 ∣ ω i ) \ell^{(0)}=0\le(1+\iota)\frac{\pi^*(s_0\mid\omega_j)}{\pi^*(s_0\mid\omega_i)} ℓ ( 0 ) = 0 ≤ ( 1 + ι ) π ∗ ( s 0 ∣ ω i ) π ∗ ( s 0 ∣ ω j ) , and r ( 0 ) = 1 G p 0 r^{(0)}=\frac{1}{G_{p_0}} r ( 0 ) = G p 0 1 . Because by definition a ~ ω j , ω i \tilde{a}_{\omega_j,\omega_i} a ~ ω j , ω i is the first indifference action between states ω i \omega_i ω i and ω j \omega_j ω j , a ~ ω j , ω i \tilde{a}_{\omega_j,\omega_i} a ~ ω j , ω i is weakly better than a ω j ∗ a^*_{\omega_j} a ω j ∗ when the signaling ratio is π ∗ ( s 0 ∣ ω j ) π ∗ ( s 0 ∣ ω i ) \frac{\pi^*(s_0\mid\omega_j)}{\pi^*(s_0\mid\omega_i)} π ∗ ( s 0 ∣ ω i ) π ∗ ( s 0 ∣ ω j ) for a myopic receiver:
v ( a ~ ω j , ω i , ω i ) − v ( a ω j ∗ , ω i ) + μ R ( ω j ) π ∗ ( s 0 ∣ ω j ) μ R ( ω i ) π ∗ ( s 0 ∣ ω i ) ( v ( a ~ ω j , ω i , ω j ) − v ( a ω j ∗ , ω j ) ) ≥ 0. v(\tilde{a}_{\omega_j,\omega_i},\omega_i)-v(a^*_{\omega_j},\omega_i)+\frac{\mu_R(\omega_j)\pi^*(s_0\mid\omega_j)}{\mu_R(\omega_i)\pi^*(s_0\mid\omega_i)}\left(v(\tilde{a}_{\omega_j,\omega_i},\omega_j)-v(a^*_{\omega_j},\omega_j)\right)\ge0. v ( a ~ ω j , ω i , ω i ) − v ( a ω j ∗ , ω i ) + μ R ( ω i ) π ∗ ( s 0 ∣ ω i ) μ R ( ω j ) π ∗ ( s 0 ∣ ω j ) ( v ( a ~ ω j , ω i , ω j ) − v ( a ω j ∗ , ω j ) ) ≥ 0.
Note that v ( a ~ ω j , ω i , ω i ) − v ( a ω j ∗ , ω i ) ≤ 1 v(\tilde{a}_{\omega_j,\omega_i},\omega_i)-v(a^*_{\omega_j},\omega_i)\le1 v ( a ~ ω j , ω i , ω i ) − v ( a ω j ∗ , ω i ) ≤ 1 and v ( a ~ ω j , ω i , ω j ) − v ( a ω j ∗ , ω j ) ≤ − G < 0 v(\tilde{a}_{\omega_j,\omega_i},\omega_j)-v(a^*_{\omega_j},\omega_j)\le-G<0 v ( a ~ ω j , ω i , ω j ) − v ( a ω j ∗ , ω j ) ≤ − G < 0 , so
1 − μ R ( ω j ) μ R ( ω i ) π ∗ ( s 0 ∣ ω j ) π ∗ ( s 0 ∣ ω i ) ⋅ G ≥ 0 , 1-\frac{\mu_R(\omega_j)}{\mu_R(\omega_i)}\frac{\pi^*(s_0\mid\omega_j)}{\pi^*(s_0\mid\omega_i)}\cdot G\ge0, 1 − μ R ( ω i ) μ R ( ω j ) π ∗ ( s 0 ∣ ω i ) π ∗ ( s 0 ∣ ω j ) ⋅ G ≥ 0 ,
which implies
π ∗ ( s 0 ∣ ω j ) π ∗ ( s 0 ∣ ω i ) ≤ μ R ( ω i ) μ R ( ω j ) G ≤ 1 p 0 G . (35) \frac{\pi^*(s_0\mid\omega_j)}{\pi^*(s_0\mid\omega_i)}
\le\frac{\mu_R(\omega_i)}{\mu_R(\omega_j)G}
\le\frac{1}{p_0G}. \tag*{(35)} π ∗ ( s 0 ∣ ω i ) π ∗ ( s 0 ∣ ω j ) ≤ μ R ( ω j ) G μ R ( ω i ) ≤ p 0 G 1 . (35)
Thus, r ( 0 ) = 1 p 0 G ≥ ( 1 − ι ) μ R ( ω i ) μ R ( ω j ) G r^{(0)}=\frac{1}{p_0G}\ge(1-\iota)\frac{\mu_R(\omega_i)}{\mu_R(\omega_j)G} r ( 0 ) = p 0 G 1 ≥ ( 1 − ι ) μ R ( ω j ) G μ R ( ω i ) .
Then, consider step k ≥ 0 k\ge0 k ≥ 0 . The algorithm chooses the midpoint q = ℓ ( k ) + r ( k ) 2 q=\frac{\ell^{(k)}+r^{(k)}}{2} q = 2 ℓ ( k ) + r ( k ) and uses a signaling scheme satisfying π ( k ) ( s 0 ∣ ω j ) π ( k ) ( s 0 ∣ ω i ) = q \frac{\pi^{(k)}(s_0\mid\omega_j)}{\pi^{(k)}(s_0\mid\omega_i)}=q π ( k ) ( s 0 ∣ ω i ) π ( k ) ( s 0 ∣ ω j ) = q . When signal s 0 s_0 s 0 is realized:
if the receiver takes action a ω i ∗ a^*_{\omega_i} a ω i ∗ , then by the second item of the Lemma 2, q ≤ ( 1 + ι ) π ∗ ( s 0 ∣ ω j ) π ∗ ( s 0 ∣ ω i ) q\le(1+\iota)\frac{\pi^*(s_0\mid\omega_j)}{\pi^*(s_0\mid\omega_i)} q ≤ ( 1 + ι ) π ∗ ( s 0 ∣ ω i ) π ∗ ( s 0 ∣ ω j ) . The algorithm sets ℓ ( k + 1 ) \ell^{(k+1)} ℓ ( k + 1 ) to be q q q , so ℓ ( k + 1 ) \ell^{(k+1)} ℓ ( k + 1 ) satisfies the claim;
if the receiver does not take action a ω i ∗ a^*_{\omega_i} a ω i ∗ , then by the first item of Lemma 2, q ≥ ( 1 − ι ) π ∗ ( s 0 ∣ ω j ) π ∗ ( s 0 ∣ ω i ) q\ge(1-\iota)\frac{\pi^*(s_0\mid\omega_j)}{\pi^*(s_0\mid\omega_i)} q ≥ ( 1 − ι ) π ∗ ( s 0 ∣ ω i ) π ∗ ( s 0 ∣ ω j ) . The algorithm sets r ( k + 1 ) r^{(k+1)} r ( k + 1 ) to be q q q , so r ( k + 1 ) r^{(k+1)} r ( k + 1 ) satisfies the claim.
Proof of Lemma 1. When Algorithm 1 ends, we have ∣ r ( k ) − ℓ ( k ) ∣ ≤ τ |r^{(k)}-\ell^{(k)}|\le\tau ∣ r ( k ) − ℓ ( k ) ∣ ≤ τ . Then, using the above claim, we have
∣ π ∗ ( s 0 ∣ ω j ) π ∗ ( s 0 ∣ ω i ) − ℓ ( k ) ∣ ≤ τ + ι π ∗ ( s 0 ∣ ω j ) π ∗ ( s 0 ∣ ω i ) , \left|\frac{\pi^*(s_0\mid\omega_j)}{\pi^*(s_0\mid\omega_i)}-\ell^{(k)}\right|
\le\tau+\iota\frac{\pi^*(s_0\mid\omega_j)}{\pi^*(s_0\mid\omega_i)}, π ∗ ( s 0 ∣ ω i ) π ∗ ( s 0 ∣ ω j ) − ℓ ( k ) ≤ τ + ι π ∗ ( s 0 ∣ ω i ) π ∗ ( s 0 ∣ ω j ) ,
which implies
∣ μ R ( ω i ) μ R ( ω j ) − ρ ^ ∣ = ∣ π ∗ ( s 0 ∣ ω j ) π ∗ ( s 0 ∣ ω i ) − ℓ ( k ) ∣ ⋅ v ( a ~ ω j , ω i , ω j ) − v ( a ω i ∗ , ω 2 ) v ( a ω i ∗ , ω i ) − v ( a ~ ω j , ω i , ω i ) ≤ ( τ + ι π ∗ ( s 0 ∣ ω j ) π ∗ ( s 0 ∣ ω i ) ) ⋅ 1 G ≤ ( τ + δ G p 0 ) ⋅ 1 G ≤ ( τ + γ m ( 1 − γ ) G 3 p 0 3 ) ⋅ 1 G ≤ ( τ + γ m ( 1 − γ ) G 3 p 0 3 ) ⋅ 1 G p 0 ⋅ μ R ( ω i ) μ R ( ω j ) . \begin{aligned}
\left|\frac{\mu_R(\omega_i)}{\mu_R(\omega_j)}-\hat{\rho}\right|
&=\left|\frac{\pi^*(s_0\mid\omega_j)}{\pi^*(s_0\mid\omega_i)}-\ell^{(k)}\right|
\cdot\frac{v(\tilde{a}_{\omega_j,\omega_i},\omega_j)-v(a^*_{\omega_i},\omega_2)}
{v(a^*_{\omega_i},\omega_i)-v(\tilde{a}_{\omega_j,\omega_i},\omega_i)}\\
&\le\left(\tau+\iota\frac{\pi^*(s_0\mid\omega_j)}{\pi^*(s_0\mid\omega_i)}\right)\cdot\frac{1}{G}\\
&\le\left(\tau+\frac{\delta}{Gp_0}\right)\cdot\frac{1}{G}\\
&\le\left(\tau+\frac{\gamma^m}{(1-\gamma)G^3p_0^3}\right)\cdot\frac{1}{G}\\
&\le\left(\tau+\frac{\gamma^m}{(1-\gamma)G^3p_0^3}\right)\cdot\frac{1}{Gp_0}\cdot\frac{\mu_R(\omega_i)}{\mu_R(\omega_j)}.
\end{aligned} μ R ( ω j ) μ R ( ω i ) − ρ ^ = π ∗ ( s 0 ∣ ω i ) π ∗ ( s 0 ∣ ω j ) − ℓ ( k ) ⋅ v ( a ω i ∗ , ω i ) − v ( a ~ ω j , ω i , ω i ) v ( a ~ ω j , ω i , ω j ) − v ( a ω i ∗ , ω 2 ) ≤ ( τ + ι π ∗ ( s 0 ∣ ω i ) π ∗ ( s 0 ∣ ω j ) ) ⋅ G 1 ≤ ( τ + G p 0 δ ) ⋅ G 1 ≤ ( τ + ( 1 − γ ) G 3 p 0 3 γ m ) ⋅ G 1 ≤ ( τ + ( 1 − γ ) G 3 p 0 3 γ m ) ⋅ G p 0 1 ⋅ μ R ( ω j ) μ R ( ω i ) .
where the last step is because μ R ( ω i ) μ R ( ω j ) ≥ p 0 \frac{\mu_R(\omega_i)}{\mu_R(\omega_j)}\ge p_0 μ R ( ω j ) μ R ( ω i ) ≥ p 0 .
Then, we consider the expected running time of Algorithm 1. Because the difference r ( k ) − ℓ ( k ) r^{(k)}-\ell^{(k)} r ( k ) − ℓ ( k ) shrinks by a half after each while loop, the number of while loops, k k k , is at most
k ≤ log 2 r ( 0 ) − ℓ ( 0 ) τ = log 2 1 G p 0 τ . (36) k \le\log_{2}\frac{r^{(0)}-\ell^{(0)}}{\tau}=\log_{2}\frac{1}{Gp_{0}\tau}. \tag*{(36)} k ≤ log 2 τ r ( 0 ) − ℓ ( 0 ) = log 2 G p 0 τ 1 . (36)
Then, we consider the number of periods needed in each while loop. By definition, this is equal to the number of periods until signal s 0 s_{0} s 0 is realized, plus m m m . Since one of π ( k ) ( s 0 ∣ ω j ) \pi^{(k)}(s_{0}\mid\omega_{j}) π ( k ) ( s 0 ∣ ω j ) and π ( k ) ( s 0 ∣ ω i ) \pi^{(k)}(s_{0}\mid\omega_{i}) π ( k ) ( s 0 ∣ ω i ) is 1 (see Algorithm 1), the probability (according to the information designer’s prior belief) that s 0 s_{0} s 0 is realized in each period is at least:
π ( k ) ( s 0 ) = μ D ( ω i ) π ( k ) ( s 0 ∣ ω i ) + μ D ( ω j ) π ( k ) ( s 0 ∣ ω j ) ≥ min { μ D ( ω i ) , μ D ( ω j ) } ≥ p 0 . \pi^{(k)}(s_{0})=\mu_{D}(\omega_{i})\pi^{(k)}(s_{0}\mid\omega_{i})+\mu_{D}(\omega_{j})\pi^{(k)}(s_{0}\mid\omega_{j})\ge\min\{\mu_{D}(\omega_{i}),\mu_{D}(\omega_{j})\}\ge p_{0}. π ( k ) ( s 0 ) = μ D ( ω i ) π ( k ) ( s 0 ∣ ω i ) + μ D ( ω j ) π ( k ) ( s 0 ∣ ω j ) ≥ min { μ D ( ω i ) , μ D ( ω j )} ≥ p 0 .
So by the property of the geometric random variable, the expected number of periods until a signal s 0 s_{0} s 0 is realized is at most
1 π ( k ) ( s 0 ) ≤ 1 p 0 , \frac{1}{\pi^{(k)}(s_{0})}\le\frac{1}{p_{0}}, π ( k ) ( s 0 ) 1 ≤ p 0 1 ,
and the total expected number of periods over k k k while loops is at most
k ⋅ ( 1 p 0 + m ) ≤ ( 1 p 0 + m ) log 2 1 G p 0 τ . k\cdot\left(\frac{1}{p_{0}}+m\right)\le\left(\frac{1}{p_{0}}+m\right)\log_{2}\frac{1}{Gp_{0}\tau}. k ⋅ ( p 0 1 + m ) ≤ ( p 0 1 + m ) log 2 G p 0 τ 1 .
Proof of Lemma 2
Proof. Case (i): π ( k ) ( s 0 ∣ ω j ) π ( k ) ( s 0 ∣ ω i ) < ( 1 − ι ) π ∗ ( s 0 ∣ ω j ) π ∗ ( s 0 ∣ ω i ) \frac{\pi^{(k)}(s_{0}\mid\omega_{j})}{\pi^{(k)}(s_{0}\mid\omega_{i})}<(1-\iota)\frac{\pi^{*}(s_{0}\mid\omega_{j})}{\pi^{*}(s_{0}\mid\omega_{i})} π ( k ) ( s 0 ∣ ω i ) π ( k ) ( s 0 ∣ ω j ) < ( 1 − ι ) π ∗ ( s 0 ∣ ω i ) π ∗ ( s 0 ∣ ω j ) . If the receiver best responds in that period, then his optimal action will be a ω i ∗ a^{*}_{\omega_{i}} a ω i ∗ , since he is indifferent between a ω i ∗ a^{*}_{\omega_{i}} a ω i ∗ and the first indifferent action a ~ ω j , ω i \widetilde{a}_{\omega_{j},\omega_{i}} a ω j , ω i at π ∗ ( s 0 ∣ ω j ) π ∗ ( s 0 ∣ ω i ) \frac{\pi^{*}(s_{0}\mid\omega_{j})}{\pi^{*}(s_{0}\mid\omega_{i})} π ∗ ( s 0 ∣ ω i ) π ∗ ( s 0 ∣ ω j ) by definition, and also due to the case with π ( k ) ( s 0 ∣ ω j ) π ( k ) ( s 0 ∣ ω i ) < π ∗ ( s 0 ∣ ω j ) π ∗ ( s 0 ∣ ω i ) \frac{\pi^{(k)}(s_{0}\mid\omega_{j})}{\pi^{(k)}(s_{0}\mid\omega_{i})}<\frac{\pi^{*}(s_{0}\mid\omega_{j})}{\pi^{*}(s_{0}\mid\omega_{i})} π ( k ) ( s 0 ∣ ω i ) π ( k ) ( s 0 ∣ ω j ) < π ∗ ( s 0 ∣ ω i ) π ∗ ( s 0 ∣ ω j ) . Suppose the strategic receiver deviates to an action a ≠ a ω i ∗ a\ne a^{*}_{\omega_{i}} a = a ω i ∗ instead. Then, he will suffer a loss in the current period, and will potentially obtain some gain in the future.
Regarding the gain, the signaling schemes used in the next additional m m m periods are independent of the receiver’s current action, and the receiver’s payoff is bounded in [ 0 , 1 ] [0,1] [ 0 , 1 ] . Thus, the receiver’s total gain in the future is at most
gain ≤ 0 + ∑ t = m + 1 T γ t ⋅ 1 ≤ γ m + 1 1 − γ . \operatorname{gain}\le0+\sum_{t=m+1}^{T}\gamma^{t}\cdot1\le\frac{\gamma^{m+1}}{1-\gamma}. gain ≤ 0 + t = m + 1 ∑ T γ t ⋅ 1 ≤ 1 − γ γ m + 1 .
Then, we compute his loss, denoted by loss ( a ω i ∗ , a ) \operatorname{loss}(a^{*}_{\omega_{i}},a) loss ( a ω i ∗ , a ) , which is the difference between the receiver’s payoff of taking action a ω i ∗ a_{\omega_i}^{*} a ω i ∗ versus the deviation action a a a in the current round.
loss ( a ω i ∗ , a ) = ∑ ω ∈ Ω Pr μ R , π ( k ) [ ω ∣ s 0 ] ( v ( a ω i ∗ , ω ) − v ( a , ω ) ) = μ R ( ω i ) π ( k ) ( s 0 ∣ ω i ) π ( k ) ( s 0 ) ( v ( a ω i ∗ , ω i ) − v ( a , ω i ) ) + μ R ( ω j ) π ( k ) ( s 0 ∣ ω j ) π ( k ) ( s 0 ) ( v ( a ω i ∗ , ω j ) − v ( a , ω j ) ) . \begin{aligned}
\operatorname{loss}(a_{\omega_i}^{*},a)
&=\sum_{\omega\in\Omega}\Pr_{\mu_R,\pi^{(k)}}[\omega\mid s_0]\left(v(a_{\omega_i}^{*},\omega)-v(a,\omega)\right)\\
&=\frac{\mu_R(\omega_i)\pi^{(k)}(s_0\mid\omega_i)}{\pi^{(k)}(s_0)}\left(v(a_{\omega_i}^{*},\omega_i)-v(a,\omega_i)\right)
+\frac{\mu_R(\omega_j)\pi^{(k)}(s_0\mid\omega_j)}{\pi^{(k)}(s_0)}\left(v(a_{\omega_i}^{*},\omega_j)-v(a,\omega_j)\right).
\end{aligned} loss ( a ω i ∗ , a ) = ω ∈ Ω ∑ μ R , π ( k ) Pr [ ω ∣ s 0 ] ( v ( a ω i ∗ , ω ) − v ( a , ω ) ) = π ( k ) ( s 0 ) μ R ( ω i ) π ( k ) ( s 0 ∣ ω i ) ( v ( a ω i ∗ , ω i ) − v ( a , ω i ) ) + π ( k ) ( s 0 ) μ R ( ω j ) π ( k ) ( s 0 ∣ ω j ) ( v ( a ω i ∗ , ω j ) − v ( a , ω j ) ) .
Claim 3. loss ( a ω i ∗ , a ) > ι G 2 p 0 2 \operatorname{loss}(a_{\omega_i}^{*},a)>\iota G^2p_0^2 loss ( a ω i ∗ , a ) > ι G 2 p 0 2 .
Proof. Consider the graph (Figure 4 a) where the horizontal axis is the ratio π ( s 0 ∣ ω j ) π ( s 0 ∣ ω i ) \frac{\pi(s_0\mid\omega_j)}{\pi(s_0\mid\omega_i)} π ( s 0 ∣ ω i ) π ( s 0 ∣ ω j ) . For each action a ∈ A a\in A a ∈ A , we plot the line representing the receiver’s single-period utility of taking action a a a (after a normalization):
y a ( x ) = v ( a , ω i ) + μ R ( ω j ) μ R ( ω i ) ⋅ x ⋅ v ( a , ω j ) , where x = π ( s 0 ∣ ω j ) π ( s 0 ∣ ω i ) . y_a(x)=v(a,\omega_i)+\frac{\mu_R(\omega_j)}{\mu_R(\omega_i)}\cdot x\cdot v(a,\omega_j),\qquad\text{where }x=\frac{\pi(s_0\mid\omega_j)}{\pi(s_0\mid\omega_i)}. y a ( x ) = v ( a , ω i ) + μ R ( ω i ) μ R ( ω j ) ⋅ x ⋅ v ( a , ω j ) , where x = π ( s 0 ∣ ω i ) π ( s 0 ∣ ω j ) .
By definition, point A A A is the indifference point between actions a ω i ∗ a_{\omega_i}^{*} a ω i ∗ and a ~ ω j , ω i \widetilde{a}_{\omega_j,\omega_i} a ω j , ω i , whose x x x -coordinate x A = π ∗ ( s 0 ∣ ω j ) π ∗ ( s 0 ∣ ω i ) x_A=\frac{\pi^{*}(s_0\mid\omega_j)}{\pi^{*}(s_0\mid\omega_i)} x A = π ∗ ( s 0 ∣ ω i ) π ∗ ( s 0 ∣ ω j ) and y y y -coordinate y A = y a ω i ∗ ( x A ) = y a ~ ω j , ω i ( x A ) y_A=y_{a_{\omega_i}^{*}}(x_A)=y_{\widetilde{a}_{\omega_j,\omega_i}}(x_A) y A = y a ω i ∗ ( x A ) = y a ω j , ω i ( x A ) . Let d x ( a ω i ∗ , a ) = y a ω i ∗ ( x ) − y a ( x ) d_x(a_{\omega_i}^{*},a)=y_{a_{\omega_i}^{*}}(x)-y_a(x) d x ( a ω i ∗ , a ) = y a ω i ∗ ( x ) − y a ( x ) be the vertical distance between lines y a ω i ∗ y_{a_{\omega_i}^{*}} y a ω i ∗ and y a y_a y a at x x x -coordinate x x x . Because A A A is the first indifference point, line y a y_a y a is below line y a ω i ∗ y_{a_{\omega_i}^{*}} y a ω i ∗ in the [ 0 , x A ] [0,x_A] [ 0 , x A ] segment: namely, d 0 ( a ω i ∗ , a ) > 0 d_0(a_{\omega_i}^{*},a)>0 d 0 ( a ω i ∗ , a ) > 0 and d x A ( a ω i ∗ , a ) ≥ 0 d_{x_A}(a_{\omega_i}^{*},a)\geq0 d x A ( a ω i ∗ , a ) ≥ 0 . Consider the line connecting C = ( 0 , y a ( 0 ) = v ( a , ω i ) ) C=(0,y_a(0)=v(a,\omega_i)) C = ( 0 , y a ( 0 ) = v ( a , ω i )) to A A A , with B B B being the point on this line at x-coordinate
x B = π ( k ) ( s 0 ∣ ω j ) π ( k ) ( s 0 ∣ ω i ) ≤ ( 1 − ι ) π ∗ ( s 0 ∣ ω j ) π ∗ ( s 0 ∣ ω i ) = ( 1 − ι ) x A . x_B = \frac{\pi^{(k)}(s_0|\omega_j)}{\pi^{(k)}(s_0|\omega_i)} \le(1-\iota)\frac{\pi^*(s_0|\omega_j)}{\pi^*(s_0|\omega_i)} = (1-\iota)x_A. x B = π ( k ) ( s 0 ∣ ω i ) π ( k ) ( s 0 ∣ ω j ) ≤ ( 1 − ι ) π ∗ ( s 0 ∣ ω i ) π ∗ ( s 0 ∣ ω j ) = ( 1 − ι ) x A .
Since B B B is between line y a y_a y a and line y a ω i ∗ y_{a_{\omega_i}^*} y a ω i ∗ , d x B ( a ω i ∗ , a ) ≥ B D d_{x_B}(a_{\omega_i}^*,a) \ge BD d x B ( a ω i ∗ , a ) ≥ B D . Triangle A B D ABD A B D is similar to triangle A C E ACE A C E , so
B D = C E ⋅ A B A C = C E ⋅ x A − x B x A − 0 = ( v ( a ω i ∗ , ω i ) − v ( a , ω i ) ) ⋅ π ∗ ( s 0 ∣ ω j ) π ∗ ( s 0 ∣ ω i ) − π ( k ) ( s 0 ∣ ω j ) π ( k ) ( s 0 ∣ ω i ) π ∗ ( s 0 ∣ ω j ) π ∗ ( s 0 ∣ ω i ) > G ⋅ ι . BD = CE \cdot\frac{AB}{AC} = CE \cdot\frac{x_A-x_B}{x_A-0} = \left(v(a_{\omega_i}^*,\omega_i)-v(a,\omega_i)\right)\cdot\frac{\frac{\pi^*(s_0|\omega_j)}{\pi^*(s_0|\omega_i)}-\frac{\pi^{(k)}(s_0|\omega_j)}{\pi^{(k)}(s_0|\omega_i)}}{\frac{\pi^*(s_0|\omega_j)}{\pi^*(s_0|\omega_i)}} > G\cdot\iota. B D = C E ⋅ A C A B = C E ⋅ x A − 0 x A − x B = ( v ( a ω i ∗ , ω i ) − v ( a , ω i ) ) ⋅ π ∗ ( s 0 ∣ ω i ) π ∗ ( s 0 ∣ ω j ) π ∗ ( s 0 ∣ ω i ) π ∗ ( s 0 ∣ ω j ) − π ( k ) ( s 0 ∣ ω i ) π ( k ) ( s 0 ∣ ω j ) > G ⋅ ι .
Therefore,
G ι < d x B ( a ω i ∗ , a ) = v ( a ω i ∗ , ω i ) + μ R ( ω j ) π ( k ) ( s 0 ∣ ω j ) μ R ( ω i ) π ( k ) ( s 0 ∣ ω i ) v ( a ω i ∗ , ω j ) − v ( a , ω i ) − μ R ( ω j ) π ( k ) ( s 0 ∣ ω j ) μ R ( ω i ) π ( k ) ( s 0 ∣ ω i ) v ( a , ω j ) . G\iota< d_{x_B}(a_{\omega_i}^*,a) = v(a_{\omega_i}^*,\omega_i)+\frac{\mu_R(\omega_j)\pi^{(k)}(s_0|\omega_j)}{\mu_R(\omega_i)\pi^{(k)}(s_0|\omega_i)}v(a_{\omega_i}^*,\omega_j)-v(a,\omega_i)-\frac{\mu_R(\omega_j)\pi^{(k)}(s_0|\omega_j)}{\mu_R(\omega_i)\pi^{(k)}(s_0|\omega_i)}v(a,\omega_j). G ι < d x B ( a ω i ∗ , a ) = v ( a ω i ∗ , ω i ) + μ R ( ω i ) π ( k ) ( s 0 ∣ ω i ) μ R ( ω j ) π ( k ) ( s 0 ∣ ω j ) v ( a ω i ∗ , ω j ) − v ( a , ω i ) − μ R ( ω i ) π ( k ) ( s 0 ∣ ω i ) μ R ( ω j ) π ( k ) ( s 0 ∣ ω j ) v ( a , ω j ) .
Multiplying both sides by μ R ( ω i ) π ( k ) ( s 0 ∣ ω i ) π ( k ) ( s 0 ) \frac{\mu_R(\omega_i)\pi^{(k)}(s_0|\omega_i)}{\pi^{(k)}(s_0)} π ( k ) ( s 0 ) μ R ( ω i ) π ( k ) ( s 0 ∣ ω i ) ,
G ι μ R ( ω i ) π ( k ) ( s 0 ∣ ω i ) π ( k ) ( s 0 ) < μ R ( ω i ) π ( k ) ( s 0 ∣ ω i ) π ( k ) ( s 0 ) ( v ( a ω i ∗ , ω i ) − v ( a , ω i ) ) + μ R ( ω j ) π ( k ) ( s 0 ∣ ω j ) π ( k ) ( s 0 ) ( v ( a ω i ∗ , ω j ) − v ( a , ω j ) ) = loss ( a ω i ∗ , a ) . \begin{aligned}
G\iota\frac{\mu_R(\omega_i)\pi^{(k)}(s_0|\omega_i)}{\pi^{(k)}(s_0)} < \frac{\mu_R(\omega_i)\pi^{(k)}(s_0|\omega_i)}{\pi^{(k)}(s_0)}\left(v(a_{\omega_i}^*,\omega_i)-v(a,\omega_i)\right)+\frac{\mu_R(\omega_j)\pi^{(k)}(s_0|\omega_j)}{\pi^{(k)}(s_0)}\left(v(a_{\omega_i}^*,\omega_j)-v(a,\omega_j)\right) \\
&= \operatorname{loss}(a_{\omega_i}^*,a).
\end{aligned} G ι π ( k ) ( s 0 ) μ R ( ω i ) π ( k ) ( s 0 ∣ ω i ) < π ( k ) ( s 0 ) μ R ( ω i ) π ( k ) ( s 0 ∣ ω i ) ( v ( a ω i ∗ , ω i ) − v ( a , ω i ) ) + π ( k ) ( s 0 ) μ R ( ω j ) π ( k ) ( s 0 ∣ ω j ) ( v ( a ω i ∗ , ω j ) − v ( a , ω j ) ) = loss ( a ω i ∗ , a ) .
We note that μ R ( ω i ) π ( k ) ( s 0 ∣ ω i ) π ( k ) ( s 0 ) ≥ G p 0 2 \frac{\mu_R(\omega_i)\pi^{(k)}(s_0|\omega_i)}{\pi^{(k)}(s_0)} \ge Gp_0^2 π ( k ) ( s 0 ) μ R ( ω i ) π ( k ) ( s 0 ∣ ω i ) ≥ G p 0 2 , because μ R ( ω i ) ≥ p 0 \mu_R(\omega_i) \ge p_0 μ R ( ω i ) ≥ p 0 , π ( k ) ( s 0 ) ≤ 1 \pi^{(k)}(s_0) \le1 π ( k ) ( s 0 ) ≤ 1 , π ( k ) ( s 0 ∣ ω i ) = min { 1 , 1 q } ≥ G p 0 \pi^{(k)}(s_0|\omega_i)=\min\{1,\frac{1}{q}\}\ge Gp_0 π ( k ) ( s 0 ∣ ω i ) = min { 1 , q 1 } ≥ G p 0 from Algorithm 1. Thus, we obtain loss ( a ω i ∗ , a ) ≥ ι G 2 p 0 2 \operatorname{loss}(a_{\omega_i}^*,a) \ge\iota G^2p_0^2 loss ( a ω i ∗ , a ) ≥ ι G 2 p 0 2 , which proves the claim. □
Comparing gain and loss, given ι = γ m ( 1 − γ ) G 2 p 0 2 \iota=\frac{\gamma^m}{(1-\gamma)G^2p_0^2} ι = ( 1 − γ ) G 2 p 0 2 γ m , we have
loss ( a ω i ∗ , a ) > ι G 2 p 0 2 > γ m + 1 1 − γ ≥ gain . \operatorname{loss}(a_{\omega_i}^*,a) > \iota G^2p_0^2 > \frac{\gamma^{m+1}}{1-\gamma} \ge\operatorname{gain}. loss ( a ω i ∗ , a ) > ι G 2 p 0 2 > 1 − γ γ m + 1 ≥ gain .
Thus, the strategic receiver should not take a a a for any a ≠ a ω i ∗ a\ne a_{\omega_i}^* a = a ω i ∗ , and thus takes a ω i ∗ a_{\omega_i}^* a ω i ∗ .
Case (ii): π ( k ) ( s 0 ∣ ω j ) π ( k ) ( s 0 ∣ ω i ) > ( 1 + ι ) π ∗ ( s 0 ∣ ω j ) π ∗ ( s 0 ∣ ω i ) \frac{\pi^{(k)}(s_0|\omega_j)}{\pi^{(k)}(s_0|\omega_i)}>(1+\iota)\frac{\pi^*(s_0|\omega_j)}{\pi^*(s_0|\omega_i)} π ( k ) ( s 0 ∣ ω i ) π ( k ) ( s 0 ∣ ω j ) > ( 1 + ι ) π ∗ ( s 0 ∣ ω i ) π ∗ ( s 0 ∣ ω j ) . See Figure 4 b for an illustration. In this case, the optimal action for a myopic receiver is not a ω i ∗ a_{\omega_i}^* a ω i ∗ , because π ( k ) ( s 0 ∣ ω j ) π ( k ) ( s 0 ∣ ω i ) > ( 1 + ι ) π ∗ ( s 0 ∣ ω j ) π ∗ ( s 0 ∣ ω i ) \frac{\pi^{(k)}(s_0|\omega_j)}{\pi^{(k)}(s_0|\omega_i)}>(1+\iota)\frac{\pi^*(s_0|\omega_j)}{\pi^*(s_0|\omega_i)} π ( k ) ( s 0 ∣ ω i ) π ( k ) ( s 0 ∣ ω j ) > ( 1 + ι ) π ∗ ( s 0 ∣ ω i ) π ∗ ( s 0 ∣ ω j ) . If a strategic receiver takes action a ω i ∗ a_{\omega_i}^* a ω i ∗ in the current period, then he suffers a loss in the current period and potentially obtains some gain in the future. As in the previous case, the gain is at most γ m + 1 1 − γ \frac{\gamma^{m+1}}{1-\gamma} 1 − γ γ m + 1 . Then, consider the loss. We lower bound the loss by the utility difference between a ~ \tilde{a} a ~ and a ω i ∗ a^*_{\omega_i} a ω i ∗ in the current period:
loss ( a ω i ∗ ) ≥ ∑ ω ∈ Ω Pr μ R , π ( k ) [ ω ∣ s 0 ] ( v ( a ~ ω j , ω i , ω ) − v ( a ω i ∗ , ω ) ) = μ R ( ω i ) π ( k ) ( s 0 ∣ ω i ) π ( k ) ( s 0 ) ( v ( a ~ , ω 1 ) − v ( a ω i ∗ , ω 1 ) ) + μ R ( ω j ) π ( k ) ( s 0 ∣ ω 2 ) π ( k ) ( s 0 ) ( v ( a ~ ω j , ω i , ω j ) − v ( a ω i ∗ , ω j ) ) = μ R ( ω i ) π ( k ) ( s 0 ∣ ω i ) π ( k ) ( s 0 ) ( v ( a ~ , ω i ) − v ( a ω i ∗ , ω i ) + μ R ( ω j ) π ( k ) ( s 0 ∣ ω j ) μ R ( ω i ) π ( k ) ( s 0 ∣ ω i ) ( v ( a ~ ω j , ω i , ω j ) − v ( a ω i ∗ , ω j ) ) ) = μ R ( ω i ) π ( k ) ( s 0 ∣ ω i ) π ( k ) ( s 0 ) B ′ D ′ (see xref‘Figure 4b‘). \begin{aligned}
\operatorname{loss}(a^*_{\omega_i}) \geq\sum_{\omega\in\Omega} \Pr_{\mu_R,\pi^{(k)}}[\omega\mid s_0]\left(v(\tilde{a}_{\omega_j,\omega_i},\omega)-v(a^*_{\omega_i},\omega)\right) \\
&= \frac{\mu_R(\omega_i)\pi^{(k)}(s_0\mid\omega_i)}{\pi^{(k)}(s_0)}\left(v(\tilde{a},\omega_1)-v(a^*_{\omega_i},\omega_1)\right)+\frac{\mu_R(\omega_j)\pi^{(k)}(s_0\mid\omega_2)}{\pi^{(k)}(s_0)}\left(v(\tilde{a}_{\omega_j,\omega_i},\omega_j)-v(a^*_{\omega_i},\omega_j)\right) \\
&= \frac{\mu_R(\omega_i)\pi^{(k)}(s_0\mid\omega_i)}{\pi^{(k)}(s_0)}\left(v(\tilde{a},\omega_i)-v(a^*_{\omega_i},\omega_i)+\frac{\mu_R(\omega_j)\pi^{(k)}(s_0\mid\omega_j)}{\mu_R(\omega_i)\pi^{(k)}(s_0\mid\omega_i)}\left(v(\tilde{a}_{\omega_j,\omega_i},\omega_j)-v(a^*_{\omega_i},\omega_j)\right)\right) \\
&= \frac{\mu_R(\omega_i)\pi^{(k)}(s_0\mid\omega_i)}{\pi^{(k)}(s_0)}B'D' \qquad\text{(see {xref}`Figure 4b`).}
\end{aligned} loss ( a ω i ∗ ) ≥ ω ∈ Ω ∑ μ R , π ( k ) Pr [ ω ∣ s 0 ] ( v ( a ~ ω j , ω i , ω ) − v ( a ω i ∗ , ω ) ) = π ( k ) ( s 0 ) μ R ( ω i ) π ( k ) ( s 0 ∣ ω i ) ( v ( a ~ , ω 1 ) − v ( a ω i ∗ , ω 1 ) ) + π ( k ) ( s 0 ) μ R ( ω j ) π ( k ) ( s 0 ∣ ω 2 ) ( v ( a ~ ω j , ω i , ω j ) − v ( a ω i ∗ , ω j ) ) = π ( k ) ( s 0 ) μ R ( ω i ) π ( k ) ( s 0 ∣ ω i ) ( v ( a ~ , ω i ) − v ( a ω i ∗ , ω i ) + μ R ( ω i ) π ( k ) ( s 0 ∣ ω i ) μ R ( ω j ) π ( k ) ( s 0 ∣ ω j ) ( v ( a ~ ω j , ω i , ω j ) − v ( a ω i ∗ , ω j ) ) ) = π ( k ) ( s 0 ) μ R ( ω i ) π ( k ) ( s 0 ∣ ω i ) B ′ D ′ (see xref ‘Figure 4b‘).
Because triangle A B ′ D ′ AB'D' A B ′ D ′ is similar to triangle A C ′ E AC'E A C ′ E , we have
B ′ D ′ = C ′ E ⋅ A B ′ A C ′ = ( v ( a ω i ∗ , ω i ) − v ( a , ω i ) ) ⋅ π ( k ) ( s 0 ∣ ω j ) π ( k ) ( s 0 ∣ ω i ) − π ∗ ( s 0 ∣ ω j ) π ∗ ( s 0 ∣ ω i ) π ∗ ( s 0 ∣ ω j ) π ∗ ( s 0 ∣ ω i ) > G ι . B'D'=C'E\cdot\frac{AB'}{AC'}=\left(v(a^*_{\omega_i},\omega_i)-v(a,\omega_i)\right)\cdot\frac{\frac{\pi^{(k)}(s_0\mid\omega_j)}{\pi^{(k)}(s_0\mid\omega_i)}-\frac{\pi^*(s_0\mid\omega_j)}{\pi^*(s_0\mid\omega_i)}}{\frac{\pi^*(s_0\mid\omega_j)}{\pi^*(s_0\mid\omega_i)}}>G\iota. B ′ D ′ = C ′ E ⋅ A C ′ A B ′ = ( v ( a ω i ∗ , ω i ) − v ( a , ω i ) ) ⋅ π ∗ ( s 0 ∣ ω i ) π ∗ ( s 0 ∣ ω j ) π ( k ) ( s 0 ∣ ω i ) π ( k ) ( s 0 ∣ ω j ) − π ∗ ( s 0 ∣ ω i ) π ∗ ( s 0 ∣ ω j ) > G ι .
Same as the previous case, μ R ( ω i ) π ( k ) ( s 0 ∣ ω i ) π ( k ) ( s 0 ) ≥ G p 0 2 \frac{\mu_R(\omega_i)\pi^{(k)}(s_0\mid\omega_i)}{\pi^{(k)}(s_0)}\geq Gp_0^2 π ( k ) ( s 0 ) μ R ( ω i ) π ( k ) ( s 0 ∣ ω i ) ≥ G p 0 2 . Therefore, loss ( a ω i ∗ ) > G 2 p 0 2 ι > γ m + 1 1 − γ ≥ gain \operatorname{loss}(a^*_{\omega_i})>G^2p_0^2\iota>\frac{\gamma^{m+1}}{1-\gamma}\geq\operatorname{gain} loss ( a ω i ∗ ) > G 2 p 0 2 ι > 1 − γ γ m + 1 ≥ gain , given ι = γ m ( 1 − γ ) G 2 p 0 2 \iota=\frac{\gamma^m}{(1-\gamma)G^2p_0^2} ι = ( 1 − γ ) G 2 p 0 2 γ m . So, the strategic receiver should not take a ω i ∗ a^*_{\omega_i} a ω i ∗ .
Proof of Lemma 3
Lemma 17. Let ε ′ = 1 G p 0 ( τ + γ m ( 1 − γ ) G 3 p 0 3 ) ≤ 1 2 \varepsilon'=\frac{1}{Gp_0}\left(\tau+\frac{\gamma^m}{(1-\gamma)G^3p_0^3}\right)\leq\frac{1}{2} ε ′ = G p 0 1 ( τ + ( 1 − γ ) G 3 p 0 3 γ m ) ≤ 2 1 . For any two states ω i , ω j ∈ Ω \omega_i,\omega_j\in\Omega ω i , ω j ∈ Ω , the output ρ ^ i j \hat{\rho}_{ij} ρ ^ ij of Algorithm 2 satisfies ρ ^ i j ∈ ( 1 ± 4 ε ′ ) μ R ( ω i ) μ R ( ω j ) \hat{\rho}_{ij}\in(1\pm4\varepsilon')\frac{\mu_R(\omega_i)}{\mu_R(\omega_j)} ρ ^ ij ∈ ( 1 ± 4 ε ′ ) μ R ( ω j ) μ R ( ω i ) . Algorithm 2 terminates in at most 2 ( 1 p 0 + m ) log 2 1 G p 0 τ 2\left(\frac{1}{p_0}+m\right)\log_2\frac{1}{Gp_0\tau} 2 ( p 0 1 + m ) log 2 G p 0 τ 1 periods in expectation.
Proof. The proof uses Lemma 1 twice. See details below. Let ε ′ = 1 G p 0 ( τ + γ m ( 1 − γ ) G 3 p 0 3 ) \varepsilon'=\frac{1}{Gp_0}\left(\tau+\frac{\gamma^m}{(1-\gamma)G^3p_0^3}\right) ε ′ = G p 0 1 ( τ + ( 1 − γ ) G 3 p 0 3 γ m ) . If ( ω i , ω j ) (\omega_i,\omega_j) ( ω i , ω j ) is a pair of distinguishable states, then Lemma 1 shows that the estimate ρ ^ i j \hat{\rho}_{ij} ρ ^ ij returned by Algorithm 1 satisfies ρ ^ i j ∈ ( 1 ± ε ′ ) μ R ( ω i ) μ R ( ω j ) \hat{\rho}_{ij}\in(1\pm\varepsilon')\frac{\mu_R(\omega_i)}{\mu_R(\omega_j)} ρ ^ ij ∈ ( 1 ± ε ′ ) μ R ( ω j ) μ R ( ω i ) .
If ( ω i , ω j ) (\omega_i,\omega_j) ( ω i , ω j ) is not a pair of distinguishable states, then by Lemma 1, we have the ratio estimates ρ ^ i k ∈ ( 1 ± ε ′ ) μ R ( ω i ) μ R ( ω k ) \hat{\rho}_{ik}\in(1\pm\varepsilon')\frac{\mu_R(\omega_i)}{\mu_R(\omega_k)} ρ ^ ik ∈ ( 1 ± ε ′ ) μ R ( ω k ) μ R ( ω i ) and ρ ^ j k ∈ ( 1 ± ε ′ ) μ R ( ω j ) μ R ( ω k ) \hat{\rho}_{jk}\in(1\pm\varepsilon')\frac{\mu_R(\omega_j)}{\mu_R(\omega_k)} ρ ^ j k ∈ ( 1 ± ε ′ ) μ R ( ω k ) μ R ( ω j ) . So,
ρ ^ i j = ρ ^ i k ρ ^ j k ≥ μ R ( ω i ) μ R ( ω k ) ( 1 − ε ′ ) μ R ( ω j ) μ R ( ω k ) ( 1 + ε ′ ) ≥ μ R ( ω i ) μ R ( ω j ) ( 1 − 2 ε ′ ) \hat{\rho}_{ij}=\frac{\hat{\rho}_{ik}}{\hat{\rho}_{jk}}\geq\frac{\frac{\mu_R(\omega_i)}{\mu_R(\omega_k)}(1-\varepsilon')}{\frac{\mu_R(\omega_j)}{\mu_R(\omega_k)}(1+\varepsilon')}\geq\frac{\mu_R(\omega_i)}{\mu_R(\omega_j)}(1-2\varepsilon') ρ ^ ij = ρ ^ j k ρ ^ ik ≥ μ R ( ω k ) μ R ( ω j ) ( 1 + ε ′ ) μ R ( ω k ) μ R ( ω i ) ( 1 − ε ′ ) ≥ μ R ( ω j ) μ R ( ω i ) ( 1 − 2 ε ′ )
and
ρ ^ i j = ρ ^ i k ρ ^ j k ≤ μ R ( ω i ) μ R ( ω k ) ( 1 + ε ′ ) μ R ( ω j ) μ R ( ω k ) ( 1 − ε ′ ) ≤ μ R ( ω i ) μ R ( ω j ) ( 1 + 4 ε ′ ) \hat{\rho}_{ij}=\frac{\hat{\rho}_{ik}}{\hat{\rho}_{jk}}\leq\frac{\frac{\mu_R(\omega_i)}{\mu_R(\omega_k)}(1+\varepsilon')}{\frac{\mu_R(\omega_j)}{\mu_R(\omega_k)}(1-\varepsilon')}\leq\frac{\mu_R(\omega_i)}{\mu_R(\omega_j)}(1+4\varepsilon') ρ ^ ij = ρ ^ j k ρ ^ ik ≤ μ R ( ω k ) μ R ( ω j ) ( 1 − ε ′ ) μ R ( ω k ) μ R ( ω i ) ( 1 + ε ′ ) ≤ μ R ( ω j ) μ R ( ω i ) ( 1 + 4 ε ′ )
given ε ′ ≤ 1 2 \varepsilon' \le\frac{1}{2} ε ′ ≤ 2 1 . The multiplicative error of ρ ^ i j \hat{\rho}_{ij} ρ ^ ij is thus at most 4 ε ′ 4\varepsilon' 4 ε ′ .
Algorithm 2 needs twice the running time of Algorithm 1, which is at most 2 ( 1 p 0 + m ) log 2 1 G p 0 2\left(\frac{1}{p_0}+m\right)\log_2\frac{1}{Gp_0} 2 ( p 0 1 + m ) log 2 G p 0 1 periods in expectation by Lemma 1.
Lemma 18. Suppose 1 G p 0 ( τ + γ m ( 1 − γ ) G 3 p 0 3 ) ≤ 1 2 \frac{1}{Gp_0}\left(\tau+\frac{\gamma^m}{(1-\gamma)G^3p_0^3}\right)\le\frac{1}{2} G p 0 1 ( τ + ( 1 − γ ) G 3 p 0 3 γ m ) ≤ 2 1 .
Proof. By Lemma 17, the estimation ρ ^ i 1 \hat{\rho}_{i1} ρ ^ i 1 satisfies ρ ^ i 1 ∈ ( 1 ± 4 ε ′ ) μ R ( ω i ) μ R ( ω 1 ) \hat{\rho}_{i1}\in(1\pm4\varepsilon')\frac{\mu_R(\omega_i)}{\mu_R(\omega_1)} ρ ^ i 1 ∈ ( 1 ± 4 ε ′ ) μ R ( ω 1 ) μ R ( ω i ) with ε ′ = 1 G p 0 ( τ + γ m ( 1 − γ ) G 3 p 0 3 ) \varepsilon'=\frac{1}{Gp_0}\left(\tau+\frac{\gamma^m}{(1-\gamma)G^3p_0^3}\right) ε ′ = G p 0 1 ( τ + ( 1 − γ ) G 3 p 0 3 γ m ) . Because μ R ( ω i ) μ R ( ω 1 ) ≤ 1 p 0 \frac{\mu_R(\omega_i)}{\mu_R(\omega_1)}\le\frac{1}{p_0} μ R ( ω 1 ) μ R ( ω i ) ≤ p 0 1 , we have
∣ ρ ^ i 1 − μ R ( ω i ) μ R ( ω 1 ) ∣ ≤ 4 ε ′ p 0 . \left|\hat{\rho}_{i1}-\frac{\mu_R(\omega_i)}{\mu_R(\omega_1)}\right|\le\frac{4\varepsilon'}{p_0}. ρ ^ i 1 − μ R ( ω 1 ) μ R ( ω i ) ≤ p 0 4 ε ′ .
Since μ R \mu_R μ R is a probability distribution, we have 1 = ∑ ω ∈ Ω μ R ( ω ) = μ R ( ω 1 ) + μ R ( ω 1 ) ∑ i = 2 ∣ Ω ∣ μ R ( ω i ) μ R ( ω 1 ) 1=\sum_{\omega\in\Omega}\mu_R(\omega)=\mu_R(\omega_1)+\mu_R(\omega_1)\sum_{i=2}^{|\Omega|}\frac{\mu_R(\omega_i)}{\mu_R(\omega_1)} 1 = ∑ ω ∈ Ω μ R ( ω ) = μ R ( ω 1 ) + μ R ( ω 1 ) ∑ i = 2 ∣Ω∣ μ R ( ω 1 ) μ R ( ω i ) , so
μ R ( ω 1 ) = 1 1 + ∑ i = 2 ∣ Ω ∣ μ R ( ω i ) μ R ( ω 1 ) . \mu_R(\omega_1)=\frac{1}{1+\sum_{i=2}^{|\Omega|}\frac{\mu_R(\omega_i)}{\mu_R(\omega_1)}}. μ R ( ω 1 ) = 1 + ∑ i = 2 ∣Ω∣ μ R ( ω 1 ) μ R ( ω i ) 1 .
Because the function f ( x ) = 1 1 + x f(x)=\frac{1}{1+x} f ( x ) = 1 + x 1 is 1-Lipschitz (∣ f ′ ( x ) ∣ = 1 ( 1 + x ) 2 ≤ 1 |f'(x)|=\frac{1}{(1+x)^2}\le1 ∣ f ′ ( x ) ∣ = ( 1 + x ) 2 1 ≤ 1 ), we have
∣ μ ^ ( ω 1 ) − μ R ( ω 1 ) ∣ = ∣ 1 1 + ∑ i = 2 ∣ Ω ∣ ρ ^ i 1 − 1 1 + ∑ i = 2 ∣ Ω ∣ μ R ( ω i ) μ R ( ω 1 ) ∣ ≤ ∣ ∑ i = 2 ∣ Ω ∣ ρ ^ i 1 − ∑ i = 2 ∣ Ω ∣ μ R ( ω i ) μ R ( ω 1 ) ∣ ≤ ∣ Ω ∣ 4 ε ′ p 0 . |\hat{\mu}(\omega_1)-\mu_R(\omega_1)|=\left|\frac{1}{1+\sum_{i=2}^{|\Omega|}\hat{\rho}_{i1}}-\frac{1}{1+\sum_{i=2}^{|\Omega|}\frac{\mu_R(\omega_i)}{\mu_R(\omega_1)}}\right|\le\left|\sum_{i=2}^{|\Omega|}\hat{\rho}_{i1}-\sum_{i=2}^{|\Omega|}\frac{\mu_R(\omega_i)}{\mu_R(\omega_1)}\right|\le|\Omega|\frac{4\varepsilon'}{p_0}. ∣ μ ^ ( ω 1 ) − μ R ( ω 1 ) ∣ = 1 + ∑ i = 2 ∣Ω∣ ρ ^ i 1 1 − 1 + ∑ i = 2 ∣Ω∣ μ R ( ω 1 ) μ R ( ω i ) 1 ≤ i = 2 ∑ ∣Ω∣ ρ ^ i 1 − i = 2 ∑ ∣Ω∣ μ R ( ω 1 ) μ R ( ω i ) ≤ ∣Ω∣ p 0 4 ε ′ .
Then, for every i = 2 , … , ∣ Ω ∣ i=2,\ldots,|\Omega| i = 2 , … , ∣Ω∣ ,
∣ μ ^ ( ω i ) − μ R ( ω i ) ∣ = ∣ ρ ^ i 1 μ ^ ( ω 1 ) − μ R ( ω i ) μ R ( ω 1 ) μ R ( ω 1 ) ∣ ≤ ∣ ρ ^ i 1 − μ R ( ω i ) μ R ( ω 1 ) ∣ μ ^ ( ω 1 ) + μ R ( ω i ) μ R ( ω 1 ) ∣ μ ^ ( ω 1 ) − μ R ( ω 1 ) ∣ ≤ 4 ε ′ p 0 μ ^ ( ω 1 ) + μ R ( ω i ) μ R ( ω 1 ) ∣ Ω ∣ 4 ε ′ p 0 . \begin{aligned}
|\hat{\mu}(\omega_i)-\mu_R(\omega_i)|
&=\left|\hat{\rho}_{i1}\hat{\mu}(\omega_1)-\frac{\mu_R(\omega_i)}{\mu_R(\omega_1)}\mu_R(\omega_1)\right|\\
&\le\left|\hat{\rho}_{i1}-\frac{\mu_R(\omega_i)}{\mu_R(\omega_1)}\right|\hat{\mu}(\omega_1)+\frac{\mu_R(\omega_i)}{\mu_R(\omega_1)}|\hat{\mu}(\omega_1)-\mu_R(\omega_1)|\\
&\le\frac{4\varepsilon'}{p_0}\hat{\mu}(\omega_1)+\frac{\mu_R(\omega_i)}{\mu_R(\omega_1)}|\Omega|\frac{4\varepsilon'}{p_0}.
\end{aligned} ∣ μ ^ ( ω i ) − μ R ( ω i ) ∣ = ρ ^ i 1 μ ^ ( ω 1 ) − μ R ( ω 1 ) μ R ( ω i ) μ R ( ω 1 ) ≤ ρ ^ i 1 − μ R ( ω 1 ) μ R ( ω i ) μ ^ ( ω 1 ) + μ R ( ω 1 ) μ R ( ω i ) ∣ μ ^ ( ω 1 ) − μ R ( ω 1 ) ∣ ≤ p 0 4 ε ′ μ ^ ( ω 1 ) + μ R ( ω 1 ) μ R ( ω i ) ∣Ω∣ p 0 4 ε ′ .
Therefore,
∥ μ ^ − μ R ∥ 1 = ∣ μ ^ ( ω 1 ) − μ R ( ω 1 ) ∣ + ∑ i = 2 ∣ Ω ∣ ∣ μ ^ ( ω i ) − μ R ( ω i ) ∣ ≤ ∣ Ω ∣ 4 ε ′ p 0 + ∣ Ω ∣ 4 ε ′ p 0 μ ^ ( ω 1 ) + ∣ Ω ∣ 4 ε ′ p 0 ∑ i = 2 ∣ Ω ∣ μ R ( ω i ) μ R ( ω 1 ) ≤ ∣ Ω ∣ 8 ε ′ p 0 + ∣ Ω ∣ 4 ε ′ p 0 1 p 0 ≤ 12 ∣ Ω ∣ ε ′ p 0 2 = 12 ∣ Ω ∣ G p 0 3 ( τ + γ m ( 1 − γ ) G 3 p 0 ) . \begin{aligned}
\lVert\hat{\mu}-\mu_R\rVert_1 = \lvert\hat{\mu}(\omega_1)-\mu_R(\omega_1)\rvert+ \sum_{i=2}^{\lvert\Omega\rvert}\lvert\hat{\mu}(\omega_i)-\mu_R(\omega_i)\rvert\\
&\leq\lvert\Omega\rvert\frac{4\varepsilon'}{p_0}+\lvert\Omega\rvert\frac{4\varepsilon'}{p_0}\hat{\mu}(\omega_1)+\lvert\Omega\rvert\frac{4\varepsilon'}{p_0}\sum_{i=2}^{\lvert\Omega\rvert}\frac{\mu_R(\omega_i)}{\mu_R(\omega_1)} \\
&\leq\lvert\Omega\rvert\frac{8\varepsilon'}{p_0}+\lvert\Omega\rvert\frac{4\varepsilon'}{p_0}\frac{1}{p_0} \\
&\leq\frac{12\lvert\Omega\rvert\varepsilon'}{p_0^2} \\
&= \frac{12\lvert\Omega\rvert}{Gp_0^3}\left(\tau+\frac{\gamma^m}{(1-\gamma)G^3p_0}\right).
\end{aligned} ∥ μ ^ − μ R ∥ 1 = ∣ μ ^ ( ω 1 ) − μ R ( ω 1 )∣ + i = 2 ∑ ∣ Ω ∣ ∣ μ ^ ( ω i ) − μ R ( ω i )∣ ≤ ∣ Ω ∣ p 0 4 ε ′ + ∣ Ω ∣ p 0 4 ε ′ μ ^ ( ω 1 ) + ∣ Ω ∣ p 0 4 ε ′ i = 2 ∑ ∣ Ω ∣ μ R ( ω 1 ) μ R ( ω i ) ≤ ∣ Ω ∣ p 0 8 ε ′ + ∣ Ω ∣ p 0 4 ε ′ p 0 1 ≤ p 0 2 12 ∣ Ω ∣ ε ′ = G p 0 3 12 ∣ Ω ∣ ( τ + ( 1 − γ ) G 3 p 0 γ m ) .
which proves the first claim of Lemma 18.
Because Algorithm 3 runs Algorithm 2 for ∣ Ω ∣ − 1 \lvert\Omega\rvert-1 ∣ Ω ∣ − 1 times and the expected running time of Algorithm 2 is at most 2 ( 1 p 0 + m ) log 2 1 G p 0 τ 2\left(\frac{1}{p_0}+m\right)\log_2\frac{1}{Gp_0\tau} 2 ( p 0 1 + m ) log 2 G p 0 τ 1 periods (by Lemma 17), the expected running time of Algorithm 2 is at most
2 ( ∣ Ω ∣ − 1 ) ( 1 p 0 + m ) log 2 1 G p 0 τ . 2(\lvert\Omega\rvert-1)\left(\frac{1}{p_0}+m\right)\log_2\frac{1}{Gp_0\tau}. 2 (∣ Ω ∣ − 1 ) ( p 0 1 + m ) log 2 G p 0 τ 1 .
periods.
Proof of Lemma 3. Lemma 3 follows from Lemma 18 by plugging in
τ = G p 0 3 ε 24 ∣ Ω ∣ , m = ⌈ 1 ln ( 1 / γ ) ln ( 24 ∣ Ω ∣ ( 1 − γ ) G 4 p 0 6 ε ) ⌉ . \tau=\frac{Gp_0^3\varepsilon}{24\lvert\Omega\rvert},\qquad m=\left\lceil\frac{1}{\ln(1/\gamma)}\ln\left(\frac{24\lvert\Omega\rvert}{(1-\gamma)G^4p_0^6\varepsilon}\right)\right\rceil. τ = 24 ∣ Ω ∣ G p 0 3 ε , m = ⌈ ln ( 1/ γ ) 1 ln ( ( 1 − γ ) G 4 p 0 6 ε 24 ∣ Ω ∣ ) ⌉ .
Proof of Lemma 4
Let μ ^ ∈ Δ ( Ω ) \hat{\mu}\in\Delta(\Omega) μ ^ ∈ Δ ( Ω ) be a receiver prior satisfying min ω ∈ Ω μ ^ ( ω ) ≥ p 0 > 0 \min_{\omega\in\Omega}\hat{\mu}(\omega)\geq p_0>0 min ω ∈ Ω μ ^ ( ω ) ≥ p 0 > 0 . Let B 1 ( μ ^ , ε ) = { μ : ∥ μ − μ ^ ∥ 1 ≤ ε } B_1(\hat{\mu},\varepsilon)=\{\mu:\lVert\mu-\hat{\mu}\rVert_1\leq\varepsilon\} B 1 ( μ ^ , ε ) = { μ : ∥ μ − μ ^ ∥ 1 ≤ ε } be the set of priors within distance ε \varepsilon ε to μ ^ \hat{\mu} μ ^ . Suppose ε ≤ p 0 2 D 4 \varepsilon\leq\frac{p_0^2D}{4} ε ≤ 4 p 0 2 D . Let π \pi π be a direct signaling scheme. We want to construct another direct signaling scheme π ~ \tilde{\pi} π ~ in two steps: (1) First, construct a non-direct signaling scheme π ~ ∘ \tilde{\pi}^{\circ} π ~ ∘ . (2) Then, convert π ~ ∘ \tilde{\pi}^{\circ} π ~ ∘ into a direct signaling scheme π ~ \tilde{\pi} π ~ that still satisfies the requirements of the lemma.
Step (1): Construct a non-direct signaling scheme π ~ ∘ \tilde{\pi}^{\circ} π ~ ∘ . Let δ \delta δ satisfy
2 ε p 0 D ≤ δ ≤ p 0 2 . \frac{2\varepsilon}{p_0D}\leq\delta\leq\frac{p_0}{2}. p 0 D 2 ε ≤ δ ≤ 2 p 0 .
Let P ^ μ ^ , π ( a ) = ∑ ω ∈ Ω μ ^ ( a ) π ( a ∣ ω ) \widehat{P}_{\widehat{\mu},\pi}(a)=\sum_{\omega\in\Omega}\widehat{\mu}(a)\pi(a\mid\omega) P μ , π ( a ) = ∑ ω ∈ Ω μ ( a ) π ( a ∣ ω ) be the unconditional probability that π \pi π sends signal a a a under prior μ ^ \widehat{\mu} μ . Let μ ^ a , π ∈ Δ ( Ω ) \widehat{\mu}_{a,\pi}\in\Delta(\Omega) μ a , π ∈ Δ ( Ω ) be the posterior belief induced by signal a a a under signaling scheme π \pi π and prior μ ^ \widehat{\mu} μ :
μ ^ a , π ( ω ) = μ ^ ( ω ) π ( a ∣ ω ) P ^ μ ^ , π ( a ) , ∀ ω ∈ Ω . \widehat{\mu}_{a,\pi}(\omega)=\frac{\widehat{\mu}(\omega)\pi(a\mid\omega)}{\widehat{P}_{\widehat{\mu},\pi}(a)},\qquad\forall\omega\in\Omega. μ a , π ( ω ) = P μ , π ( a ) μ ( ω ) π ( a ∣ ω ) , ∀ ω ∈ Ω.
According to Assumption 2, there exists a belief η a ∈ Δ ( Ω ) \eta_a\in\Delta(\Omega) η a ∈ Δ ( Ω ) for which E ω ∼ η a [ v ( a , ω ) − v ( a ′ , ω ) ] ≥ D , ∀ a ′ ≠ a \mathbb{E}_{\omega\sim\eta_a}[v(a,\omega)-v(a',\omega)]\ge D,\ \forall a'\ne a E ω ∼ η a [ v ( a , ω ) − v ( a ′ , ω )] ≥ D , ∀ a ′ = a . Consider the convex combination of μ ^ a , π \widehat{\mu}_{a,\pi} μ a , π and η a \eta_a η a with coefficients 1 − δ , δ 1-\delta,\delta 1 − δ , δ :
ξ a = ( 1 − δ ) μ ^ a , π + δ η a . (37) \xi_a=(1-\delta)\widehat{\mu}_{a,\pi}+\delta\eta_a. \tag*{(37)} ξ a = ( 1 − δ ) μ a , π + δ η a . (37)
By the linearity of expectation, we have
E ω ∼ ξ a [ v ( a , ω ) − v ( a ′ , ω ) ] = ( 1 − δ ) E ω ∼ μ ^ a , π [ v ( a , ω ) − v ( a ′ , ω ) ] + δ E ω ∼ η a [ v ( a , ω ) − v ( a ′ , ω ) ] ≥ ( 1 − δ ) E ω ∼ μ ^ a , π [ v ( a , ω ) − v ( a ′ , ω ) ] + δ D . (38) \begin{aligned}
\mathbb{E}_{\omega\sim\xi_a}[v(a,\omega)-v(a',\omega)]=(1-\delta)\mathbb{E}_{\omega\sim\widehat{\mu}_{a,\pi}}[v(a,\omega)-v(a',\omega)]+\delta\mathbb{E}_{\omega\sim\eta_a}[v(a,\omega)-v(a',\omega)] \\
&\ge(1-\delta)\mathbb{E}_{\omega\sim\widehat{\mu}_{a,\pi}}[v(a,\omega)-v(a',\omega)]+\delta D. \tag*{(38)}
\end{aligned} E ω ∼ ξ a [ v ( a , ω ) − v ( a ′ , ω )] = ( 1 − δ ) E ω ∼ μ a , π [ v ( a , ω ) − v ( a ′ , ω )] + δ E ω ∼ η a [ v ( a , ω ) − v ( a ′ , ω )] ≥ ( 1 − δ ) E ω ∼ μ a , π [ v ( a , ω ) − v ( a ′ , ω )] + δ D . (38)
Let ξ = ∑ a ∈ A P ^ μ ^ , π ( a ) ξ a ∈ Δ ( Ω ) \xi=\sum_{a\in A}\widehat{P}_{\widehat{\mu},\pi}(a)\xi_a\in\Delta(\Omega) ξ = ∑ a ∈ A P μ , π ( a ) ξ a ∈ Δ ( Ω ) , and write μ ^ \widehat{\mu} μ as the convex combination of ξ \xi ξ and another belief χ ∈ Δ ( Ω ) \chi\in\Delta(\Omega) χ ∈ Δ ( Ω ) , with some coefficient y ∈ [ 0 , 1 ] y\in[0,1] y ∈ [ 0 , 1 ] :
μ ^ = ( 1 − y ) ξ + y χ = ∑ a ∈ A ( 1 − y ) P ^ μ ^ , π ( a ) ξ a + y χ . (39) \widehat{\mu}=(1-y)\xi+y\chi=\sum_{a\in A}(1-y)\widehat{P}_{\widehat{\mu},\pi}(a)\xi_a+y\chi. \tag*{(39)} μ = ( 1 − y ) ξ + y χ = a ∈ A ∑ ( 1 − y ) P μ , π ( a ) ξ a + y χ . (39)
Lemma 19 (Proposition 1 of Zu, Iyer, and Xu, 2021). If δ ≤ p 0 2 \delta\le\frac{p_0}{2} δ ≤ 2 p 0 , then there exist χ \chi χ on the boundary of Δ ( Ω ) \Delta(\Omega) Δ ( Ω ) and y ≤ δ p 0 ≤ 1 2 y\le\frac{\delta}{p_0}\le\frac{1}{2} y ≤ p 0 δ ≤ 2 1 that satisfy (38).
Since (38) is a convex decomposition of the prior μ ^ \widehat{\mu} μ , according to Kamenica and Gentzkow, 2011, there exists a signaling scheme π ~ ∘ \widetilde{\pi}^{\circ} π ∘ that induces posterior ξ a \xi_a ξ a with total probability ( 1 − y ) P ^ μ ^ , π ( a ) , ∀ a ∈ A (1-y)\widehat{P}_{\widehat{\mu},\pi}(a),\ \forall a\in A ( 1 − y ) P μ , π ( a ) , ∀ a ∈ A , and induces the posterior that puts all probability on ω \omega ω with total probability y χ ( ω ) , ∀ ω ∈ Ω y\chi(\omega),\ \forall\omega\in\Omega y χ ( ω ) , ∀ ω ∈ Ω . Namely, π ~ ∘ \widetilde{\pi}^{\circ} π ∘ has signal space S = A ∪ Ω S=A\cup\Omega S = A ∪ Ω and conditional probability
π ~ ∘ ( s ∣ ω ) = { ( 1 − y ) P ^ μ ^ , π ( a ) ξ a ( ω ) μ ^ ( ω ) for s = a ∈ A ; y χ ( ω ) μ ^ ( ω ) for s = ω ∈ Ω ; 0 otherwise. (40) \widetilde{\pi}^{\circ}(s\mid\omega)=
\begin{cases}
\frac{(1-y)\widehat{P}_{\widehat{\mu},\pi}(a)\xi_a(\omega)}{\widehat{\mu}(\omega)} & \text{for }s=a\in A;\\
\frac{y\chi(\omega)}{\widehat{\mu}(\omega)} & \text{for }s=\omega\in\Omega;\\
0 & \text{otherwise.}
\end{cases} \tag*{(40)} π ∘ ( s ∣ ω ) = ⎩ ⎨ ⎧ μ ( ω ) ( 1 − y ) P μ , π ( a ) ξ a ( ω ) μ ( ω ) y χ ( ω ) 0 for s = a ∈ A ; for s = ω ∈ Ω ; otherwise. (40)
It is not hard to verify that, under prior μ ^ \widehat{\mu} μ and signaling scheme π ~ ∘ \widetilde{\pi}^{\circ} π ∘ , the posterior induced by signal a ∈ A a\in A a ∈ A is equal to ξ a \xi_a ξ a , and the posterior induced by signal ω \omega ω is the deterministic distribution on ω \omega ω .
Claim 4. For any prior of the receiver μ ∈ B 1 ( μ ^ , ε ) \mu\in B_1(\widehat{\mu},\varepsilon) μ ∈ B 1 ( μ , ε ) , given any action recommendation $a\in A
from π ~ ∘ \tilde{\pi}^{\circ} π ~ ∘ , the receiver’s posterior μ a , π ~ ∘ \mu_{a,\tilde{\pi}^{\circ}} μ a , π ~ ∘ satisfies
E ω ∼ μ a , π ~ ∘ [ v ( a , ω ) − v ( a ′ , ω ) ] ≥ ( 1 − δ ) E ω ∼ μ ^ a , π [ v ( a , ω ) − v ( a ′ , ω ) ] + δ D − 2 ε p 0 , ∀ a ′ ∈ A ∖ { a } . \mathbb{E}_{\omega\sim\mu_{a,\tilde{\pi}^{\circ}}}[v(a,\omega)-v(a',\omega)] \ge(1-\delta)\mathbb{E}_{\omega\sim\hat{\mu}_{a,\pi}}[v(a,\omega)-v(a',\omega)]+\delta D-\frac{2\varepsilon}{p_0},\quad\forall a'\in A\setminus\{a\}. E ω ∼ μ a , π ~ ∘ [ v ( a , ω ) − v ( a ′ , ω )] ≥ ( 1 − δ ) E ω ∼ μ ^ a , π [ v ( a , ω ) − v ( a ′ , ω )] + δ D − p 0 2 ε , ∀ a ′ ∈ A ∖ { a } .
And when π ~ ∘ \tilde{\pi}^{\circ} π ~ ∘ sends signal ω ∈ Ω \omega\in\Omega ω ∈ Ω , the receiver will believe that the state is ω \omega ω deterministically.
Proof. Under signaling scheme π ~ ∘ \tilde{\pi}^{\circ} π ~ ∘ , let μ a , π ~ ∘ \mu_{a,\tilde{\pi}^{\circ}} μ a , π ~ ∘ , ξ a \xi_a ξ a be the posteriors induced by signal a a a from priors μ \mu μ , μ ^ \hat{\mu} μ ^ . By the continuity of posterior (Lemma 15), we have ∥ μ a , π ~ ∘ − ξ a ∥ 1 ≤ 2 p 0 ∥ μ − μ ^ ∥ 1 ≤ 2 ε p 0 \|\mu_{a,\tilde{\pi}^{\circ}}-\xi_a\|_1\le\frac{2}{p_0}\|\mu-\hat{\mu}\|_1\le\frac{2\varepsilon}{p_0} ∥ μ a , π ~ ∘ − ξ a ∥ 1 ≤ p 0 2 ∥ μ − μ ^ ∥ 1 ≤ p 0 2 ε . Then, since the receiver’s utility is in [ 0 , 1 ] [0,1] [ 0 , 1 ] , for any action a ′ ≠ a a'\ne a a ′ = a , we have
E ω ∼ μ a , π ~ ∘ [ v ( a , ω ) − v ( a ′ , ω ) ] ≥ E ω ∼ ξ a [ v ( a , ω ) − v ( a ′ , ω ) ] − 2 ε p 0 \mathbb{E}_{\omega\sim\mu_{a,\tilde{\pi}^{\circ}}}[v(a,\omega)-v(a',\omega)]\ge\mathbb{E}_{\omega\sim\xi_a}[v(a,\omega)-v(a',\omega)]-\frac{2\varepsilon}{p_0} E ω ∼ μ a , π ~ ∘ [ v ( a , ω ) − v ( a ′ , ω )] ≥ E ω ∼ ξ a [ v ( a , ω ) − v ( a ′ , ω )] − p 0 2 ε
by (38)
≥ ( 1 − δ ) E ω ∼ μ ^ a , π [ v ( a , ω ) − v ( a ′ , ω ) ] + δ D − 2 ε p 0 . \ge(1-\delta)\mathbb{E}_{\omega\sim\hat{\mu}_{a,\pi}}[v(a,\omega)-v(a',\omega)]+\delta D-\frac{2\varepsilon}{p_0}. ≥ ( 1 − δ ) E ω ∼ μ ^ a , π [ v ( a , ω ) − v ( a ′ , ω )] + δ D − p 0 2 ε .
We show that the recommendation part of signaling scheme π ~ ∘ \tilde{\pi}^{\circ} π ~ ∘ is close to π \pi π , in the following sense:
Claim 5. For every ω ∈ Ω \omega\in\Omega ω ∈ Ω , the ℓ 1 \ell_1 ℓ 1 distance ∥ π ~ ∘ ( ⋅ ∣ ω ) − π ( ⋅ ∣ ω ) ∥ 1 = ∑ a ∈ A ∣ π ~ ∘ ( a ∣ ω ) − π ( a ∣ ω ) ∣ ≤ 3 δ p 0 \|\tilde{\pi}^{\circ}(\cdot|\omega)-\pi(\cdot|\omega)\|_1=\sum_{a\in A}|\tilde{\pi}^{\circ}(a|\omega)-\pi(a|\omega)|\le\frac{3\delta}{p_0} ∥ π ~ ∘ ( ⋅ ∣ ω ) − π ( ⋅ ∣ ω ) ∥ 1 = ∑ a ∈ A ∣ π ~ ∘ ( a ∣ ω ) − π ( a ∣ ω ) ∣ ≤ p 0 3 δ .
Proof. By definition,
∑ a ∈ A ∣ π ~ ∘ ( a ∣ ω ) − π ( a ∣ ω ) ∣ = ∑ a ∈ A ∣ ( 1 − y ) P μ ^ , π ( a ) ξ a ( ω ) μ ^ ( ω ) − P μ ^ , π ( a ) μ ^ a ( ω ) μ ^ ( ω ) ∣ ≤ ∑ a ∈ A ( P μ ^ , π ( a ) μ ^ ( ω ) ⋅ ∣ ξ a ( ω ) − μ ^ a ( ω ) ∣ + y ⋅ P μ ^ , π ( a ) ξ a ( ω ) μ ^ ( ω ) ) \begin{aligned}
\sum_{a\in A}|\tilde{\pi}^{\circ}(a|\omega)-\pi(a|\omega)|
=\sum_{a\in A}\left|\frac{(1-y)P_{\hat{\mu},\pi}(a)\xi_a(\omega)}{\hat{\mu}(\omega)}-\frac{P_{\hat{\mu},\pi}(a)\hat{\mu}_a(\omega)}{\hat{\mu}(\omega)}\right| \\
&\le\sum_{a\in A}\left(\frac{P_{\hat{\mu},\pi}(a)}{\hat{\mu}(\omega)}\cdot|\xi_a(\omega)-\hat{\mu}_a(\omega)|+y\cdot\frac{P_{\hat{\mu},\pi}(a)\xi_a(\omega)}{\hat{\mu}(\omega)}\right)
\end{aligned} a ∈ A ∑ ∣ π ~ ∘ ( a ∣ ω ) − π ( a ∣ ω ) ∣ = a ∈ A ∑ μ ^ ( ω ) ( 1 − y ) P μ ^ , π ( a ) ξ a ( ω ) − μ ^ ( ω ) P μ ^ , π ( a ) μ ^ a ( ω ) ≤ a ∈ A ∑ ( μ ^ ( ω ) P μ ^ , π ( a ) ⋅ ∣ ξ a ( ω ) − μ ^ a ( ω ) ∣ + y ⋅ μ ^ ( ω ) P μ ^ , π ( a ) ξ a ( ω ) )
by (37) and y ≤ 1 2 y\le\frac{1}{2} y ≤ 2 1
≤ ∑ a ∈ A ( P μ ^ , π ( a ) μ ^ ( ω ) ⋅ δ + 2 y ⋅ ( 1 − y ) P μ ^ , π ( a ) ξ a ( ω ) μ ^ ( ω ) ) . \le\sum_{a\in A}\left(\frac{P_{\hat{\mu},\pi}(a)}{\hat{\mu}(\omega)}\cdot\delta+2y\cdot\frac{(1-y)P_{\hat{\mu},\pi}(a)\xi_a(\omega)}{\hat{\mu}(\omega)}\right). ≤ a ∈ A ∑ ( μ ^ ( ω ) P μ ^ , π ( a ) ⋅ δ + 2 y ⋅ μ ^ ( ω ) ( 1 − y ) P μ ^ , π ( a ) ξ a ( ω ) ) .
We have ∑ a ∈ A P μ ^ , π ( a ) = 1 \sum_{a\in A}P_{\hat{\mu},\pi}(a)=1 ∑ a ∈ A P μ ^ , π ( a ) = 1 and ∑ a ∈ A ( 1 − y ) P μ ^ , π ( a ) ξ a ( ω ) ≤ μ ^ ( ω ) \sum_{a\in A}(1-y)P_{\hat{\mu},\pi}(a)\xi_a(\omega)\le\hat{\mu}(\omega) ∑ a ∈ A ( 1 − y ) P μ ^ , π ( a ) ξ a ( ω ) ≤ μ ^ ( ω ) by (39) . So, the above is at most
≤ 1 μ ^ ( ω ) ⋅ δ + 2 y ⋅ 1 ≤ δ p 0 + 2 δ p 0 = 3 δ p 0 , \le\frac{1}{\hat{\mu}(\omega)}\cdot\delta+2y\cdot1\le\frac{\delta}{p_0}+2\frac{\delta}{p_0}=\frac{3\delta}{p_0}, ≤ μ ^ ( ω ) 1 ⋅ δ + 2 y ⋅ 1 ≤ p 0 δ + 2 p 0 δ = p 0 3 δ ,
given y ≤ δ p 0 y\le\frac{\delta}{p_0} y ≤ p 0 δ .
Then, we show that the information designer’s utility under scheme π ~ ∘ \tilde{\pi}^{\circ} π ~ ∘ is close to her utility under π \pi π :
Claim 6. The designer’s utility U ( π ~ ∘ , ρ o b ; μ D ) ≥ U ( π , ρ o b ; μ D ) − 3 δ p 0 U(\tilde{\pi}^{\circ},\rho_{\mathrm{ob}};\mu_D)\ge U(\pi,\rho_{\mathrm{ob}};\mu_D)-\frac{3\delta}{p_0} U ( π ~ ∘ , ρ ob ; μ D ) ≥ U ( π , ρ ob ; μ D ) − p 0 3 δ .
Proof. For the non-direct signaling scheme π ~ ∘ \tilde{\pi}^{\circ} π ~ ∘ , the notation ρ o b \rho_{\mathrm{ob}} ρ ob means: when π ~ ∘ \tilde{\pi}^{\circ} π ~ ∘ recommends action, the receiver takes the recommended action, and when π ~ ∘ \tilde{\pi}^{\circ} π ~ ∘ reveals the state, the receiver takes an optimal action for that state. The designer’s expected utility is thus
U ( π ~ ∘ , ρ o b ; μ D ) = ∑ ω ∈ Ω μ D ( ω ) ( ∑ a ∈ A π ~ ∘ ( a ∣ ω ) u ( a , ω ) + π ~ ∘ ( ω ∣ ω ) u ( a ω ∗ , ω ) ) ≥ ∑ ω ∈ Ω μ D ( ω ) ∑ a ∈ A π ~ ∘ ( a ∣ ω ) u ( a , ω ) ≥ U ( π , ρ o b ; μ D ) − 3 δ p 0 \begin{aligned}
U(\tilde{\pi}^{\circ},\rho_{\mathrm{ob}};\mu_D)=\sum_{\omega\in\Omega}\mu_D(\omega)\left(\sum_{a\in A}\tilde{\pi}^{\circ}(a\mid\omega)u(a,\omega)+\tilde{\pi}^{\circ}(\omega\mid\omega)u(a_{\omega}^{*},\omega)\right) \\
&\geq\sum_{\omega\in\Omega}\mu_D(\omega)\sum_{a\in A}\tilde{\pi}^{\circ}(a\mid\omega)u(a,\omega) \\
&\geq U(\pi,\rho_{\mathrm{ob}};\mu_D)-\frac{3\delta}{p_0}
\end{aligned} U ( π ~ ∘ , ρ ob ; μ D ) = ω ∈ Ω ∑ μ D ( ω ) ( a ∈ A ∑ π ~ ∘ ( a ∣ ω ) u ( a , ω ) + π ~ ∘ ( ω ∣ ω ) u ( a ω ∗ , ω ) ) ≥ ω ∈ Ω ∑ μ D ( ω ) a ∈ A ∑ π ~ ∘ ( a ∣ ω ) u ( a , ω ) ≥ U ( π , ρ ob ; μ D ) − p 0 3 δ
by Claim 5 and U U U is 1-Lipschitz in π \pi π (Lemma 14 ).
Step (2): Convert π ~ ∘ \tilde{\pi}^{\circ} π ~ ∘ into a direct signaling scheme π ~ \tilde{\pi} π ~ . Then, we convert π ~ ∘ \tilde{\pi}^{\circ} π ~ ∘ (whose signal space is A ∪ Ω A\cup\Omega A ∪ Ω ) into a direct signaling scheme π ~ \tilde{\pi} π ~ (whose signal space is A A A ) by coalescing the signals. Specifically, for each state ω ∈ Ω \omega\in\Omega ω ∈ Ω , and action a ∈ A a\in A a ∈ A , let π ~ \tilde{\pi} π ~ send signal a a a when π ~ ∘ \tilde{\pi}^{\circ} π ~ ∘ sends signal a a a and when π ~ ∘ \tilde{\pi}^{\circ} π ~ ∘ sends signal ω \omega ω if the receiver’s optimal action a ω ∗ ∈ arg max a ∈ A v ( a , ω ) a_{\omega}^{*}\in\arg\max_{a\in A}v(a,\omega) a ω ∗ ∈ arg max a ∈ A v ( a , ω ) for state ω \omega ω equals a a a :
π ~ ( a ∣ ω ) = π ~ ∘ ( a ∣ ω ) + 1 [ a = a ω ∗ ] ⋅ π ~ ∘ ( ω ∣ ω ) . (41) \tilde{\pi}(a\mid\omega)=\tilde{\pi}^{\circ}(a\mid\omega)+\mathbb{1}[a=a_{\omega}^{*}]\cdot\tilde{\pi}^{\circ}(\omega\mid\omega). \tag*{(41)} π ~ ( a ∣ ω ) = π ~ ∘ ( a ∣ ω ) + 1 [ a = a ω ∗ ] ⋅ π ~ ∘ ( ω ∣ ω ) . (41)
By the coalescing lemma Lemma 16, when π ~ \tilde{\pi} π ~ recommends action a a a , we have: for a receiver with any prior μ ∈ B 1 ( μ ^ , ϵ ) \mu\in B_1(\hat{\mu},\epsilon) μ ∈ B 1 ( μ ^ , ϵ ) , the posterior μ a , π ~ \mu_{a,\tilde{\pi}} μ a , π ~ satisfies
E ω ∼ μ a , π ~ [ v ( a , ω ) − v ( a ′ , ω ) ] ≥ min { E ω ∼ μ a , π ~ ∘ [ v ( a , ω ) − v ( a ′ , ω ) ] , 0 } \mathbb{E}_{\omega\sim\mu_{a,\tilde{\pi}}}\left[v(a,\omega)-v(a',\omega)\right]\geq\min\left\{\mathbb{E}_{\omega\sim\mu_{a,\tilde{\pi}^{\circ}}}\left[v(a,\omega)-v(a',\omega)\right],0\right\} E ω ∼ μ a , π ~ [ v ( a , ω ) − v ( a ′ , ω ) ] ≥ min { E ω ∼ μ a , π ~ ∘ [ v ( a , ω ) − v ( a ′ , ω ) ] , 0 }
(by xref‘Claim 4‘) ≥ min { ( 1 − δ ) E ω ∼ μ ^ a , π ~ [ v ( a , ω ) − v ( a ′ , ω ) ] + δ D − 2 ϵ p 0 , 0 } . \text{(by {xref}`Claim 4`)}\quad\geq\min\left\{(1-\delta)\mathbb{E}_{\omega\sim\hat{\mu}_{a,\tilde{\pi}}}\left[v(a,\omega)-v(a',\omega)\right]+\delta D-\frac{2\epsilon}{p_0},0\right\}. (by xref ‘Claim 4‘) ≥ min { ( 1 − δ ) E ω ∼ μ ^ a , π ~ [ v ( a , ω ) − v ( a ′ , ω ) ] + δ D − p 0 2 ϵ , 0 } .
We verify that π ~ \tilde{\pi} π ~ satisfies the condition for the receiver’s posterior in Lemma 4. There are two cases:
If E ω ∼ μ ^ a , π ~ [ v ( a , ω ) − v ( a ′ , ω ) ] ≥ 0 \mathbb{E}_{\omega\sim\hat{\mu}_{a,\tilde{\pi}}}[v(a,\omega)-v(a',\omega)]\geq0 E ω ∼ μ ^ a , π ~ [ v ( a , ω ) − v ( a ′ , ω )] ≥ 0 , then because δ D − 2 ϵ p 0 ≥ 0 \delta D-\frac{2\epsilon}{p_0}\geq0 δ D − p 0 2 ϵ ≥ 0 by our condition on δ \delta δ , we have min { ( 1 − δ ) E ω ∼ μ ^ a , π ~ [ v ( a , ω ) − v ( a ′ , ω ) ] + δ D − 2 ϵ p 0 , 0 } = 0 \min\{(1-\delta)\mathbb{E}_{\omega\sim\hat{\mu}_{a,\tilde{\pi}}}[v(a,\omega)-v(a',\omega)]+\delta D-\frac{2\epsilon}{p_0},0\}=0 min {( 1 − δ ) E ω ∼ μ ^ a , π ~ [ v ( a , ω ) − v ( a ′ , ω )] + δ D − p 0 2 ϵ , 0 } = 0 , so
E ω ∼ μ a , π ~ [ v ( a , ω ) − v ( a ′ , ω ) ] ≥ 0 = min { E ω ∼ μ ^ a , π ~ [ v ( a , ω ) − v ( a ′ , ω ) ] + δ D − 2 ϵ p 0 , 0 } . \mathbb{E}_{\omega\sim\mu_{a,\tilde{\pi}}}\left[v(a,\omega)-v(a',\omega)\right]\geq0=\min\left\{\mathbb{E}_{\omega\sim\hat{\mu}_{a,\tilde{\pi}}}\left[v(a,\omega)-v(a',\omega)\right]+\delta D-\frac{2\epsilon}{p_0},0\right\}. E ω ∼ μ a , π ~ [ v ( a , ω ) − v ( a ′ , ω ) ] ≥ 0 = min { E ω ∼ μ ^ a , π ~ [ v ( a , ω ) − v ( a ′ , ω ) ] + δ D − p 0 2 ϵ , 0 } .
If E ω ∼ μ ^ a , π ~ [ v ( a , ω ) − v ( a ′ , ω ) ] < 0 \mathbb{E}_{\omega\sim\hat{\mu}_{a,\tilde{\pi}}}[v(a,\omega)-v(a',\omega)]<0 E ω ∼ μ ^ a , π ~ [ v ( a , ω ) − v ( a ′ , ω )] < 0 , then we have 0 > ( 1 − δ ) E ω ∼ μ ^ a , π ~ [ v ( a , ω ) − v ( a ′ , ω ) ] ≥ 0>(1-\delta)\mathbb{E}_{\omega\sim\hat{\mu}_{a,\tilde{\pi}}}[v(a,\omega)-v(a',\omega)]\geq 0 > ( 1 − δ ) E ω ∼ μ ^ a , π ~ [ v ( a , ω ) − v ( a ′ , ω )] ≥
E ω ∼ μ ^ a , π [ v ( a , ω ) − v ( a ′ , ω ) ] \mathbb{E}_{\omega\sim\hat{\mu}_{a,\pi}}\left[v(a,\omega)-v(a',\omega)\right] E ω ∼ μ ^ a , π [ v ( a , ω ) − v ( a ′ , ω ) ] . So,
E ω ∼ μ a , π ~ [ v ( a , ω ) − v ( a ′ , ω ) ] ≥ min { ( 1 − δ ) E ω ∼ μ ^ a , π [ v ( a , ω ) − v ( a ′ , ω ) ] + δ D − 2 ε p 0 , 0 } ≥ min { E ω ∼ μ ^ a , π [ v ( a , ω ) − v ( a ′ , ω ) ] + δ D − 2 ε p 0 , 0 } . \begin{aligned}
\mathbb{E}_{\omega\sim\mu_{a,\tilde{\pi}}}\left[v(a,\omega)-v(a',\omega)\right]\ge\min\left\{(1-\delta)\mathbb{E}_{\omega\sim\hat{\mu}_{a,\pi}}\left[v(a,\omega)-v(a',\omega)\right]+\delta D-\frac{2\varepsilon}{p_0},0\right\} \\
&\ge\min\left\{\mathbb{E}_{\omega\sim\hat{\mu}_{a,\pi}}\left[v(a,\omega)-v(a',\omega)\right]+\delta D-\frac{2\varepsilon}{p_0},0\right\}.
\end{aligned} E ω ∼ μ a , π ~ [ v ( a , ω ) − v ( a ′ , ω ) ] ≥ min { ( 1 − δ ) E ω ∼ μ ^ a , π [ v ( a , ω ) − v ( a ′ , ω ) ] + δ D − p 0 2 ε , 0 } ≥ min { E ω ∼ μ ^ a , π [ v ( a , ω ) − v ( a ′ , ω ) ] + δ D − p 0 2 ε , 0 } .
In both cases, π ~ \tilde{\pi} π ~ satisfies the condition.
We then verify the condition for the designer’s utility, which is U ( π ~ , ρ o b ; μ D ) ≥ U ( π , ρ o b ; μ D ) − 3 δ p 0 U(\tilde{\pi},\rho_{\mathrm{ob}};\mu_D)\ge U(\pi,\rho_{\mathrm{ob}};\mu_D)-\frac{3\delta}{p_0} U ( π ~ , ρ ob ; μ D ) ≥ U ( π , ρ ob ; μ D ) − p 0 3 δ . It directly follows from Claim 6, which shows U ( π ~ ∘ , ρ o b ; μ D ) ≥ U ( π , ρ o b ; μ D ) − 3 δ p 0 U(\tilde{\pi}^{\circ},\rho_{\mathrm{ob}};\mu_D)\ge U(\pi,\rho_{\mathrm{ob}};\mu_D)-\frac{3\delta}{p_0} U ( π ~ ∘ , ρ ob ; μ D ) ≥ U ( π , ρ ob ; μ D ) − p 0 3 δ , and Lemma 16, which shows U ( π ~ ∘ , ρ o b ; μ D ) = U ( π ~ , ρ o b ; μ D ) U(\tilde{\pi}^{\circ},\rho_{\mathrm{ob}};\mu_D)=U(\tilde{\pi},\rho_{\mathrm{ob}};\mu_D) U ( π ~ ∘ , ρ ob ; μ D ) = U ( π ~ , ρ ob ; μ D ) .
Finally, we verify the closeness condition between π ~ \tilde{\pi} π ~ and π \pi π . For every ω ∈ Ω \omega\in\Omega ω ∈ Ω ,
∑ a ∈ A ∣ π ~ ( a ∣ ω ) − π ( a ∣ ω ) ∣ ≤ ∑ a ∈ A ( ∣ π ~ ∘ ( a ∣ ω ) − π ( a ∣ ω ) ∣ + 1 [ a = a w ∗ ] ⋅ π ~ ∘ ( ω ∣ ω ) ) by x r e f ‘ ( 40 ) ‘ ≤ 3 δ p 0 + y χ ( ω ) μ ^ ( ω ) by xref‘Claim 5‘ and xref‘Equation (39)‘ ≤ 3 δ p 0 + δ p 0 2 because y ≤ δ p 0 and μ ^ ( ω ) ≥ p 0 ≤ 4 δ p 0 2 . \begin{aligned}
\sum_{a\in A}\left|\tilde{\pi}(a\mid\omega)-\pi(a\mid\omega)\right|\le\sum_{a\in A}\left(\left|\tilde{\pi}^{\circ}(a\mid\omega)-\pi(a\mid\omega)\right|+\mathbb{1}[a=a_w^*]\cdot\tilde{\pi}^{\circ}(\omega\mid\omega)\right) \qquad\text{by } {xref}`(40)` \\
&\le\frac{3\delta}{p_0}+y\frac{\chi(\omega)}{\hat{\mu}(\omega)} \qquad\text{by {xref}`Claim 5` and {xref}`Equation (39)`} \\
&\le\frac{3\delta}{p_0}+\frac{\delta}{p_0^2} \qquad\text{because } y\le\frac{\delta}{p_0}\text{ and }\hat{\mu}(\omega)\ge p_0 \\
&\le\frac{4\delta}{p_0^2}.
\end{aligned} a ∈ A ∑ ∣ π ~ ( a ∣ ω ) − π ( a ∣ ω ) ∣ ≤ a ∈ A ∑ ( ∣ π ~ ∘ ( a ∣ ω ) − π ( a ∣ ω ) ∣ + 1 [ a = a w ∗ ] ⋅ π ~ ∘ ( ω ∣ ω ) ) by x r e f ‘ ( 40 ) ‘ ≤ p 0 3 δ + y μ ^ ( ω ) χ ( ω ) by xref ‘Claim 5‘ and xref ‘Equation (39)‘ ≤ p 0 3 δ + p 0 2 δ because y ≤ p 0 δ and μ ^ ( ω ) ≥ p 0 ≤ p 0 2 4 δ .
Omitted Proofs in Section 4
Proof of Lemma 5
Fix any feasible designer strategy σ = ( π σ ( t ) ) t = 1 T \sigma=(\pi_{\sigma}^{(t)})_{t=1}^{T} σ = ( π σ ( t ) ) t = 1 T and let the receiver play a (history-dependent) best response ϕ ∗ ( σ , μ R ) \phi^*(\sigma,\mu_R) ϕ ∗ ( σ , μ R ) . For each period t t t and realized public history h t h_t h t (signals and actions up to t − 1 t-1 t − 1 ), π σ ( t ) \pi_{\sigma}^{(t)} π σ ( t ) induces a posterior μ t ( ⋅ ∣ h t , s t ) \mu_t(\cdot\mid h_t,s_t) μ t ( ⋅ ∣ h t , s t ) after signal s t ∼ π σ ( t ) ( ⋅ ∣ ω , h t ) s_t\sim\pi_{\sigma}^{(t)}(\cdot\mid\omega,h_t) s t ∼ π σ ( t ) ( ⋅ ∣ ω , h t ) is sent under μ R \mu_R μ R . Let B t ( h t , s t ) ⊆ A B_t(h_t,s_t)\subseteq A B t ( h t , s t ) ⊆ A be the nonempty set of myopic best replies at ( h t , s t ) (h_t,s_t) ( h t , s t ) :
B t ( h t , s t ) : = arg max a ∈ A V t ( a ; μ t ( ⋅ ∣ h t , s t ) ) , B_t(h_t,s_t):=\underset{a\in A}{\arg\max}\ V_t(a;\mu_t(\cdot\mid h_t,s_t)), B t ( h t , s t ) := a ∈ A arg max V t ( a ; μ t ( ⋅ ∣ h t , s t )) ,
and fix a measurable tie-breaking rule τ t \tau_t τ t so that b t ( h t , s t ) : = τ t ( B t ( h t , s t ) ) ∈ B t ( h t , s t ) b_t(h_t,s_t):=\tau_t(B_t(h_t,s_t))\in B_t(h_t,s_t) b t ( h t , s t ) := τ t ( B t ( h t , s t )) ∈ B t ( h t , s t ) .
Direct, dynamically persuasive transformation. Define a direct signal in period t t t by post-processing the original signal via
g t : s t ↦ a t ′ = b t ( h t , s t ) ∈ A , g_t:s_t\mapsto a'_t=b_t(h_t,s_t)\in A, g t : s t ↦ a t ′ = b t ( h t , s t ) ∈ A ,
and set π σ d ( t ) ( ⋅ ∣ ω , h t ) : = ( g t ∘ π σ ( t ) ) ( ⋅ ∣ ω , h t ) \pi_{\sigma^d}^{(t)}(\cdot\mid\omega,h_t):=(g_t\circ\pi_{\sigma}^{(t)})(\cdot\mid\omega,h_t) π σ d ( t ) ( ⋅ ∣ ω , h t ) := ( g t ∘ π σ ( t ) ) ( ⋅ ∣ ω , h t ) , where σ d = ( π σ d ( t ) ) t = 1 T \sigma^d=(\pi_{\sigma^d}^{(t)})_{t=1}^{T} σ d = ( π σ d ( t ) ) t = 1 T denotes the resulting strategy. Because g t g_t g t is a (history-dependent) garbling of s t s_t s t , the posterior belief μ t ( ⋅ ∣ h t , s t ) \mu_t(\cdot\mid h_t,s_t) μ t ( ⋅ ∣ h t , s t ) induced by π σ ( t ) \pi_{\sigma}^{(t)} π σ ( t ) is the same posterior belief the receiver holds about states upon receiving the recommendation a t ′ = a'_t= a t ′ = g t ( s t ) g_t(s_t) g t ( s t ) under strategy σ d \sigma^d σ d (Bayes plausibility is preserved by post-processing). By construction, a t ′ a'_t a t ′ maximizes V t ( ⋅ ; μ t ) V_t(\cdot;\mu_t) V t ( ⋅ ; μ t ) , so obeying the recommendation is optimal for the receiver at each ( h t , a t ′ ) (h_t,a'_t) ( h t , a t ′ ) . Therefore there is a best response ϕ o b \phi_{\mathrm{ob}} ϕ ob under σ d \sigma^d σ d that obeys the recommendation every period. That is, ϕ o b = ϕ ∗ ( σ d , μ R ) \phi_{\mathrm{ob}}=\phi^*(\sigma^d,\mu_R) ϕ ob = ϕ ∗ ( σ d , μ R ) and σ d \sigma^d σ d is dynamically persuasive.
Outcome equivalence and optimality. Consider the outcome path generated by ( σ , ϕ ∗ ( σ , μ R ) ) (\sigma,\phi^*(\sigma,\mu_R)) ( σ , ϕ ∗ ( σ , μ R )) . At each ( h t , s t ) (h_t,s_t) ( h t , s t ) , the action actually taken is some a t ∈ B t ( h t , s t ) a_t\in B_t(h_t,s_t) a t ∈ B t ( h t , s t ) . Under ( σ d , ϕ o b ) (\sigma^d,\phi_{\mathrm{ob}}) ( σ d , ϕ ob ) , the action taken at the same ( h t , s t ) (h_t,s_t) ( h t , s t ) is exactly a t ′ = b t ( h t , s t ) ∈ B t ( h t , s t ) a'_t=b_t(h_t,s_t)\in B_t(h_t,s_t) a t ′ = b t ( h t , s t ) ∈ B t ( h t , s t ) . Hence, conditional on ( ω , h t , s t ) (\omega,h_t,s_t) ( ω , h t , s t ) , the action under σ d \sigma^d σ d is a selection from the same set of best replies as under σ \sigma σ ; in particular, the receiver payoff is weakly higher and the distribution over next-period histories (and thus all future posteriors) is preserved. Therefore the designer’s expected payoff is unchanged:
U T ( σ d , ϕ o b ; μ D ) = U T ( σ , ϕ ∗ ( σ , μ R ) ; μ D ) . U_T(\sigma^d,\phi_{\mathrm{ob}};\mu_D)=U_T(\sigma,\phi^*(\sigma,\mu_R);\mu_D). U T ( σ d , ϕ ob ; μ D ) = U T ( σ , ϕ ∗ ( σ , μ R ) ; μ D ) .
Applying that transformation to an optimal σ \sigma σ yields a history-dependent, dynamically persuasive direct strategy σ ∗ \sigma^* σ ∗ that attains the dynamic optimum:
U T ∗ ∗ ( T ) = U T ( σ ∗ , ϕ o b ; μ D ) with ϕ o b = ϕ ∗ ( σ ∗ , μ R ) . U_T^{**}(T)=U_T(\sigma^*,\phi_{\mathrm{ob}};\mu_D)\quad\text{with}\quad\phi_{\mathrm{ob}}=\phi^*(\sigma^*,\mu_R). U T ∗∗ ( T ) = U T ( σ ∗ , ϕ ob ; μ D ) with ϕ ob = ϕ ∗ ( σ ∗ , μ R ) .
Proof of Lemma 6
Fix an optimal DDP strategy σ ∗ \sigma^* σ ∗ (possibly history-dependent) with an obedient receiver strategy ϕ o b \phi_{\mathrm{ob}} ϕ ob , over a finite horizon t = 1 , … , T t=1,\ldots,T t = 1 , … , T . Let the state process { ω t } t = 1 T \{\omega_t\}_{t=1}^T { ω t } t = 1 T be i.i.d. with (true) marginal λ ∈ Δ ( Ω ) \lambda\in\Delta(\Omega) λ ∈ Δ ( Ω ) , independent of past play. For each period t t t and action a ∈ A a\in A a ∈ A , define the on-path joint and conditional probabilities induced by ( σ , ϕ o b ) (\sigma,\phi_{\mathrm{ob}}) ( σ , ϕ ob ) :
Q t ( ω , a ) : = Pr σ , ϕ o b ( ω t = ω , a t = a ) , P t ( a ∣ ω ) : = Pr σ , ϕ o b ( a t = a ∣ ω t = ω ) . Q_t(\omega,a):=\Pr_{\sigma,\phi_{\mathrm{ob}}}(\omega_t=\omega,a_t=a),\qquad P_t(a\mid\omega):=\Pr_{\sigma,\phi_{\mathrm{ob}}}(a_t=a\mid\omega_t=\omega). Q t ( ω , a ) := σ , ϕ ob Pr ( ω t = ω , a t = a ) , P t ( a ∣ ω ) := σ , ϕ ob Pr ( a t = a ∣ ω t = ω ) .
where a t a_t a t denotes the action recommended (and, on-path, taken) at t t t . By the law of total probability,
Q t ( ω , a ) = λ ( ω ) P t ( a ∣ ω ) . Q_t(\omega,a)=\lambda(\omega)P_t(a\mid\omega). Q t ( ω , a ) = λ ( ω ) P t ( a ∣ ω ) .
For each t t t , define a history-independent direct scheme π σ ~ ( t ) : Ω → Δ ( A ) \pi_{\tilde{\sigma}}^{(t)}:\Omega\to\Delta(A) π σ ~ ( t ) : Ω → Δ ( A ) by
π σ ~ ( t ) ( a ∣ ω ) : = P t ( a ∣ ω ) (choose arbitrarily on { ω : λ ( ω ) = 0 } ) . \pi_{\tilde{\sigma}}^{(t)}(a\mid\omega):=P_t(a\mid\omega)\qquad\text{(choose arbitrarily on }\{\omega:\lambda(\omega)=0\}\text{)}. π σ ~ ( t ) ( a ∣ ω ) := P t ( a ∣ ω ) (choose arbitrarily on { ω : λ ( ω ) = 0 } ) .
Let σ ~ = ( π σ ~ ( 1 ) , … , π σ ~ ( T ) ) \tilde{\sigma}=(\pi_{\tilde{\sigma}}^{(1)},\ldots,\pi_{\tilde{\sigma}}^{(T)}) σ ~ = ( π σ ~ ( 1 ) , … , π σ ~ ( T ) ) , and keep the same off-path continuation rule as under σ ∗ \sigma^* σ ∗ : following any deviation, switch to the no-information punishment policy as in σ ∗ \sigma^* σ ∗ for the remainder of the game. By construction,
Pr σ ~ , ϕ o b ( ω t = ω , a t = a ) = λ ( ω ) π σ ~ ( t ) ( a ∣ ω ) = λ ( ω ) P t ( a ∣ ω ) = Q t ( ω , a ) . \Pr_{\tilde{\sigma},\phi_{\mathrm{ob}}}(\omega_t=\omega,a_t=a)=\lambda(\omega)\pi_{\tilde{\sigma}}^{(t)}(a\mid\omega)=\lambda(\omega)P_t(a\mid\omega)=Q_t(\omega,a). σ ~ , ϕ ob Pr ( ω t = ω , a t = a ) = λ ( ω ) π σ ~ ( t ) ( a ∣ ω ) = λ ( ω ) P t ( a ∣ ω ) = Q t ( ω , a ) .
so the joint law of ( ω t , a t ) (\omega_t,a_t) ( ω t , a t ) is identical under ( σ ∗ , ϕ o b ) (\sigma^*,\phi_{\mathrm{ob}}) ( σ ∗ , ϕ ob ) and ( σ ~ , ϕ o b ) (\tilde{\sigma},\phi_{\mathrm{ob}}) ( σ ~ , ϕ ob ) for every t t t . Because payoffs are additively separable across periods, the designer’s total expected payoff is the same under σ ~ \tilde{\sigma} σ ~ as under σ ∗ \sigma^* σ ∗ . Thus, σ ~ \tilde{\sigma} σ ~ is optimal if σ \sigma σ is.
Fix a period t t t and a recommended action a a a with Pr ( a t = a ) > 0 \Pr(a_t=a)>0 Pr ( a t = a ) > 0 . Let μ R \mu_R μ R be the receiver’s prior. Under σ ~ \tilde{\sigma} σ ~ , Bayes’ rule gives the posterior
μ t σ ~ ( ω ∣ a ) = μ R ( ω ) π σ ~ ( t ) ( a ∣ ω ) ∑ ω ′ μ R ( ω ′ ) π σ ~ ( t ) ( a ∣ ω ′ ) = μ R ( ω ) P t ( a ∣ ω ) ∑ ω ′ μ R ( ω ′ ) P t ( a ∣ ω ′ ) . \mu_t^{\tilde{\sigma}}(\omega\mid a)=\frac{\mu_R(\omega)\pi_{\tilde{\sigma}}^{(t)}(a\mid\omega)}{\sum_{\omega'}\mu_R(\omega')\pi_{\tilde{\sigma}}^{(t)}(a\mid\omega')}=\frac{\mu_R(\omega)P_t(a\mid\omega)}{\sum_{\omega'}\mu_R(\omega')P_t(a\mid\omega')}. μ t σ ~ ( ω ∣ a ) = ∑ ω ′ μ R ( ω ′ ) π σ ~ ( t ) ( a ∣ ω ′ ) μ R ( ω ) π σ ~ ( t ) ( a ∣ ω ) = ∑ ω ′ μ R ( ω ′ ) P t ( a ∣ ω ′ ) μ R ( ω ) P t ( a ∣ ω ) .
The right-hand side is exactly the average of the posteriors induced by ( σ ∗ , ϕ o b ) (\sigma^*,\phi_{\mathrm{ob}}) ( σ ∗ , ϕ ob ) at all on-path histories that yield recommendation a a a , weighted by their probability conditional on a a a (law of iterated expectations). So we have
μ t σ ~ ( ⋅ ∣ a ) = μ ˉ t σ ∗ ( ⋅ ∣ a ) . \mu_t^{\tilde{\sigma}}(\cdot\mid a)=\bar{\mu}_t^{\sigma^*}(\cdot\mid a). μ t σ ~ ( ⋅ ∣ a ) = μ ˉ t σ ∗ ( ⋅ ∣ a ) .
where μ ˉ t σ ∗ ( ⋅ ∣ a ) \bar{\mu}_t^{\sigma^*}(\cdot\mid a) μ ˉ t σ ∗ ( ⋅ ∣ a ) is the averaged posterior faced by the receiver under ( σ ∗ , ϕ o b ) (\sigma^*,\phi_{\mathrm{ob}}) ( σ ∗ , ϕ ob ) when recommendation a a a is observed.
Let W t + 1 g W_{t+1}^{g} W t + 1 g (resp. W t + 1 p W_{t+1}^{p} W t + 1 p ) denote the receiver’s continuation utility from t + 1 t+1 t + 1 onward on the obedience (resp. punishment) path. By our construction of off-path behavior, these continuations are the same under σ ~ \tilde{\sigma} σ ~ as under σ ∗ \sigma^* σ ∗ :
W t + 1 g , σ ~ = W t + 1 g , σ ∗ , W t + 1 p , σ ~ = W t + 1 p , σ ∗ . W_{t+1}^{g,\tilde{\sigma}}=W_{t+1}^{g,\sigma^*},\qquad W_{t+1}^{p,\tilde{\sigma}}=W_{t+1}^{p,\sigma^*}. W t + 1 g , σ ~ = W t + 1 g , σ ∗ , W t + 1 p , σ ~ = W t + 1 p , σ ∗ .
Therefore, for every deviation a ′ ≠ a a'\ne a a ′ = a , the period-t t t one-shot deviation inequality under σ ~ \tilde{\sigma} σ ~ ,
E ω ∼ μ t σ ~ ( ⋅ ∣ a ) [ v ( a , ω ) − v ( a ′ , ω ) ] + ( W t + 1 g , σ ~ − W t + 1 p , σ ~ ) ≥ 0 , \mathbb{E}_{\omega\sim\mu_t^{\tilde{\sigma}}(\cdot\mid a)}[v(a,\omega)-v(a',\omega)]+(W_{t+1}^{g,\tilde{\sigma}}-W_{t+1}^{p,\tilde{\sigma}})\ge0, E ω ∼ μ t σ ~ ( ⋅ ∣ a ) [ v ( a , ω ) − v ( a ′ , ω )] + ( W t + 1 g , σ ~ − W t + 1 p , σ ~ ) ≥ 0 ,
coincides with the corresponding inequality under ( σ ∗ , ϕ o b ) (\sigma^*,\phi_{\mathrm{ob}}) ( σ ∗ , ϕ ob ) , which holds because σ \sigma σ is dynamically persuasive. By the one-shot deviation principle, obedience is optimal at all on-path information sets under σ ~ \tilde{\sigma} σ ~ as well.
Combining the above, σ ~ \tilde{\sigma} σ ~ is an optimal DDP strategy and is history-independent by construction.
Proof of Lemma 8
To begin, we consider the restriction to history-independent DDP. By Proposition 1, the benchmark
U T ∗ ∗ ( I ) = max σ U T ( σ ; ρ o b , μ D ) U_T^{**}(\mathcal{I}) = \max_{\sigma} U_T(\sigma; \rho_{\mathrm{ob}}, \mu_D) U T ∗∗ ( I ) = σ max U T ( σ ; ρ ob , μ D )
is attained by some history-independent, dynamically persuasive DDP strategy σ = ( π σ ( 1 ) , … , π σ ( T ) ) \sigma= \left(\pi_{\sigma}^{(1)}, \ldots, \pi_{\sigma}^{(T)}\right) σ = ( π σ ( 1 ) , … , π σ ( T ) ) . So we may restrict attention to this class and impose the dynamic persuasiveness constraint in its posterior form (15) .
We then consider a promise state and two primitive constraints.
For any such σ \sigma σ , define the promised payoff above the uninformed utility from period t t t to T T T as
g t ( σ ) : = ∑ t ′ = t T γ t ′ − t ( V ( π σ ( t ′ ) , ρ o b ; μ R ) − V u n i n f o r m e d ( μ R ) ) ( ≥ 0 ) . g_t(\sigma) := \sum_{t'=t}^{T} \gamma^{t'-t}\left(V\left(\pi_{\sigma}^{(t')}, \rho_{\mathrm{ob}}; \mu_R\right) - V_{\mathrm{uninformed}}\left(\mu_R\right)\right) \quad(\ge0). g t ( σ ) := t ′ = t ∑ T γ t ′ − t ( V ( π σ ( t ′ ) , ρ ob ; μ R ) − V uninformed ( μ R ) ) ( ≥ 0 ) .
Dynamic persuasiveness (15) at period t t t recommendation a a a is equivalent to
E ω ∼ μ R ( ⋅ ∣ a , π σ ( t ) ) [ v ( a , ω ) − v ( a ′ , ω ) ] + γ ⋅ g t + 1 ( σ ) ≥ 0 ∀ a ′ ∈ A and on-path a , (42) \mathbb{E}_{\omega\sim\mu_R(\cdot\mid a,\pi_{\sigma}^{(t)})}\left[v(a,\omega)-v(a',\omega)\right] + \gamma\cdot g_{t+1}(\sigma) \ge0 \qquad\forall a' \in A \text{ and on-path } a, \tag*{(42)} E ω ∼ μ R ( ⋅ ∣ a , π σ ( t ) ) [ v ( a , ω ) − v ( a ′ , ω ) ] + γ ⋅ g t + 1 ( σ ) ≥ 0 ∀ a ′ ∈ A and on-path a , (42)
i.e., a posterior single-period deviation inequality with continuation buffer γ ⋅ g t + 1 \gamma\cdot g_{t+1} γ ⋅ g t + 1 . Moreover, g t ( σ ) g_t(\sigma) g t ( σ ) satisfies the constraint of the receiver’s ex ante obedience
( V ( π σ ( t ) , ρ o b ; μ R ) − V u n i n f o r m e d ( μ R ) ) + γ ⋅ g t + 1 ( σ ) ≥ g t ( σ ) . (43) \left(V\left(\pi_{\sigma}^{(t)}, \rho_{\mathrm{ob}}; \mu_R\right) - V_{\mathrm{uninformed}}\left(\mu_R\right)\right) + \gamma\cdot g_{t+1}(\sigma) \ge g_t(\sigma). \tag*{(43)} ( V ( π σ ( t ) , ρ ob ; μ R ) − V uninformed ( μ R ) ) + γ ⋅ g t + 1 ( σ ) ≥ g t ( σ ) . (43)
Then we show that the dynamic programming solution upper bounds the benchmark. We show by backward induction on t t t that for every feasible history-independent dynamically persuasive σ \sigma σ ,
∑ t ′ = t T U ( π σ ( t ′ ) , ρ o b ; μ D ) ≤ F ( t , g t ( σ ) ) . (44) \sum_{t'=t}^{T} U\left(\pi_{\sigma}^{(t')}, \rho_{\mathrm{ob}}; \mu_D\right) \le F\left(t,g_t(\sigma)\right). \tag*{(44)} t ′ = t ∑ T U ( π σ ( t ′ ) , ρ ob ; μ D ) ≤ F ( t , g t ( σ ) ) . (44)
For t = T t = T t = T , feasibility requires (42) (which reduces to static persuasiveness) and (43) with g T g_T g T , which is exactly the constraint of (18) . Thus, we have U ( π σ ( T ) , ρ o b ; μ D ) ≤ F ( T , g T ( σ ) ) U\left(\pi_{\sigma}^{(T)}, \rho_{\mathrm{ob}}; \mu_D\right) \le F\left(T,g_T(\sigma)\right) U ( π σ ( T ) , ρ ob ; μ D ) ≤ F ( T , g T ( σ ) ) .
Assume (44) holds for t + 1 t+1 t + 1 . At period t t t , the pair ( π σ ( t ) , g t + 1 ( σ ) ) \left(\pi_{\sigma}^{(t)},g_{t+1}(\sigma)\right) ( π σ ( t ) , g t + 1 ( σ ) ) satisfies the constraints of the Bellman step (19) with g = g t ( σ ) g = g_t(\sigma) g = g t ( σ ) by (42) and (43) . Therefore,
∑ t ′ = t T U ( π σ ( t ′ ) , ρ o b ; μ D ) = U ( π σ ( t ) , ρ o b ; μ D ) + ∑ t ′ = t + 1 T U ( π σ ( t ′ ) , ρ o b ; μ D ) ≤ U ( π σ ( t ) , ρ o b ; μ D ) + F ( t + 1 , g t + 1 ( σ ) ) ≤ F ( t , g t ( σ ) ) , \begin{aligned}
\sum_{t'=t}^{T} U\left(\pi_{\sigma}^{(t')}, \rho_{\mathrm{ob}}; \mu_D\right)
&= U\left(\pi_{\sigma}^{(t)}, \rho_{\mathrm{ob}}; \mu_D\right) + \sum_{t'=t+1}^{T} U\left(\pi_{\sigma}^{(t')}, \rho_{\mathrm{ob}}; \mu_D\right) \\
&\le U\left(\pi_{\sigma}^{(t)}, \rho_{\mathrm{ob}}; \mu_D\right) + F\left(t+1,g_{t+1}(\sigma)\right) \\
&\le F\left(t,g_t(\sigma)\right),
\end{aligned} t ′ = t ∑ T U ( π σ ( t ′ ) , ρ ob ; μ D ) = U ( π σ ( t ) , ρ ob ; μ D ) + t ′ = t + 1 ∑ T U ( π σ ( t ′ ) , ρ ob ; μ D ) ≤ U ( π σ ( t ) , ρ ob ; μ D ) + F ( t + 1 , g t + 1 ( σ ) ) ≤ F ( t , g t ( σ ) ) ,
where the last inequality uses the maximization in (19) . Taking t = 1 t=1 t = 1 and using that F ( 1 , g ) F(1,g) F ( 1 , g ) is nonincreasing in g g g (tightening the promise cannot increase designer value), we get
U T ( σ ; ϕ o b , μ D ) ≤ F ( 1 , g 1 ( σ ) ) ≤ F ( 1 , 0 ) . U_T(\sigma;\phi_{\mathrm{ob}},\mu_D) \le F(1,g_1(\sigma)) \le F(1,0). U T ( σ ; ϕ ob , μ D ) ≤ F ( 1 , g 1 ( σ )) ≤ F ( 1 , 0 ) .
Maximizing over feasible σ \sigma σ yields U T ∗ ∗ ( I ) ≤ F ( 1 , 0 ) U_T^{**}(\mathcal{I}) \le F(1,0) U T ∗∗ ( I ) ≤ F ( 1 , 0 ) .
We finally show that the dynamic programming solution is tight.
Let { ( π σ ∗ ( t ) , g t + 1 ∗ ) } t = 1 T \{(\pi_{\sigma^*}^{(t)},g_{t+1}^*)\}_{t=1}^T {( π σ ∗ ( t ) , g t + 1 ∗ ) } t = 1 T be maximizers in (18) and (19) along an optimal path starting at F ( 1 , 0 ) F(1,0) F ( 1 , 0 ) (set g 1 ∗ : = 0 g_1^*:=0 g 1 ∗ := 0 ). By definition of F ( t + 1 , g t + 1 ∗ ) F(t+1,g_{t+1}^*) F ( t + 1 , g t + 1 ∗ ) , there exists a tail strategy ( π σ ∗ ( t + 1 ) , … , π σ ∗ ( T ) ) (\pi_{\sigma^*}^{(t+1)},\ldots,\pi_{\sigma^*}^{(T)}) ( π σ ∗ ( t + 1 ) , … , π σ ∗ ( T ) ) delivering the promise g t + 1 ∗ g_{t+1}^* g t + 1 ∗ and satisfying dynamic persuasiveness from t + 1 t+1 t + 1 onward. Concatenating these choices yields a history-independent DDP strategy σ ∗ = ( π σ ∗ ( t ) ) t = 1 T \sigma^*=(\pi_{\sigma^*}^{(t)})_{t=1}^T σ ∗ = ( π σ ∗ ( t ) ) t = 1 T .
We check the feasibility of σ ∗ \sigma^* σ ∗ : the pair ( π σ ∗ ( t ) , g t + 1 ∗ ) (\pi_{\sigma^*}^{(t)},g_{t+1}^*) ( π σ ∗ ( t ) , g t + 1 ∗ ) satisfies (19) :
V ( π σ ∗ ( t ) , ρ o b ; μ R ) − V u n i n f o r m e d ( μ R ) + γ ⋅ g t + 1 ∗ ≥ g t ∗ , (45) V(\pi_{\sigma^*}^{(t)},\rho_{\mathrm{ob}};\mu_R)-V_{\mathrm{uninformed}}(\mu_R)+\gamma\cdot g_{t+1}^*\ge g_t^*, \tag*{(45)} V ( π σ ∗ ( t ) , ρ ob ; μ R ) − V uninformed ( μ R ) + γ ⋅ g t + 1 ∗ ≥ g t ∗ , (45)
E ω ∼ μ R ( ⋅ ∣ a , π σ ∗ ( t ) ) [ v ( a , ω ) − v ( a ′ , ω ) ] + γ ⋅ g t + 1 ∗ ≥ 0 ∀ a ′ ∈ A , ∀ on-path a . (46) \mathbb{E}_{\omega\sim\mu_R(\cdot\mid a,\pi_{\sigma^*}^{(t)})}[v(a,\omega)-v(a',\omega)]+\gamma\cdot g_{t+1}^*\ge0\quad\forall a'\in A,\ \forall\text{ on-path }a. \tag*{(46)} E ω ∼ μ R ( ⋅ ∣ a , π σ ∗ ( t ) ) [ v ( a , ω ) − v ( a ′ , ω )] + γ ⋅ g t + 1 ∗ ≥ 0 ∀ a ′ ∈ A , ∀ on-path a . (46)
Thus promised utilities are kept (45) . For dynamic persuasiveness, fix t t t and an on-path recommendation a a a . The deviation at period t t t changes only the current action . Then, the DDP strategy switches to the (uninformative) punishment, whose continuation equals the uninformed payoff, while obedience secures an extra γ ⋅ g t + 1 ∗ \gamma\cdot g_{t+1}^* γ ⋅ g t + 1 ∗ above that baseline by construction. Thus, the receiver’s obedience minus deviation payoff is
E [ v ( a , ω ) − v ( a ′ , ω ) ∣ a ] ⏟ current gap + γ g t + 1 ∗ ⏟ continuation gap ≥ 0 (by xref‘(45)‘) , \underbrace{\mathbb{E}[v(a,\omega)-v(a',\omega)\mid a]}_{\text{current gap}}+\underbrace{\gamma g_{t+1}^*}_{\text{continuation gap}}\ge0\quad\text{(by {xref}`(45)`)}, current gap E [ v ( a , ω ) − v ( a ′ , ω ) ∣ a ] + continuation gap γ g t + 1 ∗ ≥ 0 (by xref ‘(45)‘) ,
so obedience is optimal at every on-path history, and thus σ ∗ \sigma^* σ ∗ is dynamically persuasive.
Finally, by the Bellman equalities,
U T ( σ ∗ ; ϕ o b , μ D ) = ∑ t = 1 T U ( π σ ∗ ( t ) , ρ o b ; μ D ) = F ( 1 , 0 ) . U_T(\sigma^*;\phi_{\mathrm{ob}},\mu_D)=\sum_{t=1}^T U(\pi_{\sigma^*}^{(t)},\rho_{\mathrm{ob}};\mu_D)=F(1,0). U T ( σ ∗ ; ϕ ob , μ D ) = t = 1 ∑ T U ( π σ ∗ ( t ) , ρ ob ; μ D ) = F ( 1 , 0 ) .
Combining with the dynamic programming solution upper bounding the benchmark, we obtain U T ∗ ∗ ( I ) = F ( 1 , 0 ) U_T^{**}(\mathcal{I})=F(1,0) U T ∗∗ ( I ) = F ( 1 , 0 ) , attained by the history-independent, dynamically persuasive DDP strategy σ ∗ \sigma^* σ ∗ .
Proof of Proposition 4
Proposition 4. For any instance I \mathcal{I} I and any T ≥ 1 T\ge1 T ≥ 1 , the designer’s dynamic benchmark U T ∗ ∗ ( I ) U_T^{**}(\mathcal{I}) U T ∗∗ ( I ) is weakly increasing in the receiver’s discount factor γ ∈ [ 0 , 1 ) \gamma\in[0,1) γ ∈ [ 0 , 1 ) .
Proof. Fix γ ′ < γ ′ ′ \gamma' < \gamma'' γ ′ < γ ′′ . Consider any DDP strategy σ \sigma σ that is dynamically persuasive (i.e., induces obedience) under γ ′ \gamma' γ ′ , together with the receiver’s obedient strategy ϕ o b \phi_{\mathrm{ob}} ϕ ob . The one-step obedience constraints compare current recommended action payoffs with any deviation payoffs plus the discounted continuation values. When γ \gamma γ increases from γ ′ \gamma' γ ′ to γ ′ ′ \gamma'' γ ′′ , the continuation part of these constraints is (weakly) upweighted, making obedience easier to sustain. Hence any ( σ , ϕ o b ) (\sigma,\phi_{\mathrm{ob}}) ( σ , ϕ ob ) feasible at γ ′ \gamma' γ ′ remains feasible at γ ′ ′ \gamma'' γ ′′ .
Therefore the feasible set of DDP+obedience outcomes at γ ′ ′ \gamma'' γ ′′ contains that at γ ′ \gamma' γ ′ , so the designer’s optimal value cannot decrease:
U T ∗ ∗ ( I ; γ ′ ) ≤ U T ∗ ∗ ( I ; γ ′ ′ ) . U_T^{**}(\mathcal{I};\gamma') \le U_T^{**}(\mathcal{I};\gamma''). U T ∗∗ ( I ; γ ′ ) ≤ U T ∗∗ ( I ; γ ′′ ) .
Since γ ′ , γ ′ ′ \gamma',\gamma'' γ ′ , γ ′′ were arbitrary with γ ′ < γ ′ ′ \gamma' < \gamma'' γ ′ < γ ′′ , U T ∗ ∗ ( I ) U_T^{**}(\mathcal{I}) U T ∗∗ ( I ) is weakly increasing in γ \gamma γ .
Proof of Proposition 3
Proof. Let π ∗ \pi^* π ∗ be an optimal signaling scheme that solves problem (21) , with corresponding designer payoff U ( π ∗ , ρ o b ; μ D ) = U I R ∗ U(\pi^*,\rho_{\mathrm{ob}};\mu_D)=U_{IR}^* U ( π ∗ , ρ ob ; μ D ) = U I R ∗ .
Proof of upper bound 1 T U T ∗ ∗ ( I ; γ ) ≤ U I R ∗ \frac{1}{T}U_T^{**}(\mathcal{I};\gamma) \le U_{IR}^* T 1 U T ∗∗ ( I ; γ ) ≤ U I R ∗ . By Proposition 1, the dynamic benchmark U T ∗ ∗ ( I ; γ ) U_T^{**}(\mathcal{I};\gamma) U T ∗∗ ( I ; γ ) can be achieved by some history-independent DDP strategies σ = ( π ( t ) ) t = 1 T \sigma=(\pi^{(t)})_{t=1}^T σ = ( π ( t ) ) t = 1 T that is dynamically persuasive. Define g t = ∑ t ′ = 1 T γ t ′ − t ( V ( π ( t ′ ) , ρ o b ; μ R ) − V u n i n f o r m e d ( μ R ) ) g_t=\sum_{t'=1}^{T}\gamma^{t'-t}\left(V(\pi^{(t')},\rho_{\mathrm{ob}};\mu_R)-V_{\mathrm{uninformed}}(\mu_R)\right) g t = ∑ t ′ = 1 T γ t ′ − t ( V ( π ( t ′ ) , ρ ob ; μ R ) − V uninformed ( μ R ) ) . By dynamic persuasiveness, we have g t ≥ 0 g_t\ge0 g t ≥ 0 . We also note that
V ( π ( t ) , ρ o b ; μ R ) − V u n i n f o r m e d ( μ R ) + γ g t + 1 = g t . (47) V(\pi^{(t)},\rho_{\mathrm{ob}};\mu_R)-V_{\mathrm{uninformed}}(\mu_R)+\gamma g_{t+1}=g_t. \tag*{(47)} V ( π ( t ) , ρ ob ; μ R ) − V uninformed ( μ R ) + γ g t + 1 = g t . (47)
Let π ˉ : = 1 T ∑ t = 1 T π ( t ) \bar{\pi}:=\frac{1}{T}\sum_{t=1}^{T}\pi^{(t)} π ˉ := T 1 ∑ t = 1 T π ( t ) be the average signaling scheme of the DDP strategy σ \sigma σ . We will show ∑ t = 1 T [ V ( π ( t ) , ρ o b ; μ R ) − V u n i n f o r m e d ( μ R ) ] ≥ 0 \sum_{t=1}^{T}\left[V(\pi^{(t)},\rho_{\mathrm{ob}};\mu_R)-V_{\mathrm{uninformed}}(\mu_R)\right]\ge0 ∑ t = 1 T [ V ( π ( t ) , ρ ob ; μ R ) − V uninformed ( μ R ) ] ≥ 0 . To see that, first, summing Eq. (47) over t = 1 , … , T t=1,\ldots,T t = 1 , … , T ,
∑ t = 1 T [ V ( π ( t ) , ρ o b ; μ R ) − V u n i n f o r m e d ( μ R ) ] + γ ∑ t = 1 T g t + 1 = ∑ t = 1 T g t . \sum_{t=1}^{T}\left[V(\pi^{(t)},\rho_{\mathrm{ob}};\mu_R)-V_{\mathrm{uninformed}}(\mu_R)\right]+\gamma\sum_{t=1}^{T}g_{t+1}=\sum_{t=1}^{T}g_t. t = 1 ∑ T [ V ( π ( t ) , ρ ob ; μ R ) − V uninformed ( μ R ) ] + γ t = 1 ∑ T g t + 1 = t = 1 ∑ T g t .
Rearranging,
∑ t = 1 T [ V ( π ( t ) , ρ o b ; μ R ) − V u n i n f o r m e d ( μ R ) ] = ∑ t = 1 T g t − γ ∑ t = 1 T g t + 1 = g 1 + ( 1 − γ ) ∑ t = 2 T g t . \sum_{t=1}^{T}\left[V(\pi^{(t)},\rho_{\mathrm{ob}};\mu_R)-V_{\mathrm{uninformed}}(\mu_R)\right]=\sum_{t=1}^{T}g_t-\gamma\sum_{t=1}^{T}g_{t+1}=g_1+(1-\gamma)\sum_{t=2}^{T}g_t. t = 1 ∑ T [ V ( π ( t ) , ρ ob ; μ R ) − V uninformed ( μ R ) ] = t = 1 ∑ T g t − γ t = 1 ∑ T g t + 1 = g 1 + ( 1 − γ ) t = 2 ∑ T g t .
Because g 1 , g t ≥ 0 g_1,g_t\ge0 g 1 , g t ≥ 0 and γ ≤ 1 \gamma\le1 γ ≤ 1 , we have
∑ t = 1 T [ V ( π ( t ) , ρ o b ; μ R ) − V u n i n f o r m e d ( μ R ) ] ≥ 0. \sum_{t=1}^{T}\left[V(\pi^{(t)},\rho_{\mathrm{ob}};\mu_R)-V_{\mathrm{uninformed}}(\mu_R)\right]\ge0. t = 1 ∑ T [ V ( π ( t ) , ρ ob ; μ R ) − V uninformed ( μ R ) ] ≥ 0.
This means that the average signaling scheme π ˉ \bar{\pi} π ˉ satisfies the receiver’s individual rationality constraint in optimization problem (21) , so it is a feasible solution, and the designer’s payoff is at most
U ( π ˉ , ρ o b ; μ D ) ≤ U I R ∗ . U(\bar{\pi},\rho_{\mathrm{ob}};\mu_D) \le U^*_{IR}. U ( π ˉ , ρ ob ; μ D ) ≤ U I R ∗ .
By linearity, U ( π ˉ , ρ o b ; μ D ) = 1 T ∑ t = 1 T U ( π ( t ) , ρ o b ; μ D ) = 1 T U T ∗ ∗ ( L ; γ ) U(\bar{\pi},\rho_{\mathrm{ob}};\mu_D)=\frac{1}{T}\sum_{t=1}^{T}U(\pi^{(t)},\rho_{\mathrm{ob}};\mu_D)=\frac{1}{T}U_T^{**}(\mathcal{L};\gamma) U ( π ˉ , ρ ob ; μ D ) = T 1 ∑ t = 1 T U ( π ( t ) , ρ ob ; μ D ) = T 1 U T ∗∗ ( L ; γ ) . So 1 T U T ∗ ∗ ( L ; γ ) ≤ U I R ∗ \frac{1}{T}U_T^{**}(\mathcal{L};\gamma)\le U^*_{IR} T 1 U T ∗∗ ( L ; γ ) ≤ U I R ∗ .
Proof of lower bound lim γ → 1 lim T → ∞ 1 T U T ∗ ( L ) ≥ U I R ∗ \lim_{\gamma\to1}\lim_{T\to\infty}\frac{1}{T}U_T^*(\mathcal{L})\ge U^*_{IR} lim γ → 1 lim T → ∞ T 1 U T ∗ ( L ) ≥ U I R ∗ . Let π f u l l \pi_{\mathrm{full}} π full be the “full-information” signaling scheme that reveals the optimal action a ω ∗ a^*_\omega a ω ∗ to the receiver at every state ω ∈ Ω \omega\in\Omega ω ∈ Ω , which gives the receiver expected payoff V ( π f u l l , ρ o b ; μ R ) V(\pi_{\mathrm{full}},\rho_{\mathrm{ob}};\mu_R) V ( π full , ρ ob ; μ R ) . Under Assumptions 1 and 3, it is easy to show that V ( π f u l l , ρ o b ; μ R ) − V u n i n f o r m e d ( μ R ) ≥ p 0 G V(\pi_{\mathrm{full}},\rho_{\mathrm{ob}};\mu_R)-V_{\mathrm{uninformed}}(\mu_R)\ge p_0G V ( π full , ρ ob ; μ R ) − V uninformed ( μ R ) ≥ p 0 G . Let π ′ \pi' π ′ be a direct signaling scheme obtained by mixing π ∗ \pi^* π ∗ with weight 1 − δ ′ 1-\delta' 1 − δ ′ and π f u l l \pi_{\mathrm{full}} π full with weight δ ′ \delta' δ ′ . Consider the DDP strategy σ ′ = ( π ′ ) t = 1 T \sigma'=(\pi')_{t=1}^{T} σ ′ = ( π ′ ) t = 1 T that repeats π ′ \pi' π ′ unless the receiver has deviated. We claim that, under σ ′ \sigma' σ ′ , the receiver is willing to obey the recommended actions in every period t t t from 1 to T − T T-\sqrt{T} T − T : to see this, given the recommendation at period t t t , the receiver’s obedient payoff from period t t t to T T T is at least
γ t ⋅ 0 + ∑ t ′ = t + 1 T γ t ′ V ( π ′ , ρ o b ; μ R ) . \gamma^t\cdot0+\sum_{t'=t+1}^{T}\gamma^{t'}V(\pi',\rho_{\mathrm{ob}};\mu_R). γ t ⋅ 0 + t ′ = t + 1 ∑ T γ t ′ V ( π ′ , ρ ob ; μ R ) .
If the receiver deviates from the recommendation in period t t t , he will obtain payoff at most 1 in period t t t and be punished starting from period t + 1 t+1 t + 1 , so his total payoff is at most
γ t ⋅ 1 + ∑ t ′ = t + 1 T γ t ′ V u n i n f o r m e d ( μ R ) . \gamma^t\cdot1+\sum_{t'=t+1}^{T}\gamma^{t'}V_{\mathrm{uninformed}}(\mu_R). γ t ⋅ 1 + t ′ = t + 1 ∑ T γ t ′ V uninformed ( μ R ) .
The difference is
γ t ⋅ 0 + ∑ t ′ = t + 1 T γ t ′ V ( π ′ , ρ o b ; μ R ) − γ t ⋅ 1 − ∑ t ′ = t + 1 T γ t ′ V u n i n f o r m e d ( μ R ) = − γ t + ∑ t ′ = t + 1 T γ t ′ [ V ( π ′ , ρ o b ; μ R ) − V u n i n f o r m e d ( μ R ) ] = − γ t + ∑ t ′ = t + 1 T γ t ′ [ ( 1 − δ ′ ) ( V ( π ∗ , ρ o b ; μ R ) − V u n i n f o r m e d ( μ R ) ) + δ ′ ( V ( π f u l l , ρ o b ; μ R ) − V u n i n f o r m e d ( μ R ) ) ] ≥ − γ t + ∑ t ′ = t + 1 T γ t ′ δ ′ p 0 G = − γ t + δ ′ p 0 G ⋅ γ t + 1 ⋅ 1 − γ T − t 1 − γ ≥ 0 \begin{aligned}
&\gamma^{t}\cdot0+\sum_{t'=t+1}^{T}\gamma^{t'}V(\pi',\rho_{\mathrm{ob}};\mu_R)-\gamma^{t}\cdot1-\sum_{t'=t+1}^{T}\gamma^{t'}V_{\mathrm{uninformed}}(\mu_R)\\
&=-\gamma^{t}+\sum_{t'=t+1}^{T}\gamma^{t'}\left[V(\pi',\rho_{\mathrm{ob}};\mu_R)-V_{\mathrm{uninformed}}(\mu_R)\right]\\
&=-\gamma^{t}+\sum_{t'=t+1}^{T}\gamma^{t'}\left[(1-\delta')\left(V(\pi^*,\rho_{\mathrm{ob}};\mu_R)-V_{\mathrm{uninformed}}(\mu_R)\right)+\delta'\left(V(\pi_{\mathrm{full}},\rho_{\mathrm{ob}};\mu_R)-V_{\mathrm{uninformed}}(\mu_R)\right)\right]\\
&\ge-\gamma^{t}+\sum_{t'=t+1}^{T}\gamma^{t'}\delta'p_0G\\
&=-\gamma^{t}+\delta'p_0G\cdot\gamma^{t+1}\cdot\frac{1-\gamma^{T-t}}{1-\gamma}\\
&\ge0
\end{aligned} γ t ⋅ 0 + t ′ = t + 1 ∑ T γ t ′ V ( π ′ , ρ ob ; μ R ) − γ t ⋅ 1 − t ′ = t + 1 ∑ T γ t ′ V uninformed ( μ R ) = − γ t + t ′ = t + 1 ∑ T γ t ′ [ V ( π ′ , ρ ob ; μ R ) − V uninformed ( μ R ) ] = − γ t + t ′ = t + 1 ∑ T γ t ′ [ ( 1 − δ ′ ) ( V ( π ∗ , ρ ob ; μ R ) − V uninformed ( μ R ) ) + δ ′ ( V ( π full , ρ ob ; μ R ) − V uninformed ( μ R ) ) ] ≥ − γ t + t ′ = t + 1 ∑ T γ t ′ δ ′ p 0 G = − γ t + δ ′ p 0 G ⋅ γ t + 1 ⋅ 1 − γ 1 − γ T − t ≥ 0
by choosing δ ′ = 1 − γ p 0 G γ ( 1 − γ T − t ) \delta'=\frac{1-\gamma}{p_0G\gamma(1-\gamma^{T-t})} δ ′ = p 0 G γ ( 1 − γ T − t ) 1 − γ . Note that δ ′ → 1 − γ p 0 G γ → 0 \delta'\to\frac{1-\gamma}{p_0G\gamma}\to0 δ ′ → p 0 G γ 1 − γ → 0 as T → ∞ T\to\infty T → ∞ and γ → 1 \gamma\to1 γ → 1 . This means that the receiver will obey the recommendations from period 1 to period T − T T-\sqrt{T} T − T . So, the designer’s average payoff is at least
1 T U ( σ , ϕ ∗ ( σ ) ; μ D ) ≥ 1 T ( ∑ t = 1 T − T U ( π ′ , ρ o b ; μ D ) + 0 ) ≥ 1 T ( ∑ t = 1 T − T ( 1 − δ ′ ) U ( π ∗ , ρ o b ; μ D ) + 0 ) = T − T T ( 1 − δ ′ ) ⋅ U ( π ∗ , ρ o b ; μ D ) → U ( π ∗ , ρ o b ; μ D ) \begin{aligned}
\frac{1}{T}U(\sigma,\phi^*(\sigma);\mu_D)\ge\frac{1}{T}\left(\sum_{t=1}^{T-\sqrt{T}}U(\pi',\rho_{\mathrm{ob}};\mu_D)+0\right) \\
&\ge\frac{1}{T}\left(\sum_{t=1}^{T-\sqrt{T}}(1-\delta')U(\pi^*,\rho_{\mathrm{ob}};\mu_D)+0\right) \\
&=\frac{T-\sqrt{T}}{T}(1-\delta')\cdot U(\pi^*,\rho_{\mathrm{ob}};\mu_D) \\
&\to U(\pi^*,\rho_{\mathrm{ob}};\mu_D)
\end{aligned} T 1 U ( σ , ϕ ∗ ( σ ) ; μ D ) ≥ T 1 t = 1 ∑ T − T U ( π ′ , ρ ob ; μ D ) + 0 ≥ T 1 t = 1 ∑ T − T ( 1 − δ ′ ) U ( π ∗ , ρ ob ; μ D ) + 0 = T T − T ( 1 − δ ′ ) ⋅ U ( π ∗ , ρ ob ; μ D ) → U ( π ∗ , ρ ob ; μ D )
as T → ∞ T\to\infty T → ∞ and γ → 1 \gamma\to1 γ → 1 , where the last step is because δ ′ → 0 \delta'\to0 δ ′ → 0 .
Omitted Proofs in Section 5
Proof of Lemma 9
Proof. Let π f u l l \pi_{\mathrm{full}} π full be the “full revelation” signaling scheme that reveals the optimal action a ω ∗ = max a ∈ A v ( a , ω ) a^*_{\omega}=\max_{a\in A}v(a,\omega) a ω ∗ = max a ∈ A v ( a , ω ) for the receiver for every state ω ∈ Ω \omega\in\Omega ω ∈ Ω . By Assumption 3, π f u l l \pi_{\mathrm{full}} π full is unique. Let ∥ π σ ^ − π f u l l ∥ = max ω ∈ Ω ∥ π σ ^ ( ⋅ ∣ ω ) − π f u l l ( ⋅ ∣ ω ) ∥ 1 \lVert\pi_{\hat{\sigma}}-\pi_{\mathrm{full}}\rVert=\max_{\omega\in\Omega}\lVert\pi_{\hat{\sigma}}(\cdot\mid\omega)-\pi_{\mathrm{full}}(\cdot\mid\omega)\rVert_1 ∥ π σ ^ − π full ∥ = max ω ∈ Ω ∥ π σ ^ ( ⋅ ∣ ω ) − π full ( ⋅ ∣ ω ) ∥ 1 be the matrix ℓ 1 \ell_1 ℓ 1 -distance between direct signaling schemes π \pi π and π f u l l \pi_{\mathrm{full}} π full . Let θ = 8 δ p 0 3 G \theta=\sqrt{\frac{8\delta}{p_0^3G}} θ = p 0 3 G 8 δ .
Case (1): ∥ π σ ^ − π f u l l ∥ ≤ θ \lVert\pi_{\hat{\sigma}}-\pi_{\mathrm{full}}\rVert\le\theta ∥ π σ ^ − π full ∥ ≤ θ . In this case, we let π σ ~ \pi_{\tilde{\sigma}} π σ ~ be π f u l l \pi_{\mathrm{full}} π full . Because π f u l l \pi_{\mathrm{full}} π full is persuasive regardless of the receiver’s prior, we immediately have
E ω ∼ μ ( ⋅ ∣ a , π σ ~ ) [ v ( a , ω ) − v ( a ′ , ω ) ] ≥ 0 ≥ min { E ω ∼ μ ^ ( ⋅ ∣ a , π σ ^ ) [ v ( a , ω ) − v ( a ′ , ω ) ] + δ D − 2 ε p 0 , 0 } . \mathbb{E}_{\omega\sim\mu(\cdot\mid a,\pi_{\tilde{\sigma}})}\left[v(a,\omega)-v(a',\omega)\right] \ge0 \ge\min\left\{\mathbb{E}_{\omega\sim\hat{\mu}(\cdot\mid a,\pi_{\hat{\sigma}})}\left[v(a,\omega)-v(a',\omega)\right]+\delta D-\frac{2\varepsilon}{p_0},0\right\}. E ω ∼ μ ( ⋅ ∣ a , π σ ~ ) [ v ( a , ω ) − v ( a ′ , ω ) ] ≥ 0 ≥ min { E ω ∼ μ ^ ( ⋅ ∣ a , π σ ^ ) [ v ( a , ω ) − v ( a ′ , ω ) ] + δ D − p 0 2 ε , 0 } .
which satisfies the first claim of Lemma 9. The receiver’s utility satisfies V ( π σ ~ , ρ o b ; μ R ) ≥ V ( π σ ^ , ρ o b ; μ R ) V(\pi_{\tilde{\sigma}},\rho_{\mathrm{ob}};\mu_R) \ge V(\pi_{\hat{\sigma}},\rho_{\mathrm{ob}};\mu_R) V ( π σ ~ , ρ ob ; μ R ) ≥ V ( π σ ^ , ρ ob ; μ R ) because the full revelation signaling scheme maximizes the receiver’s utility. For the designer’s utility, by the Lipschitz continuity with respect to π \pi π (Lemma 14 ), we have
U ( π σ ~ , ρ o b ; μ D ) ≥ U ( π σ ^ , ρ o b ; μ D ) − ∥ π f u l l − π σ ^ ∥ ≥ U ( π σ ^ , ρ o b ; μ D ) − θ . U(\pi_{\tilde{\sigma}},\rho_{\mathrm{ob}};\mu_D) \ge U(\pi_{\hat{\sigma}},\rho_{\mathrm{ob}};\mu_D)-\left\|\pi_{\mathrm{full}}-\pi_{\hat{\sigma}}\right\| \ge U(\pi_{\hat{\sigma}},\rho_{\mathrm{ob}};\mu_D)-\theta. U ( π σ ~ , ρ ob ; μ D ) ≥ U ( π σ ^ , ρ ob ; μ D ) − ∥ π full − π σ ^ ∥ ≥ U ( π σ ^ , ρ ob ; μ D ) − θ .
which satisfies the third claim of Lemma 9.
Case (2): ∥ π σ ^ − π f u l l ∥ > θ \left\|\pi_{\hat{\sigma}}-\pi_{\mathrm{full}}\right\|>\theta ∥ π σ ^ − π full ∥ > θ . In this case, we apply static robustification (Lemma 4) to π \pi π to obtain direct signaling scheme π σ ~ \pi_{\tilde{\sigma}} π σ ~ . According to the third claim of Lemma 4, ∥ π σ ~ − π σ ^ ∥ ≤ 4 δ p 0 2 \left\|\pi_{\tilde{\sigma}}-\pi_{\hat{\sigma}}\right\|\le\frac{4\delta}{p_0^2} ∥ π σ ~ − π σ ^ ∥ ≤ p 0 2 4 δ , so by the Lipschitz continuity of the receiver’s utility (Lemma 14 ),
V ( π σ ~ , ρ o b ; μ R ) ≥ V ( π σ ^ , ρ o b ; μ R ) − 4 δ p 0 2 . V(\pi_{\tilde{\sigma}},\rho_{\mathrm{ob}};\mu_R)\ge V(\pi_{\hat{\sigma}},\rho_{\mathrm{ob}};\mu_R)-\frac{4\delta}{p_0^2}. V ( π σ ~ , ρ ob ; μ R ) ≥ V ( π σ ^ , ρ ob ; μ R ) − p 0 2 4 δ .
Consider the convex combination of π σ ~ \pi_{\tilde{\sigma}} π σ ~ and π f u l l \pi_{\mathrm{full}} π full with weights 1 − α 1-\alpha 1 − α , α \alpha α , denoted by π σ ~ , α \pi_{\tilde{\sigma},\alpha} π σ ~ , α . By the coalescing lemma (Lemma 16), π ~ α \tilde{\pi}_{\alpha} π ~ α automatically satisfies the first claim of Lemma 9. By linearity, the receiver’s utility satisfies
V ( π σ ~ , α , ρ o b ; μ R ) − V ( π σ ^ , ρ o b ; μ R ) = ( 1 − α ) ( V ( π σ ~ , ρ o b ; μ R ) − V ( π σ ^ , ρ o b ; μ R ) ) + α ( V ( π f u l l , ρ o b ; μ R ) − V ( π σ ^ , ρ o b ; μ R ) ) ≥ − ( 1 − α ) 4 δ p 0 2 + α ⋅ p 0 G θ 2 by xref‘Claim 7‘ ≥ 0 \begin{aligned}
V(\pi_{\tilde{\sigma},\alpha},\rho_{\mathrm{ob}};\mu_R)-V(\pi_{\hat{\sigma}},\rho_{\mathrm{ob}};\mu_R)
&=(1-\alpha)\left(V(\pi_{\tilde{\sigma}},\rho_{\mathrm{ob}};\mu_R)-V(\pi_{\hat{\sigma}},\rho_{\mathrm{ob}};\mu_R)\right)\\
&\quad+\alpha\left(V(\pi_{\mathrm{full}},\rho_{\mathrm{ob}};\mu_R)-V(\pi_{\hat{\sigma}},\rho_{\mathrm{ob}};\mu_R)\right)\\
&\ge-(1-\alpha)\frac{4\delta}{p_0^2}+\alpha\cdot\frac{p_0G\theta}{2}\qquad\text{by {xref}`Claim 7`}\\
&\ge0
\end{aligned} V ( π σ ~ , α , ρ ob ; μ R ) − V ( π σ ^ , ρ ob ; μ R ) = ( 1 − α ) ( V ( π σ ~ , ρ ob ; μ R ) − V ( π σ ^ , ρ ob ; μ R ) ) + α ( V ( π full , ρ ob ; μ R ) − V ( π σ ^ , ρ ob ; μ R ) ) ≥ − ( 1 − α ) p 0 2 4 δ + α ⋅ 2 p 0 Gθ by xref ‘Claim 7‘ ≥ 0
where we choose α = 8 δ p 0 3 G θ = 8 δ p 0 3 G \alpha=\frac{8\delta}{p_0^3G\theta}=\sqrt{\frac{8\delta}{p_0^3G}} α = p 0 3 Gθ 8 δ = p 0 3 G 8 δ (α ≤ 1 \alpha\le1 α ≤ 1 given δ ≤ p 0 3 G 8 \delta\le\frac{p_0^3G}{8} δ ≤ 8 p 0 3 G ). So, π σ ~ , α \pi_{\tilde{\sigma},\alpha} π σ ~ , α weakly improves the receiver’s utility compared to π \pi π , satisfying the second claim of Lemma 9.
The designer’s utility under π ~ σ , α \tilde{\pi}_{\sigma,\alpha} π ~ σ , α is
U ( π ~ σ , α , ρ o b ; μ D ) = ( 1 − α ) U ( π ~ , ρ o b ; μ D ) + α U ( π f u l l , ρ o b ; μ D ) ≥ ( 1 − α ) ( U ( π ^ σ , ρ o b ; μ D ) − 3 δ p 0 ) + 0 by Lemma 4 ≥ U ( π ^ σ , ρ o b ; μ D ) − 3 δ p 0 − 8 δ p 0 3 G given α = 8 δ p 0 3 G ≥ U ( π ^ σ , ρ o b ; μ D ) − 3 δ 8 p 0 − 8 δ p 0 3 G as δ ≤ p 0 3 G 8 ⟹ δ p 0 ≤ 1 8 ≥ U ( π ^ σ , ρ o b ; μ D ) − 9 δ 8 p 0 3 G − 8 δ p 0 3 G p 0 , G ≤ 1 ≥ U ( π ^ σ , ρ o b ; μ D ) − 4 δ p 0 3 G , \begin{aligned}
U(\tilde{\pi}_{\sigma,\alpha},\rho_{\mathrm{ob}};\mu_D)
&= (1-\alpha)U(\tilde{\pi},\rho_{\mathrm{ob}};\mu_D)+\alpha U(\pi_{\mathrm{full}},\rho_{\mathrm{ob}};\mu_D)\\
&\ge(1-\alpha)\left(U(\hat{\pi}_{\sigma},\rho_{\mathrm{ob}};\mu_D)-\frac{3\delta}{p_0}\right)+0 && \text{by Lemma 4}\\
&\ge U(\hat{\pi}_{\sigma},\rho_{\mathrm{ob}};\mu_D)-\frac{3\delta}{p_0}-\sqrt{\frac{8\delta}{p_0^3G}} && \text{given }\alpha=\sqrt{\frac{8\delta}{p_0^3G}}\\
&\ge U(\hat{\pi}_{\sigma},\rho_{\mathrm{ob}};\mu_D)-3\sqrt{\frac{\delta}{8p_0}}-\sqrt{\frac{8\delta}{p_0^3G}} && \text{as }\delta\le\frac{p_0^3G}{8}\Longrightarrow\frac{\delta}{p_0}\le\frac{1}{8}\\
&\ge U(\hat{\pi}_{\sigma},\rho_{\mathrm{ob}};\mu_D)-\sqrt{\frac{9\delta}{8p_0^3G}}-\sqrt{\frac{8\delta}{p_0^3G}} && p_0,G\le1\\
&\ge U(\hat{\pi}_{\sigma},\rho_{\mathrm{ob}};\mu_D)-4\sqrt{\frac{\delta}{p_0^3G}},
\end{aligned} U ( π ~ σ , α , ρ ob ; μ D ) = ( 1 − α ) U ( π ~ , ρ ob ; μ D ) + α U ( π full , ρ ob ; μ D ) ≥ ( 1 − α ) ( U ( π ^ σ , ρ ob ; μ D ) − p 0 3 δ ) + 0 ≥ U ( π ^ σ , ρ ob ; μ D ) − p 0 3 δ − p 0 3 G 8 δ ≥ U ( π ^ σ , ρ ob ; μ D ) − 3 8 p 0 δ − p 0 3 G 8 δ ≥ U ( π ^ σ , ρ ob ; μ D ) − 8 p 0 3 G 9 δ − p 0 3 G 8 δ ≥ U ( π ^ σ , ρ ob ; μ D ) − 4 p 0 3 G δ , by Lemma 4 given α = p 0 3 G 8 δ as δ ≤ 8 p 0 3 G ⟹ p 0 δ ≤ 8 1 p 0 , G ≤ 1
which proves the lemma.
Claim 7. ∥ π ^ σ − π f u l l ∥ > θ ⟹ V ( π f u l l , ρ o b ; μ R ) − V ( π ^ σ , ρ o b ; μ R ) > p 0 G θ 2 \lVert\hat{\pi}_{\sigma}-\pi_{\mathrm{full}}\rVert>\theta\Longrightarrow V(\pi_{\mathrm{full}},\rho_{\mathrm{ob}};\mu_R)-V(\hat{\pi}_{\sigma},\rho_{\mathrm{ob}};\mu_R)>\frac{p_0G\theta}{2} ∥ π ^ σ − π full ∥ > θ ⟹ V ( π full , ρ ob ; μ R ) − V ( π ^ σ , ρ ob ; μ R ) > 2 p 0 Gθ
Proof. For any state ω ∈ Ω \omega\in\Omega ω ∈ Ω , define the difference
Δ ( ω ) : = v ( a ω ∗ , ω ) − ∑ a π ^ σ ( a ∣ ω ) v ( a , ω ) = ∑ a ≠ a ω ∗ π ^ σ ( a ∣ ω ) ( v ( a ω ∗ , ω ) − v ( a , ω ) ) ≥ G ( 1 − π ^ σ ( a ω ∗ ∣ ω ) ) = G 2 ∥ π ^ σ ( ⋅ ∣ ω ) − π f u l l ( ⋅ ∣ ω ) ∥ 1 . \begin{aligned}
\Delta(\omega)&:=v(a_{\omega}^{*},\omega)-\sum_a\hat{\pi}_{\sigma}(a\mid\omega)v(a,\omega)\\
&=\sum_{a\ne a_{\omega}^{*}}\hat{\pi}_{\sigma}(a\mid\omega)\bigl(v(a_{\omega}^{*},\omega)-v(a,\omega)\bigr)\\
&\ge G(1-\hat{\pi}_{\sigma}(a_{\omega}^{*}\mid\omega))
=\frac{G}{2}\lVert\hat{\pi}_{\sigma}(\cdot\mid\omega)-\pi_{\mathrm{full}}(\cdot\mid\omega)\rVert_1.
\end{aligned} Δ ( ω ) := v ( a ω ∗ , ω ) − a ∑ π ^ σ ( a ∣ ω ) v ( a , ω ) = a = a ω ∗ ∑ π ^ σ ( a ∣ ω ) ( v ( a ω ∗ , ω ) − v ( a , ω ) ) ≥ G ( 1 − π ^ σ ( a ω ∗ ∣ ω )) = 2 G ∥ π ^ σ ( ⋅ ∣ ω ) − π full ( ⋅ ∣ ω ) ∥ 1 . The condition ∥ π ^ σ − π f u l l ∥ > θ \lVert\hat{\pi}_{\sigma}-\pi_{\mathrm{full}}\rVert>\theta ∥ π ^ σ − π full ∥ > θ implies ∃ ω ˉ ∈ Ω \exists\bar{\omega}\in\Omega ∃ ω ˉ ∈ Ω with ∥ π ^ σ ( ⋅ ∣ ω ˉ ) − π f u l l ( ⋅ ∣ ω ˉ ) ∥ 1 > θ \lVert\hat{\pi}_{\sigma}(\cdot\mid\bar{\omega})-\pi_{\mathrm{full}}(\cdot\mid\bar{\omega})\rVert_1>\theta ∥ π ^ σ ( ⋅ ∣ ω ˉ ) − π full ( ⋅ ∣ ω ˉ ) ∥ 1 > θ , hence Δ ( ω ˉ ) ≥ G θ 2 \Delta(\bar{\omega})\ge\frac{G\theta}{2} Δ ( ω ˉ ) ≥ 2 Gθ . Therefore,
V ( π f u l l , ρ o b ; μ R ) − V ( π ^ σ , ρ o b ; μ R ) = ∑ ω μ R ( ω ) Δ ( ω ) ≥ μ R ( ω ˉ ) Δ ( ω ˉ ) ≥ p 0 G θ 2 . V(\pi_{\mathrm{full}},\rho_{\mathrm{ob}};\mu_R)-V(\hat{\pi}_{\sigma},\rho_{\mathrm{ob}};\mu_R)
=\sum_{\omega}\mu_R(\omega)\Delta(\omega)
\ge\mu_R(\bar{\omega})\Delta(\bar{\omega})
\ge\frac{p_0G\theta}{2}. V ( π full , ρ ob ; μ R ) − V ( π ^ σ , ρ ob ; μ R ) = ω ∑ μ R ( ω ) Δ ( ω ) ≥ μ R ( ω ˉ ) Δ ( ω ˉ ) ≥ 2 p 0 Gθ .
Proof of Theorem 2
Proof of Theorem 2. We convert σ ^ = ( π ^ σ ( 1 ) , … , π ^ σ ( T ) ) \hat{\sigma}=(\hat{\pi}_{\sigma}^{(1)},\ldots,\hat{\pi}_{\sigma}^{(T)}) σ ^ = ( π ^ σ ( 1 ) , … , π ^ σ ( T ) ) to σ ~ = ( π ~ σ ( 1 ) , … , π ~ σ ( T ) ) \tilde{\sigma}=(\tilde{\pi}_{\sigma}^{(1)},\ldots,\tilde{\pi}_{\sigma}^{(T)}) σ ~ = ( π ~ σ ( 1 ) , … , π ~ σ ( T ) ) by applying utility-improving robustification (Lemma 9) to the direct signaling schemes of σ \sigma σ in the backward order: namely, from π σ ( T ) \pi_{\sigma}^{(T)} π σ ( T ) to π σ ( 1 ) \pi_{\sigma}^{(1)} π σ ( 1 ) .
Specifically, for each period t ≤ T t\le T t ≤ T , having robustified π σ ( T ) , … , π σ ( t + 1 ) \pi_{\sigma}^{(T)},\ldots,\pi_{\sigma}^{(t+1)} π σ ( T ) , … , π σ ( t + 1 ) , we robustify π σ ( t ) \pi_{\sigma}^{(t)} π σ ( t ) so that ( π σ ( t ) , π σ ( t + 1 ) , … , π σ ( T ) ) (\pi_{\sigma}^{(t)},\pi_{\sigma}^{(t+1)},\ldots,\pi_{\sigma}^{(T)}) ( π σ ( t ) , π σ ( t + 1 ) , … , π σ ( T ) ) is dynamically persuasive with respect to μ R \mu_R μ R satisfying ∥ μ R − μ ^ ∥ 1 ≤ ε \lVert\mu_R-\hat{\mu}\rVert_1\le\varepsilon ∥ μ R − μ ^ ∥ 1 ≤ ε . In particular, we want the following to hold: for any action a a a recommended by π σ ~ ( t ) \pi_{\tilde{\sigma}}^{(t)} π σ ~ ( t ) and any a ′ ∈ A a' \in A a ′ ∈ A ,
γ t E ω ∼ μ R ( ⋅ ∣ a , π σ ~ ( t ) ) [ v ( a , ω ) − v ( a ′ , ω ) ] + ∑ t ′ = t + 1 T γ t ′ ( V ( π σ ~ ( t ′ ) , ρ o b ; μ R ) − V u n i n f o r m e d ( μ R ) ) ≥ 0. (48) \gamma^t \mathbb{E}_{\omega\sim\mu_R(\cdot\mid a,\pi_{\tilde{\sigma}}^{(t)})}\left[v(a,\omega)-v(a',\omega)\right]+\sum_{t'=t+1}^{T}\gamma^{t'}\left(V\left(\pi_{\tilde{\sigma}}^{(t')},\rho_{\mathrm{ob}};\mu_R\right)-V_{\mathrm{uninformed}}(\mu_R)\right)\ge0. \tag*{(48)} γ t E ω ∼ μ R ( ⋅ ∣ a , π σ ~ ( t ) ) [ v ( a , ω ) − v ( a ′ , ω ) ] + t ′ = t + 1 ∑ T γ t ′ ( V ( π σ ~ ( t ′ ) , ρ ob ; μ R ) − V uninformed ( μ R ) ) ≥ 0. (48)
Applying Lemma 9 to π σ ~ ( t ) \pi_{\tilde{\sigma}}^{(t)} π σ ~ ( t ) with parameter δ t > 0 \delta_t>0 δ t > 0 (to be chosen later), we obtain another direct signaling scheme π σ ~ ( t ) \pi_{\tilde{\sigma}}^{(t)} π σ ~ ( t ) that satisfies E ω ∼ μ R ( ⋅ ∣ a , π σ ~ ( t ) ) [ v ( a , ω ) − v ( a ′ , ω ) ] ≥ min { E ω ∼ μ ^ ( ⋅ ∣ a , π σ ~ ( t ) ) [ v ( a , ω ) − v ( a ′ , ω ) ] + δ t D − 2 ε p 0 , 0 } \mathbb{E}_{\omega\sim\mu_R(\cdot\mid a,\pi_{\tilde{\sigma}}^{(t)})}[v(a,\omega)-v(a',\omega)]\ge\min\{\mathbb{E}_{\omega\sim\hat{\mu}(\cdot\mid a,\pi_{\tilde{\sigma}}^{(t)})}[v(a,\omega)-v(a',\omega)]+\delta_tD-\frac{2\varepsilon}{p_0},0\} E ω ∼ μ R ( ⋅ ∣ a , π σ ~ ( t ) ) [ v ( a , ω ) − v ( a ′ , ω )] ≥ min { E ω ∼ μ ^ ( ⋅ ∣ a , π σ ~ ( t ) ) [ v ( a , ω ) − v ( a ′ , ω )] + δ t D − p 0 2 ε , 0 } . There are two cases:
Case (1): E ω ∼ μ R ( ⋅ ∣ a , π σ ~ ( t ) ) [ v ( a , ω ) − v ( a ′ , ω ) ] ≥ 0 \mathbb{E}_{\omega\sim\mu_R(\cdot\mid a,\pi_{\tilde{\sigma}}^{(t)})}[v(a,\omega)-v(a',\omega)]\ge0 E ω ∼ μ R ( ⋅ ∣ a , π σ ~ ( t ) ) [ v ( a , ω ) − v ( a ′ , ω )] ≥ 0 : In this case, by induction, we have
∑ t ′ = t + 1 T γ t ′ ( V ( π σ ~ ( t ′ ) , ρ o b ; μ R ) − V u n i n f o r m e d ( μ R ) ) ≥ 0 \sum_{t'=t+1}^{T}\gamma^{t'}\left(V\left(\pi_{\tilde{\sigma}}^{(t')},\rho_{\mathrm{ob}};\mu_R\right)-V_{\mathrm{uninformed}}(\mu_R)\right)\ge0 t ′ = t + 1 ∑ T γ t ′ ( V ( π σ ~ ( t ′ ) , ρ ob ; μ R ) − V uninformed ( μ R ) ) ≥ 0
because the robustified DDP strategy ( π σ ~ ( t ′ ) ) t ′ = t + 1 T \left(\pi_{\tilde{\sigma}}^{(t')}\right)_{t'=t+1}^{T} ( π σ ~ ( t ′ ) ) t ′ = t + 1 T is dynamically persuasive. So, we immediately obtain (47).
Case (2): E ω ∼ μ R ( ⋅ ∣ a , π σ ~ ( t ) ) [ v ( a , ω ) − v ( a ′ , ω ) ] < 0 \mathbb{E}_{\omega\sim\mu_R(\cdot\mid a,\pi_{\tilde{\sigma}}^{(t)})}[v(a,\omega)-v(a',\omega)]<0 E ω ∼ μ R ( ⋅ ∣ a , π σ ~ ( t ) ) [ v ( a , ω ) − v ( a ′ , ω )] < 0 : in this case, by the first claim of Lemma 9,
0 > E ω ∼ μ R ( ⋅ ∣ a , π σ ~ ( t ) ) [ v ( a , ω ) − v ( a ′ , ω ) ] ≥ E ω ∼ μ ^ ( ⋅ ∣ a , π σ ~ ( t ) ) [ v ( a , ω ) − v ( a ′ , ω ) ] + δ t D − 2 ε p 0 . (49) 0>\mathbb{E}_{\omega\sim\mu_R(\cdot\mid a,\pi_{\tilde{\sigma}}^{(t)})}[v(a,\omega)-v(a',\omega)]\ge\mathbb{E}_{\omega\sim\hat{\mu}(\cdot\mid a,\pi_{\tilde{\sigma}}^{(t)})}[v(a,\omega)-v(a',\omega)]+\delta_tD-\frac{2\varepsilon}{p_0}. \tag*{(49)} 0 > E ω ∼ μ R ( ⋅ ∣ a , π σ ~ ( t ) ) [ v ( a , ω ) − v ( a ′ , ω )] ≥ E ω ∼ μ ^ ( ⋅ ∣ a , π σ ~ ( t ) ) [ v ( a , ω ) − v ( a ′ , ω )] + δ t D − p 0 2 ε . (49)
We then consider the receiver’s continuation utility from period t + 1 t+1 t + 1 to T T T .
∑ t ′ = t + 1 T γ t ′ ( V ( π σ ~ ( t ′ ) , ρ o b ; μ R ) − V u n i n f o r m e d ( μ R ) ) ≥ ∑ t ′ = t + 1 T γ t ′ ( V ( π σ ~ ( t ′ ) , ρ o b ; μ R ) − V u n i n f o r m e d ( μ R ) ) by the second claim of Lemma 9 ≥ ∑ t ′ = t + 1 T γ t ′ ( V ( π σ ~ ( t ′ ) , ρ o b ; μ ^ ) − V u n i n f o r m e d ( μ ^ ) − 2 ∥ μ R − μ ^ ∥ 1 ) V is 1-Lipschitz (Lemma 14) ≥ ∑ t ′ = t + 1 T γ t ′ ( V ( π σ ~ ( t ′ ) , ρ o b ; μ ^ ) − V u n i n f o r m e d ( μ ^ ) ) − γ t + 1 1 − γ 2 ε . (50) \begin{aligned}
\sum_{t'=t+1}^{T}\gamma^{t'}\left(V\left(\pi_{\tilde{\sigma}}^{(t')},\rho_{\mathrm{ob}};\mu_R\right)-V_{\mathrm{uninformed}}(\mu_R)\right)
&\ge\sum_{t'=t+1}^{T}\gamma^{t'}\left(V\left(\pi_{\tilde{\sigma}}^{(t')},\rho_{\mathrm{ob}};\mu_R\right)-V_{\mathrm{uninformed}}(\mu_R)\right) && \text{by the second claim of Lemma 9}\\
&\ge\sum_{t'=t+1}^{T}\gamma^{t'}\left(V\left(\pi_{\tilde{\sigma}}^{(t')},\rho_{\mathrm{ob}};\hat{\mu}\right)-V_{\mathrm{uninformed}}(\hat{\mu})-2\lVert\mu_R-\hat{\mu}\rVert_1\right) && V\text{ is 1-Lipschitz (Lemma 14)}\\
&\ge\sum_{t'=t+1}^{T}\gamma^{t'}\left(V\left(\pi_{\tilde{\sigma}}^{(t')},\rho_{\mathrm{ob}};\hat{\mu}\right)-V_{\mathrm{uninformed}}(\hat{\mu})\right)-\frac{\gamma^{t+1}}{1-\gamma}2\varepsilon. \tag*{(50)}
\end{aligned} t ′ = t + 1 ∑ T γ t ′ ( V ( π σ ~ ( t ′ ) , ρ ob ; μ R ) − V uninformed ( μ R ) ) ≥ t ′ = t + 1 ∑ T γ t ′ ( V ( π σ ~ ( t ′ ) , ρ ob ; μ R ) − V uninformed ( μ R ) ) ≥ t ′ = t + 1 ∑ T γ t ′ ( V ( π σ ~ ( t ′ ) , ρ ob ; μ ^ ) − V uninformed ( μ ^ ) − 2 ∥ μ R − μ ^ ∥ 1 ) ≥ t ′ = t + 1 ∑ T γ t ′ ( V ( π σ ~ ( t ′ ) , ρ ob ; μ ^ ) − V uninformed ( μ ^ ) ) − 1 − γ γ t + 1 2 ε . by the second claim of Lemma 9 V is 1-Lipschitz (Lemma 14) (50)
Adding (49) and (50) ,
γ t E ω ∼ μ R ( ⋅ ∣ a , π σ ( t ) ) [ v ( a , ω ) − v ( a ′ , ω ) ] + ∑ t ′ = t + 1 T γ t ′ ( V ( π σ ′ ( t ′ ) , ρ o b ; μ R ) − V u n i n f o r m e d ( μ R ) ) ≥ γ t E ω ∼ μ ^ ( ⋅ ∣ a , π σ ^ ( t ) ) [ v ( a , ω ) − v ( a ′ , ω ) ] + ∑ t ′ = t + 1 T γ t ′ ( V ( π σ ^ ( t ′ ) , ρ o b ; μ ^ ) − V u n i n f o r m e d ( μ ^ ) ) \begin{aligned}
\gamma^t \mathbb{E}_{\omega\sim\mu_R(\cdot\mid a,\pi_\sigma^{(t)})}\left[v(a,\omega)-v(a',\omega)\right]+\sum_{t'=t+1}^{T}\gamma^{t'}\left(V(\pi_{\sigma'}^{(t')},\rho_{\mathrm{ob}};\mu_R)-V_{\mathrm{uninformed}}(\mu_R)\right) \\
&\geq\gamma^t \mathbb{E}_{\omega\sim\hat{\mu}(\cdot\mid a,\pi_{\hat{\sigma}}^{(t)})}\left[v(a,\omega)-v(a',\omega)\right]+\sum_{t'=t+1}^{T}\gamma^{t'}\left(V(\pi_{\hat{\sigma}}^{(t')},\rho_{\mathrm{ob}};\hat{\mu})-V_{\mathrm{uninformed}}(\hat{\mu})\right)
\end{aligned} γ t E ω ∼ μ R ( ⋅ ∣ a , π σ ( t ) ) [ v ( a , ω ) − v ( a ′ , ω ) ] + t ′ = t + 1 ∑ T γ t ′ ( V ( π σ ′ ( t ′ ) , ρ ob ; μ R ) − V uninformed ( μ R ) ) ≥ γ t E ω ∼ μ ^ ( ⋅ ∣ a , π σ ^ ( t ) ) [ v ( a , ω ) − v ( a ′ , ω ) ] + t ′ = t + 1 ∑ T γ t ′ ( V ( π σ ^ ( t ′ ) , ρ ob ; μ ^ ) − V uninformed ( μ ^ ) )
∑ t ′ = t + 1 T γ t E ω ∼ μ ^ ( ⋅ ∣ a , π σ ^ ( t ) ) [ v ( a , ω ) − v ( a ′ , ω ) ] + ∑ t ′ = t + 1 T γ t ′ ( V ( π σ ^ ( t ′ ) , ρ o b ; μ ^ ) − V u n i n f o r m e d ( μ ^ ) ) ⏟ ≥ 0 because ( π σ ^ ( t ) , … , π σ ^ ( T ) ) is dynamically persuasive for μ ^ + γ t δ t D − γ t 2 ε p 0 − γ t + 1 1 − γ 2 ε \underset{\geq0\text{ because }(\pi_{\hat{\sigma}}^{(t)},\ldots,\pi_{\hat{\sigma}}^{(T)})\text{ is dynamically persuasive for }\hat{\mu}}{\underbrace{\vphantom{\sum_{t'=t+1}^{T}}\phantom{\gamma^t \mathbb{E}_{\omega\sim\hat{\mu}(\cdot\mid a,\pi_{\hat{\sigma}}^{(t)})}\left[v(a,\omega)-v(a',\omega)\right]+\sum_{t'=t+1}^{T}\gamma^{t'}\left(V(\pi_{\hat{\sigma}}^{(t')},\rho_{\mathrm{ob}};\hat{\mu})-V_{\mathrm{uninformed}(\hat{\mu})}\right)}}}
+\gamma^t\delta_tD-\gamma^t\frac{2\varepsilon}{p_0}-\frac{\gamma^{t+1}}{1-\gamma}2\varepsilon ≥ 0 because ( π σ ^ ( t ) , … , π σ ^ ( T ) ) is dynamically persuasive for μ ^ t ′ = t + 1 ∑ T γ t E ω ∼ μ ^ ( ⋅ ∣ a , π σ ^ ( t ) ) [ v ( a , ω ) − v ( a ′ , ω ) ] + t ′ = t + 1 ∑ T γ t ′ ( V ( π σ ^ ( t ′ ) , ρ ob ; μ ^ ) − V uninformed ( μ ^ ) ) + γ t δ t D − γ t p 0 2 ε − 1 − γ γ t + 1 2 ε
where we choose δ t = ( 1 p 0 + γ 1 − γ ) 2 ε D \delta_t=\left(\frac{1}{p_0}+\frac{\gamma}{1-\gamma}\right)\frac{2\varepsilon}{D} δ t = ( p 0 1 + 1 − γ γ ) D 2 ε . Thus, ( π σ ~ ( t ) , π σ ~ ( t + 1 ) , … , π σ ~ ( T ) ) (\pi_{\tilde{\sigma}}^{(t)},\pi_{\tilde{\sigma}}^{(t+1)},\ldots,\pi_{\tilde{\sigma}}^{(T)}) ( π σ ~ ( t ) , π σ ~ ( t + 1 ) , … , π σ ~ ( T ) ) is dynamically persuasive for a receiver with prior μ R \mu_R μ R . Performing the robustification from t = T t=T t = T to t = 1 t=1 t = 1 , we therefore obtain a DDP strategy σ ~ = ( π σ ~ ( 1 ) , … , π σ ~ ( T ) ) \tilde{\sigma}=(\pi_{\tilde{\sigma}}^{(1)},\ldots,\pi_{\tilde{\sigma}}^{(T)}) σ ~ = ( π σ ~ ( 1 ) , … , π σ ~ ( T ) ) that is dynamically persuasive for a receiver with prior μ R \mu_R μ R .
The designer’s T T T -period total utility under the original DDP strategy σ = ( π σ ( 1 ) , … , π σ ( T ) ) \sigma=(\pi_\sigma^{(1)},\ldots,\pi_\sigma^{(T)}) σ = ( π σ ( 1 ) , … , π σ ( T ) ) is U T ( σ , ϕ o b ; μ D ) = ∑ t = 1 T U ( π σ ( t ) , ρ o b ; μ D ) U_T(\sigma,\phi_{\mathrm{ob}};\mu_D)=\sum_{t=1}^{T}U(\pi_\sigma^{(t)},\rho_{\mathrm{ob}};\mu_D) U T ( σ , ϕ ob ; μ D ) = ∑ t = 1 T U ( π σ ( t ) , ρ ob ; μ D ) . Under the robustified DDP strategy σ ~ = ( π σ ~ ( 1 ) , … , π σ ~ ( T ) ) \tilde{\sigma}=(\pi_{\tilde{\sigma}}^{(1)},\ldots,\pi_{\tilde{\sigma}}^{(T)}) σ ~ = ( π σ ~ ( 1 ) , … , π σ ~ ( T ) ) , the designer’s utility is
U T ( σ ~ , ϕ o b ; μ D ) = ∑ t = 1 T U ( π σ ~ ( t ) , ρ o b ; μ D ) ≥ ∑ t = 1 T ( U ( π σ ( t ) , ρ o b ; μ D ) − 4 δ t p 0 3 G ) \begin{aligned}
U_T(\tilde{\sigma},\phi_{\mathrm{ob}};\mu_D)=\sum_{t=1}^{T}U(\pi_{\tilde{\sigma}}^{(t)},\rho_{\mathrm{ob}};\mu_D) \\
&\geq\sum_{t=1}^{T}\left(U(\pi_\sigma^{(t)},\rho_{\mathrm{ob}};\mu_D)-4\sqrt{\frac{\delta_t}{p_0^3G}}\right)
\end{aligned} U T ( σ ~ , ϕ ob ; μ D ) = t = 1 ∑ T U ( π σ ~ ( t ) , ρ ob ; μ D ) ≥ t = 1 ∑ T ( U ( π σ ( t ) , ρ ob ; μ D ) − 4 p 0 3 G δ t )
by the third claim of Lemma 9
= U T ( σ , ϕ o b ; μ D ) − 4 T ( 1 p 0 + γ 1 − γ ) 2 ε p 0 3 G D , =U_T(\sigma,\phi_{\mathrm{ob}};\mu_D)-4T\sqrt{\left(\frac{1}{p_0}+\frac{\gamma}{1-\gamma}\right)\frac{2\varepsilon}{p_0^3GD}}, = U T ( σ , ϕ ob ; μ D ) − 4 T ( p 0 1 + 1 − γ γ ) p 0 3 G D 2 ε ,
which proves Theorem 2.
Proof of Lemma 10
Proof. Let σ T ∗ = ( π σ ∗ ( 1 ) , … , π σ ∗ ( T ) ) \sigma_T^*=(\pi_{\sigma^*}^{(1)},\ldots,\pi_{\sigma^*}^{(T)}) σ T ∗ = ( π σ ∗ ( 1 ) , … , π σ ∗ ( T ) ) be an optimal history-independent DDP strategy for the T T T -period game, which returns U T σ ∗ ( μ D , μ ^ ) U_T^{\sigma^*}(\mu_D,\hat{\mu}) U T σ ∗ ( μ D , μ ^ ) .
Let σ t a i l = ( π σ ∗ ( T − T 0 + 1 ) , … , π σ ∗ ( T ) ) \sigma^{\mathrm{tail}}=(\pi_{\sigma^*}^{(T-T_0+1)},\ldots,\pi_{\sigma^*}^{(T)}) σ tail = ( π σ ∗ ( T − T 0 + 1 ) , … , π σ ∗ ( T ) ) be the tail strategy for σ T ∗ \sigma_T^* σ T ∗ . Because the DDP strategy ensures the receiver’s obedience through the continuation incentives that depend only on the future, truncating the first T 0 T_0 T 0 periods does not affect incentives from period T 0 + 1 T_0+1 T 0 + 1 onward. Thus σ t a i l \sigma^{\mathrm{tail}} σ tail is a feasible ( T − T 0 ) (T-T_0) ( T − T 0 ) -period DDP strategy for the same prior.
On the obedient path, the designer’s per-period payoff is U ( π ( t ) , ρ o b ; μ D ) ∈ [ 0 , 1 ] U(\pi^{(t)},\rho_{\mathrm{ob}};\mu_D)\in[0,1] U ( π ( t ) , ρ ob ; μ D ) ∈ [ 0 , 1 ] . Therefore, the first T 0 T_0 T 0 periods contribute at most T 0 T_0 T 0 to the total:
U T ∗ ∗ ( μ D , μ ^ ) = ∑ t = 1 T 0 U ( π σ ∗ ( t ) , ρ o b ; μ D ) + ∑ t = T 0 + 1 T U ( π σ ∗ ( t ) , ρ o b ; μ D ) ≤ T 0 + U ( σ t a i l , ρ o b ; μ D ) U_T^{**}(\mu_D,\hat{\mu})=\sum_{t=1}^{T_0}U(\pi_{\sigma_*}^{(t)},\rho_{\mathrm{ob}};\mu_D)+\sum_{t=T_0+1}^{T}U(\pi_{\sigma_*}^{(t)},\rho_{\mathrm{ob}};\mu_D)\leq T_0+U(\sigma^{\mathrm{tail}},\rho_{\mathrm{ob}};\mu_D) U T ∗∗ ( μ D , μ ^ ) = t = 1 ∑ T 0 U ( π σ ∗ ( t ) , ρ ob ; μ D ) + t = T 0 + 1 ∑ T U ( π σ ∗ ( t ) , ρ ob ; μ D ) ≤ T 0 + U ( σ tail , ρ ob ; μ D )
Maximizing over all feasible ( T − T 0 ) (T-T_0) ( T − T 0 ) -period strategies gives
U T − T 0 ∗ ∗ ( μ D , μ ^ ) ≥ U ( σ t a i l , ρ o b ; μ D ) ≥ U T ∗ ∗ ( μ D , μ ^ ) − T 0 . U_{T-T_0}^{**}(\mu_D,\hat{\mu})\geq U(\sigma^{\mathrm{tail}},\rho_{\mathrm{ob}};\mu_D)\geq U_T^{**}(\mu_D,\hat{\mu})-T_0. U T − T 0 ∗∗ ( μ D , μ ^ ) ≥ U ( σ tail , ρ ob ; μ D ) ≥ U T ∗∗ ( μ D , μ ^ ) − T 0 .
Omitted Proofs in Section 6
Proof of Lemma 11
Given an incentive compatible and individually rational menu M = { π μ } \mathcal{M}=\{\pi_\mu\} M = { π μ } , the receiver will report his prior μ R \mu_R μ R truthfully. Consider a weighted welfare of the game, which is the sum of the information designer’s utility and ( 1 + μ ~ R ) (1+\tilde{\mu}_R) ( 1 + μ ~ R ) times the receiver’s utility:
U s p ( M , μ R ) + ( 1 + μ ~ R ) V s p ( μ R , μ R ) = ( 1 − p 0 ) π μ R ( a 1 ∣ ω 0 ) + p 0 π μ R ( a 1 ∣ ω 1 ) + μ ~ R π μ R ( a 1 ∣ ω 1 ) − π ( a 1 ∣ ω 0 ) = ( μ ~ R + p 0 ) π μ R ( a 1 ∣ ω 1 ) − p 0 π μ R ( a 1 ∣ ω 0 ) . (51) U^{\mathrm{sp}}(\mathcal{M},\mu_R)+(1+\tilde{\mu}_R)V^{\mathrm{sp}}(\mu_R,\mu_R)
=(1-p_0)\pi_{\mu_R}(a_1\mid\omega_0)+p_0\pi_{\mu_R}(a_1\mid\omega_1)+\tilde{\mu}_R\pi_{\mu_R}(a_1\mid\omega_1)-\pi(a_1\mid\omega_0)
=(\tilde{\mu}_R+p_0)\pi_{\mu_R}(a_1\mid\omega_1)-p_0\pi_{\mu_R}(a_1\mid\omega_0). \tag*{(51)} U sp ( M , μ R ) + ( 1 + μ ~ R ) V sp ( μ R , μ R ) = ( 1 − p 0 ) π μ R ( a 1 ∣ ω 0 ) + p 0 π μ R ( a 1 ∣ ω 1 ) + μ ~ R π μ R ( a 1 ∣ ω 1 ) − π ( a 1 ∣ ω 0 ) = ( μ ~ R + p 0 ) π μ R ( a 1 ∣ ω 1 ) − p 0 π μ R ( a 1 ∣ ω 0 ) . (51)
Replacing μ R \mu_R μ R by μ \mu μ (and replacing μ ~ R \tilde{\mu}_R μ ~ R by μ ~ \tilde{\mu} μ ~ ) in (51) and noting that V s p ( μ , μ ) ≥ V u n i n f o r m e d ≥ 0 V^{\mathrm{sp}}(\mu,\mu)\geq V^{\mathrm{uninformed}}\geq0 V sp ( μ , μ ) ≥ V uninformed ≥ 0 by individual rationality, we have
U s p ( M , μ ) ≤ ( μ ~ + p 0 ) π μ ( a 1 ∣ ω 1 ) − p 0 π μ ( a 1 ∣ ω 0 ) . (52) U^{\mathrm{sp}}(\mathcal{M},\mu)\leq(\tilde{\mu}+p_0)\pi_\mu(a_1\mid\omega_1)-p_0\pi_\mu(a_1\mid\omega_0). \tag*{(52)} U sp ( M , μ ) ≤ ( μ ~ + p 0 ) π μ ( a 1 ∣ ω 1 ) − p 0 π μ ( a 1 ∣ ω 0 ) . (52)
Proof of Lemma 12
The information designer’s single-period regret
Reg s p ( M , μ R ) = ( 1 − p 0 ) μ ~ R + p 0 − U s p ( M , μ R ) ≥ π μ R ( a 1 ∣ ω 1 ) ⋅ ( ( 1 − p 0 ) μ ~ R + p 0 ) − U s p ( M , μ R ) = ( μ ~ R + p 0 ) π μ R ( a 1 ∣ ω 1 ) − U s p ( M , μ R ) − p 0 μ ~ R π μ R ( a 1 ∣ ω 1 ) = by ( 50 ) ( 1 + μ ~ R ) V s p ( μ R , μ R ) + p 0 π μ R ( a 1 ∣ ω 0 ) − p 0 μ ~ R π μ R ( a 1 ∣ ω 1 ) ≥ by IC ( 1 + μ ~ R ) V s p ( μ R , μ ) + p 0 π μ R ( a 1 ∣ ω 0 ) − p 0 μ ~ R π μ R ( a 1 ∣ ω 1 ) = ( 1 + μ ~ R ) 1 1 + μ ~ R ( μ R π μ ( a 1 ∣ ω 1 ) − π μ ( a 1 ∣ ω 0 ) ) + p 0 π μ R ( a 1 ∣ ω 0 ) − p 0 μ ~ R π μ R ( a 1 ∣ ω 1 ) ≥ by ( 51 ) ( μ ~ R μ ~ + p 0 ( U s p ( M , μ ) + p 0 π μ ( a 1 ∣ ω 0 ) ) − π μ ( a 1 ∣ ω 0 ) ) + p 0 π μ R ( a 1 ∣ ω 0 ) − p 0 μ ~ R π μ R ( a 1 ∣ ω 1 ) = by ( 30 ) ( μ ~ R μ ~ + p 0 ( U s p ( M , μ ) + p 0 π μ ( a 1 ∣ ω 0 ) ) − U s p ( M , μ ) − p 0 π μ ( a 1 ∣ ω 0 ) + p 0 π μ ( a 1 ∣ ω 1 ) ) + p 0 π μ R ( a 1 ∣ ω 0 ) − p 0 μ ~ R π μ R ( a 1 ∣ ω 1 ) = μ ~ R − μ ~ − p 0 μ ~ + p 0 U s p ( M , μ ) + p 0 ( μ ~ R − μ ~ − p 0 μ ~ + p 0 π μ ( a 1 ∣ ω 0 ) + π μ ( a 1 ∣ ω 1 ) + π μ R ( a 1 ∣ ω 0 ) − μ ~ R π μ R ( a 1 ∣ ω 1 ) ) ≥ μ ~ R − μ ~ − p 0 μ ~ + p 0 U s p ( M , μ ) + p 0 ( μ ~ R − μ ~ − p 0 μ ~ + p 0 π μ ( a 1 ∣ ω 0 ) − μ ~ R ) . \begin{aligned}
\operatorname{Reg}^{\mathrm{sp}}(\mathcal{M},\mu_R)
&=(1-p_0)\tilde{\mu}_R+p_0-U^{\mathrm{sp}}(\mathcal{M},\mu_R)\\
&\ge\pi_{\mu_R}(a_1\mid\omega_1)\cdot\left((1-p_0)\tilde{\mu}_R+p_0\right)-U^{\mathrm{sp}}(\mathcal{M},\mu_R)\\
&=(\tilde{\mu}_R+p_0)\pi_{\mu_R}(a_1\mid\omega_1)-U^{\mathrm{sp}}(\mathcal{M},\mu_R)-p_0\tilde{\mu}_R\pi_{\mu_R}(a_1\mid\omega_1)\\
&\mathrel{\stackrel{\text{by }(50)}{=}}(1+\tilde{\mu}_R)V^{\mathrm{sp}}(\mu_R,\mu_R)+p_0\pi_{\mu_R}(a_1\mid\omega_0)-p_0\tilde{\mu}_R\pi_{\mu_R}(a_1\mid\omega_1)\\
&\mathrel{\stackrel{\text{by IC}}{\ge}}(1+\tilde{\mu}_R)V^{\mathrm{sp}}(\mu_R,\mu)+p_0\pi_{\mu_R}(a_1\mid\omega_0)-p_0\tilde{\mu}_R\pi_{\mu_R}(a_1\mid\omega_1)\\
&=(1+\tilde{\mu}_R)\frac{1}{1+\tilde{\mu}_R}\left(\mu_R\pi_\mu(a_1\mid\omega_1)-\pi_\mu(a_1\mid\omega_0)\right)+p_0\pi_{\mu_R}(a_1\mid\omega_0)-p_0\tilde{\mu}_R\pi_{\mu_R}(a_1\mid\omega_1)\\
&\mathrel{\stackrel{\text{by }(51)}{\ge}}\left(\frac{\tilde{\mu}_R}{\tilde{\mu}+p_0}\left(U^{\mathrm{sp}}(\mathcal{M},\mu)+p_0\pi_\mu(a_1\mid\omega_0)\right)-\pi_\mu(a_1\mid\omega_0)\right)+p_0\pi_{\mu_R}(a_1\mid\omega_0)-p_0\tilde{\mu}_R\pi_{\mu_R}(a_1\mid\omega_1)\\
&\mathrel{\stackrel{\text{by }(30)}{=}}\left(\frac{\tilde{\mu}_R}{\tilde{\mu}+p_0}\left(U^{\mathrm{sp}}(\mathcal{M},\mu)+p_0\pi_\mu(a_1\mid\omega_0)\right)-U^{\mathrm{sp}}(\mathcal{M},\mu)-p_0\pi_\mu(a_1\mid\omega_0)+p_0\pi_\mu(a_1\mid\omega_1)\right)\\
&\qquad{}+p_0\pi_{\mu_R}(a_1\mid\omega_0)-p_0\tilde{\mu}_R\pi_{\mu_R}(a_1\mid\omega_1)\\
&=\frac{\tilde{\mu}_R-\tilde{\mu}-p_0}{\tilde{\mu}+p_0}U^{\mathrm{sp}}(\mathcal{M},\mu)\\
&\qquad{}+p_0\left(\frac{\tilde{\mu}_R-\tilde{\mu}-p_0}{\tilde{\mu}+p_0}\pi_\mu(a_1\mid\omega_0)+\pi_\mu(a_1\mid\omega_1)+\pi_{\mu_R}(a_1\mid\omega_0)-\tilde{\mu}_R\pi_{\mu_R}(a_1\mid\omega_1)\right)\\
&\ge\frac{\tilde{\mu}_R-\tilde{\mu}-p_0}{\tilde{\mu}+p_0}U^{\mathrm{sp}}(\mathcal{M},\mu)+p_0\left(\frac{\tilde{\mu}_R-\tilde{\mu}-p_0}{\tilde{\mu}+p_0}\pi_\mu(a_1\mid\omega_0)-\tilde{\mu}_R\right).
\end{aligned} Reg sp ( M , μ R ) = ( 1 − p 0 ) μ ~ R + p 0 − U sp ( M , μ R ) ≥ π μ R ( a 1 ∣ ω 1 ) ⋅ ( ( 1 − p 0 ) μ ~ R + p 0 ) − U sp ( M , μ R ) = ( μ ~ R + p 0 ) π μ R ( a 1 ∣ ω 1 ) − U sp ( M , μ R ) − p 0 μ ~ R π μ R ( a 1 ∣ ω 1 ) = by ( 50 ) ( 1 + μ ~ R ) V sp ( μ R , μ R ) + p 0 π μ R ( a 1 ∣ ω 0 ) − p 0 μ ~ R π μ R ( a 1 ∣ ω 1 ) ≥ by IC ( 1 + μ ~ R ) V sp ( μ R , μ ) + p 0 π μ R ( a 1 ∣ ω 0 ) − p 0 μ ~ R π μ R ( a 1 ∣ ω 1 ) = ( 1 + μ ~ R ) 1 + μ ~ R 1 ( μ R π μ ( a 1 ∣ ω 1 ) − π μ ( a 1 ∣ ω 0 ) ) + p 0 π μ R ( a 1 ∣ ω 0 ) − p 0 μ ~ R π μ R ( a 1 ∣ ω 1 ) ≥ by ( 51 ) ( μ ~ + p 0 μ ~ R ( U sp ( M , μ ) + p 0 π μ ( a 1 ∣ ω 0 ) ) − π μ ( a 1 ∣ ω 0 ) ) + p 0 π μ R ( a 1 ∣ ω 0 ) − p 0 μ ~ R π μ R ( a 1 ∣ ω 1 ) = by ( 30 ) ( μ ~ + p 0 μ ~ R ( U sp ( M , μ ) + p 0 π μ ( a 1 ∣ ω 0 ) ) − U sp ( M , μ ) − p 0 π μ ( a 1 ∣ ω 0 ) + p 0 π μ ( a 1 ∣ ω 1 ) ) + p 0 π μ R ( a 1 ∣ ω 0 ) − p 0 μ ~ R π μ R ( a 1 ∣ ω 1 ) = μ ~ + p 0 μ ~ R − μ ~ − p 0 U sp ( M , μ ) + p 0 ( μ ~ + p 0 μ ~ R − μ ~ − p 0 π μ ( a 1 ∣ ω 0 ) + π μ ( a 1 ∣ ω 1 ) + π μ R ( a 1 ∣ ω 0 ) − μ ~ R π μ R ( a 1 ∣ ω 1 ) ) ≥ μ ~ + p 0 μ ~ R − μ ~ − p 0 U sp ( M , μ ) + p 0 ( μ ~ + p 0 μ ~ R − μ ~ − p 0 π μ ( a 1 ∣ ω 0 ) − μ ~ R ) .
Let μ ~ R = 1 \tilde{\mu}_R=1 μ ~ R = 1 and μ ~ = 1 2 − p 0 \tilde{\mu}=\frac{1}{2}-p_0 μ ~ = 2 1 − p 0 . If Reg s p ( M , μ R ) ≥ 1 4 \operatorname{Reg}^{\mathrm{sp}}(\mathcal{M},\mu_R)\ge\frac{1}{4} Reg sp ( M , μ R ) ≥ 4 1 , then the lemma is proved. If Reg s p ( M , μ R ) < 1 4 \operatorname{Reg}^{\mathrm{sp}}(\mathcal{M},\mu_R)<\frac{1}{4} Reg sp ( M , μ R ) < 4 1 , then we have
1 4 > Reg s p ( M , μ R ) ≥ U s p ( M , μ ) + p 0 ( π μ ( a 1 ∣ ω 0 ) − 1 ) ≥ U s p ( M , μ ) − p 0 . \frac{1}{4}>\operatorname{Reg}^{\mathrm{sp}}(\mathcal{M},\mu_R)\ge U^{\mathrm{sp}}(\mathcal{M},\mu)+p_0\left(\pi_\mu(a_1\mid\omega_0)-1\right)\ge U^{\mathrm{sp}}(\mathcal{M},\mu)-p_0. 4 1 > Reg sp ( M , μ R ) ≥ U sp ( M , μ ) + p 0 ( π μ ( a 1 ∣ ω 0 ) − 1 ) ≥ U sp ( M , μ ) − p 0 .
which implies that the information designer’s regret under receiver prior μ \mu μ is
Reg s p ( M , μ ) = ( 1 − p 0 ) μ ~ + p 0 − U s p ( M , μ ) ≥ ( 1 − p 0 ) ( 1 2 − p 0 ) + p 0 − 1 4 − p 0 ≥ 1 4 − 3 2 p 0 ≥ 1 16 \begin{aligned}
\operatorname{Reg}^{\mathrm{sp}}(\mathcal{M},\mu)
&=(1-p_0)\tilde{\mu}+p_0-U^{\mathrm{sp}}(\mathcal{M},\mu)\\
&\ge(1-p_0)\left(\frac{1}{2}-p_0\right)+p_0-\frac{1}{4}-p_0\\
&\ge\frac{1}{4}-\frac{3}{2}p_0\ge\frac{1}{16}
\end{aligned} Reg sp ( M , μ ) = ( 1 − p 0 ) μ ~ + p 0 − U sp ( M , μ ) ≥ ( 1 − p 0 ) ( 2 1 − p 0 ) + p 0 − 4 1 − p 0 ≥ 4 1 − 2 3 p 0 ≥ 16 1
given p 0 = 1 8 p_0=\frac{1}{8} p 0 = 8 1 . Using μ ( ω 0 ) = 1 1 + μ ~ \mu(\omega_0)=\frac{1}{1+\tilde{\mu}} μ ( ω 0 ) = 1 + μ ~ 1 and μ ( ω 1 ) = μ ~ 1 + μ ~ \mu(\omega_1)=\frac{\tilde{\mu}}{1+\tilde{\mu}} μ ( ω 1 ) = 1 + μ ~ μ ~ , by direct calculation, one can verify that the condition μ ( ω ) ≥ p 0 \mu(\omega)\ge p_0 μ ( ω ) ≥ p 0 , ∀ ω ∈ Ω \forall\omega\in\Omega ∀ ω ∈ Ω , is satisfied. So, the lemma holds with μ ~ \tilde{\mu} μ ~ .
Proof of Lemma 12
We summarize the information design instance I \mathcal{I} I here:
I = { designer’s utility: u ( a 1 , ω ) = 1 , u ( a 0 , ω ) = 0 , ∀ ω ∈ Ω = { ω 0 , ω 1 } ; receiver’s utility: v ( a 0 , ω 0 ) = 0 , v ( a 0 , ω 1 ) = 0 , v ( a 1 , ω 0 ) = − 1 , v ( a 1 , ω 1 ) = 1 ; designer’s prior: μ D ( ω 0 ) = 1 − p 0 , μ D ( ω 1 ) = p 0 , 0 < p 0 < 1 / 2 ; receiver’s prior: μ R ( ω 0 ) = 1 1 + μ ~ R , μ R ( ω 1 ) = μ ~ R 1 + μ ~ R , 0 ≤ μ R ≤ 1. \mathcal{I}=\left\{
\begin{aligned}
&\text{designer’s utility:}\quad u(a_1,\omega)=1,\quad u(a_0,\omega)=0,\quad\forall\omega\in\Omega=\{\omega_0,\omega_1\};\\
&\text{receiver’s utility:}\quad v(a_0,\omega_0)=0,\quad v(a_0,\omega_1)=0,\quad v(a_1,\omega_0)=-1,\quad v(a_1,\omega_1)=1;\\
&\text{designer’s prior:}\quad\mu_D(\omega_0)=1-p_0,\quad\mu_D(\omega_1)=p_0,\quad0<p_0<1/2;\\
&\text{receiver’s prior:}\quad\mu_R(\omega_0)=\frac{1}{1+\widetilde{\mu}_R},\quad\mu_R(\omega_1)=\frac{\widetilde{\mu}_R}{1+\widetilde{\mu}_R},\quad0\leq\mu_R\leq1.
\end{aligned}
\right. I = ⎩ ⎨ ⎧ designer’s utility: u ( a 1 , ω ) = 1 , u ( a 0 , ω ) = 0 , ∀ ω ∈ Ω = { ω 0 , ω 1 } ; receiver’s utility: v ( a 0 , ω 0 ) = 0 , v ( a 0 , ω 1 ) = 0 , v ( a 1 , ω 0 ) = − 1 , v ( a 1 , ω 1 ) = 1 ; designer’s prior: μ D ( ω 0 ) = 1 − p 0 , μ D ( ω 1 ) = p 0 , 0 < p 0 < 1/2 ; receiver’s prior: μ R ( ω 0 ) = 1 + μ R 1 , μ R ( ω 1 ) = 1 + μ R μ R , 0 ≤ μ R ≤ 1.
We present some useful facts before proving Lemma 12. Because μ R ( ω 1 ) ≤ μ R ( ω 0 ) \mu_R(\omega_1)\leq\mu_R(\omega_0) μ R ( ω 1 ) ≤ μ R ( ω 0 ) , the receiver’s ex-ante utility when receiving no information is
V u n i n f o r m e d ( μ R ) = max a ∈ A ∑ ω ∈ Ω μ R ( ω ) v ( a , ω ) = max { 0 ⏟ a 0 ’s utility , μ R ( ω 1 ) − μ R ( ω 0 ) ⏟ a 1 ’s utility } = 0. (53) V_{\mathrm{uninformed}}(\mu_R)=\max_{a\in A}\sum_{\omega\in\Omega}\mu_R(\omega)v(a,\omega)=\max\left\{\underbrace{0}_{a_0\text{'s utility}},\underbrace{\mu_R(\omega_1)-\mu_R(\omega_0)}_{a_1\text{'s utility}}\right\}=0. \tag*{(53)} V uninformed ( μ R ) = a ∈ A max ω ∈ Ω ∑ μ R ( ω ) v ( a , ω ) = max ⎩ ⎨ ⎧ a 0 ’s utility 0 , a 1 ’s utility μ R ( ω 1 ) − μ R ( ω 0 ) ⎭ ⎬ ⎫ = 0. (53)
When the designer uses a direct signaling scheme π : Ω → Δ ( A ) \pi:\Omega\to\Delta(A) π : Ω → Δ ( A ) , the ex-ante utility of an obedient receiver is
V ( π , ρ o b ; μ R ) = ∑ ω ∈ Ω μ R ( ω ) ∑ a ∈ A π ( a ∣ ω ) v ( a , ω ) = − μ R ( ω 0 ) π ( a 1 ∣ ω 0 ) + μ R ( ω 1 ) π ( a 1 ∣ ω 1 ) , (54) V(\pi,\rho_{\mathrm{ob}};\mu_R)=\sum_{\omega\in\Omega}\mu_R(\omega)\sum_{a\in A}\pi(a\mid\omega)v(a,\omega)=-\mu_R(\omega_0)\pi(a_1\mid\omega_0)+\mu_R(\omega_1)\pi(a_1\mid\omega_1), \tag*{(54)} V ( π , ρ ob ; μ R ) = ω ∈ Ω ∑ μ R ( ω ) a ∈ A ∑ π ( a ∣ ω ) v ( a , ω ) = − μ R ( ω 0 ) π ( a 1 ∣ ω 0 ) + μ R ( ω 1 ) π ( a 1 ∣ ω 1 ) , (54)
and the ex-ante utility of the designer is
U ( π , ρ o b ; μ D ) = ∑ ω ∈ Ω μ R ( ω ) ∑ a ∈ A π ( a ∣ ω ) u ( a , ω ) = ( 1 − p 0 ) π ( a 1 ∣ ω 0 ) + p 0 π ( a 1 ∣ ω 1 ) . (55) U(\pi,\rho_{\mathrm{ob}};\mu_D)=\sum_{\omega\in\Omega}\mu_R(\omega)\sum_{a\in A}\pi(a\mid\omega)u(a,\omega)=(1-p_0)\pi(a_1\mid\omega_0)+p_0\pi(a_1\mid\omega_1). \tag*{(55)} U ( π , ρ ob ; μ D ) = ω ∈ Ω ∑ μ R ( ω ) a ∈ A ∑ π ( a ∣ ω ) u ( a , ω ) = ( 1 − p 0 ) π ( a 1 ∣ ω 0 ) + p 0 π ( a 1 ∣ ω 1 ) . (55)
The persuasiveness constraints for a direct signaling scheme π \pi π are:
E ω ∼ μ R ( ⋅ ∣ a 0 , π ) [ v ( a 0 , ω ) − v ( a 1 , ω ) ] = μ R ( ω 0 ) π ( a 0 ∣ ω 0 ) − μ R ( ω 1 ) π ( a 0 ∣ ω 1 ) μ R ( ω 0 ) π ( a 0 ∣ ω 0 ) + μ R ( ω 1 ) π ( a 0 ∣ ω 1 ) ≥ 0 ; (56) \mathbb{E}_{\omega\sim\mu_R(\cdot\mid a_0,\pi)}[v(a_0,\omega)-v(a_1,\omega)]=\frac{\mu_R(\omega_0)\pi(a_0\mid\omega_0)-\mu_R(\omega_1)\pi(a_0\mid\omega_1)}{\mu_R(\omega_0)\pi(a_0\mid\omega_0)+\mu_R(\omega_1)\pi(a_0\mid\omega_1)}\geq0; \tag*{(56)} E ω ∼ μ R ( ⋅ ∣ a 0 , π ) [ v ( a 0 , ω ) − v ( a 1 , ω )] = μ R ( ω 0 ) π ( a 0 ∣ ω 0 ) + μ R ( ω 1 ) π ( a 0 ∣ ω 1 ) μ R ( ω 0 ) π ( a 0 ∣ ω 0 ) − μ R ( ω 1 ) π ( a 0 ∣ ω 1 ) ≥ 0 ; (56)
E ω ∼ μ R ( ⋅ ∣ a 1 , π ) [ v ( a 1 , ω ) − v ( a 0 , ω ) ] = − μ R ( ω 0 ) π ( a 1 ∣ ω 0 ) + μ R ( ω 1 ) π ( a 1 ∣ ω 1 ) μ R ( ω 0 ) π ( a 1 ∣ ω 0 ) + μ R ( ω 1 ) π ( a 1 ∣ ω 1 ) ≥ 0 ; (57) \mathbb{E}_{\omega\sim\mu_R(\cdot\mid a_1,\pi)}[v(a_1,\omega)-v(a_0,\omega)]=\frac{-\mu_R(\omega_0)\pi(a_1\mid\omega_0)+\mu_R(\omega_1)\pi(a_1\mid\omega_1)}{\mu_R(\omega_0)\pi(a_1\mid\omega_0)+\mu_R(\omega_1)\pi(a_1\mid\omega_1)}\geq0; \tag*{(57)} E ω ∼ μ R ( ⋅ ∣ a 1 , π ) [ v ( a 1 , ω ) − v ( a 0 , ω )] = μ R ( ω 0 ) π ( a 1 ∣ ω 0 ) + μ R ( ω 1 ) π ( a 1 ∣ ω 1 ) − μ R ( ω 0 ) π ( a 1 ∣ ω 0 ) + μ R ( ω 1 ) π ( a 1 ∣ ω 1 ) ≥ 0 ; (57)
The Bayesian persuasion benchmark U B P ∗ ( I ) U_{\mathrm{BP}}^{*}(\mathcal{I}) U BP ∗ ( I ) is the designer’s optimal utility from a persuasive direct signaling scheme, when knowing the receiver’s prior and not using menu:
U B P ∗ ( I ) = max π : Ω → Δ ( A ) U ( π , ρ o b ; μ D ) U_{\mathrm{BP}}^{*}(\mathcal{I})=\max_{\pi:\Omega\to\Delta(A)} U(\pi,\rho_{\mathrm{ob}};\mu_D) U BP ∗ ( I ) = π : Ω → Δ ( A ) max U ( π , ρ ob ; μ D )
s.t. ( for a 0 ) μ R ( ω 0 ) π ( a 0 ∣ ω 0 ) − μ R ( ω 1 ) π ( a 0 ∣ ω 1 ) ≥ 0 ; ( for a 1 ) − μ R ( ω 0 ) π ( a 1 ∣ ω 0 ) + μ R ( ω 1 ) π ( a 1 ∣ ω 1 ) ≥ 0. \begin{aligned}
\text{s.t.}\quad&( \text{for }a_0)\quad\mu_R(\omega_0)\pi(a_0\mid\omega_0)-\mu_R(\omega_1)\pi(a_0\mid\omega_1)\ge0;\\
&( \text{for }a_1)\quad-\mu_R(\omega_0)\pi(a_1\mid\omega_0)+\mu_R(\omega_1)\pi(a_1\mid\omega_1)\ge0.
\end{aligned} s.t. ( for a 0 ) μ R ( ω 0 ) π ( a 0 ∣ ω 0 ) − μ R ( ω 1 ) π ( a 0 ∣ ω 1 ) ≥ 0 ; ( for a 1 ) − μ R ( ω 0 ) π ( a 1 ∣ ω 0 ) + μ R ( ω 1 ) π ( a 1 ∣ ω 1 ) ≥ 0.
Claim 8. U B P ∗ ( I ) = ( 1 − p 0 ) μ ~ R + p 0 = U s p ∗ ( μ R ) U_{\mathrm{BP}}^{*}(\mathcal{I})=(1-p_0)\widetilde{\mu}_R+p_0=U_{\mathrm{sp}}^{*}(\mu_R) U BP ∗ ( I ) = ( 1 − p 0 ) μ R + p 0 = U sp ∗ ( μ R ) .
Proof. In the U B P ∗ ( I ) U_{\mathrm{BP}}^{*}(\mathcal{I}) U BP ∗ ( I ) optimization problem, the persuasiveness constraint for a 0 a_0 a 0 is
μ R ( ω 0 ) π ( a 0 ∣ ω 0 ) − μ R ( ω 1 ) π ( a 0 ∣ ω 1 ) = μ R ( ω 0 ) ( 1 − π ( a 1 ∣ ω 0 ) ) − μ R ( ω 1 ) ( 1 − π ( a 1 ∣ ω 1 ) ) = μ R ( ω 0 ) − μ R ( ω 1 ) ⏟ ≥ 0 by construction − μ R ( ω 0 ) π ( a 1 ∣ ω 0 ) + μ R ( ω 1 ) π ( a 1 ∣ ω 1 ) ⏟ ≥ 0 when a 1 is persuasive ≥ 0 , \begin{aligned}
\mu_R(\omega_0)\pi(a_0\mid\omega_0)-\mu_R(\omega_1)\pi(a_0\mid\omega_1)
&=\mu_R(\omega_0)(1-\pi(a_1\mid\omega_0))-\mu_R(\omega_1)(1-\pi(a_1\mid\omega_1))\\
&=\underbrace{\mu_R(\omega_0)-\mu_R(\omega_1)}_{\ge0\text{ by construction}}-\underbrace{\mu_R(\omega_0)\pi(a_1\mid\omega_0)+\mu_R(\omega_1)\pi(a_1\mid\omega_1)}_{\ge0\text{ when }a_1\text{ is persuasive}}\ge0,
\end{aligned} μ R ( ω 0 ) π ( a 0 ∣ ω 0 ) − μ R ( ω 1 ) π ( a 0 ∣ ω 1 ) = μ R ( ω 0 ) ( 1 − π ( a 1 ∣ ω 0 )) − μ R ( ω 1 ) ( 1 − π ( a 1 ∣ ω 1 )) = ≥ 0 by construction μ R ( ω 0 ) − μ R ( ω 1 ) − ≥ 0 when a 1 is persuasive μ R ( ω 0 ) π ( a 1 ∣ ω 0 ) + μ R ( ω 1 ) π ( a 1 ∣ ω 1 ) ≥ 0 ,
which is automatically satisfied when a 1 a_1 a 1 is persuasive. So, we can drop the persuasiveness constraint for a 0 a_0 a 0 and write the optimization problem as
U B P ∗ ( I ) = max π : Ω → Δ ( A ) ( 1 − p 0 ) π ( a 1 ∣ ω 0 ) + p 0 π ( a 1 ∣ ω 1 ) U_{\mathrm{BP}}^{*}(\mathcal{I})=\max_{\pi:\Omega\to\Delta(A)}(1-p_0)\pi(a_1\mid\omega_0)+p_0\pi(a_1\mid\omega_1) U BP ∗ ( I ) = π : Ω → Δ ( A ) max ( 1 − p 0 ) π ( a 1 ∣ ω 0 ) + p 0 π ( a 1 ∣ ω 1 )
s.t. − μ R ( ω 0 ) π ( a 1 ∣ ω 0 ) + μ R ( ω 1 ) π ( a 1 ∣ ω 1 ) ≥ 0. \text{s.t.}\quad-\mu_R(\omega_0)\pi(a_1\mid\omega_0)+\mu_R(\omega_1)\pi(a_1\mid\omega_1)\ge0. s.t. − μ R ( ω 0 ) π ( a 1 ∣ ω 0 ) + μ R ( ω 1 ) π ( a 1 ∣ ω 1 ) ≥ 0.
The solution to the above optimization problem is clearly π ( a 1 ∣ ω 1 ) = 1 \pi(a_1\mid\omega_1)=1 π ( a 1 ∣ ω 1 ) = 1 and π ( a 1 ∣ ω 0 ) = μ R ( ω 1 ) μ R ( ω 0 ) = μ ~ R \pi(a_1\mid\omega_0)=\frac{\mu_R(\omega_1)}{\mu_R(\omega_0)}=\widetilde{\mu}_R π ( a 1 ∣ ω 0 ) = μ R ( ω 0 ) μ R ( ω 1 ) = μ R , with optimal objective value U B P ∗ ( I ) = ( 1 − p 0 ) μ ~ R + p 0 U_{\mathrm{BP}}^{*}(\mathcal{I})=(1-p_0)\widetilde{\mu}_R+p_0 U BP ∗ ( I ) = ( 1 − p 0 ) μ R + p 0 , which equals U s p ∗ ( μ R ) U_{\mathrm{sp}}^{*}(\mu_R) U sp ∗ ( μ R ) .
To prove Lemma 12 , we then analyze the designer’s dynamic benchmark U T ∗ ∗ ( I ) U_T^{**}(\mathcal{I}) U T ∗∗ ( I ) . According to Proposition 1, the dynamic benchmark U T ∗ ∗ ( I ) U_T^{**}(\mathcal{I}) U T ∗∗ ( I ) can be achieved by some history-independent dynamically persuasive DDP strategy σ = ( π ( 1 ) , … , π ( T ) ) \sigma=(\pi^{(1)},\ldots,\pi^{(T)}) σ = ( π ( 1 ) , … , π ( T ) ) : namely,
U T ∗ ∗ ( I ) = max σ = ( π ( 1 ) , … , π ( T ) ) ∑ t = 1 T U ( π ( t ) , ρ o b ; μ D ) U_T^{**}(\mathcal{I})=\max_{\sigma=(\pi^{(1)},\ldots,\pi^{(T)})}\sum_{t=1}^{T}U(\pi^{(t)},\rho_{\mathrm{ob}};\mu_D) U T ∗∗ ( I ) = σ = ( π ( 1 ) , … , π ( T ) ) max t = 1 ∑ T U ( π ( t ) , ρ ob ; μ D )
subject to ∀ t ∈ { 1 , … , T } , ∀ a , a ′ ∈ A \forall t\in\{1,\ldots,T\},\ \forall a,a'\in A ∀ t ∈ { 1 , … , T } , ∀ a , a ′ ∈ A ,
γ t E ω ∼ μ R ( ⋅ ∣ a , π ( t ) ) [ v ( a , ω ) − v ( a ′ , ω ) ] + ∑ t ′ = t + 1 T γ t ′ ( V ( π ( t ′ ) , ρ o b ; μ R ) − V u n i n f o r m e d ( μ R ) ) ≥ 0. \gamma^t\mathbb{E}_{\omega\sim\mu_R(\cdot\mid a,\pi^{(t)})}\left[v(a,\omega)-v(a',\omega)\right]+\sum_{t'=t+1}^{T}\gamma^{t'}\left(V(\pi^{(t')},\rho_{\mathrm{ob}};\mu_R)-V_{\mathrm{uninformed}}(\mu_R)\right)\ge0. γ t E ω ∼ μ R ( ⋅ ∣ a , π ( t ) ) [ v ( a , ω ) − v ( a ′ , ω ) ] + t ′ = t + 1 ∑ T γ t ′ ( V ( π ( t ′ ) , ρ ob ; μ R ) − V uninformed ( μ R ) ) ≥ 0.
Plugging in Equations (53) –(57) , we obtain
U T ∗ ∗ ( I ) = max σ = ( π ( 1 ) , … , π ( T ) ) ∑ t = 1 T ( ( 1 − p 0 ) π ( t ) ( a 1 ∣ ω 0 ) + p 0 π ( t ) ( a 1 ∣ ω 1 ) ) \mathcal{U}^{**}_{T}(\mathcal{I})=\max_{\sigma=(\pi^{(1)},\ldots,\pi^{(T)})}\sum_{t=1}^{T}\left((1-p_{0})\pi^{(t)}(a_{1}\mid\omega_{0})+p_{0}\pi^{(t)}(a_{1}\mid\omega_{1})\right) U T ∗∗ ( I ) = σ = ( π ( 1 ) , … , π ( T ) ) max t = 1 ∑ T ( ( 1 − p 0 ) π ( t ) ( a 1 ∣ ω 0 ) + p 0 π ( t ) ( a 1 ∣ ω 1 ) )
subject to ∀ t ∈ { 1 , … , T } \forall t\in\{1,\ldots,T\} ∀ t ∈ { 1 , … , T } ,
γ t μ R ( ω 0 ) π ( t ) ( a 0 ∣ ω 0 ) − μ R ( ω 1 ) π ( t ) ( a 0 ∣ ω 1 ) μ R ( ω 0 ) π ( t ) ( a 0 ∣ ω 0 ) + μ R ( ω 1 ) π ( t ) ( a 0 ∣ ω 1 ) + ∑ t ′ = t + 1 T γ t ′ ( − μ R ( ω 0 ) π ( t ′ ) ( a 1 ∣ ω 0 ) + μ R ( ω 1 ) π ( t ′ ) ( a 1 ∣ ω 1 ) ) ≥ 0 , \gamma^{t}\frac{\mu_{R}(\omega_{0})\pi^{(t)}(a_{0}\mid\omega_{0})-\mu_{R}(\omega_{1})\pi^{(t)}(a_{0}\mid\omega_{1})}{\mu_{R}(\omega_{0})\pi^{(t)}(a_{0}\mid\omega_{0})+\mu_{R}(\omega_{1})\pi^{(t)}(a_{0}\mid\omega_{1})}+\sum_{t'=t+1}^{T}\gamma^{t'}\left(-\mu_{R}(\omega_{0})\pi^{(t')}(a_{1}\mid\omega_{0})+\mu_{R}(\omega_{1})\pi^{(t')}(a_{1}\mid\omega_{1})\right)\geq0, γ t μ R ( ω 0 ) π ( t ) ( a 0 ∣ ω 0 ) + μ R ( ω 1 ) π ( t ) ( a 0 ∣ ω 1 ) μ R ( ω 0 ) π ( t ) ( a 0 ∣ ω 0 ) − μ R ( ω 1 ) π ( t ) ( a 0 ∣ ω 1 ) + t ′ = t + 1 ∑ T γ t ′ ( − μ R ( ω 0 ) π ( t ′ ) ( a 1 ∣ ω 0 ) + μ R ( ω 1 ) π ( t ′ ) ( a 1 ∣ ω 1 ) ) ≥ 0 ,
γ t − μ R ( ω 0 ) π ( t ) ( a 1 ∣ ω 0 ) + μ R ( ω 1 ) π ( t ) ( a 1 ∣ ω 1 ) μ R ( ω 0 ) π ( t ) ( a 1 ∣ ω 0 ) + μ R ( ω 1 ) π ( t ) ( a 1 ∣ ω 1 ) + ∑ t ′ = t + 1 T γ t ′ ( − μ R ( ω 0 ) π ( t ′ ) ( a 1 ∣ ω 0 ) + μ R ( ω 1 ) π ( t ′ ) ( a 1 ∣ ω 1 ) ) ≥ 0 , \gamma^{t}\frac{-\mu_{R}(\omega_{0})\pi^{(t)}(a_{1}\mid\omega_{0})+\mu_{R}(\omega_{1})\pi^{(t)}(a_{1}\mid\omega_{1})}{\mu_{R}(\omega_{0})\pi^{(t)}(a_{1}\mid\omega_{0})+\mu_{R}(\omega_{1})\pi^{(t)}(a_{1}\mid\omega_{1})}+\sum_{t'=t+1}^{T}\gamma^{t'}\left(-\mu_{R}(\omega_{0})\pi^{(t')}(a_{1}\mid\omega_{0})+\mu_{R}(\omega_{1})\pi^{(t')}(a_{1}\mid\omega_{1})\right)\geq0, γ t μ R ( ω 0 ) π ( t ) ( a 1 ∣ ω 0 ) + μ R ( ω 1 ) π ( t ) ( a 1 ∣ ω 1 ) − μ R ( ω 0 ) π ( t ) ( a 1 ∣ ω 0 ) + μ R ( ω 1 ) π ( t ) ( a 1 ∣ ω 1 ) + t ′ = t + 1 ∑ T γ t ′ ( − μ R ( ω 0 ) π ( t ′ ) ( a 1 ∣ ω 0 ) + μ R ( ω 1 ) π ( t ′ ) ( a 1 ∣ ω 1 ) ) ≥ 0 ,
and
∑ t ′ = t + 1 T γ t ′ ( − μ R ( ω 0 ) π ( t ′ ) ( a 1 ∣ ω 0 ) + μ R ( ω 1 ) π ( t ′ ) ( a 1 ∣ ω 1 ) ) ≥ 0. \sum_{t'=t+1}^{T}\gamma^{t'}\left(-\mu_{R}(\omega_{0})\pi^{(t')}(a_{1}\mid\omega_{0})+\mu_{R}(\omega_{1})\pi^{(t')}(a_{1}\mid\omega_{1})\right)\geq0. t ′ = t + 1 ∑ T γ t ′ ( − μ R ( ω 0 ) π ( t ′ ) ( a 1 ∣ ω 0 ) + μ R ( ω 1 ) π ( t ′ ) ( a 1 ∣ ω 1 ) ) ≥ 0.
We note that the optimal solution to the above problem must have π ( t ) ( a 1 ∣ ω 1 ) = 1 \pi^{(t)}(a_{1}\mid\omega_{1})=1 π ( t ) ( a 1 ∣ ω 1 ) = 1 for every t t t . This is because: (1) increasing π ( t ) ( a 1 ∣ ω 1 ) \pi^{(t)}(a_{1}\mid\omega_{1}) π ( t ) ( a 1 ∣ ω 1 ) increases the designer’s utility; (2) increasing π ( t ) ( a 1 ∣ ω 1 ) \pi^{(t)}(a_{1}\mid\omega_{1}) π ( t ) ( a 1 ∣ ω 1 ) will not violate the three constraints because the right-hand-sides of the constraints are all increasing in π ( t ) ( a 1 ∣ ω 1 ) \pi^{(t)}(a_{1}\mid\omega_{1}) π ( t ) ( a 1 ∣ ω 1 ) . Note that in first constraint, the right-hand-side is decreasing in π ( t ) ( a 0 ∣ ω 1 ) \pi^{(t)}(a_{0}\mid\omega_{1}) π ( t ) ( a 0 ∣ ω 1 ) while π ( t ) ( a 0 ∣ ω 1 ) = 1 − π ( t ) ( a 1 ∣ ω 1 ) \pi^{(t)}(a_{0}\mid\omega_{1})=1-\pi^{(t)}(a_{1}\mid\omega_{1}) π ( t ) ( a 0 ∣ ω 1 ) = 1 − π ( t ) ( a 1 ∣ ω 1 ) is decreasing in π ( t ) ( a 1 ∣ ω 1 ) \pi^{(t)}(a_{1}\mid\omega_{1}) π ( t ) ( a 1 ∣ ω 1 ) . So, we can increase π ( t ) ( a 1 ∣ ω 1 ) \pi^{(t)}(a_{1}\mid\omega_{1}) π ( t ) ( a 1 ∣ ω 1 ) to the maximal value of 1. With π ( t ) ( a 1 ∣ ω 1 ) = 1 \pi^{(t)}(a_{1}\mid\omega_{1})=1 π ( t ) ( a 1 ∣ ω 1 ) = 1 , we can rewrite U T ∗ ∗ ( I ) \mathcal{U}^{**}_{T}(\mathcal{I}) U T ∗∗ ( I ) as
U T ∗ ∗ ( I ) = max σ = ( π ( 1 ) , … , π ( T ) ) ∑ t = 1 T ( ( 1 − p 0 ) π ( t ) ( a 1 ∣ ω 0 ) + p 0 ) \mathcal{U}^{**}_{T}(\mathcal{I})=\max_{\sigma=(\pi^{(1)},\ldots,\pi^{(T)})}\sum_{t=1}^{T}\left((1-p_{0})\pi^{(t)}(a_{1}\mid\omega_{0})+p_{0}\right) U T ∗∗ ( I ) = σ = ( π ( 1 ) , … , π ( T ) ) max t = 1 ∑ T ( ( 1 − p 0 ) π ( t ) ( a 1 ∣ ω 0 ) + p 0 )
s.t. γ t μ R ( ω 0 ) π ( t ) ( a 0 ∣ ω 0 ) − 0 μ R ( ω 0 ) π ( t ) ( a 0 ∣ ω 0 ) + 0 + ∑ t ′ = t + 1 T γ t ′ ( − μ R ( ω 0 ) π ( t ′ ) ( a 1 ∣ ω 0 ) + μ R ( ω 1 ) ) ≥ 0 , \gamma^{t}\frac{\mu_{R}(\omega_{0})\pi^{(t)}(a_{0}\mid\omega_{0})-0}{\mu_{R}(\omega_{0})\pi^{(t)}(a_{0}\mid\omega_{0})+0}+\sum_{t'=t+1}^{T}\gamma^{t'}\left(-\mu_{R}(\omega_{0})\pi^{(t')}(a_{1}\mid\omega_{0})+\mu_{R}(\omega_{1})\right)\geq0, γ t μ R ( ω 0 ) π ( t ) ( a 0 ∣ ω 0 ) + 0 μ R ( ω 0 ) π ( t ) ( a 0 ∣ ω 0 ) − 0 + ∑ t ′ = t + 1 T γ t ′ ( − μ R ( ω 0 ) π ( t ′ ) ( a 1 ∣ ω 0 ) + μ R ( ω 1 ) ) ≥ 0 ,
γ t − μ R ( ω 0 ) π ( t ) ( a 1 ∣ ω 0 ) + μ R ( ω 1 ) μ R ( ω 0 ) π ( t ) ( a 1 ∣ ω 0 ) + μ R ( ω 1 ) + ∑ t ′ = t + 1 T γ t ′ ( − μ R ( ω 0 ) π ( t ′ ) ( a 1 ∣ ω 0 ) + μ R ( ω 1 ) ) ≥ 0 , \gamma^{t}\frac{-\mu_{R}(\omega_{0})\pi^{(t)}(a_{1}\mid\omega_{0})+\mu_{R}(\omega_{1})}{\mu_{R}(\omega_{0})\pi^{(t)}(a_{1}\mid\omega_{0})+\mu_{R}(\omega_{1})}+\sum_{t'=t+1}^{T}\gamma^{t'}\left(-\mu_{R}(\omega_{0})\pi^{(t')}(a_{1}\mid\omega_{0})+\mu_{R}(\omega_{1})\right)\geq0, γ t μ R ( ω 0 ) π ( t ) ( a 1 ∣ ω 0 ) + μ R ( ω 1 ) − μ R ( ω 0 ) π ( t ) ( a 1 ∣ ω 0 ) + μ R ( ω 1 ) + t ′ = t + 1 ∑ T γ t ′ ( − μ R ( ω 0 ) π ( t ′ ) ( a 1 ∣ ω 0 ) + μ R ( ω 1 ) ) ≥ 0 ,
and
∑ t ′ = t + 1 T γ t ′ ( − μ R ( ω 0 ) π ( t ′ ) ( a 1 ∣ ω 0 ) + μ R ( ω 1 ) ) ≥ 0. \sum_{t'=t+1}^{T}\gamma^{t'}\left(-\mu_{R}(\omega_{0})\pi^{(t')}(a_{1}\mid\omega_{0})+\mu_{R}(\omega_{1})\right)\geq0. t ′ = t + 1 ∑ T γ t ′ ( − μ R ( ω 0 ) π ( t ′ ) ( a 1 ∣ ω 0 ) + μ R ( ω 1 ) ) ≥ 0.
Because μ R ( ω 0 ) π ( t ) ( a 0 ∣ ω 0 ) − 0 μ R ( ω 0 ) π ( t ) ( a 0 ∣ ω 0 ) + 0 = 1 \frac{\mu_{R}(\omega_{0})\pi^{(t)}(a_{0}\mid\omega_{0})-0}{\mu_{R}(\omega_{0})\pi^{(t)}(a_{0}\mid\omega_{0})+0}=1 μ R ( ω 0 ) π ( t ) ( a 0 ∣ ω 0 ) + 0 μ R ( ω 0 ) π ( t ) ( a 0 ∣ ω 0 ) − 0 = 1 , the first constraint is implied by the third constraint. So we can drop the first constraint. Letting x ( t ) = π ( t ) ( a 1 ∣ ω 0 ) x^{(t)}=\pi^{(t)}(a_1\mid\omega_0) x ( t ) = π ( t ) ( a 1 ∣ ω 0 ) , we have
U T ∗ ∗ ( I ) = max 0 ≤ x ( t ) ≤ 1 ∑ t = 1 T ( ( 1 − p 0 ) x ( t ) + p 0 ) or equivalently max 0 ≤ x ( t ) ≤ 1 ∑ t = 1 T x ( t ) (58) \mathcal{U}^{**}_T(\mathcal{I})=\max_{0\leq x^{(t)}\leq1}\sum_{t=1}^{T}\left((1-p_0)x^{(t)}+p_0\right)
\quad\text{or equivalently}\quad
\max_{0\leq x^{(t)}\leq1}\sum_{t=1}^{T}x^{(t)} \tag*{(58)} U T ∗∗ ( I ) = 0 ≤ x ( t ) ≤ 1 max t = 1 ∑ T ( ( 1 − p 0 ) x ( t ) + p 0 ) or equivalently 0 ≤ x ( t ) ≤ 1 max t = 1 ∑ T x ( t ) (58)
s.t. γ t − μ R ( ω 0 ) x ( t ) + μ R ( ω 1 ) μ R ( ω 0 ) x ( t ) + μ R ( ω 1 ) + ∑ t ′ = t + 1 T γ t ′ ( − μ R ( ω 0 ) x ( t ′ ) + μ R ( ω 1 ) ) ≥ 0 , (59) \text{s.t.}\quad
\gamma^t\frac{-\mu_R(\omega_0)x^{(t)}+\mu_R(\omega_1)}
{\mu_R(\omega_0)x^{(t)}+\mu_R(\omega_1)}
+\sum_{t'=t+1}^{T}\gamma^{t'}\left(-\mu_R(\omega_0)x^{(t')}+\mu_R(\omega_1)\right)\geq0, \tag*{(59)} s.t. γ t μ R ( ω 0 ) x ( t ) + μ R ( ω 1 ) − μ R ( ω 0 ) x ( t ) + μ R ( ω 1 ) + t ′ = t + 1 ∑ T γ t ′ ( − μ R ( ω 0 ) x ( t ′ ) + μ R ( ω 1 ) ) ≥ 0 , (59)
and ∑ t ′ = t + 1 T γ t ′ ( − μ R ( ω 0 ) x ( t ′ ) + μ R ( ω 1 ) ) ≥ 0. (60) \text{and}\quad
\sum_{t'=t+1}^{T}\gamma^{t'}\left(-\mu_R(\omega_0)x^{(t')}+\mu_R(\omega_1)\right)\geq0. \tag*{(60)} and t ′ = t + 1 ∑ T γ t ′ ( − μ R ( ω 0 ) x ( t ′ ) + μ R ( ω 1 ) ) ≥ 0. (60)
Claim 9. A solution to the optimization problem (57)–(59) is x ( t ) = μ R ( ω 1 ) μ R ( ω 0 ) x^{(t)}=\frac{\mu_R(\omega_1)}{\mu_R(\omega_0)} x ( t ) = μ R ( ω 0 ) μ R ( ω 1 ) for every t ∈ { 1 , … , T } t\in\{1,\ldots,T\} t ∈ { 1 , … , T } .
Proof. We prove this claim by induction on T T T . When T = 1 T=1 T = 1 , the optimization problem becomes
max 0 ≤ x ( 1 ) ≤ 1 x ( 1 ) \max_{0\leq x^{(1)}\leq1}x^{(1)} 0 ≤ x ( 1 ) ≤ 1 max x ( 1 )
s.t. γ 1 − μ R ( ω 0 ) x ( 1 ) + μ R ( ω 1 ) μ R ( ω 0 ) x ( 1 ) + μ R ( ω 1 ) ≥ 0 , \text{s.t.}\quad
\gamma^1\frac{-\mu_R(\omega_0)x^{(1)}+\mu_R(\omega_1)}
{\mu_R(\omega_0)x^{(1)}+\mu_R(\omega_1)}\geq0, s.t. γ 1 μ R ( ω 0 ) x ( 1 ) + μ R ( ω 1 ) − μ R ( ω 0 ) x ( 1 ) + μ R ( ω 1 ) ≥ 0 ,
which is clearly maximized by x ( 1 ) = μ R ( ω 1 ) μ R ( ω 0 ) x^{(1)}=\frac{\mu_R(\omega_1)}{\mu_R(\omega_0)} x ( 1 ) = μ R ( ω 0 ) μ R ( ω 1 ) .
Consider T ≥ 2 T\geq2 T ≥ 2 . Let x = ( x ( t ) ) t = 1 T x=(x^{(t)})_{t=1}^{T} x = ( x ( t ) ) t = 1 T be any feasible solution where x ( T ) < μ R ( ω 1 ) μ R ( ω 0 ) x^{(T)}<\frac{\mu_R(\omega_1)}{\mu_R(\omega_0)} x ( T ) < μ R ( ω 0 ) μ R ( ω 1 ) . We define x ~ = ( x ~ ( t ) ) t = 1 T \tilde{x}=(\tilde{x}^{(t)})_{t=1}^{T} x ~ = ( x ~ ( t ) ) t = 1 T by
x ~ ( T ) = μ R ( ω 1 ) μ R ( ω 0 ) , x ~ ( T − 1 ) = x ( T − 1 ) − γ T γ T − 1 ( μ R ( ω 1 ) μ R ( ω 0 ) − x ( T ) ) , x ~ ( t ) = x ( t ) ∀ t ≤ T − 2. \tilde{x}^{(T)}=\frac{\mu_R(\omega_1)}{\mu_R(\omega_0)},\qquad
\tilde{x}^{(T-1)}=x^{(T-1)}-\frac{\gamma^T}{\gamma^{T-1}}\left(\frac{\mu_R(\omega_1)}{\mu_R(\omega_0)}-x^{(T)}\right),\qquad
\tilde{x}^{(t)}=x^{(t)}\ \ \forall t\leq T-2. x ~ ( T ) = μ R ( ω 0 ) μ R ( ω 1 ) , x ~ ( T − 1 ) = x ( T − 1 ) − γ T − 1 γ T ( μ R ( ω 0 ) μ R ( ω 1 ) − x ( T ) ) , x ~ ( t ) = x ( t ) ∀ t ≤ T − 2.
We show that x ~ \tilde{x} x ~ is feasible and has a weakly larger objective value than x x x . To verify the feasibility of x ~ \tilde{x} x ~ , we verify the first constraint (59) for every t t t :
γ t − μ R ( ω 0 ) x ~ ( t ) + μ R ( ω 1 ) μ R ( ω 0 ) x ~ ( t ) + μ R ( ω 1 ) + ∑ t ′ = t + 1 T γ t ′ ( − μ R ( ω 0 ) x ~ ( t ′ ) + μ R ( ω 1 ) ) = γ t − μ R ( ω 0 ) x ( t ) + μ R ( ω 1 ) μ R ( ω 0 ) x ( t ) + μ R ( ω 1 ) + ∑ t ′ = t + 1 T − 2 γ t ′ ( − μ R ( ω 0 ) x ( t ′ ) + μ R ( ω 1 ) ) + γ T − 1 ( − μ R ( ω 0 ) x ~ ( T − 1 ) + μ R ( ω 1 ) ) + γ T ⋅ 0 = γ t − μ R ( ω 0 ) x ( t ) + μ R ( ω 1 ) μ R ( ω 0 ) x ( t ) + μ R ( ω 1 ) + ∑ t ′ = t + 1 T − 2 γ t ′ ( − μ R ( ω 0 ) x ( t ′ ) + μ R ( ω 1 ) ) + γ T − 1 ( − μ R ( ω 0 ) ( x ( T − 1 ) − γ T γ T − 1 ( μ R ( ω 1 ) μ R ( ω 0 ) − x ( T ) ) ) + μ R ( ω 1 ) ) = γ t − μ R ( ω 0 ) x ( t ) + μ R ( ω 1 ) μ R ( ω 0 ) x ( t ) + μ R ( ω 1 ) + ∑ t ′ = t + 1 T − 2 γ t ′ ( − μ R ( ω 0 ) x ( t ′ ) + μ R ( ω 1 ) ) + γ T − 1 ( − μ R ( ω 0 ) x ( T − 1 ) + μ R ( ω 1 ) ) + γ T ( − μ R ( ω 0 ) x ( T ) + μ R ( ω 1 ) ) ≥ 0 \begin{aligned}
\gamma^t \frac{-\mu_R(\omega_0)\tilde{x}^{(t)}+\mu_R(\omega_1)}{\mu_R(\omega_0)\tilde{x}^{(t)}+\mu_R(\omega_1)}+\sum_{t'=t+1}^{T}\gamma^{t'}\left(-\mu_R(\omega_0)\tilde{x}^{(t')}+\mu_R(\omega_1)\right)
&=\gamma^t\frac{-\mu_R(\omega_0)x^{(t)}+\mu_R(\omega_1)}{\mu_R(\omega_0)x^{(t)}+\mu_R(\omega_1)}+\sum_{t'=t+1}^{T-2}\gamma^{t'}\left(-\mu_R(\omega_0)x^{(t')}+\mu_R(\omega_1)\right)\\
&\quad+\gamma^{T-1}\left(-\mu_R(\omega_0)\tilde{x}^{(T-1)}+\mu_R(\omega_1)\right)+\gamma^T\cdot0\\
&=\gamma^t\frac{-\mu_R(\omega_0)x^{(t)}+\mu_R(\omega_1)}{\mu_R(\omega_0)x^{(t)}+\mu_R(\omega_1)}+\sum_{t'=t+1}^{T-2}\gamma^{t'}\left(-\mu_R(\omega_0)x^{(t')}+\mu_R(\omega_1)\right)\\
&\quad+\gamma^{T-1}\left(-\mu_R(\omega_0)\left(x^{(T-1)}-\frac{\gamma^T}{\gamma^{T-1}}\left(\frac{\mu_R(\omega_1)}{\mu_R(\omega_0)}-x^{(T)}\right)\right)+\mu_R(\omega_1)\right)\\
&=\gamma^t\frac{-\mu_R(\omega_0)x^{(t)}+\mu_R(\omega_1)}{\mu_R(\omega_0)x^{(t)}+\mu_R(\omega_1)}+\sum_{t'=t+1}^{T-2}\gamma^{t'}\left(-\mu_R(\omega_0)x^{(t')}+\mu_R(\omega_1)\right)\\
&\quad+\gamma^{T-1}\left(-\mu_R(\omega_0)x^{(T-1)}+\mu_R(\omega_1)\right)+\gamma^T\left(-\mu_R(\omega_0)x^{(T)}+\mu_R(\omega_1)\right)\\
&\ge0
\end{aligned} γ t μ R ( ω 0 ) x ~ ( t ) + μ R ( ω 1 ) − μ R ( ω 0 ) x ~ ( t ) + μ R ( ω 1 ) + t ′ = t + 1 ∑ T γ t ′ ( − μ R ( ω 0 ) x ~ ( t ′ ) + μ R ( ω 1 ) ) = γ t μ R ( ω 0 ) x ( t ) + μ R ( ω 1 ) − μ R ( ω 0 ) x ( t ) + μ R ( ω 1 ) + t ′ = t + 1 ∑ T − 2 γ t ′ ( − μ R ( ω 0 ) x ( t ′ ) + μ R ( ω 1 ) ) + γ T − 1 ( − μ R ( ω 0 ) x ~ ( T − 1 ) + μ R ( ω 1 ) ) + γ T ⋅ 0 = γ t μ R ( ω 0 ) x ( t ) + μ R ( ω 1 ) − μ R ( ω 0 ) x ( t ) + μ R ( ω 1 ) + t ′ = t + 1 ∑ T − 2 γ t ′ ( − μ R ( ω 0 ) x ( t ′ ) + μ R ( ω 1 ) ) + γ T − 1 ( − μ R ( ω 0 ) ( x ( T − 1 ) − γ T − 1 γ T ( μ R ( ω 0 ) μ R ( ω 1 ) − x ( T ) ) ) + μ R ( ω 1 ) ) = γ t μ R ( ω 0 ) x ( t ) + μ R ( ω 1 ) − μ R ( ω 0 ) x ( t ) + μ R ( ω 1 ) + t ′ = t + 1 ∑ T − 2 γ t ′ ( − μ R ( ω 0 ) x ( t ′ ) + μ R ( ω 1 ) ) + γ T − 1 ( − μ R ( ω 0 ) x ( T − 1 ) + μ R ( ω 1 ) ) + γ T ( − μ R ( ω 0 ) x ( T ) + μ R ( ω 1 ) ) ≥ 0
where the “≥ 0 \ge0 ≥ 0 ” is because x x x satisfies (59) .
γ t − μ R ( ω 0 ) x ~ ( t ) + μ R ( ω 1 ) μ R ( ω 0 ) x ~ ( t ) + μ R ( ω 1 ) + ∑ t ′ = t + 1 T γ t ′ ( − μ R ( ω 0 ) x ~ ( t ′ ) + μ R ( ω 1 ) ) = γ T − 1 − μ R ( ω 0 ) x ~ ( T − 1 ) + μ R ( ω 1 ) μ R ( ω 0 ) x ~ ( T − 1 ) + μ R ( ω 1 ) + γ T ⋅ 0 = γ T − 1 − μ R ( ω 0 ) ( x ( T − 1 ) − γ T γ T − 1 ( μ R ( ω 1 ) μ R ( ω 0 ) − x ( T ) ) ) + μ R ( ω 1 ) μ R ( ω 0 ) x ~ ( T − 1 ) + μ R ( ω 1 ) = γ T − 1 ( − μ R ( ω 0 ) x ( T − 1 ) + μ R ( ω 1 ) ) + γ T ( − μ R ( ω 0 ) x ( T ) + μ R ( ω 1 ) ) μ R ( ω 0 ) x ~ ( T − 1 ) + μ R ( ω 1 ) ≥ 0 \begin{aligned}
\gamma^t\frac{-\mu_R(\omega_0)\tilde{x}^{(t)}+\mu_R(\omega_1)}{\mu_R(\omega_0)\tilde{x}^{(t)}+\mu_R(\omega_1)}+\sum_{t'=t+1}^{T}\gamma^{t'}\left(-\mu_R(\omega_0)\tilde{x}^{(t')}+\mu_R(\omega_1)\right)
&=\gamma^{T-1}\frac{-\mu_R(\omega_0)\tilde{x}^{(T-1)}+\mu_R(\omega_1)}{\mu_R(\omega_0)\tilde{x}^{(T-1)}+\mu_R(\omega_1)}+\gamma^T\cdot0\\
&=\gamma^{T-1}\frac{-\mu_R(\omega_0)\left(x^{(T-1)}-\frac{\gamma^T}{\gamma^{T-1}}\left(\frac{\mu_R(\omega_1)}{\mu_R(\omega_0)}-x^{(T)}\right)\right)+\mu_R(\omega_1)}{\mu_R(\omega_0)\tilde{x}^{(T-1)}+\mu_R(\omega_1)}\\
&=\frac{\gamma^{T-1}\left(-\mu_R(\omega_0)x^{(T-1)}+\mu_R(\omega_1)\right)+\gamma^T\left(-\mu_R(\omega_0)x^{(T)}+\mu_R(\omega_1)\right)}{\mu_R(\omega_0)\tilde{x}^{(T-1)}+\mu_R(\omega_1)}\\
&\ge0
\end{aligned} γ t μ R ( ω 0 ) x ~ ( t ) + μ R ( ω 1 ) − μ R ( ω 0 ) x ~ ( t ) + μ R ( ω 1 ) + t ′ = t + 1 ∑ T γ t ′ ( − μ R ( ω 0 ) x ~ ( t ′ ) + μ R ( ω 1 ) ) = γ T − 1 μ R ( ω 0 ) x ~ ( T − 1 ) + μ R ( ω 1 ) − μ R ( ω 0 ) x ~ ( T − 1 ) + μ R ( ω 1 ) + γ T ⋅ 0 = γ T − 1 μ R ( ω 0 ) x ~ ( T − 1 ) + μ R ( ω 1 ) − μ R ( ω 0 ) ( x ( T − 1 ) − γ T − 1 γ T ( μ R ( ω 0 ) μ R ( ω 1 ) − x ( T ) ) ) + μ R ( ω 1 ) = μ R ( ω 0 ) x ~ ( T − 1 ) + μ R ( ω 1 ) γ T − 1 ( − μ R ( ω 0 ) x ( T − 1 ) + μ R ( ω 1 ) ) + γ T ( − μ R ( ω 0 ) x ( T ) + μ R ( ω 1 ) ) ≥ 0
where the “≥ 0 \ge0 ≥ 0 ” is because x x x satisfies (60) .
γ t − μ R ( ω 0 ) x ~ ( T ) + μ R ( ω 1 ) μ R ( ω 0 ) x ~ ( T ) + μ R ( ω 1 ) = 0 ≥ 0. \gamma^t\frac{-\mu_R(\omega_0)\tilde{x}^{(T)}+\mu_R(\omega_1)}{\mu_R(\omega_0)\tilde{x}^{(T)}+\mu_R(\omega_1)}=0\ge0. γ t μ R ( ω 0 ) x ~ ( T ) + μ R ( ω 1 ) − μ R ( ω 0 ) x ~ ( T ) + μ R ( ω 1 ) = 0 ≥ 0.
Thus, x ~ \tilde{x} x ~ satisfies the first constraint (59) . One can similarly verify that x ~ \tilde{x} x ~ satisfies the second constraint (60) . We then show that x ~ \tilde{x} x ~ has a weakly larger objective value than x x x :
∑ t = 1 T x ~ ( t ) = ∑ t = 1 T − 2 x ( t ) + x ( T − 1 ) − γ T γ T − 1 ( μ R ( ω 1 ) μ R ( ω 0 ) − x ( T ) ) + μ R ( ω 1 ) μ R ( ω 0 ) ≥ ∑ t = 1 T − 2 x ( t ) + x ( T − 1 ) − ( μ R ( ω 1 ) μ R ( ω 0 ) − x ( T ) ) + μ R ( ω 1 ) μ R ( ω 0 ) = ∑ t = 1 T x ( t ) . \begin{aligned}
\sum_{t=1}^{T}\tilde{x}^{(t)}
&= \sum_{t=1}^{T-2}x^{(t)}+x^{(T-1)}-\frac{\gamma^{T}}{\gamma^{T-1}}\left(\frac{\mu_R(\omega_1)}{\mu_R(\omega_0)}-x^{(T)}\right)+\frac{\mu_R(\omega_1)}{\mu_R(\omega_0)} \\
&\ge\sum_{t=1}^{T-2}x^{(t)}+x^{(T-1)}-\left(\frac{\mu_R(\omega_1)}{\mu_R(\omega_0)}-x^{(T)}\right)+\frac{\mu_R(\omega_1)}{\mu_R(\omega_0)} \\
&= \sum_{t=1}^{T}x^{(t)}.
\end{aligned} t = 1 ∑ T x ~ ( t ) = t = 1 ∑ T − 2 x ( t ) + x ( T − 1 ) − γ T − 1 γ T ( μ R ( ω 0 ) μ R ( ω 1 ) − x ( T ) ) + μ R ( ω 0 ) μ R ( ω 1 ) ≥ t = 1 ∑ T − 2 x ( t ) + x ( T − 1 ) − ( μ R ( ω 0 ) μ R ( ω 1 ) − x ( T ) ) + μ R ( ω 0 ) μ R ( ω 1 ) = t = 1 ∑ T x ( t ) .
Since x ~ \tilde{x} x ~ is feasible and weakly better than x x x , we can safely set x ( T ) = μ R ( ω 1 ) μ R ( ω 0 ) x^{(T)}=\frac{\mu_R(\omega_1)}{\mu_R(\omega_0)} x ( T ) = μ R ( ω 0 ) μ R ( ω 1 ) and reduce the optimization problem to the T − 1 T-1 T − 1 period problem:
max 0 ≤ x ( t ) ≤ 1 ∑ t = 1 T − 1 x ( t ) s.t. γ t − μ R ( ω 0 ) x ( t ) + μ R ( ω 1 ) μ R ( ω 0 ) x ( t ) + μ R ( ω 1 ) + ∑ t ′ = t + 1 T − 1 γ t ′ ( − μ R ( ω 0 ) x ( t ′ ) + μ R ( ω 1 ) ) ≥ 0 , and ∑ t ′ = t + 1 T − 1 γ t ′ ( − μ R ( ω 0 ) x ( t ′ ) + μ R ( ω 1 ) ) ≥ 0. \begin{aligned}
\max_{0\le x^{(t)}\le1}\quad&\sum_{t=1}^{T-1}x^{(t)} \\
\text{s.t.}\quad&\gamma^{t}\frac{-\mu_R(\omega_0)x^{(t)}+\mu_R(\omega_1)}{\mu_R(\omega_0)x^{(t)}+\mu_R(\omega_1)}
+\sum_{t'=t+1}^{T-1}\gamma^{t'}\left(-\mu_R(\omega_0)x^{(t')}+\mu_R(\omega_1)\right)\ge0, \\
&\text{and}\quad\sum_{t'=t+1}^{T-1}\gamma^{t'}\left(-\mu_R(\omega_0)x^{(t')}+\mu_R(\omega_1)\right)\ge0.
\end{aligned} 0 ≤ x ( t ) ≤ 1 max s.t. t = 1 ∑ T − 1 x ( t ) γ t μ R ( ω 0 ) x ( t ) + μ R ( ω 1 ) − μ R ( ω 0 ) x ( t ) + μ R ( ω 1 ) + t ′ = t + 1 ∑ T − 1 γ t ′ ( − μ R ( ω 0 ) x ( t ′ ) + μ R ( ω 1 ) ) ≥ 0 , and t ′ = t + 1 ∑ T − 1 γ t ′ ( − μ R ( ω 0 ) x ( t ′ ) + μ R ( ω 1 ) ) ≥ 0.
By induction, the T − 1 T-1 T − 1 period problem has solution ( x ( t ) = μ R ( ω 1 ) μ R ( ω 0 ) ) t = 1 T − 1 \left(x^{(t)}=\frac{\mu_R(\omega_1)}{\mu_R(\omega_0)}\right)_{t=1}^{T-1} ( x ( t ) = μ R ( ω 0 ) μ R ( ω 1 ) ) t = 1 T − 1 . Thus, the T T T period problem has solution ( x ( t ) = μ R ( ω 1 ) μ R ( ω 0 ) ) t = 1 T \left(x^{(t)}=\frac{\mu_R(\omega_1)}{\mu_R(\omega_0)}\right)_{t=1}^{T} ( x ( t ) = μ R ( ω 0 ) μ R ( ω 1 ) ) t = 1 T .
By Claim 9, we have x ( t ) = μ R ( ω 1 ) μ R ( ω 0 ) x^{(t)}=\frac{\mu_R(\omega_1)}{\mu_R(\omega_0)} x ( t ) = μ R ( ω 0 ) μ R ( ω 1 ) and thus
U T ∗ ( I ) = ∑ t = 1 T ( ( 1 − p 0 ) μ R ( ω 1 ) μ R ( ω 0 ) + p 0 ) = T ⋅ ( ( 1 − p 0 ) μ ~ R + p 0 ) = by Claim 8 T ⋅ U B P ∗ ( I ) = U T ∗ ( I ) U_T^{*}(\mathcal{I})=\sum_{t=1}^{T}\left((1-p_0)\frac{\mu_R(\omega_1)}{\mu_R(\omega_0)}+p_0\right)=T\cdot\left((1-p_0)\tilde{\mu}_R+p_0\right)\mathrel{\overset{\text{by Claim 8}}{=} }T\cdot U_{\mathrm{BP}}^{*}(\mathcal{I})=U_T^{*}(\mathcal{I}) U T ∗ ( I ) = t = 1 ∑ T ( ( 1 − p 0 ) μ R ( ω 0 ) μ R ( ω 1 ) + p 0 ) = T ⋅ ( ( 1 − p 0 ) μ ~ R + p 0 ) = by Claim 8 T ⋅ U BP ∗ ( I ) = U T ∗ ( I )
which proves Lemma 12 .
Proof of Lemma 13
Under menu M \mathcal{M} M , the receiver’s expected utility with prior μ R \mu_R μ R and a report μ \mu μ in the single-period is
V s p ( M , μ R , μ ) = ∑ ω ∈ Ω μ R ( ω ) ∑ a ∈ A σ μ ( a ∣ ω ) v ( a , ω ) = E ( π ( t ) , ρ ( t ) ) ∼ G , ϕ ∗ ( G , μ ) [ 1 T γ ∑ t = 1 T γ t ∑ ω ∈ Ω μ R ( ω ) ∑ s ∈ S π ( t ) ( s ∣ ω ) ∑ a ∈ A ρ ( t ) ( a ∣ s ) v ( a , ω ) ] ≤ E ( π ( t ) , ρ ( t ) ) ∼ G , ϕ ∗ ( G , μ R ) [ 1 T γ ∑ t = 1 T γ t ∑ ω ∈ Ω μ R ( ω ) ∑ s ∈ S π ( t ) ( s ∣ ω ) ∑ a ∈ A ρ ( t ) ( a ∣ s ) v ( a , ω ) ] = V s p ( M , μ R , μ R ) , \begin{aligned}
V^{\mathrm{sp}}(\mathcal{M},\mu_R,\mu)
&= \sum_{\omega\in\Omega}\mu_R(\omega)\sum_{a\in A}\sigma_\mu(a\mid\omega)v(a,\omega)\\
&= \mathbb{E}_{(\pi^{(t)},\rho^{(t)})\sim\mathcal{G},\phi^*(\mathcal{G},\mu)}
\left[
\frac{1}{T_\gamma}\sum_{t=1}^{T}\gamma^t\sum_{\omega\in\Omega}\mu_R(\omega)
\sum_{s\in S}\pi^{(t)}(s\mid\omega)\sum_{a\in A}\rho^{(t)}(a\mid s)v(a,\omega)
\right]\\
&\leq\mathbb{E}_{(\pi^{(t)},\rho^{(t)})\sim\mathcal{G},\phi^*(\mathcal{G},\mu_R)}
\left[
\frac{1}{T_\gamma}\sum_{t=1}^{T}\gamma^t\sum_{\omega\in\Omega}\mu_R(\omega)
\sum_{s\in S}\pi^{(t)}(s\mid\omega)\sum_{a\in A}\rho^{(t)}(a\mid s)v(a,\omega)
\right]\\
&= V^{\mathrm{sp}}(\mathcal{M},\mu_R,\mu_R),
\end{aligned} V sp ( M , μ R , μ ) = ω ∈ Ω ∑ μ R ( ω ) a ∈ A ∑ σ μ ( a ∣ ω ) v ( a , ω ) = E ( π ( t ) , ρ ( t ) ) ∼ G , ϕ ∗ ( G , μ ) [ T γ 1 t = 1 ∑ T γ t ω ∈ Ω ∑ μ R ( ω ) s ∈ S ∑ π ( t ) ( s ∣ ω ) a ∈ A ∑ ρ ( t ) ( a ∣ s ) v ( a , ω ) ] ≤ E ( π ( t ) , ρ ( t ) ) ∼ G , ϕ ∗ ( G , μ R ) [ T γ 1 t = 1 ∑ T γ t ω ∈ Ω ∑ μ R ( ω ) s ∈ S ∑ π ( t ) ( s ∣ ω ) a ∈ A ∑ ρ ( t ) ( a ∣ s ) v ( a , ω ) ] = V sp ( M , μ R , μ R ) ,
where the ≤ \leq ≤ is because ϕ ∗ ( G , μ R ) \phi^*(\mathcal{G},\mu_R) ϕ ∗ ( G , μ R ) is a best response to G \mathcal{G} G for a strategic receiver with prior μ R \mu_R μ R in the T T T -period game. So, M \mathcal{M} M is incentive compatible. By a similar argument, one can verify that M \mathcal{M} M is individually rational. So the claim is proven.
Proof of Lemma 13
We will prove Reg ( G ; μ D , μ R ) ≥ T γ Reg s p ( M , μ R ) \operatorname{Reg}(\mathcal{G};\mu_D,\mu_R) \ge T_\gamma\operatorname{Reg}^{\mathrm{sp}}(\mathcal{M},\mu_R) Reg ( G ; μ D , μ R ) ≥ T γ Reg sp ( M , μ R ) . Consider:
T γ Reg s p ( M , μ R ) = T γ [ U s p ∗ ( μ R ) − U s p ( M , μ R ) ] = T γ [ U s p ∗ ( μ R ) − ∑ ω ∈ Ω μ D ( ω ) ∑ a ∈ A σ μ R ( a ∣ ω ) u ( a , ω ) ] = T γ U s p ∗ ( μ R ) − E ( π ( t ) , ρ ( t ) ) ∼ G , ϕ ∗ ( G , μ R ) [ ∑ t = 1 T γ t ∑ ω ∈ Ω μ D ( ω ) ∑ s ∈ S π ( t ) ( s ∣ ω ) ∑ a ∈ A ρ ( t ) ( a ∣ s ) u ( a , ω ) ] = T U s p ∗ ( μ R ) − ( T − T γ ) U s p ∗ ( μ R ) − E ( π ( t ) , ρ ( t ) ) ∼ G , ϕ ∗ ( G , μ R ) [ ∑ t = 1 T ( 1 − 1 + γ t ) ∑ ω ∈ Ω μ D ( ω ) ∑ s ∈ S π ( t ) ( s ∣ ω ) ∑ a ∈ A ρ ( t ) ( a ∣ s ) u ( a , ω ) ] = T U s p ∗ ( μ R ) − E ( π ( t ) , ρ ( t ) ) ∼ G , ϕ ∗ ( G , μ R ) [ ∑ t = 1 T ∑ ω ∈ Ω μ D ( ω ) ∑ s ∈ S π ( t ) ( s ∣ ω ) ∑ a ∈ A ρ ( t ) ( a ∣ s ) u ( a , ω ) ] − ( T − T γ ) U s p ∗ ( μ R ) + E ( π ( t ) , ρ ( t ) ) ∼ G , ϕ ∗ ( G , μ R ) [ ∑ t = 1 T ( 1 − γ t ) ∑ ω ∈ Ω μ D ( ω ) ∑ s ∈ S π ( t ) ( s ∣ ω ) ∑ a ∈ A ρ ( t ) ( a ∣ s ) u ( a , ω ) ] = Reg ( G ; μ D , μ R ) − E ( π ( t ) , ρ ( t ) ) ∼ G , ϕ ∗ ( G , μ R ) [ ∑ t = 1 T ( 1 − γ t ) ( U s p ∗ ( μ R ) − ∑ ω ∈ Ω μ D ( ω ) ∑ s ∈ S π ( t ) ( s ∣ ω ) ∑ a ∈ A ρ ( t ) ( a ∣ s ) u ( a , ω ) ) ] . \begin{aligned}
T_\gamma\operatorname{Reg}^{\mathrm{sp}}(\mathcal{M},\mu_R)
&=T_\gamma\left[U_{\mathrm{sp}}^*(\mu_R)-U^{\mathrm{sp}}(\mathcal{M},\mu_R)\right]\\
&=T_\gamma\left[U_{\mathrm{sp}}^*(\mu_R)-\sum_{\omega\in\Omega}\mu_D(\omega)\sum_{a\in A}\sigma_{\mu_R}(a\mid\omega)u(a,\omega)\right]\\
&=T_\gamma U_{\mathrm{sp}}^*(\mu_R)-\mathbb{E}_{(\pi^{(t)},\rho^{(t)})\sim\mathcal{G},\phi^*(\mathcal{G},\mu_R)}\left[\sum_{t=1}^{T}\gamma^t\sum_{\omega\in\Omega}\mu_D(\omega)\sum_{s\in S}\pi^{(t)}(s\mid\omega)\sum_{a\in A}\rho^{(t)}(a\mid s)u(a,\omega)\right]\\
&=TU_{\mathrm{sp}}^*(\mu_R)-(T-T_\gamma)U_{\mathrm{sp}}^*(\mu_R)\\
&\quad-\mathbb{E}_{(\pi^{(t)},\rho^{(t)})\sim\mathcal{G},\phi^*(\mathcal{G},\mu_R)}\left[\sum_{t=1}^{T}(1-1+\gamma^t)\sum_{\omega\in\Omega}\mu_D(\omega)\sum_{s\in S}\pi^{(t)}(s\mid\omega)\sum_{a\in A}\rho^{(t)}(a\mid s)u(a,\omega)\right]\\
&=TU_{\mathrm{sp}}^*(\mu_R)-\mathbb{E}_{(\pi^{(t)},\rho^{(t)})\sim\mathcal{G},\phi^*(\mathcal{G},\mu_R)}\left[\sum_{t=1}^{T}\sum_{\omega\in\Omega}\mu_D(\omega)\sum_{s\in S}\pi^{(t)}(s\mid\omega)\sum_{a\in A}\rho^{(t)}(a\mid s)u(a,\omega)\right]\\
&\quad-(T-T_\gamma)U_{\mathrm{sp}}^*(\mu_R)+\mathbb{E}_{(\pi^{(t)},\rho^{(t)})\sim\mathcal{G},\phi^*(\mathcal{G},\mu_R)}\left[\sum_{t=1}^{T}(1-\gamma^t)\sum_{\omega\in\Omega}\mu_D(\omega)\sum_{s\in S}\pi^{(t)}(s\mid\omega)\sum_{a\in A}\rho^{(t)}(a\mid s)u(a,\omega)\right]\\
&=\operatorname{Reg}(\mathcal{G};\mu_D,\mu_R)\\
&\quad-\mathbb{E}_{(\pi^{(t)},\rho^{(t)})\sim\mathcal{G},\phi^*(\mathcal{G},\mu_R)}\left[\sum_{t=1}^{T}(1-\gamma^t)\left(U_{\mathrm{sp}}^*(\mu_R)-\sum_{\omega\in\Omega}\mu_D(\omega)\sum_{s\in S}\pi^{(t)}(s\mid\omega)\sum_{a\in A}\rho^{(t)}(a\mid s)u(a,\omega)\right)\right].
\end{aligned} T γ Reg sp ( M , μ R ) = T γ [ U sp ∗ ( μ R ) − U sp ( M , μ R ) ] = T γ [ U sp ∗ ( μ R ) − ω ∈ Ω ∑ μ D ( ω ) a ∈ A ∑ σ μ R ( a ∣ ω ) u ( a , ω ) ] = T γ U sp ∗ ( μ R ) − E ( π ( t ) , ρ ( t ) ) ∼ G , ϕ ∗ ( G , μ R ) [ t = 1 ∑ T γ t ω ∈ Ω ∑ μ D ( ω ) s ∈ S ∑ π ( t ) ( s ∣ ω ) a ∈ A ∑ ρ ( t ) ( a ∣ s ) u ( a , ω ) ] = T U sp ∗ ( μ R ) − ( T − T γ ) U sp ∗ ( μ R ) − E ( π ( t ) , ρ ( t ) ) ∼ G , ϕ ∗ ( G , μ R ) [ t = 1 ∑ T ( 1 − 1 + γ t ) ω ∈ Ω ∑ μ D ( ω ) s ∈ S ∑ π ( t ) ( s ∣ ω ) a ∈ A ∑ ρ ( t ) ( a ∣ s ) u ( a , ω ) ] = T U sp ∗ ( μ R ) − E ( π ( t ) , ρ ( t ) ) ∼ G , ϕ ∗ ( G , μ R ) [ t = 1 ∑ T ω ∈ Ω ∑ μ D ( ω ) s ∈ S ∑ π ( t ) ( s ∣ ω ) a ∈ A ∑ ρ ( t ) ( a ∣ s ) u ( a , ω ) ] − ( T − T γ ) U sp ∗ ( μ R ) + E ( π ( t ) , ρ ( t ) ) ∼ G , ϕ ∗ ( G , μ R ) [ t = 1 ∑ T ( 1 − γ t ) ω ∈ Ω ∑ μ D ( ω ) s ∈ S ∑ π ( t ) ( s ∣ ω ) a ∈ A ∑ ρ ( t ) ( a ∣ s ) u ( a , ω ) ] = Reg ( G ; μ D , μ R ) − E ( π ( t ) , ρ ( t ) ) ∼ G , ϕ ∗ ( G , μ R ) [ t = 1 ∑ T ( 1 − γ t ) ( U sp ∗ ( μ R ) − ω ∈ Ω ∑ μ D ( ω ) s ∈ S ∑ π ( t ) ( s ∣ ω ) a ∈ A ∑ ρ ( t ) ( a ∣ s ) u ( a , ω ) ) ] .
Then, we want to show A ≥ 0 A\ge0 A ≥ 0 . According to the definition of utility (30) (31) in the two-state, two-action single-period game defined previously,
U s p ∗ ( μ R ) − ∑ ω ∈ Ω μ D ( ω ) ∑ s ∈ S π ( t ) ( s ∣ ω ) ∑ a ∈ A ρ ( t ) ( a ∣ s ) u ( a , ω ) = ( 1 − p 0 ) μ ~ R + p 0 − ( 1 − p 0 ) ∑ s ∈ S π ( t ) ( s ∣ ω 0 ) ρ ( t ) ( a 1 ∣ s ) − p 0 ∑ s ∈ S π ( t ) ( s ∣ ω 1 ) ρ ( t ) ( a 1 ∣ s ) = ( 1 − p 0 ) ( μ ~ R − ∑ s ∈ S π ( t ) ( s ∣ ω 0 ) ρ ( t ) ( a 1 ∣ s ) ) + p 0 ( 1 − ∑ s ∈ S π ( t ) ( s ∣ ω 1 ) ρ ( t ) ( a 1 ∣ s ) ) ≥ ( 1 − p 0 ) ( 1 + μ ~ R ) V s p ( π ( t ) , ρ ( t ) ) + 0 , \begin{aligned}
U_{\mathrm{sp}}^*(\mu_R)-\sum_{\omega\in\Omega}\mu_D(\omega)\sum_{s\in S}\pi^{(t)}(s\mid\omega)\sum_{a\in A}\rho^{(t)}(a\mid s)u(a,\omega)
&=(1-p_0)\tilde{\mu}_R+p_0-(1-p_0)\sum_{s\in S}\pi^{(t)}(s\mid\omega_0)\rho^{(t)}(a_1\mid s)-p_0\sum_{s\in S}\pi^{(t)}(s\mid\omega_1)\rho^{(t)}(a_1\mid s)\\
&=(1-p_0)\left(\tilde{\mu}_R-\sum_{s\in S}\pi^{(t)}(s\mid\omega_0)\rho^{(t)}(a_1\mid s)\right)+p_0\left(1-\sum_{s\in S}\pi^{(t)}(s\mid\omega_1)\rho^{(t)}(a_1\mid s)\right)\\
&\ge(1-p_0)(1+\tilde{\mu}_R)V^{\mathrm{sp}}(\pi^{(t)},\rho^{(t)})+0,
\end{aligned} U sp ∗ ( μ R ) − ω ∈ Ω ∑ μ D ( ω ) s ∈ S ∑ π ( t ) ( s ∣ ω ) a ∈ A ∑ ρ ( t ) ( a ∣ s ) u ( a , ω ) = ( 1 − p 0 ) μ ~ R + p 0 − ( 1 − p 0 ) s ∈ S ∑ π ( t ) ( s ∣ ω 0 ) ρ ( t ) ( a 1 ∣ s ) − p 0 s ∈ S ∑ π ( t ) ( s ∣ ω 1 ) ρ ( t ) ( a 1 ∣ s ) = ( 1 − p 0 ) ( μ ~ R − s ∈ S ∑ π ( t ) ( s ∣ ω 0 ) ρ ( t ) ( a 1 ∣ s ) ) + p 0 ( 1 − s ∈ S ∑ π ( t ) ( s ∣ ω 1 ) ρ ( t ) ( a 1 ∣ s ) ) ≥ ( 1 − p 0 ) ( 1 + μ ~ R ) V sp ( π ( t ) , ρ ( t ) ) + 0 ,
where
V s p ( π ( t ) , ρ ( t ) ) = 1 1 + μ ~ R ( μ ~ R ∑ s ∈ S π ( t ) ( s ∣ ω 1 ) ρ ( t ) ( a 1 ∣ s ) − ∑ s ∈ S π ( t ) ( s ∣ ω 0 ) ρ ( t ) ( a 1 ∣ s ) ) V^{\mathrm{sp}}(\pi^{(t)},\rho^{(t)})=\frac{1}{1+\tilde{\mu}_R}\left(\tilde{\mu}_R\sum_{s\in S}\pi^{(t)}(s\mid\omega_1)\rho^{(t)}(a_1\mid s)-\sum_{s\in S}\pi^{(t)}(s\mid\omega_0)\rho^{(t)}(a_1\mid s)\right) V sp ( π ( t ) , ρ ( t ) ) = 1 + μ ~ R 1 ( μ ~ R s ∈ S ∑ π ( t ) ( s ∣ ω 1 ) ρ ( t ) ( a 1 ∣ s ) − s ∈ S ∑ π ( t ) ( s ∣ ω 0 ) ρ ( t ) ( a 1 ∣ s ) )
is the receiver’s single-period utility under signaling scheme π ( t ) \pi^{(t)} π ( t ) and responding strategy ρ ( t ) \rho^{(t)} ρ ( t ) . The following lemma shows that E [ ∑ t = 1 T ( 1 − γ t ) V s p ( π ( t ) , ρ ( t ) ) ] ≥ 0 \mathbb{E}\left[\sum_{t=1}^{T}(1-\gamma^{t})V^{\mathrm{sp}}(\pi^{(t)},\rho^{(t)})\right]\ge0 E [ ∑ t = 1 T ( 1 − γ t ) V sp ( π ( t ) , ρ ( t ) ) ] ≥ 0 . Note that this lemma is not immediate because the receiver’s utility can be negative in some periods. See Appendix § 13 .§ 6 for the proof of this lemma.
Lemma 20. Assume that the receiver’s discount factor sequence satisfies 1 ≥ γ t ≥ 0 1 \ge\gamma^{t} \ge0 1 ≥ γ t ≥ 0 and γ t ≥ γ t + 1 \gamma^{t} \ge\gamma^{t+1} γ t ≥ γ t + 1 , ∀ t \forall t ∀ t . For any learning algorithm G \mathcal{G} G of the information designer, with the receiver best responding by ϕ ∗ ( G , μ R ) \phi^{*}(\mathcal{G},\mu_{R}) ϕ ∗ ( G , μ R ) , we have E [ ∑ t = 1 T ( 1 − γ t ) V s p ( π ( t ) , ρ ( t ) ) ] ≥ 0 \mathbb{E}\left[\sum_{t=1}^{T}(1-\gamma^{t})V^{\mathrm{sp}}(\pi^{(t)},\rho^{(t)})\right]\ge0 E [ ∑ t = 1 T ( 1 − γ t ) V sp ( π ( t ) , ρ ( t ) ) ] ≥ 0 .
So, we obtain
A ≥ ( 1 − p 0 ) ( 1 + μ ~ R ) E [ ∑ t = 1 T ( 1 − γ t ) V s p ( π ( t ) , ρ ( t ) ) ] ≥ 0 , A \ge(1-p_{0})(1+\widetilde{\mu}_{R})\mathbb{E}\left[\sum_{t=1}^{T}(1-\gamma^{t})V^{\mathrm{sp}}(\pi^{(t)},\rho^{(t)})\right]\ge0, A ≥ ( 1 − p 0 ) ( 1 + μ R ) E [ t = 1 ∑ T ( 1 − γ t ) V sp ( π ( t ) , ρ ( t ) ) ] ≥ 0 ,
which implies
Reg ( G ; μ D , μ R ) ≥ T γ Reg s p ( M , μ R ) . \operatorname{Reg}(\mathcal{G};\mu_{D},\mu_{R}) \ge T_{\gamma}\operatorname{Reg}^{\mathrm{sp}}(\mathcal{M},\mu_{R}). Reg ( G ; μ D , μ R ) ≥ T γ Reg sp ( M , μ R ) .
Proof of Lemma 20
In this proof, the expectation is over the stochastic process { ( π ( t ) , ρ ( t ) ) } t = 1 T \{(\pi^{(t)},\rho^{(t)})\}_{t=1}^{T} {( π ( t ) , ρ ( t ) ) } t = 1 T generated by G \mathcal{G} G and ϕ ∗ ( G , μ R ) \phi^{*}(\mathcal{G},\mu_{R}) ϕ ∗ ( G , μ R ) . Let
E [ V s p ( π ( t ) , ρ ( t ) ) ] = E [ 1 1 + μ ~ R ( μ ~ R ∑ s ∈ S π ( t ) ( s ∣ ω 1 ) ρ ( t ) ( a 1 ∣ ω 1 ) − ∑ s ∈ S π ( t ) ( s ∣ ω 0 ) ρ ( t ) ( a 1 ∣ ω 0 ) ) ] \mathbb{E}[V^{\mathrm{sp}}(\pi^{(t)},\rho^{(t)})]=\mathbb{E}\left[\frac{1}{1+\widetilde{\mu}_{R}}\left(\widetilde{\mu}_{R}\sum_{s\in\mathcal{S}}\pi^{(t)}(s\mid\omega_{1})\rho^{(t)}(a_{1}\mid\omega_{1})-\sum_{s\in\mathcal{S}}\pi^{(t)}(s\mid\omega_{0})\rho^{(t)}(a_{1}\mid\omega_{0})\right)\right] E [ V sp ( π ( t ) , ρ ( t ) )] = E [ 1 + μ R 1 ( μ R s ∈ S ∑ π ( t ) ( s ∣ ω 1 ) ρ ( t ) ( a 1 ∣ ω 1 ) − s ∈ S ∑ π ( t ) ( s ∣ ω 0 ) ρ ( t ) ( a 1 ∣ ω 0 ) ) ]
be the receiver’s expected utility at period t t t , where the signaling scheme is π ( t ) \pi^{(t)} π ( t ) and the receiver responds by strategy ρ ( t ) \rho^{(t)} ρ ( t ) . Denote the receiver’s discounted and undiscounted total utility from period t 1 t_{1} t 1 to t 2 t_{2} t 2 by
D V ( t 1 , t 2 ) = ∑ t = t 1 t 2 γ t E [ V s p ( π ( t ) , ρ ( t ) ) ] , U V ( t 1 , t 2 ) = ∑ t = t 1 t 2 E [ V s p ( π ( t ) , ρ ( t ) ) ] . DV(t_{1},t_{2})=\sum_{t=t_{1}}^{t_{2}}\gamma^{t}\mathbb{E}[V^{\mathrm{sp}}(\pi^{(t)},\rho^{(t)})],\qquad UV(t_{1},t_{2})=\sum_{t=t_{1}}^{t_{2}}\mathbb{E}[V^{\mathrm{sp}}(\pi^{(t)},\rho^{(t)})]. D V ( t 1 , t 2 ) = t = t 1 ∑ t 2 γ t E [ V sp ( π ( t ) , ρ ( t ) )] , U V ( t 1 , t 2 ) = t = t 1 ∑ t 2 E [ V sp ( π ( t ) , ρ ( t ) )] .
We will use an induction to prove 0 ≤ D V ( t , T ) ≤ γ t U V ( t , T ) 0\le DV(t,T)\le\gamma^{t}UV(t,T) 0 ≤ D V ( t , T ) ≤ γ t U V ( t , T ) for every t ∈ { 1 , … , T , T + 1 } t\in\{1,\ldots,T,T+1\} t ∈ { 1 , … , T , T + 1 } . The base case where t = T + 1 t=T+1 t = T + 1 trivially holds. Consider any t ≤ T t\le T t ≤ T . First, we have D V ( t , T ) ≥ ∑ t ′ = 1 T γ t ′ V u n i n f o r m e d ≥ 0 DV(t,T)\ge\sum_{t'=1}^{T}\gamma^{t'}V_{\mathrm{uninformed}}\ge0 D V ( t , T ) ≥ ∑ t ′ = 1 T γ t ′ V uninformed ≥ 0 because the receiver is best-responding using ϕ ∗ ( G , μ R ) \phi^{*}(\mathcal{G},\mu_{R}) ϕ ∗ ( G , μ R ) and he can always guarantee the uninformed utility by ignoring the signals. We then note that
D V ( t , T ) = D V ( t , t ) + D V ( t + 1 , T ) = γ t U V ( t , t ) + D V ( t + 1 , T ) ≤ γ t U V ( t , t ) + γ t + 1 U V ( t + 1 , T ) by induction. \begin{aligned}
DV(t,T)&=DV(t,t)+DV(t+1,T)\\
&=\gamma^{t}UV(t,t)+DV(t+1,T)\\
&\le\gamma^{t}UV(t,t)+\gamma^{t+1}UV(t+1,T)\quad\text{by induction.}
\end{aligned} D V ( t , T ) = D V ( t , t ) + D V ( t + 1 , T ) = γ t U V ( t , t ) + D V ( t + 1 , T ) ≤ γ t U V ( t , t ) + γ t + 1 U V ( t + 1 , T ) by induction.
Because γ t ≥ γ t + 1 \gamma^t \ge\gamma^{t+1} γ t ≥ γ t + 1 and U V ( t + 1 , T ) ≥ γ t + 1 D V ( t + 1 , T ) ≥ 0 UV(t+1,T) \ge\gamma^{t+1}DV(t+1,T) \ge0 U V ( t + 1 , T ) ≥ γ t + 1 D V ( t + 1 , T ) ≥ 0 by induction, we have
D V ( t , T ) ≤ γ t U V ( t , t ) + γ t U V ( t + 1 , T ) = γ t U V ( t , T ) , DV(t,T) \le\gamma^t UV(t,t) + \gamma^t UV(t+1,T) = \gamma^t UV(t,T), D V ( t , T ) ≤ γ t U V ( t , t ) + γ t U V ( t + 1 , T ) = γ t U V ( t , T ) ,
which proves the induction.
For t = 1 t=1 t = 1 , γ t ≤ 1 \gamma^t \le1 γ t ≤ 1 , so D V ( 1 , T ) ≤ γ 1 U V ( 1 , T ) ≤ U V ( 1 , T ) DV(1,T) \le\gamma^1 UV(1,T) \le UV(1,T) D V ( 1 , T ) ≤ γ 1 U V ( 1 , T ) ≤ U V ( 1 , T ) , which implies
0 ≤ U V ( 1 , T ) − D V ( 1 , T ) = E [ ∑ t = 1 T ( 1 − γ t ) V s p ( π ( t ) , ρ ( t ) ) ] 0 \le UV(1,T) - DV(1,T) = \mathbb{E}\left[\sum_{t=1}^{T}(1-\gamma^t)V^{\mathrm{sp}}(\pi^{(t)},\rho^{(t)})\right] 0 ≤ U V ( 1 , T ) − D V ( 1 , T ) = E [ t = 1 ∑ T ( 1 − γ t ) V sp ( π ( t ) , ρ ( t ) ) ]
and proves the lemma.