Introduction
Automated mechanism design (AMD) uses computation and data to optimize auction mechanisms when analytical solutions are unavailable. Early AMD approaches formulate mechanism design as large-scale mathematical programs ([9 , 10 ] ), while recent work uses neural networks to learn approximately revenue-optimal mechanisms from valuation samples ([13 , 15 , 35 , 31 ] ). These methods primarily focus on truthful mechanisms, where incentive compatibility makes truthful reporting an equilibrium and therefore removes the need to compute bidders’ strategic responses.
Many widely deployed auctions, however, are non-truthful ([7 , 18 , 17 ] ). Although the revelation principle often permits mechanism designers to restrict their attention to truthful mechanisms without loss, it does not apply in many settings, such as when bidders have type-dependent constraints on their feasible actions [3 ] . Moreover, practical considerations may limit the mechanism to a prescribed family of non-truthful auctions, such as first-price auctions with reserves. Such auction formats are common in practice; for example, first-price auctions are widespread in online advertising due to their simplicity and transparency [20 ] . However, optimizing non-truthful mechanisms is substantially harder: changing the mechanism changes bidders’ equilibrium strategies, so the revenue must be optimized through the equilibrium of an induced multi-player game.
We formulate non-truthful AMD as a multi-follower Stackelberg problem, where the auctioneer is the leader and bidders are strategic followers. This formulation connects mechanism design to bilevel optimization. Existing approaches to non-truthful mechanism optimization either use zeroth-order optimization ([16 ] ) or assume access to a differentiable equilibrium oracle ([33 , 34 ] ). Meanwhile, current fully first-order bilevel methods primarily consider a single-follower setting ([28 , 26 ] ). Unlike the single-follower bilevel problem, however, a multi-follower equilibrium couples the responses of all followers: changing one follower’s strategy affects the incentives of the others, and these cross-player effects are generally asymmetric, so standard first-order bilevel constructions do not extend directly. This leaves open how to develop a scalable fully first-order method for multiple strategic followers.
Main contributions. Our contributions are threefold. First, we formulate non-truthful auction design as a Stackelberg game with N > 1 N > 1 N > 1 followers and develop a fully first-order method for symmetric-Jacobian games, achieving ϵ \epsilon ϵ -stationarity with O ( N ϵ − 2 ) O(N\epsilon^{-2}) O ( N ϵ − 2 ) gradient oracle complexity. We also show why this approach fails in asymmetric games: the true hypergradient requires Jacobian J − T J^{-\mathsf{T}} J − T whereas our method yields J − 1 J^{-1} J − 1 . Second, we resolve this mismatch using a finite-difference potential function that induces an auxiliary game with a local Jacobian approximating J − T J^{-\mathsf{T}} J − T . We prove local strong monotonicity and non-asymptotic hypergradient guarantees for the auxiliary game, provide a fully first-order algorithm that converges to ϵ \epsilon ϵ -stationarity using O ~ ( N 2 ϵ − 2 ) \widetilde{O}(N^2\epsilon^{-2}) O ( N 2 ϵ − 2 ) oracle calls, and extend the analysis to linear inequality-constrained follower games. Lastly, we formulate a neural-network-parameterized auction design problem in which the auctioneer optimizes revenue while bidders respond strategically. We develop a warm-start strategy for tracking differentiable branches of local Nash equilibria, allowing the first-order method to be applied without assuming a globally unique equilibrium. We evaluate the approach on symmetric and asymmetric single-item auctions and multi-item auctions, demonstrating its applicability to equilibrium-dependent mechanism design.
Related Works
Neural Approaches to Learning Truthful Auctions. Classical approaches to revenue-maximizing mechanism design face two obstacles in practice. First, they typically assume the designer knows the distribution of buyers’ values in advance [e.g., 29]—information that is often unavailable. Second, a general closed-form characterization of revenue-optimal multi-item auctions is unknown [e.g., 12, 32]. Automated mechanism design (AMD) aims to circumvent these challenges by optimizing for revenue-maximizing mechanisms using sample valuation profiles and computational search [9 , 10 ] . The first neural approach to AMD is the work of Dütting et al. [13 ] , which represents allocation and payment rules using neural networks and penalizing bidders’ gains from misreporting to approximately enforce dominant-strategy incentive compatibility. More recently, GemNet [35 ] uses bidder-independent menus and a post-training transformation to guarantee incentive compatibility. In contrast, we optimize neural pay-your-bid mechanisms whose revenue depends on bidders’ strategic responses, requiring us to account for how equilibrium strategies change with the mechanism.
Non-truthful Auction Design: Optimizing non-truthful auctions requires accounting for how mechanism changes affect bidders’ equilibrium strategies. Several papers show that gradient dynamics empirically converge to Nash equilibria in auction games, but focus on computing bidders’ strategies rather than optimizing the auction itself [4 , 5 , 24 , 21 ] . More directly related, Feng et al. [16 ] optimize first-price auction reserves via zeroth-order gradient descent. They compute gradient estimates based on bidders’ responsiveness to experimental shocks in reserves, rather than differentiating through a computed equilibrium. Unlike reserve optimization, our framework learns the allocation rule itself, including for multi-item auctions. Meanwhile, [33 ] formulate mechanism design as a bilevel optimization problem with (coarse) correlated equilibria as the lower-level solution. They assume access to a neural network that outputs a differentiable equilibrium for discrete-action games and its gradient with respect to mechanism parameters. In contrast, we develop novel fully first-order bilevel optimization algorithms with convergence guarantees for Stackelberg games with multiple followers, and apply them to auction design. We design auction mechanisms parameterized by neural networks and compute Nash equilibria through gradient dynamics. Scalability is a key limitation of [33 ] ; their models are only trained on two-player games with at most 16 discrete actions for each player. In contrast, our action space is continuous.
Bilevel Optimization: Bilevel optimization (BO) is an optimization problem constrained by the solution of another optimization problem. It is essentially a Stackelberg game with a single follower. A body of work studies fully first-order methods for BO. These works typically assume a strongly convex lower-level problem to ensure a unique lower-level solution and an invertible KKT matrix. For an unconstrained lower-level problem, [28 ] derives an approximate hypergradient using a penalty function and achieves a complexity of O ( ϵ − 1.5 ) O(\epsilon^{-1.5}) O ( ϵ − 1.5 ) in deterministic BO. For linearly constrained lower-level problems, [26 , 37 ] achieves a complexity of O ( ϵ − 3 δ − 1 ) O(\epsilon^{-3}\delta^{-1}) O ( ϵ − 3 δ − 1 ) to converge to a ( ϵ , δ ) (\epsilon,\delta) ( ϵ , δ ) -Goldstein stationary point. To the best of our knowledge, existing first-order algorithms for solving Stackelberg games with multiple followers with implicit differentiation is only applicable when Jacobian is block-diagonal [30 ] .
Non-truthful Auction Design as a Stackelberg Game
Consider a sealed-bid auction of N N N bidders and J J J items. Bidder i i i has a private valuation t i ∈ T i ⊆ R + J t_i \in\mathcal{T}_i \subseteq\mathbb{R}^{J}_{+} t i ∈ T i ⊆ R + J , drawn independently from a distribution F i F_i F i , and submits a bid a i ∈ A i ⊆ R + J a_i \in\mathcal{A}_i \subseteq\mathbb{R}^{J}_{+} a i ∈ A i ⊆ R + J . Let t = ( t 1 , … , t N ) t = (t_1, \ldots, t_N) t = ( t 1 , … , t N ) and a = ( a 1 , … , a N ) a = (a_1, \ldots, a_N) a = ( a 1 , … , a N ) . An auction mechanism is parametrized by x ∈ X ⊆ R p x \in\mathcal{X} \subseteq\mathbb{R}^{p} x ∈ X ⊆ R p and consists of a randomized allocation rule ρ x \rho_x ρ x and a payment rule P x P_x P x . Given the auction x x x , a bid profile a a a , and valuation t i t_i t i for bidder i i i , bidder i i i 's expected utility can be computed by u i ( x , a , t i ) u_i(x, a, t_i) u i ( x , a , t i ) .
For a given mechanism x x x , each bidder i i i chooses a parametrized bidding strategy β y i : T i → A i \beta_{y_i}: \mathcal{T}_i \to\mathcal{A}_i β y i : T i → A i . This strategy maps the bidder's valuation t i t_i t i to a bid in A i \mathcal{A}_i A i , and y i y_i y i is a parameter vector defining the strategy. Unless stated otherwise, y i ∈ Y i ( x ) : = R n i y_i \in\mathcal{Y}_i(x) := \mathbb{R}^{n_i} y i ∈ Y i ( x ) := R n i . Let y : = ( y 1 , ⋯ , y N ) ∈ Y ( x ) : = ∏ i = 1 N Y i ( x ) ⊆ R n y := (y_1, \cdots, y_N) \in\mathcal{Y}(x) := \prod_{i=1}^{N}\mathcal{Y}_i(x) \subseteq\mathbb{R}^{n} y := ( y 1 , ⋯ , y N ) ∈ Y ( x ) := ∏ i = 1 N Y i ( x ) ⊆ R n and write y ¬ i : = ( y j ) j ≠ i y_{\neg i} := (y_j)_{j \ne i} y ¬ i := ( y j ) j = i . The mechanism and bidding strategies induce an ex-ante expected cost for each bidder. We work with costs rather than utilities so that the follower game can be expressed as a minimization problem.
Definition 3.1 (Ex-ante expected cost). Bidder i i i 's ex-ante expected cost is
g i ( x , y i , y ¬ i ) : = − E t 1 , … , t N [ u i ( x , { β y j ( t j ) } j = 1 N , t i ) ] . g_i(x, y_i, y_{\neg i}) := -\mathbb{E}_{t_1,\ldots,t_N}\left[u_i\left(x,\{\beta_{y_j}(t_j)\}_{j=1}^{N},t_i\right)\right]. g i ( x , y i , y ¬ i ) := − E t 1 , … , t N [ u i ( x , { β y j ( t j ) } j = 1 N , t i ) ] .
Thus, minimizing g i g_i g i is equivalent to maximizing bidder i i i 's ex-ante expected utility.
For a fixed mechanism x x x , the bidders therefore play a normal-form game G ( x ) : = ( x , [ N ] , { Y i } i ∈ [ N ] , { g i } i ∈ [ N ] ) \mathcal{G}(x) := (x, [N], \{\mathcal{Y}_i\}_{i \in[N]}, \{g_i\}_{i \in[N]}) G ( x ) := ( x , [ N ] , { Y i } i ∈ [ N ] , { g i } i ∈ [ N ] ) . We assume bidders follow a Nash equilibrium with respect to the ex-ante expected cost functions. A Nash equilibrium is a strategy profile in which no player can reduce their expected cost by unilaterally changing their strategy parameter while holding everyone else's strategies fixed.
Definition 3.2 (Nash equilibrium). We say y ∗ ∈ Y ( x ) y^* \in\mathcal{Y}(x) y ∗ ∈ Y ( x ) is a Nash equilibrium of the game G ( x ) \mathcal{G}(x) G ( x ) if and only if
g i ( x , y ∗ ) ≤ g i ( x , y i , y ¬ i ∗ ) , ∀ i ∈ [ N ] , ∀ y i ∈ Y i ( x ) . g_i(x,y^*) \leq g_i(x,y_i,y^*_{\neg i}),\quad\forall i \in[N],\ \forall y_i \in\mathcal{Y}_i(x). g i ( x , y ∗ ) ≤ g i ( x , y i , y ¬ i ∗ ) , ∀ i ∈ [ N ] , ∀ y i ∈ Y i ( x ) . We denote the set of Nash equilibria as NE ( G ( x ) ) \operatorname{NE}(\mathcal{G}(x)) NE ( G ( x )) .
The auction design problem is to choose the expected payment rule P x P_x P x and allocation rule ρ x \rho_x ρ x that maximize some objective, assuming bidders play Nash equilibrium strategies. We focus on maximizing the auctioneer's expected revenue. Because our framework is written as a minimization problem, let f ( x , y ) f(x,y) f ( x , y ) denote the negative expected revenue under mechanism x x x and bidding-strategy profile y y y . We first consider the setting in which the induced lower-level game has a unique Nash equilibrium y ∗ ( x ) y^*(x) y ∗ ( x ) and formulate the auction design problem as a Stackelberg Game.
min x ∈ X F ( x ) : = f ( x , y ∗ ( x ) ) s.t. y ∗ ( x ) ∈ NE ( G ( x ) ) . (1) \min_{x \in\mathcal{X}} F(x) := f(x,y^*(x)) \quad\text{s.t. } y^*(x) \in\operatorname{NE}(\mathcal{G}(x)). \tag*{(1)} x ∈ X min F ( x ) := f ( x , y ∗ ( x )) s.t. y ∗ ( x ) ∈ NE ( G ( x )) . (1)
§ 6 extends this formulation to auction games with multiple equilibria.
To solve Problem P1, we run gradient descent on F ( x ) F(x) F ( x ) to optimize the parameter x x x . Our goal is to find an ϵ \epsilon ϵ -stationary point, which is defined as any point x ∈ R p x \in\mathbb{R}^{p} x ∈ R p such that ∥ ∇ F ( x ) ∥ ≤ ϵ \|\nabla F(x)\| \leq\epsilon ∥∇ F ( x ) ∥ ≤ ϵ .
Hypergradient and second-order information: Due to the nested structure in Problem P1, computing the gradient of the function F ( x ) F(x) F ( x ) requires differentiating through the Nash equilibrium y ∗ ( x ) y^*(x) y ∗ ( x ) . To show how to compute d y ∗ d x \frac{d y^*}{d x} d x d y ∗ , we define the gradient operator:
Definition 3.3 (gradient operator). V ( x , y ) : = ( ∇ y 1 g 1 ( x , y ) , ⋯ , ∇ y N g N ( x , y ) ) ∈ R n \mathcal{V}(x,y) := (\nabla_{y_1}g_1(x,y), \cdots, \nabla_{y_N}g_N(x,y)) \in\mathbb{R}^{n} V ( x , y ) := ( ∇ y 1 g 1 ( x , y ) , ⋯ , ∇ y N g N ( x , y )) ∈ R n .
We can write the optimality conditions of the Nash equilibrium y ∗ y^* y ∗ using the gradient operator, and apply the Implicit Function Theorem to solve d y ∗ d x \frac{d y^*}{d x} d x d y ∗ :
V ( x , y ∗ ) = 0 (optimality) ⟹ ∂ ∂ x V ( x , y ∗ ) + ∂ ∂ y V ( x , y ∗ ) d d x y ∗ ( x ) = 0 (2) \mathcal{V}(x,y^*) = 0 \ \text{(optimality)} \quad\Longrightarrow\quad\frac{\partial}{\partial x}\mathcal{V}(x,y^*) + \frac{\partial}{\partial y}\mathcal{V}(x,y^*)\frac{d}{d x}y^*(x) = 0 \tag*{(2)} V ( x , y ∗ ) = 0 (optimality) ⟹ ∂ x ∂ V ( x , y ∗ ) + ∂ y ∂ V ( x , y ∗ ) d x d y ∗ ( x ) = 0 (2)
Solving Equation (2) to get d y ∗ d x \frac{d y^*}{d x} d x d y ∗ , we get the following exact hypergradient:
∇ F ( x ) = ∇ x f ( x , y ∗ ( x ) ) + ( d d x y ∗ ( x ) ) T ∇ y f ( x , y ∗ ( x ) ) = ∇ x f ( x , y ∗ ( x ) ) − ( ∂ ∂ x V ( x , y ∗ ) ) T J ( x , y ∗ ) − T ∇ y f ( x , y ∗ ) . (3) \begin{aligned}
\nabla F(x) = \nabla_x f(x,y^*(x)) + \left(\frac{d}{d x}y^*(x)\right)^{\mathsf{T}}\nabla_y f(x,y^*(x)) \\
&= \nabla_x f(x,y^*(x)) - \left(\frac{\partial}{\partial x}\mathcal{V}(x,y^*)\right)^{\mathsf{T}}J(x,y^*)^{-\mathsf{T}}\nabla_y f(x,y^*). \tag*{(3)}
\end{aligned} ∇ F ( x ) = ∇ x f ( x , y ∗ ( x )) + ( d x d y ∗ ( x ) ) T ∇ y f ( x , y ∗ ( x )) = ∇ x f ( x , y ∗ ( x )) − ( ∂ x ∂ V ( x , y ∗ ) ) T J ( x , y ∗ ) − T ∇ y f ( x , y ∗ ) . (3)
where J ( x , y ) : = ∂ ∂ y V ( x , y ) J(x,y) := \frac{\partial}{\partial y}\mathcal{V}(x,y) J ( x , y ) := ∂ y ∂ V ( x , y ) is the Jacobian of the gradient operator.
Equation (3) gives us the exact hypergradient to solve Problem P1. However, the hypergradient involves the Jacobian J ( x , y ∗ ) J(x,y^*) J ( x , y ∗ ) of the gradient operator, which is second-order and can be computationally intractable. The main goal of this work is to construct an approximate hypergradient estimator that bypasses second-order information and solves Problem P1 efficiently.
Definitions and Assumptions: Below we summarize the definitions and assumptions needed for the later hypergradient and convergence analysis for solving Problem P1.
Assumption 3.4 (Lower-level game). We assume the lower-level game G \mathcal{G} G is μ \mu μ -strongly monotone, C C C -smooth and S S S -Hessian smooth. We assume V V V to be
μ \mu μ -strongly monotone: ⟨ V ( x , y ′ ) − V ( x , y ) , y ′ − y ⟩ ≥ μ ∥ y ′ − y ∥ 2 \langle V(x,y')-V(x,y),y'-y\rangle\ge\mu\lVert y'-y\rVert^2 ⟨ V ( x , y ′ ) − V ( x , y ) , y ′ − y ⟩ ≥ μ ∥ y ′ − y ∥ 2 , ∀ y ′ , y ∈ R n \forall y',y\in\mathbb{R}^n ∀ y ′ , y ∈ R n ,
Jointly C C C -Lipschitz: ∥ V ( x ′ , y ′ ) − V ( x , y ) ∥ ≤ C ∥ ( x ′ , y ′ ) − ( x , y ) ∥ \lVert V(x',y')-V(x,y)\rVert\le C\lVert(x',y')-(x,y)\rVert ∥ V ( x ′ , y ′ ) − V ( x , y )∥ ≤ C ∥( x ′ , y ′ ) − ( x , y )∥ , ∀ ( x ′ , y ′ ) , ( x , y ) ∈ X × R n \forall(x',y'),(x,y)\in X\times\mathbb{R}^n ∀ ( x ′ , y ′ ) , ( x , y ) ∈ X × R n ,
Jointly S S S -Hessian Lipschitz: ∥ ∂ ∂ ( x , y ) V ( x ′ , y ′ ) − ∂ ∂ ( x , y ) V ( x , y ) ∥ ≤ S ∥ ( x ′ , y ′ ) − ( x , y ) ∥ \left\lVert\frac{\partial}{\partial(x,y)}V(x',y')-\frac{\partial}{\partial(x,y)}V(x,y)\right\rVert\le S\lVert(x',y')-(x,y)\rVert ∂ ( x , y ) ∂ V ( x ′ , y ′ ) − ∂ ( x , y ) ∂ V ( x , y ) ≤ S ∥( x ′ , y ′ ) − ( x , y )∥ , ∀ ( x ′ , y ′ ) , ( x , y ) ∈ X × R n \forall(x',y'),(x,y)\in X\times\mathbb{R}^n ∀ ( x ′ , y ′ ) , ( x , y ) ∈ X × R n .
The strong monotonicity assumption guarantees a unique lower-level Nash equilibrium [14 ] .
Assumption 3.5 (Upper-level objective). We assume f f f to be
Jointly L f L_f L f -Lipschitz ∥ f ( x ′ , y ′ ) − f ( x , y ) ∥ ≤ L f ∥ ( x ′ , y ′ ) − ( x , y ) ∥ \lVert f(x',y')-f(x,y)\rVert\le L_f\lVert(x',y')-(x,y)\rVert ∥ f ( x ′ , y ′ ) − f ( x , y )∥ ≤ L f ∥( x ′ , y ′ ) − ( x , y )∥ , ∀ ( x ′ , y ′ ) , ( x , y ) ∈ X × R n \forall(x',y'),(x,y)\in X\times\mathbb{R}^n ∀ ( x ′ , y ′ ) , ( x , y ) ∈ X × R n ,
C f C_f C f -gradient Lipschitz ∥ ∇ ( x , y ) f ( x ′ , y ′ ) − ∇ ( x , y ) f ( x , y ) ∥ ≤ C f ∥ ( x ′ , y ′ ) − ( x , y ) ∥ \lVert\nabla_{(x,y)}f(x',y')-\nabla_{(x,y)}f(x,y)\rVert\le C_f\lVert(x',y')-(x,y)\rVert ∥ ∇ ( x , y ) f ( x ′ , y ′ ) − ∇ ( x , y ) f ( x , y )∥ ≤ C f ∥( x ′ , y ′ ) − ( x , y )∥ , ∀ ( x ′ , y ′ ) , ( x , y ) ∈ X × R n \forall(x',y'),(x,y)\in X\times\mathbb{R}^n ∀ ( x ′ , y ′ ) , ( x , y ) ∈ X × R n ,
S f S_f S f -Hessian Lipschitz ∥ ∂ ∂ ( x , y ) ∇ ( x , y ) f ( x ′ , y ′ ) − ∂ ∂ ( x , y ) ∇ ( x , y ) f ( x , y ) ∥ ≤ S f ∥ ( x ′ , y ′ ) − ( x , y ) ∥ \left\lVert\frac{\partial}{\partial(x,y)}\nabla_{(x,y)}f(x',y')-\frac{\partial}{\partial(x,y)}\nabla_{(x,y)}f(x,y)\right\rVert\le S_f\lVert(x',y')-(x,y)\rVert ∂ ( x , y ) ∂ ∇ ( x , y ) f ( x ′ , y ′ ) − ∂ ( x , y ) ∂ ∇ ( x , y ) f ( x , y ) ≤ S f ∥( x ′ , y ′ ) − ( x , y )∥ , ∀ ( x ′ , y ′ ) , ( x , y ) ∈ X × R n \forall(x',y'),(x,y)\in X\times\mathbb{R}^n ∀ ( x ′ , y ′ ) , ( x , y ) ∈ X × R n .
Stackelberg Games with Symmetric Jacobian
We first consider Stackelberg games where the Jacobian J = ∂ ∂ y V ( x , y ∗ ) J=\frac{\partial}{\partial y}V(x,y^*) J = ∂ y ∂ V ( x , y ∗ ) is symmetric, i.e., J = J ⊤ J=J^\top J = J ⊤ . This condition holds in the special case where followers do not interact through their decision variables, or when the pairwise strategic interactions between every follower i i i and j j j are reciprocal. This section proposes a first-order method for solving Stackelberg games with a symmetric Jacobian.
Consider problem P1. Define a perturbed game G δ : = ( x , [ N ] , { Y i } i ∈ [ N ] , { g i + δ f } i ∈ [ N ] ) \mathcal{G}_\delta:= (x,[N],\{Y_i\}_{i\in[N]},\{g_i+\delta f\}_{i\in[N]}) G δ := ( x , [ N ] , { Y i } i ∈ [ N ] , { g i + δ f } i ∈ [ N ] ) , where the followers’ objectives are perturbed by the leader’s objective δ f \delta f δ f with a small number δ > 0 \delta>0 δ > 0 . Let y δ ∗ y_\delta^* y δ ∗ be the new Nash equilibrium for the perturbed game G δ \mathcal{G}_\delta G δ . We construct the following finite difference:
v δ ( x , y δ ∗ ) = 1 δ ∑ i = 1 N ∇ x g i ( x , y δ , i ∗ , y − i ∗ ) − ∇ x g i ( x , y i ∗ , y − i ∗ ) (4) v_\delta(x,y_\delta^*)=\frac{1}{\delta}\sum_{i=1}^{N}\nabla_x g_i(x,y_{\delta,i}^*,y_{-i}^*)-\nabla_x g_i(x,y_i^*,y_{-i}^*) \tag*{(4)} v δ ( x , y δ ∗ ) = δ 1 i = 1 ∑ N ∇ x g i ( x , y δ , i ∗ , y − i ∗ ) − ∇ x g i ( x , y i ∗ , y − i ∗ ) (4)
We first quantify the error of the finite-difference estimator in the idealized setting where the perturbed equilibrium y δ ∗ y_\delta^* y δ ∗ is computed exactly.
Theorem 4.1. If the Jacobian J = ∂ ∂ y V ( x , y ∗ ) J=\frac{\partial}{\partial y}V(x,y^*) J = ∂ y ∂ V ( x , y ∗ ) is symmetric, then:
∥ v δ ( x , y δ ∗ ) + ( ∂ ∂ x V ( x , y ∗ ) ) ⊤ J ( x , y ∗ ) − ⊤ ∇ y f ( x , y ∗ ) ∥ ≤ O ( N δ ) . (5) \left\lVert v_\delta(x,y_\delta^*)+\left(\frac{\partial}{\partial x}V(x,y^*)\right)^\top J(x,y^*)^{-\top}\nabla_y f(x,y^*)\right\rVert\le O(N\delta). \tag*{(5)} v δ ( x , y δ ∗ ) + ( ∂ x ∂ V ( x , y ∗ ) ) ⊤ J ( x , y ∗ ) − ⊤ ∇ y f ( x , y ∗ ) ≤ O ( N δ ) . (5)
Proof sketch. The perturbed equilibrium y δ ∗ y_\delta^* y δ ∗ satisfies V ( x , y δ ∗ ) + δ ∇ y f ( x , y δ ∗ ) = 0 V(x,y_\delta^*)+\delta\nabla_y f(x,y_\delta^*)=0 V ( x , y δ ∗ ) + δ ∇ y f ( x , y δ ∗ ) = 0 . Differentiating with respect to δ \delta δ at 0 0 0 gives lim δ → 0 y δ ∗ − y ∗ δ = d d δ y δ ∗ ∣ δ = 0 = − J ( x , y ∗ ) − 1 ∇ y f ( x , y ∗ ( x ) ) \lim_{\delta\to0}\frac{y_\delta^*-y^*}{\delta}=\left.\frac{\mathrm{d}}{\mathrm{d}\delta}y_\delta^*\right|_{\delta=0}=-J(x,y^*)^{-1}\nabla_y f(x,y^*(x)) lim δ → 0 δ y δ ∗ − y ∗ = d δ d y δ ∗ δ = 0 = − J ( x , y ∗ ) − 1 ∇ y f ( x , y ∗ ( x )) . By the fundamental Theorem of calculus we have
v δ ( x , y δ ∗ ) = ( ∂ V ( x , y ∗ ) ∂ x ) ⊤ y δ ∗ − y ∗ δ + O ( N δ ) v_\delta(x,y_\delta^*)=\left(\frac{\partial V(x,y^*)}{\partial x}\right)^\top\frac{y_\delta^*-y^*}{\delta}+O(N\delta) v δ ( x , y δ ∗ ) = ( ∂ x ∂ V ( x , y ∗ ) ) ⊤ δ y δ ∗ − y ∗ + O ( N δ )
where the remainder term comes from summing up the error ∥ y δ ∗ − y ∗ ∥ = O ( δ ) \lVert y_\delta^*-y^*\rVert=O(\delta) ∥ y δ ∗ − y ∗ ∥ = O ( δ ) for each player. Thus,
v δ ( x , y δ ∗ ) ≃ − ( ∂ ∂ x V ( x , y ∗ ) ) ⊤ J ( x , y ∗ ) − 1 ∇ y f ( x , y ∗ ) + O ( N δ ) (6) v_\delta(x,y_\delta^*)\simeq-\left(\frac{\partial}{\partial x}V(x,y^*)\right)^\top J(x,y^*)^{-1}\nabla_y f(x,y^*)+O(N\delta) \tag*{(6)} v δ ( x , y δ ∗ ) ≃ − ( ∂ x ∂ V ( x , y ∗ ) ) ⊤ J ( x , y ∗ ) − 1 ∇ y f ( x , y ∗ ) + O ( N δ ) (6)
= − ( ∂ ∂ x V ( x , y ∗ ) ) ⊤ J ( x , y ∗ ) − ⊤ ∇ y f ( x , y ∗ ) + O ( N δ ) (7) =-\left(\frac{\partial}{\partial x}V(x,y^*)\right)^\top J(x,y^*)^{-\top}\nabla_y f(x,y^*)+O(N\delta) \tag*{(7)} = − ( ∂ x ∂ V ( x , y ∗ ) ) ⊤ J ( x , y ∗ ) − ⊤ ∇ y f ( x , y ∗ ) + O ( N δ ) (7)
The last equality in Equation (7) is due to the symmetry of J J J . □
This approximation guarantee suggests that we can run inexact gradient descent to obtain a fully first-order approximation of the hypergradient with a non-asymptotic rate. This intuition is formalized in Theorem C.6 . The proof of Theorem 4.1 highlights that the naive finite difference construction fails when J ( x , y ∗ ) ≠ J ( x , y ∗ ) ⊤ J(x,y^*)\ne J(x,y^*)^\top J ( x , y ∗ ) = J ( x , y ∗ ) ⊤ .
Stackelberg Games with Asymmetric Jacobian
As alluded to above, the current perturbed game cannot be used to solve the Stackelberg game without a symmetric Jacobian. We propose a novel approach to resolve the Jacobian mismatch in this section.
Finite difference approximation for unconstrained Stackelberg game
Consider differentiating ⟨ V ( x , y ) , y − y ∗ ⟩ \langle V(x,y),y-y^*\rangle ⟨ V ( x , y ) , y − y ∗ ⟩ with respect to y y y . The product rule gives ∇ y ⟨ V ( x , y ) , y − y ∗ ⟩ − V ( x , y ) = J ( x , y ) ⊤ ( y − y ∗ ) \nabla_y\langle V(x,y),y-y^*\rangle-V(x,y)=J(x,y)^\top(y-y^*) ∇ y ⟨ V ( x , y ) , y − y ∗ ⟩ − V ( x , y ) = J ( x , y ) ⊤ ( y − y ∗ ) . This is exactly the transpose operator missing in (7) . However, differentiating the scalar ⟨ V ( x , y ) , y − y ∗ ⟩ \langle V(x,y),y-y^*\rangle ⟨ V ( x , y ) , y − y ∗ ⟩ through y y y requires second-order derivatives of the follower objectives. Thus, we construct a finite-difference potential whose gradient approximates ∇ y ⟨ V ( x , y ) , y − y ∗ ⟩ \nabla_y\langle V(x,y),y-y^*\rangle ∇ y ⟨ V ( x , y ) , y − y ∗ ⟩ .
Definition 5.1 (Potential function). Given a small number ϵ > 0 \epsilon>0 ϵ > 0 , define
ϕ ϵ ( x , y ) : = 1 ϵ ∑ j = 1 N [ g j ( x , y j + ϵ ( y j − y j ∗ ) , y − j ) − g j ( x , y ) ] . \phi_\epsilon(x,y):=\frac{1}{\epsilon}\sum_{j=1}^{N}\left[g_j(x,y_j+\epsilon(y_j-y_j^*),y_{-j})-g_j(x,y)\right]. ϕ ϵ ( x , y ) := ϵ 1 j = 1 ∑ N [ g j ( x , y j + ϵ ( y j − y j ∗ ) , y − j ) − g j ( x , y ) ] .
By definition of directional derivative, each finite-difference term in ϕ ϵ ( x , y ) \phi_\epsilon(x,y) ϕ ϵ ( x , y ) converges to ⟨ ∇ y i g i ( x , y ) , y i − y i ∗ ⟩ \langle\nabla_{y_i}g_i(x,y),y_i-y_i^*\rangle ⟨ ∇ y i g i ( x , y ) , y i − y i ∗ ⟩ . So ϕ ϵ ( x , y ) \phi_\epsilon(x,y) ϕ ϵ ( x , y ) approaches ⟨ V ( x , y ) , y − y ∗ ⟩ \langle V(x,y),y-y^*\rangle ⟨ V ( x , y ) , y − y ∗ ⟩ and ∇ y ϕ ϵ ( x , y ) \nabla_y\phi_\epsilon(x,y) ∇ y ϕ ϵ ( x , y ) approaches to ∇ y ⟨ V ( x , y ) , y − y ∗ ⟩ \nabla_y\langle V(x,y),y-y^*\rangle ∇ y ⟨ V ( x , y ) , y − y ∗ ⟩ as ϵ → 0 \epsilon\to0 ϵ → 0 . Therefore, ∇ y ϕ ϵ ( x , y ) − V ( x , y ) \nabla_y\phi_\epsilon(x,y)-V(x,y) ∇ y ϕ ϵ ( x , y ) − V ( x , y ) approximates J ( x , y ) ⊤ ( y − y ∗ ) J(x,y)^\top(y-y^*) J ( x , y ) ⊤ ( y − y ∗ ) . Consider an unconstrained perturbed game G ˉ δ : = ( x , [ N ] , { Y i } i ∈ [ N ] , { g ˉ i } i ∈ [ N ] ) \bar{G}_\delta:=(x,[N],\{Y_i\}_{i\in[N]},\{\bar{g}_i\}_{i\in[N]}) G ˉ δ := ( x , [ N ] , { Y i } i ∈ [ N ] , { g ˉ i } i ∈ [ N ] ) with cost function g ˉ i ( x , y ) = ϕ ϵ ( x , y ) − g i ( x , y ) + δ f ( x , y ) \bar{g}_i(x,y)=\phi_\epsilon(x,y)-g_i(x,y)+\delta f(x,y) g ˉ i ( x , y ) = ϕ ϵ ( x , y ) − g i ( x , y ) + δ f ( x , y ) . Denote the gradient operator for the perturbed game G ˉ δ \bar{G}_\delta G ˉ δ as V ˉ ( x , y ) = ( ∇ y 1 g ˉ 1 ( x , y ) , ⋯ , ∇ y N g ˉ N ( x , y ) ) ∈ R n \bar{V}(x,y)=(\nabla_{y_1}\bar{g}_1(x,y),\cdots,\nabla_{y_N}\bar{g}_N(x,y))\in\mathbb{R}^n V ˉ ( x , y ) = ( ∇ y 1 g ˉ 1 ( x , y ) , ⋯ , ∇ y N g ˉ N ( x , y )) ∈ R n . Then we have V ˉ = ∇ y ϕ ϵ − V + δ ∇ y f \bar{V}=\nabla_y\phi_\epsilon-V+\delta\nabla_yf V ˉ = ∇ y ϕ ϵ − V + δ ∇ y f where the term ∇ y ϕ ϵ − V \nabla_y\phi_\epsilon-V ∇ y ϕ ϵ − V is exactly what we need to recover the Jacobian. Now consider the perturbed equilibrium condition
0 = V ˉ ( x , y ˉ δ ∗ ) ≈ J ( x , y ) ⊤ ( y ˉ δ ∗ − y ∗ ) + δ ∇ y f ( x , y ∗ ) ⟹ y ˉ δ ∗ − y ∗ δ ≈ − J ( x , y ) − ⊤ ∇ y f ( x , y ∗ ) 0=\bar{V}(x,\bar{y}_\delta^*)\approx J(x,y)^\top(\bar{y}_\delta^*-y^*)+\delta\nabla_yf(x,y^*)\quad\Longrightarrow\quad\frac{\bar{y}_\delta^*-y^*}{\delta}\approx-J(x,y)^{-\top}\nabla_yf(x,y^*) 0 = V ˉ ( x , y ˉ δ ∗ ) ≈ J ( x , y ) ⊤ ( y ˉ δ ∗ − y ∗ ) + δ ∇ y f ( x , y ∗ ) ⟹ δ y ˉ δ ∗ − y ∗ ≈ − J ( x , y ) − ⊤ ∇ y f ( x , y ∗ )
Thus, the perturbed equilibrium y ˉ δ ∗ \bar{y}_\delta^* y ˉ δ ∗ satisfies y ˉ δ ∗ ≈ y ∗ \bar{y}_\delta^*\approx y^* y ˉ δ ∗ ≈ y ∗ and thus d d δ y ˉ δ ∗ ∣ δ = 0 ≈ − J ( x , y ∗ ) − ⊤ ∇ y f ( x , y ∗ ) \left.\frac{d}{d\delta}\bar{y}_\delta^*\right|_{\delta=0}\approx-J(x,y^*)^{-\top}\nabla_yf(x,y^*) d δ d y ˉ δ ∗ δ = 0 ≈ − J ( x , y ∗ ) − ⊤ ∇ y f ( x , y ∗ ) , as required by the Stackelberg hypergradient. As a consequence, we use the finite difference v δ ( x , y ˉ δ ∗ ) v_\delta(x,\bar{y}_\delta^*) v δ ( x , y ˉ δ ∗ ) induced by the perturbed equilibrium to recover the correct Jacobian:
Even though the construction resolves the Jacobian mismatch, it does not yield a fully first-order algorithm immediately. This is because the auxiliary game operator V ˉ \bar{V} V ˉ does not necessarily inherit strong monotonicity from V V V . Thus the next technical challenge is to prove that the auxiliary game is strongly monotone locally around the neighborhood of the true equilibrium y ∗ y^* y ∗ (Lemma F.1 ), and that the perturbed equilibrium y ˉ δ ∗ \bar{y}_\delta^* y ˉ δ ∗ stays inside this neighborhood (Lemma F.2). The following theorem puts the ideas together and shows that v δ ( x , y ˉ δ ∗ ) v_\delta(x,\bar{y}_\delta^*) v δ ( x , y ˉ δ ∗ ) is a fully first-order approximation with a non-asymptotic rate.
Theorem 5.2 (Non-Asymptotic hypergradient approximation). For 0 < δ < μ 2 C f 0<\delta<\frac{\mu}{2C_f} 0 < δ < 2 C f μ and 0 < R < μ − δ C f S ( N + 1 ) ( 2 + N ) 0<R<\frac{\mu-\delta C_f}{S(N+1)(2+\sqrt{N})} 0 < R < S ( N + 1 ) ( 2 + N ) μ − δ C f and 0 < ϵ < 1 0<\epsilon<1 0 < ϵ < 1 , V ˉ \bar{V} V ˉ is locally μ ˉ \bar{\mu} μ ˉ -strongly monotone and C ˉ \bar{C} C ˉ -smooth on B ‾ R ( y ∗ ) = { y ∈ R n : ∥ y − y ∗ ∥ ≤ R } \overline{B}_R(y^*)=\{y\in\mathbb{R}^n:\|y-y^*\|\leq R\} B R ( y ∗ ) = { y ∈ R n : ∥ y − y ∗ ∥ ≤ R } for some μ ˉ , C ˉ > 0 \bar{\mu},\bar{C}>0 μ ˉ , C ˉ > 0 . Furthermore, if δ < min ( μ 2 C f , μ ˉ R 2 C f ) \delta<\min\left(\frac{\mu}{2C_f},\frac{\bar{\mu}R}{2C_f}\right) δ < min ( 2 C f μ , 2 C f μ ˉ R ) , then we have a fully first-order hypergradient approximation with a non-asymptotic rate:
∥ v δ ( x , y ˉ δ ∗ ) − ∂ ∂ x V ( x , y ∗ ) ⊤ J ( x , y ∗ ) − ⊤ ( − ∇ y f ( x , y ∗ ) ) ∥ ≤ O ( N δ μ ˉ 2 + ϵ ) . \left\|v_\delta(x,\bar{y}_\delta^*)-\frac{\partial}{\partial x}V(x,y^*)^\top J(x,y^*)^{-\top}\left(-\nabla_yf(x,y^*)\right)\right\|\leq O\left(\frac{N\delta}{\bar{\mu}^2}+\epsilon\right). v δ ( x , y ˉ δ ∗ ) − ∂ x ∂ V ( x , y ∗ ) ⊤ J ( x , y ∗ ) − ⊤ ( − ∇ y f ( x , y ∗ ) ) ≤ O ( μ ˉ 2 N δ + ϵ ) .
Proof Sketch. Lemma F.1 shows that the auxiliary operator V ˉ \bar{V} V ˉ is locally μ ˉ \bar{\mu} μ ˉ -strongly monotone and C ˉ \bar{C} C ˉ -smooth on B ‾ R ( y ∗ ) \overline{B}_R(y^*) B R ( y ∗ ) . Under the stated bound on δ \delta δ , Lemma F.2 shows that the map T ( y ) = y − η V ˉ ( x , y ) T(y)=y-\eta\bar{V}(x,y) T ( y ) = y − η V ˉ ( x , y ) is a contraction on B ‾ R ( y ∗ ) \overline{B}_R(y^*) B R ( y ∗ ) . Hence the auxiliary game admits a unique Nash equilibrium y ˉ δ ∗ \bar{y}_\delta^* y ˉ δ ∗ in this neighborhood with ∥ y ˉ δ ∗ − y ∗ ∥ ≤ 2 L f δ μ ˉ \|\bar{y}_\delta^*-y^*\|\leq\frac{2L_f\delta}{\bar{\mu}} ∥ y ˉ δ ∗ − y ∗ ∥ ≤ μ ˉ 2 L f δ . Now we relate the displacement of this auxiliary equilibrium to the exact hypergradient. Let v ϵ ∗ : = ( J ( x , y ∗ ) + ϵ D ( x , y ∗ ) ) − ⊤ ( − ∇ y f ( x , y ∗ ) ) v_\epsilon^*:=(J(x,y^*)+\epsilon D(x,y^*))^{-\top}(-\nabla_yf(x,y^*)) v ϵ ∗ := ( J ( x , y ∗ ) + ϵD ( x , y ∗ ) ) − ⊤ ( − ∇ y f ( x , y ∗ )) and v ∗ : = ( J ( x , y ∗ ) − ⊤ ( − ∇ y f ( x , y ∗ ) ) ) v^*:=(J(x,y^*)^{-\top}(-\nabla_yf(x,y^*))) v ∗ := ( J ( x , y ∗ ) − ⊤ ( − ∇ y f ( x , y ∗ ))) . Lemma F.3 shows that the normalized equilibrium displacement approximates v ϵ ∗ v_\epsilon^* v ϵ ∗ : ∥ y ˉ δ ∗ − y ∗ δ − v ϵ ∗ ∥ ≤ O ( N δ μ ˉ 2 ) \left\|\frac{\bar{y}_\delta^*-y^*}{\delta}-v_\epsilon^*\right\|\leq O\left(\frac{N\delta}{\bar{\mu}^2}\right) δ y ˉ δ ∗ − y ∗ − v ϵ ∗ ≤ O ( μ ˉ 2 N δ ) . Lemma F.4 controls the bias introduced by the ϵ D \epsilon D ϵD regularization: ∥ v ϵ ∗ − v ∗ ∥ ≤ O ( ϵ ) \|v_\epsilon^*-v^*\|\leq O(\epsilon) ∥ v ϵ ∗ − v ∗ ∥ ≤ O ( ϵ ) . It remains to relate the equilibrium displacement to the finite-difference expression. By Taylor expanding v δ ( x , y ˉ δ ∗ ) v_\delta(x,\bar{y}_\delta^*) v δ ( x , y ˉ δ ∗ ) around y ∗ y^* y ∗ , we have v δ ( x , y ˉ δ ∗ ) = ∂ ∂ x V ( x , y ∗ ) ⊤ y ˉ δ ∗ − y ∗ δ + r δ v_\delta(x,\bar{y}_\delta^*)=\frac{\partial}{\partial x}V(x,y^*)^\top\frac{\bar{y}_\delta^*-y^*}{\delta}+r_\delta v δ ( x , y ˉ δ ∗ ) = ∂ x ∂ V ( x , y ∗ ) ⊤ δ y ˉ δ ∗ − y ∗ + r δ where ∥ r δ ∥ ≤ O ( N δ μ ˉ 2 ) \|r_\delta\|\leq O\left(\frac{N\delta}{\bar{\mu}^2}\right) ∥ r δ ∥ ≤ O ( μ ˉ 2 N δ ) . The sum of these three terms gives an upper bound on ∥ v δ ( x , y ˉ δ ∗ ) − ∂ ∂ x V ( x , y ∗ ) ⊤ v ∗ ∥ \|v_\delta(x,\bar{y}_\delta^*)-\frac{\partial}{\partial x}V(x,y^*)^\top v^*\| ∥ v δ ( x , y ˉ δ ∗ ) − ∂ x ∂ V ( x , y ∗ ) ⊤ v ∗ ∥ which yields the result. □ \square □
Algorithm 1 summarizes our fully first-order method for both symmetric and asymmetric Stackelberg games. At each outer iteration, we approximately solve the original follower game and the perturbed game, construct the finite-difference hypergradient estimator, and update the leader variable by inexact gradient descent. We now analyze the algorithm’s convergence.
Theorem 5.3. Consider Problem P1. Under Assumptions 3.4 and 3.5, Algorithm 1 outputs an O ( ϵ ) O(\epsilon) O ( ϵ ) -stationary point of the hyperobjective F F F with O ~ ( N 2 ϵ − 2 ) \widetilde{O}(N^2\epsilon^{-2}) O ( N 2 ϵ − 2 ) oracle calls to f f f and g i g_i g i .
Proof. By the unconstrained version of Theorem B.7, the hyperobjective F F F is smooth. Choose δ ≤ ϵ 2 N \delta\le\frac{\epsilon}{2N} δ ≤ 2 N ϵ and use the potential function ϕ ϵ / 2 \phi_{\epsilon/2} ϕ ϵ /2 . By Lemma D.1, if the approximate lower-level solutions y ^ \hat{y} y ^ , y ^ δ \hat{y}_{\delta} y ^ δ satisfy ∥ y ∗ − y ^ ∥ ≤ δ 2 \lVert y^*-\hat{y}\rVert\le\delta^2 ∥ y ∗ − y ^ ∥ ≤ δ 2 and ∥ y ˉ δ − y ^ δ ∥ ≤ δ 2 \lVert\bar{y}_{\delta}-\hat{y}_{\delta}\rVert\le\delta^2 ∥ y ˉ δ − y ^ δ ∥ ≤ δ 2 , then the inexact hypergradient ∇ ^ F ( x ) = ∇ x f ( x , y ^ ) + v ^ δ ( x , y ^ δ ) \widehat{\nabla}F(x)=\nabla_x f(x,\hat{y})+\hat{v}_{\delta}(x,\hat{y}_{\delta}) ∇ F ( x ) = ∇ x f ( x , y ^ ) + v ^ δ ( x , y ^ δ ) satisfies ∥ ∇ ^ F ( x ) − ∇ F ( x ) ∥ ≤ O ( N δ + ϵ / 2 ) ≤ O ( ϵ ) \lVert\widehat{\nabla}F(x)-\nabla F(x)\rVert\le O(N\delta+\epsilon/2)\le O(\epsilon) ∥ ∇ F ( x ) − ∇ F ( x )∥ ≤ O ( N δ + ϵ /2 ) ≤ O ( ϵ ) . Therefore, by Lemma D.2, inexact gradient descent requires O ( ϵ − 2 ) O(\epsilon^{-2}) O ( ϵ − 2 ) hypergradient evaluations to obtain an O ( ϵ ) O(\epsilon) O ( ϵ ) -stationary point of F F F . It remains to bound the cost of each hypergradient evaluation. Since the potential function is constructed using the approximate equilibrium y ^ \hat{y} y ^ , its gradient operator is also inexact, with an error of O ( ϵ ) O(\epsilon) O ( ϵ ) (Lemma G.6). Moreover, the auxiliary game is only locally strongly monotone and smooth on B ˉ R ( y ∗ ) \bar{B}_{R}(y^*) B ˉ R ( y ∗ ) . We therefore choose δ ≤ min { ϵ / ( 2 N ) , μ ˉ R / ( 4 L f ) , R / 2 } \delta\le\min\{\epsilon/(2N),\bar{\mu}R/(4L_f),\sqrt{R/2}\} δ ≤ min { ϵ / ( 2 N ) , μ ˉ R / ( 4 L f ) , R /2 } , which ensures ∥ y ˉ δ ∗ − y ^ ∥ ≤ ∥ y ˉ δ ∗ − y ∗ ∥ + ∥ y ∗ − y ^ ∥ ≤ R \lVert\bar{y}_{\delta}^*-\hat{y}\rVert\le\lVert\bar{y}_{\delta}^*-y^*\rVert+\lVert y^*-\hat{y}\rVert\le R ∥ y ˉ δ ∗ − y ^ ∥ ≤ ∥ y ˉ δ ∗ − y ∗ ∥ + ∥ y ∗ − y ^ ∥ ≤ R . Hence, gradient descent initialized at y ^ \hat{y} y ^ remains in the local neighborhood and converges linearly to the auxiliary equilibrium. For each call to ∇ ^ F ( x ) \widehat{\nabla}F(x) ∇ F ( x ) , O ( log ( 1 / δ 2 ) ) O(\log(1/\delta^2)) O ( log ( 1/ δ 2 )) lower-level iterations are sufficient to obtain the required δ 2 \delta^2 δ 2 -accurate solutions. Each evaluation of the auxiliary game operator requires gradients of all N N N surrogate costs, and evaluating each surrogate cost requires O ( N ) O(N) O ( N ) calls to the original follower objectives g i g_i g i . Thus, each lower-level iteration uses O ( N 2 ) O(N^2) O ( N 2 ) first-order oracle calls. Combining this with O ( ϵ − 2 ) O(\epsilon^{-2}) O ( ϵ − 2 ) outer iterations and O ( log ( 1 / δ 2 ) ) O(\log(1/\delta^2)) O ( log ( 1/ δ 2 )) lower-level iterations gives O ( N 2 ϵ − 2 log ( 1 / δ 2 ) ) = O ~ ( N 2 ϵ − 2 ) O(N^2\epsilon^{-2}\log(1/\delta^2))=\widetilde{O}(N^2\epsilon^{-2}) O ( N 2 ϵ − 2 log ( 1/ δ 2 )) = O ( N 2 ϵ − 2 ) .
Note that, compared to the symmetric case, there is an extra factor of N N N due to the sum over all N N N players’ objectives in the potential function.
Constrained Stackelberg game
We now extend the framework to follower games with inequality constraints. For each player i i i , let Y i = { y i ∈ R n i : e ( i ) ( x , y i ) : = A ( i ) x + B ( i ) y i + b ( i ) ≤ 0 } Y_i=\{y_i\in\mathbb{R}^{n_i}:e^{(i)}(x,y_i):=A^{(i)}x+B^{(i)}y_i+b^{(i)}\le0\} Y i = { y i ∈ R n i : e ( i ) ( x , y i ) := A ( i ) x + B ( i ) y i + b ( i ) ≤ 0 } . Linear inequality constraints are more challenging because the hyperobjective may be piecewise smooth. However, only the active inequalities affect the local equilibrium sensitivity, so the problem can be reduced locally to a linear equality-constrained Stackelberg game. Appendix H formalizes this reduction. Thus, it suffices to consider the case where Y i Y_i Y i is equality-constrained.
Definition 5.4 (Functional constraints). Each e ( i ) : X × Y i → R m i e^{(i)}:X\times Y_i\to\mathbb{R}^{m_i} e ( i ) : X × Y i → R m i is a function denoting equality constraints for each player. Each e ( i ) e^{(i)} e ( i ) is private to player i i i . Denote λ ∗ = ∑ i = 1 N λ i ∗ \lambda^*=\sum_{i=1}^{N}\lambda_i^* λ ∗ = ∑ i = 1 N λ i ∗ . Denote λ = ( λ 1 , … , λ N ) ∈ R m \lambda=(\lambda_1,\ldots,\lambda_N)\in\mathbb{R}^m λ = ( λ 1 , … , λ N ) ∈ R m as the Lagrange multipliers for constraints of the players. Denote λ ∗ : = ( λ 1 ∗ , … , λ N ∗ ) ∈ R m \lambda^*:=(\lambda_1^*,\ldots,\lambda_N^*)\in\mathbb{R}^m λ ∗ := ( λ 1 ∗ , … , λ N ∗ ) ∈ R m as the Lagrange multipliers satisfying the KKT condition of min y i ∈ Y i ( x ) g i ( x , y i , y − i ∗ ) \min_{y_i\in Y_i(x)}g_i(x,y_i,y_{-i}^*) min y i ∈ Y i ( x ) g i ( x , y i , y − i ∗ ) , ∀ i ∈ [ N ] \forall i\in[N] ∀ i ∈ [ N ] .
In this setting, we use the same potential ϕ ϵ \phi_{\epsilon} ϕ ϵ as Definition 5.1 and define a constrained auxiliary game G ˉ δ \bar{G}_{\delta} G ˉ δ by g ˉ i ( x , y ) = ϕ ϵ ( x , y ) − g i ( x , y ) − ( λ ∗ ) T B y + δ f ( x , y ) \bar{g}_i(x,y)=\phi_{\epsilon}(x,y)-g_i(x,y)-(\lambda^*)^{\mathsf T}By+\delta f(x,y) g ˉ i ( x , y ) = ϕ ϵ ( x , y ) − g i ( x , y ) − ( λ ∗ ) T B y + δ f ( x , y ) . It follows that G ˉ δ \bar{G}_{\delta} G ˉ δ has the same local strong monotonicity modulus μ ˉ \bar{\mu} μ ˉ and local smoothness modulus C ˉ \bar{C} C ˉ on the ball B ˉ R ( y ∗ ) \bar{B}_{R}(y^*) B ˉ R ( y ∗ ) as the unconstrained case. Let y ˉ δ ∗ \bar{y}_{\delta}^* y ˉ δ ∗ be the Nash equilibrium of G ˉ δ \bar{G}_{\delta} G ˉ δ and λ δ ∗ \lambda_{\delta}^* λ δ ∗ be the corresponding Lagrange multiplier. We modify the finite difference to address the constraints as: v δ ( x , y δ ∗ , λ δ ∗ ) = 1 δ ∑ i = 1 N [ ∇ x g i ( x , y δ , i ∗ , y δ , − i ∗ ) − ∇ x g i ( x , y ∗ ) + ∂ ∂ x e ( i ) ( x , y i ∗ ) T ( λ δ ∗ − λ ∗ ) ] v_{\delta}(x,y_{\delta}^*,\lambda_{\delta}^*)=\frac{1}{\delta}\sum_{i=1}^{N}[\nabla_x g_i(x,y_{\delta,i}^*,y_{\delta,-i}^*)-\nabla_x g_i(x,y^*)+\frac{\partial}{\partial x}e^{(i)}(x,y_i^*)^{\mathsf T}(\lambda_{\delta}^*-\lambda^*)] v δ ( x , y δ ∗ , λ δ ∗ ) = δ 1 ∑ i = 1 N [ ∇ x g i ( x , y δ , i ∗ , y δ , − i ∗ ) − ∇ x g i ( x , y ∗ ) + ∂ x ∂ e ( i ) ( x , y i ∗ ) T ( λ δ ∗ − λ ∗ )] .
The constrained analysis follows the same argument as in the unconstrained case after replacing the game Jacobian by the corresponding KKT system. The transpose-aware construction in Section 5.1 therefore yields the same hypergradient approximation error order. The resulting auxiliary game remains locally strongly monotone, so the lower-level equilibrium can be computed to the required accuracy by first-order methods, and the approximate hypergradient can be used in inexact gradient descent. The full approximation and complexity guarantees are deferred to Appendices G and H.
Learning Non-truthful Auctions
In this section, we instantiate the general first-order Stackelberg framework for learning non-truthful auction mechanisms, where the mechanism is parameterized by neural networks, and bidders respond strategically. Figure 1 and Algorithm 1 illustrate our framework.
Figure 1. In forward pass, we compute the Nash equilibrium y ∗ y^{*} y ∗ of the original game and compute the revenue f ( x , y ∗ ) f(x,y^{*}) f ( x , y ∗ ) . a i , j a_{i,j} a i , j is the bidder i i i ’s bid on item j j j . In the backward pass, we solve the perturbed game defined by g ˉ i ( x , y ) = ϕ ϵ ( x , y ) − g i ( x , y ) + δ f ( x , y ) \bar{g}_{i}(x,y)=\phi_{\epsilon}(x,y)-g_{i}(x,y)+\delta f(x,y) g ˉ i ( x , y ) = ϕ ϵ ( x , y ) − g i ( x , y ) + δ f ( x , y ) and compute v δ ( x , y ˉ δ ∗ ) ≈ d d x y ∗ ( x ) ⊤ ∇ y f ( x , y ∗ ) v_{\delta}(x,\bar{y}_{\delta}^{*})\approx\frac{d}{dx}y^{*}(x)^{\top}\nabla_{y}f(x,y^{*}) v δ ( x , y ˉ δ ∗ ) ≈ d x d y ∗ ( x ) ⊤ ∇ y f ( x , y ∗ ) .
Parameterization of Auction Mechanism
An N N N -bidder, J J J -item mechanism is given by an expected payment rule P x : ∏ A i → R N P_x:\prod A_i\to\mathbb{R}^{N} P x : ∏ A i → R N and a randomized allocation function ρ x : ∏ A i → ∏ j = 1 N Δ N + 1 \rho_x:\prod A_i\to\prod_{j=1}^{N}\Delta^{N+1} ρ x : ∏ A i → ∏ j = 1 N Δ N + 1 . Without loss of generality, we assume 0 ≤ t i ≤ ω i , ∀ i ∈ [ N ] 0\le t_i\le\omega_i,\forall i\in[N] 0 ≤ t i ≤ ω i , ∀ i ∈ [ N ] , for some
Algorithm 1 First-Order Algorithm for Solving Stackelberg Game with Multiple Followers
notation: Let S M ( x , y ) : X × R n → Y ( x ) S_{\mathcal{M}}(x,y): X \times\mathbb{R}^{n} \to Y(x) S M ( x , y ) : X × R n → Y ( x ) be a solver that returns a Nash equilibrium of an arbitrary game M ( x ) \mathcal{M}(x) M ( x ) given initial iterate y y y . Define Q δ : = G δ Q_{\delta}:=G_{\delta} Q δ := G δ if ∂ ∂ y V ( x , y ∗ ( x ) ) \frac{\partial}{\partial y}V(x,y^{*}(x)) ∂ y ∂ V ( x , y ∗ ( x )) is symmetric, otherwise define Q δ : = G ˉ δ Q_{\delta}:=\bar{G}_{\delta} Q δ := G ˉ δ .
initialize: Initial upper-level variable x 0 ∈ X x_{0}\in X x 0 ∈ X , initial lower-level variable y 0 ∈ R n y_{0}\in\mathbb{R}^{n} y 0 ∈ R n , finite difference parameter 0 < δ < 1 0<\delta<1 0 < δ < 1 , potential function parameter 0 < ϵ < 1 0<\epsilon<1 0 < ϵ < 1 , learning rate η > 0 \eta>0 η > 0 .
for t = 0 , 1 , 2 , … , T − 1 t=0,1,2,\ldots,T-1 t = 0 , 1 , 2 , … , T − 1 do
Compute Nash equilibrium y ^ = S G ( x t , y t ) \hat{y}=S_{\mathcal{G}}(x_{t},y_{t}) y ^ = S G ( x t , y t ) (and the dual solution λ ^ \hat{\lambda} λ ^ if constrained)
Compute perturbed Nash equilibrium y ^ δ = S Q δ ( x t , y ^ ) \hat{y}_{\delta}=S_{Q_{\delta}}(x_{t},\hat{y}) y ^ δ = S Q δ ( x t , y ^ ) (and the dual solution λ ^ δ \hat{\lambda}_{\delta} λ ^ δ if constrained).
Compute approximate hypergradient ∇ F ^ ( x t ) = ∇ x f ( x t , y ^ ) + v ^ δ ( x , y ^ δ ) \widehat{\nabla F}(x_{t})=\nabla_{x}f(x_{t},\hat{y})+\hat{v}_{\delta}(x,\hat{y}_{\delta}) ∇ F ( x t ) = ∇ x f ( x t , y ^ ) + v ^ δ ( x , y ^ δ ) (use v ^ δ ( x , y ^ δ , λ ^ δ ) \hat{v}_{\delta}(x,\hat{y}_{\delta},\hat{\lambda}_{\delta}) v ^ δ ( x , y ^ δ , λ ^ δ ) if constrained)
y t + 1 = y ^ y_{t+1}=\hat{y} y t + 1 = y ^
x t + 1 = x t − η ∇ F ^ ( x t ) x_{t+1}=x_{t}-\eta\widehat{\nabla F}(x_{t}) x t + 1 = x t − η ∇ F ( x t )
end for
return x t , y t x_{t},y_{t} x t , y t
ω i ∈ R + J \omega_{i}\in\mathbb{R}_{+}^{J} ω i ∈ R + J . Let ρ x , i , j \rho_{x,i,j} ρ x , i , j denote the probability that player i i i receives item j j j . Define the additive expected utility function as u i ( x , y , t i ) = ∑ j = 1 J t i , j ρ x , i , j ( { β y k ( t k ) } k ∈ [ N ] ) − P x , i ( { β y k ( t k ) } k ∈ [ N ] ) u_{i}(x,y,t_{i})=\sum_{j=1}^{J}t_{i,j}\rho_{x,i,j}(\{\beta_{y_{k}}(t_{k})\}_{k\in[N]})-P_{x,i}(\{\beta_{y_{k}}(t_{k})\}_{k\in[N]}) u i ( x , y , t i ) = ∑ j = 1 J t i , j ρ x , i , j ({ β y k ( t k ) } k ∈ [ N ] ) − P x , i ({ β y k ( t k ) } k ∈ [ N ] ) , with the bidding strategies { β y k } k = 1 N \{\beta_{y_{k}}\}_{k=1}^{N} { β y k } k = 1 N being neural networks. We model the mechanism with temperature τ > 0 \tau>0 τ > 0 , neural networks l x : ∏ A i → R J l_{x}:\prod\mathcal{A}_{i}\to\mathbb{R}^{J} l x : ∏ A i → R J , and h x ( i ) : A i → R J h_{x}^{(i)}:\mathcal{A}_{i}\to\mathbb{R}^{J} h x ( i ) : A i → R J .
ρ x , i , j ( a ) = exp ( h x , j ( i ) ( a i ) / τ ) ∑ k = 1 N exp ( h x , j ( k ) ( a k ) / τ ) + exp ( l x , j ( a ) / τ ) , P x , i ( a ) = ∑ j = 1 J ρ x , i , j ( a ) a i , j \rho_{x,i,j}(a)=\frac{\exp(h_{x,j}^{(i)}(a_{i})/\tau)}{\sum_{k=1}^{N}\exp(h_{x,j}^{(k)}(a_{k})/\tau)+\exp(l_{x,j}(a)/\tau)},\qquad P_{x,i}(a)=\sum_{j=1}^{J}\rho_{x,i,j}(a)a_{i,j} ρ x , i , j ( a ) = ∑ k = 1 N exp ( h x , j ( k ) ( a k ) / τ ) + exp ( l x , j ( a ) / τ ) exp ( h x , j ( i ) ( a i ) / τ ) , P x , i ( a ) = j = 1 ∑ J ρ x , i , j ( a ) a i , j
Similar to a first-price auction, bidders pay their own bids if they receive an item. The function l x l_{x} l x controls the probability that no items are allocated.
Tracking a Local Equilibrium Branch
Algorithm 1 is used to find optimal non-truthful auctions. However, Nash equilibria may not be unique or even exist in auction games [6 ] , in which case Problem P1 is ill-defined. When there exists multiple Nash equilibria, we select a differentiable branch of Nash equilibria z ∗ : X → Y ( x ) z^{*}:X\to Y(x) z ∗ : X → Y ( x ) such that z ∗ ( x ) ∈ NE ( G ( x ) ) , ∀ x ∈ X z^{*}(x)\in\operatorname{NE}(\mathcal{G}(x)),\forall x\in X z ∗ ( x ) ∈ NE ( G ( x )) , ∀ x ∈ X , and formulate the following Stackelberg game: min x ∈ X F ( x ) : = f ( x , z ∗ ( x ) ) : = − E t [ ∑ i = 1 N P x , i ( { β z j ∗ ( x ) ( t j ) } j = 1 N ) ] \min_{x\in X}F(x):=f(x,z^{*}(x)):= -\mathbb{E}_{t}\left[\sum_{i=1}^{N}P_{x,i}(\{\beta_{z_{j}^{*}(x)}(t_{j})\}_{j=1}^{N})\right] min x ∈ X F ( x ) := f ( x , z ∗ ( x )) := − E t [ ∑ i = 1 N P x , i ({ β z j ∗ ( x ) ( t j ) } j = 1 N ) ] . When Nash equilibrium does not exist, we may replace NE ( G ) \operatorname{NE}(\mathcal{G}) NE ( G ) with LNE ( G ) \operatorname{LNE}(\mathcal{G}) LNE ( G ) . The problem is how we can select a differentiable branch of local Nash equilibria. We may assume local strong monotonicity around every local Nash equilibrium y ∗ ( x ) y^{*}(x) y ∗ ( x ) . Local Nash equilibria vary smoothly if the initial iterate is accurate and the mechanism parameters vary slightly (the hyperobjective’s learning rate η \eta η is small). To ensure the initial game can be solved accurately, we pretrain the mechanism neural network to the allocation rule of a simultaneous first-price auction. We also initialize the lower-level solution as a parameter y y y that induces a truthful bidding strategy (β y i ( t i ) = t i \beta_{y_i}(t_i) = t_i β y i ( t i ) = t i , ∀ t i ∈ T i \forall t_i \in T_i ∀ t i ∈ T i ). The following theorem formalizes the warm-start strategy for auction design (see the proof in Appendix § 22 ).
Theorem 6.1. Suppose G ( x ) G(x) G ( x ) is locally strongly monotone on a ball B ˉ r ( y ∗ ( x ) ) \bar{B}_{r}(y^*(x)) B ˉ r ( y ∗ ( x )) with radius r > 0 r > 0 r > 0 for all y ∗ ( x ) ∈ L N E ( G ( x ) ) y^*(x) \in LNE(G(x)) y ∗ ( x ) ∈ L N E ( G ( x )) and x ∈ X x \in X x ∈ X . Given initial iterate y y y , let S ( x , y , δ ^ ) : X × R n × R + → Y ( x ) S(x, y, \hat{\delta}) : X \times\mathbb{R}^n \times\mathbb{R}_{+} \to Y(x) S ( x , y , δ ^ ) : X × R n × R + → Y ( x ) be a solver can always return y ^ \hat{y} y ^ such that ∥ y ^ − y ∗ ( x ) ∥ ≤ δ ^ \lVert\hat{y} - y^*(x) \rVert\le\hat{\delta} ∥ y ^ − y ∗ ( x )∥ ≤ δ ^ whenever ∥ y − y ∗ ( x ) ∥ ≤ r \lVert y - y^*(x) \rVert\le r ∥ y − y ∗ ( x )∥ ≤ r and y ∗ ( x ) ∈ L N E ( G ( x ) ) y^*(x) \in LNE(G(x)) y ∗ ( x ) ∈ L N E ( G ( x )) .
In Algorithm 1, suppose ∥ y 0 − y ∗ ( x 0 ) ∥ ≤ r 2 \lVert y_0 - y^*(x_0) \rVert\le\frac{r}{2} ∥ y 0 − y ∗ ( x 0 )∥ ≤ 2 r for some y ∗ ( x 0 ) ∈ L N E ( G ( x 0 ) ) y^*(x_0) \in LNE(G(x_0)) y ∗ ( x 0 ) ∈ L N E ( G ( x 0 )) and the learning rate η \eta η is sufficiently small, then y k = z ∗ ( x k ) y_k = z^*(x_k) y k = z ∗ ( x k ) , ∀ k ∈ [ T ] \forall k \in[T] ∀ k ∈ [ T ] for some continuously differentiable function z ∗ : X → Y ( x ) z^* : X \to Y(x) z ∗ : X → Y ( x ) such that z ∗ ( x ) ∈ L N E ( G ( x ) ) z^*(x) \in LNE(G(x)) z ∗ ( x ) ∈ L N E ( G ( x )) .
Experiments
Experimental settings. We evaluate our method in three two-bidder auction settings. The first is a symmetric single-item auction, with bidders’ values drawn independently and identically (iid) from Uniform[ 0 , 1 ] [0, 1] [ 0 , 1 ] . In this setting, a second-price auction with a reserve of 0.5 is revenue-optimal. A first-price auction with the same reserve achieves the same expected revenue at its symmetric Bayes-Nash equilibrium [29 , 27 ] . The second setting is an asymmetric single-item auction, with values drawn independently from Uniform[ 0 , 4 3 ] [0, \frac{4}{3}] [ 0 , 3 4 ] (strong bidder) and Uniform[ 0 , 4 5 ] [0, \frac{4}{5}] [ 0 , 5 4 ] (weak bidder). Under the optimal auction [29 ] , the item is allocated to the bidder with the highest non-negative virtual value, where the strong and weak bidders’ virtual values are 2 t 1 − 4 3 2t_1 - \frac{4}{3} 2 t 1 − 3 4 and 2 t 2 − 4 5 2t_2 - \frac{4}{5} 2 t 2 − 5 4 , with corresponding reserve values of 0.667 and 0.4, respectively. The third setting has two items and additive bidders, with per-item valuations sampled iid from Uniform[ 0 , 1 ] [0, 1] [ 0 , 1 ] . We use GemNet [35 ] as a learned truthful benchmark; no closed-form revenue-optimal mechanism is known.
At the start of each training run, we sample 10,000 random valuation profiles and reuse this fixed sample to estimate expected revenue throughout training. We replace softmax with max when reporting the expected revenue.
Optimization performance. Figure 2 compares our first-order method with a second-order method (exact hypergradient) and a zeroth-order method [25 ] . Across all three settings, our method converges faster to a higher revenue. Moreover, the second-order method may be more sensitive to bias and noise based on its high variance. Figure 3 evaluates computational scaling by increasing the number of bidders in the symmetric single-item auction. Our first-order method has a significantly lower backward-pass runtime compared to the second-order implementation. This is consistent with the growing computational cost of the linear system required by the second-order method.
Figure 2. Convergence of negative expected revenue. Symmetric single-item auction (left), asymmetric single-item auction (middle), multi-item auction (right). The truthful multi-item auction baseline is GemNet.
Figure 3. Backward time vs number of players in symmetric single-item auction
Comparison with truthful benchmarks. Our goal in these experiments is to assess whether optimizing pay-your-bid mechanisms can achieve revenues close to truthful benchmarks while retaining a simple, transparent payment rule—not to establish a revenue advantage of non-truthfulness itself. Indeed, Table 1 demonstrates that the revenues of our non-truthful auction approximate those of the optimal auctions in single-item settings and GemNet in the multi-item setting. Appendix § 22 and Figure 4 include allocation visualizations that provide qualitative comparisons with these benchmarks.
Table 1. Expected revenue comparison
Figure 4. Equilibrium bidding strategy (left) and allocation rule (middle, right) for symmetric single-item auction. For allocation probability, x-axis and y-axis are the bid of bidder 1 and bidder 2 respectively. As shown in the leftmost plot, the learned auction is non-truthful, yet its revenue matches second-price auction with optimal reserve price of 0.5.
Conclusion
We formulate non-truthful auction design as a Stackelberg game and develop a novel fully first-order algorithm to solve it. Our key contribution is establishing non-asymptotic convergence guarantees even when the Jacobian is asymmetric. Our algorithm discovers non-truthful auction with revenue close to optimal truthful auctions even when the mechanism and bidders are neural networks.
Statement of LLM Use
In this work, we used generative AI tools for assistance with mathematical derivation and method implementation. We have not used generative AI tools for other tasks with required disclosure. Additionally, we used generative AI tools for assistance with debugging and copy-editing to improve readability. We have reviewed all AI-assisted work. We take responsibility for the final content of this work, including text, claims or artifacts produced with the aid of generative AI.
Ethics Statement
Our first order algorithm solves Stackelberg games, which is a widely used model in Economics. We recommend domain-specific validation when our algorithm is deployed in the real world.
Reproducibility Statement
For reproducibility, all assumptions and mathematical proofs are provided in the main text and Appendix with sufficient explanation. Experimental details are listed in Appendix § 22 .
Auxiliary Lemmas
Lemma A.1 (Variational inequality characterization of Nash Equilibrium). Suppose feasible set Y i ( x ) Y_i(x) Y i ( x ) is convex, for all i ∈ [ N ] i \in[N] i ∈ [ N ] . If ( y 1 ∗ , ⋯ , y N ∗ ) ∈ LNE ( G ( x ) ) (y_1^*, \cdots, y_N^*) \in\operatorname{LNE}(\mathcal{G}(x)) ( y 1 ∗ , ⋯ , y N ∗ ) ∈ LNE ( G ( x )) , then
∀ y i ∈ U i : ⟨ ∇ y i g i ( x , y i ∗ , y − i ∗ ) , y i − y i ∗ ⟩ ≥ 0 \forall y_i \in U_i:\ \langle\nabla_{y_i}g_i(x,y_i^*,y_{-i}^*),y_i-y_i^*\rangle\ge0 ∀ y i ∈ U i : ⟨ ∇ y i g i ( x , y i ∗ , y − i ∗ ) , y i − y i ∗ ⟩ ≥ 0 for some neighborhood U i ⊆ Y i U_i \subseteq Y_i U i ⊆ Y i of y i ∗ y_i^* y i ∗ , for all i ∈ [ N ] i \in[N] i ∈ [ N ] . Under Assumption 3.4, each g i g_i g i is convex in y i y_i y i , the reverse direction also holds with ( y 1 ∗ , ⋯ , y N ∗ ) ∈ NE ( G ( x ) ) (y_1^*, \cdots, y_N^*) \in\operatorname{NE}(\mathcal{G}(x)) ( y 1 ∗ , ⋯ , y N ∗ ) ∈ NE ( G ( x )) . Thus, local Nash equilibrium is a necessary and sufficient condition for Nash equilibrium in a strongly monotone and smooth game.
Proof. Let U = ∏ i U i U=\prod_i U_i U = ∏ i U i and Y ( x ) = ∏ i Y i ( x ) Y(x)=\prod_i Y_i(x) Y ( x ) = ∏ i Y i ( x ) . Suppose y ∗ = ( y 1 ∗ , ⋯ , y N ∗ ) ∈ LNE ( G ( x ) ) y^*=(y_1^*,\cdots,y_N^*)\in\operatorname{LNE}(\mathcal{G}(x)) y ∗ = ( y 1 ∗ , ⋯ , y N ∗ ) ∈ LNE ( G ( x )) , then for all i ∈ [ N ] i\in[N] i ∈ [ N ] , y i ∗ ∈ arg min y i ∈ U i g i ( x , y i , y − i ∗ ) y_i^*\in\operatorname*{arg\,min}_{y_i\in U_i}g_i(x,y_i,y_{-i}^*) y i ∗ ∈ arg min y i ∈ U i g i ( x , y i , y − i ∗ ) is a local minimum. Then, ⟨ ∇ y i g i ( x , y i ∗ , y − i ∗ ) , d ⟩ ≥ 0 \langle\nabla_{y_i}g_i(x,y_i^*,y_{-i}^*),d\rangle\ge0 ⟨ ∇ y i g i ( x , y i ∗ , y − i ∗ ) , d ⟩ ≥ 0 for all feasible directions d ∈ R n i d\in\mathbb{R}^{n_i} d ∈ R n i such that y i ∗ + t d ∈ U i y_i^*+td\in U_i y i ∗ + t d ∈ U i for some t > 0 t>0 t > 0 . Since U i ⊆ Y i ( x ) U_i\subseteq Y_i(x) U i ⊆ Y i ( x ) is convex, ⟨ ∇ y i g i ( x , y i ∗ , y − i ∗ ) , y i − y i ∗ ⟩ ≥ 0 \langle\nabla_{y_i}g_i(x,y_i^*,y_{-i}^*),y_i-y_i^*\rangle\ge0 ⟨ ∇ y i g i ( x , y i ∗ , y − i ∗ ) , y i − y i ∗ ⟩ ≥ 0 , for all y i ∈ U i y_i\in U_i y i ∈ U i . Then ⟨ V ( x , y ∗ ) , y − y ∗ ⟩ ≥ 0 \langle V(x,y^*),y-y^*\rangle\ge0 ⟨ V ( x , y ∗ ) , y − y ∗ ⟩ ≥ 0 , for all y ∈ U y\in U y ∈ U .
Suppose ⟨ V ( x , y ∗ ) , y − y ∗ ⟩ ≥ 0 \langle V(x,y^*),y-y^*\rangle\ge0 ⟨ V ( x , y ∗ ) , y − y ∗ ⟩ ≥ 0 , for all y ∈ Y ( x ) y\in Y(x) y ∈ Y ( x ) . Then, for all i ∈ [ N ] i\in[N] i ∈ [ N ] , ⟨ ∇ y i g i ( x , y i ∗ , y − i ∗ ) , y i − y i ∗ ⟩ ≥ 0 \langle\nabla_{y_i}g_i(x,y_i^*,y_{-i}^*),y_i-y_i^*\rangle\ge0 ⟨ ∇ y i g i ( x , y i ∗ , y − i ∗ ) , y i − y i ∗ ⟩ ≥ 0 . Hence, y i ∗ y_i^* y i ∗ is a stationary point. If V V V is monotone (which implies each g i g_i g i is convex in y i y_i y i ), then y i ∗ ∈ arg min y i ∈ Y i ( x ) g i ( x , y i , y − i ∗ ) y_i^*\in\operatorname*{arg\,min}_{y_i\in Y_i(x)}g_i(x,y_i,y_{-i}^*) y i ∗ ∈ arg min y i ∈ Y i ( x ) g i ( x , y i , y − i ∗ ) is global minimum, then y ∗ ∈ NE ( G ( x ) ) ⊆ LNE ( G ( x ) ) y^*\in\operatorname{NE}(\mathcal{G}(x))\subseteq\operatorname{LNE}(\mathcal{G}(x)) y ∗ ∈ NE ( G ( x )) ⊆ LNE ( G ( x )) .
Since feasible set for each player is represented by linear constraints, we provide an alternative characterization of Nash equilibrium using KKT condition,
Corollary A.2 (KKT characterization of Nash Equilibrium). Under Assumption G.1, if ( y 1 ∗ , ⋯ , y N ∗ ) ∈ LNE ( G ( x ) ) (y_1^*,\cdots,y_N^*)\in\operatorname{LNE}(\mathcal{G}(x)) ( y 1 ∗ , ⋯ , y N ∗ ) ∈ LNE ( G ( x )) , then there exists λ ∗ : = ( λ 1 ∗ , ⋯ , λ N ∗ ) ∈ R m \lambda^*:=(\lambda_1^*,\cdots,\lambda_N^*)\in\mathbb{R}^m λ ∗ := ( λ 1 ∗ , ⋯ , λ N ∗ ) ∈ R m such that
( y i ∗ , λ i ∗ ) satisfies the KKT condition of min y i ∈ Y i ( x ) g i ( x , y i , y − i ∗ ) , ∀ i ∈ [ N ] (y_i^*,\lambda_i^*)\text{ satisfies the KKT condition of }\min_{y_i\in Y_i(x)}g_i(x,y_i,y_{-i}^*),\ \forall i\in[N] ( y i ∗ , λ i ∗ ) satisfies the KKT condition of y i ∈ Y i ( x ) min g i ( x , y i , y − i ∗ ) , ∀ i ∈ [ N ] Under Assumption 3.4, each cost function g i g_i g i is convex in y i y_i y i . There exists optimal dual solution λ ∗ : = ( λ 1 ∗ , ⋯ , λ N ∗ ) ∈ R m \lambda^*:=(\lambda_1^*,\cdots,\lambda_N^*)\in\mathbb{R}^m λ ∗ := ( λ 1 ∗ , ⋯ , λ N ∗ ) ∈ R m such that ( y 1 ∗ , ⋯ , y N ∗ ) ∈ NE ( G ) (y_1^*,\cdots,y_N^*)\in\operatorname{NE}(\mathcal{G}) ( y 1 ∗ , ⋯ , y N ∗ ) ∈ NE ( G ) if and only if
( y i ∗ , λ i ∗ ) satisfies the KKT condition of min y i ∈ Y i ( x ) g i ( x , y i , y − i ∗ ) , ∀ i ∈ [ N ] (y_i^*,\lambda_i^*)\text{ satisfies the KKT condition of }\min_{y_i\in Y_i(x)}g_i(x,y_i,y_{-i}^*),\ \forall i\in[N] ( y i ∗ , λ i ∗ ) satisfies the KKT condition of y i ∈ Y i ( x ) min g i ( x , y i , y − i ∗ ) , ∀ i ∈ [ N ]
Lemma A.3 (Fixed point characterization of variational inequality). Let Y ⊆ R n Y\subseteq\mathbb{R}^n Y ⊆ R n be a closed, convex set. Denote the Euclidean projection operator as Π Y ( z ) = arg min y ∈ Y 1 2 ( y − z ) ⊤ ( y − z ) \Pi_Y(z)=\operatorname*{arg\,min}_{y\in Y}\frac{1}{2}(y-z)^\top(y-z) Π Y ( z ) = arg min y ∈ Y 2 1 ( y − z ) ⊤ ( y − z ) . Let V : R n → R n V:\mathbb{R}^n\to\mathbb{R}^n V : R n → R n . Let η > 0 \eta>0 η > 0 be an arbitrary number. y ∗ ∈ Y y^*\in Y y ∗ ∈ Y satisfies ⟨ V ( y ∗ ) , y − y ∗ ⟩ ≥ 0 , ∀ y ∈ Y \langle V(y^*),y-y^*\rangle\ge0,\ \forall y\in Y ⟨ V ( y ∗ ) , y − y ∗ ⟩ ≥ 0 , ∀ y ∈ Y if and only if y ∗ = Π Y ( y ∗ − η V ( y ∗ ) ) y^*=\Pi_Y(y^*-\eta V(y^*)) y ∗ = Π Y ( y ∗ − η V ( y ∗ )) .
Proof. By optimality condition of projection, y ∗ = Π Y ( y ∗ − η V ( y ∗ ) ) y^*=\Pi_Y(y^*-\eta V(y^*)) y ∗ = Π Y ( y ∗ − η V ( y ∗ )) if and only if ∀ y ∈ Y \forall y\in Y ∀ y ∈ Y ,
⟨ y ∗ − ( y ∗ − η V ( y ∗ ) ) , y − y ∗ ⟩ ≥ 0 \langle y^*-(y^*-\eta V(y^*)),y-y^*\rangle\ge0 ⟨ y ∗ − ( y ∗ − η V ( y ∗ )) , y − y ∗ ⟩ ≥ 0
⟨ V ( y ∗ ) , y − y ∗ ⟩ ≥ 0 \langle V(y^*),y-y^*\rangle\ge0 ⟨ V ( y ∗ ) , y − y ∗ ⟩ ≥ 0
Lemma A.4. Let A ∈ R m × n A\in\mathbb{R}^{m\times n} A ∈ R m × n . Let σ max ( A ) = σ 1 ( A ) ≥ σ 2 ( A ) ≥ ⋯ ≥ σ R ( A ) = σ min ( A ) > 0 \sigma_{\max}(A)=\sigma_1(A)\ge\sigma_2(A)\ge\cdots\ge\sigma_R(A)=\sigma_{\min}(A)>0 σ m a x ( A ) = σ 1 ( A ) ≥ σ 2 ( A ) ≥ ⋯ ≥ σ R ( A ) = σ m i n ( A ) > 0 be non-zero singular values of A A A . Then,
∥ A ∥ = max x ≠ 0 ∥ A x ∥ ∥ x ∥ = σ max ( A ) \lVert A\rVert=\max_{x\ne0}\frac{\lVert Ax\rVert}{\lVert x\rVert}=\sigma_{\max}(A) ∥ A ∥ = x = 0 max ∥ x ∥ ∥ A x ∥ = σ m a x ( A ) If A A A is an invertible square matrix or A A A has full column rank, then
min x ≠ 0 ∥ A x ∥ ∥ x ∥ = σ min ( A ) \min_{x\ne0}\frac{\lVert Ax\rVert}{\lVert x\rVert}=\sigma_{\min}(A) x = 0 min ∥ x ∥ ∥ A x ∥ = σ m i n ( A )
Proof. Let A = U Σ V T A = U\Sigma V^{\mathsf{T}} A = U Σ V T be the singular value decomposition of A A A where U ∈ R m × m U \in\mathbb{R}^{m \times m} U ∈ R m × m , V ∈ R n × n V \in\mathbb{R}^{n \times n} V ∈ R n × n and Σ ∈ R m × n \Sigma\in\mathbb{R}^{m \times n} Σ ∈ R m × n . U U U and V V V are orthogonal matrix with orthonormal columns. Σ = diag ( σ 1 , ⋯ , σ R , 0 , ⋯ , 0 ) \Sigma= \operatorname{diag}(\sigma_1,\cdots,\sigma_R,0,\cdots,0) Σ = diag ( σ 1 , ⋯ , σ R , 0 , ⋯ , 0 ) . Let x x x be a unit vector,
∥ A x ∥ 2 = x T A T A x = x T V Σ T U T U Σ V T x = x T V Σ T Σ V T x \lVert Ax\rVert^2 = x^{\mathsf{T}}A^{\mathsf{T}}Ax
= x^{\mathsf{T}}V\Sigma^{\mathsf{T}}U^{\mathsf{T}}U\Sigma V^{\mathsf{T}}x
= x^{\mathsf{T}}V\Sigma^{\mathsf{T}}\Sigma V^{\mathsf{T}}x ∥ A x ∥ 2 = x T A T A x = x T V Σ T U T U Σ V T x = x T V Σ T Σ V T x
Let y = V T x y = V^{\mathsf{T}}x y = V T x , ∥ y ∥ = ∥ x ∥ = 1 \lVert y\rVert= \lVert x\rVert= 1 ∥ y ∥ = ∥ x ∥ = 1 ,
x T A T A x = y T Σ T Σ y ≤ σ max 2 ( A ) x^{\mathsf{T}}A^{\mathsf{T}}Ax = y^{\mathsf{T}}\Sigma^{\mathsf{T}}\Sigma y
\le\sigma_{\max}^{2}(A) x T A T A x = y T Σ T Σ y ≤ σ m a x 2 ( A )
which is an achievable upper bound if we choose x x x to be the corresponding right singular vector. If A A A is an invertible square matrix, then the diagonal Σ \Sigma Σ does not contain any zeros. If A A A has full column rank, then Σ = [ diag ( σ 1 , ⋯ , σ R ) 0 ] \Sigma= \begin{bmatrix}\operatorname{diag}(\sigma_1,\cdots,\sigma_R) \\ 0\end{bmatrix} Σ = [ diag ( σ 1 , ⋯ , σ R ) 0 ] , and the diagonal of Σ T Σ \Sigma^{\mathsf{T}}\Sigma Σ T Σ does not contain any zeros. We can minimize over x x x to get the minimum singular value.
Lemma A.5. If the function V ( y ) V(y) V ( y ) is μ \mu μ -strongly monotone with respect to y ∈ R n y \in\mathbb{R}^{n} y ∈ R n , then ⟨ d , d d y V ( y ) d ⟩ ≥ μ ∥ d ∥ 2 \left\langle d,\frac{d}{dy}V(y)d\right\rangle\ge\mu\lVert d\rVert^2 ⟨ d , d y d V ( y ) d ⟩ ≥ μ ∥ d ∥ 2 , ∀ y ∈ Y \forall y \in Y ∀ y ∈ Y , ∀ d ∈ R n \forall d \in\mathbb{R}^{n} ∀ d ∈ R n . Furthermore, ∥ d d y V ( y ) ∥ ≥ σ min ( d d y V ( y ) ) ≥ μ \left\lVert\frac{d}{dy}V(y)\right\rVert\ge\sigma_{\min}\left(\frac{d}{dy}V(y)\right) \ge\mu d y d V ( y ) ≥ σ m i n ( d y d V ( y ) ) ≥ μ and ∥ d d y V ( y ) − 1 ∥ = σ max ( d d y V ( y ) − 1 ) ≤ 1 μ \left\lVert\frac{d}{dy}V(y)^{-1}\right\rVert= \sigma_{\max}\left(\frac{d}{dy}V(y)^{-1}\right) \le\frac{1}{\mu} d y d V ( y ) − 1 = σ m a x ( d y d V ( y ) − 1 ) ≤ μ 1 .
Proof. Let d ∈ R n d \in\mathbb{R}^{n} d ∈ R n be an arbitrary vector,
⟨ V ( y + t d ) − V ( y ) , t d ⟩ ≥ μ ∥ t d ∥ 2 \left\langle V(y+td)-V(y),td\right\rangle\ge\mu\lVert td\rVert^2 ⟨ V ( y + t d ) − V ( y ) , t d ⟩ ≥ μ ∥ t d ∥ 2
⟨ V ( y + t d ) − V ( y ) , d ⟩ t ≥ μ ∥ d ∥ 2 \frac{\left\langle V(y+td)-V(y),d\right\rangle}{t} \ge\mu\lVert d\rVert^2 t ⟨ V ( y + t d ) − V ( y ) , d ⟩ ≥ μ ∥ d ∥ 2
⟨ d d y V ( y ) d , d ⟩ ≥ μ ∥ d ∥ 2 ( take t → 0 ) \left\langle\frac{d}{dy}V(y)d,d\right\rangle\ge\mu\lVert d\rVert^2 \quad(\text{take }t\to0) ⟨ d y d V ( y ) d , d ⟩ ≥ μ ∥ d ∥ 2 ( take t → 0 )
Thus, d d y V ( y ) \frac{d}{dy}V(y) d y d V ( y ) is invertible and has full rank. The singular value decomposition of d d y V ( y ) \frac{d}{dy}V(y) d y d V ( y ) is given by U Σ V T U\Sigma V^{\mathsf{T}} U Σ V T where U ∈ R n × n U \in\mathbb{R}^{n \times n} U ∈ R n × n , V ∈ R n × n V \in\mathbb{R}^{n \times n} V ∈ R n × n and Σ ∈ R n × n \Sigma\in\mathbb{R}^{n \times n} Σ ∈ R n × n . U U U and V V V are orthogonal matrix. Σ = diag ( σ 1 , ⋯ , σ n ) \Sigma= \operatorname{diag}(\sigma_1,\cdots,\sigma_n) Σ = diag ( σ 1 , ⋯ , σ n ) . Following the proof in lemma Lemma A.4 , we have
min ∥ v ∥ = 1 ∥ d d y V ( y ) v ∥ = σ min ( d d y V ( y ) ) \min_{\lVert v\rVert=1}\left\lVert\frac{d}{dy}V(y)v\right\rVert= \sigma_{\min}\left(\frac{d}{dy}V(y)\right) ∥ v ∥ = 1 min d y d V ( y ) v = σ m i n ( d y d V ( y ) )
Let v ∈ R n v \in\mathbb{R}^{n} v ∈ R n be a unit vector with ∥ v ∥ = 1 \lVert v\rVert= 1 ∥ v ∥ = 1 ,
∥ d d y V ( y ) v ∥ = ∥ d d y V ( y ) v ∥ ∥ v ∥ ≥ ⟨ d d y V ( y ) v , v ⟩ ≥ μ ∥ v ∥ 2 = μ \left\lVert\frac{d}{dy}V(y)v\right\rVert
= \left\lVert\frac{d}{dy}V(y)v\right\rVert\lVert v\rVert
\ge\left\langle\frac{d}{dy}V(y)v,v\right\rangle
\ge\mu\lVert v\rVert^2
= \mu d y d V ( y ) v = d y d V ( y ) v ∥ v ∥ ≥ ⟨ d y d V ( y ) v , v ⟩ ≥ μ ∥ v ∥ 2 = μ
Taking minimum over v v v yields σ min ( d d y V ( y ) ) ≥ μ \sigma_{\min}\left(\frac{d}{dy}V(y)\right) \ge\mu σ m i n ( d y d V ( y ) ) ≥ μ . Inverting d d y V ( y ) \frac{d}{dy}V(y) d y d V ( y ) using its singular value decomposition yields ∥ d d y V ( y ) − 1 ∥ = σ max ( d d y V ( y ) − 1 ) ≤ 1 μ \left\lVert\frac{d}{dy}V(y)^{-1}\right\rVert= \sigma_{\max}\left(\frac{d}{dy}V(y)^{-1}\right) \le\frac{1}{\mu} d y d V ( y ) − 1 = σ m a x ( d y d V ( y ) − 1 ) ≤ μ 1 .
Lemma A.6. If the function V ( y ) V(y) V ( y ) is C C C -Lipschitz with respect to y ∈ R n y \in\mathbb{R}^{n} y ∈ R n , then ∥ d d y V ( y ) ∥ = σ max ( d d y V ( y ) ) ≤ C \left\lVert\frac{d}{dy}V(y)\right\rVert= \sigma_{\max}\left(\frac{d}{dy}V(y)\right) \le C d y d V ( y ) = σ m a x ( d y d V ( y ) ) ≤ C . If d d y V ( y ) − 1 \frac{d}{dy}V(y)^{-1} d y d V ( y ) − 1 exists, then ∥ d d y V ( y ) − 1 ∥ ≥ σ min ( d d y V ( y ) − 1 ) ≥ 1 C \left\lVert\frac{d}{dy}V(y)^{-1}\right\rVert\ge\sigma_{\min}\left(\frac{d}{dy}V(y)^{-1}\right) \ge\frac{1}{C} d y d V ( y ) − 1 ≥ σ m i n ( d y d V ( y ) − 1 ) ≥ C 1 .
Proof. By definition of C C C -Lipschitzness and using Lemma Lemma A.4 , ∥ d d y V ( y ) ∥ = σ max ( d d y V ( y ) ) ≤ C \left\lVert\frac{d}{dy}V(y)\right\rVert= \sigma_{\max}\left(\frac{d}{dy}V(y)\right) \le C d y d V ( y ) = σ m a x ( d y d V ( y ) ) ≤ C . If d d y V ( y ) − 1 \frac{d}{dy}V(y)^{-1} d y d V ( y ) − 1 exists, using singular value decomposition of a full rank matrix, we get σ min ( d d y V ( y ) − 1 ) ≥ 1 C \sigma_{\min}\left(\frac{d}{dy}V(y)^{-1}\right) \ge\frac{1}{C} σ m i n ( d y d V ( y ) − 1 ) ≥ C 1 .
Lemma A.7. Let G G G be a square matrix such that ⟨ d , G d ⟩ ≥ μ ∥ d ∥ 2 \langle d,Gd\rangle\ge\mu\lVert d\rVert^2 ⟨ d , G d ⟩ ≥ μ ∥ d ∥ 2 , ∀ d ∈ R n \forall d \in\mathbb{R}^n ∀ d ∈ R n for some μ > 0 \mu> 0 μ > 0 . Let B B B be a matrix with full row rank, then B G − 1 B T BG^{-1}B^{\mathsf{T}} B G − 1 B T is invertible. If σ max ( G ) ≤ C \sigma_{\max}(G) \le C σ m a x ( G ) ≤ C , then σ min ( B G − 1 B T ) ≥ μ C 2 σ min ( B ) 2 \sigma_{\min}(BG^{-1}B^{\mathsf{T}}) \ge\frac{\mu}{C^2}\sigma_{\min}(B)^2 σ m i n ( B G − 1 B T ) ≥ C 2 μ σ m i n ( B ) 2 and σ max ( B G − 1 B T ) ≤ σ max ( B ) 2 μ \sigma_{\max}(BG^{-1}B^{\mathsf{T}}) \le\frac{\sigma_{\max}(B)^2}{\mu} σ m a x ( B G − 1 B T ) ≤ μ σ m a x ( B ) 2 .
Proof. G G G is invertible. Let 0 ≠ v ∈ R n 0 \ne v \in\mathbb{R}^n 0 = v ∈ R n , d = G − 1 v d=G^{-1}v d = G − 1 v
v T G − 1 v = v T d = d T G T d = d T G d > 0 v^{\mathsf{T}}G^{-1}v=v^{\mathsf{T}}d=d^{\mathsf{T}}G^{\mathsf{T}}d=d^{\mathsf{T}}Gd>0 v T G − 1 v = v T d = d T G T d = d T G d > 0
Let 0 ≠ x ∈ R n 0 \ne x \in\mathbb{R}^n 0 = x ∈ R n , suppose
B G − 1 B T x = 0 BG^{-1}B^{\mathsf{T}}x=0 B G − 1 B T x = 0
x T B G − 1 B T x = 0 x^{\mathsf{T}}BG^{-1}B^{\mathsf{T}}x=0 x T B G − 1 B T x = 0
Since B B B has full row rank and v T G − 1 v > 0 v^{\mathsf{T}}G^{-1}v>0 v T G − 1 v > 0 for any v ≠ 0 v \ne0 v = 0 , this results in a contradiction. Hence, B G − 1 B T BG^{-1}B^{\mathsf{T}} B G − 1 B T is invertible.
σ max ( B G − 1 B T ) ≤ ∥ B ∥ 2 ∥ G − 1 ∥ ≤ ∥ B ∥ 2 μ = σ max ( B ) 2 μ \sigma_{\max}(BG^{-1}B^{\mathsf{T}})\le\lVert B\rVert^2\lVert G^{-1}\rVert
\le\frac{\lVert B\rVert^2}{\mu}
= \frac{\sigma_{\max}(B)^2}{\mu} σ m a x ( B G − 1 B T ) ≤ ∥ B ∥ 2 ∥ G − 1 ∥ ≤ μ ∥ B ∥ 2 = μ σ m a x ( B ) 2
Let x x x be a unit vector. Let v = B T x v=B^{\mathsf{T}}x v = B T x and d = G − 1 v d=G^{-1}v d = G − 1 v ,
∥ B G − 1 B T x ∥ = ∥ B G − 1 B T x ∥ ∥ x ∥ ≥ x T B G − 1 B T x = d T G d ≥ μ ∥ d ∥ 2 = μ ∥ G − 1 B T x ∥ 2 ≥ μ C 2 ∥ B T x ∥ 2 ≥ μ C 2 σ min ( B ) 2 \begin{aligned}
\lVert BG^{-1}B^{\mathsf{T}}x\rVert
&=\lVert BG^{-1}B^{\mathsf{T}}x\rVert\lVert x\rVert\\
&\ge x^{\mathsf{T}}BG^{-1}B^{\mathsf{T}}x \\
&=d^{\mathsf{T}}Gd \\
&\ge\mu\lVert d\rVert^2 \\
&=\mu\lVert G^{-1}B^{\mathsf{T}}x\rVert^2 \\
&\ge\frac{\mu}{C^2}\lVert B^{\mathsf{T}}x\rVert^2 \\
&\ge\frac{\mu}{C^2}\sigma_{\min}(B)^2
\end{aligned} ∥ B G − 1 B T x ∥ = ∥ B G − 1 B T x ∥ ∥ x ∥ ≥ x T B G − 1 B T x = d T G d ≥ μ ∥ d ∥ 2 = μ ∥ G − 1 B T x ∥ 2 ≥ C 2 μ ∥ B T x ∥ 2 ≥ C 2 μ σ m i n ( B ) 2
Minimizing over x x x , we get σ min ( B G − 1 B T ) ≥ μ C 2 σ min ( B ) 2 \sigma_{\min}(BG^{-1}B^{\mathsf{T}})\ge\frac{\mu}{C^2}\sigma_{\min}(B)^2 σ m i n ( B G − 1 B T ) ≥ C 2 μ σ m i n ( B ) 2 .
Lemma A.8. Given a block matrix H = [ G B T B 0 ] H=\begin{bmatrix}G & B^{\mathsf{T}} \\ B & 0\end{bmatrix} H = [ G B B T 0 ] . If B B B has full row rank and ( ∀ d ≠ 0 s.t. B d = 0 ) ( d T G d > 0 ) (\forall d \ne0\ \text{s.t.}\ Bd=0)\ (d^{\mathsf{T}}Gd>0) ( ∀ d = 0 s.t. B d = 0 ) ( d T G d > 0 ) , then H H H is invertible.
Proof. We want to show that H [ a b ] = 0 ⟹ [ a b ] = 0 H\begin{bmatrix}a\\b\end{bmatrix}=0\Longrightarrow\begin{bmatrix}a\\b\end{bmatrix}=0 H [ a b ] = 0 ⟹ [ a b ] = 0 . Suppose H [ a b ] = 0 H\begin{bmatrix}a\\b\end{bmatrix}=0 H [ a b ] = 0 . Suppose B a ≠ 0 Ba\ne0 B a = 0 , then H [ a b ] = 0 H\begin{bmatrix}a\\b\end{bmatrix}=0 H [ a b ] = 0 is impossible by construction. Suppose B a = 0 Ba=0 B a = 0 . If a ≠ 0 a\ne0 a = 0 , a T G a > 0 a^{\mathsf{T}}Ga>0 a T G a > 0 , then [ a b ] H [ a b ] = a T G a + 2 b T B a > 0 \begin{bmatrix}a&b\end{bmatrix}H\begin{bmatrix}a\\b\end{bmatrix}=a^{\mathsf{T}}Ga+2b^{\mathsf{T}}Ba>0 [ a b ] H [ a b ] = a T G a + 2 b T B a > 0 , so H [ a b ] ≠ 0 H\begin{bmatrix}a\\b\end{bmatrix}\ne0 H [ a b ] = 0 which violates the hypothesis. If a = 0 a=0 a = 0 , then H [ a b ] = [ B T b 0 ] = [ 0 0 ] H\begin{bmatrix}a\\b\end{bmatrix}=\begin{bmatrix}B^{\mathsf{T}}b\\0\end{bmatrix}=\begin{bmatrix}0\\0\end{bmatrix} H [ a b ] = [ B T b 0 ] = [ 0 0 ] . Since B T B^{\mathsf{T}} B T has full column rank, b = 0 b=0 b = 0 .
Lipschitzness and Smoothness of Stackelberg Game with Linear Equality Constraints
Lemma B.1 (Implicit differentiation of KKT system).
[ d d x y ∗ ( x ) d d x λ ∗ ( x ) ] = [ ∂ ∂ y V ( x , y ∗ ) B T B 0 ] − 1 [ − ∂ ∂ x V ( x , y ∗ ) − A ] (8) \begin{bmatrix}
\frac{d}{dx}y^*(x) \\
\frac{d}{dx}\lambda^*(x)
\end{bmatrix}
=
\begin{bmatrix}
\frac{\partial}{\partial y}V(x,y^*) & B^{\mathsf{T}} \\
B & 0
\end{bmatrix}^{-1}
\begin{bmatrix}
-\frac{\partial}{\partial x}V(x,y^*) \\
-A \tag*{(8)}
\end{bmatrix} [ d x d y ∗ ( x ) d x d λ ∗ ( x ) ] = [ ∂ y ∂ V ( x , y ∗ ) B B T 0 ] − 1 [ − ∂ x ∂ V ( x , y ∗ ) − A ] (8)
Proof. Consider problem P1. By strong monotonicity assumption 3.4, each cost function g i g_i g i is convex in y i y_i y i . Hence, KKT condition is the necessary and sufficient condition for each y i ∗ y_i^* y i ∗ to be the Nash equilibrium. Denote B : = diag ( ( ∂ ∂ y i e ( i ) ( x , y i ∗ ) ) i ∈ [ N ] ) ∈ R m × n B := \operatorname{diag}\left(\left(\frac{\partial}{\partial y_i}e^{(i)}(x,y_i^*)\right)_{i\in[N]}\right) \in\mathbb{R}^{m\times n} B := diag ( ( ∂ y i ∂ e ( i ) ( x , y i ∗ ) ) i ∈ [ N ] ) ∈ R m × n as a block diagonal matrix, and A : = [ ∂ ∂ x e ( 1 ) ( x , y 1 ∗ ) ⋮ ∂ ∂ x e ( N ) ( x , y N ∗ ) ] ∈ R m × p A := \begin{bmatrix}\frac{\partial}{\partial x}e^{(1)}(x,y_1^*)\\ \vdots\\ \frac{\partial}{\partial x}e^{(N)}(x,y_N^*)\end{bmatrix} \in\mathbb{R}^{m\times p} A := ∂ x ∂ e ( 1 ) ( x , y 1 ∗ ) ⋮ ∂ x ∂ e ( N ) ( x , y N ∗ ) ∈ R m × p . The KKT condition is
Γ ( x , y ∗ , λ ∗ ) = [ ∇ V ( x , y ∗ ) + B T λ ∗ [ e ( 1 ) ( x , y 1 ∗ ) ⋮ e ( N ) ( x , y N ∗ ) ] ] = 0 \Gamma(x,y^*,\lambda^*) =
\begin{bmatrix}
\nabla V(x,y^*) + B^\mathsf{T}\lambda^*\\
\begin{bmatrix}
e^{(1)}(x,y_1^*)\\
\vdots\\
e^{(N)}(x,y_N^*)
\end{bmatrix}
\end{bmatrix}
= 0 Γ ( x , y ∗ , λ ∗ ) = ∇ V ( x , y ∗ ) + B T λ ∗ e ( 1 ) ( x , y 1 ∗ ) ⋮ e ( N ) ( x , y N ∗ ) = 0
By Lemma A.8, the KKT matrix ∂ ∂ ( y ∗ , λ ∗ ) Γ ( x , y ∗ , λ ∗ ) = [ ∂ ∂ y V ( x , y ∗ ) B T B 0 ] \frac{\partial}{\partial(y^*,\lambda^*)}\Gamma(x,y^*,\lambda^*) = \begin{bmatrix}\frac{\partial}{\partial y}V(x,y^*) & B^\mathsf{T}\\ B & 0\end{bmatrix} ∂ ( y ∗ , λ ∗ ) ∂ Γ ( x , y ∗ , λ ∗ ) = [ ∂ y ∂ V ( x , y ∗ ) B B T 0 ] is invertible. Finally, by implicit function theorem, y ∗ ( x ) , λ ∗ ( x ) y^*(x),\lambda^*(x) y ∗ ( x ) , λ ∗ ( x ) are continuously differentiable functions of x x x and their derivatives match the statement of this Lemma. □
Corollary B.2 (Hypergradient).
∇ F ( x ) = ∇ x f ( x , y ∗ ) + [ ∂ ∂ x V ( x , y ∗ ) T A T ] [ ∂ ∂ y V ( x , y ∗ ) B T B 0 ] − T [ − ∇ y f ( x , y ∗ ) 0 ] (9) \nabla F(x) = \nabla_x f(x,y^*) + \begin{bmatrix}\frac{\partial}{\partial x}V(x,y^*)^\mathsf{T} & A^\mathsf{T}\end{bmatrix}
\begin{bmatrix}
\frac{\partial}{\partial y}V(x,y^*) & B^\mathsf{T}\\
B & 0
\end{bmatrix}^{-\mathsf{T}}
\begin{bmatrix}
-\nabla_y f(x,y^*)\\
0
\end{bmatrix} \tag*{(9)} ∇ F ( x ) = ∇ x f ( x , y ∗ ) + [ ∂ x ∂ V ( x , y ∗ ) T A T ] [ ∂ y ∂ V ( x , y ∗ ) B B T 0 ] − T [ − ∇ y f ( x , y ∗ ) 0 ] (9)
Proof.
( d d x y ∗ ( x ) ) T ∇ y f ( x , y ∗ ( x ) ) = [ d d x y ∗ ( x ) T d d x λ ∗ ( x ) T ] [ ∇ y f ( x , y ∗ ) 0 ] = [ ∂ ∂ x V ( x , y ∗ ) T A T ] [ ∂ ∂ y V ( x , y ∗ ) B T B 0 ] − T [ − ∇ y f ( x , y ∗ ) 0 ] \begin{aligned}
\left(\frac{d}{dx}y^*(x)\right)^\mathsf{T}\nabla_y f(x,y^*(x)) \\
&= \begin{bmatrix}\frac{d}{dx}y^*(x)^\mathsf{T} & \frac{d}{dx}\lambda^*(x)^\mathsf{T}\end{bmatrix}
\begin{bmatrix}
\nabla_y f(x,y^*)\\
0
\end{bmatrix} \\
&= \begin{bmatrix}\frac{\partial}{\partial x}V(x,y^*)^\mathsf{T} & A^\mathsf{T}\end{bmatrix}
\begin{bmatrix}
\frac{\partial}{\partial y}V(x,y^*) & B^\mathsf{T}\\
B & 0
\end{bmatrix}^{-\mathsf{T}}
\begin{bmatrix}
-\nabla_y f(x,y^*)\\
0
\end{bmatrix}
\end{aligned} ( d x d y ∗ ( x ) ) T ∇ y f ( x , y ∗ ( x )) = [ d x d y ∗ ( x ) T d x d λ ∗ ( x ) T ] [ ∇ y f ( x , y ∗ ) 0 ] = [ ∂ x ∂ V ( x , y ∗ ) T A T ] [ ∂ y ∂ V ( x , y ∗ ) B B T 0 ] − T [ − ∇ y f ( x , y ∗ ) 0 ]
□
Lemma B.3. Suppose
[ G B T B 0 ] [ d y d λ ] = [ − G x y − A ] (10) \begin{bmatrix}
G & B^\mathsf{T}\\
B & 0
\end{bmatrix}
\begin{bmatrix}
d_y\\
d_\lambda
\end{bmatrix}
=
\begin{bmatrix}
-G_{xy}\\
-A \tag*{(10)}
\end{bmatrix} [ G B B T 0 ] [ d y d λ ] = [ − G x y − A ] (10) where G G G is a square matrix satisfying ⟨ d , G d ⟩ ≥ μ ∥ d ∥ 2 \langle d,Gd\rangle\ge\mu\lVert d\rVert^2 ⟨ d , G d ⟩ ≥ μ ∥ d ∥ 2 , ∀ d ∈ R n \forall d\in\mathbb{R}^n ∀ d ∈ R n for some μ > 0 \mu>0 μ > 0 , and B B B has full row rank, then
d y = ( I − P B G ) B T ( − A ) + P B G ( − G − 1 G x y ) d_y = (I-P_B^G)B^\mathsf{T}(-A) + P_B^G(-G^{-1}G_{xy}) d y = ( I − P B G ) B T ( − A ) + P B G ( − G − 1 G x y ) where P B G : = I − G − 1 B T ( B G − 1 B T ) − 1 B P_B^G := I-G^{-1}B^\mathsf{T}(BG^{-1}B^\mathsf{T})^{-1}B P B G := I − G − 1 B T ( B G − 1 B T ) − 1 B
Proof. Expanding the equations,
d y = − G − 1 B T d λ − G − 1 G x y , (11) d_y=-G^{-1}B^\mathsf{T}d_\lambda-G^{-1}G_{xy}, \tag*{(11)} d y = − G − 1 B T d λ − G − 1 G x y , (11)
B d y = − A . (12) Bd_y=-A. \tag*{(12)} B d y = − A . (12)
Solving for d λ d_\lambda d λ , we have
d λ = ( B G − 1 B T ) − 1 [ A − B G − 1 G x y ] = ( B B T ) − 1 B G ( I − P B G ) ( B T A − G − 1 G x y ) = ( B T ) T G ( I − P B G ) ( B T A − G − 1 G x y ) \begin{aligned}
d_\lambda=(BG^{-1}B^\mathsf{T})^{-1}[A-BG^{-1}G_{xy}] \\
&=(BB^\mathsf{T})^{-1}BG(I-P_B^G)(B^\mathsf{T}A-G^{-1}G_{xy}) \\
&=(B^\mathsf{T})^\mathsf{T}G(I-P_B^G)(B^\mathsf{T}A-G^{-1}G_{xy})
\end{aligned} d λ = ( B G − 1 B T ) − 1 [ A − B G − 1 G x y ] = ( B B T ) − 1 B G ( I − P B G ) ( B T A − G − 1 G x y ) = ( B T ) T G ( I − P B G ) ( B T A − G − 1 G x y )
Solving for d y d_y d y , we have
d y = − G − 1 B T ( B G − 1 B T ) − 1 [ A − B G − 1 G x y ] − G − 1 G x y = ( I − P B G ) B T ( − A ) + P B G ( − G − 1 G x y ) \begin{aligned}
d_y=-G^{-1}B^{\mathsf{T}}(BG^{-1}B^{\mathsf{T}})^{-1}[A-BG^{-1}G_{xy}]-G^{-1}G_{xy} \\
&=(I-P_B^G)B^{\mathsf{T}}(-A)+P_B^G(-G^{-1}G_{xy})
\end{aligned} d y = − G − 1 B T ( B G − 1 B T ) − 1 [ A − B G − 1 G x y ] − G − 1 G x y = ( I − P B G ) B T ( − A ) + P B G ( − G − 1 G x y )
Lemma B.4. Let P B G ( x ) = I − G − 1 B T ( B G − 1 B T ) − 1 B P_B^G(x)=I-G^{-1}B^{\mathsf{T}}(BG^{-1}B^{\mathsf{T}})^{-1}B P B G ( x ) = I − G − 1 B T ( B G − 1 B T ) − 1 B , where square matrix G G G satisfies ⟨ d , G d ⟩ ≥ μ g ∥ d ∥ 2 \langle d,Gd\rangle\ge\mu_g\lVert d\rVert^2 ⟨ d , G d ⟩ ≥ μ g ∥ d ∥ 2 for some μ g > 0 \mu_g>0 μ g > 0 and matrix B B B has full row rank. Given σ max ( G ) ≤ C g \sigma_{\max}(G)\le C_g σ m a x ( G ) ≤ C g for some C g C_g C g , let κ g = C g μ g \kappa_g=\frac{C_g}{\mu_g} κ g = μ g C g , then ∥ P B G ∥ ≤ 1 + σ max ( B ) 2 σ min ( B ) 2 κ g 2 = : M P \lVert P_B^G\rVert\le1+\frac{\sigma_{\max}(B)^2}{\sigma_{\min}(B)^2}\kappa_g^2=:M_P ∥ P B G ∥ ≤ 1 + σ m i n ( B ) 2 σ m a x ( B ) 2 κ g 2 =: M P .
Proof.
∥ P B G ∥ ≤ 1 + ∥ B ∥ 2 μ g 1 σ min ( B G − 1 B T ) \lVert P_B^G\rVert\le1+\frac{\lVert B\rVert^2}{\mu_g}\frac{1}{\sigma_{\min}(BG^{-1}B^{\mathsf{T}})} ∥ P B G ∥ ≤ 1 + μ g ∥ B ∥ 2 σ m i n ( B G − 1 B T ) 1
≤ 1 + ∥ B ∥ 2 μ g C g 2 μ g σ min ( B ) 2 = 1 + σ max ( B ) 2 σ min ( B ) 2 κ g 2 \begin{aligned}
\le1+\frac{\lVert B\rVert^2}{\mu_g}\frac{C_g^2}{\mu_g\sigma_{\min}(B)^2} \\
&=1+\frac{\sigma_{\max}(B)^2}{\sigma_{\min}(B)^2}\kappa_g^2
\end{aligned} ≤ 1 + μ g ∥ B ∥ 2 μ g σ m i n ( B ) 2 C g 2 = 1 + σ m i n ( B ) 2 σ m a x ( B ) 2 κ g 2
The first inequality uses Lemma A.5 and Lemma A.4 . The second inequality uses Lemma A.7. The last equality uses Lemma A.4 .
Lemma B.5. Let P B G : = I − G − 1 B T ( B G − 1 B T ) − 1 B P_B^G:=I-G^{-1}B^{\mathsf{T}}(BG^{-1}B^{\mathsf{T}})^{-1}B P B G := I − G − 1 B T ( B G − 1 B T ) − 1 B , where square matrix G G G satisfies ⟨ d , G d ⟩ ≥ μ g ∥ d ∥ 2 \langle d,Gd\rangle\ge\mu_g\lVert d\rVert^2 ⟨ d , G d ⟩ ≥ μ g ∥ d ∥ 2 for some μ g > 0 \mu_g>0 μ g > 0 and matrix B B B has full row rank. Given σ max ( G ) ≤ C g \sigma_{\max}(G)\le C_g σ m a x ( G ) ≤ C g for some C g C_g C g . Suppose another square matrix G ^ \widehat{G} G also satisfies the above properties with the same constants μ g \mu_g μ g and C g C_g C g . Let κ g = C g μ g \kappa_g=\frac{C_g}{\mu_g} κ g = μ g C g , then ∥ P B G − P B G ^ ∥ ≤ L P ∥ G − G ^ ∥ \lVert P_B^G-P_B^{\widehat{G}}\rVert\le L_P\lVert G-\widehat{G}\rVert ∥ P B G − P B G ∥ ≤ L P ∥ G − G ∥ where L P : = C g 2 σ max ( B ) 2 μ g 3 σ min ( B ) 2 + σ max ( B ) 2 μ g C g 4 ∥ B ∥ 2 μ g 4 σ min ( B ) 4 L_P:=\frac{C_g^2\sigma_{\max}(B)^2}{\mu_g^3\sigma_{\min}(B)^2}+\frac{\sigma_{\max}(B)^2}{\mu_g}\frac{C_g^4\lVert B\rVert^2}{\mu_g^4\sigma_{\min}(B)^4} L P := μ g 3 σ m i n ( B ) 2 C g 2 σ m a x ( B ) 2 + μ g σ m a x ( B ) 2 μ g 4 σ m i n ( B ) 4 C g 4 ∥ B ∥ 2 .
Proof. Denote M = B G − 1 B T M=BG^{-1}B^{\mathsf{T}} M = B G − 1 B T and M ^ = B G ^ − 1 B T \widehat{M}=B\widehat{G}^{-1}B^{\mathsf{T}} M = B G − 1 B T ,
P B G ^ − P B G = G − 1 B T M − 1 B − G ^ − 1 B T M ^ − 1 B = ( G − 1 B T M − 1 B − G ^ − 1 B T M − 1 B ) + ( G ^ − 1 B T M − 1 B − G ^ − 1 B T M ^ − 1 B ) = ( G − 1 − G ^ − 1 ) B T M − 1 B + G ^ − 1 B T ( M − 1 − M ^ − 1 ) B \begin{aligned}
P_B^{\widehat{G}}-P_B^G
=G^{-1}B^{\mathsf{T}}M^{-1}B-\widehat{G}^{-1}B^{\mathsf{T}}\widehat{M}^{-1}B \\
&=(G^{-1}B^{\mathsf{T}}M^{-1}B-\widehat{G}^{-1}B^{\mathsf{T}}M^{-1}B)+(\widehat{G}^{-1}B^{\mathsf{T}}M^{-1}B-\widehat{G}^{-1}B^{\mathsf{T}}\widehat{M}^{-1}B) \\
&=(G^{-1}-\widehat{G}^{-1})B^{\mathsf{T}}M^{-1}B+\widehat{G}^{-1}B^{\mathsf{T}}(M^{-1}-\widehat{M}^{-1})B
\end{aligned} P B G − P B G = G − 1 B T M − 1 B − G − 1 B T M − 1 B = ( G − 1 B T M − 1 B − G − 1 B T M − 1 B ) + ( G − 1 B T M − 1 B − G − 1 B T M − 1 B ) = ( G − 1 − G − 1 ) B T M − 1 B + G − 1 B T ( M − 1 − M − 1 ) B
Now we proceed to bound ∥ G − 1 − G ^ − 1 ∥ \lVert G^{-1}-\widehat{G}^{-1}\rVert ∥ G − 1 − G − 1 ∥ and ∥ M − 1 − M ^ − 1 ∥ \lVert M^{-1}-\widehat{M}^{-1}\rVert ∥ M − 1 − M − 1 ∥ using the identity C − 1 − D − 1 = C − 1 ( D − C ) D − 1 C^{-1}-D^{-1}=C^{-1}(D-C)D^{-1} C − 1 − D − 1 = C − 1 ( D − C ) D − 1 for any invertible matrices C C C and D D D ,
∥ G − 1 − G ^ − 1 ∥ ≤ 1 μ g 2 ∥ G − G ^ ∥ \lVert G^{-1}-\widehat{G}^{-1}\rVert
\le\frac{1}{\mu_g^2}\lVert G-\widehat{G}\rVert ∥ G − 1 − G − 1 ∥ ≤ μ g 2 1 ∥ G − G ∥
Using Lemma A.7, we have σ min ( B G − 1 B T ) ≥ μ g C g 2 σ min ( B ) 2 \sigma_{\min}(BG^{-1}B^{\mathsf{T}})\ge\frac{\mu_g}{C_g^2}\sigma_{\min}(B)^2 σ m i n ( B G − 1 B T ) ≥ C g 2 μ g σ m i n ( B ) 2 and σ max ( B G − 1 B T ) ≤ σ max ( B ) 2 μ g \sigma_{\max}(BG^{-1}B^{\mathsf{T}})\le\frac{\sigma_{\max}(B)^2}{\mu_g} σ m a x ( B G − 1 B T ) ≤ μ g σ m a x ( B ) 2 .
∥ M − 1 − M ^ − 1 ∥ ≤ 1 σ min ( B G − 1 B T ) 2 ∥ M − M ^ ∥ ≤ C g 4 ∥ B ∥ 2 μ g 2 σ min ( B ) 4 ∥ G − 1 − G ^ − 1 ∥ ≤ C g 4 ∥ B ∥ 2 μ g 4 σ min ( B ) 4 ∥ G − G ^ ∥ \begin{aligned}
\lVert M^{-1}-\widehat{M}^{-1}\rVert
\le\frac{1}{\sigma_{\min}(BG^{-1}B^{\mathsf{T}})^2}\lVert M-\widehat{M}\rVert\\
&\le\frac{C_g^4\lVert B\rVert^2}{\mu_g^2\sigma_{\min}(B)^4}\lVert G^{-1}-\widehat{G}^{-1}\rVert\\
&\le\frac{C_g^4\lVert B\rVert^2}{\mu_g^4\sigma_{\min}(B)^4}\lVert G-\widehat{G}\rVert
\end{aligned} ∥ M − 1 − M − 1 ∥ ≤ σ m i n ( B G − 1 B T ) 2 1 ∥ M − M ∥ ≤ μ g 2 σ m i n ( B ) 4 C g 4 ∥ B ∥ 2 ∥ G − 1 − G − 1 ∥ ≤ μ g 4 σ m i n ( B ) 4 C g 4 ∥ B ∥ 2 ∥ G − G ∥
Now we combine the results together,
∥ P B G ^ − P B G ∥ ≤ ∥ G − 1 − G ^ − 1 ∥ ∥ B ∥ 2 ∥ M − 1 ∥ + ∥ G ^ − 1 ∥ ∥ B ∥ 2 ∥ M − 1 − M ^ − 1 ∥ ≤ ( C g 2 σ max ( B ) 2 μ g 3 σ min ( B ) 2 + σ max ( B ) 2 μ g 4 σ min ( B ) 4 C g 4 ∥ B ∥ 2 ) ∥ G − G ^ ∥ \begin{aligned}
\lVert P_{B}^{\widehat{G}}-P_{B}^{G}\rVert
&\le\lVert G^{-1}-\widehat{G}^{-1}\rVert\lVert B\rVert^{2}\lVert M^{-1}\rVert+\lVert\widehat{G}^{-1}\rVert\lVert B\rVert^{2}\lVert M^{-1}-\widehat{M}^{-1}\rVert\\
&\le\left(\frac{C_{g}^{2}\sigma_{\max}(B)^{2}}{\mu_{g}^{3}\sigma_{\min}(B)^{2}}+\frac{\sigma_{\max}(B)^{2}}{\mu_{g}^{4}\sigma_{\min}(B)^{4}}C_{g}^{4}\lVert B\rVert^{2}\right)\lVert G-\widehat{G}\rVert
\end{aligned} ∥ P B G − P B G ∥ ≤ ∥ G − 1 − G − 1 ∥ ∥ B ∥ 2 ∥ M − 1 ∥ + ∥ G − 1 ∥ ∥ B ∥ 2 ∥ M − 1 − M − 1 ∥ ≤ ( μ g 3 σ m i n ( B ) 2 C g 2 σ m a x ( B ) 2 + μ g 4 σ m i n ( B ) 4 σ m a x ( B ) 2 C g 4 ∥ B ∥ 2 ) ∥ G − G ∥
Lemma B.6 (Lipschitzness and smoothness of lower-level solution mappings). Denote κ = C μ \kappa=\frac{C}{\mu} κ = μ C , y ∗ ( x ) y^*(x) y ∗ ( x ) is L y L_y L y -Lipschitz and C y C_y C y -smooth in x x x for some constants L y L_y L y and C y C_y C y . Moreover, λ ∗ ( x ) \lambda^*(x) λ ∗ ( x ) is L λ L_\lambda L λ -Lipschitz and C λ C_\lambda C λ -smooth in x x x for some constants L λ L_\lambda L λ and C λ C_\lambda C λ .
Proof. Denote G = ∂ ∂ y V ( x , y ∗ ( x ) ) G=\frac{\partial}{\partial y}V(x,y^*(x)) G = ∂ y ∂ V ( x , y ∗ ( x )) , G x y = ∂ ∂ x V ( x , y ∗ ( x ) ) G_{xy}=\frac{\partial}{\partial x}V(x,y^*(x)) G x y = ∂ x ∂ V ( x , y ∗ ( x )) . Directly applying Lemma B.3 and Lemma B.4, we can prove Lipschitzness of y ∗ ( x ) y^*(x) y ∗ ( x ) ,
∥ d d x y ∗ ( x ) ∥ = ∥ ( I − P B G ) B † ( − A ) + P B G ( − G − 1 G x y ) ∥ ≤ ∥ I − P B G ∥ ∥ B † A ∥ + ∥ P B G ∥ ∥ G − 1 ∥ ∥ G x y ∥ ≤ ( 1 + M P ) ∥ B † A ∥ + M P C μ = : L y \begin{aligned}
\left\lVert\frac{d}{dx}y^*(x)\right\rVert
&=\left\lVert(I-P_{B}^{G})B^{\dagger}(-A)+P_{B}^{G}(-G^{-1}G_{xy})\right\rVert\\
&\le\lVert I-P_{B}^{G}\rVert\lVert B^{\dagger}A\rVert+\lVert P_{B}^{G}\rVert\lVert G^{-1}\rVert\lVert G_{xy}\rVert\\
&\le(1+M_{P})\lVert B^{\dagger}A\rVert+M_{P}\frac{C}{\mu}\\
&=:L_y
\end{aligned} d x d y ∗ ( x ) = ( I − P B G ) B † ( − A ) + P B G ( − G − 1 G x y ) ≤ ∥ I − P B G ∥ ∥ B † A ∥ + ∥ P B G ∥ ∥ G − 1 ∥ ∥ G x y ∥ ≤ ( 1 + M P ) ∥ B † A ∥ + M P μ C =: L y
Denote y ^ = y ∗ ( x ^ ) \widehat{y}=y^*(\widehat{x}) y = y ∗ ( x ) , G ^ = ∂ ∂ y V ( x ^ , y ^ ) \widehat{G}=\frac{\partial}{\partial y}V(\widehat{x},\widehat{y}) G = ∂ y ∂ V ( x , y ) and G ^ x y = ∂ ∂ x V ( x ^ , y ^ ) \widehat{G}_{xy}=\frac{\partial}{\partial x}V(\widehat{x},\widehat{y}) G x y = ∂ x ∂ V ( x , y ) . We can also prove smoothness of y ∗ ( x ) y^*(x) y ∗ ( x ) ,
∥ d d x y ∗ ( x ^ ) − d d x y ∗ ( x ) ∥ = ∥ ( I − P B G ^ ) B † ( − A ) + P B G ^ ( − G ^ − 1 G ^ x y ) − ( I − P B G ) B † ( − A ) − P B G ( − G − 1 G x y ) ∥ ≤ ∥ ( P B G − P B G ^ ) B † ( − A ) ∥ + ∥ ( P B G ^ − P B G ) ( − G ^ − 1 G ^ x y ) ∥ + ∥ P B G ( G − 1 G x y − G ^ − 1 G ^ x y ) ∥ ≤ L P ∥ G − G ^ ∥ ∥ B † A ∥ + L P ∥ G − G ^ ∥ C μ + M P ∥ G − 1 G x y − G ^ − 1 G ^ x y ∥ \begin{aligned}
\left\lVert\frac{d}{dx}y^*(\widehat{x})-\frac{d}{dx}y^*(x)\right\rVert
&=\left\lVert(I-P_{B}^{\widehat{G}})B^{\dagger}(-A)+P_{B}^{\widehat{G}}(-\widehat{G}^{-1}\widehat{G}_{xy})-(I-P_{B}^{G})B^{\dagger}(-A)-P_{B}^{G}(-G^{-1}G_{xy})\right\rVert\\
&\le\left\lVert(P_{B}^{G}-P_{B}^{\widehat{G}})B^{\dagger}(-A)\right\rVert+\left\lVert(P_{B}^{\widehat{G}}-P_{B}^{G})(-\widehat{G}^{-1}\widehat{G}_{xy})\right\rVert+\left\lVert P_{B}^{G}(G^{-1}G_{xy}-\widehat{G}^{-1}\widehat{G}_{xy})\right\rVert\\
&\le L_{P}\lVert G-\widehat{G}\rVert\lVert B^{\dagger}A\rVert+L_{P}\lVert G-\widehat{G}\rVert\frac{C}{\mu}+M_{P}\lVert G^{-1}G_{xy}-\widehat{G}^{-1}\widehat{G}_{xy}\rVert
\end{aligned} d x d y ∗ ( x ) − d x d y ∗ ( x ) = ( I − P B G ) B † ( − A ) + P B G ( − G − 1 G x y ) − ( I − P B G ) B † ( − A ) − P B G ( − G − 1 G x y ) ≤ ( P B G − P B G ) B † ( − A ) + ( P B G − P B G ) ( − G − 1 G x y ) + P B G ( G − 1 G x y − G − 1 G x y ) ≤ L P ∥ G − G ∥ ∥ B † A ∥ + L P ∥ G − G ∥ μ C + M P ∥ G − 1 G x y − G − 1 G x y ∥
where Lemma B.5 and B.4 are applied in the last inequality. Now we can bound ∥ G − G ^ ∥ \lVert G-\widehat{G}\rVert ∥ G − G ∥ . Denote L G : = S ( 1 + L y ) L_G:=S(1+L_y) L G := S ( 1 + L y ) , we have
∥ G − G ^ ∥ ≤ ∥ ∂ ∂ y V ( x , y ) − ∂ ∂ y V ( x ^ , y ) ∥ + ∥ ∂ ∂ y V ( x ^ , y ) − ∂ ∂ y V ( x ^ , y ^ ) ∥ ≤ S ( 1 + L y ) ∥ x − x ^ ∥ = L G ∥ x − x ^ ∥ \begin{aligned}
\lVert G-\widehat{G}\rVert
&\le\left\lVert\frac{\partial}{\partial y}V(x,y)-\frac{\partial}{\partial y}V(\widehat{x},y)\right\rVert+\left\lVert\frac{\partial}{\partial y}V(\widehat{x},y)-\frac{\partial}{\partial y}V(\widehat{x},\widehat{y})\right\rVert\\
&\le S(1+L_y)\lVert x-\widehat{x}\rVert\\
&=L_G\lVert x-\widehat{x}\rVert
\end{aligned} ∥ G − G ∥ ≤ ∂ y ∂ V ( x , y ) − ∂ y ∂ V ( x , y ) + ∂ y ∂ V ( x , y ) − ∂ y ∂ V ( x , y ) ≤ S ( 1 + L y ) ∥ x − x ∥ = L G ∥ x − x ∥
and we can bound the difference between unconstrained implicit gradients ∥ G − 1 G x y − G ^ − 1 G ^ x y ∥ \lVert G^{-1}G_{xy}-\widehat{G}^{-1}\widehat{G}_{xy}\rVert ∥ G − 1 G x y − G − 1 G x y ∥ , denote L u : = S ( 1 + L y ) μ + C S ( 1 + L y ) μ 2 L_u:=\frac{S(1+L_y)}{\mu}+\frac{CS(1+L_y)}{\mu^2} L u := μ S ( 1 + L y ) + μ 2 C S ( 1 + L y ) , we have
∥ G − 1 G x y − G ^ − 1 G ^ x y ∥ ≤ ∥ G − 1 G x y − G − 1 G ^ x y ∥ + ∥ G − 1 G ^ x y − G ^ − 1 G ^ x y ∥ ≤ 1 μ ∥ G x y − G ^ x y ∥ + ∥ G ^ x y ∥ ∥ G − 1 ( G ^ − G ) G ^ − 1 ∥ ≤ ( S ( 1 + L y ) μ + C S ( 1 + L y ) μ 2 ) ∥ x − x ^ ∥ = L u ∥ x − x ^ ∥ \begin{aligned}
\lVert G^{-1}G_{xy}-\widehat{G}^{-1}\widehat{G}_{xy}\rVert
&\le\lVert G^{-1}G_{xy}-G^{-1}\widehat{G}_{xy}\rVert+\lVert G^{-1}\widehat{G}_{xy}-\widehat{G}^{-1}\widehat{G}_{xy}\rVert\\
&\le\frac{1}{\mu}\lVert G_{xy}-\widehat{G}_{xy}\rVert+\lVert\widehat{G}_{xy}\rVert\lVert G^{-1}(\widehat{G}-G)\widehat{G}^{-1}\rVert\\
&\le\left(\frac{S(1+L_y)}{\mu}+\frac{CS(1+L_y)}{\mu^2}\right)\lVert x-\widehat{x}\rVert\\
&=L_u\lVert x-\widehat{x}\rVert
\end{aligned} ∥ G − 1 G x y − G − 1 G x y ∥ ≤ ∥ G − 1 G x y − G − 1 G x y ∥ + ∥ G − 1 G x y − G − 1 G x y ∥ ≤ μ 1 ∥ G x y − G x y ∥ + ∥ G x y ∥ ∥ G − 1 ( G − G ) G − 1 ∥ ≤ ( μ S ( 1 + L y ) + μ 2 C S ( 1 + L y ) ) ∥ x − x ∥ = L u ∥ x − x ∥
We have obtained the smoothness bound for y ∗ ( x ) y^*(x) y ∗ ( x ) ,
∥ d d x y ∗ ( x ) − d d x y ∗ ( x ^ ) ∥ ≤ ( L P L G ∥ B † A ∥ + C L P L G μ + M P ( L u ) ) ∥ x − x ^ ∥ : = C y ∥ x − x ^ ∥ \left\|\frac{d}{dx}y^*(x)-\frac{d}{dx}y^*(\hat{x})\right\|
\leq\left(L_P L_G\left\|B^\dagger A\right\|+\frac{C L_P L_G}{\mu}+M_P(L_u)\right)\left\|x-\hat{x}\right\|
:=C_y\left\|x-\hat{x}\right\| d x d y ∗ ( x ) − d x d y ∗ ( x ^ ) ≤ ( L P L G B † A + μ C L P L G + M P ( L u ) ) ∥ x − x ^ ∥ := C y ∥ x − x ^ ∥
Directly applying Lemma B.3 and Lemma B.4, we can prove Lipschitzness of λ ∗ ( x ) \lambda^*(x) λ ∗ ( x ) ,
∥ d d x λ ∗ ( x ) ∥ = ∥ ( B † ) ⊤ G ( I − P B G ) ( B † A − G − 1 G x y ) ∥ ≤ ∥ B † ∥ ∥ G ∥ ( 1 + M P ) ( ∥ B † A ∥ + C μ ) : = L λ \left\|\frac{d}{dx}\lambda^*(x)\right\|=\left\|(B^\dagger)^\top G(I-P_B^G)(B^\dagger A-G^{-1}G_{xy})\right\|
\leq\left\|B^\dagger\right\|\left\|G\right\|(1+M_P)\left(\left\|B^\dagger A\right\|+\frac{C}{\mu}\right)
:=L_\lambda d x d λ ∗ ( x ) = ( B † ) ⊤ G ( I − P B G ) ( B † A − G − 1 G x y ) ≤ B † ∥ G ∥ ( 1 + M P ) ( B † A + μ C ) := L λ
The smoothness of λ ∗ ( x ) \lambda^*(x) λ ∗ ( x ) is given by
∥ d d x λ ∗ ( x ) − d d x λ ∗ ( x ^ ) ∥ = ∥ ( B † ) ⊤ G ( I − P B G ) ( B † A − G − 1 G x y ) − ( B † ) ⊤ G ^ ( I − P B G ) ( B † A − G − 1 G x y ) ∥ + ∥ ( B † ) ⊤ G ^ ( I − P B G ) ( B † A − G − 1 G x y ) − ( B † ) ⊤ G ^ ( I − P B ^ G ) ( B † A − G − 1 G x y ) ∥ + ∥ ( B † ) ⊤ G ^ ( I − P B ^ G ) ( B † A − G − 1 G x y ) − ( B † ) ⊤ G ^ ( I − P B ^ G ^ ) ( B † A − G ^ − 1 G ^ x y ) ∥ ≤ ∥ B † ∥ ( 1 + M P ) ( ∥ B † A ∥ + C μ ) ∥ G − G ^ ∥ + C ∥ B † ∥ ( ∥ B † A ∥ + C μ ) ∥ P B ^ G − P B G ∥ + C ∥ B † ∥ ( 1 + M P ) ∥ G − 1 G x y − G ^ − 1 G ^ x y ∥ ≤ ( L G ∥ B † ∥ ( 1 + M P ) ( ∥ B † A ∥ + C μ ) + L P C ∥ B † ∥ ( ∥ B † A ∥ + C μ ) + C L u ∥ B † ∥ ( 1 + M P ) ) ∥ x − x ^ ∥ : = C λ ∥ x − x ^ ∥ \begin{aligned}
\left\|\frac{d}{dx}\lambda^*(x)-\frac{d}{dx}\lambda^*(\hat{x})\right\|
={}&\left\|(B^\dagger)^\top G(I-P_B^G)(B^\dagger A-G^{-1}G_{xy})\right.\\
&\left.\quad-(B^\dagger)^\top\hat{G}(I-P_B^G)(B^\dagger A-G^{-1}G_{xy})\right\|\\
&+\left\|(B^\dagger)^\top\hat{G}(I-P_B^G)(B^\dagger A-G^{-1}G_{xy})\right.\\
&\left.\quad-(B^\dagger)^\top\hat{G}(I-P_{\hat{B}}^G)(B^\dagger A-G^{-1}G_{xy})\right\|\\
&+\left\|(B^\dagger)^\top\hat{G}(I-P_{\hat{B}}^G)(B^\dagger A-G^{-1}G_{xy})\right.\\
&\left.\quad-(B^\dagger)^\top\hat{G}(I-P_{\hat{B}}^{\hat{G}})(B^\dagger A-\hat{G}^{-1}\hat{G}_{xy})\right\|\\
\leq{}&\left\|B^\dagger\right\|(1+M_P)\left(\left\|B^\dagger A\right\|+\frac{C}{\mu}\right)\left\|G-\hat{G}\right\|\\
&+C\left\|B^\dagger\right\|\left(\left\|B^\dagger A\right\|+\frac{C}{\mu}\right)\left\|P_{\hat{B}}^G-P_B^G\right\|\\
&+C\left\|B^\dagger\right\|(1+M_P)\left\|G^{-1}G_{xy}-\hat{G}^{-1}\hat{G}_{xy}\right\|\\
\leq{}&\left(L_G\left\|B^\dagger\right\|(1+M_P)\left(\left\|B^\dagger A\right\|+\frac{C}{\mu}\right)+L_P C\left\|B^\dagger\right\|\left(\left\|B^\dagger A\right\|+\frac{C}{\mu}\right)\right.\\
&\left.\quad+C L_u\left\|B^\dagger\right\|(1+M_P)\right)\left\|x-\hat{x}\right\|\\
:={}&C_\lambda\left\|x-\hat{x}\right\|
\end{aligned} d x d λ ∗ ( x ) − d x d λ ∗ ( x ^ ) = ≤ ≤ := ( B † ) ⊤ G ( I − P B G ) ( B † A − G − 1 G x y ) − ( B † ) ⊤ G ^ ( I − P B G ) ( B † A − G − 1 G x y ) + ( B † ) ⊤ G ^ ( I − P B G ) ( B † A − G − 1 G x y ) − ( B † ) ⊤ G ^ ( I − P B ^ G ) ( B † A − G − 1 G x y ) + ( B † ) ⊤ G ^ ( I − P B ^ G ) ( B † A − G − 1 G x y ) − ( B † ) ⊤ G ^ ( I − P B ^ G ^ ) ( B † A − G ^ − 1 G ^ x y ) B † ( 1 + M P ) ( B † A + μ C ) G − G ^ + C B † ( B † A + μ C ) P B ^ G − P B G + C B † ( 1 + M P ) G − 1 G x y − G ^ − 1 G ^ x y ( L G B † ( 1 + M P ) ( B † A + μ C ) + L P C B † ( B † A + μ C ) + C L u B † ( 1 + M P ) ) ∥ x − x ^ ∥ C λ ∥ x − x ^ ∥
Theorem B.7 (Lipschitzness and smoothness of constrained hyper-objective F F F ). The hyper-objective F F F is L F L_F L F -Lipschitz and C F C_F C F -smooth in x x x , Where L F : = L f ( 1 + L y ) L_F:=L_f(1+L_y) L F := L f ( 1 + L y ) and C F : = ( C f ( 1 + L y ) + L f C y + L y C f + L y 2 C f ) C_F:=(C_f(1+L_y)+L_fC_y+L_yC_f+L_y^2C_f) C F := ( C f ( 1 + L y ) + L f C y + L y C f + L y 2 C f ) .
Proof. Let x , x ^ ∈ X x,\hat{x}\in X x , x ^ ∈ X . Using Lemma B.6, Lipschitzness of F F F is given by
∥ ∇ F ( x ) ∥ ≤ ∥ ∇ x f ( x , y ∗ ( x ) ) ∥ + ∥ d d x y ∗ ( x ) ∥ ∥ ∇ y f ( x , y ∗ ( x ) ) ∥ = L f ( 1 + L y ) : = L F \left\|\nabla F(x)\right\|\leq\left\|\nabla_x f(x,y^*(x))\right\|+\left\|\frac{d}{dx}y^*(x)\right\|\left\|\nabla_y f(x,y^*(x))\right\|
=L_f(1+L_y)
:=L_F ∥ ∇ F ( x ) ∥ ≤ ∥ ∇ x f ( x , y ∗ ( x )) ∥ + d x d y ∗ ( x ) ∥ ∇ y f ( x , y ∗ ( x )) ∥ = L f ( 1 + L y ) := L F
Smoothness of F F F is given by
∥ ∇ F ( x ) − ∇ F ( x ^ ) ∥ ≤ ∥ ∇ x f ( x , y ∗ ( x ) ) − ∇ x f ( x ^ , y ∗ ( x ^ ) ) ∥ + ∥ d d x y ∗ ( x ) ⊤ ∇ y f ( x , y ∗ ( x ) ) − d d x y ∗ ( x ^ ) ⊤ ∇ y f ( x ^ , y ∗ ( x ^ ) ) ∥ ≤ ( C f ( 1 + L y ) + L f C y + L y C f + L y 2 C f ) ∥ x − x ^ ∥ : = C F ∥ x − x ^ ∥ \begin{aligned}
\left\|\nabla F(x)-\nabla F(\hat{x})\right\|
\leq{}&\left\|\nabla_xf(x,y^*(x))-\nabla_xf(\hat{x},y^*(\hat{x}))\right\|\\
&+\left\|\frac{d}{dx}y^*(x)^\top\nabla_yf(x,y^*(x))-\frac{d}{dx}y^*(\hat{x})^\top\nabla_yf(\hat{x},y^*(\hat{x}))\right\|\\
\leq{}&(C_f(1+L_y)+L_fC_y+L_yC_f+L_y^2C_f)\left\|x-\hat{x}\right\|\\
:={}&C_F\left\|x-\hat{x}\right\|
\end{aligned} ∥ ∇ F ( x ) − ∇ F ( x ^ ) ∥ ≤ ≤ := ∥ ∇ x f ( x , y ∗ ( x )) − ∇ x f ( x ^ , y ∗ ( x ^ )) ∥ + d x d y ∗ ( x ) ⊤ ∇ y f ( x , y ∗ ( x )) − d x d y ∗ ( x ^ ) ⊤ ∇ y f ( x ^ , y ∗ ( x ^ )) ( C f ( 1 + L y ) + L f C y + L y C f + L y 2 C f ) ∥ x − x ^ ∥ C F ∥ x − x ^ ∥
Corollary B.8 (Smoothness of the unconstrained hyper-objective). Consider Problem P1 with unconstrained follower strategy sets Y i = R n i Y_i=\mathbb{R}^{n_i} Y i = R n i for all i ∈ [ N ] i\in[N] i ∈ [ N ] . Suppose the equilibrium mapping y ∗ ( x ) y^*(x) y ∗ ( x ) is L y L_y L y -Lipschitz and its Jacobian d d x y ∗ ( x ) \frac{d}{dx}y^*(x) d x d y ∗ ( x ) is C y C_y C y -Lipschitz. Then the hyper-objective
F ( x ) = f ( x , y ∗ ( x ) ) F(x)=f(x,y^*(x)) F ( x ) = f ( x , y ∗ ( x ))
is L F L_F L F -Lipschitz and C F C_F C F -smooth, where
L F : = L f ( 1 + L y ) L_F := L_f(1 + L_y) L F := L f ( 1 + L y )
and
C F : = C f ( 1 + L y ) + L f C y + L y C f + L y 2 C f . C_F := C_f(1 + L_y) + L_f C_y + L_y C_f + L_y^2 C_f. C F := C f ( 1 + L y ) + L f C y + L y C f + L y 2 C f .
Proof. In the unconstrained setting, the hypergradient is
∇ F ( x ) = ∇ x f ( x , y ∗ ( x ) ) + ( d d x y ∗ ( x ) ) ⊤ ∇ y f ( x , y ∗ ( x ) ) . \nabla F(x) = \nabla_x f(x, y^*(x)) + \left(\frac{d}{dx}y^*(x)\right)^\top\nabla_y f(x, y^*(x)). ∇ F ( x ) = ∇ x f ( x , y ∗ ( x )) + ( d x d y ∗ ( x ) ) ⊤ ∇ y f ( x , y ∗ ( x )) .
The proof of Theorem B.7 only uses the L y L_y L y -Lipschitzness of y ∗ ( x ) y^*(x) y ∗ ( x ) , the C y C_y C y -Lipschitzness of d d x y ∗ ( x ) \frac{d}{dx}y^*(x) d x d y ∗ ( x ) , and the Lipschitzness and smoothness of f f f . Hence, after removing all constraint and dual-variable terms, the same argument gives
∥ ∇ F ( x ) ∥ ≤ L f ( 1 + L y ) = L F , \|\nabla F(x)\| \le L_f(1 + L_y) = L_F, ∥∇ F ( x ) ∥ ≤ L f ( 1 + L y ) = L F ,
and, for any x , x ^ ∈ X x,\hat{x} \in X x , x ^ ∈ X ,
∥ ∇ F ( x ) − ∇ F ( x ^ ) ∥ ≤ ( C f ( 1 + L y ) + L f C y + L y C f + L y 2 C f ) ∥ x − x ^ ∥ . \|\nabla F(x) - \nabla F(\hat{x})\| \le(C_f(1 + L_y) + L_f C_y + L_y C_f + L_y^2 C_f)\|x - \hat{x}\|. ∥∇ F ( x ) − ∇ F ( x ^ ) ∥ ≤ ( C f ( 1 + L y ) + L f C y + L y C f + L y 2 C f ) ∥ x − x ^ ∥.
Therefore, F F F is L F L_F L F -Lipschitz and C F C_F C F -smooth.
Symmetric Jacobian Stackelberg Games: Unconstrained and Equality-Constrained Cases
We first provide auxiliary lemmas for § 4 with linear equality constraints. We then provide complete proof for theorem 4.1 and C.6 specialized to the unconstrained case.
Auxiliary Lemmas in constrained setting
Lemma C.1. The perturbed game G δ : = ( x , [ N ] , { Y i } i ∈ [ N ] , { g i + δ f } i ∈ [ N ] ) \mathcal{G}_\delta:= (x, [N], \{Y_i\}_{i\in[N]}, \{g_i + \delta f\}_{i\in[N]}) G δ := ( x , [ N ] , { Y i } i ∈ [ N ] , { g i + δ f } i ∈ [ N ] ) . G δ \mathcal{G}_\delta G δ is a C δ : = ( C + δ C f ) C_\delta:= (C + \delta C_f) C δ := ( C + δ C f ) -smooth, μ δ : = ( μ − δ C f ) \mu_\delta:= (\mu- \delta C_f) μ δ := ( μ − δ C f ) -strongly monotone and S δ : = ( S + δ S f ) S_\delta:= (S + \delta S_f) S δ := ( S + δ S f ) -Hessian smooth. If we assume δ ≤ μ 2 C f \delta\le\frac{\mu}{2C_f} δ ≤ 2 C f μ , then we set μ δ = μ 2 \mu_\delta= \frac{\mu}{2} μ δ = 2 μ , C δ = C + μ 2 C_\delta= C + \frac{\mu}{2} C δ = C + 2 μ and S δ = S + S f μ 2 C f S_\delta= S + \frac{S_f\mu}{2C_f} S δ = S + 2 C f S f μ , so that μ δ \mu_\delta μ δ , C δ C_\delta C δ and S δ S_\delta S δ are independent of δ \delta δ .
Proof. Let x , x ′ ∈ X x,x' \in X x , x ′ ∈ X . y , y ′ ∈ Y y,y' \in Y y , y ′ ∈ Y , To prove strong monotonicity,
⟨ V ( x , y ′ ) + δ ∇ y f ( x , y ′ ) − ( V ( x , y ) + δ ∇ y f ( x , y ) ) , y ′ − y ⟩ ≥ μ ∥ y ′ − y ∥ 2 + ⟨ δ ∇ y f ( x , y ′ ) − δ ∇ y f ( x , y ) , y ′ − y ⟩ ≥ μ ∥ y ′ − y ∥ 2 − δ ∥ ∇ y f ( x , y ′ ) − ∇ y f ( x , y ) ∥ ∥ y ′ − y ∥ ≥ ( μ − δ C f ) ∥ y ′ − y ∥ 2 \begin{aligned}
\langle V(x,y') + \delta\nabla_y f(x,y') - (V(x,y) + \delta\nabla_y f(x,y)), y' - y\rangle
&\ge\mu\|y' - y\|^2 + \langle\delta\nabla_y f(x,y') - \delta\nabla_y f(x,y), y' - y\rangle\\
&\ge\mu\|y' - y\|^2 - \delta\|\nabla_y f(x,y') - \nabla_y f(x,y)\|\|y' - y\| \\
&\ge(\mu- \delta C_f)\|y' - y\|^2
\end{aligned} ⟨ V ( x , y ′ ) + δ ∇ y f ( x , y ′ ) − ( V ( x , y ) + δ ∇ y f ( x , y )) , y ′ − y ⟩ ≥ μ ∥ y ′ − y ∥ 2 + ⟨ δ ∇ y f ( x , y ′ ) − δ ∇ y f ( x , y ) , y ′ − y ⟩ ≥ μ ∥ y ′ − y ∥ 2 − δ ∥ ∇ y f ( x , y ′ ) − ∇ y f ( x , y ) ∥∥ y ′ − y ∥ ≥ ( μ − δ C f ) ∥ y ′ − y ∥ 2
To prove smoothness,
∥ V ( x ′ , y ′ ) + δ ∇ y f ( x ′ , y ′ ) − ( V ( x , y ) + δ ∇ y f ( x , y ) ) ∥ ≤ C ∥ y ′ − y ∥ + δ ∥ ∇ y f ( x , y ′ ) − ∇ y f ( x , y ) ∥ ≤ ( C + δ C f ) ∥ y ′ − y ∥ \begin{aligned}
\|V(x',y') + \delta\nabla_y f(x',y') - (V(x,y) + \delta\nabla_y f(x,y))\|
&\le C\|y' - y\| + \delta\|\nabla_y f(x,y') - \nabla_y f(x,y)\| \\
&\le(C + \delta C_f)\|y' - y\|
\end{aligned} ∥ V ( x ′ , y ′ ) + δ ∇ y f ( x ′ , y ′ ) − ( V ( x , y ) + δ ∇ y f ( x , y )) ∥ ≤ C ∥ y ′ − y ∥ + δ ∥ ∇ y f ( x , y ′ ) − ∇ y f ( x , y ) ∥ ≤ ( C + δ C f ) ∥ y ′ − y ∥
To prove Hessian-smoothness,
∥ ∂ ∂ ( x , y ) V ( x ′ , y ′ ) + δ ∂ ∂ ( x , y ) ∇ y f ( x ′ , y ′ ) − ∂ ∂ ( x , y ) V ( x , y ) − δ ∂ ∂ ( x , y ) ∇ y f ( x , y ) ∥ ≤ S ∥ ( x ′ , y ′ ) − ( x , y ) ∥ + δ ∥ ∂ ∂ ( x , y ) ∇ y f ( x ′ , y ′ ) − ∂ ∂ ( x , y ) ∇ y f ( x , y ) ∥ ≤ ( S + δ S f ) ∥ ( x ′ , y ′ ) − ( x , y ) ∥ \begin{aligned}
\left\|\frac{\partial}{\partial(x,y)}V(x',y') + \delta\frac{\partial}{\partial(x,y)}\nabla_y f(x',y') - \frac{\partial}{\partial(x,y)}V(x,y) - \delta\frac{\partial}{\partial(x,y)}\nabla_y f(x,y)\right\|
&\le S\|(x',y') - (x,y)\| + \delta\left\|\frac{\partial}{\partial(x,y)}\nabla_y f(x',y') - \frac{\partial}{\partial(x,y)}\nabla_y f(x,y)\right\| \\
&\le(S + \delta S_f)\|(x',y') - (x,y)\|
\end{aligned} ∂ ( x , y ) ∂ V ( x ′ , y ′ ) + δ ∂ ( x , y ) ∂ ∇ y f ( x ′ , y ′ ) − ∂ ( x , y ) ∂ V ( x , y ) − δ ∂ ( x , y ) ∂ ∇ y f ( x , y ) ≤ S ∥ ( x ′ , y ′ ) − ( x , y ) ∥ + δ ∂ ( x , y ) ∂ ∇ y f ( x ′ , y ′ ) − ∂ ( x , y ) ∂ ∇ y f ( x , y ) ≤ ( S + δ S f ) ∥ ( x ′ , y ′ ) − ( x , y ) ∥
If we assume δ ≤ μ 2 C f \delta\le\frac{\mu}{2C_f} δ ≤ 2 C f μ , then μ δ ≥ μ 2 \mu_\delta\ge\frac{\mu}{2} μ δ ≥ 2 μ , C δ ≤ C + μ 2 C_\delta\le C + \frac{\mu}{2} C δ ≤ C + 2 μ and S δ ≤ S + S f μ 2 C f S_\delta\le S + \frac{S_f\mu}{2C_f} S δ ≤ S + 2 C f S f μ .
Lemma C.2. Let y δ ∗ y_\delta^* y δ ∗ be the Nash equilibrium of the perturbed game G δ G_\delta G δ . If δ ∈ [ 0 , Δ ] \delta\in[0,\Delta] δ ∈ [ 0 , Δ ] where Δ ≤ μ 2 C f \Delta\le\frac{\mu}{2C_f} Δ ≤ 2 C f μ , then
∥ y δ ∗ − y ∗ ∥ ≤ 2 L f μ δ = O ( δ ) \lVert y_\delta^* - y^* \rVert\le\frac{2L_f}{\mu}\delta= O(\delta) ∥ y δ ∗ − y ∗ ∥ ≤ μ 2 L f δ = O ( δ ) Proof. By optimality of y δ ∗ y_\delta^* y δ ∗ and y ∗ y^* y ∗ in their respective games,
⟨ V ( x , y δ ∗ ) + δ ∇ y f ( x , y δ ∗ ) , y ∗ − y δ ∗ ⟩ ≥ 0 \left\langle V(x,y_\delta^*)+\delta\nabla_y f(x,y_\delta^*),y^*-y_\delta^*\right\rangle\ge0 ⟨ V ( x , y δ ∗ ) + δ ∇ y f ( x , y δ ∗ ) , y ∗ − y δ ∗ ⟩ ≥ 0 ⟨ V ( x , y ∗ ) , y δ ∗ − y ∗ ⟩ ≥ 0 \left\langle V(x,y^*),y_\delta^*-y^*\right\rangle\ge0 ⟨ V ( x , y ∗ ) , y δ ∗ − y ∗ ⟩ ≥ 0 By summing up the inequalities,
⟨ V ( x , y ∗ ) − V ( x , y δ ∗ ) − δ ∇ y f ( x , y δ ∗ ) , y δ ∗ − y ∗ ⟩ ≥ 0 \left\langle V(x,y^*)-V(x,y_\delta^*)-\delta\nabla_y f(x,y_\delta^*),y_\delta^*-y^*\right\rangle\ge0 ⟨ V ( x , y ∗ ) − V ( x , y δ ∗ ) − δ ∇ y f ( x , y δ ∗ ) , y δ ∗ − y ∗ ⟩ ≥ 0 By srtong monotonicity of V + δ ∇ y f V+\delta\nabla_y f V + δ ∇ y f ,
⟨ V ( x , y δ ∗ ) + δ ∇ y f ( x , y δ ∗ ) − V ( x , y ∗ ) − δ ∇ y f ( x , y ∗ ) , y δ ∗ − y ∗ ⟩ ≥ ( μ − δ C f ) ∥ y δ ∗ − y ∗ ∥ 2 \left\langle V(x,y_\delta^*)+\delta\nabla_y f(x,y_\delta^*)-V(x,y^*)-\delta\nabla_y f(x,y^*),y_\delta^*-y^*\right\rangle\ge(\mu-\delta C_f)\lVert y_\delta^*-y^*\rVert^2 ⟨ V ( x , y δ ∗ ) + δ ∇ y f ( x , y δ ∗ ) − V ( x , y ∗ ) − δ ∇ y f ( x , y ∗ ) , y δ ∗ − y ∗ ⟩ ≥ ( μ − δ C f ) ∥ y δ ∗ − y ∗ ∥ 2 We can bound the left-hand side by
⟨ V ( x , y δ ∗ ) + δ ∇ y f ( x , y δ ∗ ) − V ( x , y ∗ ) − δ ∇ y f ( x , y ∗ ) , y δ ∗ − y ∗ ⟩ = ⟨ V ( x , y δ ∗ ) + δ ∇ y f ( x , y δ ∗ ) − V ( x , y ∗ ) + V ( x , y ∗ ) − ( V ( x , y ∗ ) + δ ∇ y f ( x , y ∗ ) ) , y δ ∗ − y ∗ ⟩ ≤ ⟨ V ( x , y ∗ ) − ( V ( x , y ∗ ) + δ ∇ y f ( x , y ∗ ) ) , y δ ∗ − y ∗ ⟩ ≤ δ ∥ ∇ y f ( x , y ∗ ) ∥ ∥ y δ ∗ − y ∗ ∥ ≤ δ L f ∥ y δ ∗ − y ∗ ∥ \begin{aligned}
&\left\langle V(x,y_\delta^*)+\delta\nabla_y f(x,y_\delta^*)-V(x,y^*)-\delta\nabla_y f(x,y^*),y_\delta^*-y^*\right\rangle\\
={}&\left\langle V(x,y_\delta^*)+\delta\nabla_y f(x,y_\delta^*)-V(x,y^*)+V(x,y^*)-(V(x,y^*)+\delta\nabla_y f(x,y^*)),y_\delta^*-y^*\right\rangle\\
\le{}&\left\langle V(x,y^*)-(V(x,y^*)+\delta\nabla_y f(x,y^*)),y_\delta^*-y^*\right\rangle\\
\le{}&\delta\lVert\nabla_y f(x,y^*)\rVert\lVert y_\delta^*-y^*\rVert\\
\le{}&\delta L_f\lVert y_\delta^*-y^*\rVert
\end{aligned} = ≤ ≤ ≤ ⟨ V ( x , y δ ∗ ) + δ ∇ y f ( x , y δ ∗ ) − V ( x , y ∗ ) − δ ∇ y f ( x , y ∗ ) , y δ ∗ − y ∗ ⟩ ⟨ V ( x , y δ ∗ ) + δ ∇ y f ( x , y δ ∗ ) − V ( x , y ∗ ) + V ( x , y ∗ ) − ( V ( x , y ∗ ) + δ ∇ y f ( x , y ∗ )) , y δ ∗ − y ∗ ⟩ ⟨ V ( x , y ∗ ) − ( V ( x , y ∗ ) + δ ∇ y f ( x , y ∗ )) , y δ ∗ − y ∗ ⟩ δ ∥ ∇ y f ( x , y ∗ )∥ ∥ y δ ∗ − y ∗ ∥ δ L f ∥ y δ ∗ − y ∗ ∥ Hence, we get the desired inequality,
∥ y δ ∗ − y ∗ ∥ ≤ δ μ − δ C f L f \lVert y_\delta^*-y^*\rVert\le\frac{\delta}{\mu-\delta C_f}L_f ∥ y δ ∗ − y ∗ ∥ ≤ μ − δ C f δ L f ≤ 2 L f μ δ \le\frac{2L_f}{\mu}\delta ≤ μ 2 L f δ
Lemma C.3 (Implicit differentiation of perturbed game).
[ d d δ y δ ∗ ∣ δ = 0 d d δ λ δ ∗ ∣ δ = 0 ] = [ ∂ ∂ y V ( x , y ∗ ) B ⊤ B 0 ] − 1 [ − ∇ y f ( x , y ∗ ) 0 ] \begin{bmatrix}
\left.\frac{d}{d\delta}y_\delta^*\right|_{\delta=0}\\
\left.\frac{d}{d\delta}\lambda_\delta^*\right|_{\delta=0}
\end{bmatrix}
=
\begin{bmatrix}
\frac{\partial}{\partial y}V(x,y^*) & B^\top\\
B & 0
\end{bmatrix}^{-1}
\begin{bmatrix}
-\nabla_y f(x,y^*)\\
0
\end{bmatrix} [ d δ d y δ ∗ δ = 0 d δ d λ δ ∗ δ = 0 ] = [ ∂ y ∂ V ( x , y ∗ ) B B ⊤ 0 ] − 1 [ − ∇ y f ( x , y ∗ ) 0 ] Proof. By Lemma C.1, G δ G_\delta G δ is a ( μ − δ C f ) (\mu-\delta C_f) ( μ − δ C f ) -strongly monotone, ( C + δ C f ) (C+\delta C_f) ( C + δ C f ) -smooth and ( S + δ S f (S+\delta S_f ( S + δ S f -Hessian smooth. Let y δ ∗ , λ δ ∗ y_\delta^*,\lambda_\delta^* y δ ∗ , λ δ ∗ be the Nash equilibrium and lagrange multiplier for G δ G_\delta G δ . We can again apply implicit function theorem and show that y δ ∗ y_\delta^* y δ ∗ and λ δ ∗ \lambda_\delta^* λ δ ∗ are continuously differentiable functions of δ \delta δ and the derivatives of them are given by
[ ∂ ∂ y V ( x , y δ ∗ ) + δ ∇ y y 2 f ( x , y δ ∗ ) B ⊤ B 0 ] [ d d δ y δ ∗ d d δ λ δ ∗ ] = [ − ∇ y f ( x , y δ ∗ ) 0 ] (13) \begin{bmatrix}
\frac{\partial}{\partial y}V(x,y_\delta^*)+\delta\nabla^2_{yy}f(x,y_\delta^*) & B^\top\\
B & 0
\end{bmatrix}
\begin{bmatrix}
\frac{d}{d\delta}y_\delta^*\\
\frac{d}{d\delta}\lambda_\delta^*
\end{bmatrix}
=
\begin{bmatrix}
-\nabla_y f(x,y_\delta^*)\\
0 \tag*{(13)}
\end{bmatrix} [ ∂ y ∂ V ( x , y δ ∗ ) + δ ∇ y y 2 f ( x , y δ ∗ ) B B ⊤ 0 ] [ d δ d y δ ∗ d δ d λ δ ∗ ] = [ − ∇ y f ( x , y δ ∗ ) 0 ] (13) we can prove that y ∗ = lim δ → 0 y δ ∗ y^*=\lim_{\delta\to0}y_\delta^* y ∗ = lim δ → 0 y δ ∗ (Lemma C.2 ). Take δ → 0 \delta\to0 δ → 0 , we have the desired result.
Lemma C.4. Assume that ∂ ∂ y V ( x , y ∗ ) \frac{\partial}{\partial y}V(x,y^*) ∂ y ∂ V ( x , y ∗ ) is symmetric. Denote y ∗ , λ ∗ y^*,\lambda^* y ∗ , λ ∗ as constants with respect to x x x , then the hypergradient ∇ F ( x ) \nabla F(x) ∇ F ( x ) can be rewritten without second-order derivatives:
v δ ( x , y δ ∗ , λ δ ∗ ) = 1 δ ∑ i = 1 N ∇ x [ g i ( x , y δ , i ∗ , y δ ¬ i ∗ ) + ( λ δ , i ∗ ) ⊤ e ( i ) ( x , y i ∗ ) ] − ∇ x [ g i ( x , y i ∗ , y ¬ i ∗ ) + ( λ i ∗ ) ⊤ e ( i ) ( x , y i ∗ ) ] v_\delta(x,y_\delta^*,\lambda_\delta^*)=\frac{1}{\delta}\sum_{i=1}^{N}\nabla_x\left[g_i(x,y_{\delta,i}^*,y_{\delta\neg i}^*)+(\lambda_{\delta,i}^*)^\top e^{(i)}(x,y_i^*)\right]-\nabla_x\left[g_i(x,y_i^*,y_{\neg i}^*)+(\lambda_i^*)^\top e^{(i)}(x,y_i^*)\right] v δ ( x , y δ ∗ , λ δ ∗ ) = δ 1 i = 1 ∑ N ∇ x [ g i ( x , y δ , i ∗ , y δ ¬ i ∗ ) + ( λ δ , i ∗ ) ⊤ e ( i ) ( x , y i ∗ ) ] − ∇ x [ g i ( x , y i ∗ , y ¬ i ∗ ) + ( λ i ∗ ) ⊤ e ( i ) ( x , y i ∗ ) ] ( d d x y ∗ ( x ) ) ⊤ ∇ y f ( x , y ∗ ( x ) ) = lim δ → 0 v δ ( x , y δ ∗ , λ δ ∗ ) \left(\frac{d}{dx}y^*(x)\right)^\top\nabla_y f(x,y^*(x))=\lim_{\delta\to0}v_\delta(x,y_\delta^*,\lambda_\delta^*) ( d x d y ∗ ( x ) ) ⊤ ∇ y f ( x , y ∗ ( x )) = δ → 0 lim v δ ( x , y δ ∗ , λ δ ∗ )
Proof.
\begin{aligned}
\left(\frac{d}{dx}y^*(x)\right)^\top\nabla_y f(x,y^*(x)) \\
&= \left[\frac{d}{dx}y^*(x)^\top\quad\frac{d}{dx}\lambda^*(x)^\top\right]
\begin{bmatrix}
\nabla_y f(x,y^*)\\
0
\end{bmatrix} \\
&= \left[\frac{\partial}{\partial x}V(x,y^*)^\top\quad A^\top\right]
\begin{bmatrix}
\frac{\partial}{\partial y}V(x,y^*) & B^\top\\
B & 0
\end{bmatrix}^{-1}
\begin{bmatrix}
-\nabla_y f(x,y^*)\\
0
\end{bmatrix} \\
&= \left[\frac{\partial}{\partial x}V(x,y^*)^\top\quad A^\top\right]
\left[\frac{d}{d\delta}y_\delta^*
\frac{d}{d\delta}\lambda_\delta^*\right]_{\delta=0} \\
&= \left[
\frac{\partial}{\partial x_1}\nabla_y g_1(x,y^*) \quad\cdots\quad\frac{\partial}{\partial x_p}\nabla_y g_1(x,y^*) \atop
\vdots\quad\cdots\quad\vdots\atop
\frac{\partial}{\partial x_1}\nabla_y g_N(x,y^*) \quad\cdots\quad\frac{\partial}{\partial x_p}\nabla_y g_N(x,y^*)
\right]^\top
\frac{d}{d\delta}y_\delta^*\bigg|_{\delta=0}
+\sum_{i=1}^{N}\left(\frac{\partial}{\partial x}e^{(i)}(x,y_i^*)\right)^\top
\frac{d}{d\delta}\lambda_{\delta,i}^*\bigg|_{\delta=0} \\
&= \sum_{i=1}^{N}\frac{\partial}{\partial y_i}\nabla_x g_i(x,y^*)^\top\frac{d}{d\delta}y_{\delta,i}^*\bigg|_{\delta=0}
+\frac{\partial}{\partial x}e^{(i)}(x,y_i^*)^\top\frac{d}{d\delta}\lambda_{\delta,i}^*\bigg|_{\delta=0} \\
&= \sum_{i=1}^{N}\lim_{\delta\to0}\frac{1}{\delta}\left[\nabla_x g_i(x,y_{\delta,i}^*,y_{-i}^*)-\nabla_x g_i(x,y_i^*,y_{-i}^*)\right]
+\frac{\partial}{\partial x}e^{(i)}(x,y_i^*)^\top\lim_{\delta\to0}\frac{1}{\delta}\left[\lambda_{\delta,i}^*-\lambda_i^*\right] \\
&= \lim_{\delta\to0}\frac{1}{\delta}\sum_{i=1}^{N}\nabla_x\left[g_i(x,y_{\delta,i}^*,y_{-i}^*)+(\lambda_{\delta,i}^*)^\top e^{(i)}(x,y_i^*)\right]
-\nabla_x\left[g_i(x,y_i^*,y_{-i}^*)+(\lambda_i^*)^\top e^{(i)}(x,y_i^*)\right]
\end{aligned}
Lemma C.5 (Lipshitzness and smoothness of perturbed lower-level solution mappings). Denote κ δ = C δ μ δ \kappa_\delta=\frac{C_\delta}{\mu_\delta} κ δ = μ δ C δ and suppose δ ≤ Δ : = μ 2 C f \delta\le\Delta:=\frac{\mu}{2C_f} δ ≤ Δ := 2 C f μ , y δ ∗ y_\delta^* y δ ∗ is L y , δ L_{y,\delta} L y , δ -Lipschitz and C y , δ C_{y,\delta} C y , δ -smooth in δ ∈ R > 0 \delta\in\mathbb{R}_{>0} δ ∈ R > 0 . λ δ ∗ \lambda_\delta^* λ δ ∗ is L λ , δ L_{\lambda,\delta} L λ , δ -Lipschitz and C λ , δ C_{\lambda,\delta} C λ , δ -smooth in δ \delta δ . Moreover, L y , δ , C y , δ , L λ , δ , C λ , δ L_{y,\delta}, C_{y,\delta}, L_{\lambda,\delta}, C_{\lambda,\delta} L y , δ , C y , δ , L λ , δ , C λ , δ are independent of the value of δ \delta δ , and can be treated as constants.
Proof. ( d d δ y δ ∗ , d d δ λ δ ∗ ) \left(\frac{d}{d\delta}y_\delta^*,\frac{d}{d\delta}\lambda_\delta^*\right) ( d δ d y δ ∗ , d δ d λ δ ∗ ) are the solutions to equation (13) . Let 0 < δ , δ ^ ≤ Δ 0<\delta,\hat{\delta}\le\Delta 0 < δ , δ ^ ≤ Δ . Using Lemma B.6, substitute G = ∂ ∂ y V ( x , y δ ∗ ) + δ ∇ y y 2 f ( x , y δ ∗ ) G=\frac{\partial}{\partial y}V(x,y_\delta^*)+\delta\nabla_{yy}^2f(x,y_\delta^*) G = ∂ y ∂ V ( x , y δ ∗ ) + δ ∇ y y 2 f ( x , y δ ∗ ) , G x y = ∇ y f ( x , y δ ∗ ) G_{xy}=\nabla_yf(x,y_\delta^*) G x y = ∇ y f ( x , y δ ∗ ) , A = 0 A=0 A = 0 , y ^ = y δ ^ ∗ \hat{y}=y_{\hat{\delta}}^* y ^ = y δ ^ ∗ , G ^ = ∂ ∂ y V ( x , y δ ^ ∗ ) + δ ^ ∇ y y 2 f ( x , y δ ^ ∗ ) \hat{G}=\frac{\partial}{\partial y}V(x,y_{\hat{\delta}}^*)+\hat{\delta}\nabla_{yy}^2f(x,y_{\hat{\delta}}^*) G ^ = ∂ y ∂ V ( x , y δ ^ ∗ ) + δ ^ ∇ y y 2 f ( x , y δ ^ ∗ ) , G ^ x y = ∇ y f ( x , y δ ^ ∗ ) \hat{G}_{xy}=\nabla_yf(x,y_{\hat{\delta}}^*) G ^ x y = ∇ y f ( x , y δ ^ ∗ ) . κ δ : = C δ μ δ \kappa_\delta:=\frac{C_\delta}{\mu_\delta} κ δ := μ δ C δ . Denote ∥ P B G − P B G ^ ∥ ≤ L P , δ ∥ G − G ^ ∥ \|P_B^G-P_B^{\hat{G}}\|\le L_{P,\delta}\|G-\hat{G}\| ∥ P B G − P B G ^ ∥ ≤ L P , δ ∥ G − G ^ ∥ and ∥ P B G ∥ ≤ M P , δ \|P_B^G\|\le M_{P,\delta} ∥ P B G ∥ ≤ M P , δ according to Lemmas B.4 and B.5. The Lipschitzness of y δ ∗ y_\delta^* y δ ∗ is given by
∥ d d δ y δ ∗ ∥ = ∥ ( I − P B G ) B ⊤ ( − A ) + P B G ( − G − 1 G x y ) ∥ ≤ ∥ I − P B G ∥ ∥ B † A ∥ + ∥ P B G ∥ ∥ G − 1 ∥ ∥ G x y ∥ ≤ ( 1 + M P , δ ) ( 0 + L f μ δ ) = : L y , δ \begin{aligned}
\left\|\frac{d}{d\delta}y_\delta^*\right\|
=\left\|(I-P_B^G)B^\top(-A)+P_B^G(-G^{-1}G_{xy})\right\| \\
&\le\|I-P_B^G\|\|B^\dagger A\|+\|P_B^G\|\|G^{-1}\|\|G_{xy}\| \\
&\le(1+M_{P,\delta})\left(0+\frac{L_f}{\mu_\delta}\right) \\
&=:L_{y,\delta}
\end{aligned} d δ d y δ ∗ = ( I − P B G ) B ⊤ ( − A ) + P B G ( − G − 1 G x y ) ≤ ∥ I − P B G ∥∥ B † A ∥ + ∥ P B G ∥∥ G − 1 ∥∥ G x y ∥ ≤ ( 1 + M P , δ ) ( 0 + μ δ L f ) =: L y , δ
Denote L G , δ : = ( S L y , δ + C f + Δ S f L y , δ ) L_{G,\delta}:=(SL_{y,\delta}+C_f+\Delta S_fL_{y,\delta}) L G , δ := ( S L y , δ + C f + Δ S f L y , δ ) , we have
∥ G − G ^ ∥ = ∥ ∂ ∂ y V ( x , y δ ∗ ) + δ ∇ y y 2 f ( x , y δ ∗ ) − ( ∂ ∂ y V ( x , y δ ^ ∗ ) + δ ^ ∇ y y 2 f ( x , y δ ^ ∗ ) ) ∥ ≤ S L y , δ ∥ δ − δ ^ ∥ + ∥ δ ∇ y y 2 f ( x , y δ ∗ ) − δ ^ ∇ y y 2 f ( x , y δ ∗ ) ∥ + ∥ δ ^ ∇ y y 2 f ( x , y δ ∗ ) − δ ^ ∇ y y 2 f ( x , y δ ^ ∗ ) ∥ ≤ ( S L y , δ + C f + Δ S f L y , δ ) ∥ δ − δ ^ ∥ \begin{aligned}
\|G-\hat{G}\|
=\left\|\frac{\partial}{\partial y}V(x,y_\delta^*)+\delta\nabla_{yy}^2f(x,y_\delta^*)-\left(\frac{\partial}{\partial y}V(x,y_{\hat{\delta}}^*)+\hat{\delta}\nabla_{yy}^2f(x,y_{\hat{\delta}}^*)\right)\right\| \\
&\le SL_{y,\delta}\|\delta-\hat{\delta}\|+\|\delta\nabla_{yy}^2f(x,y_\delta^*)-\hat{\delta}\nabla_{yy}^2f(x,y_\delta^*)\|
+\|\hat{\delta}\nabla_{yy}^2f(x,y_\delta^*)-\hat{\delta}\nabla_{yy}^2f(x,y_{\hat{\delta}}^*)\| \\
&\le(SL_{y,\delta}+C_f+\Delta S_fL_{y,\delta})\|\delta-\hat{\delta}\|
\end{aligned} ∥ G − G ^ ∥ = ∂ y ∂ V ( x , y δ ∗ ) + δ ∇ y y 2 f ( x , y δ ∗ ) − ( ∂ y ∂ V ( x , y δ ^ ∗ ) + δ ^ ∇ y y 2 f ( x , y δ ^ ∗ ) ) ≤ S L y , δ ∥ δ − δ ^ ∥ + ∥ δ ∇ y y 2 f ( x , y δ ∗ ) − δ ^ ∇ y y 2 f ( x , y δ ∗ ) ∥ + ∥ δ ^ ∇ y y 2 f ( x , y δ ∗ ) − δ ^ ∇ y y 2 f ( x , y δ ^ ∗ ) ∥ ≤ ( S L y , δ + C f + Δ S f L y , δ ) ∥ δ − δ ^ ∥
: = L G , δ ∥ δ − δ ^ ∥ :=L_{G,\delta}\|\delta-\hat{\delta}\| := L G , δ ∥ δ − δ ^ ∥
Denote L u , δ = C f L y , δ + L f μ δ 2 L G , δ L_{u,\delta}=C_fL_{y,\delta}+\frac{L_f}{\mu_\delta^2}L_{G,\delta} L u , δ = C f L y , δ + μ δ 2 L f L G , δ , we have
∥ G − 1 G x y − G ^ − 1 G ^ x y ∥ ≤ ∥ G − 1 G x y − G − 1 G ^ x y ∥ + ∥ G − 1 G ^ x y − G ^ − 1 G ^ x y ∥ ≤ 1 μ δ ∥ G x y − G ^ x y ∥ + ∥ G ^ x y ∥ ∥ G − 1 ( G ^ − G ) G ^ − 1 ∥ ≤ ( C f L y , δ + L f μ δ 2 L G , δ ) ∥ δ − δ ^ ∥ : = L u , δ ∥ δ − δ ^ ∥ \begin{aligned}
\left\|G^{-1}G_{xy}-\widehat{G}^{-1}\widehat{G}_{xy}\right\|
&\leq\left\|G^{-1}G_{xy}-G^{-1}\widehat{G}_{xy}\right\|+\left\|G^{-1}\widehat{G}_{xy}-\widehat{G}^{-1}\widehat{G}_{xy}\right\|\\
&\leq\frac{1}{\mu_\delta}\left\|G_{xy}-\widehat{G}_{xy}\right\|+\left\|\widehat{G}_{xy}\right\|\left\|G^{-1}(\widehat{G}-G)\widehat{G}^{-1}\right\|\\
&\leq\left(C_fL_{y,\delta}+\frac{L_f}{\mu_\delta^2}L_{G,\delta}\right)\|\delta-\widehat{\delta}\|\\
&:=L_{u,\delta}\|\delta-\widehat{\delta}\|
\end{aligned} G − 1 G x y − G − 1 G x y ≤ G − 1 G x y − G − 1 G x y + G − 1 G x y − G − 1 G x y ≤ μ δ 1 G x y − G x y + G x y G − 1 ( G − G ) G − 1 ≤ ( C f L y , δ + μ δ 2 L f L G , δ ) ∥ δ − δ ∥ := L u , δ ∥ δ − δ ∥
The smoothness of y δ ∗ y_\delta^* y δ ∗ is given by
∥ d d x y δ ∗ − d d x y δ ^ ∗ ∥ \left\|\frac{d}{dx}y_\delta^*-\frac{d}{dx}y_{\widehat{\delta}}^*\right\| d x d y δ ∗ − d x d y δ ∗
≤ L P , δ ∥ G − G ^ ∥ 0 + L P , δ ∥ G − G ^ ∥ L f μ δ + M P , δ ∥ G − 1 G x y − G ^ − 1 G ^ x y ∥ ≤ ( L P , δ + L P , δ L G , δ L f μ δ + M P , δ L u , δ ) ∥ δ − δ ^ ∥ : = C y , δ ∥ δ − δ ^ ∥ \begin{aligned}
&\leq L_{P,\delta}\|G-\widehat{G}\|_0+L_{P,\delta}\|G-\widehat{G}\|\frac{L_f}{\mu_\delta}+M_{P,\delta}\left\|G^{-1}G_{xy}-\widehat{G}^{-1}\widehat{G}_{xy}\right\|\\
&\leq\left(L_{P,\delta}+L_{P,\delta}L_{G,\delta}\frac{L_f}{\mu_\delta}+M_{P,\delta}L_{u,\delta}\right)\|\delta-\widehat{\delta}\|\\
&:=C_{y,\delta}\|\delta-\widehat{\delta}\|
\end{aligned} ≤ L P , δ ∥ G − G ∥ 0 + L P , δ ∥ G − G ∥ μ δ L f + M P , δ G − 1 G x y − G − 1 G x y ≤ ( L P , δ + L P , δ L G , δ μ δ L f + M P , δ L u , δ ) ∥ δ − δ ∥ := C y , δ ∥ δ − δ ∥
The Lipschitzness of λ δ ∗ \lambda_\delta^* λ δ ∗ is given by
∥ d d δ λ δ ∗ ∥ = ∥ ( B † ) ⊤ G ( I − P B G ) ( B † A − G − 1 G x y ) ∥ ≤ ∥ B † ∥ ∥ C δ ∥ ( 1 + M P , δ ) ( 0 + L f μ δ ) \begin{aligned}
\left\|\frac{d}{d\delta}\lambda_\delta^*\right\|=\left\|(B^\dagger)^\top G(I-P_B^G)(B^\dagger A-G^{-1}G_{xy})\right\| \\
&\leq\|B^\dagger\|\|C_\delta\|(1+M_{P,\delta})\left(0+\frac{L_f}{\mu_\delta}\right)
\end{aligned} d δ d λ δ ∗ = ( B † ) ⊤ G ( I − P B G ) ( B † A − G − 1 G x y ) ≤ ∥ B † ∥∥ C δ ∥ ( 1 + M P , δ ) ( 0 + μ δ L f )
: = L λ , δ :=L_{\lambda,\delta} := L λ , δ
The smoothness of λ δ ∗ \lambda_\delta^* λ δ ∗ is given by
∥ d d δ λ δ ∗ − d d δ λ δ ^ ∗ ∥ = ∥ ( B † ) ⊤ G ( I − P B G ) ( B † A − G − 1 G x y ) − B † ⊤ G ^ ( I − P B G ) ( B † A − G − 1 G x y ) ∥ + ∥ B † ⊤ G ^ ( I − P B G ) ( B † A − G − 1 G x y ) − B † ⊤ G ^ ( I − P B G ^ ) ( B † A − G − 1 G x y ) ∥ + ∥ B † ⊤ G ^ ( I − P B G ^ ) ( B † A − G − 1 G x y ) − B † ⊤ G ^ ( I − P B G ^ ) ( B † A − G ^ − 1 G ^ x y ) ∥ ≤ ∥ B † ∥ ( 1 + M P , δ ) ( 0 + L f μ δ ) ∥ G − G ^ ∥ + C δ ∥ B † ∥ ( 0 + L f μ δ ) ∥ P B G ^ − P B G ∥ + C δ ∥ B † ∥ ( 1 + M P , δ ) ∥ G − 1 G x y − G ^ − 1 G ^ x y ∥ ≤ ( ∥ B † ∥ ( 1 + M P , δ ) L f μ δ L G , δ + C δ ∥ B † ∥ L f μ δ L P , δ + C δ ∥ B † ∥ ( 1 + M P , δ ) L u , δ ) ∥ δ − δ ^ ∥ : = C λ , δ ∥ δ − δ ^ ∥ \begin{aligned}
\left\|\frac{d}{d\delta}\lambda_\delta^*-\frac{d}{d\delta}\lambda_{\widehat{\delta}}^*\right\|
&=\left\|(B^\dagger)^\top G(I-P_B^G)(B^\dagger A-G^{-1}G_{xy})\right.\\
&\qquad\left.-B^{\dagger\top}\widehat{G}(I-P_B^G)(B^\dagger A-G^{-1}G_{xy})\right\|\\
&\quad+\left\|B^{\dagger\top}\widehat{G}(I-P_B^G)(B^\dagger A-G^{-1}G_{xy})\right.\\
&\qquad\left.-B^{\dagger\top}\widehat{G}(I-P_B^{\widehat{G}})(B^\dagger A-G^{-1}G_{xy})\right\|\\
&\quad+\left\|B^{\dagger\top}\widehat{G}(I-P_B^{\widehat{G}})(B^\dagger A-G^{-1}G_{xy})\right.\\
&\qquad\left.-B^{\dagger\top}\widehat{G}(I-P_B^{\widehat{G}})(B^\dagger A-\widehat{G}^{-1}\widehat{G}_{xy})\right\|\\
&\leq\|B^\dagger\|(1+M_{P,\delta})\left(0+\frac{L_f}{\mu_\delta}\right)\|G-\widehat{G}\|+C_\delta\|B^\dagger\|\left(0+\frac{L_f}{\mu_\delta}\right)\|P_B^{\widehat{G}}-P_B^G\|\\
&\qquad+C_\delta\|B^\dagger\|(1+M_{P,\delta})\left\|G^{-1}G_{xy}-\widehat{G}^{-1}\widehat{G}_{xy}\right\|\\
&\leq\left(\|B^\dagger\|(1+M_{P,\delta})\frac{L_f}{\mu_\delta}L_{G,\delta}+C_\delta\|B^\dagger\|\frac{L_f}{\mu_\delta}L_{P,\delta}+C_\delta\|B^\dagger\|(1+M_{P,\delta})L_{u,\delta}\right)\|\delta-\widehat{\delta}\|\\
&:=C_{\lambda,\delta}\|\delta-\widehat{\delta}\|
\end{aligned} d δ d λ δ ∗ − d δ d λ δ ∗ = ( B † ) ⊤ G ( I − P B G ) ( B † A − G − 1 G x y ) − B † ⊤ G ( I − P B G ) ( B † A − G − 1 G x y ) + B † ⊤ G ( I − P B G ) ( B † A − G − 1 G x y ) − B † ⊤ G ( I − P B G ) ( B † A − G − 1 G x y ) + B † ⊤ G ( I − P B G ) ( B † A − G − 1 G x y ) − B † ⊤ G ( I − P B G ) ( B † A − G − 1 G x y ) ≤ ∥ B † ∥ ( 1 + M P , δ ) ( 0 + μ δ L f ) ∥ G − G ∥ + C δ ∥ B † ∥ ( 0 + μ δ L f ) ∥ P B G − P B G ∥ + C δ ∥ B † ∥ ( 1 + M P , δ ) G − 1 G x y − G − 1 G x y ≤ ( ∥ B † ∥ ( 1 + M P , δ ) μ δ L f L G , δ + C δ ∥ B † ∥ μ δ L f L P , δ + C δ ∥ B † ∥ ( 1 + M P , δ ) L u , δ ) ∥ δ − δ ∥ := C λ , δ ∥ δ − δ ∥
Theorem 4.1. If the Jacobian J = ∂ ∂ y V ( x , y ∗ ) J=\frac{\partial}{\partial y}V(x,y^*) J = ∂ y ∂ V ( x , y ∗ ) is symmetric, then:
∥ v δ ( x , y δ ∗ ) + ( ∂ ∂ x V ( x , y δ ∗ ) ) ⊤ J ( x , y ∗ ) − ⊤ ∇ y f ( x , y δ ∗ ) ∥ ≤ O ( N δ ) . (14) \left\|v_\delta(x,y_\delta^*)+\left(\frac{\partial}{\partial x}V(x,y_\delta^*)\right)^\top J(x,y^*)^{-\top}\nabla_y f(x,y_\delta^*)\right\|\leq O(N\delta). \tag*{(14)} v δ ( x , y δ ∗ ) + ( ∂ x ∂ V ( x , y δ ∗ ) ) ⊤ J ( x , y ∗ ) − ⊤ ∇ y f ( x , y δ ∗ ) ≤ O ( N δ ) . (14)
Proof. Let y t ∗ y_t^* y t ∗ denote the Nash equilibrium of the perturbed game
G t : = ( x , [ N ] , { R n i } i ∈ [ N ] , { g i + t f } i ∈ [ N ] ) , t ∈ [ 0 , δ ] . G_t:=\left(x,[N],\{\mathbb{R}^{n_i}\}_{i\in[N]},\{g_i+tf\}_{i\in[N]}\right),\qquad t\in[0,\delta]. G t := ( x , [ N ] , { R n i } i ∈ [ N ] , { g i + t f } i ∈ [ N ] ) , t ∈ [ 0 , δ ] .
The equilibrium condition is
V ( x , y t ∗ ) + t ∇ y f ( x , y t ∗ ) = 0. V(x,y_t^*)+t\nabla_y f(x,y_t^*)=0. V ( x , y t ∗ ) + t ∇ y f ( x , y t ∗ ) = 0.
Differentiating with respect to t t t gives
( J ( x , y t ∗ ) + t ∇ y y 2 f ( x , y t ∗ ) ) d d t y t ∗ = − ∇ y f ( x , y t ∗ ) . \left(J(x,y_t^*)+t\nabla_{yy}^2f(x,y_t^*)\right)\frac{d}{dt}y_t^*=-\nabla_y f(x,y_t^*). ( J ( x , y t ∗ ) + t ∇ y y 2 f ( x , y t ∗ ) ) d t d y t ∗ = − ∇ y f ( x , y t ∗ ) .
Taking t = 0 t=0 t = 0 ,
d d t y t ∗ ∣ t = 0 = J ( x , y ∗ ) − 1 ( − ∇ y f ( x , y ∗ ) ) . \left.\frac{d}{dt}y_t^*\right|_{t=0}=J(x,y^*)^{-1}\left(-\nabla_y f(x,y^*)\right). d t d y t ∗ t = 0 = J ( x , y ∗ ) − 1 ( − ∇ y f ( x , y ∗ ) ) .
By implicit differentiation of the original equilibrium condition V ( x , y ∗ ) = 0 V(x,y^*)=0 V ( x , y ∗ ) = 0 ,
( d d x y ∗ ( x ) ) T ∇ y f ( x , y ∗ ) = ∂ ∂ x V ( x , y ∗ ) T J ( x , y ∗ ) − T ( − ∇ y f ( x , y ∗ ) ) . \left(\frac{d}{dx}y^*(x)\right)^{\mathsf T}\nabla_y f(x,y^*)=\frac{\partial}{\partial x}V(x,y^*)^{\mathsf T}J(x,y^*)^{-\mathsf T}\left(-\nabla_y f(x,y^*)\right). ( d x d y ∗ ( x ) ) T ∇ y f ( x , y ∗ ) = ∂ x ∂ V ( x , y ∗ ) T J ( x , y ∗ ) − T ( − ∇ y f ( x , y ∗ ) ) .
Since J ( x , y ∗ ) J(x,y^*) J ( x , y ∗ ) is symmetric,
J ( x , y ∗ ) − T = J ( x , y ∗ ) − 1 . J(x,y^*)^{-\mathsf T}=J(x,y^*)^{-1}. J ( x , y ∗ ) − T = J ( x , y ∗ ) − 1 .
Hence,
( d d x y ∗ ( x ) ) T ∇ y f ( x , y ∗ ) = ∂ ∂ x V ( x , y ∗ ) T d d t y t ∗ ∣ t = 0 . \left(\frac{d}{dx}y^*(x)\right)^{\mathsf T}\nabla_y f(x,y^*)=\left.\frac{\partial}{\partial x}V(x,y^*)^{\mathsf T}\frac{d}{dt}y_t^*\right|_{t=0}. ( d x d y ∗ ( x ) ) T ∇ y f ( x , y ∗ ) = ∂ x ∂ V ( x , y ∗ ) T d t d y t ∗ t = 0 .
By equality of mixed partial derivatives,
∂ ∂ x V ( x , y ∗ ) T d d t y t ∗ ∣ t = 0 = ∑ i = 1 N ∂ ∂ y i ∇ x g i ( x , y ∗ ) T d d t y t , i ∗ ∣ t = 0 . \left.\frac{\partial}{\partial x}V(x,y^*)^{\mathsf T}\frac{d}{dt}y_t^*\right|_{t=0}
=\sum_{i=1}^{N}\frac{\partial}{\partial y_i}\nabla_x g_i(x,y^*)^{\mathsf T}\left.\frac{d}{dt}y_{t,i}^*\right|_{t=0}. ∂ x ∂ V ( x , y ∗ ) T d t d y t ∗ t = 0 = i = 1 ∑ N ∂ y i ∂ ∇ x g i ( x , y ∗ ) T d t d y t , i ∗ t = 0 .
For 0 ≤ t ≤ δ ≤ μ / ( 2 C f ) 0\leq t\leq\delta\leq\mu/(2C_f) 0 ≤ t ≤ δ ≤ μ / ( 2 C f ) , the perturbed operator V + t ∇ y f V+t\nabla_y f V + t ∇ y f is at least μ / 2 \mu/2 μ /2 -strongly monotone. Under Assumptions 3.4 and 3.5, there exist constants L y , δ L_{y,\delta} L y , δ and C y , δ C_{y,\delta} C y , δ , independent of δ \delta δ , such that
∥ y t ∗ − y ∗ ∥ ≤ L y , δ t , ∥ d d t y t ∗ ∥ ≤ L y , δ , \|y_t^*-y^*\|\leq L_{y,\delta}t,\qquad
\left\|\frac{d}{dt}y_t^*\right\|\leq L_{y,\delta}, ∥ y t ∗ − y ∗ ∥ ≤ L y , δ t , d t d y t ∗ ≤ L y , δ ,
and
∥ d d t y t ∗ − d d t y t ∗ ∣ t = 0 ∥ ≤ C y , δ t . \left\|\left.\frac{d}{dt}y_t^*-\frac{d}{dt}y_t^*\right|_{t=0}\right\|\leq C_{y,\delta}t. d t d y t ∗ − d t d y t ∗ t = 0 ≤ C y , δ t .
By definition of finite difference in (4) ,
v δ ( x , y δ ∗ ) = 1 δ ∑ i = 1 N [ ∇ x g i ( x , y δ , i ∗ , y − i ∗ ) − ∇ x g i ( x , y i ∗ , y − i ∗ ) ] . v_\delta(x,y_\delta^*)=\frac{1}{\delta}\sum_{i=1}^{N}\left[\nabla_xg_i(x,y_{\delta,i}^*,y_{-i}^*)-\nabla_xg_i(x,y_i^*,y_{-i}^*)\right]. v δ ( x , y δ ∗ ) = δ 1 i = 1 ∑ N [ ∇ x g i ( x , y δ , i ∗ , y − i ∗ ) − ∇ x g i ( x , y i ∗ , y − i ∗ ) ] .
For all i ∈ [ N ] i\in[N] i ∈ [ N ] , by the fundamental theorem of calculus,
∇ x g i ( x , y δ , i ∗ , y − i ∗ ) − ∇ x g i ( x , y i ∗ , y − i ∗ ) = ∫ 0 δ ∂ ∂ y i ∇ x g i ( x , y t , i ∗ , y − i ∗ ) d d t y t , i ∗ d t . \nabla_xg_i(x,y_{\delta,i}^*,y_{-i}^*)-\nabla_xg_i(x,y_i^*,y_{-i}^*)
=\int_0^\delta\frac{\partial}{\partial y_i}\nabla_xg_i(x,y_{t,i}^*,y_{-i}^*)\frac{d}{dt}y_{t,i}^*\,dt. ∇ x g i ( x , y δ , i ∗ , y − i ∗ ) − ∇ x g i ( x , y i ∗ , y − i ∗ ) = ∫ 0 δ ∂ y i ∂ ∇ x g i ( x , y t , i ∗ , y − i ∗ ) d t d y t , i ∗ d t .
We bound each term by
∥ ∇ x g i ( x , y δ , i ∗ , y − i ∗ ) − ∇ x g i ( x , y i ∗ , y − i ∗ ) δ − ∂ ∂ y i ∇ x g i ( x , y ∗ ) d d t y t , i ∗ ∣ t = 0 ∥ ≤ 1 δ ∫ 0 δ ∥ ∂ ∂ y i ∇ x g i ( x , y t , i ∗ , y − i ∗ ) d d t y t , i ∗ − ∂ ∂ y i ∇ x g i ( x , y ∗ ) d d t y t , i ∗ ∣ t = 0 ∥ d t ≤ 1 δ ∫ 0 δ ( S L y , δ 2 t + C C y , δ t ) d t = 1 2 ( S L y , δ 2 + C C y , δ ) δ . \begin{aligned}
&\left\|
\frac{\nabla_xg_i(x,y_{\delta,i}^*,y_{-i}^*)-\nabla_xg_i(x,y_i^*,y_{-i}^*)}{\delta}
-\left.\frac{\partial}{\partial y_i}\nabla_xg_i(x,y^*)\frac{d}{dt}y_{t,i}^*\right|_{t=0}
\right\| \\
&\leq\frac{1}{\delta}\int_0^\delta
\left\|
\frac{\partial}{\partial y_i}\nabla_xg_i(x,y_{t,i}^*,y_{-i}^*)\frac{d}{dt}y_{t,i}^*
-\left.\frac{\partial}{\partial y_i}\nabla_xg_i(x,y^*)\frac{d}{dt}y_{t,i}^*\right|_{t=0}
\right\|\,dt \\
&\leq\frac{1}{\delta}\int_0^\delta\left(SL_{y,\delta}^2t+CC_{y,\delta}t\right)\,dt \\
&=\frac{1}{2}\left(SL_{y,\delta}^2+CC_{y,\delta}\right)\delta.
\end{aligned} δ ∇ x g i ( x , y δ , i ∗ , y − i ∗ ) − ∇ x g i ( x , y i ∗ , y − i ∗ ) − ∂ y i ∂ ∇ x g i ( x , y ∗ ) d t d y t , i ∗ t = 0 ≤ δ 1 ∫ 0 δ ∂ y i ∂ ∇ x g i ( x , y t , i ∗ , y − i ∗ ) d t d y t , i ∗ − ∂ y i ∂ ∇ x g i ( x , y ∗ ) d t d y t , i ∗ t = 0 d t ≤ δ 1 ∫ 0 δ ( S L y , δ 2 t + C C y , δ t ) d t = 2 1 ( S L y , δ 2 + C C y , δ ) δ .
Summing over i = 1 , … , N i=1,\ldots,N i = 1 , … , N , we obtain
∥ v δ ( x , y δ ∗ ) − ( d d x y ∗ ( x ) ) T ∇ y f ( x , y ∗ ( x ) ) ∥ ≤ N 2 ( S L y , δ 2 + C C y , δ ) δ = O ( N δ ) . \left\|v_{\delta}(x,y_{\delta}^{*})-\left(\frac{d}{dx}y^{*}(x)\right)^{\mathsf{T}}\nabla_y f(x,y^{*}(x))\right\|
\le\frac{N}{2}\left(SL_{y,\delta}^{2}+CC_{y,\delta}\right)\delta=O(N\delta). v δ ( x , y δ ∗ ) − ( d x d y ∗ ( x ) ) T ∇ y f ( x , y ∗ ( x )) ≤ 2 N ( S L y , δ 2 + C C y , δ ) δ = O ( N δ ) .
Theorem C.6. Consider problem P1 and assume that the Jacobian J = ∂ ∂ y V ( x , y ∗ ) J=\frac{\partial}{\partial y}V(x,y^{*}) J = ∂ y ∂ V ( x , y ∗ ) is symmetric. Under assumptions 3.4 and 3.5, Algorithm 1 specialized to unconstrained case outputs an O ( ϵ ) O(\epsilon) O ( ϵ ) -stationary point of the hyperobjective F F F with O ~ ( N ϵ − 2 ) \widetilde{O}(N\epsilon^{-2}) O ( N ϵ − 2 ) oracle calls to f f f and g i g_i g i .
Proof. By Assumptions 3.4 and 3.5 and the implicit function theorem, the lower-level solution mapping y ∗ ( x ) y^{*}(x) y ∗ ( x ) is Lipschitz and smooth. By Corollary B.8, the hyperobjective F F F is C F C_F C F -smooth for some constant C F > 0 C_F>0 C F > 0 .
Suppose we have access to approximate lower-level solutions y ^ , y ^ δ \hat{y},\hat{y}_{\delta} y ^ , y ^ δ such that
∥ y ∗ − y ^ ∥ ≤ δ 2 , ∥ y δ ∗ − y ^ δ ∥ ≤ δ 2 . \|y^{*}-\hat{y}\|\le\delta^{2},\qquad\|y_{\delta}^{*}-\hat{y}_{\delta}\|\le\delta^{2}. ∥ y ∗ − y ^ ∥ ≤ δ 2 , ∥ y δ ∗ − y ^ δ ∥ ≤ δ 2 .
We construct the inexact hypergradient
∇ F ^ ( x ) = ∇ x f ( x , y ^ ) + v ^ δ ( x , y ^ δ ) , \nabla\hat{F}(x)=\nabla_x f(x,\hat{y})+\hat{v}_{\delta}(x,\hat{y}_{\delta}), ∇ F ^ ( x ) = ∇ x f ( x , y ^ ) + v ^ δ ( x , y ^ δ ) ,
where
v ^ δ ( x , y ^ δ ) = 1 δ ∑ i = 1 N [ ∇ x g i ( x , y ^ δ , i , y ^ − i ) − ∇ x g i ( x , y ^ i , y ^ − i ) ] . \hat{v}_{\delta}(x,\hat{y}_{\delta})=\frac{1}{\delta}\sum_{i=1}^{N}\left[\nabla_x g_i(x,\hat{y}_{\delta,i},\hat{y}_{-i})-\nabla_x g_i(x,\hat{y}_i,\hat{y}_{-i})\right]. v ^ δ ( x , y ^ δ ) = δ 1 i = 1 ∑ N [ ∇ x g i ( x , y ^ δ , i , y ^ − i ) − ∇ x g i ( x , y ^ i , y ^ − i ) ] .
We first bound the error due to the approximate lower-level solutions:
∥ v ^ δ ( x , y ^ δ ) − v δ ( x , y δ ∗ ) ∥ ≤ 1 δ ∑ i = 1 N ∥ ∇ x g i ( x , y ^ δ , i , y ^ − i ) − ∇ x g i ( x , y δ , i ∗ , y − i ∗ ) ∥ + 1 δ ∑ i = 1 N ∥ ∇ x g i ( x , y i ∗ , y − i ∗ ) − ∇ x g i ( x , y ^ i , y ^ − i ) ∥ ≤ 1 δ ∑ i = 1 N ( 2 C δ 2 + C δ 2 ) = 3 C N δ = O ( N δ ) . \begin{aligned}
\|\hat{v}_{\delta}(x,\hat{y}_{\delta})-v_{\delta}(x,y_{\delta}^{*})\|
&\le\frac{1}{\delta}\sum_{i=1}^{N}\|\nabla_x g_i(x,\hat{y}_{\delta,i},\hat{y}_{-i})-\nabla_x g_i(x,y_{\delta,i}^{*},y_{-i}^{*})\|\\
&\quad+\frac{1}{\delta}\sum_{i=1}^{N}\|\nabla_x g_i(x,y_i^{*},y_{-i}^{*})-\nabla_x g_i(x,\hat{y}_i,\hat{y}_{-i})\|\\
&\le\frac{1}{\delta}\sum_{i=1}^{N}(2C\delta^{2}+C\delta^{2})\\
&=3CN\delta=O(N\delta).
\end{aligned} ∥ v ^ δ ( x , y ^ δ ) − v δ ( x , y δ ∗ ) ∥ ≤ δ 1 i = 1 ∑ N ∥ ∇ x g i ( x , y ^ δ , i , y ^ − i ) − ∇ x g i ( x , y δ , i ∗ , y − i ∗ ) ∥ + δ 1 i = 1 ∑ N ∥ ∇ x g i ( x , y i ∗ , y − i ∗ ) − ∇ x g i ( x , y ^ i , y ^ − i ) ∥ ≤ δ 1 i = 1 ∑ N ( 2 C δ 2 + C δ 2 ) = 3 C N δ = O ( N δ ) .
Also,
∥ ∇ x f ( x , y ^ ) − ∇ x f ( x , y ∗ ) ∥ ≤ C f ∥ y ^ − y ∗ ∥ ≤ C f δ 2 . \|\nabla_x f(x,\hat{y})-\nabla_x f(x,y^{*})\|\le C_f\|\hat{y}-y^{*}\|\le C_f\delta^{2}. ∥ ∇ x f ( x , y ^ ) − ∇ x f ( x , y ∗ ) ∥ ≤ C f ∥ y ^ − y ∗ ∥ ≤ C f δ 2 .
Combining the above inequalities with Theorem 4.2,
∥ ∇ F ^ ( x ) − ∇ F ( x ) ∥ ≤ ∥ ∇ x f ( x , y ^ ) − ∇ x f ( x , y ∗ ) ∥ + ∥ v ^ δ ( x , y ^ δ ) − v δ ( x , y δ ∗ ) ∥ + ∥ v δ ( x , y δ ∗ ) − ( d d x y ∗ ( x ) ) T ∇ y f ( x , y ∗ ) ∥ ≤ C f δ 2 + O ( N δ ) = O ( N δ ) . \begin{aligned}
\|\nabla\hat{F}(x)-\nabla F(x)\|
&\le\|\nabla_x f(x,\hat{y})-\nabla_x f(x,y^{*})\|\\
&\quad+\|\hat{v}_{\delta}(x,\hat{y}_{\delta})-v_{\delta}(x,y_{\delta}^{*})\|\\
&\quad+\left\|v_{\delta}(x,y_{\delta}^{*})-\left(\frac{d}{dx}y^{*}(x)\right)^{\mathsf{T}}\nabla_y f(x,y^{*})\right\|\\
&\le C_f\delta^{2}+O(N\delta)\\
&=O(N\delta).
\end{aligned} ∥∇ F ^ ( x ) − ∇ F ( x ) ∥ ≤ ∥ ∇ x f ( x , y ^ ) − ∇ x f ( x , y ∗ ) ∥ + ∥ v ^ δ ( x , y ^ δ ) − v δ ( x , y δ ∗ ) ∥ + v δ ( x , y δ ∗ ) − ( d x d y ∗ ( x ) ) T ∇ y f ( x , y ∗ ) ≤ C f δ 2 + O ( N δ ) = O ( N δ ) .
Set
δ = ϵ N . \delta=\frac{\epsilon}{N}. δ = N ϵ .
Then
∥ ∇ F ^ ( x ) − ∇ F ( x ) ∥ ≤ O ( ϵ ) . \lVert\nabla\widehat{F}(x)-\nabla F(x)\rVert\le O(\epsilon). ∥ ∇ F ( x ) − ∇ F ( x )∥ ≤ O ( ϵ ) .
Using ∇ F ^ ( x ) \nabla\widehat{F}(x) ∇ F ( x ) as the inexact gradient in Lemma D.2 , an O ( ϵ ) O(\epsilon) O ( ϵ ) -stationary point of F F F can be obtained with
O ( ϵ − 2 ) O(\epsilon^{-2}) O ( ϵ − 2 )
calls to ∇ F ^ \nabla\widehat{F} ∇ F . It remains to bound the oracle complexity for computing each ∇ F ^ \nabla\widehat{F} ∇ F . Since the lower-level game is unconstrained, PGD reduces to gradient descent,
y k + 1 = y k − η V ( x , y k ) . y^{k+1}=y^k-\eta V(x,y^k). y k + 1 = y k − η V ( x , y k ) .
By Assumption 3.4 and Lemma E.1 , gradient descent converges linearly to y ∗ y^* y ∗ . Moreover, for δ ≤ μ / ( 2 C f ) \delta\le\mu/(2C_f) δ ≤ μ / ( 2 C f ) , the perturbed operator
V δ ( x , y ) = V ( x , y ) + δ ∇ y f ( x , y ) V_\delta(x,y)=V(x,y)+\delta\nabla_y f(x,y) V δ ( x , y ) = V ( x , y ) + δ ∇ y f ( x , y )
is at least μ / 2 \mu/2 μ /2 -strongly monotone and ( C + δ C f ) (C+\delta C_f) ( C + δ C f ) -smooth, so gradient descent also converges linearly to y δ ∗ y_\delta^* y δ ∗ . For each call to ∇ F ^ ( x ) \nabla\widehat{F}(x) ∇ F ( x ) ,
O ( log 1 δ 2 ) = O ( log ( N 2 ϵ − 2 ) ) \mathcal{O}\left(\log\frac{1}{\delta^2}\right)=\mathcal{O}\left(\log(N^2\epsilon^{-2})\right) O ( log δ 2 1 ) = O ( log ( N 2 ϵ − 2 ) )
lower-level iterations are sufficient to obtain δ 2 \delta^2 δ 2 -accurate solutions y ^ \widehat{y} y and y ^ δ \widehat{y}_\delta y δ . Each lower-level iteration requires one first-order oracle call to each of the N N N follower cost functions. Therefore, the total oracle complexity is
O ( N ϵ − 2 log ( N 2 ϵ − 2 ) ) = O ~ ( N ϵ − 2 ) . \mathcal{O}\left(N\epsilon^{-2}\log(N^2\epsilon^{-2})\right)
=\widetilde{\mathcal{O}}(N\epsilon^{-2}). O ( N ϵ − 2 log ( N 2 ϵ − 2 ) ) = O ( N ϵ − 2 ) .
Inexactness in Lower-Level solutions and Hypergradient
Lemma D.1 (Inexactness in lower-level solutions). Assume we have access to approximate lower-level solutions y ^ \widehat{y} y , λ ^ \widehat{\lambda} λ , y ^ δ \widehat{y}_\delta y δ , λ ^ δ \widehat{\lambda}_\delta λ δ such that for some δ ^ > 0 \widehat{\delta}>0 δ > 0 , ∥ y ∗ − y ^ ∥ ≤ δ ^ \lVert y^*-\widehat{y}\rVert\le\widehat{\delta} ∥ y ∗ − y ∥ ≤ δ , ∥ λ ∗ − λ ^ ∥ ≤ δ ^ \lVert\lambda^*-\widehat{\lambda}\rVert\le\widehat{\delta} ∥ λ ∗ − λ ∥ ≤ δ , ∥ y δ ∗ − y ^ δ ∥ ≤ δ ^ \lVert y_\delta^*-\widehat{y}_\delta\rVert\le\widehat{\delta} ∥ y δ ∗ − y δ ∥ ≤ δ , ∥ λ δ ∗ − λ ^ δ ∥ ≤ δ ^ \lVert\lambda_\delta^*-\widehat{\lambda}_\delta\rVert\le\widehat{\delta} ∥ λ δ ∗ − λ δ ∥ ≤ δ . If δ ^ = δ 2 \widehat{\delta}=\delta^2 δ = δ 2 , then the approximate hypergradient ∇ ^ F ( x ) : = ∇ x f ( x , y ∗ ( x ) ) + v ^ δ ( x , y ^ δ , λ ^ δ ) \widehat{\nabla}F(x):=\nabla_x f(x,y^*(x))+\widehat{v}_\delta(x,\widehat{y}_\delta,\widehat{\lambda}_\delta) ∇ F ( x ) := ∇ x f ( x , y ∗ ( x )) + v δ ( x , y δ , λ δ ) , has an error of
∥ ∇ ^ F ( x ) − ∇ F ( x ) ∥ ≤ O ( N δ ) \lVert\widehat{\nabla}F(x)-\nabla F(x)\rVert\le O(N\delta) ∥ ∇ F ( x ) − ∇ F ( x )∥ ≤ O ( N δ )
Proof.
v δ ( x , y δ ∗ , λ δ ∗ ) = 1 δ ∑ i = 1 N ∇ x [ g i ( x , y δ , i ∗ , y ¬ i ∗ ) + ( λ δ , i ∗ ) ⊤ e ( i ) ( x , y i ∗ ) ] − ∇ x [ g i ( x , y i ∗ , y ¬ i ∗ ) + ( λ i ∗ ) ⊤ e ( i ) ( x , y i ∗ ) ] v_\delta(x,y_\delta^*,\lambda_\delta^*)=\frac{1}{\delta}\sum_{i=1}^{N}\nabla_x\left[g_i(x,y_{\delta,i}^*,y_{\neg i}^*)+(\lambda_{\delta,i}^*)^\top e^{(i)}(x,y_i^*)\right]-\nabla_x\left[g_i(x,y_i^*,y_{\neg i}^*)+(\lambda_i^*)^\top e^{(i)}(x,y_i^*)\right] v δ ( x , y δ ∗ , λ δ ∗ ) = δ 1 i = 1 ∑ N ∇ x [ g i ( x , y δ , i ∗ , y ¬ i ∗ ) + ( λ δ , i ∗ ) ⊤ e ( i ) ( x , y i ∗ ) ] − ∇ x [ g i ( x , y i ∗ , y ¬ i ∗ ) + ( λ i ∗ ) ⊤ e ( i ) ( x , y i ∗ ) ]
∥ v ^ δ ( x , y ^ δ , λ ^ δ ) − v δ ( x , y δ ∗ , λ δ ∗ ) ∥ ≤ 1 δ ∑ i = 1 N ∥ ∇ x g i ( x , y ^ δ , i , y ^ ¬ i ) − ∇ x g i ( x , y δ , i ∗ , y ¬ i ∗ ) + ∇ x g i ( x , y i ∗ , y ¬ i ∗ ) − ∇ x g i ( x , y ^ i , y ^ ¬ i ) ∥ + 1 δ ∑ i = 1 N ∥ ∇ x [ ( λ ^ δ , i ) ⊤ e ( i ) ( x , y ^ i ) − ( λ δ , i ∗ ) ⊤ e ( i ) ( x , y i ∗ ) + ( λ i ∗ ) ⊤ e ( i ) ( x , y i ∗ ) − ( λ ^ i ) ⊤ e ( i ) ( x , y ^ i ) ] ∥ ≤ 1 δ ∑ i = 1 N ( 3 C δ ^ + 2 ∥ A ( i ) ∥ δ ) = O ( N δ ) \begin{aligned}
\lVert\widehat{v}_\delta(x,\widehat{y}_\delta,\widehat{\lambda}_\delta)-v_\delta(x,y_\delta^*,\lambda_\delta^*)\rVert
&\le\frac{1}{\delta}\sum_{i=1}^{N}\Bigl\lVert\nabla_xg_i(x,\widehat{y}_{\delta,i},\widehat{y}_{\neg i})-\nabla_xg_i(x,y_{\delta,i}^*,y_{\neg i}^*)+\nabla_xg_i(x,y_i^*,y_{\neg i}^*)-\nabla_xg_i(x,\widehat{y}_i,\widehat{y}_{\neg i})\Bigr\rVert\\
&\quad+\frac{1}{\delta}\sum_{i=1}^{N}\Bigl\lVert\nabla_x\bigl[(\widehat{\lambda}_{\delta,i})^\top e^{(i)}(x,\widehat{y}_i)-(\lambda_{\delta,i}^*)^\top e^{(i)}(x,y_i^*)+(\lambda_i^*)^\top e^{(i)}(x,y_i^*)-(\widehat{\lambda}_i)^\top e^{(i)}(x,\widehat{y}_i)\bigr]\Bigr\rVert\\
&\le\frac{1}{\delta}\sum_{i=1}^{N}(3C\widehat{\delta}+2\lVert A^{(i)}\rVert\delta) \\
&=O(N\delta)
\end{aligned} ∥ v δ ( x , y δ , λ δ ) − v δ ( x , y δ ∗ , λ δ ∗ )∥ ≤ δ 1 i = 1 ∑ N ∇ x g i ( x , y δ , i , y ¬ i ) − ∇ x g i ( x , y δ , i ∗ , y ¬ i ∗ ) + ∇ x g i ( x , y i ∗ , y ¬ i ∗ ) − ∇ x g i ( x , y i , y ¬ i ) + δ 1 i = 1 ∑ N ∇ x [ ( λ δ , i ) ⊤ e ( i ) ( x , y i ) − ( λ δ , i ∗ ) ⊤ e ( i ) ( x , y i ∗ ) + ( λ i ∗ ) ⊤ e ( i ) ( x , y i ∗ ) − ( λ i ) ⊤ e ( i ) ( x , y i ) ] ≤ δ 1 i = 1 ∑ N ( 3 C δ + 2 ∥ A ( i ) ∥ δ ) = O ( N δ )
∥ ∇ ^ F ( x ) − ∇ F ( x ) ∥ ≤ ∥ ∇ x f ( x , y ^ ) − ∇ x f ( x , y ∗ ) ∥ + ∥ v ^ δ ( x , y ^ δ , λ ^ δ ) − v δ ( x , y δ ∗ , λ δ ∗ ) ∥ + ∥ v δ ( x , y δ ∗ , λ δ ∗ ) − ( d d x y ∗ ( x ) ) T ∇ y f ( x , y ∗ ) ∥ ≤ C f δ ^ + O ( N δ ) = O ( N δ ) \begin{aligned}
\lVert\widehat{\nabla} F(x)-\nabla F(x)\rVert
&\le\lVert\nabla_x f(x,\widehat{y})-\nabla_x f(x,y^*)\rVert+\lVert\widehat{v}_{\delta}(x,\widehat{y}_{\delta},\widehat{\lambda}_{\delta})-v_{\delta}(x,y_{\delta}^*,\lambda_{\delta}^*)\rVert\\
&\quad+\left\lVert v_{\delta}(x,y_{\delta}^*,\lambda_{\delta}^*)-\left(\frac{d}{dx}y^*(x)\right)^{\mathsf{T}}\nabla_y f(x,y^*)\right\rVert\\
&\le C_f\widehat{\delta}+O(N\delta)\\
&=O(N\delta)
\end{aligned} ∥ ∇ F ( x ) − ∇ F ( x )∥ ≤ ∥ ∇ x f ( x , y ) − ∇ x f ( x , y ∗ )∥ + ∥ v δ ( x , y δ , λ δ ) − v δ ( x , y δ ∗ , λ δ ∗ )∥ + v δ ( x , y δ ∗ , λ δ ∗ ) − ( d x d y ∗ ( x ) ) T ∇ y f ( x , y ∗ ) ≤ C f δ + O ( N δ ) = O ( N δ )
Lemma D.2 (Inexact gradient descent on smooth function). Suppose the inexactness in the gradient of C F C_F C F -smooth function F F F is given by ∥ ∇ ^ F ( x ) − ∇ F ( x ) ∥ ≤ α = O ( ϵ ) \lVert\widehat{\nabla}F(x)-\nabla F(x)\rVert\le\alpha=O(\epsilon) ∥ ∇ F ( x ) − ∇ F ( x )∥ ≤ α = O ( ϵ ) . If learning rate η = 1 C F \eta=\frac{1}{C_F} η = C F 1 , then gradient descent using the inexact gradient ∇ ^ F ( x ) \widehat{\nabla}F(x) ∇ F ( x ) requires O ( ϵ − 2 ) O(\epsilon^{-2}) O ( ϵ − 2 ) to reach O ( ϵ ) O(\epsilon) O ( ϵ ) -stationarity.
Proof. Suppose ( ∀ x ∈ X ) (\forall x\in X) ( ∀ x ∈ X ) ( ∇ ^ F ( x ) = ∇ F ( x ) + τ ( x ) ) (\widehat{\nabla}F(x)=\nabla F(x)+\tau(x)) ( ∇ F ( x ) = ∇ F ( x ) + τ ( x )) where ∥ τ ( x ) ∥ ≤ O ( α ) \lVert\tau(x)\rVert\le O(\alpha) ∥ τ ( x )∥ ≤ O ( α ) . Denote x + = x − η ∇ ^ F ( x ) x^+=x-\eta\widehat{\nabla}F(x) x + = x − η ∇ F ( x ) ,
F ( x + ) ≤ F ( x ) + ∇ F ( x ) T ( x + − x ) + C F 2 ∥ x + − x ∥ 2 = F ( x ) − η ∇ F ( x ) T ∇ ^ F ( x ) + C F 2 ∥ η ∇ ^ F ( x ) ∥ 2 = F ( x ) − η ∥ ∇ F ( x ) ∥ 2 + η 2 C F 2 ∥ ∇ F ( x ) + τ ( x ) ∥ 2 − η ∇ F ( x ) T τ ( x ) = F ( x ) − η ∥ ∇ F ( x ) ∥ 2 + η 2 C F 2 ∥ ∇ F ( x ) ∥ 2 + ( η 2 C F − η ) ∇ F ( x ) T τ ( x ) + η 2 C F 2 ∥ τ ( x ) ∥ 2 = F ( x ) − 2 η − η 2 C F 2 ∥ ∇ F ( x ) ∥ 2 + ( η 2 C F − η ) ∇ F ( x ) T τ ( x ) + η 2 C F 2 ∥ τ ( x ) ∥ 2 = F ( x ) − η 2 ∥ ∇ F ( x ) ∥ 2 + 0 + η 2 ∥ τ ( x ) ∥ 2 ≤ F ( x ) − η 2 ∥ ∇ F ( x ) ∥ 2 + η 2 α 2 \begin{aligned}
F(x^+)
&\le F(x)+\nabla F(x)^{\mathsf{T}}(x^+-x)+\frac{C_F}{2}\lVert x^+-x\rVert^2\\
&=F(x)-\eta\nabla F(x)^{\mathsf{T}}\widehat{\nabla}F(x)+\frac{C_F}{2}\lVert\eta\widehat{\nabla}F(x)\rVert^2\\
&=F(x)-\eta\lVert\nabla F(x)\rVert^2+\frac{\eta^2C_F}{2}\lVert\nabla F(x)+\tau(x)\rVert^2-\eta\nabla F(x)^{\mathsf{T}}\tau(x)\\
&=F(x)-\eta\lVert\nabla F(x)\rVert^2+\frac{\eta^2C_F}{2}\lVert\nabla F(x)\rVert^2+(\eta^2C_F-\eta)\nabla F(x)^{\mathsf{T}}\tau(x)+\frac{\eta^2C_F}{2}\lVert\tau(x)\rVert^2\\
&=F(x)-\frac{2\eta-\eta^2C_F}{2}\lVert\nabla F(x)\rVert^2+(\eta^2C_F-\eta)\nabla F(x)^{\mathsf{T}}\tau(x)+\frac{\eta^2C_F}{2}\lVert\tau(x)\rVert^2\\
&=F(x)-\frac{\eta}{2}\lVert\nabla F(x)\rVert^2+0+\frac{\eta}{2}\lVert\tau(x)\rVert^2\\
&\le F(x)-\frac{\eta}{2}\lVert\nabla F(x)\rVert^2+\frac{\eta}{2}\alpha^2
\end{aligned} F ( x + ) ≤ F ( x ) + ∇ F ( x ) T ( x + − x ) + 2 C F ∥ x + − x ∥ 2 = F ( x ) − η ∇ F ( x ) T ∇ F ( x ) + 2 C F ∥ η ∇ F ( x ) ∥ 2 = F ( x ) − η ∥ ∇ F ( x ) ∥ 2 + 2 η 2 C F ∥ ∇ F ( x ) + τ ( x ) ∥ 2 − η ∇ F ( x ) T τ ( x ) = F ( x ) − η ∥ ∇ F ( x ) ∥ 2 + 2 η 2 C F ∥ ∇ F ( x ) ∥ 2 + ( η 2 C F − η ) ∇ F ( x ) T τ ( x ) + 2 η 2 C F ∥ τ ( x ) ∥ 2 = F ( x ) − 2 2 η − η 2 C F ∥ ∇ F ( x ) ∥ 2 + ( η 2 C F − η ) ∇ F ( x ) T τ ( x ) + 2 η 2 C F ∥ τ ( x ) ∥ 2 = F ( x ) − 2 η ∥ ∇ F ( x ) ∥ 2 + 0 + 2 η ∥ τ ( x ) ∥ 2 ≤ F ( x ) − 2 η ∥ ∇ F ( x ) ∥ 2 + 2 η α 2
Denote each step of inexact gradient descent by x k = x k − 1 + η ∇ ^ F ( x k − 1 ) x_k=x_{k-1}+\eta\widehat{\nabla}F(x_{k-1}) x k = x k − 1 + η ∇ F ( x k − 1 ) for k = 0 , 1 , 2 , ⋯ , T − 1 k=0,1,2,\cdots,T-1 k = 0 , 1 , 2 , ⋯ , T − 1 . Applying a telescope sum, we have
F ( x T ) − F ( x 0 ) ≤ − η 2 ∑ k = 0 T − 1 ∥ ∇ F ( x k ) ∥ 2 + η 2 ∑ k = 0 T − 1 α 2 F(x_T)-F(x_0)\le-\frac{\eta}{2}\sum_{k=0}^{T-1}\lVert\nabla F(x_k)\rVert^2+\frac{\eta}{2}\sum_{k=0}^{T-1}\alpha^2 F ( x T ) − F ( x 0 ) ≤ − 2 η k = 0 ∑ T − 1 ∥ ∇ F ( x k ) ∥ 2 + 2 η k = 0 ∑ T − 1 α 2
1 T ∑ k = 0 T − 1 ∥ ∇ F ( x k ) ∥ 2 ≤ 2 ( F ( x 0 ) − F ( x T ) ) η T + α 2 \frac{1}{T}\sum_{k=0}^{T-1}\lVert\nabla F(x_k)\rVert^2\le\frac{2(F(x_0)-F(x_T))}{\eta T}+\alpha^2 T 1 k = 0 ∑ T − 1 ∥ ∇ F ( x k ) ∥ 2 ≤ η T 2 ( F ( x 0 ) − F ( x T )) + α 2
min k = 0 , 1 , ⋯ , T − 1 ∥ ∇ F ( x k ) ∥ 2 ≤ 2 ( F ( x 0 ) − F ( x ∗ ) ) η T + α 2 ( x ∗ is the minimizer of F ) \min_{k=0,1,\cdots,T-1}\lVert\nabla F(x_k)\rVert^2\le\frac{2(F(x_0)-F(x^*))}{\eta T}+\alpha^2\qquad(x^*\text{ is the minimizer of }F) k = 0 , 1 , ⋯ , T − 1 min ∥ ∇ F ( x k ) ∥ 2 ≤ η T 2 ( F ( x 0 ) − F ( x ∗ )) + α 2 ( x ∗ is the minimizer of F )
When T = O ( ϵ − 2 ) T=O(\epsilon^{-2}) T = O ( ϵ − 2 ) and α = O ( ϵ ) \alpha=O(\epsilon) α = O ( ϵ ) , we have min k = 0 , 1 , ⋯ , T − 1 ∥ ∇ F ( x k ) ∥ ≤ O ( ϵ ) \min_{k=0,1,\cdots,T-1}\lVert\nabla F(x_k)\rVert\le O(\epsilon) min k = 0 , 1 , ⋯ , T − 1 ∥ ∇ F ( x k )∥ ≤ O ( ϵ ) .
Solving Smooth and Strongly Monotone Game
Smooth and Strongly monotone game can be solved using projected gradient descent (PGD) ([14 ] ). Let η > 0 \eta>0 η > 0 be the learning rate and let { y k } \{y_k\} { y k } be the iterates that converges to the Nash equilibrium, the update rule is given by y k + 1 = Π Y ( y k − η V ( y k ) ) y_{k+1}=\Pi_Y(y_k-\eta V(y_k)) y k + 1 = Π Y ( y k − η V ( y k )) . Here, we omit the upper-level variable for convenience. Following [14 ] , we can show a linear convergence guarantee. We extend the guarantee to the case when we only have access to approximate projection and approximate gradient operator V V V in Lemma E.2 .
Finding a Nash equilibrium is equivalent to solving a variational inequality in smooth and strongly monotone game (Lemma A.1 ). We consider solving for y ∗ y^* y ∗ in the following variational inequality
∀ y ∈ Y : ⟨ V ( y ∗ ) , y − y ∗ ⟩ ≥ 0 \forall y \in Y : \langle V(y^*),y-y^*\rangle\ge0 ∀ y ∈ Y : ⟨ V ( y ∗ ) , y − y ∗ ⟩ ≥ 0
where Y Y Y is a closed convex set and V V V is a μ \mu μ -strongly monotone and C C C -Lipschitz vector-valued function. We also define Euclidean projection as Π Y ( z ) = arg min y ∈ Y ∥ y − z ∥ 2 \Pi_Y(z)=\arg\min_{y\in Y}\lVert y-z\rVert^2 Π Y ( z ) = arg min y ∈ Y ∥ y − z ∥ 2 . Lemma A.3 shows that y ∗ y^* y ∗ satisfies the variational inequality above if and only y ∗ = Π Y ( y ∗ − η V ( y ∗ ) ) y^*=\Pi_Y(y^*-\eta V(y^*)) y ∗ = Π Y ( y ∗ − η V ( y ∗ )) for some η > 0 \eta>0 η > 0 .
Lemma E.1. Let γ = 1 − 2 μ η + η 2 C 2 ∀ x , y ∈ Y \gamma=1-2\mu\eta+\eta^2C^2\ \forall x,y\in Y γ = 1 − 2 μ η + η 2 C 2 ∀ x , y ∈ Y ,
∥ Π Y ( x − η V ( x ) ) − Π Y ( y − η V ( y ) ) ∥ 2 ≤ γ ∥ y − x ∥ 2 \lVert\Pi_Y(x-\eta V(x))-\Pi_Y(y-\eta V(y))\rVert^2\le\gamma\lVert y-x\rVert^2 ∥ Π Y ( x − η V ( x )) − Π Y ( y − η V ( y )) ∥ 2 ≤ γ ∥ y − x ∥ 2 If 0 < η < 2 μ C 2 0<\eta<\frac{2\mu}{C^2} 0 < η < C 2 2 μ , then 0 < γ < 1 0<\gamma<1 0 < γ < 1 and thus Π Y ( y − η V ( y ) ) \Pi_Y(y-\eta V(y)) Π Y ( y − η V ( y )) is a contraction mapping in y y y .
Proof.
∥ Π Y ( x − η V ( x ) ) − Π Y ( y − η V ( y ) ) ∥ 2 ≤ ∥ x − η V ( x ) − y + η V ( y ) ∥ 2 = ∥ x − y ∥ 2 − 2 η ⟨ y − x , V ( y ) − V ( x ) ⟩ + η 2 ∥ V ( y ) − V ( x ) ∥ 2 ≤ ∥ x − y ∥ 2 − 2 μ η ∥ x − y ∥ 2 + η 2 C 2 ∥ y − x ∥ 2 = ( 1 − 2 μ η + η 2 C 2 ) ∥ y − x ∥ 2 \begin{aligned}
\lVert\Pi_Y(x-\eta V(x))-\Pi_Y(y-\eta V(y))\rVert^2
&\le\lVert x-\eta V(x)-y+\eta V(y)\rVert^2\\
&=\lVert x-y\rVert^2-2\eta\langle y-x,V(y)-V(x)\rangle+\eta^2\lVert V(y)-V(x)\rVert^2\\
&\le\lVert x-y\rVert^2-2\mu\eta\lVert x-y\rVert^2+\eta^2C^2\lVert y-x\rVert^2\\
&=(1-2\mu\eta+\eta^2C^2)\lVert y-x\rVert^2
\end{aligned} ∥ Π Y ( x − η V ( x )) − Π Y ( y − η V ( y )) ∥ 2 ≤ ∥ x − η V ( x ) − y + η V ( y ) ∥ 2 = ∥ x − y ∥ 2 − 2 η ⟨ y − x , V ( y ) − V ( x )⟩ + η 2 ∥ V ( y ) − V ( x ) ∥ 2 ≤ ∥ x − y ∥ 2 − 2 μ η ∥ x − y ∥ 2 + η 2 C 2 ∥ y − x ∥ 2 = ( 1 − 2 μ η + η 2 C 2 ) ∥ y − x ∥ 2
Solving ( 1 − 2 μ η + η 2 C 2 ) < 1 (1-2\mu\eta+\eta^2C^2)<1 ( 1 − 2 μ η + η 2 C 2 ) < 1 , we get the 0 < η < 2 μ C 2 0<\eta<\frac{2\mu}{C^2} 0 < η < C 2 2 μ .
Lemma E.2. (PGD with inexact projection and gradient). Suppose for any vector z z z , we only have access to an approximate projection Π ^ Y ( z ) \widehat{\Pi}_Y(z) Π Y ( z ) such that ∥ Π ^ Y ( z ) − Π Y ( z ) ∥ ≤ ϵ 0 \lVert\widehat{\Pi}_Y(z)-\Pi_Y(z)\rVert\le\epsilon_0 ∥ Π Y ( z ) − Π Y ( z )∥ ≤ ϵ 0 for some ϵ 0 \epsilon_0 ϵ 0 . Let V ^ ( z ) \widehat{V}(z) V ( z ) be an approximate gradient operator such that ∥ V ^ ( z ) − V ( z ) ∥ ≤ δ 0 \lVert\widehat{V}(z)-V(z)\rVert\le\delta_0 ∥ V ( z ) − V ( z )∥ ≤ δ 0 for some δ 0 > 0 \delta_0>0 δ 0 > 0 . Let γ = 1 − 2 μ η + η 2 C 2 \gamma=1-2\mu\eta+\eta^2C^2 γ = 1 − 2 μ η + η 2 C 2 and 0 < η < 2 μ C 2 0<\eta<\frac{2\mu}{C^2} 0 < η < C 2 2 μ . Define an arbitrary initial iterate y 0 ∈ Y y_0\in Y y 0 ∈ Y and apply the following update rule recursively:
y t + 1 = Π ^ Y ( y t − η V ^ ( y t ) ) y_{t+1}=\widehat{\Pi}_Y(y_t-\eta\widehat{V}(y_t)) y t + 1 = Π Y ( y t − η V ( y t )) If ϵ 0 = ( 1 − γ ) ϵ 4 \epsilon_0=\frac{(1-\sqrt{\gamma})\epsilon}{4} ϵ 0 = 4 ( 1 − γ ) ϵ and δ 0 = ( 1 − γ ) ϵ 4 η \delta_0=\frac{(1-\sqrt{\gamma})\epsilon}{4\eta} δ 0 = 4 η ( 1 − γ ) ϵ , then ∥ y t − y ∗ ∥ ≤ ϵ \lVert y_t-y^*\rVert\le\epsilon ∥ y t − y ∗ ∥ ≤ ϵ when T ≥ O ( log ( 2 ∥ y 0 − y ∗ ∥ ϵ ) ) T\ge O\left(\log\left(\frac{2\lVert y_0-y^*\rVert}{\epsilon}\right)\right) T ≥ O ( log ( ϵ 2 ∥ y 0 − y ∗ ∥ ) )
Proof. For every vector z z z , Π ^ Y ( z ) = Π Y ( z ) + τ 0 \widehat{\Pi}_Y(z)=\Pi_Y(z)+\tau_0 Π Y ( z ) = Π Y ( z ) + τ 0 for some vector τ 0 \tau_0 τ 0 such that ∥ τ 0 ∥ ≤ ϵ 0 \lVert\tau_0\rVert\le\epsilon_0 ∥ τ 0 ∥ ≤ ϵ 0 ,
∥ y t + 1 − y ∗ ∥ = ∥ Π ^ Y ( y t − η V ^ ( y t ) ) − Π Y ( y t − η V ( y t ) ) + Π Y ( y t − η V ( y t ) ) − Π Y ( y ∗ − η V ( y ∗ ) ) ∥ ≤ ∥ Π Y ( y t − η V ^ ( y t ) ) − Π Y ( y t − η V ( y t ) ) ∥ + ∥ Π Y ( y t − η V ( y t ) ) − Π Y ( y ∗ − η V ( y ∗ ) ) ∥ + ϵ 0 ≤ ∥ − η V ^ ( y t ) + η V ( y t ) ∥ + ∥ Π Y ( y t − η V ( y t ) ) − Π Y ( y ∗ − η V ( y ∗ ) ) ∥ + ϵ 0 ≤ γ ∥ y t − y ∗ ∥ + η δ 0 + ϵ 0 \begin{aligned}
\lVert y_{t+1}-y^*\rVert
&=\lVert\widehat{\Pi}_Y(y_t-\eta\widehat{V}(y_t))-\Pi_Y(y_t-\eta V(y_t))+\Pi_Y(y_t-\eta V(y_t))-\Pi_Y(y^*-\eta V(y^*))\rVert\\
&\le\lVert\Pi_Y(y_t-\eta\widehat{V}(y_t))-\Pi_Y(y_t-\eta V(y_t))\rVert+\lVert\Pi_Y(y_t-\eta V(y_t))-\Pi_Y(y^*-\eta V(y^*))\rVert+\epsilon_0\\
&\le\lVert-\eta\widehat{V}(y_t)+\eta V(y_t)\rVert+\lVert\Pi_Y(y_t-\eta V(y_t))-\Pi_Y(y^*-\eta V(y^*))\rVert+\epsilon_0\\
&\le\sqrt{\gamma}\lVert y_t-y^*\rVert+\eta\delta_0+\epsilon_0
\end{aligned} ∥ y t + 1 − y ∗ ∥ = ∥ Π Y ( y t − η V ( y t )) − Π Y ( y t − η V ( y t )) + Π Y ( y t − η V ( y t )) − Π Y ( y ∗ − η V ( y ∗ ))∥ ≤ ∥ Π Y ( y t − η V ( y t )) − Π Y ( y t − η V ( y t ))∥ + ∥ Π Y ( y t − η V ( y t )) − Π Y ( y ∗ − η V ( y ∗ ))∥ + ϵ 0 ≤ ∥ − η V ( y t ) + η V ( y t )∥ + ∥ Π Y ( y t − η V ( y t )) − Π Y ( y ∗ − η V ( y ∗ ))∥ + ϵ 0 ≤ γ ∥ y t − y ∗ ∥ + η δ 0 + ϵ 0 ∥ y t + 1 − y ∗ ∥ ≤ ( γ ) t + 1 ∥ y 0 − y ∗ ∥ + ∑ k = 0 t ( γ ) k ϵ 0 + η ∑ k = 0 t ( γ ) k δ 0 = ( γ ) t + 1 ∥ y 0 − y ∗ ∥ + 1 − γ t + 1 1 − γ ϵ 0 + 1 − γ t + 1 1 − γ η δ 0 ≤ ( γ ) t + 1 ∥ y 0 − y ∗ ∥ + 1 1 − γ ( ϵ 0 + η δ 0 ) = ( γ ) t + 1 ∥ y 0 − y ∗ ∥ + ϵ 2 \begin{aligned}
\lVert y_{t+1}-y^*\rVert
&\le(\sqrt{\gamma})^{t+1}\lVert y_0-y^*\rVert+\sum_{k=0}^{t}(\sqrt{\gamma})^k\epsilon_0+\eta\sum_{k=0}^{t}(\sqrt{\gamma})^k\delta_0\\
&=(\sqrt{\gamma})^{t+1}\lVert y_0-y^*\rVert+\frac{1-\sqrt{\gamma}^{t+1}}{1-\sqrt{\gamma}}\epsilon_0+\frac{1-\sqrt{\gamma}^{t+1}}{1-\sqrt{\gamma}}\eta\delta_0\\
&\le(\sqrt{\gamma})^{t+1}\lVert y_0-y^*\rVert+\frac{1}{1-\sqrt{\gamma}}(\epsilon_0+\eta\delta_0)\\
&=(\sqrt{\gamma})^{t+1}\lVert y_0-y^*\rVert+\frac{\epsilon}{2}
\end{aligned} ∥ y t + 1 − y ∗ ∥ ≤ ( γ ) t + 1 ∥ y 0 − y ∗ ∥ + k = 0 ∑ t ( γ ) k ϵ 0 + η k = 0 ∑ t ( γ ) k δ 0 = ( γ ) t + 1 ∥ y 0 − y ∗ ∥ + 1 − γ 1 − γ t + 1 ϵ 0 + 1 − γ 1 − γ t + 1 η δ 0 ≤ ( γ ) t + 1 ∥ y 0 − y ∗ ∥ + 1 − γ 1 ( ϵ 0 + η δ 0 ) = ( γ ) t + 1 ∥ y 0 − y ∗ ∥ + 2 ϵ
Solving for ( γ ) t + 1 ∥ y 0 − y ∗ ∥ ≤ ϵ 2 (\sqrt{\gamma})^{t+1}\lVert y_0-y^*\rVert\leq\frac{\epsilon}{2} ( γ ) t + 1 ∥ y 0 − y ∗ ∥ ≤ 2 ϵ , we have
( γ ) t + 1 ∥ y 0 − y ∗ ∥ ≤ ϵ 2 (\sqrt{\gamma})^{t+1}\lVert y_0-y^*\rVert\leq\frac{\epsilon}{2} ( γ ) t + 1 ∥ y 0 − y ∗ ∥ ≤ 2 ϵ
( t + 1 ) log ( 1 γ ) + log ( 1 ∥ y 0 − y ∗ ∥ ) ≥ log ( 2 ϵ ) (t+1)\log\left(\frac{1}{\sqrt{\gamma}}\right)+\log\left(\frac{1}{\lVert y_0-y^*\rVert}\right)\geq\log\left(\frac{2}{\epsilon}\right) ( t + 1 ) log ( γ 1 ) + log ( ∥ y 0 − y ∗ ∥ 1 ) ≥ log ( ϵ 2 )
t + 1 ≥ log ( 2 ∥ y 0 − y ∗ ∥ ϵ ) / log ( 1 γ ) t+1\geq\log\left(\frac{2\lVert y_0-y^*\rVert}{\epsilon}\right)/\log\left(\frac{1}{\sqrt{\gamma}}\right) t + 1 ≥ log ( ϵ 2 ∥ y 0 − y ∗ ∥ ) / log ( γ 1 )
We leverage Lemma E.3 to verify convergence of PGD in practice.
Lemma E.3 (Sufficient condition for convergence to y ∗ y^* y ∗ ). Let r ( y ) : = y − Π Y ( y − V ( y ) ) r(y):=y-\Pi_Y(y-V(y)) r ( y ) := y − Π Y ( y − V ( y )) be the natural residual . Suppose ∥ r ( y ^ ) ∥ ≤ β \lVert r(\hat{y})\rVert\leq\beta ∥ r ( y ^ )∥ ≤ β , then ∥ y ^ − y ∗ ∥ ≤ C + 1 μ β \lVert\hat{y}-y^*\rVert\leq\frac{C+1}{\mu}\beta ∥ y ^ − y ∗ ∥ ≤ μ C + 1 β .
Proof. Let u : = y ^ − V ( y ^ ) u:=\hat{y}-V(\hat{y}) u := y ^ − V ( y ^ ) , by first-order optimality of Euclidean projection,
⟨ Π Y ( u ) − u , z − Π Y ( u ) ⟩ ≥ 0 , ∀ z ∈ Y \langle\Pi_Y(u)-u,z-\Pi_Y(u)\rangle\geq0,\quad\forall z\in Y ⟨ Π Y ( u ) − u , z − Π Y ( u )⟩ ≥ 0 , ∀ z ∈ Y ⟨ V ( y ^ ) − r ( y ^ ) , y ∗ − y ^ + r ( y ^ ) ⟩ ≥ 0 , substitute z = y ∗ \langle V(\hat{y})-r(\hat{y}),y^*-\hat{y}+r(\hat{y})\rangle\geq0,\quad\text{substitute }z=y^* ⟨ V ( y ^ ) − r ( y ^ ) , y ∗ − y ^ + r ( y ^ )⟩ ≥ 0 , substitute z = y ∗ Since y ∗ y^* y ∗ solves the variational inequality ⟨ V ( y ∗ ) , y − y ∗ ⟩ ≥ 0 , ∀ y ∈ Y \langle V(y^*),y-y^*\rangle\geq0,\ \forall y\in Y ⟨ V ( y ∗ ) , y − y ∗ ⟩ ≥ 0 , ∀ y ∈ Y , we have
⟨ V ( y ∗ ) , y ^ − r ( y ^ ) − y ∗ ⟩ ≥ 0 \langle V(y^*),\hat{y}-r(\hat{y})-y^*\rangle\geq0 ⟨ V ( y ∗ ) , y ^ − r ( y ^ ) − y ∗ ⟩ ≥ 0 Summing the inequalities together, we have
⟨ V ( y ^ ) − V ( y ∗ ) − r ( y ^ ) , y ∗ − y ^ + r ( y ^ ) ⟩ ≥ 0 \langle V(\hat{y})-V(y^*)-r(\hat{y}),y^*-\hat{y}+r(\hat{y})\rangle\geq0 ⟨ V ( y ^ ) − V ( y ∗ ) − r ( y ^ ) , y ∗ − y ^ + r ( y ^ )⟩ ≥ 0 Rearranging terms and applying Lipschitzness of V V V ,
⟨ V ( y ^ ) − V ( y ∗ ) , y ^ − y ∗ ⟩ ≤ ⟨ V ( y ^ ) − V ( y ∗ ) , r ( y ^ ) ⟩ + ⟨ r ( y ^ ) , y ^ − y ∗ ⟩ ≤ ∥ V ( y ^ ) − V ( y ∗ ) ∥ ∥ r ( y ^ ) ∥ + ∥ r ( y ^ ) ∥ ∥ y ^ − y ∗ ∥ ≤ β ( C + 1 ) ∥ y ^ − y ∗ ∥ \begin{aligned}
\langle V(\hat{y})-V(y^*),\hat{y}-y^*\rangle
&\leq\langle V(\hat{y})-V(y^*),r(\hat{y})\rangle+\langle r(\hat{y}),\hat{y}-y^*\rangle\\
&\leq\lVert V(\hat{y})-V(y^*)\rVert\lVert r(\hat{y})\rVert+\lVert r(\hat{y})\rVert\lVert\hat{y}-y^*\rVert\\
&\leq\beta(C+1)\lVert\hat{y}-y^*\rVert
\end{aligned} ⟨ V ( y ^ ) − V ( y ∗ ) , y ^ − y ∗ ⟩ ≤ ⟨ V ( y ^ ) − V ( y ∗ ) , r ( y ^ )⟩ + ⟨ r ( y ^ ) , y ^ − y ∗ ⟩ ≤ ∥ V ( y ^ ) − V ( y ∗ )∥ ∥ r ( y ^ )∥ + ∥ r ( y ^ )∥ ∥ y ^ − y ∗ ∥ ≤ β ( C + 1 ) ∥ y ^ − y ∗ ∥ By strong monotonicity and rearranging terms, we get the desired bound,
μ ∥ y ^ − y ∗ ∥ 2 ≤ ⟨ V ( y ^ ) − V ( y ∗ ) , y ^ − y ∗ ⟩ ≤ β ( C + 1 ) ∥ y ^ − y ∗ ∥ \begin{aligned}
\mu\lVert\hat{y}-y^*\rVert^2
&\leq\langle V(\hat{y})-V(y^*),\hat{y}-y^*\rangle\\
&\leq\beta(C+1)\lVert\hat{y}-y^*\rVert
\end{aligned} μ ∥ y ^ − y ∗ ∥ 2 ≤ ⟨ V ( y ^ ) − V ( y ∗ ) , y ^ − y ∗ ⟩ ≤ β ( C + 1 ) ∥ y ^ − y ∗ ∥
Lemma E.4 (Inexact dual solution oracle). Consider a μ \mu μ -strongly monotone and C C C -smooth game G : = ( x , [ N ] , { Y i } i ∈ [ N ] , { g i } i ∈ [ N ] ) \mathcal{G}:=(x,[N],\{Y_i\}_{i\in[N]},\{g_i\}_{i\in[N]}) G := ( x , [ N ] , { Y i } i ∈ [ N ] , { g i } i ∈ [ N ] ) . Given an approximate lower-level primal solution y ^ \hat{y} y ^ such that ∥ y ^ − y ∗ ∥ ≤ δ ^ \lVert\hat{y}-y^*\rVert\leq\hat{\delta} ∥ y ^ − y ∗ ∥ ≤ δ ^ , we can obtain an approximate lower-level dual solution λ ^ \hat{\lambda} λ ^ such that ∥ λ ^ − λ ∗ ∥ ≤ δ ^ \lVert\hat{\lambda}-\lambda^*\rVert\leq\hat{\delta} ∥ λ ^ − λ ∗ ∥ ≤ δ ^ using the same method in [26 ] within O ( 1 ) O(1) O ( 1 ) oracle calls.
Proof. The KKT system for the Nash equilibrium of G \mathcal{G} G is given by
[ V ( x , y ∗ ) + B ⊤ λ ∗ ϵ ( 1 ) ( x , y 1 ∗ ) ⋮ ϵ ( N ) ( x , y N ∗ ) ] = 0 \left[
\begin{array}{c}
V(x,y^*)+B^\top\lambda^*\\
\epsilon^{(1)}(x,y_1^*)\\
\vdots\\
\epsilon^{(N)}(x,y_N^*)
\end{array}
\right]=0 V ( x , y ∗ ) + B ⊤ λ ∗ ϵ ( 1 ) ( x , y 1 ∗ ) ⋮ ϵ ( N ) ( x , y N ∗ ) = 0 [ V ( x , y ∗ ) + B ⊤ λ ∗ B y ∗ ] = [ 0 − A x − b ] \left[
\begin{array}{c}
V(x,y^*)+B^\top\lambda^*\\
By^*
\end{array}
\right]
=
\left[
\begin{array}{c}
0\\
-Ax-b
\end{array}
\right] [ V ( x , y ∗ ) + B ⊤ λ ∗ B y ∗ ] = [ 0 − A x − b ]
Denote c x = − A x − b c_x=-Ax-b c x = − A x − b , ( y ∗ , λ ∗ ) (y^*,\lambda^*) ( y ∗ , λ ∗ ) are given by
[ I B T B 0 ] [ y ∗ λ ∗ ] = [ − V ( x , y ∗ ) + y ∗ c x ] \begin{bmatrix} I & B^{\mathsf{T}} \\ B & 0 \end{bmatrix}
\begin{bmatrix} y^* \\ \lambda^* \end{bmatrix}
=
\begin{bmatrix} -V(x,y^*)+y^* \\ c_x \end{bmatrix} [ I B B T 0 ] [ y ∗ λ ∗ ] = [ − V ( x , y ∗ ) + y ∗ c x ]
and the approximate lower-level solution ( y ^ , λ ^ ) (\hat{y},\hat{\lambda}) ( y ^ , λ ^ ) are given by
[ I B T B 0 ] [ y ^ λ ^ ] = [ B T λ ^ + y ^ c x ] \begin{bmatrix} I & B^{\mathsf{T}} \\ B & 0 \end{bmatrix}
\begin{bmatrix} \hat{y} \\ \hat{\lambda} \end{bmatrix}
=
\begin{bmatrix} B^{\mathsf{T}}\hat{\lambda}+\hat{y} \\ c_x \end{bmatrix} [ I B B T 0 ] [ y ^ λ ^ ] = [ B T λ ^ + y ^ c x ]
Given y ^ \hat{y} y ^ , we can obtain λ ^ \hat{\lambda} λ ^ by solving
B T λ ^ = arg min s ∈ span ( B T ) ∥ − V ( x , y ^ ) − s ∥ 2 B^{\mathsf{T}}\hat{\lambda}
=
\underset{s\in\operatorname{span}(B^{\mathsf{T}})}{\arg\min}
\left\|-V(x,\hat{y})-s\right\|^2 B T λ ^ = s ∈ span ( B T ) arg min ∥ − V ( x , y ^ ) − s ∥ 2
From the KKT system, observe that
− V ( x , y ∗ ) = B T λ ∗ = arg min s ∈ span ( B T ) ∥ − V ( x , y ∗ ) − s ∥ 2 -V(x,y^*)=B^{\mathsf{T}}\lambda^*
=
\underset{s\in\operatorname{span}(B^{\mathsf{T}})}{\arg\min}
\left\|-V(x,y^*)-s\right\|^2 − V ( x , y ∗ ) = B T λ ∗ = s ∈ span ( B T ) arg min ∥ − V ( x , y ∗ ) − s ∥ 2
∥ B T λ ^ − ( − V ( x , y ∗ ) ) ∥ = ∥ B T λ ^ − B T λ ∗ ∥ ≤ ∥ V ( x , y ∗ ) − V ( x , y ^ ) ∥ ≤ C ∥ y ^ − y ∗ ∥ ≤ C δ ^ \begin{aligned}
\left\|B^{\mathsf{T}}\hat{\lambda}-(-V(x,y^*))\right\|
&=\left\|B^{\mathsf{T}}\hat{\lambda}-B^{\mathsf{T}}\lambda^*\right\|\\
&\leq\left\|V(x,y^*)-V(x,\hat{y})\right\|\\
&\leq C\left\|\hat{y}-y^*\right\|\\
&\leq C\hat{\delta}
\end{aligned} B T λ ^ − ( − V ( x , y ∗ )) = B T λ ^ − B T λ ∗ ≤ ∥ V ( x , y ∗ ) − V ( x , y ^ ) ∥ ≤ C ∥ y ^ − y ∗ ∥ ≤ C δ ^
For some τ \tau τ such that ∥ τ ∥ ≤ ( 1 + C ) δ ^ \|\tau\|\leq(1+C)\hat{\delta} ∥ τ ∥ ≤ ( 1 + C ) δ ^ ,
[ y ^ λ ^ ] − [ y ∗ λ ∗ ] = ∥ [ I B T B 0 ] − 1 [ τ 0 ] ∥ ≤ ∥ [ I B T B 0 ] − 1 ∥ ( 1 + C ) δ ^ \begin{aligned}
\begin{bmatrix} \hat{y} \\ \hat{\lambda} \end{bmatrix}
-
\begin{bmatrix} y^* \\ \lambda^* \end{bmatrix}
=
\left\|
\begin{bmatrix} I & B^{\mathsf{T}} \\ B & 0 \end{bmatrix}^{-1}
\begin{bmatrix} \tau\\ 0 \end{bmatrix}
\right\| \\
&\leq
\left\|
\begin{bmatrix} I & B^{\mathsf{T}} \\ B & 0 \end{bmatrix}^{-1}
\right\|(1+C)\hat{\delta}
\end{aligned} [ y ^ λ ^ ] − [ y ∗ λ ∗ ] = [ I B B T 0 ] − 1 [ τ 0 ] ≤ [ I B B T 0 ] − 1 ( 1 + C ) δ ^
Unconstrained Stackelberg Game
The gradient of the potential function is
∇ y i ϕ ϵ ( x , y ) = 1 ϵ [ ∇ y i g i ( x , y i + ϵ ( y i − y i ∗ ) , y − i ) ( 1 + ϵ ) − ∇ y i g i ( x , y ) ] + 1 ϵ ∑ j ≠ i [ ∇ y i g j ( x , y j + ϵ ( y j − y j ∗ ) , y − j ) − ∇ y i g j ( x , y ) ] = 1 ϵ ∑ j = 1 N [ ∇ y i g j ( x , y j + ϵ ( y j − y j ∗ ) , y − j ) − ∇ y i g j ( x , y ) ] + g i ( x , y i + ϵ ( y i − y i ∗ ) , y − i ) \begin{aligned}
\nabla_{y_i}\phi_{\epsilon}(x,y)
={}&\frac{1}{\epsilon}\left[\nabla_{y_i}g_i(x,y_i+\epsilon(y_i-y_i^*),y_{-i})(1+\epsilon)-\nabla_{y_i}g_i(x,y)\right]\\
&+\frac{1}{\epsilon}\sum_{j\ne i}\left[\nabla_{y_i}g_j(x,y_j+\epsilon(y_j-y_j^*),y_{-j})-\nabla_{y_i}g_j(x,y)\right]\\
={}&\frac{1}{\epsilon}\sum_{j=1}^{N}\left[\nabla_{y_i}g_j(x,y_j+\epsilon(y_j-y_j^*),y_{-j})-\nabla_{y_i}g_j(x,y)\right]\\
&+g_i(x,y_i+\epsilon(y_i-y_i^*),y_{-i})
\end{aligned} ∇ y i ϕ ϵ ( x , y ) = = ϵ 1 [ ∇ y i g i ( x , y i + ϵ ( y i − y i ∗ ) , y − i ) ( 1 + ϵ ) − ∇ y i g i ( x , y ) ] + ϵ 1 j = i ∑ [ ∇ y i g j ( x , y j + ϵ ( y j − y j ∗ ) , y − j ) − ∇ y i g j ( x , y ) ] ϵ 1 j = 1 ∑ N [ ∇ y i g j ( x , y j + ϵ ( y j − y j ∗ ) , y − j ) − ∇ y i g j ( x , y ) ] + g i ( x , y i + ϵ ( y i − y i ∗ ) , y − i )
Lemma F.1 (Local strong Monotonicity and smoothness of the perturbed game G ˉ δ \bar{G}_{\delta} G ˉ δ ). Consider a closed ball around y ∗ y^* y ∗ , B ˉ R ( y ∗ ) = { y ∈ R n : ∥ y − y ∗ ∥ ≤ R } \bar{B}_R(y^*)=\{y\in\mathbb{R}^n:\|y-y^*\|\leq R\} B ˉ R ( y ∗ ) = { y ∈ R n : ∥ y − y ∗ ∥ ≤ R } . For 0 < δ < μ 2 C f 0<\delta<\frac{\mu}{2C_f} 0 < δ < 2 C f μ and 0 < R < μ − δ C f S ( N + 1 ) ( 2 + N ) 0<R<\frac{\mu-\delta C_f}{S(N+1)(2+\sqrt{N})} 0 < R < S ( N + 1 ) ( 2 + N ) μ − δ C f and 0 < ϵ < 1 0<\epsilon<1 0 < ϵ < 1 , the perturbed game G ˉ δ \bar{G}_{\delta} G ˉ δ is locally μ ˉ \bar{\mu} μ ˉ -strongly monotone:
⟨ V ˉ ( x , y ) − V ˉ ( x , z ) , y − z ⟩ ≥ μ ˉ ∥ y − z ∥ 2 , ∀ y , z ∈ B ˉ R ( y ∗ ) \langle\bar{V}(x,y)-\bar{V}(x,z),y-z\rangle\geq\bar{\mu}\|y-z\|^2,\quad\forall y,z\in\bar{B}_R(y^*) ⟨ V ˉ ( x , y ) − V ˉ ( x , z ) , y − z ⟩ ≥ μ ˉ ∥ y − z ∥ 2 , ∀ y , z ∈ B ˉ R ( y ∗ ) For any ϵ > 0 \epsilon>0 ϵ > 0 , δ > 0 \delta>0 δ > 0 and R > 0 R>0 R > 0 , G ˉ δ \bar{G}_{\delta} G ˉ δ is locally C ˉ \bar{C} C ˉ -smooth:
∥ V ˉ ( x , y ) − V ˉ ( x , z ) ∥ ≤ C ˉ ∥ y − z ∥ , ∀ y , z ∈ B ˉ R ( y ∗ ) \|\bar{V}(x,y)-\bar{V}(x,z)\|\leq\bar{C}\|y-z\|,\quad\forall y,z\in\bar{B}_R(y^*) ∥ V ˉ ( x , y ) − V ˉ ( x , z ) ∥ ≤ C ˉ ∥ y − z ∥ , ∀ y , z ∈ B ˉ R ( y ∗ ) The local strong monotonicity modulus is μ ˉ : = μ − S R N 2 ϵ − ( S R N ) ( 1 + ϵ 2 + N ) + ϵ μ − S R ϵ ( 1 + ϵ 2 + N ) − δ C f > 0 \bar{\mu}:=\mu-\frac{SRN}{2}\epsilon-(SRN)(1+\frac{\epsilon}{2}+\sqrt{N})+\epsilon\mu-SR\epsilon(1+\frac{\epsilon}{2}+\sqrt{N})-\delta C_f>0 μ ˉ := μ − 2 S R N ϵ − ( S R N ) ( 1 + 2 ϵ + N ) + ϵ μ − S R ϵ ( 1 + 2 ϵ + N ) − δ C f > 0 .
The local smoothness modulus is C ˉ : = N { [ C N + S R ( N ( 1 + ϵ 2 ) + N ) + δ C f ] + ϵ [ C + S R ( 2 + ϵ 2 ) ] } \bar{C}:=\sqrt{N}\{[C\sqrt{N}+SR(\sqrt{N}(1+\frac{\epsilon}{2})+N)+\delta C_f]+\epsilon[C+SR(2+\frac{\epsilon}{2})]\} C ˉ := N {[ C N + S R ( N ( 1 + 2 ϵ ) + N ) + δ C f ] + ϵ [ C + S R ( 2 + 2 ϵ )]}
Proof. By the Fundamental Theorem of Calculus,
∇ y i g j ( x , y j + ϵ ( y j − y j ∗ ) , y − j ) − ∇ y i g j ( x , y ) = ϵ ∫ 0 1 ∂ ∂ y j ∇ y i g j ( x , y j + t ϵ ( y j − y j ∗ ) , y − j ) ( y j − y j ∗ ) d t \nabla_{y_i}g_j(x,y_j+\epsilon(y_j-y_j^*),y_{-j})-\nabla_{y_i}g_j(x,y)=\epsilon\int_0^1\frac{\partial}{\partial y_j}\nabla_{y_i}g_j(x,y_j+t\epsilon(y_j-y_j^*),y_{-j})(y_j-y_j^*)\,dt ∇ y i g j ( x , y j + ϵ ( y j − y j ∗ ) , y − j ) − ∇ y i g j ( x , y ) = ϵ ∫ 0 1 ∂ y j ∂ ∇ y i g j ( x , y j + t ϵ ( y j − y j ∗ ) , y − j ) ( y j − y j ∗ ) d t
We can express the gradient of the perturbed cost function as
∇ y i g ˉ i ( x , y ) = ∫ 0 1 ∑ j = 1 N ∂ ∂ y j ∇ y i g j ( x , y j + t ϵ ( y j − y j ∗ ) , y − j ) ( y j − y j ∗ ) d t + ϵ ∫ 0 1 ∂ ∂ y i ∇ y i g i ( x , y i + t ϵ ( y i − y i ∗ ) , y − i ) ( y i − y i ∗ ) d t + δ ∇ y i f ( x , y ) = ∫ 0 1 ∑ j = 1 N J j i ( x , y j + t ϵ ( y j − y j ∗ ) , y − j ) T ( y j − y j ∗ ) d t + ϵ ∫ 0 1 J i i ( x , y i + t ϵ ( y i − y i ∗ ) , y − i ) ( y i − y i ∗ ) d t + δ ∇ y i f ( x , y ) \begin{aligned}
\nabla_{y_i}\bar{g}_i(x,y)
&=\int_0^1\sum_{j=1}^{N}\frac{\partial}{\partial y_j}\nabla_{y_i}g_j(x,y_j+t\epsilon(y_j-y_j^*),y_{-j})(y_j-y_j^*)\,dt\\
&\quad+\epsilon\int_0^1\frac{\partial}{\partial y_i}\nabla_{y_i}g_i(x,y_i+t\epsilon(y_i-y_i^*),y_{-i})(y_i-y_i^*)\,dt+\delta\nabla_{y_i}f(x,y)\\
&=\int_0^1\sum_{j=1}^{N}J_{ji}(x,y_j+t\epsilon(y_j-y_j^*),y_{-j})^{\mathsf{T}}(y_j-y_j^*)\,dt\\
&\quad+\epsilon\int_0^1J_{ii}(x,y_i+t\epsilon(y_i-y_i^*),y_{-i})(y_i-y_i^*)\,dt+\delta\nabla_{y_i}f(x,y)
\end{aligned} ∇ y i g ˉ i ( x , y ) = ∫ 0 1 j = 1 ∑ N ∂ y j ∂ ∇ y i g j ( x , y j + t ϵ ( y j − y j ∗ ) , y − j ) ( y j − y j ∗ ) d t + ϵ ∫ 0 1 ∂ y i ∂ ∇ y i g i ( x , y i + t ϵ ( y i − y i ∗ ) , y − i ) ( y i − y i ∗ ) d t + δ ∇ y i f ( x , y ) = ∫ 0 1 j = 1 ∑ N J j i ( x , y j + t ϵ ( y j − y j ∗ ) , y − j ) T ( y j − y j ∗ ) d t + ϵ ∫ 0 1 J ii ( x , y i + t ϵ ( y i − y i ∗ ) , y − i ) ( y i − y i ∗ ) d t + δ ∇ y i f ( x , y )
We can further expand it according to the identity A ( y ) ( y − y ∗ ) − A ( z ) ( z − y ∗ ) = A ( y ) ( y − z ) + [ A ( y ) − A ( z ) ] ( z − y ∗ ) A(y)(y-y^*)-A(z)(z-y^*)=A(y)(y-z)+[A(y)-A(z)](z-y^*) A ( y ) ( y − y ∗ ) − A ( z ) ( z − y ∗ ) = A ( y ) ( y − z ) + [ A ( y ) − A ( z )] ( z − y ∗ ) for arbitrary matrices A ( y ) A(y) A ( y ) and A ( z ) A(z) A ( z ) .
⟨ ∇ y i g ˉ i ( x , y ) − ∇ y i g ˉ i ( x , z ) , y i − z i ⟩ = ∫ 0 1 ∑ j = 1 N ⟨ y i − z i , J j i ( x , y j + t ϵ ( y j − y j ∗ ) , y − j ) T ( y j − z j ) ⟩ d t + ∫ 0 1 ∑ j = 1 N ⟨ y i − z i , [ J j i ( x , y j + t ϵ ( y j − y j ∗ ) , y − j ) T − J j i ( x , z j + t ϵ ( z j − y j ∗ ) , z − j ) T ] ( z j − y j ∗ ) ⟩ d t + ϵ ∫ 0 1 ⟨ y i − z i , J i i ( x , y i + t ϵ ( y i − y i ∗ ) , y − i ) ( y i − z i ) ⟩ d t + ϵ ∫ 0 1 ⟨ y i − z i , [ J i i ( x , y i + t ϵ ( y i − y i ∗ ) , y − i ) − J i i ( x , z i + t ϵ ( z i − y i ∗ ) , z − i ) ] ( z i − y i ∗ ) ⟩ d t + δ ⟨ y i − z i , ∇ y i f ( x , y ) − ∇ y i f ( x , z ) ⟩ \begin{aligned}
\langle\nabla_{y_i}\bar{g}_i(x,y)-\nabla_{y_i}\bar{g}_i(x,z),y_i-z_i\rangle
&=\int_0^1\sum_{j=1}^{N}\langle y_i-z_i,J_{ji}(x,y_j+t\epsilon(y_j-y_j^*),y_{-j})^{\mathsf{T}}(y_j-z_j)\rangle\,dt\\
&\quad+\int_0^1\sum_{j=1}^{N}\langle y_i-z_i,[J_{ji}(x,y_j+t\epsilon(y_j-y_j^*),y_{-j})^{\mathsf{T}}\\
&\qquad\qquad-J_{ji}(x,z_j+t\epsilon(z_j-y_j^*),z_{-j})^{\mathsf{T}}](z_j-y_j^*)\rangle\,dt\\
&\quad+\epsilon\int_0^1\langle y_i-z_i,J_{ii}(x,y_i+t\epsilon(y_i-y_i^*),y_{-i})(y_i-z_i)\rangle\,dt\\
&\quad+\epsilon\int_0^1\langle y_i-z_i,[J_{ii}(x,y_i+t\epsilon(y_i-y_i^*),y_{-i})\\
&\qquad\qquad-J_{ii}(x,z_i+t\epsilon(z_i-y_i^*),z_{-i})](z_i-y_i^*)\rangle\,dt\\
&\quad+\delta\langle y_i-z_i,\nabla_{y_i}f(x,y)-\nabla_{y_i}f(x,z)\rangle
\end{aligned} ⟨ ∇ y i g ˉ i ( x , y ) − ∇ y i g ˉ i ( x , z ) , y i − z i ⟩ = ∫ 0 1 j = 1 ∑ N ⟨ y i − z i , J j i ( x , y j + t ϵ ( y j − y j ∗ ) , y − j ) T ( y j − z j )⟩ d t + ∫ 0 1 j = 1 ∑ N ⟨ y i − z i , [ J j i ( x , y j + t ϵ ( y j − y j ∗ ) , y − j ) T − J j i ( x , z j + t ϵ ( z j − y j ∗ ) , z − j ) T ] ( z j − y j ∗ )⟩ d t + ϵ ∫ 0 1 ⟨ y i − z i , J ii ( x , y i + t ϵ ( y i − y i ∗ ) , y − i ) ( y i − z i )⟩ d t + ϵ ∫ 0 1 ⟨ y i − z i , [ J ii ( x , y i + t ϵ ( y i − y i ∗ ) , y − i ) − J ii ( x , z i + t ϵ ( z i − y i ∗ ) , z − i )] ( z i − y i ∗ )⟩ d t + δ ⟨ y i − z i , ∇ y i f ( x , y ) − ∇ y i f ( x , z )⟩
We can further decompose term (a) according to the identity
J j i ( x , y j + t ϵ ( y j − y j ∗ ) , y − j ) T = J j i ( x , y ) T + ( J j i ( x , y j + t ϵ ( y j − y j ∗ ) , y − j ) T − J j i ( x , y ) T ) J_{ji}(x,y_j+t\epsilon(y_j-y_j^*),y_{-j})^{\mathsf{T}}
=J_{ji}(x,y)^{\mathsf{T}}+\left(J_{ji}(x,y_j+t\epsilon(y_j-y_j^*),y_{-j})^{\mathsf{T}}-J_{ji}(x,y)^{\mathsf{T}}\right) J j i ( x , y j + t ϵ ( y j − y j ∗ ) , y − j ) T = J j i ( x , y ) T + ( J j i ( x , y j + t ϵ ( y j − y j ∗ ) , y − j ) T − J j i ( x , y ) T )
For all y , z ∈ B R ( y ∗ ) y,z \in B_R(y^*) y , z ∈ B R ( y ∗ )
⟨ V ˉ ( x , y ) − V ˉ ( x , z ) , y − z ⟩ = ∑ i = 1 N ⟨ ∇ y i g ˉ i ( x , y ) − ∇ y i g ˉ i ( x , z ) , y i − z i ⟩ ≥ μ ∥ y − z ∥ 2 − ∫ 0 1 ∑ i = 1 N ∑ j = 1 N ∥ y i − z i ∥ ∥ y j − z j ∥ S ∥ t ε ( y j − y j ∗ ) ∥ d t ⏟ ( a ) − ∫ 0 1 ∑ i = 1 N ∑ j = 1 N ∥ y i − z i ∥ S ( ( 1 + t ε ) ∥ y j − z j ∥ + ∥ y ¬ j − z ¬ j ∥ ) ∥ z j − y j ∗ ∥ d t ⏟ ( b ) + ε μ ∥ y − z ∥ 2 − ε ∫ 0 1 ∑ i = 1 N ∥ y i − z i ∥ S ( ( 1 + t ε ) ∥ y i − z i ∥ + ∥ y ¬ i − z ¬ i ∥ ) ∥ z i − y i ∗ ∥ d t ⏟ ( c ) − δ C f ∥ y − z ∥ 2 \begin{aligned}
\left\langle\bar{V}(x,y)-\bar{V}(x,z),y-z\right\rangle
&=\sum_{i=1}^{N}\left\langle\nabla_{y_i}\bar{g}_i(x,y)-\nabla_{y_i}\bar{g}_i(x,z),y_i-z_i\right\rangle\\
&\ge\mu\lVert y-z\rVert^2-\underbrace{\int_0^1\sum_{i=1}^{N}\sum_{j=1}^{N}\lVert y_i-z_i\rVert\lVert y_j-z_j\rVert S\lVert t\varepsilon(y_j-y_j^*)\rVert\,dt}_{(a)}\\
&\quad-\underbrace{\int_0^1\sum_{i=1}^{N}\sum_{j=1}^{N}\lVert y_i-z_i\rVert S\left((1+t\varepsilon)\lVert y_j-z_j\rVert+\lVert y_{\neg j}-z_{\neg j}\rVert\right)\lVert z_j-y_j^*\rVert\,dt}_{(b)}\\
&\quad+\varepsilon\mu\lVert y-z\rVert^2-\varepsilon\underbrace{\int_0^1\sum_{i=1}^{N}\lVert y_i-z_i\rVert S\left((1+t\varepsilon)\lVert y_i-z_i\rVert+\lVert y_{\neg i}-z_{\neg i}\rVert\right)\lVert z_i-y_i^*\rVert\,dt}_{(c)}\\
&\quad-\delta C_f\lVert y-z\rVert^2
\end{aligned} ⟨ V ˉ ( x , y ) − V ˉ ( x , z ) , y − z ⟩ = i = 1 ∑ N ⟨ ∇ y i g ˉ i ( x , y ) − ∇ y i g ˉ i ( x , z ) , y i − z i ⟩ ≥ μ ∥ y − z ∥ 2 − ( a ) ∫ 0 1 i = 1 ∑ N j = 1 ∑ N ∥ y i − z i ∥ ∥ y j − z j ∥ S ∥ tε ( y j − y j ∗ )∥ d t − ( b ) ∫ 0 1 i = 1 ∑ N j = 1 ∑ N ∥ y i − z i ∥ S ( ( 1 + tε ) ∥ y j − z j ∥ + ∥ y ¬ j − z ¬ j ∥ ) ∥ z j − y j ∗ ∥ d t + ε μ ∥ y − z ∥ 2 − ε ( c ) ∫ 0 1 i = 1 ∑ N ∥ y i − z i ∥ S ( ( 1 + tε ) ∥ y i − z i ∥ + ∥ y ¬ i − z ¬ i ∥ ) ∥ z i − y i ∗ ∥ d t − δ C f ∥ y − z ∥ 2
( a ) = S ε ∫ 0 1 ∑ i = 1 N t ∥ y i − z i ∥ ( ∑ j ∥ y j − z j ∥ ∥ y j − y j ∗ ∥ ) d t ≤ S N ε ∫ 0 1 t ∥ y − z ∥ ( ∑ j ∥ y j − z j ∥ ∥ y j − y j ∗ ∥ ) d t ≤ S N ε ∫ 0 1 t ∥ y − z ∥ ( ∑ j ∥ y j − z j ∥ ∥ y − y ∗ ∥ ) d t ≤ S N 2 ε ∥ y − z ∥ 2 ∥ y − y ∗ ∥ ≤ S R N 2 ε ∥ y − z ∥ 2 \begin{aligned}
(a)&=S\varepsilon\int_0^1\sum_{i=1}^{N}t\lVert y_i-z_i\rVert\left(\sum_j\lVert y_j-z_j\rVert\lVert y_j-y_j^*\rVert\right)\,dt\\
&\le S\sqrt{N}\varepsilon\int_0^1t\lVert y-z\rVert\left(\sum_j\lVert y_j-z_j\rVert\lVert y_j-y_j^*\rVert\right)\,dt\\
&\le S\sqrt{N}\varepsilon\int_0^1t\lVert y-z\rVert\left(\sum_j\lVert y_j-z_j\rVert\lVert y-y^*\rVert\right)\,dt\\
&\le\frac{SN}{2}\varepsilon\lVert y-z\rVert^2\lVert y-y^*\rVert\\
&\le\frac{SRN}{2}\varepsilon\lVert y-z\rVert^2
\end{aligned} ( a ) = S ε ∫ 0 1 i = 1 ∑ N t ∥ y i − z i ∥ ( j ∑ ∥ y j − z j ∥ ∥ y j − y j ∗ ∥ ) d t ≤ S N ε ∫ 0 1 t ∥ y − z ∥ ( j ∑ ∥ y j − z j ∥ ∥ y j − y j ∗ ∥ ) d t ≤ S N ε ∫ 0 1 t ∥ y − z ∥ ( j ∑ ∥ y j − z j ∥ ∥ y − y ∗ ∥ ) d t ≤ 2 S N ε ∥ y − z ∥ 2 ∥ y − y ∗ ∥ ≤ 2 S R N ε ∥ y − z ∥ 2
where the first and third inequalities are obtained using the identity 1 N ∥ y ∥ = 1 N ∑ i ∥ y i ∥ 2 ≥ ∑ i 1 N ∥ y i ∥ \frac{1}{\sqrt{N}}\lVert y\rVert=\sqrt{\frac{1}{N}\sum_i\lVert y_i\rVert^2}\ge\sum_i\frac{1}{N}\lVert y_i\rVert N 1 ∥ y ∥ = N 1 ∑ i ∥ y i ∥ 2 ≥ ∑ i N 1 ∥ y i ∥ derived from Jensen’s inequality. Using similar techniques, we can bound
( b ) ≤ S N ∥ y − z ∥ 2 ∥ z − y ∗ ∥ + S N 2 ε ∥ y − z ∥ 2 ∥ z − y ∗ ∥ + S N N ∥ y − z ∥ 2 ∥ z − y ∗ ∥ ≤ ( S R N ) ( 1 + ε 2 + N ) ∥ y − z ∥ 2 \begin{aligned}
(b)&\le SN\lVert y-z\rVert^2\lVert z-y^*\rVert+\frac{SN}{2}\varepsilon\lVert y-z\rVert^2\lVert z-y^*\rVert+SN\sqrt{N}\lVert y-z\rVert^2\lVert z-y^*\rVert\\
&\le(SRN)\left(1+\frac{\varepsilon}{2}+\sqrt{N}\right)\lVert y-z\rVert^2
\end{aligned} ( b ) ≤ S N ∥ y − z ∥ 2 ∥ z − y ∗ ∥ + 2 S N ε ∥ y − z ∥ 2 ∥ z − y ∗ ∥ + S N N ∥ y − z ∥ 2 ∥ z − y ∗ ∥ ≤ ( S R N ) ( 1 + 2 ε + N ) ∥ y − z ∥ 2
( c ) ≤ S ε ( ( 1 + ε 2 ) ∥ y − z ∥ 2 ∥ z − y ∗ ∥ + N ∥ y − z ∥ 2 ∥ z − y ∗ ∥ ) ≤ S R ε ( 1 + ε 2 + N ) ∥ y − z ∥ 2 \begin{aligned}
(c)&\le S\varepsilon\left((1+\frac{\varepsilon}{2})\lVert y-z\rVert^2\lVert z-y^*\rVert+\sqrt{N}\lVert y-z\rVert^2\lVert z-y^*\rVert\right)\\
&\le SR\varepsilon\left(1+\frac{\varepsilon}{2}+\sqrt{N}\right)\lVert y-z\rVert^2
\end{aligned} ( c ) ≤ S ε ( ( 1 + 2 ε ) ∥ y − z ∥ 2 ∥ z − y ∗ ∥ + N ∥ y − z ∥ 2 ∥ z − y ∗ ∥ ) ≤ S R ε ( 1 + 2 ε + N ) ∥ y − z ∥ 2
Putting these together, we have local strong monotonicity with modulus μ ˉ \bar{\mu} μ ˉ for sufficiently small ε \varepsilon ε , δ \delta δ and R R R
⟨ V ˉ ( x , y ) − V ˉ ( x , z ) , y − z ⟩ ≥ [ μ − S R N 2 ε − ( S R N ) ( 1 + ε 2 + N ) + ε μ − S R ε ( 1 + ε 2 + N ) − δ C f ] ∥ y − z ∥ 2 \left\langle\bar{V}(x,y)-\bar{V}(x,z),y-z\right\rangle
\ge\left[\mu-\frac{SRN}{2}\varepsilon-(SRN)\left(1+\frac{\varepsilon}{2}+\sqrt{N}\right)+\varepsilon\mu-SR\varepsilon\left(1+\frac{\varepsilon}{2}+\sqrt{N}\right)-\delta C_f\right]\lVert y-z\rVert^2 ⟨ V ˉ ( x , y ) − V ˉ ( x , z ) , y − z ⟩ ≥ [ μ − 2 S R N ε − ( S R N ) ( 1 + 2 ε + N ) + ε μ − S R ε ( 1 + 2 ε + N ) − δ C f ] ∥ y − z ∥ 2
Rearranging terms, we get μ ˉ > 0 \bar{\mu} > 0 μ ˉ > 0 when
R < ( 1 + ϵ ) μ − δ C f S N ( 1 + ϵ + N ) + S ϵ ( 1 + ϵ 2 + N ) R < \frac{(1+\epsilon)\mu-\delta C_f}{SN(1+\epsilon+\sqrt{N})+S\epsilon(1+\frac{\epsilon}{2}+\sqrt{N})} R < S N ( 1 + ϵ + N ) + S ϵ ( 1 + 2 ϵ + N ) ( 1 + ϵ ) μ − δ C f
( 1 + ϵ ) μ − δ C f S N ( 1 + ϵ + N ) + S ϵ ( 1 + ϵ 2 + N ) > μ − δ C f S N ( 1 + ϵ + N ) + S ϵ ( 1 + ϵ + N ) > μ − δ C f S ( N + 1 ) ( 2 + N ) (assume 0 < ϵ < 1 ) \frac{(1+\epsilon)\mu-\delta C_f}{SN(1+\epsilon+\sqrt{N})+S\epsilon(1+\frac{\epsilon}{2}+\sqrt{N})}
>
\frac{\mu-\delta C_f}{SN(1+\epsilon+\sqrt{N})+S\epsilon(1+\epsilon+\sqrt{N})}
>
\frac{\mu-\delta C_f}{S(N+1)(2+\sqrt{N})}
\quad\text{(assume }0<\epsilon<1\text{)} S N ( 1 + ϵ + N ) + S ϵ ( 1 + 2 ϵ + N ) ( 1 + ϵ ) μ − δ C f > S N ( 1 + ϵ + N ) + S ϵ ( 1 + ϵ + N ) μ − δ C f > S ( N + 1 ) ( 2 + N ) μ − δ C f (assume 0 < ϵ < 1 )
0 < δ < μ 2 C f 0<\delta<\frac{\mu}{2C_f} 0 < δ < 2 C f μ and 0 < R < μ − δ C f S ( N + 1 ) ( 2 + N ) 0<R<\frac{\mu-\delta C_f}{S(N+1)(2+\sqrt{N})} 0 < R < S ( N + 1 ) ( 2 + N ) μ − δ C f is a sufficient condition for μ ˉ > 0 \bar{\mu}>0 μ ˉ > 0 .
To derive the smoothness bound,
∥ ∇ y i g ˉ i ( x , y ) − ∇ y i g ˉ i ( x , z ) ∥ = ∥ ∫ 0 1 ∑ j = 1 N J j i ( x , y j + t ϵ ( y j − y j ∗ ) , y ¬ j ) ⊤ ( y j − z j ) d t + ∫ 0 1 ∑ j = 1 N [ J j i ( x , y j + t ϵ ( y j − y j ∗ ) , y ¬ j ) ⊤ − J j i ( x , z j + t ϵ ( z j − y j ∗ ) , z ¬ j ) ⊤ ] ( z j − y j ∗ ) d t + ϵ ∫ 0 1 J i i ( x , y i + t ϵ ( y i − y i ∗ ) , y ¬ i ) ( y i − z i ) d t + ϵ ∫ 0 1 [ J i i ( x , y i + t ϵ ( y i − y i ∗ ) , y ¬ i ) − J i i ( x , z i + t ϵ ( z i − y i ∗ ) , z ¬ i ) ] ( z i − y i ∗ ) d t + δ ∇ y i f ( x , y ) − ∇ y i f ( x , z ) ∥ ≤ C N ∥ y − z ∥ + S ∥ z − y ∗ ∥ [ N ( 1 + ϵ 2 ) ∥ y − z ∥ + N ∥ y − z ∥ ] + ϵ C ∥ y i − z i ∥ + ϵ S ∥ z − y ∗ ∥ [ ( 1 + ϵ 2 ) ∥ y i − z i ∥ + ∥ y − z ∥ ] + δ C f ∥ y − z ∥ ≤ [ C N + S R ( N ( 1 + ϵ 2 ) + N ) + δ C f ] ∥ y − z ∥ + ϵ [ C + S R ( 2 + ϵ 2 ) ] ∥ y − z ∥ . \begin{aligned}
\|\nabla_{y_i}\bar{g}_i(x,y)-\nabla_{y_i}\bar{g}_i(x,z)\|
={}&\left\|\int_0^1\sum_{j=1}^N J_{ji}(x,y_j+t\epsilon(y_j-y_j^*),y_{\neg j})^\top(y_j-z_j)\,dt\right.\\
&+\int_0^1\sum_{j=1}^N[J_{ji}(x,y_j+t\epsilon(y_j-y_j^*),y_{\neg j})^\top-J_{ji}(x,z_j+t\epsilon(z_j-y_j^*),z_{\neg j})^\top](z_j-y_j^*)\,dt\\
&+\epsilon\int_0^1 J_{ii}(x,y_i+t\epsilon(y_i-y_i^*),y_{\neg i})(y_i-z_i)\,dt\\
&+\epsilon\int_0^1[J_{ii}(x,y_i+t\epsilon(y_i-y_i^*),y_{\neg i})-J_{ii}(x,z_i+t\epsilon(z_i-y_i^*),z_{\neg i})](z_i-y_i^*)\,dt\\
&\left.+\delta\nabla_{y_i}f(x,y)-\nabla_{y_i}f(x,z)\right\|\\
\le{}& C\sqrt{N}\|y-z\|+S\|z-y^*\|[\sqrt{N}(1+\frac{\epsilon}{2})\|y-z\|+N\|y-z\|]\\
&+\epsilon C\|y_i-z_i\|+\epsilon S\|z-y^*\|[(1+\frac{\epsilon}{2})\|y_i-z_i\|+\|y-z\|]+\delta C_f\|y-z\|\\
\le{}&[C\sqrt{N}+SR(\sqrt{N}(1+\frac{\epsilon}{2})+N)+\delta C_f]\|y-z\|+\epsilon[C+SR(2+\frac{\epsilon}{2})]\|y-z\|.
\end{aligned} ∥ ∇ y i g ˉ i ( x , y ) − ∇ y i g ˉ i ( x , z ) ∥ = ≤ ≤ ∫ 0 1 j = 1 ∑ N J j i ( x , y j + t ϵ ( y j − y j ∗ ) , y ¬ j ) ⊤ ( y j − z j ) d t + ∫ 0 1 j = 1 ∑ N [ J j i ( x , y j + t ϵ ( y j − y j ∗ ) , y ¬ j ) ⊤ − J j i ( x , z j + t ϵ ( z j − y j ∗ ) , z ¬ j ) ⊤ ] ( z j − y j ∗ ) d t + ϵ ∫ 0 1 J ii ( x , y i + t ϵ ( y i − y i ∗ ) , y ¬ i ) ( y i − z i ) d t + ϵ ∫ 0 1 [ J ii ( x , y i + t ϵ ( y i − y i ∗ ) , y ¬ i ) − J ii ( x , z i + t ϵ ( z i − y i ∗ ) , z ¬ i )] ( z i − y i ∗ ) d t + δ ∇ y i f ( x , y ) − ∇ y i f ( x , z ) ∥ C N ∥ y − z ∥ + S ∥ z − y ∗ ∥ [ N ( 1 + 2 ϵ ) ∥ y − z ∥ + N ∥ y − z ∥ ] + ϵ C ∥ y i − z i ∥ + ϵ S ∥ z − y ∗ ∥ [( 1 + 2 ϵ ) ∥ y i − z i ∥ + ∥ y − z ∥ ] + δ C f ∥ y − z ∥ [ C N + S R ( N ( 1 + 2 ϵ ) + N ) + δ C f ] ∥ y − z ∥ + ϵ [ C + S R ( 2 + 2 ϵ )] ∥ y − z ∥.
Hence,
∥ V ˉ ( x , y ) − V ˉ ( x , z ) ∥ ≤ N { [ C N + S R ( N ( 1 + ϵ 2 ) + N ) + δ C f ] + ϵ [ C + S R ( 2 + ϵ 2 ) ] } ∥ y − z ∥ . \begin{aligned}
\|\bar{V}(x,y)-\bar{V}(x,z)\|
\le\sqrt{N}\{[C\sqrt{N}+SR(\sqrt{N}(1+\frac{\epsilon}{2})+N)+\delta C_f]+\epsilon[C+SR(2+\frac{\epsilon}{2})]\}\|y-z\|.
\end{aligned} ∥ V ˉ ( x , y ) − V ˉ ( x , z ) ∥ ≤ N {[ C N + S R ( N ( 1 + 2 ϵ ) + N ) + δ C f ] + ϵ [ C + S R ( 2 + 2 ϵ )]} ∥ y − z ∥.
We need to guarantee the existence of Nash equilibrium y ˉ δ ∗ \bar{y}_{\delta}^* y ˉ δ ∗ in the perturbed game G ˉ δ \bar{G}_{\delta} G ˉ δ , and guarantee that ∥ y ˉ δ ∗ − y ∗ ∥ \|\bar{y}_{\delta}^*-y^*\| ∥ y ˉ δ ∗ − y ∗ ∥ shrinks with decreasing δ > 0 \delta>0 δ > 0 .
Lemma F.2 (Existence of Nash equilibrium in the perturbed game). Let R R R be the radius of the closed ball B ˉ R ( y ∗ ) \bar{B}_R(y^*) B ˉ R ( y ∗ ) on which V ˉ \bar{V} V ˉ is μ ˉ \bar{\mu} μ ˉ -strongly-monotone and C ˉ \bar{C} C ˉ -Lipschitz. For 0 < δ ≤ min { μ ˉ R 2 L f , μ 2 C f } 0<\delta\le\min\{\frac{\bar{\mu}R}{2L_f},\frac{\mu}{2C_f}\} 0 < δ ≤ min { 2 L f μ ˉ R , 2 C f μ } , the perturbed game G ˉ δ \bar{G}_{\delta} G ˉ δ has a locally unique Nash equilibrium y ˉ δ ∗ \bar{y}_{\delta}^* y ˉ δ ∗ in B ˉ R ( y ∗ ) \bar{B}_R(y^*) B ˉ R ( y ∗ ) , and
∥ y ˉ δ ∗ − y ∗ ∥ ≤ 2 L f δ μ ˉ \|\bar{y}_{\delta}^*-y^*\|\le\frac{2L_f\delta}{\bar{\mu}} ∥ y ˉ δ ∗ − y ∗ ∥ ≤ μ ˉ 2 L f δ
Proof. Let γ = 1 − 2 μ ˉ η + η 2 C ˉ 2 \gamma=1-2\bar{\mu}\eta+\eta^2\bar{C}^2 γ = 1 − 2 μ ˉ η + η 2 C ˉ 2 . Denote T ( y ) = y − η V ˉ ( x , y ) T(y)=y-\eta\bar{V}(x,y) T ( y ) = y − η V ˉ ( x , y ) . From Lemm E.1, we have ∀ y , z ∈ B ˉ R ( y ∗ ) \forall y,z\in\bar{B}_R(y^*) ∀ y , z ∈ B ˉ R ( y ∗ ) ,
∥ T ( y ) − T ( z ) ∥ 2 ≤ γ ∥ y − z ∥ 2 \|T(y)-T(z)\|^2\le\gamma\|y-z\|^2 ∥ T ( y ) − T ( z ) ∥ 2 ≤ γ ∥ y − z ∥ 2
If 0 < η < 2 μ ˉ C ˉ 2 0<\eta<\frac{2\bar{\mu}}{\bar{C}^2} 0 < η < C ˉ 2 2 μ ˉ , then 0 < γ < 1 0<\gamma<1 0 < γ < 1 and thus T ( y ) T(y) T ( y ) is a contraction mapping in y y y . We want to prove that T T T maps from B ˉ R ( y ∗ ) \bar{B}_R(y^*) B ˉ R ( y ∗ ) to B ˉ R ( y ∗ ) \bar{B}_R(y^*) B ˉ R ( y ∗ ) . Since T T T is a contraction, there exists a unique fixed point y ˉ δ ∗ = T ( y ˉ δ ∗ ) \bar{y}_{\delta}^*=T(\bar{y}_{\delta}^*) y ˉ δ ∗ = T ( y ˉ δ ∗ ) by Banach fixed-point theorem. The fixed point is the Nash equilibrium by Lemma A.3. Let y ∈ B ˉ R ( y ∗ ) y \in\bar{B}_{R}(y^*) y ∈ B ˉ R ( y ∗ ) ,
∥ T ( y ) − y ∗ ∥ ≤ ∥ T ( y ) − T ( y ∗ ) ∥ + ∥ T ( y ∗ ) − y ∗ ∥ ≤ γ ∥ y − y ∗ ∥ + η ∥ V ˉ ( y ∗ ) ∥ ≤ γ R + η L f δ \begin{aligned}
\|T(y)-y^*\| \leq\|T(y)-T(y^*)\|+\|T(y^*)-y^*\| \\
&\leq\sqrt{\gamma}\|y-y^*\|+\eta\|\bar{V}(y^*)\| \\
&\leq\sqrt{\gamma}R+\eta L_f\delta
\end{aligned} ∥ T ( y ) − y ∗ ∥ ≤ ∥ T ( y ) − T ( y ∗ ) ∥ + ∥ T ( y ∗ ) − y ∗ ∥ ≤ γ ∥ y − y ∗ ∥ + η ∥ V ˉ ( y ∗ ) ∥ ≤ γ R + η L f δ
Set η = μ ˉ C ˉ 2 \eta=\frac{\bar{\mu}}{\bar{C}^2} η = C ˉ 2 μ ˉ , we have 1 − γ = 1 − 1 − μ ˉ 2 C ˉ 2 1-\sqrt{\gamma}=1-\sqrt{1-\frac{\bar{\mu}^2}{\bar{C}^2}} 1 − γ = 1 − 1 − C ˉ 2 μ ˉ 2 . We need
δ ≤ ( 1 − γ ) R η L f = ( 1 − 1 − μ ˉ 2 C ˉ 2 ) R μ ˉ C ˉ 2 L f \delta\leq\frac{(1-\sqrt{\gamma})R}{\eta L_f}
=\frac{\left(1-\sqrt{1-\frac{\bar{\mu}^2}{\bar{C}^2}}\right)R}{\frac{\bar{\mu}}{\bar{C}^2}L_f} δ ≤ η L f ( 1 − γ ) R = C ˉ 2 μ ˉ L f ( 1 − 1 − C ˉ 2 μ ˉ 2 ) R
For any concave function h : R → R h:\mathbb{R}\to\mathbb{R} h : R → R , ∀ a , b ∈ R \forall a,b\in\mathbb{R} ∀ a , b ∈ R , h ( b ) ≤ h ( a ) + ∇ h ( a ) ⊤ ( b − a ) h(b)\leq h(a)+\nabla h(a)^{\top}(b-a) h ( b ) ≤ h ( a ) + ∇ h ( a ) ⊤ ( b − a ) . Set h ( b ) = 1 − b h(b)=\sqrt{1-b} h ( b ) = 1 − b , a = 0 a=0 a = 0 , we have the identity 1 − b ≤ 1 − b 2 \sqrt{1-b}\leq1-\frac{b}{2} 1 − b ≤ 1 − 2 b . Then
( 1 − 1 − μ ˉ 2 C ˉ 2 ) R μ ˉ C ˉ 2 L f ≥ ( 1 − ( 1 − μ ˉ 2 2 C ˉ 2 ) ) R μ ˉ C ˉ 2 L f = μ ˉ R 2 L f \frac{\left(1-\sqrt{1-\frac{\bar{\mu}^2}{\bar{C}^2}}\right)R}{\frac{\bar{\mu}}{\bar{C}^2}L_f}
\geq\frac{\left(1-\left(1-\frac{\bar{\mu}^2}{2\bar{C}^2}\right)\right)R}{\frac{\bar{\mu}}{\bar{C}^2}L_f}
=\frac{\bar{\mu}R}{2L_f} C ˉ 2 μ ˉ L f ( 1 − 1 − C ˉ 2 μ ˉ 2 ) R ≥ C ˉ 2 μ ˉ L f ( 1 − ( 1 − 2 C ˉ 2 μ ˉ 2 ) ) R = 2 L f μ ˉ R
A sufficient condition for the fixed point y ˉ δ ∗ \bar{y}^*_\delta y ˉ δ ∗ to exist is δ ≤ μ ˉ R 2 L f \delta\leq\frac{\bar{\mu}R}{2L_f} δ ≤ 2 L f μ ˉ R . We can bound the distance to y ∗ y^* y ∗ in terms of δ \delta δ ,
∥ y ˉ δ ∗ − y ∗ ∥ ≤ γ ∥ y ˉ δ ∗ − y ∗ ∥ + η L f δ \|\bar{y}^*_\delta-y^*\|\leq\sqrt{\gamma}\|\bar{y}^*_\delta-y^*\|+\eta L_f\delta ∥ y ˉ δ ∗ − y ∗ ∥ ≤ γ ∥ y ˉ δ ∗ − y ∗ ∥ + η L f δ
∥ y ˉ δ ∗ − y ∗ ∥ ≤ η L f δ 1 − γ \|\bar{y}^*_\delta-y^*\|\leq\frac{\eta L_f\delta}{1-\sqrt{\gamma}} ∥ y ˉ δ ∗ − y ∗ ∥ ≤ 1 − γ η L f δ
∥ y ˉ δ ∗ − y ∗ ∥ ≤ 2 L f δ μ ˉ \|\bar{y}^*_\delta-y^*\|\leq\frac{2L_f\delta}{\bar{\mu}} ∥ y ˉ δ ∗ − y ∗ ∥ ≤ μ ˉ 2 L f δ
Lemma F.3.
∥ y ˉ δ ∗ − y ∗ δ − ( J ( x , y ∗ ) ⊤ + ϵ D ( x , y ∗ ) ) − 1 ( − ∇ y f ( x , y ∗ ) ) ∥ ≤ N μ ˉ 2 O ( δ ) \left\|\frac{\bar{y}^*_\delta-y^*}{\delta}-(J(x,y^*)^\top+\epsilon D(x,y^*))^{-1}(-\nabla_y f(x,y^*))\right\|\leq\frac{\sqrt{N}}{\bar{\mu}^2}O(\delta) δ y ˉ δ ∗ − y ∗ − ( J ( x , y ∗ ) ⊤ + ϵD ( x , y ∗ ) ) − 1 ( − ∇ y f ( x , y ∗ )) ≤ μ ˉ 2 N O ( δ )
Proof. Let y = y ˉ δ ∗ y=\bar{y}_{\delta}^{*} y = y ˉ δ ∗ , for all i ∈ [ N ] i\in[N] i ∈ [ N ] , we have
0 = ∇ y i g ˉ i ( x , y ) = ∫ 0 1 ∑ j = 1 N J j i ( x , y j + t ϵ ( y j − y j ∗ ) , y ¬ j ) ⊤ ( y j − y j ∗ ) d t + ϵ ∫ 0 1 J i i ( x , y i + t ϵ ( y i − y i ∗ ) , y ¬ i ) ( y i − y i ∗ ) d t + δ ∇ y i f ( x , y ) = ∫ 0 1 ∑ j = 1 N [ J j i ( x , y j + t ϵ ( y j − y j ∗ ) , y ¬ j ) ⊤ − J j i ( x , y j ∗ , y ¬ j ) ⊤ ] ( y j − y j ∗ ) d t + ϵ ∫ 0 1 [ J i i ( x , y i + t ϵ ( y i − y i ∗ ) , y ¬ i ) − J i i ( x , y i ∗ , y ¬ i ) ] ( y i − y i ∗ ) d t + δ ( ∇ y i f ( x , y ) − ∇ y i f ( x , y ∗ ) ) + ∑ j = 1 N [ J j i ( x , y j ∗ , y ¬ j ) ⊤ − J j i ( x , y ∗ ) ⊤ ] ( y j − y j ∗ ) + [ J i i ( x , y i ∗ , y ¬ i ) ⊤ − J i i ( x , y ∗ ) ⊤ ] ( y i − y i ∗ ) + ∑ j = 1 N J j i ( x , y ∗ ) ⊤ ( y j − y j ∗ ) + ϵ J i i ( x , y ∗ ) ( y i − y i ∗ ) + δ ∇ y i f ( x , y ∗ ) \begin{aligned}
0&=\nabla_{y_i}\bar{g}_i(x,y)\\
&=\int_0^1\sum_{j=1}^{N}J_{ji}(x,y_j+t\epsilon(y_j-y_j^*),y_{\neg j})^\top(y_j-y_j^*)\,dt\\
&\quad+\epsilon\int_0^1J_{ii}(x,y_i+t\epsilon(y_i-y_i^*),y_{\neg i})(y_i-y_i^*)\,dt+\delta\nabla_{y_i}f(x,y)\\
&=\int_0^1\sum_{j=1}^{N}\left[J_{ji}(x,y_j+t\epsilon(y_j-y_j^*),y_{\neg j})^\top-J_{ji}(x,y_j^*,y_{\neg j})^\top\right](y_j-y_j^*)\,dt\\
&\quad+\epsilon\int_0^1\left[J_{ii}(x,y_i+t\epsilon(y_i-y_i^*),y_{\neg i})-J_{ii}(x,y_i^*,y_{\neg i})\right](y_i-y_i^*)\,dt\\
&\quad+\delta\left(\nabla_{y_i}f(x,y)-\nabla_{y_i}f(x,y^*)\right)\\
&\quad+\sum_{j=1}^{N}\left[J_{ji}(x,y_j^*,y_{\neg j})^\top-J_{ji}(x,y^*)^\top\right](y_j-y_j^*)\\
&\quad+\left[J_{ii}(x,y_i^*,y_{\neg i})^\top-J_{ii}(x,y^*)^\top\right](y_i-y_i^*)\\
&\quad+\sum_{j=1}^{N}J_{ji}(x,y^*)^\top(y_j-y_j^*)+\epsilon J_{ii}(x,y^*)(y_i-y_i^*)+\delta\nabla_{y_i}f(x,y^*)
\end{aligned} 0 = ∇ y i g ˉ i ( x , y ) = ∫ 0 1 j = 1 ∑ N J j i ( x , y j + t ϵ ( y j − y j ∗ ) , y ¬ j ) ⊤ ( y j − y j ∗ ) d t + ϵ ∫ 0 1 J ii ( x , y i + t ϵ ( y i − y i ∗ ) , y ¬ i ) ( y i − y i ∗ ) d t + δ ∇ y i f ( x , y ) = ∫ 0 1 j = 1 ∑ N [ J j i ( x , y j + t ϵ ( y j − y j ∗ ) , y ¬ j ) ⊤ − J j i ( x , y j ∗ , y ¬ j ) ⊤ ] ( y j − y j ∗ ) d t + ϵ ∫ 0 1 [ J ii ( x , y i + t ϵ ( y i − y i ∗ ) , y ¬ i ) − J ii ( x , y i ∗ , y ¬ i ) ] ( y i − y i ∗ ) d t + δ ( ∇ y i f ( x , y ) − ∇ y i f ( x , y ∗ ) ) + j = 1 ∑ N [ J j i ( x , y j ∗ , y ¬ j ) ⊤ − J j i ( x , y ∗ ) ⊤ ] ( y j − y j ∗ ) + [ J ii ( x , y i ∗ , y ¬ i ) ⊤ − J ii ( x , y ∗ ) ⊤ ] ( y i − y i ∗ ) + j = 1 ∑ N J j i ( x , y ∗ ) ⊤ ( y j − y j ∗ ) + ϵ J ii ( x , y ∗ ) ( y i − y i ∗ ) + δ ∇ y i f ( x , y ∗ )
∥ a ∥ ≤ S ( 1 + ϵ 2 ) ∥ y − y ∗ ∥ 2 ∥ b ∥ ≤ S ( 1 + ϵ 2 ) ∥ y − y ∗ ∥ 2 ∥ c ∥ ≤ δ L f ∥ y − y ∗ ∥ ∥ d ∥ ≤ S ∥ y − y ∗ ∥ 2 ∥ e ∥ ≤ ϵ S ∥ y − y ∗ ∥ 2 \begin{aligned}
\|a\|&\le S\left(1+\frac{\epsilon}{2}\right)\|y-y^*\|^2\\
\|b\|&\le S\left(1+\frac{\epsilon}{2}\right)\|y-y^*\|^2\\
\|c\|&\le\delta L_f\|y-y^*\|\\
\|d\|&\le S\|y-y^*\|^2\\
\|e\|&\le\epsilon S\|y-y^*\|^2
\end{aligned} ∥ a ∥ ∥ b ∥ ∥ c ∥ ∥ d ∥ ∥ e ∥ ≤ S ( 1 + 2 ϵ ) ∥ y − y ∗ ∥ 2 ≤ S ( 1 + 2 ϵ ) ∥ y − y ∗ ∥ 2 ≤ δ L f ∥ y − y ∗ ∥ ≤ S ∥ y − y ∗ ∥ 2 ≤ ϵ S ∥ y − y ∗ ∥ 2
Combining the equations for i = 1 , 2 , ⋯ , N i=1,2,\cdots,N i = 1 , 2 , ⋯ , N ,
J ( x , y ∗ ) ⊤ ( y − y ∗ ) + ϵ D ( x , y ∗ ) ( y − y ∗ ) + δ ∇ y f ( x , y ∗ ) + e x t r a = 0 J(x,y^*)^\top(y-y^*)+\epsilon D(x,y^*)(y-y^*)+\delta\nabla_y f(x,y^*)+\mathrm{extra}=0 J ( x , y ∗ ) ⊤ ( y − y ∗ ) + ϵD ( x , y ∗ ) ( y − y ∗ ) + δ ∇ y f ( x , y ∗ ) + extra = 0
where ∥ e x t r a ∥ ≤ N S ( 1 + ϵ 2 ) ∥ y − y ∗ ∥ 2 + S ( 1 + ϵ 2 ) ∥ y − y ∗ ∥ 2 + δ L f ∥ y − y ∗ ∥ + S ∥ y − y ∗ ∥ 2 + ϵ S ∥ y − y ∗ ∥ 2 \|\mathrm{extra}\|\le\sqrt{N}S\left(1+\frac{\epsilon}{2}\right)\|y-y^*\|^2+S\left(1+\frac{\epsilon}{2}\right)\|y-y^*\|^2+\delta L_f\|y-y^*\|+S\|y-y^*\|^2+\epsilon S\|y-y^*\|^2 ∥ extra ∥ ≤ N S ( 1 + 2 ϵ ) ∥ y − y ∗ ∥ 2 + S ( 1 + 2 ϵ ) ∥ y − y ∗ ∥ 2 + δ L f ∥ y − y ∗ ∥ + S ∥ y − y ∗ ∥ 2 + ϵ S ∥ y − y ∗ ∥ 2 . Solving the equation, we have
y ˉ δ ∗ − y ∗ δ = ( J ( x , y ∗ ) ⊤ + ϵ D ( x , y ∗ ) ) − 1 ( − ∇ y f ( x , y ∗ ) − e x t r a δ ) \frac{\bar{y}_{\delta}^{*}-y^*}{\delta}=\left(J(x,y^*)^\top+\epsilon D(x,y^*)\right)^{-1}\left(-\nabla_y f(x,y^*)-\frac{\mathrm{extra}}{\delta}\right) δ y ˉ δ ∗ − y ∗ = ( J ( x , y ∗ ) ⊤ + ϵD ( x , y ∗ ) ) − 1 ( − ∇ y f ( x , y ∗ ) − δ extra )
Using Lemma F.2, we have
∥ y ˉ δ ∗ − y ∗ δ − ( J ( x , y ∗ ) ⊤ + ϵ D ( x , y ∗ ) ) − 1 ( − ∇ y f ( x , y ∗ ) ) ∥ ≤ ∥ ( J ( x , y ∗ ) ⊤ + ϵ D ( x , y ∗ ) ) − 1 ∥ ∥ e x t r a δ ∥ ≤ 1 μ ( 1 + ϵ ) [ O ( N δ μ ˉ 2 ) + O ( N δ ϵ μ ˉ 2 ) ] ≤ O ( N δ μ ˉ 2 ) (we ignore O ( δ ϵ ) ) \begin{aligned}
\left\|\frac{\bar{y}_{\delta}^{*}-y^*}{\delta}-\left(J(x,y^*)^\top+\epsilon D(x,y^*)\right)^{-1}\left(-\nabla_y f(x,y^*)\right)\right\|
\le\left\|\left(J(x,y^*)^\top+\epsilon D(x,y^*)\right)^{-1}\right\|\left\|\frac{\mathrm{extra}}{\delta}\right\| \\
&\le\frac{1}{\mu(1+\epsilon)}\left[O\left(\frac{\sqrt{N}\delta}{\bar{\mu}^{2}}\right)+O\left(\frac{\sqrt{N}\delta\epsilon}{\bar{\mu}^{2}}\right)\right] \\
&\le O\left(\frac{\sqrt{N}\delta}{\bar{\mu}^{2}}\right)\qquad\text{(we ignore }O(\delta\epsilon)\text{)}
\end{aligned} δ y ˉ δ ∗ − y ∗ − ( J ( x , y ∗ ) ⊤ + ϵD ( x , y ∗ ) ) − 1 ( − ∇ y f ( x , y ∗ ) ) ≤ ( J ( x , y ∗ ) ⊤ + ϵD ( x , y ∗ ) ) − 1 δ extra ≤ μ ( 1 + ϵ ) 1 [ O ( μ ˉ 2 N δ ) + O ( μ ˉ 2 N δ ϵ ) ] ≤ O ( μ ˉ 2 N δ ) (we ignore O ( δ ϵ ) )
Lemma F.4.
∥ ( J ( x , y ∗ ) ⊤ + ϵ D ( x , y ∗ ) ) − 1 ( − ∇ y f ( x , y ∗ ) ) − J ( x , y ∗ ) − ⊤ ( − ∇ y f ( x , y ∗ ) ) ∥ ≤ O ( ϵ ) \left\|(J(x,y^*)^\top+\epsilon D(x,y^*))^{-1}(-\nabla_y f(x,y^*))-J(x,y^*)^{-\top}(-\nabla_y f(x,y^*))\right\|\le O(\epsilon) ( J ( x , y ∗ ) ⊤ + ϵD ( x , y ∗ ) ) − 1 ( − ∇ y f ( x , y ∗ )) − J ( x , y ∗ ) − ⊤ ( − ∇ y f ( x , y ∗ )) ≤ O ( ϵ ) Proof.
∥ ( J ( x , y ∗ ) ⊤ + ϵ D ( x , y ∗ ) ) − 1 ( − ∇ y f ( x , y ∗ ) ) − J ( x , y ∗ ) − ⊤ ( − ∇ y f ( x , y ∗ ) ) ∥ = ∥ ( J ( x , y ∗ ) ⊤ + ϵ D ( x , y ∗ ) ) − 1 [ J ( x , y ∗ ) ⊤ − ( J ( x , y ∗ ) ⊤ + ϵ D ( x , y ∗ ) ) ] J ( x , y ∗ ) − ⊤ ( − ∇ y f ( x , y ∗ ) ) ∥ ≤ ϵ μ 2 ( 1 + ϵ ) L f = O ( ϵ ) \begin{aligned}
&\left\|(J(x,y^*)^\top+\epsilon D(x,y^*))^{-1}(-\nabla_y f(x,y^*))-J(x,y^*)^{-\top}(-\nabla_y f(x,y^*))\right\|\\
&=\left\|(J(x,y^*)^\top+\epsilon D(x,y^*))^{-1}\left[J(x,y^*)^\top-(J(x,y^*)^\top+\epsilon D(x,y^*))\right]J(x,y^*)^{-\top}(-\nabla_y f(x,y^*))\right\|\\
&\le\frac{\epsilon}{\mu^2(1+\epsilon)}L_f\\
&=O(\epsilon)
\end{aligned} ( J ( x , y ∗ ) ⊤ + ϵD ( x , y ∗ ) ) − 1 ( − ∇ y f ( x , y ∗ )) − J ( x , y ∗ ) − ⊤ ( − ∇ y f ( x , y ∗ )) = ( J ( x , y ∗ ) ⊤ + ϵD ( x , y ∗ ) ) − 1 [ J ( x , y ∗ ) ⊤ − ( J ( x , y ∗ ) ⊤ + ϵD ( x , y ∗ )) ] J ( x , y ∗ ) − ⊤ ( − ∇ y f ( x , y ∗ )) ≤ μ 2 ( 1 + ϵ ) ϵ L f = O ( ϵ ) The first equality uses the matrix identity A − 1 − B − 1 = A − 1 ( B − A ) B − 1 A^{-1}-B^{-1}=A^{-1}(B-A)B^{-1} A − 1 − B − 1 = A − 1 ( B − A ) B − 1 for any invertible matrices A A A and B B B of the same dimensions. □ \square □
Proof of Theorem 5.2
Proof.
v δ ( x , y ˉ δ ∗ ) = 1 δ ∑ i = 1 N [ ∇ x g i ( x , y ˉ δ , i ∗ , y − i ∗ ) − ∇ x g i ( x , y ∗ ) ] = 1 δ ∑ i = 1 N [ ∂ ∂ y i ∇ x g i ( x , y ∗ ) ( y ˉ δ , i ∗ − y i ∗ ) + H . O . T . ] = ∂ ∂ x V ( x , y ∗ ) ⊤ y ˉ δ ∗ − y ∗ δ + N H . O . T . δ \begin{aligned}
v_\delta(x,\bar y_\delta^*)=\frac{1}{\delta}\sum_{i=1}^{N}\left[\nabla_xg_i(x,\bar y_{\delta,i}^*,y_{-i}^*)-\nabla_xg_i(x,y^*)\right] \\
&=\frac{1}{\delta}\sum_{i=1}^{N}\left[\frac{\partial}{\partial y_i}\nabla_xg_i(x,y^*)(\bar y_{\delta,i}^*-y_i^*)+\mathrm{H.O.T.}\right] \\
&=\frac{\partial}{\partial x}V(x,y^*)^\top\frac{\bar y_\delta^*-y^*}{\delta}+N\frac{\mathrm{H.O.T.}}{\delta}
\end{aligned} v δ ( x , y ˉ δ ∗ ) = δ 1 i = 1 ∑ N [ ∇ x g i ( x , y ˉ δ , i ∗ , y − i ∗ ) − ∇ x g i ( x , y ∗ ) ] = δ 1 i = 1 ∑ N [ ∂ y i ∂ ∇ x g i ( x , y ∗ ) ( y ˉ δ , i ∗ − y i ∗ ) + H.O.T. ] = ∂ x ∂ V ( x , y ∗ ) ⊤ δ y ˉ δ ∗ − y ∗ + N δ H.O.T.
where ∥ H . O . T . ∥ = O ( ∥ y ˉ δ ∗ − y ∗ ∥ 2 ) ≤ O ( δ 2 μ ˉ 2 ) \|\mathrm{H.O.T.}\|=O(\|\bar y_\delta^*-y^*\|^2)\le O\left(\frac{\delta^2}{\bar\mu^2}\right) ∥ H.O.T. ∥ = O ( ∥ y ˉ δ ∗ − y ∗ ∥ 2 ) ≤ O ( μ ˉ 2 δ 2 ) by Lemma F.2. Using Lemma F.3 and Lemma F.4, we have
∥ v δ ( x , y ˉ δ ∗ ) − ∂ ∂ x V ( x , y ∗ ) ⊤ ∂ ∂ y V ( x , y ∗ ) − ⊤ ( − ∇ y f ( x , y ∗ ) ) ∥ \left\|v_\delta(x,\bar y_\delta^*)-\frac{\partial}{\partial x}V(x,y^*)^\top\frac{\partial}{\partial y}V(x,y^*)^{-\top}(-\nabla_yf(x,y^*))\right\| v δ ( x , y ˉ δ ∗ ) − ∂ x ∂ V ( x , y ∗ ) ⊤ ∂ y ∂ V ( x , y ∗ ) − ⊤ ( − ∇ y f ( x , y ∗ ))
≤ ∥ v δ ( x , y ˉ δ ∗ ) − ∂ ∂ x V ( x , y ∗ ) ⊤ ( J ( x , y ∗ ) ⊤ + ϵ D ( x , y ∗ ) ) − 1 ( − ∇ y f ( x , y ∗ ) ) ∥ + ∥ ∂ ∂ x V ( x , y ∗ ) ⊤ ( J ( x , y ∗ ) ⊤ + ϵ D ( x , y ∗ ) ) − 1 ( − ∇ y f ( x , y ∗ ) ) − ∂ ∂ x V ( x , y ∗ ) ⊤ ∂ ∂ y V ( x , y ∗ ) − ⊤ ( − ∇ y f ( x , y ∗ ) ) ∥ ≤ C ( O ( N δ μ ˉ 2 ) + O ( N δ μ ˉ 2 ) + O ( ϵ ) ) = O ( N δ μ ˉ 2 + ϵ ) \begin{aligned}
&\le\left\|v_\delta(x,\bar y_\delta^*)-\frac{\partial}{\partial x}V(x,y^*)^\top(J(x,y^*)^\top+\epsilon D(x,y^*))^{-1}(-\nabla_yf(x,y^*))\right\|\\
&\quad+\left\|\frac{\partial}{\partial x}V(x,y^*)^\top(J(x,y^*)^\top+\epsilon D(x,y^*))^{-1}(-\nabla_yf(x,y^*))\right.\\
&\qquad\left.-\frac{\partial}{\partial x}V(x,y^*)^\top\frac{\partial}{\partial y}V(x,y^*)^{-\top}(-\nabla_yf(x,y^*))\right\|\\
&\le C\left(O\left(\frac{\sqrt{N}\delta}{\bar\mu^2}\right)+O\left(\frac{N\delta}{\bar\mu^2}\right)+O(\epsilon)\right)\\
&=O\left(\frac{N\delta}{\bar\mu^2}+\epsilon\right)
\end{aligned} ≤ v δ ( x , y ˉ δ ∗ ) − ∂ x ∂ V ( x , y ∗ ) ⊤ ( J ( x , y ∗ ) ⊤ + ϵD ( x , y ∗ ) ) − 1 ( − ∇ y f ( x , y ∗ )) + ∂ x ∂ V ( x , y ∗ ) ⊤ ( J ( x , y ∗ ) ⊤ + ϵD ( x , y ∗ ) ) − 1 ( − ∇ y f ( x , y ∗ )) − ∂ x ∂ V ( x , y ∗ ) ⊤ ∂ y ∂ V ( x , y ∗ ) − ⊤ ( − ∇ y f ( x , y ∗ )) ≤ C ( O ( μ ˉ 2 N δ ) + O ( μ ˉ 2 N δ ) + O ( ϵ ) ) = O ( μ ˉ 2 N δ + ϵ )
□ \square □
Stackelberg Game with Linear Equality Constraints
By construction, G ˉ δ \bar G_\delta G ˉ δ has the same local strong monotonicity modulus μ ˉ \bar\mu μ ˉ and local smoothness modulus C ˉ \bar C C ˉ on the ball B ˉ R ˉ ( y ∗ ) \bar B_{\bar R}(y^*) B ˉ R ˉ ( y ∗ ) as the unconstrained case. We are making the following assumption.
Assumption G.1 (Constraint qualification (LICQ)). Assume the rows of the Jacobian of equality constraints and active inequality constraints are linearly independent for each player.
Lemma G.2 (Existence of Nash equilibrium in the perturbed game). Let R R R be the radius of the closed ball B ˉ R ˉ ( y ∗ ) \bar B_{\bar R}(y^*) B ˉ R ˉ ( y ∗ ) on which V ˉ \bar V V ˉ is μ ˉ \bar\mu μ ˉ -strongly-monotone and C ˉ \bar C C ˉ -Lipschitz. For 0 < δ ≤ μ ˉ R 2 L f 0<\delta\le\frac{\bar\mu R}{2L_f} 0 < δ ≤ 2 L f μ ˉ R , the perturbed game G ˉ δ \bar G_\delta G ˉ δ has a locally unique Nash equilibrium y ˉ δ ∗ \bar y_\delta^* y ˉ δ ∗ in B ˉ R ˉ ( y ∗ ) \bar B_{\bar R}(y^*) B ˉ R ˉ ( y ∗ ) , and
∥ y ˉ δ ∗ − y ∗ ∥ ≤ 2 L f δ μ ˉ \|\bar y_\delta^*-y^*\|\le\frac{2L_f\delta}{\bar\mu} ∥ y ˉ δ ∗ − y ∗ ∥ ≤ μ ˉ 2 L f δ
∥ λ ˉ δ ∗ − λ ∗ ∥ ≤ ∥ B ⊤ ∥ L f ( 1 + 2 C ˉ μ ˉ ) δ \lVert\bar{\lambda}^{*}_{\delta}-\lambda^{*}\rVert\le\lVert B^{\top}\rVert L_f\left(1+\frac{2\bar{C}}{\bar{\mu}}\right)\delta ∥ λ ˉ δ ∗ − λ ∗ ∥ ≤ ∥ B ⊤ ∥ L f ( 1 + μ ˉ 2 C ˉ ) δ
Proof. Let γ = 1 − 2 μ ˉ η + η 2 C ˉ 2 \gamma=1-2\bar{\mu}\eta+\eta^2\bar{C}^2 γ = 1 − 2 μ ˉ η + η 2 C ˉ 2 . The feasible set is Y = { ( y 1 , ⋯ , y N ) ∈ R n : e ( i ) ( x , y i ) = 0 , ∀ i ∈ [ N ] } Y=\{(y_1,\cdots,y_N)\in\mathbb{R}^n:e^{(i)}(x,y_i)=0,\ \forall i\in[N]\} Y = {( y 1 , ⋯ , y N ) ∈ R n : e ( i ) ( x , y i ) = 0 , ∀ i ∈ [ N ]} . Denote T ( y ) = Π Y ( y − η V ˉ ( x , y ) ) T(y)=\Pi_Y(y-\eta\bar{V}(x,y)) T ( y ) = Π Y ( y − η V ˉ ( x , y )) . From Lemma E.1, we have ∀ y , z ∈ B ˉ R ( y ∗ ) \forall y,z\in\bar{B}_R(y^*) ∀ y , z ∈ B ˉ R ( y ∗ ) ,
∥ T ( y ) − T ( z ) ∥ 2 ≤ γ ∥ y − z ∥ 2 \lVert T(y)-T(z)\rVert^2\le\gamma\lVert y-z\rVert^2 ∥ T ( y ) − T ( z ) ∥ 2 ≤ γ ∥ y − z ∥ 2
If 0 < η < 2 μ ˉ C ˉ 2 0<\eta<\frac{2\bar{\mu}}{\bar{C}^2} 0 < η < C ˉ 2 2 μ ˉ , then 0 < γ < 1 0<\gamma<1 0 < γ < 1 and thus T ( y ) T(y) T ( y ) is a contraction mapping in y y y . We want to prove that T T T maps from B ˉ R ( y ∗ ) \bar{B}_R(y^*) B ˉ R ( y ∗ ) to B ˉ R ( y ∗ ) \bar{B}_R(y^*) B ˉ R ( y ∗ ) . Since T T T is a contraction, there exists a unique fixed point y ˉ δ ∗ = T ( y ˉ δ ∗ ) \bar{y}^*_{\delta}=T(\bar{y}^*_{\delta}) y ˉ δ ∗ = T ( y ˉ δ ∗ ) by Banach fixed-point theorem. The fixed point is the Nash equilibrium by Lemma A.3. Denote H ( y ) = Π Y ( y − η V ( x , y ) ) H(y)=\Pi_Y(y-\eta V(x,y)) H ( y ) = Π Y ( y − η V ( x , y )) . We know that y ∗ = H ( y ∗ ) y^*=H(y^*) y ∗ = H ( y ∗ ) . KKT condition says that V ( x , y ∗ ) + B ⊤ λ ∗ = 0 V(x,y^*)+B^{\top}\lambda^*=0 V ( x , y ∗ ) + B ⊤ λ ∗ = 0 . Let y ∈ B ˉ R ( y ∗ ) y\in\bar{B}_R(y^*) y ∈ B ˉ R ( y ∗ ) ,
∥ T ( y ) − y ∗ ∥ ≤ ∥ T ( y ) − T ( y ∗ ) ∥ + ∥ T ( y ∗ ) − H ( y ∗ ) ∥ + ∥ H ( y ∗ ) − y ∗ ∥ ≤ γ ∥ y − y ∗ ∥ + η ∥ − V ( x , y ∗ ) − B ⊤ λ ∗ − δ ∇ y f ( x , y ∗ ) ∥ + 0 ≤ γ ∥ y − y ∗ ∥ + η L f δ ≤ γ R + η δ L f \begin{aligned}
\lVert T(y)-y^*\rVert
&\le\lVert T(y)-T(y^*)\rVert+\lVert T(y^*)-H(y^*)\rVert+\lVert H(y^*)-y^*\rVert\\
&\le\sqrt{\gamma}\lVert y-y^*\rVert+\eta\lVert-V(x,y^*)-B^{\top}\lambda^*-\delta\nabla_y f(x,y^*)\rVert+0\\
&\le\sqrt{\gamma}\lVert y-y^*\rVert+\eta L_f\delta\\
&\le\sqrt{\gamma}R+\eta\delta L_f
\end{aligned} ∥ T ( y ) − y ∗ ∥ ≤ ∥ T ( y ) − T ( y ∗ )∥ + ∥ T ( y ∗ ) − H ( y ∗ )∥ + ∥ H ( y ∗ ) − y ∗ ∥ ≤ γ ∥ y − y ∗ ∥ + η ∥ − V ( x , y ∗ ) − B ⊤ λ ∗ − δ ∇ y f ( x , y ∗ )∥ + 0 ≤ γ ∥ y − y ∗ ∥ + η L f δ ≤ γ R + η δ L f
Thus, we can show the existence of the Nash equilibrium y ˉ δ ∗ \bar{y}^*_{\delta} y ˉ δ ∗ and the desired bound on ∥ y ˉ δ ∗ − y ∗ ∥ \lVert\bar{y}^*_{\delta}-y^*\rVert ∥ y ˉ δ ∗ − y ∗ ∥ using the same steps in Lemma F.2.
By KKT condition,
V ˉ ( x , y ˉ δ ∗ ) + B ⊤ λ ˉ δ ∗ = 0 \bar{V}(x,\bar{y}^*_{\delta})+B^{\top}\bar{\lambda}^*_{\delta}=0 V ˉ ( x , y ˉ δ ∗ ) + B ⊤ λ ˉ δ ∗ = 0
V ( x , y ∗ ) + B ⊤ λ ∗ = 0 V(x,y^*)+B^{\top}\lambda^*=0 V ( x , y ∗ ) + B ⊤ λ ∗ = 0
subtracting these two equations and rearranging terms,
∥ λ ˉ δ ∗ − λ ∗ ∥ ≤ ∥ B ⊤ ∥ ∥ V ˉ ( x , y ˉ δ ∗ ) − V ˉ ( x , y ∗ ) + V ˉ ( x , y ∗ ) − V ( x , y ∗ ) ∥ ≤ ∥ B ⊤ ∥ ( C ˉ ∥ y ˉ δ ∗ − y ∗ ∥ + δ L f ) ≤ ∥ B ⊤ ∥ L f ( 1 + 2 C ˉ μ ˉ ) δ \begin{aligned}
\lVert\bar{\lambda}^*_{\delta}-\lambda^*\rVert
&\le\lVert B^{\top}\rVert\lVert\bar{V}(x,\bar{y}^*_{\delta})-\bar{V}(x,y^*)+\bar{V}(x,y^*)-V(x,y^*)\rVert\\
&\le\lVert B^{\top}\rVert(\bar{C}\lVert\bar{y}^*_{\delta}-y^*\rVert+\delta L_f)\\
&\le\lVert B^{\top}\rVert L_f\left(1+\frac{2\bar{C}}{\bar{\mu}}\right)\delta
\end{aligned} ∥ λ ˉ δ ∗ − λ ∗ ∥ ≤ ∥ B ⊤ ∥ ∥ V ˉ ( x , y ˉ δ ∗ ) − V ˉ ( x , y ∗ ) + V ˉ ( x , y ∗ ) − V ( x , y ∗ )∥ ≤ ∥ B ⊤ ∥ ( C ˉ ∥ y ˉ δ ∗ − y ∗ ∥ + δ L f ) ≤ ∥ B ⊤ ∥ L f ( 1 + μ ˉ 2 C ˉ ) δ
Let K ( x , y ) = [ ∂ ∂ y V ( x , y ) B ⊤ B 0 ] K(x,y)=\begin{bmatrix}\frac{\partial}{\partial y}V(x,y)&B^{\top}\\B&0\end{bmatrix} K ( x , y ) = [ ∂ y ∂ V ( x , y ) B B ⊤ 0 ] . Let K ϵ ( x , y ) = [ ∂ ∂ y V ( x , y ) + ϵ D ( x , y ) B ⊤ B 0 ] K_{\epsilon}(x,y)=\begin{bmatrix}\frac{\partial}{\partial y}V(x,y)+\epsilon D(x,y)&B^{\top}\\B&0\end{bmatrix} K ϵ ( x , y ) = [ ∂ y ∂ V ( x , y ) + ϵD ( x , y ) B B ⊤ 0 ] .
Lemma G.3. Given a matrix M = [ J B ⊤ B 0 ] M=\begin{bmatrix}J&B^{\top}\\B&0\end{bmatrix} M = [ J B B ⊤ 0 ] where J J J is invertible. Then,
∥ M − 1 ∥ ≤ ( 1 + ∥ J − 1 ∥ ∥ B ∥ ) 2 max ( ∥ J − 1 ∥ , ∥ ( B J − 1 B ⊤ ) − 1 ∥ ) \lVert M^{-1}\rVert\le(1+\lVert J^{-1}\rVert\lVert B\rVert)^2\max(\lVert J^{-1}\rVert,\lVert(BJ^{-1}B^{\top})^{-1}\rVert) ∥ M − 1 ∥ ≤ ( 1 + ∥ J − 1 ∥ ∥ B ∥ ) 2 max (∥ J − 1 ∥ , ∥( B J − 1 B ⊤ ) − 1 ∥)
Proof. Let S = − B J − 1 B ⊤ S=-BJ^{-1}B^{\top} S = − B J − 1 B ⊤ . Let L = [ I 0 − B J − 1 I ] L=\begin{bmatrix}I&0\\-BJ^{-1}&I\end{bmatrix} L = [ I − B J − 1 0 I ] and U = [ I − J − 1 B ⊤ 0 I ] U=\begin{bmatrix}I&-J^{-1}B^{\top}\\0&I\end{bmatrix} U = [ I 0 − J − 1 B ⊤ I ] . Using L L L to eliminate rows of M M M and U U U to eliminates columns of M M M , we have
M = L − 1 [ J 0 0 S ] U − 1 M=L^{-1}\begin{bmatrix}J&0\\0&S\end{bmatrix}U^{-1} M = L − 1 [ J 0 0 S ] U − 1
Taking inverse,
M − 1 = U [ J − 1 0 0 S − 1 ] L = [ I − J − 1 B ⊤ 0 I ] [ J − 1 0 0 S − 1 ] [ I 0 − B J − 1 I ] \begin{aligned}
M^{-1}=U\begin{bmatrix}J^{-1}&0\\0&S^{-1}\end{bmatrix}L \\
&=\begin{bmatrix}I&-J^{-1}B^{\top}\\0&I\end{bmatrix}
\begin{bmatrix}J^{-1}&0\\0&S^{-1}\end{bmatrix}
\begin{bmatrix}I&0\\-BJ^{-1}&I\end{bmatrix}
\end{aligned} M − 1 = U [ J − 1 0 0 S − 1 ] L = [ I 0 − J − 1 B ⊤ I ] [ J − 1 0 0 S − 1 ] [ I − B J − 1 0 I ]
Then,
∥ M − 1 ∥ ≤ ( 1 + ∥ J − 1 ∥ ∥ B ∥ ) 2 max ( ∥ J − 1 ∥ , ∥ ( B J − 1 B T ) − 1 ∥ ) \lVert M^{-1}\rVert\leq\left(1+\lVert J^{-1}\rVert\lVert B\rVert\right)^2\max\left(\lVert J^{-1}\rVert,\left\lVert\left(BJ^{-1}B^{\mathsf{T}}\right)^{-1}\right\rVert\right) ∥ M − 1 ∥ ≤ ( 1 + ∥ J − 1 ∥ ∥ B ∥ ) 2 max ( ∥ J − 1 ∥ , ( B J − 1 B T ) − 1 )
Lemma G.4.
∥ [ y ˉ δ ∗ − y ∗ δ λ ˉ δ ∗ − λ ∗ δ ] − ( K ϵ ( x , y ∗ ) ) − T ( − [ ∇ y f ( x , y ∗ ) 0 ] ) ∥ ≤ O ( N δ μ ˉ 2 ) \left\lVert
\begin{bmatrix}
\dfrac{\bar{y}_{\delta}^{*}-y^{*}}{\delta}\\
\dfrac{\bar{\lambda}_{\delta}^{*}-\lambda^{*}}{\delta}
\end{bmatrix}
-
\left(K_{\epsilon}(x,y^{*})\right)^{-\mathsf{T}}
\left(
-\begin{bmatrix}
\nabla_y f(x,y^{*})\\
0
\end{bmatrix}
\right)
\right\rVert
\leq O\left(\dfrac{\sqrt{N}\delta}{\bar{\mu}^{2}}\right) δ y ˉ δ ∗ − y ∗ δ λ ˉ δ ∗ − λ ∗ − ( K ϵ ( x , y ∗ ) ) − T ( − [ ∇ y f ( x , y ∗ ) 0 ] ) ≤ O ( μ ˉ 2 N δ )
Proof. Let y = y ˉ δ ∗ y=\bar{y}_{\delta}^{*} y = y ˉ δ ∗ and λ = λ ˉ δ ∗ \lambda=\bar{\lambda}_{\delta}^{*} λ = λ ˉ δ ∗ . for all i ∈ [ N ] i\in[N] i ∈ [ N ] , we have
0 = ∇ y i g ˉ i ( x , y ) + ( B T λ ) i = ∫ 0 1 ∑ j = 1 N J j i ( x , y j + t ϵ ( y j − y j ∗ ) , y ¬ j ) T ( y j − y j ∗ ) d t + ϵ ∫ 0 1 J i i ( x , y i + t ϵ ( y i − y i ∗ ) , y ¬ i ) ( y i − y i ∗ ) d t + δ ∇ y i f ( x , y ) + ( B T ( λ − λ ∗ ) ) i = ∫ 0 1 ∑ j = 1 N [ J j i ( x , y j + t ϵ ( y j − y j ∗ ) , y ¬ j ) T − J j i ( x , y j ∗ , y ¬ j ) T ] ( y j − y j ∗ ) d t ⏟ (a) + ϵ ∫ 0 1 [ J i i ( x , y i + t ϵ ( y i − y i ∗ ) , y ¬ i ) − J i i ( x , y i ∗ , y ¬ i ) ] ( y i − y i ∗ ) d t ⏟ (b) + δ ( ∇ y i f ( x , y ) − ∇ y i f ( x , y ∗ ) ) ⏟ (c) + ∑ j = 1 N [ J j i ( x , y j ∗ , y ¬ j ) T − J j i ( x , y ∗ ) T ] ( y j − y j ∗ ) ⏟ (d) + [ J i i ( x , y i ∗ , y ¬ i ) T − J i i ( x , y ∗ ) T ] ( y i − y i ∗ ) ⏟ (e) + ∑ j = 1 N J j i ( x , y ∗ ) T ( y j − y j ∗ ) + ϵ J i i ( x , y ∗ ) ( y i − y i ∗ ) + δ ∇ y i f ( x , y ∗ ) + ( B T ( λ − λ ∗ ) ) i \begin{aligned}
0=\nabla_{y_i}\bar{g}_i(x,y)+(B^{\mathsf{T}}\lambda)_i \\
&=\int_0^1\sum_{j=1}^{N}J_{ji}(x,y_j+t\epsilon(y_j-y_j^{*}),y_{\neg j})^{\mathsf{T}}(y_j-y_j^{*})\,dt \\
&+\epsilon\int_0^1J_{ii}(x,y_i+t\epsilon(y_i-y_i^{*}),y_{\neg i})(y_i-y_i^{*})\,dt+\delta\nabla_{y_i}f(x,y)+(B^{\mathsf{T}}(\lambda-\lambda^{*}))_i \\
&=\underbrace{\int_0^1\sum_{j=1}^{N}\left[J_{ji}(x,y_j+t\epsilon(y_j-y_j^{*}),y_{\neg j})^{\mathsf{T}}-J_{ji}(x,y_j^{*},y_{\neg j})^{\mathsf{T}}\right](y_j-y_j^{*})\,dt}_{\text{(a)}} \\
&+\epsilon\underbrace{\int_0^1\left[J_{ii}(x,y_i+t\epsilon(y_i-y_i^{*}),y_{\neg i})-J_{ii}(x,y_i^{*},y_{\neg i})\right](y_i-y_i^{*})\,dt}_{\text{(b)}} \\
&+\delta\underbrace{\left(\nabla_{y_i}f(x,y)-\nabla_{y_i}f(x,y^{*})\right)}_{\text{(c)}} \\
&+\underbrace{\sum_{j=1}^{N}\left[J_{ji}(x,y_j^{*},y_{\neg j})^{\mathsf{T}}-J_{ji}(x,y^{*})^{\mathsf{T}}\right](y_j-y_j^{*})}_{\text{(d)}} \\
&+\underbrace{\left[J_{ii}(x,y_i^{*},y_{\neg i})^{\mathsf{T}}-J_{ii}(x,y^{*})^{\mathsf{T}}\right](y_i-y_i^{*})}_{\text{(e)}} \\
&+\sum_{j=1}^{N}J_{ji}(x,y^{*})^{\mathsf{T}}(y_j-y_j^{*})+\epsilon J_{ii}(x,y^{*})(y_i-y_i^{*})+\delta\nabla_{y_i}f(x,y^{*})+(B^{\mathsf{T}}(\lambda-\lambda^{*}))_i
\end{aligned} 0 = ∇ y i g ˉ i ( x , y ) + ( B T λ ) i = ∫ 0 1 j = 1 ∑ N J j i ( x , y j + t ϵ ( y j − y j ∗ ) , y ¬ j ) T ( y j − y j ∗ ) d t + ϵ ∫ 0 1 J ii ( x , y i + t ϵ ( y i − y i ∗ ) , y ¬ i ) ( y i − y i ∗ ) d t + δ ∇ y i f ( x , y ) + ( B T ( λ − λ ∗ ) ) i = (a) ∫ 0 1 j = 1 ∑ N [ J j i ( x , y j + t ϵ ( y j − y j ∗ ) , y ¬ j ) T − J j i ( x , y j ∗ , y ¬ j ) T ] ( y j − y j ∗ ) d t + ϵ (b) ∫ 0 1 [ J ii ( x , y i + t ϵ ( y i − y i ∗ ) , y ¬ i ) − J ii ( x , y i ∗ , y ¬ i ) ] ( y i − y i ∗ ) d t + δ (c) ( ∇ y i f ( x , y ) − ∇ y i f ( x , y ∗ ) ) + (d) j = 1 ∑ N [ J j i ( x , y j ∗ , y ¬ j ) T − J j i ( x , y ∗ ) T ] ( y j − y j ∗ ) + (e) [ J ii ( x , y i ∗ , y ¬ i ) T − J ii ( x , y ∗ ) T ] ( y i − y i ∗ ) + j = 1 ∑ N J j i ( x , y ∗ ) T ( y j − y j ∗ ) + ϵ J ii ( x , y ∗ ) ( y i − y i ∗ ) + δ ∇ y i f ( x , y ∗ ) + ( B T ( λ − λ ∗ ) ) i
∥ a ∥ ≤ S ( 1 + ϵ 2 ) ∥ y − y ∗ ∥ 2 \lVert a\rVert\leq S\left(1+\dfrac{\epsilon}{2}\right)\lVert y-y^{*}\rVert^{2} ∥ a ∥ ≤ S ( 1 + 2 ϵ ) ∥ y − y ∗ ∥ 2
∥ b ∥ ≤ S ( 1 + ϵ 2 ) ∥ y − y ∗ ∥ 2 \lVert b\rVert\leq S\left(1+\dfrac{\epsilon}{2}\right)\lVert y-y^{*}\rVert^{2} ∥ b ∥ ≤ S ( 1 + 2 ϵ ) ∥ y − y ∗ ∥ 2
∥ c ∥ ≤ δ L f ∥ y − y ∗ ∥ \lVert c\rVert\leq\delta L_f\lVert y-y^{*}\rVert ∥ c ∥ ≤ δ L f ∥ y − y ∗ ∥
∥ d ∥ ≤ S ∥ y − y ∗ ∥ 2 \lVert d\rVert\leq S\lVert y-y^{*}\rVert^{2} ∥ d ∥ ≤ S ∥ y − y ∗ ∥ 2
∥ e ∥ ≤ ϵ S ∥ y − y ∗ ∥ 2 \lVert e\rVert\leq\epsilon S\lVert y-y^{*}\rVert^{2} ∥ e ∥ ≤ ϵ S ∥ y − y ∗ ∥ 2
Combining the equations for i = 1 , 2 , ⋯ , N i=1,2,\cdots,N i = 1 , 2 , ⋯ , N ,
J ( x , y ∗ ) T ( y − y ∗ ) + ϵ D ( x , y ∗ ) ( y − y ∗ ) + δ ∇ y f ( x , y ∗ ) + B T ( λ − λ ∗ ) + e x t r a = 0 J(x,y^{*})^{\mathsf{T}}(y-y^{*})+\epsilon D(x,y^{*})(y-y^{*})+\delta\nabla_y f(x,y^{*})+B^{\mathsf{T}}(\lambda-\lambda^{*})+\mathrm{extra}=0 J ( x , y ∗ ) T ( y − y ∗ ) + ϵD ( x , y ∗ ) ( y − y ∗ ) + δ ∇ y f ( x , y ∗ ) + B T ( λ − λ ∗ ) + extra = 0
where ∥ e x t r a ∥ ≤ N ( S ( 1 + ϵ 2 ) ∥ y − y ∗ ∥ 2 + S ( 1 + ϵ 2 ) ∥ y − y ∗ ∥ 2 ) + δ L f ∥ y − y ∗ ∥ + S ∥ y − y ∗ ∥ 2 + ϵ S ∥ y − y ∗ ∥ 2 \lVert\mathrm{extra}\rVert\leq\sqrt{N}\left(S\left(1+\dfrac{\epsilon}{2}\right)\lVert y-y^{*}\rVert^{2}+S\left(1+\dfrac{\epsilon}{2}\right)\lVert y-y^{*}\rVert^{2}\right)+\delta L_f\lVert y-y^{*}\rVert+S\lVert y-y^{*}\rVert^{2}+\epsilon S\lVert y-y^{*}\rVert^{2} ∥ extra ∥ ≤ N ( S ( 1 + 2 ϵ ) ∥ y − y ∗ ∥ 2 + S ( 1 + 2 ϵ ) ∥ y − y ∗ ∥ 2 ) + δ L f ∥ y − y ∗ ∥ + S ∥ y − y ∗ ∥ 2 + ϵ S ∥ y − y ∗ ∥ 2
Solving the equation with the constraint that B ( y − y ∗ ) = 0 B(y-y^{*})=0 B ( y − y ∗ ) = 0 , we have
[ y ˉ δ ∗ − y ∗ δ λ ˉ δ ∗ − λ ∗ δ ] = ( K ϵ ( x , y ∗ ) ) − T ( − [ ∇ y f ( x , y ∗ ) + e x t r a δ 0 ] ) \begin{bmatrix}
\dfrac{\bar{y}_{\delta}^{*}-y^{*}}{\delta}\\
\dfrac{\bar{\lambda}_{\delta}^{*}-\lambda^{*}}{\delta}
\end{bmatrix}
=
\left(K_{\epsilon}(x,y^{*})\right)^{-\mathsf{T}}
\left(
-\begin{bmatrix}
\nabla_y f(x,y^{*})+\dfrac{\mathrm{extra}}{\delta}\\
0
\end{bmatrix}
\right) δ y ˉ δ ∗ − y ∗ δ λ ˉ δ ∗ − λ ∗ = ( K ϵ ( x , y ∗ ) ) − T ( − [ ∇ y f ( x , y ∗ ) + δ extra 0 ] )
Using Lemma G.2, we have
∥ [ y ˉ δ − y ∗ δ λ ˉ δ ∗ − λ ∗ δ ] − ( K ϵ ( x , y ∗ ) ) − ⊤ ( − [ ∇ y f ( x , y ∗ ) 0 ] ) ∥ ≤ ∥ K ϵ ( x , y ∗ ) − ⊤ ∥ ∥ e x t r a δ ∥ ≤ ( 1 + 1 μ ( 1 + ϵ ) ∥ B ∥ ) 2 max ( 1 μ ( 1 + ϵ ) , C 2 μ ( 1 + ϵ ) σ min ( B ) 2 ) [ O ( N δ μ ˉ 2 ) + O ( N δ ϵ μ ˉ 2 ) ] ≤ O ( N δ μ ˉ 2 ) \begin{aligned}
\left\|
\begin{bmatrix}
\frac{\bar y_\delta-y^*}{\delta}\\
\frac{\bar\lambda_\delta^*-\lambda^*}{\delta}
\end{bmatrix}
-
(K_\epsilon(x,y^*))^{-\top}
\left(
-\begin{bmatrix}
\nabla_y f(x,y^*)\\
0
\end{bmatrix}
\right)
\right\|
\le\|K_\epsilon(x,y^*)^{-\top}\|\left\|\frac{\mathrm{extra}}{\delta}\right\| \\
&\le\left(1+\frac{1}{\mu(1+\epsilon)}\|B\|\right)^2
\max\left(\frac{1}{\mu(1+\epsilon)},\frac{C^2}{\mu(1+\epsilon)\sigma_{\min}(B)^2}\right)
\left[O\left(\frac{\sqrt{N}\delta}{\bar\mu^2}\right)+O\left(\frac{\sqrt{N}\delta\epsilon}{\bar\mu^2}\right)\right] \\
&\le O\left(\frac{\sqrt{N}\delta}{\bar\mu^2}\right)
\end{aligned} [ δ y ˉ δ − y ∗ δ λ ˉ δ ∗ − λ ∗ ] − ( K ϵ ( x , y ∗ ) ) − ⊤ ( − [ ∇ y f ( x , y ∗ ) 0 ] ) ≤ ∥ K ϵ ( x , y ∗ ) − ⊤ ∥ δ extra ≤ ( 1 + μ ( 1 + ϵ ) 1 ∥ B ∥ ) 2 max ( μ ( 1 + ϵ ) 1 , μ ( 1 + ϵ ) σ m i n ( B ) 2 C 2 ) [ O ( μ ˉ 2 N δ ) + O ( μ ˉ 2 N δ ϵ ) ] ≤ O ( μ ˉ 2 N δ )
(we ignore O ( δ ϵ ) O(\delta\epsilon) O ( δ ϵ ) )
Denote G = J ( x , y ∗ ) + ϵ D ( x , y ∗ ) G=J(x,y^*)+\epsilon D(x,y^*) G = J ( x , y ∗ ) + ϵD ( x , y ∗ ) . The second inequality uses Lemma G.3 and the fact that d ⊤ G d ≥ ( μ + ϵ μ ) ∥ d ∥ 2 d^\top Gd\ge(\mu+\epsilon\mu)\|d\|^2 d ⊤ G d ≥ ( μ + ϵ μ ) ∥ d ∥ 2 . To bound ∥ ( B G − ⊤ B ⊤ ) − 1 ∥ \|(BG^{-\top}B^\top)^{-1}\| ∥ ( B G − ⊤ B ⊤ ) − 1 ∥ , we use Lemma A.7 and Lemma A.5:
∥ ( B G − ⊤ B ⊤ ) − 1 ∥ ≤ 1 σ min ( B G − ⊤ B ⊤ ) ≤ C 2 μ ( 1 + ϵ ) σ min ( B ) 2 \begin{aligned}
\|(BG^{-\top}B^\top)^{-1}\|\le\frac{1}{\sigma_{\min}(BG^{-\top}B^\top)} \\
&\le\frac{C^2}{\mu(1+\epsilon)\sigma_{\min}(B)^2}
\end{aligned} ∥ ( B G − ⊤ B ⊤ ) − 1 ∥ ≤ σ m i n ( B G − ⊤ B ⊤ ) 1 ≤ μ ( 1 + ϵ ) σ m i n ( B ) 2 C 2
Lemma G.5.
∥ K ϵ ( x , y ∗ ) − ⊤ ( − ∇ y f ( x , y ∗ ) ) − K ( x , y ∗ ) − ⊤ ( − ∇ y f ( x , y ∗ ) ) ∥ ≤ O ( ϵ ) \|K_\epsilon(x,y^*)^{-\top}(-\nabla_yf(x,y^*))-K(x,y^*)^{-\top}(-\nabla_yf(x,y^*))\|\le O(\epsilon) ∥ K ϵ ( x , y ∗ ) − ⊤ ( − ∇ y f ( x , y ∗ )) − K ( x , y ∗ ) − ⊤ ( − ∇ y f ( x , y ∗ )) ∥ ≤ O ( ϵ )
Proof.
∥ K ϵ ( x , y ∗ ) − ⊤ [ − ∇ y f ( x , y ∗ ) 0 ] − K ( x , y ∗ ) − ⊤ [ − ∇ y f ( x , y ∗ ) 0 ] ∥ = ∥ K ϵ ( x , y ∗ ) − ⊤ [ K ( x , y ∗ ) ⊤ − K ϵ ( x , y ∗ ) ⊤ ] K ( x , y ∗ ) − ⊤ ( − ∇ y f ( x , y ∗ ) ) ∥ ≤ O ( L f ϵ ) = O ( ϵ ) \begin{aligned}
\left\|K_\epsilon(x,y^*)^{-\top}
\begin{bmatrix}
-\nabla_yf(x,y^*)\\
0
\end{bmatrix}
-
K(x,y^*)^{-\top}
\begin{bmatrix}
-\nabla_yf(x,y^*)\\
0
\end{bmatrix}\right\| \\
&=\left\|K_\epsilon(x,y^*)^{-\top}
\left[K(x,y^*)^\top-K_\epsilon(x,y^*)^\top\right]
K(x,y^*)^{-\top}
\left(-\nabla_yf(x,y^*)\right)\right\| \\
&\le O(L_f\epsilon) \\
&=O(\epsilon)
\end{aligned} K ϵ ( x , y ∗ ) − ⊤ [ − ∇ y f ( x , y ∗ ) 0 ] − K ( x , y ∗ ) − ⊤ [ − ∇ y f ( x , y ∗ ) 0 ] = K ϵ ( x , y ∗ ) − ⊤ [ K ( x , y ∗ ) ⊤ − K ϵ ( x , y ∗ ) ⊤ ] K ( x , y ∗ ) − ⊤ ( − ∇ y f ( x , y ∗ ) ) ≤ O ( L f ϵ ) = O ( ϵ )
The first equality uses the matrix identity A − 1 − B − 1 = A − 1 ( B − A ) B − 1 A^{-1}-B^{-1}=A^{-1}(B-A)B^{-1} A − 1 − B − 1 = A − 1 ( B − A ) B − 1 for any invertible matrices A A A and B B B of the same dimensions. The second inequality uses lemma G.3 which says ∥ K ϵ ( x , y ∗ ) − 1 ∥ = O ( 1 ) \|K_\epsilon(x,y^*)^{-1}\|=O(1) ∥ K ϵ ( x , y ∗ ) − 1 ∥ = O ( 1 ) and ∥ K ( x , y ∗ ) − 1 ∥ = O ( 1 ) \|K(x,y^*)^{-1}\|=O(1) ∥ K ( x , y ∗ ) − 1 ∥ = O ( 1 ) as ϵ \epsilon ϵ decreases.
Proof of Theorem G.7
Proof.
v δ ( x , y ˉ δ ∗ , λ ˉ δ ∗ ) = 1 δ ∑ i = 1 N [ ∇ x g i ( x , y ˉ δ , i , y ¬ i ∗ ) − ∇ x g i ( x , y ∗ ) + ∂ ∂ x e ( i ) ( x , y ∗ ) ⊤ ( λ ˉ δ ∗ − λ ∗ ) ] = 1 δ ∑ i = 1 N [ ∂ ∂ y i ∇ x g i ( x , y ∗ ) ( y ˉ δ , i − y i ∗ ) + H . O . T . + ∂ ∂ x e ( i ) ( x , y ∗ ) ⊤ ( λ ˉ δ ∗ − λ ∗ ) ] = ∂ ∂ x V ( x , y ∗ ) ⊤ y ˉ δ ∗ − y ∗ δ + N H . O . T . δ + A ⊤ λ ˉ δ ∗ − λ ∗ δ \begin{aligned}
v_\delta(x,\bar y_\delta^*,\bar\lambda_\delta^*)=
\frac{1}{\delta}\sum_{i=1}^{N}
\left[
\nabla_xg_i(x,\bar y_{\delta,i},y^*_{\neg i})-\nabla_xg_i(x,y^*)
+\frac{\partial}{\partial x}e^{(i)}(x,y^*)^\top(\bar\lambda_\delta^*-\lambda^*)
\right] \\
&=\frac{1}{\delta}\sum_{i=1}^{N}
\left[
\frac{\partial}{\partial y_i}\nabla_xg_i(x,y^*)(\bar y_{\delta,i}-y_i^*)
+\mathrm{H.O.T.}
+\frac{\partial}{\partial x}e^{(i)}(x,y^*)^\top(\bar\lambda_\delta^*-\lambda^*)
\right] \\
&=\frac{\partial}{\partial x}V(x,y^*)^\top\frac{\bar y_\delta^*-y^*}{\delta}
+\frac{N\,\mathrm{H.O.T.}}{\delta}
+A^\top\frac{\bar\lambda_\delta^*-\lambda^*}{\delta}
\end{aligned} v δ ( x , y ˉ δ ∗ , λ ˉ δ ∗ ) = δ 1 i = 1 ∑ N [ ∇ x g i ( x , y ˉ δ , i , y ¬ i ∗ ) − ∇ x g i ( x , y ∗ ) + ∂ x ∂ e ( i ) ( x , y ∗ ) ⊤ ( λ ˉ δ ∗ − λ ∗ ) ] = δ 1 i = 1 ∑ N [ ∂ y i ∂ ∇ x g i ( x , y ∗ ) ( y ˉ δ , i − y i ∗ ) + H.O.T. + ∂ x ∂ e ( i ) ( x , y ∗ ) ⊤ ( λ ˉ δ ∗ − λ ∗ ) ] = ∂ x ∂ V ( x , y ∗ ) ⊤ δ y ˉ δ ∗ − y ∗ + δ N H.O.T. + A ⊤ δ λ ˉ δ ∗ − λ ∗
where ∥ H . O . T . ∥ = O ( ∥ y ˉ δ ∗ − y ∗ ∥ 2 ) ≤ O ( δ 2 μ ˉ 2 ) \|\mathrm{H.O.T.}\|=O(\|\bar y_\delta^*-y^*\|^2)\le O\left(\frac{\delta^2}{\bar\mu^2}\right) ∥ H.O.T. ∥ = O ( ∥ y ˉ δ ∗ − y ∗ ∥ 2 ) ≤ O ( μ ˉ 2 δ 2 ) by Lemma G.2. Let v ϵ ∗ : = K ϵ ( x , y ∗ ) − ⊤ [ − ∇ y f ( x , y ∗ ) 0 ] v_\epsilon^*:=K_\epsilon(x,y^*)^{-\top}\begin{bmatrix}-\nabla_yf(x,y^*)\\0\end{bmatrix} v ϵ ∗ := K ϵ ( x , y ∗ ) − ⊤ [ − ∇ y f ( x , y ∗ ) 0 ] and $v^*:= K ( x , y ∗ ) − ⊤ [ − ∇ y f ( x , y ∗ ) 0 ] K(x,y^*)^{-\top}\begin{bmatrix}-\nabla_y f(x,y^*)\\0\end{bmatrix} K ( x , y ∗ ) − ⊤ [ − ∇ y f ( x , y ∗ ) 0 ] . Using Lemma G.4 and G.5, we have
∥ v δ ( x , y ˉ δ , λ ˉ δ ∗ ) − [ ∂ ∂ x V ( x , y ∗ ) ⊤ A ⊤ ] [ ∂ ∂ y V ( x , y ∗ ) B ⊤ B 0 ] − ⊤ [ − ∇ y f ( x , y ∗ ) 0 ] ∥ ≤ ∥ v δ ( x , y ˉ δ ∗ , λ ˉ δ ∗ ) − [ ∂ ∂ x V ( x , y ∗ ) ⊤ A ⊤ ] v ϵ ∗ ∥ + ∥ [ ∂ ∂ x V ( x , y ∗ ) ⊤ A ⊤ ] ( v ϵ ∗ − v ∗ ) ∥ ≤ ( C + ∥ A ∥ ) ( O ( N δ μ ˉ 2 ) + O ( N δ μ ˉ 2 ) + O ( ϵ ) ) = O ( N δ μ ˉ 2 + ϵ ) \begin{aligned}
\left\|v_\delta(x,\bar y_\delta,\bar\lambda_\delta^*)-\begin{bmatrix}\frac{\partial}{\partial x}V(x,y^*)^\top& A^\top\end{bmatrix}\begin{bmatrix}\frac{\partial}{\partial y}V(x,y^*) & B^\top\\B&0\end{bmatrix}^{-\top}\begin{bmatrix}-\nabla_y f(x,y^*)\\0\end{bmatrix}\right\| \\
&\leq\left\|v_\delta(x,\bar y_\delta^*,\bar\lambda_\delta^*)-\begin{bmatrix}\frac{\partial}{\partial x}V(x,y^*)^\top& A^\top\end{bmatrix}v_\epsilon^*\right\|
+\left\|\begin{bmatrix}\frac{\partial}{\partial x}V(x,y^*)^\top& A^\top\end{bmatrix}(v_\epsilon^*-v^*)\right\| \\
&\leq(C+\|A\|)\left(O\left(\frac{\sqrt{N}\delta}{\bar\mu^2}\right)+O\left(\frac{N\delta}{\bar\mu^2}\right)+O(\epsilon)\right) \\
&=O\left(\frac{N\delta}{\bar\mu^2}+\epsilon\right)
\end{aligned} v δ ( x , y ˉ δ , λ ˉ δ ∗ ) − [ ∂ x ∂ V ( x , y ∗ ) ⊤ A ⊤ ] [ ∂ y ∂ V ( x , y ∗ ) B B ⊤ 0 ] − ⊤ [ − ∇ y f ( x , y ∗ ) 0 ] ≤ v δ ( x , y ˉ δ ∗ , λ ˉ δ ∗ ) − [ ∂ x ∂ V ( x , y ∗ ) ⊤ A ⊤ ] v ϵ ∗ + [ ∂ x ∂ V ( x , y ∗ ) ⊤ A ⊤ ] ( v ϵ ∗ − v ∗ ) ≤ ( C + ∥ A ∥ ) ( O ( μ ˉ 2 N δ ) + O ( μ ˉ 2 N δ ) + O ( ϵ ) ) = O ( μ ˉ 2 N δ + ϵ )
Lemma G.6. Let y ^ \hat y y ^ and λ ^ \hat\lambda λ ^ be approximate solutions of the original lower-level game such that ∥ y ∗ − y ^ ∥ ≤ δ ^ \|y^*-\hat y\|\leq\hat\delta ∥ y ∗ − y ^ ∥ ≤ δ ^ and ∥ λ ∗ − λ ^ ∥ ≤ δ ^ \|\lambda^*-\hat\lambda\|\leq\hat\delta ∥ λ ∗ − λ ^ ∥ ≤ δ ^ . Define ϕ ^ ϵ ( x , y ) = 1 ϵ ∑ j = 1 N g j ( x , y j + ϵ ( y j − y ^ j , y ¬ j ) ) − g j ( x , y ) \hat\phi_\epsilon(x,y)=\frac{1}{\epsilon}\sum_{j=1}^N g_j(x,y_j+\epsilon(y_j-\hat y_j,y_{\neg j}))-g_j(x,y) ϕ ^ ϵ ( x , y ) = ϵ 1 ∑ j = 1 N g j ( x , y j + ϵ ( y j − y ^ j , y ¬ j )) − g j ( x , y ) , g ^ i ( x , y ) = ϕ ^ ϵ ( x , y ) − g i ( x , y ) + δ f ( x , y ) − ( λ ^ ) ⊤ B y \hat g_i(x,y)=\hat\phi_\epsilon(x,y)-g_i(x,y)+\delta f(x,y)-(\hat\lambda)^\top By g ^ i ( x , y ) = ϕ ^ ϵ ( x , y ) − g i ( x , y ) + δ f ( x , y ) − ( λ ^ ) ⊤ B y and V ^ ( x , y ) = ( ∇ y 1 g ^ 1 ( x , y ) , ⋯ , ∇ y N g ^ N ( x , y ) ) \hat V(x,y)=(\nabla_{y_1}\hat g_1(x,y),\cdots,\nabla_{y_N}\hat g_N(x,y)) V ^ ( x , y ) = ( ∇ y 1 g ^ 1 ( x , y ) , ⋯ , ∇ y N g ^ N ( x , y )) . Then
∥ V ˉ ( x , y ) − V ^ ( x , y ) ∥ ≤ ( C N + ∥ B ∥ ) δ ^ \|\bar V(x,y)-\hat V(x,y)\|\leq(CN+\|B\|)\hat\delta ∥ V ˉ ( x , y ) − V ^ ( x , y ) ∥ ≤ ( C N + ∥ B ∥ ) δ ^ Proof.
∥ ∇ y i ϕ ^ ϵ ( x , y ) − ∇ y i ϕ ϵ ( x , y ) ∥ ≤ 1 ϵ ∑ j C ∥ ϵ ( y j ∗ − y ^ j ) ∥ = C N ∥ y ∗ − y ^ ∥ \|\nabla_{y_i}\hat\phi_\epsilon(x,y)-\nabla_{y_i}\phi_\epsilon(x,y)\|\leq\frac{1}{\epsilon}\sum_j C\|\epsilon(y_j^*-\hat y_j)\|=C\sqrt{N}\|y^*-\hat y\| ∥ ∇ y i ϕ ^ ϵ ( x , y ) − ∇ y i ϕ ϵ ( x , y ) ∥ ≤ ϵ 1 j ∑ C ∥ ϵ ( y j ∗ − y ^ j ) ∥ = C N ∥ y ∗ − y ^ ∥ ∥ V ( x , y ) − V ^ ( x , y ) ∥ ≤ C N ∥ y ∗ − y ^ ∥ + ∥ B ∥ ∥ λ ∗ − λ ^ ∥ ≤ ( C N + ∥ B ∥ ) δ ^ \|V(x,y)-\hat V(x,y)\|\leq CN\|y^*-\hat y\|+\|B\|\|\lambda^*-\hat\lambda\|\leq(CN+\|B\|)\hat\delta ∥ V ( x , y ) − V ^ ( x , y ) ∥ ≤ C N ∥ y ∗ − y ^ ∥ + ∥ B ∥∥ λ ∗ − λ ^ ∥ ≤ ( C N + ∥ B ∥ ) δ ^
Theorem G.7. For 0 < δ < μ 2 C f 0<\delta<\frac{\mu}{2C_f} 0 < δ < 2 C f μ and 0 < R < μ − δ C f S ( N + 1 ) ( 2 + N ) 0<R<\frac{\mu-\delta C_f}{S(N+1)(2+\sqrt{N})} 0 < R < S ( N + 1 ) ( 2 + N ) μ − δ C f and 0 < ϵ < 1 0<\epsilon<1 0 < ϵ < 1 , V ˉ \bar V V ˉ is locally μ ˉ \bar\mu μ ˉ -strongly monotone and C ˉ \bar C C ˉ -smooth on B R ( y ∗ ) = { y ∈ R n : ∥ y − y ∗ ∥ ≤ R } B_R(y^*)=\{y\in\mathbb{R}^n:\|y-y^*\|\leq R\} B R ( y ∗ ) = { y ∈ R n : ∥ y − y ∗ ∥ ≤ R } for some μ ˉ , C ˉ > 0 \bar\mu,\bar C>0 μ ˉ , C ˉ > 0 . Furthermore, if δ < min ( μ 2 C f , μ ˉ R 2 L f ) \delta<\min\left(\frac{\mu}{2C_f},\frac{\bar\mu R}{2L_f}\right) δ < min ( 2 C f μ , 2 L f μ ˉ R ) , then we have a fully first-order hypergradient approximation with non-asymptotic rate
∥ v δ ( x , y ˉ δ ∗ , λ ˉ δ ∗ ) − [ ∂ ∂ x V ( x , y ∗ ) ⊤ A ⊤ ] [ ∂ ∂ y V ( x , y ∗ ) B ⊤ B 0 ] − ⊤ [ − ∇ y f ( x , y ∗ ) 0 ] ∥ ≤ O ( N δ μ ˉ 2 + ϵ ) \left\|v_\delta(x,\bar y_\delta^*,\bar\lambda_\delta^*)-\begin{bmatrix}\frac{\partial}{\partial x}V(x,y^*)^\top& A^\top\end{bmatrix}\begin{bmatrix}\frac{\partial}{\partial y}V(x,y^*)&B^\top\\B&0\end{bmatrix}^{-\top}\begin{bmatrix}-\nabla_y f(x,y^*)\\0\end{bmatrix}\right\|\leq O\left(\frac{N\delta}{\bar\mu^2}+\epsilon\right) v δ ( x , y ˉ δ ∗ , λ ˉ δ ∗ ) − [ ∂ x ∂ V ( x , y ∗ ) ⊤ A ⊤ ] [ ∂ y ∂ V ( x , y ∗ ) B B ⊤ 0 ] − ⊤ [ − ∇ y f ( x , y ∗ ) 0 ] ≤ O ( μ ˉ 2 N δ + ϵ ) Proof Sketch. Lemma F.1 proves local strong monotonicity and smoothness. For δ < min ( μ 2 C f , μ ˉ R 2 L f ) \delta<\min\left(\frac{\mu}{2C_f},\frac{\bar\mu R}{2L_f}\right) δ < min ( 2 C f μ , 2 L f μ ˉ R ) , Lemma G.2 shows that T ( y ) = Π Y ( y − η V ˉ ( x , y ) ) T(y)=\Pi_Y(y-\eta\bar V(x,y)) T ( y ) = Π Y ( y − η V ˉ ( x , y )) is a contraction on B R ( y ∗ ) ∩ Y B_R(y^*)\cap Y B R ( y ∗ ) ∩ Y . It proves the existence of Nash equilibrium y ˉ δ ∗ \bar y_\delta^* y ˉ δ ∗ and ∥ y ˉ δ ∗ − y ∗ ∥ ≤ 2 L f δ μ ˉ \|\bar y_\delta^*-y^*\|\leq\frac{2L_f\delta}{\bar\mu} ∥ y ˉ δ ∗ − y ∗ ∥ ≤ μ ˉ 2 L f δ . Let v ϵ ∗ : = K ϵ ( x , y ∗ ) − ⊤ [ − ∇ y f ( x , y ∗ ) 0 ] v_\epsilon^*:=K_\epsilon(x,y^*)^{-\top}\begin{bmatrix}-\nabla_y f(x,y^*)\\0\end{bmatrix} v ϵ ∗ := K ϵ ( x , y ∗ ) − ⊤ [ − ∇ y f ( x , y ∗ ) 0 ] and v ∗ : = K ( x , y ∗ ) − ⊤ [ − ∇ y f ( x , y ∗ ) 0 ] v^*:=K(x,y^*)^{-\top}\begin{bmatrix}-\nabla_y f(x,y^*)\\0\end{bmatrix} v ∗ := K ( x , y ∗ ) − ⊤ [ − ∇ y f ( x , y ∗ ) 0 ] . Triangle inequality reduces the proof into bounding ∥ v δ ( x , y ˉ δ ∗ , λ ˉ δ ∗ ) − [ ∂ ∂ x V ( x , y ∗ ) ⊤ A ⊤ ] v ϵ ∗ ∥ + ∥ [ ∂ ∂ x V ( x , y ∗ ) ⊤ A ⊤ ] ( v ϵ ∗ − v ∗ ) ∥ \|v_\delta(x,\bar y_\delta^*,\bar\lambda_\delta^*)-\begin{bmatrix}\frac{\partial}{\partial x}V(x,y^*)^\top&A^\top\end{bmatrix}v_\epsilon^*\|+\|\begin{bmatrix}\frac{\partial}{\partial x}V(x,y^*)^\top&A^\top\end{bmatrix}(v_\epsilon^*-v^*)\| ∥ v δ ( x , y ˉ δ ∗ , λ ˉ δ ∗ ) − [ ∂ x ∂ V ( x , y ∗ ) ⊤ A ⊤ ] v ϵ ∗ ∥ + ∥ [ ∂ x ∂ V ( x , y ∗ ) ⊤ A ⊤ ] ( v ϵ ∗ − v ∗ ) ∥ . Combining the following equations yields the desired result,
∥ v ϵ ∗ − v ∗ ∥ ≤ O ( ϵ ) (Lemma xref‘G.5‘) \|v_\epsilon^*-v^*\|\leq O(\epsilon)\quad\text{(Lemma {xref}`G.5`)} ∥ v ϵ ∗ − v ∗ ∥ ≤ O ( ϵ ) (Lemma xref ‘G.5‘) ∥ [ y ˉ δ ∗ − y ∗ δ λ ˉ δ ∗ − λ ∗ δ ] − v ϵ ∗ ∥ ≤ N μ ˉ 2 O ( δ ) (Lemma xref‘G.4‘) \left\|\begin{bmatrix}\frac{\bar y_\delta^*-y^*}{\delta}\\\frac{\bar\lambda_\delta^*-\lambda^*}{\delta}\end{bmatrix}-v_\epsilon^*\right\|\leq\frac{\sqrt{N}}{\bar\mu^2}O(\delta)\quad\text{(Lemma {xref}`G.4`)} [ δ y ˉ δ ∗ − y ∗ δ λ ˉ δ ∗ − λ ∗ ] − v ϵ ∗ ≤ μ ˉ 2 N O ( δ ) (Lemma xref ‘G.4‘) v δ ( x , y ˉ δ ∗ , λ ˉ δ ∗ ) = ∂ ∂ x V ( x , y ∗ ) ⊤ y ˉ δ ∗ − y ∗ δ + N H . O . T . δ + A ⊤ λ ˉ δ ∗ − λ ∗ δ (Taylor’s expansion) v_\delta(x,\bar y_\delta^*,\bar\lambda_\delta^*)=\frac{\partial}{\partial x}V(x,y^*)^\top\frac{\bar y_\delta^*-y^*}{\delta}+N\frac{\mathrm{H.O.T.}}{\delta}+A^\top\frac{\bar\lambda_\delta^*-\lambda^*}{\delta}\quad\text{(Taylor's expansion)} v δ ( x , y ˉ δ ∗ , λ ˉ δ ∗ ) = ∂ x ∂ V ( x , y ∗ ) ⊤ δ y ˉ δ ∗ − y ∗ + N δ H.O.T. + A ⊤ δ λ ˉ δ ∗ − λ ∗ (Taylor’s expansion) where ∥ H . O . T . ∥ = O ( ∥ y ˉ δ ∗ − y ∗ ∥ 2 ) ≤ O ( δ 2 μ ˉ 2 ) \|\mathrm{H.O.T.}\|=O(\|\bar y_\delta^*-y^*\|^2)\leq O\left(\frac{\delta^2}{\bar\mu^2}\right) ∥ H.O.T. ∥ = O ( ∥ y ˉ δ ∗ − y ∗ ∥ 2 ) ≤ O ( μ ˉ 2 δ 2 ) .
Theorem G.8. Consider problem P 1 P1 P 1 . Under assumptions 3.4, 3.5 and G.1, inexact gradient descent (algorithm 1) outputs an O ( ϵ ) \mathcal{O}(\epsilon) O ( ϵ ) -stationary point of the hyperobjective F F F with O ~ ( N 2 ϵ − 2 ) \widetilde{\mathcal{O}}(N^{2}\epsilon^{-2}) O ( N 2 ϵ − 2 ) first-order oracle calls to f f f and g i g_i g i in both equality constrained and unconstrained settings.
Proof. Following the argument of Theorem B.7, the hyperobjective F F F is smooth. Choose δ ≤ ϵ 2 N δ 2 \delta\le\frac{\epsilon}{2N\delta^2} δ ≤ 2 N δ 2 ϵ and use the potential function ϕ ϵ 2 \phi_{\frac{\epsilon}{2}} ϕ 2 ϵ . By Lemma D.1, if the approximate lower-level solutions satisfy ∥ y ∗ − y ^ ∥ ≤ δ 2 \lVert y^*-\hat{y}\rVert\le\delta^2 ∥ y ∗ − y ^ ∥ ≤ δ 2 and ∥ y ˉ δ − y ^ δ ∥ ≤ δ 2 \lVert\bar{y}_{\delta}-\hat{y}_{\delta}\rVert\le\delta^2 ∥ y ˉ δ − y ^ δ ∥ ≤ δ 2 , then the inexact hypergradient ∇ ^ F ( x ) = ∇ x f ( x , y ^ ) + v ^ δ ( x , y ^ δ ) \hat{\nabla}F(x)=\nabla_x f(x,\hat{y})+\hat{v}_{\delta}(x,\hat{y}_{\delta}) ∇ ^ F ( x ) = ∇ x f ( x , y ^ ) + v ^ δ ( x , y ^ δ ) satisfies ∥ ∇ ^ F ( x ) − ∇ F ( x ) ∥ ≤ O ( N δ + ϵ / 2 ) ≤ O ( ϵ ) \lVert\hat{\nabla}F(x)-\nabla F(x)\rVert\le\mathcal{O}(N\delta+\epsilon/2)\le\mathcal{O}(\epsilon) ∥ ∇ ^ F ( x ) − ∇ F ( x )∥ ≤ O ( N δ + ϵ /2 ) ≤ O ( ϵ ) . Therefore, by Lemma D.2, inexact gradient descent requires O ( ϵ − 2 ) \mathcal{O}(\epsilon^{-2}) O ( ϵ − 2 ) hypergradient evaluations to reach an O ( ϵ ) \mathcal{O}(\epsilon) O ( ϵ ) -stationary point of F F F . It remains to bound the cost of each hypergradient evaluation. Since the potential is constructed using the approximate equilibrium y ^ \hat{y} y ^ , its gradient operator is also inexact, with an error of O ( ϵ ) \mathcal{O}(\epsilon) O ( ϵ ) (Lemma G.6). Moreover, the auxiliary game is only locally strongly monotone on B ˉ R ( y ∗ ) \bar{B}_{R}(y^*) B ˉ R ( y ∗ ) . We therefore choose δ ≤ min { ϵ / ( 2 N ) , μ ˉ R / ( 4 L f ) , R / 2 } \delta\le\min\{\epsilon/(2N),\bar{\mu}R/(4L_f),R/2\} δ ≤ min { ϵ / ( 2 N ) , μ ˉ R / ( 4 L f ) , R /2 } , which ensures ∥ y ˉ δ − y ^ ∥ ≤ ∥ y ˉ δ − y ∗ ∥ + ∥ y ∗ − y ^ ∥ ≤ R \lVert\bar{y}_{\delta}-\hat{y}\rVert\le\lVert\bar{y}_{\delta}-y^*\rVert+\lVert y^*-\hat{y}\rVert\le R ∥ y ˉ δ − y ^ ∥ ≤ ∥ y ˉ δ − y ∗ ∥ + ∥ y ∗ − y ^ ∥ ≤ R . Hence, gradient descent initialized at y ^ \hat{y} y ^ remains in the local neighborhood and converges linearly to the auxiliary equilibrium. By Lemma E.2, O ( log ( 1 / δ 2 ) ) \mathcal{O}(\log(1/\delta^2)) O ( log ( 1/ δ 2 )) lower-level iterations are sufficient to obtain the required δ 2 \delta^2 δ 2 -accurate solutions. Each evaluation of the auxiliary game operator requires gradients of all N N N surrogate costs, and evaluating each surrogate cost requires O ( N ) \mathcal{O}(N) O ( N ) calls to the original follower objectives g i g_i g i . Thus, each lower-level iteration uses O ( N 2 ) \mathcal{O}(N^2) O ( N 2 ) first-order oracle calls. Combining this with O ( ϵ − 2 ) \mathcal{O}(\epsilon^{-2}) O ( ϵ − 2 ) outer iterations and O ( log ( 1 / δ 2 ) ) \mathcal{O}(\log(1/\delta^2)) O ( log ( 1/ δ 2 )) lower-level iterations gives a total oracle complexity of O ( N 2 ϵ − 2 log ( 1 / δ 2 ) ) = O ~ ( N 2 ϵ − 2 ) \mathcal{O}(N^2\epsilon^{-2}\log(1/\delta^2))=\widetilde{\mathcal{O}}(N^2\epsilon^{-2}) O ( N 2 ϵ − 2 log ( 1/ δ 2 )) = O ( N 2 ϵ − 2 ) .
Extension to Stackelberg Game with Linear Inequality Constraints
Consider the following Stackelberg game with linear inequality constraints,
min x ∈ X F ( x ) : = f ( x , y ∗ ( x ) ) s.t. g i ( x , y ∗ ( x ) ) ≤ g i ( x , y i , y − i ∗ ( x ) ) , ∀ i ∈ [ N ] , ∀ y i ∈ Y i ( x ) Y i ( x ) = { y i ∈ R n i : e ( i ) ( x , y i ) ≤ 0 } e ( i ) ( x , y i ) = A ( i ) x + B ( i ) y i + b ( i ) (15) \begin{aligned}
\min_{x\in X}\quad&F(x):=f(x,y^*(x))\\
\text{s.t.}\quad&g_i(x,y^*(x))\le g_i(x,y_i,y_{-i}^*(x)),\quad\forall i\in[N],\ \forall y_i\in Y_i(x)\\
&Y_i(x)=\{y_i\in\mathbb{R}^{n_i}:e^{(i)}(x,y_i)\le0\}\\
&e^{(i)}(x,y_i)=A^{(i)}x+B^{(i)}y_i+b^{(i)} \tag*{(15)}
\end{aligned} x ∈ X min s.t. F ( x ) := f ( x , y ∗ ( x )) g i ( x , y ∗ ( x )) ≤ g i ( x , y i , y − i ∗ ( x )) , ∀ i ∈ [ N ] , ∀ y i ∈ Y i ( x ) Y i ( x ) = { y i ∈ R n i : e ( i ) ( x , y i ) ≤ 0 } e ( i ) ( x , y i ) = A ( i ) x + B ( i ) y i + b ( i ) (15)
In problem Q 1 Q1 Q 1 , the hyper-objective F F F is Lipschitz continuous but nonsmooth because the active set may change discontinuously. ( δ , ϵ ) (\delta,\epsilon) ( δ , ϵ ) -stationarity is used to measure convergence because obtaining an ϵ \epsilon ϵ -stationary point of a nonsmooth nonconvex function is intractable ([36 ] ).
Definition H.1 ((δ , ϵ \delta,\epsilon δ , ϵ )-stationarity). Let F : R d → R F:\mathbb{R}^{d}\to\mathbb{R} F : R d → R be a locally Lipschitz continuous function. Let x ∈ R d x\in\mathbb{R}^{d} x ∈ R d . Let B δ ( x ) B_{\delta}(x) B δ ( x ) be the Euclidean ball of radius δ > 0 \delta>0 δ > 0 centered at x x x . The generalized gradient of F F F at x x x is the set ∂ F ( x ) = conv ( { g ∣ g = lim k → ∞ ∇ F ( x k ) , x k → x } ) \partial F(x)=\operatorname{conv}(\{g\mid g=\lim_{k\to\infty}\nabla F(x_k),\ x_k\to x\}) ∂ F ( x ) = conv ({ g ∣ g = lim k → ∞ ∇ F ( x k ) , x k → x }) ([8 ] ). Also, ∂ δ F ( x ) = ⋂ δ > 0 ⋃ y ∈ B δ ( x ) ∂ F ( y ) \partial_{\delta}F(x)=\bigcap_{\delta>0}\bigcup_{y\in B_{\delta}(x)}\partial F(y) ∂ δ F ( x ) = ⋂ δ > 0 ⋃ y ∈ B δ ( x ) ∂ F ( y ) . The Goldstein subdifferential of F F F at x x x is the set ∂ δ F ( x ) = conv ( ⋃ y ∈ B δ ( x ) ∂ F ( y ) ) \partial_{\delta}F(x)=\operatorname{conv}(\bigcup_{y\in B_{\delta}(x)}\partial F(y)) ∂ δ F ( x ) = conv ( ⋃ y ∈ B δ ( x ) ∂ F ( y )) ([19 ] ). x x x is called a ( δ , ϵ ) (\delta,\epsilon) ( δ , ϵ ) -stationary point of a locally Lipschitz continuous function F F F if dist ( 0 , ∂ δ F ( x ) ) : = inf g ∈ ∂ δ F ( x ) ∥ x − y ∥ ≤ ϵ \operatorname{dist}(0,\partial_{\delta}F(x)):=\inf_{g\in\partial_{\delta}F(x)}\lVert x-y\rVert\le\epsilon dist ( 0 , ∂ δ F ( x )) := inf g ∈ ∂ δ F ( x ) ∥ x − y ∥ ≤ ϵ .
Assumption H.2 (Active set identification). For problem Q 1 Q1 Q 1 , assume that the active constraints can be correctly identified.
By identifying the active set at the Nash equilibrium (similar to [37 ] ), we can transform an arbitrary smooth and strongly monotone game G : = ( x , [ N ] , { Y i } i ∈ [ N ] , { g i } i ∈ [ N ] ) \mathcal{G}:=(x,[N],\{Y_i\}_{i\in[N]},\{g_i\}_{i\in[N]}) G := ( x , [ N ] , { Y i } i ∈ [ N ] , { g i } i ∈ [ N ] ) with linear inequality constraints into an equivalent game with linear equality constraints. Then, we can directly apply the hypergradient approximation derived for Stackelberg game with linear equality constraints in § 4 or G.
Given an arbitrary reference point x ˉ ∈ X \bar{x}\in X x ˉ ∈ X . For each player i ∈ [ N ] i\in[N] i ∈ [ N ] , we define the strict active set I i : = { j ∈ [ m i ] : ( λ j ( i ) ) > 0 } \mathcal{I}_i:=\{j\in[m_i]:(\lambda_j^{(i)})>0\} I i := { j ∈ [ m i ] : ( λ j ( i ) ) > 0 } , active dual variable λ I i : = ( λ j ( i ) ) j ∈ I i \lambda_{\mathcal{I}_i}:=(\lambda_j^{(i)})_{j\in\mathcal{I}_i} λ I i := ( λ j ( i ) ) j ∈ I i and the active equality constraint function e I i ( i ) : = ( e j ( i ) ) j ∈ I i e_{\mathcal{I}_i}^{(i)}:=(e_j^{(i)})_{j\in\mathcal{I}_i} e I i ( i ) := ( e j ( i ) ) j ∈ I i . We define the ghost cost function
g ~ i ( x , y i ) = g i ( x , y i ) + ( λ I i ∗ ) T e ( i ) ( x , y i ) \tilde{g}_i(x,y_i)=g_i(x,y_i)+(\lambda_{\mathcal{I}_i}^*)^\mathsf{T}e^{(i)}(x,y_i) g ~ i ( x , y i ) = g i ( x , y i ) + ( λ I i ∗ ) T e ( i ) ( x , y i )
and the ghost equality constraint Y ~ i = { y i ∈ R n i : e ~ ( i ) ( x , y i ) = 0 } \widetilde{Y}_i=\{y_i\in\mathbb{R}^{n_i}:\tilde{e}^{(i)}(x,y_i)=0\} Y i = { y i ∈ R n i : e ~ ( i ) ( x , y i ) = 0 } , where
e ~ ( i ) ( x , y i ) = ∂ ∂ x e I i ( i ) ( x ˉ , y i ∗ ) ( x − x ˉ ) + ∂ ∂ y e I i ( i ) ( x ˉ , y i ∗ ) ( y i − y i ∗ ) \tilde{e}^{(i)}(x,y_i)=\frac{\partial}{\partial x}e_{\mathcal{I}_i}^{(i)}(\bar{x},y_i^*)(x-\bar{x})+\frac{\partial}{\partial y}e_{\mathcal{I}_i}^{(i)}(\bar{x},y_i^*)(y_i-y_i^*) e ~ ( i ) ( x , y i ) = ∂ x ∂ e I i ( i ) ( x ˉ , y i ∗ ) ( x − x ˉ ) + ∂ y ∂ e I i ( i ) ( x ˉ , y i ∗ ) ( y i − y i ∗ )
are the active constraints. We replace the original lower-level game G = ( x , [ N ] , { Y i } i ∈ [ N ] , { g i } i ∈ [ N ] ) \mathcal{G} = (x, [N], \{Y_i\}_{i\in[N]}, \{g_i\}_{i\in[N]}) G = ( x , [ N ] , { Y i } i ∈ [ N ] , { g i } i ∈ [ N ] ) with ghost lower-level game G ~ : = ( x , [ N ] , { Y ~ i } i ∈ [ N ] , { g ~ i } i ∈ [ N ] ) \widetilde{\mathcal{G}} := (x, [N], \{\widetilde{Y}_i\}_{i\in[N]}, \{\widetilde{g}_i\}_{i\in[N]}) G := ( x , [ N ] , { Y i } i ∈ [ N ] , { g i } i ∈ [ N ] ) . Denote y ~ ∗ \widetilde{y}^{*} y ∗ as the Nash equilibrium of G ~ \widetilde{\mathcal{G}} G and λ ~ ∗ \widetilde{\lambda}^{*} λ ∗ as the optimal dual solution for G ~ \widetilde{\mathcal{G}} G . The ghost problem is defined as
min x ∈ X F ~ ( x ) : = f ( x , y ~ ∗ ( x ) ) s.t. g ~ i ( x , y ~ ∗ ( x ) ) ≤ g ~ i ( x , y i , y ~ − i ∗ ( x ) ) , ∀ i ∈ [ N ] , ∀ y i ∈ Y ~ i ( x ) Y ~ i ( x ) = { y i ∈ R n i : ∂ ∂ x e I i ( i ) ( x ˉ , y i ∗ ) ( x − x ˉ ) + ∂ ∂ y e I i ( i ) ( x ˉ , y i ∗ ) ( y i − y i ∗ ) = 0 } (16) \begin{aligned}
\min_{x\in X}\quad&\widetilde{F}(x) := f(x,\widetilde{y}^{*}(x))\\
\text{s.t.}\quad&\widetilde{g}_i(x,\widetilde{y}^{*}(x)) \leq\widetilde{g}_i(x,y_i,\widetilde{y}_{-i}^{*}(x)),\ \forall i\in[N],\ \forall y_i\in\widetilde{Y}_i(x)\\
\widetilde{Y}_i(x) ={}& \left\{y_i\in\mathbb{R}^{n_i}:\frac{\partial}{\partial x}e_{I_i}^{(i)}(\bar{x},y_i^{*})(x-\bar{x})+\frac{\partial}{\partial y}e_{I_i}^{(i)}(\bar{x},y_i^{*})(y_i-y_i^{*})=0\right\} \tag*{(16)}
\end{aligned} x ∈ X min s.t. Y i ( x ) = F ( x ) := f ( x , y ∗ ( x )) g i ( x , y ∗ ( x )) ≤ g i ( x , y i , y − i ∗ ( x )) , ∀ i ∈ [ N ] , ∀ y i ∈ Y i ( x ) { y i ∈ R n i : ∂ x ∂ e I i ( i ) ( x ˉ , y i ∗ ) ( x − x ˉ ) + ∂ y ∂ e I i ( i ) ( x ˉ , y i ∗ ) ( y i − y i ∗ ) = 0 } (16)
By comparing KKT conditions of the original and ghost lower-level problem (Lemma H.4), the ghost problem Q2 is in fact equivalent to the original Stackelberg game with inequality constraints Q1, such that y ~ ∗ ( x ˉ ) = y ∗ ( x ˉ ) \widetilde{y}^{*}(\bar{x})=y^{*}(\bar{x}) y ∗ ( x ˉ ) = y ∗ ( x ˉ ) and d d x y ~ ∗ ( x ˉ ) = d d x y ∗ ( x ˉ ) \frac{d}{dx}\widetilde{y}^{*}(\bar{x})=\frac{d}{dx}y^{*}(\bar{x}) d x d y ∗ ( x ˉ ) = d x d y ∗ ( x ˉ ) . Hence, F ~ ( x ˉ ) = F ( x ˉ ) \widetilde{F}(\bar{x})=F(\bar{x}) F ( x ˉ ) = F ( x ˉ ) and ∇ F ~ ( x ˉ ) = ∇ F ( x ˉ ) \nabla\widetilde{F}(\bar{x})=\nabla F(\bar{x}) ∇ F ( x ˉ ) = ∇ F ( x ˉ ) .
Lemma H.3. The ghost lower-level game G ~ : = ( x , [ N ] , { Y ~ i } i ∈ [ N ] , { g ~ i } i ∈ [ N ] ) \widetilde{\mathcal{G}} := (x,[N],\{\widetilde{Y}_i\}_{i\in[N]},\{\widetilde{g}_i\}_{i\in[N]}) G := ( x , [ N ] , { Y i } i ∈ [ N ] , { g i } i ∈ [ N ] ) in problem Q2 is μ \mu μ -strongly monotone, C C C -smooth and S S S -Hessian smooth.
Proof. Define V ~ ( x , y ) : = ( ∇ y 1 g ~ 1 ( x , y ) , ⋯ , ∇ y N g ~ N ( x , y ) ) ∈ R n \widetilde{V}(x,y):=(\nabla_{y_1}\widetilde{g}_1(x,y),\cdots,\nabla_{y_N}\widetilde{g}_N(x,y))\in\mathbb{R}^n V ( x , y ) := ( ∇ y 1 g 1 ( x , y ) , ⋯ , ∇ y N g N ( x , y )) ∈ R n . Let x , x ′ ∈ X x,x'\in X x , x ′ ∈ X and y , y ′ ∈ Y ~ y,y'\in\widetilde{Y} y , y ′ ∈ Y ,
V ~ ( x ′ , y ′ ) − V ~ ( x , y ) = V ( x ′ , y ′ ) − V ( x , y ) + B T λ ∗ − B T λ ∗ = V ( x ′ , y ′ ) − V ( x , y ) \begin{aligned}
\widetilde{V}(x',y')-\widetilde{V}(x,y)=V(x',y')-V(x,y)+B^{\mathsf{T}}\lambda^{*}-B^{\mathsf{T}}\lambda^{*} \\
&=V(x',y')-V(x,y)
\end{aligned} V ( x ′ , y ′ ) − V ( x , y ) = V ( x ′ , y ′ ) − V ( x , y ) + B T λ ∗ − B T λ ∗ = V ( x ′ , y ′ ) − V ( x , y )
We can conclude the proof using the definitions of strong monotonicity, smoothness and Hessian smoothness in 3.4. □
Lemma H.4 (Equivalence of the ghost problem and the original problem). Let y ~ ∗ ( x ) ∈ N E ( G ~ ) \widetilde{y}^{*}(x)\in NE(\widetilde{\mathcal{G}}) y ∗ ( x ) ∈ N E ( G ) be the Nash equilibrium of the ghost lower-level game Q2 and let y ∗ ( x ) ∈ N E ( G ) y^{*}(x)\in NE(\mathcal{G}) y ∗ ( x ) ∈ N E ( G ) be the Nash equilibrium of the original lower-level game Q1. For all x ˉ ∈ X \bar{x}\in X x ˉ ∈ X where active set is locally constant, y ~ ∗ ( x ˉ ) = y ∗ ( x ˉ ) \widetilde{y}^{*}(\bar{x})=y^{*}(\bar{x}) y ∗ ( x ˉ ) = y ∗ ( x ˉ ) , d d x y ~ ∗ ( x ˉ ) = d d x y ∗ ( x ˉ ) \frac{d}{dx}\widetilde{y}^{*}(\bar{x})=\frac{d}{dx}y^{*}(\bar{x}) d x d y ∗ ( x ˉ ) = d x d y ∗ ( x ˉ ) , λ ~ ∗ ( x ˉ ) = 0 \widetilde{\lambda}^{*}(\bar{x})=0 λ ∗ ( x ˉ ) = 0 where λ ~ ∗ \widetilde{\lambda}^{*} λ ∗ is the optimal dual variable of G ~ \widetilde{\mathcal{G}} G
Proof. Denote B I : = diag ( ( ∂ ∂ y e I i ( i ) ( x ˉ , y i ∗ ) ) i ∈ [ N ] ) B_I:=\operatorname{diag}((\frac{\partial}{\partial y}e_{I_i}^{(i)}(\bar{x},y_i^{*}))_{i\in[N]}) B I := diag (( ∂ y ∂ e I i ( i ) ( x ˉ , y i ∗ ) ) i ∈ [ N ] ) as a block diagonal matrix corresponding to Jacobians of active inequality constraints, and
A I : = [ ∂ ∂ x e I 1 ( 1 ) ( x ˉ , y 1 ∗ ) ⋮ ∂ ∂ x e I N ( N ) ( x ˉ , y N ∗ ) ] . A_I :=
\begin{bmatrix}
\frac{\partial}{\partial x}e_{I_1}^{(1)}(\bar{x},y_1^{*})\\
\vdots\\
\frac{\partial}{\partial x}e_{I_N}^{(N)}(\bar{x},y_N^{*})
\end{bmatrix}. A I := ∂ x ∂ e I 1 ( 1 ) ( x ˉ , y 1 ∗ ) ⋮ ∂ x ∂ e I N ( N ) ( x ˉ , y N ∗ ) .
The KKT condition of the original problem Q1 is
Γ ( x ˉ , y ∗ , λ ∗ ) = [ V ( x ˉ , y ∗ ) + B T λ ∗ [ diag ( λ 1 , I 1 ∗ ) e I 1 ( 1 ) ( x ˉ , y 1 ∗ ) ⋮ diag ( λ N , I N ∗ ) e I N ( N ) ( x ˉ , y N ∗ ) ] ] = 0. \Gamma(\bar{x},y^{*},\lambda^{*})=
\begin{bmatrix}
V(\bar{x},y^{*})+B^{\mathsf{T}}\lambda^{*}\\
\begin{bmatrix}
\operatorname{diag}(\lambda_{1,I_1}^{*})e_{I_1}^{(1)}(\bar{x},y_1^{*})\\
\vdots\\
\operatorname{diag}(\lambda_{N,I_N}^{*})e_{I_N}^{(N)}(\bar{x},y_N^{*})
\end{bmatrix}
\end{bmatrix}
=0. Γ ( x ˉ , y ∗ , λ ∗ ) = V ( x ˉ , y ∗ ) + B T λ ∗ diag ( λ 1 , I 1 ∗ ) e I 1 ( 1 ) ( x ˉ , y 1 ∗ ) ⋮ diag ( λ N , I N ∗ ) e I N ( N ) ( x ˉ , y N ∗ ) = 0.
The jacobian
∂ ∂ ( y ∗ , λ I ∗ ) Γ ( x ˉ , y ∗ , λ I ∗ ) = [ ∂ ∂ y V ( x ˉ , y ∗ ) B I T diag ( λ I ∗ ) B I 0 ] \frac{\partial}{\partial(y^{*},\lambda_I^{*})}\Gamma(\bar{x},y^{*},\lambda_I^{*})=
\begin{bmatrix}
\frac{\partial}{\partial y}V(\bar{x},y^{*}) & B_I^{\mathsf{T}}\\
\operatorname{diag}(\lambda_I^{*})B_I & 0
\end{bmatrix} ∂ ( y ∗ , λ I ∗ ) ∂ Γ ( x ˉ , y ∗ , λ I ∗ ) = [ ∂ y ∂ V ( x ˉ , y ∗ ) diag ( λ I ∗ ) B I B I T 0 ]
is invertible following the same steps in Lemma A.8. Then by implicit function theorem, y ∗ ( x ˉ ) , λ I ∗ ( x ˉ ) y^{*}(\bar{x}),\lambda_I^{*}(\bar{x}) y ∗ ( x ˉ ) , λ I ∗ ( x ˉ ) are continuously differentiable functions of x x x and their derivatives are given by
[ ∂ ∂ y V ( x ˉ , y ∗ ) B I T diag ( λ I ∗ ) B I 0 ] [ d d x y ∗ ( x ˉ ) d d x λ I ∗ ( x ˉ ) ] = [ − ∂ ∂ x V ( x ˉ , y ∗ ) − diag ( λ I ∗ ) A I ] (17) \begin{bmatrix}
\frac{\partial}{\partial y}V(\bar{x},y^{*}) & B_I^{\mathsf{T}}\\
\operatorname{diag}(\lambda_I^{*})B_I & 0
\end{bmatrix}
\begin{bmatrix}
\frac{d}{dx}y^{*}(\bar{x})\\
\frac{d}{dx}\lambda_I^{*}(\bar{x})
\end{bmatrix}
=
\begin{bmatrix}
-\frac{\partial}{\partial x}V(\bar{x},y^{*})\\
-\operatorname{diag}(\lambda_I^{*})A_I \tag*{(17)}
\end{bmatrix} [ ∂ y ∂ V ( x ˉ , y ∗ ) diag ( λ I ∗ ) B I B I T 0 ] [ d x d y ∗ ( x ˉ ) d x d λ I ∗ ( x ˉ ) ] = [ − ∂ x ∂ V ( x ˉ , y ∗ ) − diag ( λ I ∗ ) A I ] (17)
[ ∂ ∂ y V ( x ˉ , y ∗ ) B I T B I 0 ] [ d d x y ∗ ( x ˉ ) d d x λ I ∗ ( x ˉ ) ] = [ − ∂ ∂ x V ( x ˉ , y ∗ ) − A I ] (18) \begin{bmatrix}
\frac{\partial}{\partial y}V(\bar{x},y^{*}) & B_I^{\mathsf{T}}\\
B_I & 0
\end{bmatrix}
\begin{bmatrix}
\frac{d}{dx}y^{*}(\bar{x})\\
\frac{d}{dx}\lambda_I^{*}(\bar{x})
\end{bmatrix}
=
\begin{bmatrix}
-\frac{\partial}{\partial x}V(\bar{x},y^{*})\\
-A_I \tag*{(18)}
\end{bmatrix} [ ∂ y ∂ V ( x ˉ , y ∗ ) B I B I T 0 ] [ d x d y ∗ ( x ˉ ) d x d λ I ∗ ( x ˉ ) ] = [ − ∂ x ∂ V ( x ˉ , y ∗ ) − A I ] (18)
On the other hand, the KKT condition of the ghost problem Q2 is
Γ ~ ( x ˉ , y ~ ∗ , λ ~ ∗ ) = [ V ( x ˉ , y ~ ∗ ) + B I ⊤ λ I ∗ + B I ⊤ λ ~ ∗ [ e ~ I ( 1 ) ( x ˉ , y ~ ∗ ) ⋮ e ~ I ( N ) ( x ˉ , y ~ ∗ ) ] ] = 0 \widetilde{\Gamma}(\bar{x},\widetilde{y}^{*},\widetilde{\lambda}^{*})=
\begin{bmatrix}
V(\bar{x},\widetilde{y}^{*})+B_{I}^{\top}\lambda_{I}^{*}+B_{I}^{\top}\widetilde{\lambda}^{*}\\
\begin{bmatrix}
\widetilde{e}_{I}^{(1)}(\bar{x},\widetilde{y}^{*})\\
\vdots\\
\widetilde{e}_{I}^{(N)}(\bar{x},\widetilde{y}^{*})
\end{bmatrix}
\end{bmatrix}=0 Γ ( x ˉ , y ∗ , λ ∗ ) = V ( x ˉ , y ∗ ) + B I ⊤ λ I ∗ + B I ⊤ λ ∗ e I ( 1 ) ( x ˉ , y ∗ ) ⋮ e I ( N ) ( x ˉ , y ∗ ) = 0
Substituting y ~ ∗ = y ∗ \widetilde{y}^{*}=y^{*} y ∗ = y ∗ and λ ~ ∗ = 0 \widetilde{\lambda}^{*}=0 λ ∗ = 0 , the KKT condition is satisfied. By LICQ and strong monotonicity (assumptions G.1, 3.4), the unique Nash equilibrium and optimal dual variable of the ghost lower-level game are y ~ ∗ = y ∗ \widetilde{y}^{*}=y^{*} y ∗ = y ∗ and λ ~ ∗ = 0 \widetilde{\lambda}^{*}=0 λ ∗ = 0 respectively. Using implicit function theorem, the derivatives of y ~ ∗ ( x ˉ ) \widetilde{y}^{*}(\bar{x}) y ∗ ( x ˉ ) and λ ~ ∗ ( x ˉ ) \widetilde{\lambda}^{*}(\bar{x}) λ ∗ ( x ˉ ) are given by
[ ∂ ∂ y V ( x ˉ , y ~ ∗ ) B I ⊤ B I 0 ] [ d d x y ~ ∗ ( x ˉ ) d d x λ ~ ∗ ( x ˉ ) ] = [ − ∂ ∂ x V ( x ˉ , y ~ ∗ ) − A I ] (19) \begin{bmatrix}
\frac{\partial}{\partial y}V(\bar{x},\widetilde{y}^{*}) & B_{I}^{\top}\\
B_{I} & 0
\end{bmatrix}
\begin{bmatrix}
\frac{d}{dx}\widetilde{y}^{*}(\bar{x})\\
\frac{d}{dx}\widetilde{\lambda}^{*}(\bar{x})
\end{bmatrix}
=
\begin{bmatrix}
-\frac{\partial}{\partial x}V(\bar{x},\widetilde{y}^{*})\\
-A_{I} \tag*{(19)}
\end{bmatrix} [ ∂ y ∂ V ( x ˉ , y ∗ ) B I B I ⊤ 0 ] [ d x d y ∗ ( x ˉ ) d x d λ ∗ ( x ˉ ) ] = [ − ∂ x ∂ V ( x ˉ , y ∗ ) − A I ] (19)
The derivatives of the KKT conditions of Q1 and Q2 are the same. Hence, d d x y ~ ∗ ( x ˉ ) = d d x y ∗ ( x ˉ ) \frac{d}{dx}\widetilde{y}^{*}(\bar{x})=\frac{d}{dx}y^{*}(\bar{x}) d x d y ∗ ( x ˉ ) = d x d y ∗ ( x ˉ ) .
Assuming exact active set identification (assumption H.2), we can directly apply the finite difference hypergradient approximation in problem P1 and the gradient approximation guarantees follow. However, the hyper-objective F F F is nonsmooth since the active set can change with x x x discontinuously. We can utilize Lemma H.5 (see [26 ] ) for minimizing Lipschitz nonsmooth nonconvex function F F F , and then we can conclude the convergence rate for problem Q1 with theorem H.6. Note that O ( δ ^ − 1 ϵ − 3 ) \mathcal{O}(\hat{\delta}^{-1}\epsilon^{-3}) O ( δ ^ − 1 ϵ − 3 ) for reaching a ( δ ^ , ϵ ) (\hat{\delta},\epsilon) ( δ ^ , ϵ ) -stationary point is the optimal rate for nonsmooth nonconvex optimization [11 ] .
Lemma H.5. Suppose F : R d → R F:\mathbb{R}^{d}\to\mathbb{R} F : R d → R is L L L -Lipschitz, and that ∥ ∇ ^ F ( ⋅ ) − ∇ F ( ⋅ ) ∥ ≤ α \lVert\widehat{\nabla}F(\cdot)-\nabla F(\cdot)\rVert\leq\alpha ∥ ∇ F ( ⋅ ) − ∇ F ( ⋅ )∥ ≤ α . Then there exists an algorithm which outputs a ( δ ^ , ϵ + α ) (\hat{\delta},\epsilon+\alpha) ( δ ^ , ϵ + α ) -stationary point with T = O ( δ ^ − 1 ϵ − 3 ) T=\mathcal{O}(\hat{\delta}^{-1}\epsilon^{-3}) T = O ( δ ^ − 1 ϵ − 3 ) calls to ∇ ^ F ( ⋅ ) \widehat{\nabla}F(\cdot) ∇ F ( ⋅ ) for arbitrary δ ^ > 0 \hat{\delta}>0 δ ^ > 0 .
Theorem H.6. Consider problem Q1. Under assumptions 3.4, 3.5, G.1 and H.2, there exists an algorithm that outputs an ( δ ^ , 2 ϵ ) (\hat{\delta},2\epsilon) ( δ ^ , 2 ϵ ) -stationary point of the hyper-objective F F F with O ~ ( δ ^ − 1 ϵ − 3 ) \widetilde{\mathcal{O}}(\hat{\delta}^{-1}\epsilon^{-3}) O ( δ ^ − 1 ϵ − 3 ) oracle calls to f f f and g i g_i g i for δ ^ > 0 \hat{\delta}>0 δ ^ > 0 .
Proof. The proof is identical to that of theorem C.6 and G.8. Except that the complexity of the outer loop is O ( δ ^ − 1 ϵ − 3 ) \mathcal{O}(\hat{\delta}^{-1}\epsilon^{-3}) O ( δ ^ − 1 ϵ − 3 ) instead of O ( ϵ − 2 ) \mathcal{O}(\epsilon^{-2}) O ( ϵ − 2 )
Discussion: Challenges of Fully First-Order Hypergradient Approximation in Asymmetric Game
Let x ∈ R x\in\mathbb{R} x ∈ R . Suppose the upper-level objective function is f ( x , y ) = u ⊤ y f(x,y)=u^{\top}y f ( x , y ) = u ⊤ y . Consider an unconstrained game ( x , [ N ] , { R n i } i ∈ [ N ] , { g i } i ∈ [ N ] ) (x,[N],\{\mathbb{R}^{n_i}\}_{i\in[N]},\{g_i\}_{i\in[N]}) ( x , [ N ] , { R n i } i ∈ [ N ] , { g i } i ∈ [ N ] ) , where the the cost function for each player i ∈ [ N ] i\in[N] i ∈ [ N ] is
g i ( x , y ) = 1 2 y i ⊤ Q i i y i + y i ⊤ ∑ j ≠ i Q i j y j + x c i ⊤ y i g_i(x,y)=\frac{1}{2}y_i^{\top}Q_{ii}y_i+y_i^{\top}\sum_{j\ne i}Q_{ij}y_j+xc_i^{\top}y_i g i ( x , y ) = 2 1 y i ⊤ Q ii y i + y i ⊤ j = i ∑ Q ij y j + x c i ⊤ y i
At equilibrium y ∗ y^{*} y ∗ , ∇ y i g i ( x , y i ∗ , y − i ∗ ) = 0 \nabla_{y_i}g_i(x,y_i^{*},y_{-i}^{*})=0 ∇ y i g i ( x , y i ∗ , y − i ∗ ) = 0 , ∀ i ∈ [ N ] \forall i\in[N] ∀ i ∈ [ N ] . Then,
V ( x , y ∗ ) = [ ∇ y 1 g 1 ( x , y ∗ ) ⋮ ∇ y N g N ( x , y ∗ ) ] = Q y + x c V(x,y^{*})=
\begin{bmatrix}
\nabla_{y_1}g_1(x,y^{*})\\
\vdots\\
\nabla_{y_N}g_N(x,y^{*})
\end{bmatrix}
=Qy+xc V ( x , y ∗ ) = ∇ y 1 g 1 ( x , y ∗ ) ⋮ ∇ y N g N ( x , y ∗ ) = Q y + x c
Solving V ( x , y ∗ ) = 0 V(x,y^{*})=0 V ( x , y ∗ ) = 0 , we have
y ∗ = − Q − 1 ( x c ) y^{*}=-Q^{-1}(xc) y ∗ = − Q − 1 ( x c )
Using implicit differentiation, the exact hypergradient term to be approximated is
( d d x y ∗ ( x ) ) ⊤ ∇ y f ( x , y ∗ ( x ) ) = ( − Q − 1 c ) ⊤ u = − c ⊤ Q − ⊤ u (20) \begin{aligned}
\left(\frac{d}{dx}y^{*}(x)\right)^{\top}\nabla_y f(x,y^{*}(x))=(-Q^{-1}c)^{\top}u \\
&=-c^{\top}Q^{-\top}u \tag*{(20)}
\end{aligned} ( d x d y ∗ ( x ) ) ⊤ ∇ y f ( x , y ∗ ( x )) = ( − Q − 1 c ) ⊤ u = − c ⊤ Q − ⊤ u (20)
We may add perturbation δ f ( x , y ) \delta f(x,y) δ f ( x , y ) to each player’s cost function. The equilibrium y δ ∗ y_\delta^* y δ ∗ of the perturbed game is given by
y δ ∗ = − Q − 1 ( x c + δ u ) = y ∗ − δ Q − 1 u y_\delta^*=-Q^{-1}(xc+\delta u)=y^*-\delta Q^{-1}u y δ ∗ = − Q − 1 ( x c + δ u ) = y ∗ − δ Q − 1 u
The first-order hypergradient approximation in § 4 only matches equation (20) when Q Q Q is symmetric
v δ ( x , y δ ∗ ) = 1 δ ∑ i = 1 N ∇ x g i ( x , y δ , i ∗ , y δ , ¬ i ∗ ) − ∇ x g i ( x , y ∗ ) = 1 δ ∑ i = 1 N c i ⊤ ( y δ , i ∗ − y i ∗ ) = 1 δ ∑ i = 1 N c i ⊤ [ δ Q − 1 u ] i = − c ⊤ Q − 1 u \begin{aligned}
v_\delta(x,y_\delta^*)&=\frac{1}{\delta}\sum_{i=1}^{N}\nabla_x g_i(x,y_{\delta,i}^*,y_{\delta,\neg i}^*)-\nabla_x g_i(x,y^*)\\
&=\frac{1}{\delta}\sum_{i=1}^{N}c_i^\top(y_{\delta,i}^*-y_i^*)\\
&=\frac{1}{\delta}\sum_{i=1}^{N}c_i^\top[\delta Q^{-1}u]_i\\
&=-c^\top Q^{-1}u
\end{aligned} v δ ( x , y δ ∗ ) = δ 1 i = 1 ∑ N ∇ x g i ( x , y δ , i ∗ , y δ , ¬ i ∗ ) − ∇ x g i ( x , y ∗ ) = δ 1 i = 1 ∑ N c i ⊤ ( y δ , i ∗ − y i ∗ ) = δ 1 i = 1 ∑ N c i ⊤ [ δ Q − 1 u ] i = − c ⊤ Q − 1 u
Alternatively, using results in § 5 , we can construct a perturbed game with cost function g ˉ i ( x , y ) = ϕ ϵ ( x , y ) − g i ( x , y ) + δ f ( x , y ) \bar{g}_i(x,y)=\phi_\epsilon(x,y)-g_i(x,y)+\delta f(x,y) g ˉ i ( x , y ) = ϕ ϵ ( x , y ) − g i ( x , y ) + δ f ( x , y ) , where ϕ ϵ ( x , y ) = 1 ϵ ∑ j = 1 N [ g j ( x , y j + ϵ ( y j − y j ∗ ) , y ¬ j ) − g j ( x , y ) ] \phi_\epsilon(x,y)=\frac{1}{\epsilon}\sum_{j=1}^{N}[g_j(x,y_j+\epsilon(y_j-y_j^*),y_{\neg j})-g_j(x,y)] ϕ ϵ ( x , y ) = ϵ 1 ∑ j = 1 N [ g j ( x , y j + ϵ ( y j − y j ∗ ) , y ¬ j ) − g j ( x , y )] is a potential function. Its equilibrium y ˉ δ ∗ \bar{y}_\delta^* y ˉ δ ∗ satisfies lim δ → 0 y ˉ δ ∗ = y ∗ \lim_{\delta\to0}\bar{y}_\delta^*=y^* lim δ → 0 y ˉ δ ∗ = y ∗ and d y ˉ δ ∗ d δ ∣ δ = 0 = − ( Q + ϵ D ) − ⊤ u \left.\frac{d\bar{y}_\delta^*}{d\delta}\right|_{\delta=0}=-(Q+\epsilon D)^{-\top}u d δ d y ˉ δ ∗ δ = 0 = − ( Q + ϵD ) − ⊤ u , where D = diag ( Q 11 , ⋯ , Q N N ) D=\operatorname{diag}(Q_{11},\cdots,Q_{NN}) D = diag ( Q 11 , ⋯ , Q N N ) is a block-diagonal matrix. Then the first-order hypergradient approximation v δ ( x , y ˉ δ ∗ ) v_\delta(x,\bar{y}_\delta^*) v δ ( x , y ˉ δ ∗ ) exactly matches the exact hypergradient in this toy example. The intuition of the construction is that the solution of V ˉ ( x , y ) = ( ∇ y 1 g ˉ 1 ( x , y ) , ⋯ , ∇ y N g ˉ N ( x , y ) ) = 0 \bar{V}(x,y)=(\nabla_{y_1}\bar{g}_1(x,y),\cdots,\nabla_{y_N}\bar{g}_N(x,y))=0 V ˉ ( x , y ) = ( ∇ y 1 g ˉ 1 ( x , y ) , ⋯ , ∇ y N g ˉ N ( x , y )) = 0 approximates the solution of Q ⊤ ( y − y ∗ ) + δ u = 0 Q^\top(y-y^*)+\delta u=0 Q ⊤ ( y − y ∗ ) + δ u = 0 as ϵ → 0 \epsilon\to0 ϵ → 0 . Solving ∇ y i ϕ ϵ ( x , y ˉ δ ∗ ) − ∇ y i g i ( x , y ˉ δ ∗ ) + δ ∇ y i f ( x , y ˉ δ ∗ ) = 0 \nabla_{y_i}\phi_\epsilon(x,\bar{y}_\delta^*)-\nabla_{y_i}g_i(x,\bar{y}_\delta^*)+\delta\nabla_{y_i}f(x,\bar{y}_\delta^*)=0 ∇ y i ϕ ϵ ( x , y ˉ δ ∗ ) − ∇ y i g i ( x , y ˉ δ ∗ ) + δ ∇ y i f ( x , y ˉ δ ∗ ) = 0 , ∀ i ∈ [ N ] \forall i\in[N] ∀ i ∈ [ N ] :
∇ y i ϕ ϵ ( x , y ) − ∇ y i g i ( x , y ) + δ ∇ y i f ( x , y ) = 1 ϵ ∑ j = 1 N [ ∇ y i g j ( x , y j + ϵ ( y j − y j ∗ ) , y ¬ j ) − ∇ y i g j ( x , y ) ] + ∇ y i g i ( x , y i + ϵ ( y i − y i ∗ ) , y ¬ i ) − ∇ y i g i ( x , y ) + δ ∇ y i f ( x , y ) = 1 ϵ [ Q i i ( y i + ϵ ( y i − y i ∗ ) ) + ∑ j ≠ i Q i j y j − Q i i y i − ∑ j ≠ i Q i j y j ] + 1 ϵ [ ∑ j ≠ i Q j i ⊤ ( y j + ϵ ( y j − y j ∗ ) ) − ∑ j ≠ i Q j i ⊤ y j ] + [ Q i i ( y i + ϵ ( y i − y i ∗ ) ) + ∑ j ≠ i Q i j y j ] − [ Q i i y i + ∑ j ≠ i Q i j y j ] + δ u i = Q i i ( y i − y i ∗ ) + ∑ j ≠ i Q j i ⊤ ( y j − y j ∗ ) + Q i i ϵ ( y i − y i ∗ ) = 0 \begin{aligned}
\nabla_{y_i}\phi_\epsilon(x,y)-\nabla_{y_i}g_i(x,y)+\delta\nabla_{y_i}f(x,y)
&=\frac{1}{\epsilon}\sum_{j=1}^{N}\left[\nabla_{y_i}g_j(x,y_j+\epsilon(y_j-y_j^*),y_{\neg j})-\nabla_{y_i}g_j(x,y)\right]\\
&\quad+\nabla_{y_i}g_i(x,y_i+\epsilon(y_i-y_i^*),y_{\neg i})-\nabla_{y_i}g_i(x,y)+\delta\nabla_{y_i}f(x,y)\\
&=\frac{1}{\epsilon}\left[Q_{ii}(y_i+\epsilon(y_i-y_i^*))+\sum_{j\ne i}Q_{ij}y_j-Q_{ii}y_i-\sum_{j\ne i}Q_{ij}y_j\right]\\
&\quad+\frac{1}{\epsilon}\left[\sum_{j\ne i}Q_{ji}^{\top}(y_j+\epsilon(y_j-y_j^*))-\sum_{j\ne i}Q_{ji}^{\top}y_j\right]\\
&\quad+\left[Q_{ii}(y_i+\epsilon(y_i-y_i^*))+\sum_{j\ne i}Q_{ij}y_j\right]\\
&\quad-\left[Q_{ii}y_i+\sum_{j\ne i}Q_{ij}y_j\right]+\delta u_i\\
&=Q_{ii}(y_i-y_i^*)+\sum_{j\ne i}Q_{ji}^{\top}(y_j-y_j^*)+Q_{ii}\epsilon(y_i-y_i^*)=0
\end{aligned} ∇ y i ϕ ϵ ( x , y ) − ∇ y i g i ( x , y ) + δ ∇ y i f ( x , y ) = ϵ 1 j = 1 ∑ N [ ∇ y i g j ( x , y j + ϵ ( y j − y j ∗ ) , y ¬ j ) − ∇ y i g j ( x , y ) ] + ∇ y i g i ( x , y i + ϵ ( y i − y i ∗ ) , y ¬ i ) − ∇ y i g i ( x , y ) + δ ∇ y i f ( x , y ) = ϵ 1 Q ii ( y i + ϵ ( y i − y i ∗ )) + j = i ∑ Q ij y j − Q ii y i − j = i ∑ Q ij y j + ϵ 1 j = i ∑ Q j i ⊤ ( y j + ϵ ( y j − y j ∗ )) − j = i ∑ Q j i ⊤ y j + Q ii ( y i + ϵ ( y i − y i ∗ )) + j = i ∑ Q ij y j − Q ii y i + j = i ∑ Q ij y j + δ u i = Q ii ( y i − y i ∗ ) + j = i ∑ Q j i ⊤ ( y j − y j ∗ ) + Q ii ϵ ( y i − y i ∗ ) = 0
Stacking these equations together, we have
Q ⊤ ( y − y ∗ ) + ϵ D ( y − y ∗ ) + δ u = 0 Q^\top(y-y^*)+\epsilon D(y-y^*)+\delta u=0 Q ⊤ ( y − y ∗ ) + ϵD ( y − y ∗ ) + δ u = 0
y = y ∗ + ( Q ⊤ + ϵ D ) − 1 ( − δ u ) y=y^*+(Q^\top+\epsilon D)^{-1}(-\delta u) y = y ∗ + ( Q ⊤ + ϵD ) − 1 ( − δ u )
We have achieved a desired formula for y ˉ δ ∗ \bar{y}_\delta^* y ˉ δ ∗ in this toy example.
Failure of Revelation Principle
To understand why non-truthful auction is important, we construct single-bidder and two-bidder settings where there exist non-truthful auctions with more revenue than any truthful auctions. Inspired by Example B.1. in [3 ] , the key idea is to introduce valuation-dependent constraints over bids.
Single-Item Single-Bidder
Let the valuation be t ∼ Uniform [ 0 , 1 ] t \sim\operatorname{Uniform}[0,1] t ∼ Uniform [ 0 , 1 ] . Fix α > 1 \alpha> 1 α > 1 . { a ∈ R + : a ≤ k ( t ) } \{a \in\mathbb{R}_{+}: a \le k(t)\} { a ∈ R + : a ≤ k ( t )} is the feasible set of the bid given valuation t t t , where
k ( t ) = { α if t < 2 3 , 1 if t ≥ 2 3 . k(t) =
\begin{cases}
\alpha& \text{if } t < \frac{2}{3},\\
1 & \text{if } t \ge\frac{2}{3}.
\end{cases} k ( t ) = { α 1 if t < 3 2 , if t ≥ 3 2 .
Consider the following allocation ρ : [ 0 , 1 ] → { 0 , 1 } \rho: [0,1] \to\{0,1\} ρ : [ 0 , 1 ] → { 0 , 1 } and payment rule P : [ 0 , 1 ] → R P : [0,1] \to\mathbb{R} P : [ 0 , 1 ] → R ,
ρ ( a ) = { 1 if a > 0 , 0 if a = 0. \rho(a) =
\begin{cases}
1 & \text{if } a > 0,\\
0 & \text{if } a = 0.
\end{cases} ρ ( a ) = { 1 0 if a > 0 , if a = 0.
P ( a ) = { 0 if a = 0 , 2 3 if 0 < a ≤ 1 , 1 3 if a > 1. P(a) =
\begin{cases}
0 & \text{if } a = 0,\\
\frac{2}{3} & \text{if } 0 < a \le1,\\
\frac{1}{3} & \text{if } a > 1.
\end{cases} P ( a ) = ⎩ ⎨ ⎧ 0 3 2 3 1 if a = 0 , if 0 < a ≤ 1 , if a > 1.
The utility of the bidder is u ( a , t ) = t ρ ( a ) − P ( a ) u(a,t) = t\rho(a) - P(a) u ( a , t ) = tρ ( a ) − P ( a ) . We can now derive the best response of the bidder that maximizes the utility. When t ≤ 1 3 t \le\frac{1}{3} t ≤ 3 1 , bidding anything above zero would results in negative utility. The best response is to bid zero. If 1 3 < t < 2 3 \frac{1}{3} < t < \frac{2}{3} 3 1 < t < 3 2 , the constraint is a ≤ α a \le\alpha a ≤ α . The best response is to bid a > 1 a > 1 a > 1 and pay 1 3 \frac{1}{3} 3 1 . If t ≥ 2 3 t \ge\frac{2}{3} t ≥ 3 2 , the constraint is a ≤ 1 a \le1 a ≤ 1 . The best response is to bid any 0 < a ≤ 1 0 < a \le1 0 < a ≤ 1 and pay 2 3 \frac{2}{3} 3 2 . Hence, the expected revenue of this auction is
0 + E t ∈ [ 1 3 , 2 3 ] [ 1 3 ] + E t ≥ 2 3 [ 2 3 ] = 1 3 ⋅ 1 3 + 1 3 ⋅ 2 3 = 1 3 \begin{aligned}
0 + \mathbb{E}_{t \in[\frac{1}{3},\frac{2}{3}]}\left[\frac{1}{3}\right] + \mathbb{E}_{t \ge\frac{2}{3}}\left[\frac{2}{3}\right] = \frac{1}{3}\cdot\frac{1}{3} + \frac{1}{3}\cdot\frac{2}{3} \\
&= \frac{1}{3}
\end{aligned} 0 + E t ∈ [ 3 1 , 3 2 ] [ 3 1 ] + E t ≥ 3 2 [ 3 2 ] = 3 1 ⋅ 3 1 + 3 1 ⋅ 3 2 = 3 1
On the other hand, the optimal truthful auction is Myerson’s optimal auction [29 ] with an expected revenue of 1 4 \frac{1}{4} 4 1 . Hence, the non-truthful auction we propose in fact has a higher revenue than any truthful auction in this setting.
Single-Item Two-Bidder
Let the valuation of bidder i i i be t i ∼ Uniform [ 0 , 1 ] t_i \sim\operatorname{Uniform}[0,1] t i ∼ Uniform [ 0 , 1 ] sampled independently. Consider a singleton B 1 = { l 1 = 0 } B_1 = \{l_1 = 0\} B 1 = { l 1 = 0 } and contiguous intervals B j = ( l j , m j ] B_j = (l_j,m_j] B j = ( l j , m j ] for j = 2 , 3 , 4 j = 2,3,4 j = 2 , 3 , 4 . m j = l j + 1 m_j = l_{j+1} m j = l j + 1 for j = 2 , 3 j = 2,3 j = 2 , 3 and m j > l j > l 1 m_j > l_j > l_1 m j > l j > l 1 for j = 2 , 3 , 4 j = 2,3,4 j = 2 , 3 , 4 . ⋃ j = 1 4 B j \bigcup_{j=1}^{4} B_j ⋃ j = 1 4 B j is the set of possible bids. Let 0 < c 1 < c 2 < c 3 < 1 0 < c_1 < c_2 < c_3 < 1 0 < c 1 < c 2 < c 3 < 1 . { a i ∈ R + : a i ≤ k ( t i ) } \{a_i \in\mathbb{R}_{+}: a_i \le k(t_i)\} { a i ∈ R + : a i ≤ k ( t i )} is the feasible set of the i i i th bidder’s bid, where
k ( t ) = { m 4 if t < c 2 , m 3 if c 2 ≤ t < c 3 , m 2 if c 3 ≤ t . k(t) =
\begin{cases}
m_4 & \text{if } t < c_2,\\
m_3 & \text{if } c_2 \le t < c_3,\\
m_2 & \text{if } c_3 \le t.
\end{cases} k ( t ) = ⎩ ⎨ ⎧ m 4 m 3 m 2 if t < c 2 , if c 2 ≤ t < c 3 , if c 3 ≤ t .
Consider the following allocation ρ : [ 0 , 1 ] × [ 0 , 1 ] → Δ 2 \rho: [0,1] \times[0,1] \to\Delta^2 ρ : [ 0 , 1 ] × [ 0 , 1 ] → Δ 2 and payment rule P : [ 0 , 1 ] × [ 0 , 1 ] → R 2 P : [0,1] \times[0,1] \to\mathbb{R}^2 P : [ 0 , 1 ] × [ 0 , 1 ] → R 2 ,
ρ i ( a 1 , a 2 ) = { 1 if a i ≥ a ¬ i , 0 if a i < a ¬ i or a i = 0. \rho_i(a_1,a_2) =
\begin{cases}
1 & \text{if } a_i \ge a_{\neg i},\\
0 & \text{if } a_i < a_{\neg i}\text{ or }a_i = 0.
\end{cases} ρ i ( a 1 , a 2 ) = { 1 0 if a i ≥ a ¬ i , if a i < a ¬ i or a i = 0.
P i ( a 1 , a 2 ) = { 0 if a i ∈ B 1 , c 3 ρ i ( a ) if a i ∈ B 2 , c 2 ρ i ( a ) if a i ∈ B 3 , c 1 ρ i ( a ) if a i ∈ B 4 . P_i(a_1,a_2) =
\begin{cases}
0 & \text{if } a_i \in B_1,\\
c_3\rho_i(a) & \text{if } a_i \in B_2,\\
c_2\rho_i(a) & \text{if } a_i \in B_3,\\
c_1\rho_i(a) & \text{if } a_i \in B_4.
\end{cases} P i ( a 1 , a 2 ) = ⎩ ⎨ ⎧ 0 c 3 ρ i ( a ) c 2 ρ i ( a ) c 1 ρ i ( a ) if a i ∈ B 1 , if a i ∈ B 2 , if a i ∈ B 3 , if a i ∈ B 4 .
The utility of the bidder i i i given valuation t i t_i t i is u i ( a , t i ) = t i ρ i ( a ) − P i ( a ) u_i(a,t_i) = t_i\rho_i(a) - P_i(a) u i ( a , t i ) = t i ρ i ( a ) − P i ( a ) . Now we derive the Nash equilibrium. Consider player i i i . When t i < c 1 t_i < c_1 t i < c 1 , The dominant strategy is to bid zero, because the utility will be negative even if bidder i i i wins. When t i ∈ [ c 1 , c 2 ) t_i \in[c_1,c_2) t i ∈ [ c 1 , c 2 ) , the constraint is a i ≤ m 4 a_i \le m_4 a i ≤ m 4 , the dominant strategy is bidding a i ∈ B 4 a_i \in B_4 a i ∈ B 4 because it results in the least payment and maximizes the chance of winning. Following this logic, a Nash equilibrium bidding strategy is
β ( t i ) = { 0 if t < c 1 any a i ∈ B 4 if t ∈ [ c 1 , c 2 ) , any a i ∈ B 3 if t ∈ [ c 2 , c 3 ) , any a i ∈ B 2 if t ∈ [ c 3 , 1 ) . \beta(t_i) =
\begin{cases}
0 & \text{if } t < c_1 \\
\text{any } a_i \in B_4 & \text{if } t \in[c_1,c_2), \\
\text{any } a_i \in B_3 & \text{if } t \in[c_2,c_3), \\
\text{any } a_i \in B_2 & \text{if } t \in[c_3,1).
\end{cases} β ( t i ) = ⎩ ⎨ ⎧ 0 any a i ∈ B 4 any a i ∈ B 3 any a i ∈ B 2 if t < c 1 if t ∈ [ c 1 , c 2 ) , if t ∈ [ c 2 , c 3 ) , if t ∈ [ c 3 , 1 ) .
Denote P 1 = Pr ( t i < c 1 ) \mathbb{P}_1 = \Pr(t_i < c_1) P 1 = Pr ( t i < c 1 ) , P 2 = Pr ( t i ∈ [ c 1 , c 2 ) ) \mathbb{P}_2 = \Pr(t_i \in[c_1,c_2)) P 2 = Pr ( t i ∈ [ c 1 , c 2 )) , P 3 = Pr ( t i ∈ [ c 2 , c 3 ) ) \mathbb{P}_3 = \Pr(t_i \in[c_2,c_3)) P 3 = Pr ( t i ∈ [ c 2 , c 3 )) , P 4 = Pr ( t i ∈ [ c 3 , c 4 ) ) \mathbb{P}_4 = \Pr(t_i \in[c_3,c_4)) P 4 = Pr ( t i ∈ [ c 3 , c 4 )) . The expected revenue can be computed by the following Python program:
def two_bidder_four_interval(C):
# C is a list containing c1, c2, c3
C = numpy.array(C)
P = (C[1:]-C[:-1])
P = numpy.concatenate([ [C[0]], P, [(1-C[-1])] ])
C_dim = len(C)
P_dim = len(P)
c1, c2, c3 = C[0], C[1], C[2]
payment_matrix = np.array([
[0, c1, c2, c3],
[c1, c1, c1, c1],
[c2, c1, c2, c2],
[c3, c1, c2, c3]
])
revenue = 0
for i in range(P_dim):
for j in range(P_dim):
revenue += payment_matrix[i, j] * P[i] * P[j]
return revenue
Substitute c 1 = 0.4 c_1 = 0.4 c 1 = 0.4 , c 2 = 0.6 c_2 = 0.6 c 2 = 0.6 , c 3 = 0.8 c_3 = 0.8 c 3 = 0.8 , B 2 = ( 0 , 1 ] B_2 = (0,1] B 2 = ( 0 , 1 ] , B 3 = ( 1 , 1.5 ] B_3 = (1,1.5] B 3 = ( 1 , 1.5 ] , B 4 = ( 1.5 , 2 ] B_4 = (1.5,2] B 4 = ( 1.5 , 2 ] . The expected revenue is 0.472. Myerson’s optimal auction is the optimal truthful auction and has an expected revenue 5 12 = 0.417 \frac{5}{12} = 0.417 12 5 = 0.417 . The proposed non-truthful auction has a revenue higher than any truthful auction.
Experiments
Theorem 6.1. Suppose G ( x ) G(x) G ( x ) is locally strongly monotone on a ball B ‾ r ( y ∗ ( x ) ) \overline{B}_r(y^*(x)) B r ( y ∗ ( x )) with radius r > 0 r > 0 r > 0 for all y ∗ ( x ) ∈ L N E ( G ( x ) ) y^*(x) \in LNE(G(x)) y ∗ ( x ) ∈ L N E ( G ( x )) and x ∈ X x \in X x ∈ X . Given initial iterate y y y , let S ( x , y , δ ^ ) : X × R n × R + → Y ( x ) S(x,y,\hat{\delta}): X \times\mathbb{R}^n \times\mathbb{R}_+ \to Y(x) S ( x , y , δ ^ ) : X × R n × R + → Y ( x ) be a solver can always return y ^ \hat{y} y ^ such that ∥ y ^ − y ∗ ( x ) ∥ ≤ δ ^ \lVert\hat{y} - y^*(x)\rVert\le\hat{\delta} ∥ y ^ − y ∗ ( x )∥ ≤ δ ^ whenever ∥ y − y ∗ ( x ) ∥ ≤ r \lVert y - y^*(x)\rVert\le r ∥ y − y ∗ ( x )∥ ≤ r and y ∗ ( x ) ∈ L N E ( G ( x ) ) y^*(x) \in LNE(G(x)) y ∗ ( x ) ∈ L N E ( G ( x )) .
In Algorithm 1, suppose ∥ y 0 − y ∗ ( x 0 ) ∥ ≤ r 2 \lVert y_0 - y^*(x_0)\rVert\le\frac{r}{2} ∥ y 0 − y ∗ ( x 0 )∥ ≤ 2 r for some y ∗ ( x 0 ) ∈ L N E ( G ( x 0 ) ) y^*(x_0) \in LNE(G(x_0)) y ∗ ( x 0 ) ∈ L N E ( G ( x 0 )) and the learning rate η \eta η is sufficiently small, then y k = z ∗ ( x k ) y_k = z^*(x_k) y k = z ∗ ( x k ) , ∀ k ∈ [ T ] \forall k \in[T] ∀ k ∈ [ T ] for some continuously differentiable function z ∗ : X → Y ( x ) z^*: X \to Y(x) z ∗ : X → Y ( x ) such that z ∗ ( x ) ∈ L N E ( G ( x ) ) z^*(x) \in LNE(G(x)) z ∗ ( x ) ∈ L N E ( G ( x )) .
Proof. Let y ∗ ( x k ) ∈ L N E ( G ( x k ) ) y^*(x_k) \in LNE(G(x_k)) y ∗ ( x k ) ∈ L N E ( G ( x k )) . Since the KKT matrix K ( x k , y ∗ ( x k ) ) K(x_k,y^*(x_k)) K ( x k , y ∗ ( x k )) is invertible, by implicit function theorem, there exists a neighborhood N x k × N y ∗ ( x k ) N_{x_k} \times N_{y^*(x_k)} N x k × N y ∗ ( x k ) of ( x k , y ∗ ( x k ) ) (x_k,y^*(x_k)) ( x k , y ∗ ( x k )) and a continuously differentiable and L k L_k L k -Lipschitz continuous function z ∗ : N x k → N y ∗ ( x k ) z^*: N_{x_k} \to N_{y^*(x_k)} z ∗ : N x k → N y ∗ ( x k ) such that z ∗ ( x ) ∈ L N E ( G ( x ) ) z^*(x) \in LNE(G(x)) z ∗ ( x ) ∈ L N E ( G ( x )) , ∀ x ∈ N x k \forall x \in N_{x_k} ∀ x ∈ N x k , and z ∗ ( x k ) = y ∗ ( x k ) z^*(x_k) = y^*(x_k) z ∗ ( x k ) = y ∗ ( x k ) . Suppose ∥ x k − x k + 1 ∥ ≤ O ( η ) \lVert x_k-x_{k+1}\rVert\le O(\eta) ∥ x k − x k + 1 ∥ ≤ O ( η ) is sufficiently small such that x k + 1 ∈ N x k x_{k+1} \in N_{x_k} x k + 1 ∈ N x k and ∥ x k − x k + 1 ∥ ≤ r 2 \lVert x_k-x_{k+1}\rVert\le\frac{r}{2} ∥ x k − x k + 1 ∥ ≤ 2 r , then ∥ z ∗ ( x k ) − z ∗ ( x k + 1 ) ∥ ≤ r 2 \lVert z^*(x_k)-z^*(x_{k+1})\rVert\le\frac{r}{2} ∥ z ∗ ( x k ) − z ∗ ( x k + 1 )∥ ≤ 2 r . Suppose ∥ y k − y ∗ ( x k ) ∥ ≤ r 2 \lVert y_k-y^*(x_k)\rVert\le\frac{r}{2} ∥ y k − y ∗ ( x k )∥ ≤ 2 r , ∥ y k − z ∗ ( x k + 1 ) ∥ ≤ r \lVert y_k-z^*(x_{k+1})\rVert\le r ∥ y k − z ∗ ( x k + 1 )∥ ≤ r by triangle inequality. For arbitrary precision δ ^ \hat{\delta} δ ^ , the solver outputs y k + 1 = S ( x k + 1 , y k , δ ^ ) y_{k+1}=S(x_{k+1},y_k,\hat{\delta}) y k + 1 = S ( x k + 1 , y k , δ ^ ) such that ∥ y k + 1 − z ∗ ( x k + 1 ) ∥ ≤ δ ^ \lVert y_{k+1}-z^*(x_{k+1})\rVert\le\hat{\delta} ∥ y k + 1 − z ∗ ( x k + 1 )∥ ≤ δ ^ . By induction, the claim is proved. ▫
Experimental Details
Each bidding strategy β y i \beta_{y_i} β y i is two-layer fully connected neural network of width 10 with SeLU activation ([23 ] ) and ReLU activation ([1 ] ) for the output layer. The allocation rule is a two-layer fully connected neural network of width 10 with SeLU activation and Layer Normalization ([2 ] ). We use Adam ([22 ] ) to optimize the parameters of the auction mechanism. We choose the hyperobjective step size η = 10 − 5 \eta= 10^{-5} η = 1 0 − 5 for single-item auction and η = 10 − 4 \eta= 10^{-4} η = 1 0 − 4 for multi-item auction. For single-item auction, we run gradient descent with step size 10 − 4 10^{-4} 1 0 − 4 for 30000 iterations to solve the original game, and with step size 10 − 4 10^{-4} 1 0 − 4 for 1000 iterations to solve the perturbed game. For multi-item auction, we run gradient descent with step size 10 − 4 10^{-4} 1 0 − 4 for 40000 iterations to solve the original game, and with step size 10 − 4 10^{-4} 1 0 − 4 for 2000 iterations to solve the perturbed game.
We initialize the allocation rule of the mechanism to mimick first-price auction so that the initial lower-level game is easy to solve. Let p ∈ Δ N + 1 p \in\Delta^{N+1} p ∈ Δ N + 1 be the output of the allocation rule of first-price auction. Let q ∈ Δ N + 1 q \in\Delta^{N+1} q ∈ Δ N + 1 be the output of the allocation rule of the neural-network-parameterized allocation rule. Then, we run Adam with step size 10 − 4 10^{-4} 1 0 − 4 for 10 6 10^{6} 1 0 6 iterations to minimize their KL divergence D K L ( p ∥ q ) ∝ ∑ i p i log q i D_{\mathrm{KL}}(p \| q) \propto\sum_i p_i \log q_i D KL ( p ∥ q ) ∝ ∑ i p i log q i .
Figure 5. Equilibrium bidding strategy and allocation rule for asymmetric single-item auction. For allocation probability, x-axis and y-axis are the bid of bidder 1 and bidder 2 respectively.
Figure 6. Comparison between non-truthful auction discovered by our method and truthful auction discovered by GemNet. For each plot, x-axis and y-axis is the bid of bidder 1 for item 1 and item 2 respectively. The top row and bottom row show the probability of item 1 and item 2 being allocated to bidder 1 respectively, given the bid of bidder 2 is ( 0 , 0.6 ) (0,0.6) ( 0 , 0.6 ) .