Introduction
Optimal Transport (OT) is a standard way to lift any distance defined over a Polish space ( Ω , d ) (\Omega,d) ( Ω , d ) to the space of probability measures defined over Ω \Omega Ω [57 , 45 ] . Due to their geometrical properties the distances induced by OT problems, i.e., the Wasserstein distances, have been proven to be a valuable tool to address several key sub-tasks in central computer science problems, e.g., Machine Learning [47 , 19 , 24 , 2 , 41 ] , Computer Vision [43 , 44 , 40 , 3 ] , and Natural Language Processing [31 , 42 ] . Owing to the growing interest for OT-related distances, the demand for fast and efficient computation methods grew accordingly. Indeed, in its modern formulation, Wasserstein Distance induced by a cost function c : X × Y → [ 0 , + ∞ [ c:\mathcal{X}\times\mathcal{Y}\to[0,+\infty[ c : X × Y → [ 0 , + ∞ [ between two probability measures μ ∈ P ( X ) \mu\in\mathcal{P}(\mathcal{X}) μ ∈ P ( X ) and ν ∈ P ( Y ) \nu\in\mathcal{P}(\mathcal{Y}) ν ∈ P ( Y ) is defined as the minimum of the following linear problem
min π ∈ Π ( μ , ν ) ∫ X × Y c ( x , y ) π ( d x , d y ) , (1) \min_{\pi\in\Pi(\mu,\nu)} \int_{\mathcal{X}\times\mathcal{Y}} c(x,y)\,\pi(dx,dy), \tag*{(1)} π ∈ Π ( μ , ν ) min ∫ X × Y c ( x , y ) π ( d x , d y ) , (1)
where Π ( μ , ν ) = { π ∈ P ( X × Y ) ∣ π ( 1 ) = μ , π ( 2 ) = ν } \Pi(\mu,\nu)=\{\pi\in\mathcal{P}(\mathcal{X}\times\mathcal{Y})\mid\pi^{(1)}=\mu,\ \pi^{(2)}=\nu\} Π ( μ , ν ) = { π ∈ P ( X × Y ) ∣ π ( 1 ) = μ , π ( 2 ) = ν } with π ( 1 ) \pi^{(1)} π ( 1 ) and π ( 2 ) \pi^{(2)} π ( 2 ) being the marginal of π \pi π over X \mathcal{X} X and the marginal of π \pi π over Y \mathcal{Y} Y , respectively. For discrete probability measures, problem (1.1) becomes a Linear Programming (LP) problem, whose computational cost becomes prohibitive when the dimension of the problem increases [29 ] . The demand for fast algorithms capable of solving the OT problem has spurred extensive research aimed at reducing the computational cost of these methods by considering a Linear Programming problem that exploits the structure of the problem [37 , 4 ] or by approaching an entropic regularization version of the problem [18 , 54 , 1 ] .
Despite the success of the Wasserstein Distances, they are limited as they compare only positive measures with the same total mass. This factor represents a flaw in any application where the total mass of the object under study is important, e.g., in shape analysis [16 , 54 ] or in persistence diagrams comparison [56 , 32 ] . Moreover, since the OT problem imposes the conservation of the mass, the resulting Wasserstein Distances are sensitive to noise [46 ] , hindering the applicability of the Wasserstein Distances to fields such as robust learning [6 ] , domain adaptation [21 ] , imaging [34 ] , biology [49 ] , and text matching [55 ] . To overcome these shortcomings, there have been several proposals to extend the Wasserstein Distances to the set of positive measures, such as partial optimal transport [22 ] , the Wasserstein Fisher-Rao distance [28 , 16 ] , or the Kantorovich Rubinstein distance [34 ] . In this paper, we focus on the proposal presented by Liero, Mielke and Savaré, where the hard constraint on the marginal of the transportation plan between the two probability measures is replaced by a suitable Csiszár penalty term in the objective function [36 ] . More formally, given two positive measures μ \mu μ and ν \nu ν , the problem we consider is as follows
min π ∈ M + ( X × Y ) U T ϕ , c ( π ) : = min π ∈ M + ( X × Y ) ∫ X × Y c ( x , y ) π ( d x , d y ) + D ϕ ( π ( 1 ) ∣ μ ) + D ϕ ( π ( 2 ) ∣ ν ) , \min_{\pi\in\mathcal{M}_{+}(\mathcal{X}\times\mathcal{Y})}\mathrm{UT}_{\phi,c}(\pi):=\min_{\pi\in\mathcal{M}_{+}(\mathcal{X}\times\mathcal{Y})}\int_{\mathcal{X}\times\mathcal{Y}}c(x,y)\pi(\mathrm{d}x,\mathrm{d}y)+D_{\phi}(\pi^{(1)}\mid\mu)+D_{\phi}(\pi^{(2)}\mid\nu), π ∈ M + ( X × Y ) min UT ϕ , c ( π ) := π ∈ M + ( X × Y ) min ∫ X × Y c ( x , y ) π ( d x , d y ) + D ϕ ( π ( 1 ) ∣ μ ) + D ϕ ( π ( 2 ) ∣ ν ) ,
where D ϕ D_{\phi} D ϕ is the Csiszár divergence induced by a convex function ϕ \phi ϕ . Unlike the classic Wasserstein Distance, the unbalanced optimal transportation problem proposed in [36 ] can be naturally formulated as the minimization of a convex function over a convex set, enabling the use of classic routines for convex programming.
Related Works
As per the standard OT problem, most approaches for the UOT rely on a convex relaxation of the objective function that is then handled through the Sinkhorn algorithm [18 , 53 ] . The first Sinkhorn-like algorithm for unbalanced optimal transport was proposed in [17 ] , where the authors showed how to extend the entropic regularization techniques used for the classic OT problem. This approach was subsequently improved in [50 ] , where the authors enhanced the numerical stability of the method. For the one dimensional case, the state-of-the-art solver of the UOT was introduced in [52 ] , where the authors considered a translation-invariant version of the dual UOT problem that can be efficiently solved using the Frank-Wolfe algorithm. The results presented in [52 ] have led to the study of a Sliced Unbalanced Wasserstein Distances [12 ] in which the UOT distance is computed by taking the average over all the UOT distances computed over all the possible one-dimensional projections. A different approach based on entropic regularization is presented in [48 ] , where the authors impose an additional low-rank constraint to the UOT problem, enabling a more compact representation of the feasible set of transport plans.
An alternative regularization approach was presented in [8 , 39 ] , where the authors define a UOT problem regularization via an ℓ 2 \ell_2 ℓ 2 penalty on the transport plan rather than an entropic one, yielding a strongly convex objective function that can be tackled via standard gradient schemes [39 ] . In [15 ] , the authors approach the solution of the UOT problem by recasting it as a convex penalized linear regression problem, which has been proven to be efficient when handling specific Csiszár divergences. Lastly, in [25 ] , the authors consider a semi-unbalanced problem in which the penalty is imposed only on one of the two marginals [38 ] .
To the best of our knowledge, most of the works within this field are based on approaching the problem from the dual point of view and by adding a regularization term to the objective function, be it the Kullback Leibler divergence [17 ] or be it an ℓ 2 \ell_2 ℓ 2 penalization term [39 ] . A relatively under-explored research direction is to leverage some well-established algorithm for constrained optimization, such as the Frank-Wolfe (FW) algorithm, to retrieve the optimal solution for the primal formulation. Indeed, to the best of our knowledge, the only papers that use the FW routine to retrieve the solution to the UOT problem are [52 ] , where the authors use the FW routine to approach the dual formulation induced by the one dimensional UOT problem, and [25 ] , where the authors define a block-coordinate version of the algorithm that exploits the structure of their semi-unbalanced problem. Both papers showed that it is possible to define fast and efficient Frank-Wolfe routines for the UOT whenever one can harness the structure of the problem: in [52 ] they do so by showing that the Linear Minimization Oracle associated with each iteration can be performed in closed form, while in [25 ] it is done by decomposing the feasible set into the direct product of small simplex sets.
The Frank-Wolfe algorithm, also known as the conditional gradient method, is a classic optimization algorithm for constrained problems introduced by Frank and Wolfe [23 ] . Classical convergence guarantees have been extensively studied [14 ] along with the conditions that guarantee linear convergence [35 , 20 ] and enable to identify the solution support in a finite number of steps [11 ] . To ease the conditions needed to make the standard FW algorithm converge linearly, several variants, including away-step [58 ] , pairwise [20 ] , and fully corrective Frank-Wolfe, have been proposed to improve convergence rates under suitable geometric assumptions [33 ] . In recent years, the interest for the Frank-Wolfe algorithm and its variants has renewed due to its applications in sub-modular optimization problems [5 ] , variational inference problems [30 ] , traffic routing [26 ] , clustering [10 ] , and sparse neural network training [27 ] . We refer the reader to [9 ] and [13 ] for a survey on the family of Frank-Wolfe algorithms.
Our Contribution
In this paper, we propose a novel approach to solve the Unbalanced Optimal Transportation (UOT) problem presented in [36 ] that hinges on a clever exploitation of the structure of the UOT solutions. Our paper is structured as follows.
First, we study the properties of the solution to the UOT problem. In particular, given any couple of discrete positive measures μ \mu μ and ν \nu ν , we show that there exists at least an optimal transportation plan between μ \mu μ and ν \nu ν that is discrete and characterize the finite set of points on which such solution is supported. Building on this preliminary result, we show that any solution has finite measure and provide an upper and lower bound on the marginals of any solution that depend only on the specific of the problem. In particular, we provide an upper bound on the measure that any solution assigns to any couple of points in its support. These upper bounds reveal that, under suitable assumptions, the maximum distance between couples of points that belong to the support is bounded by a global constant depending on the cost function and the Csiszár divergence defining the problem. While a similar characterization was known for the Total Variation case [51 ] , our approach can be applied to any UOT problem following the formulation presented in [36 ] .
We then propose a truncated version of the UOT problem that is obtained by introducing a parameter R R R that saturates the cost function c c c . We show that the truncated problem admits a simple formulation, reducing the total number of variables needed to describe the associated UOT problem. Moreover, any truncated problem naturally yields a feasible point for the original UOT problem that acts as an approximated solution of the problem. We provide an upper bound on the primal sub-optimality gap between the real solution and the approximated solution retrieved from the truncated formulation, revealing that the gap decreases as a polynomial function of R R R . Noticeably, all of our proposed bounds on the optimal solution apply to general framework.
We then specialize our study to the case in which the Csiszár divergence belongs to the class of powerlike entropies, a class of entropies that includes the Kullback Leibler and the quadratic penalty function as specific sub-cases. To approach this class of problems, we define an efficient implementation of the Pairwise Frank-Wolfe routine capable of solving the UOT problem for any powerlike entropy and any couple of positive measures. Our routine cleverly combines our theoretical results to save the iterates of the problem in a fast and efficient way. As a by product, we also study the convergence speed of the Pairwise Frank-Wolfe algorithm under the assumptions that the objective has local Hölder continuous gradient and is p p p -uniformly convex. Moreover, we provide a support identifiability property for the Pairwise Frank-Wolfe algorithm, which ensures that our routine finds the support of the optimal solution within a finite number of steps.
Notation and problem formulation
We collect the notation and assumptions used throughout the paper.
Basic notation
We write R + = [ 0 , + ∞ [ \mathbb{R}_{+} = [0,+\infty[ R + = [ 0 , + ∞ [ , R + + = ] 0 , + ∞ [ \mathbb{R}_{++} = ]0,+\infty[ R ++ = ] 0 , + ∞ [ , and [ d ] = { 1 , … , d } [d] = \{1,\ldots,d\} [ d ] = { 1 , … , d } . The class Γ 0 ( R d ) \Gamma_{0}(\mathbb{R}^{d}) Γ 0 ( R d ) consists of proper, lower semicontinuous convex functions from R d \mathbb{R}^{d} R d to ] − ∞ , + ∞ ] ]-\infty,+\infty] ] − ∞ , + ∞ ] . For a set C C C , ι C \iota_{C} ι C denotes its indicator function, equal to zero on C C C and + ∞ +\infty + ∞ elsewhere. We use ∂ f \partial f ∂ f for the convex subdifferential and N C = ∂ ι C N_{C} = \partial\iota_{C} N C = ∂ ι C for the normal cone of a closed convex set. Interior and relative interior are denoted by int \operatorname{int} int and ri \operatorname{ri} ri , respectively.
For a finite-dimensional vector z z z , ∥ z ∥ p \lVert z\rVert_{p} ∥ z ∥ p is its usual ℓ p \ell_{p} ℓ p norm; ∥ z ∥ \lVert z\rVert ∥ z ∥ denotes the Euclidean norm and ∥ z ∥ ∞ : = max i ∈ [ d ] ∣ z i ∣ \lVert z\rVert_{\infty} := \max_{i\in[d]} |z_{i}| ∥ z ∥ ∞ := max i ∈ [ d ] ∣ z i ∣ denotes the maximum norm. Under these notations, we set B h , p ( z ) = { w : ∥ w − z ∥ p ≤ h } B_{h,p}(z) = \{w : \lVert w-z\rVert_{p} \le h\} B h , p ( z ) = { w : ∥ w − z ∥ p ≤ h } . The inner product on matrices is ⟨ a , b ⟩ = ∑ i , j a i j b i j \langle a,b\rangle= \sum_{i,j} a_{ij}b_{ij} ⟨ a , b ⟩ = ∑ i , j a ij b ij , and 1 d \mathbf{1}_{d} 1 d is the vector of d d d ones. For a nonnegative vector or matrix, ∥ x ∥ 1 \lVert x\rVert_{1} ∥ x ∥ 1 is the sum of its entries. For a real number t t t , put t + = max { t , 0 } t_{+} = \max\{t,0\} t + = max { t , 0 } .
For a compact metric space Z \mathcal{Z} Z , M + ( Z ) \mathcal{M}_{+}(\mathcal{Z}) M + ( Z ) is the set of finite nonnegative Borel measures and C ( Z ) C(\mathcal{Z}) C ( Z ) is the space of continuous real functions. Given Z ′ ⊆ Z \mathcal{Z}' \subseteq\mathcal{Z} Z ′ ⊆ Z and μ ∈ M + ( Z ) \mu\in\mathcal{M}_{+}(\mathcal{Z}) μ ∈ M + ( Z ) , we denote by μ ∣ Z ′ \mu|_{\mathcal{Z}'} μ ∣ Z ′ the restriction of μ \mu μ to Z ′ \mathcal{Z}' Z ′ , that is μ ∣ Z ′ ( A ) = μ ( A ∩ Z ′ ) \mu|_{\mathcal{Z}'}(A) = \mu(A \cap\mathcal{Z}') μ ∣ Z ′ ( A ) = μ ( A ∩ Z ′ ) for every measurable A ⊆ Z A \subseteq\mathcal{Z} A ⊆ Z . The support of a measure μ \mu μ is denoted by supp ( μ ) \operatorname{supp}(\mu) supp ( μ ) . In what follows, we say that a measure is a discrete measure if it has finite support. If T T T is measurable, T # μ ( E ) = μ ( T − 1 ( E ) ) T_{\#}\mu(E) = \mu(T^{-1}(E)) T # μ ( E ) = μ ( T − 1 ( E )) denotes the pushforward. For a measure π ∈ M + ( X × Y ) \pi\in\mathcal{M}_{+}(\mathcal{X}\times\mathcal{Y}) π ∈ M + ( X × Y ) , its marginals are π ( 1 ) = ( pr X ) # π \pi^{(1)} = (\operatorname{pr}_{\mathcal{X}})_{\#}\pi π ( 1 ) = ( pr X ) # π and π ( 2 ) = ( pr Y ) # π \pi^{(2)} = (\operatorname{pr}_{\mathcal{Y}})_{\#}\pi π ( 2 ) = ( pr Y ) # π .
Let f f f be real-valued and differentiable on an open set containing Ω ⊂ R d \Omega\subset\mathbb{R}^{d} Ω ⊂ R d . Its gradient is α \alpha α -Hölder continuous on Ω \Omega Ω , with α ∈ ] 0 , 1 ] \alpha\in]0,1] α ∈ ] 0 , 1 ] , if there is a finite H α ≥ 0 H_{\alpha} \ge0 H α ≥ 0 such that
∥ ∇ f ( x ) − ∇ f ( y ) ∥ ≤ H α ∥ x − y ∥ α ( x , y ∈ Ω ) . \lVert\nabla f(x) - \nabla f(y)\rVert\le H_{\alpha}\lVert x-y\rVert^{\alpha}
\qquad(x,y \in\Omega). ∥ ∇ f ( x ) − ∇ f ( y )∥ ≤ H α ∥ x − y ∥ α ( x , y ∈ Ω ) .
The case α = 1 \alpha= 1 α = 1 is Lipschitz continuity of the gradient. For p ≥ 2 p \ge2 p ≥ 2 , f f f is p p p -uniformly convex on Ω \Omega Ω with constant κ p > 0 \kappa_{p} > 0 κ p > 0 if
f ( y ) ≥ f ( x ) + ⟨ ∇ f ( x ) , y − x ⟩ + κ p p ∥ y − x ∥ p ( x , y ∈ Ω ) . (2) f(y) \ge f(x) + \langle\nabla f(x), y-x\rangle+ \frac{\kappa_{p}}{p}\lVert y-x\rVert^{p}
\qquad(x,y \in\Omega). \tag*{(2)} f ( y ) ≥ f ( x ) + ⟨ ∇ f ( x ) , y − x ⟩ + p κ p ∥ y − x ∥ p ( x , y ∈ Ω ) . (2)
For p = 2 p=2 p = 2 , this is strong convexity. Notice that these constants may depend on Ω \Omega Ω ; a global assertion requires the estimate on the whole set under consideration with one constant.
Finally, if f f f has nonempty minimizer set X ⋆ \mathcal{X}^{\star} X ⋆ and minimum f ⋆ f^{\star} f ⋆ , it is p p p -conditioned on Ω ⊃ X ⋆ \Omega\supset\mathcal{X}^{\star} Ω ⊃ X ⋆ if there is a constant v p > 0 v_p > 0 v p > 0 such that
f ( x ) − f ⋆ ≥ v p dist ( x , X ⋆ ) p ( x ∈ Ω ) , (3) f(x)-f^{\star} \ge v_p \operatorname{dist}(x,\mathcal{X}^{\star})^p \qquad(x \in\Omega), \tag*{(3)} f ( x ) − f ⋆ ≥ v p dist ( x , X ⋆ ) p ( x ∈ Ω ) , (3)
where dist ( x , X ⋆ ) : = min x ⋆ ∈ X ⋆ ∥ x − x ⋆ ∥ \operatorname{dist}(x,\mathcal{X}^{\star}) := \min_{x^{\star} \in\mathcal{X}^{\star}} \lVert x-x^{\star}\rVert dist ( x , X ⋆ ) := min x ⋆ ∈ X ⋆ ∥ x − x ⋆ ∥ is the standard element-to-set distance.
Let X \mathcal{X} X and Y \mathcal{Y} Y be nonempty compact metric spaces, let c : X × Y → [ 0 , + ∞ [ c:\mathcal{X} \times\mathcal{Y} \to[0,+\infty[ c : X × Y → [ 0 , + ∞ [ be continuous, and let μ ∈ M + ( X ) \mu\in\mathcal{M}_{+}(\mathcal{X}) μ ∈ M + ( X ) and ν ∈ M + ( Y ) \nu\in\mathcal{M}_{+}(\mathcal{Y}) ν ∈ M + ( Y ) be finite nonnegative measures. Given x ∈ X x \in\mathcal{X} x ∈ X , we denote by δ x ∈ M + ( X ) \delta_x \in\mathcal{M}_{+}(\mathcal{X}) δ x ∈ M + ( X ) the Dirac delta centred in x x x . In what follows, we always assume that the support of μ \mu μ and ν \nu ν are finite so that μ \mu μ and ν \nu ν are positive linear combination of different centred Dirac deltas. Moreover, ϕ : R → ] − ∞ , + ∞ ] \phi:\mathbb{R} \to]-\infty,+\infty] ϕ : R → ] − ∞ , + ∞ ] is proper, lower semicontinuous, and convex, and satisfies
ϕ ≥ 0 , ] 0 , + ∞ [ ⊆ dom ( ϕ ) ⊆ [ 0 , + ∞ [ , and 0 ∈ ∂ ϕ ( 1 ) , (4) \phi\ge0,\qquad]0,+\infty[ \subseteq\operatorname{dom}(\phi) \subseteq[0,+\infty[,\qquad\text{and}\qquad0 \in\partial\phi(1), \tag*{(4)} ϕ ≥ 0 , ] 0 , + ∞ [ ⊆ dom ( ϕ ) ⊆ [ 0 , + ∞ [ , and 0 ∈ ∂ ϕ ( 1 ) , (4)
where dom ( ϕ ) \operatorname{dom}(\phi) dom ( ϕ ) denotes the domain of ϕ \phi ϕ . In particular, ϕ \phi ϕ is finite on ] 0 , + ∞ [ ]0,+\infty[ ] 0 , + ∞ [ , whereas ϕ ( 0 ) \phi(0) ϕ ( 0 ) may be infinite. In what follows, we assume that
ϕ ∞ ′ : = lim t → + ∞ ϕ ( t ) t ∈ ] 0 , + ∞ ] , ϕ 0 ′ : = lim t ↓ 0 ϕ + ′ ( t ) ∈ [ − ∞ , 0 ] . (5) \phi'_{\infty} := \lim_{t \to+\infty} \frac{\phi(t)}{t} \in]0,+\infty],\qquad\phi'_0 := \lim_{t \downarrow0} \phi'_+(t) \in[-\infty,0]. \tag*{(5)} ϕ ∞ ′ := t → + ∞ lim t ϕ ( t ) ∈ ] 0 , + ∞ ] , ϕ 0 ′ := t ↓ 0 lim ϕ + ′ ( t ) ∈ [ − ∞ , 0 ] . (5)
When ϕ ( 0 ) < + ∞ \phi(0) < +\infty ϕ ( 0 ) < + ∞ , ϕ 0 ′ \phi'_0 ϕ 0 ′ is the right derivative of ϕ \phi ϕ at zero; when ϕ ( 0 ) = + ∞ \phi(0) = +\infty ϕ ( 0 ) = + ∞ , ϕ 0 ′ = − ∞ \phi'_0=-\infty ϕ 0 ′ = − ∞ . For t > 0 t>0 t > 0 , let ϕ + ′ ( t ) \phi'_+(t) ϕ + ′ ( t ) and ϕ − ′ ( t ) \phi'_-(t) ϕ − ′ ( t ) denote the right and left derivatives at t t t , then
∂ ϕ ( t ) = [ ϕ − ′ ( t ) , ϕ + ′ ( t ) ] , ϕ 0 ′ ≤ ϕ − ′ ( t ) ≤ ϕ + ′ ( t ) ≤ ϕ ∞ ′ . (6) \partial\phi(t) = [\phi'_-(t),\phi'_+(t)],\qquad\phi'_0 \le\phi'_-(t) \le\phi'_+(t) \le\phi'_{\infty}. \tag*{(6)} ∂ ϕ ( t ) = [ ϕ − ′ ( t ) , ϕ + ′ ( t )] , ϕ 0 ′ ≤ ϕ − ′ ( t ) ≤ ϕ + ′ ( t ) ≤ ϕ ∞ ′ . (6)
At zero, ∂ ϕ ( 0 ) = ] − ∞ , ϕ 0 ′ ] \partial\phi(0) = ]-\infty,\phi'_0] ∂ ϕ ( 0 ) = ] − ∞ , ϕ 0 ′ ] if ϕ 0 ′ \phi'_0 ϕ 0 ′ is finite, and ∂ ϕ ( 0 ) = ∅ \partial\phi(0)=\varnothing ∂ ϕ ( 0 ) = ∅ otherwise.
For a reference m ≥ 0 m \ge0 m ≥ 0 , define the closed marginal penalty
φ m ( t ) : = { m ϕ ( t / m ) , m > 0 , t ≥ 0 , ϕ ∞ ′ t , m = 0 , t ≥ 0 , ϕ ∞ ′ < + ∞ , 0 , m = 0 , t = 0 , ϕ ∞ ′ = + ∞ , + ∞ , otherwise . (7) \varphi_m(t) :=
\begin{cases}
m\phi(t/m), & m>0,\ t \ge0,\\
\phi'_{\infty}t, & m=0,\ t \ge0,\ \phi'_{\infty}<+\infty,\\
0, & m=0,\ t=0,\ \phi'_{\infty}=+\infty,\\
+\infty, & \text{otherwise}.
\end{cases} \tag*{(7)} φ m ( t ) := ⎩ ⎨ ⎧ m ϕ ( t / m ) , ϕ ∞ ′ t , 0 , + ∞ , m > 0 , t ≥ 0 , m = 0 , t ≥ 0 , ϕ ∞ ′ < + ∞ , m = 0 , t = 0 , ϕ ∞ ′ = + ∞ , otherwise . (7)
Thus φ 0 = ι { 0 } \varphi_0 = \iota_{\{0\}} φ 0 = ι { 0 } when ϕ ∞ ′ = + ∞ \phi'_{\infty}=+\infty ϕ ∞ ′ = + ∞ . The function in (2.6) is also known as the perspective of ϕ \phi ϕ with the reference m m m fixed. By definition, the subdifferential of φ m \varphi_m φ m is
∂ φ m ( t ) = { ∂ ϕ ( t / m ) , m > 0 , { ϕ ∞ ′ } , m = 0 , ϕ ∞ ′ < + ∞ , t > 0 , ] − ∞ , ϕ 0 ′ ] , m = 0 , ϕ ∞ ′ < + ∞ , t = 0 , R , m = 0 , ϕ ∞ ′ = + ∞ , t = 0 , ∅ , otherwise . (8) \partial\varphi_m(t) =
\begin{cases}
\partial\phi(t/m), & m>0,\\
\{\phi'_{\infty}\}, & m=0,\ \phi'_{\infty}<+\infty,\ t>0,\\
]-\infty,\phi'_0], & m=0,\ \phi'_{\infty}<+\infty,\ t=0,\\
\mathbb{R}, & m=0,\ \phi'_{\infty}=+\infty,\ t=0,\\
\varnothing, & \text{otherwise}.
\end{cases} \tag*{(8)} ∂ φ m ( t ) = ⎩ ⎨ ⎧ ∂ ϕ ( t / m ) , { ϕ ∞ ′ } , ] − ∞ , ϕ 0 ′ ] , R , ∅ , m > 0 , m = 0 , ϕ ∞ ′ < + ∞ , t > 0 , m = 0 , ϕ ∞ ′ < + ∞ , t = 0 , m = 0 , ϕ ∞ ′ = + ∞ , t = 0 , otherwise . (8)
Under these assumptions, the following increment bound holds
φ m ( x + δ ) − φ m ( x ) ≤ ϕ ∞ ′ δ ( ϕ ∞ ′ < + ∞ , x ∈ dom ( φ m ) , δ ≥ 0 ) . (9) \varphi_m(x+\delta)-\varphi_m(x) \le\phi'_{\infty}\delta\qquad(\phi'_{\infty}<+\infty,\ x \in\operatorname{dom}(\varphi_m),\ \delta\ge0). \tag*{(9)} φ m ( x + δ ) − φ m ( x ) ≤ ϕ ∞ ′ δ ( ϕ ∞ ′ < + ∞ , x ∈ dom ( φ m ) , δ ≥ 0 ) . (9)
For δ = 0 \delta=0 δ = 0 the bound is trivial. If m > 0 m>0 m > 0 and δ > 0 \delta>0 δ > 0 , it follows from the convexity of ϕ \phi ϕ
φ m ( x + δ ) − φ m ( x ) δ = ϕ ( ( x + δ ) / m ) − ϕ ( x / m ) δ / m ≤ ϕ − ′ ( ( x + δ ) / m ) ≤ ϕ ∞ ′ . (10) \frac{\varphi_m(x+\delta)-\varphi_m(x)}{\delta}
=
\frac{\phi((x+\delta)/m)-\phi(x/m)}{\delta/m}
\le\phi'_-\bigl((x+\delta)/m\bigr)
\le\phi'_{\infty}. \tag*{(10)} δ φ m ( x + δ ) − φ m ( x ) = δ / m ϕ (( x + δ ) / m ) − ϕ ( x / m ) ≤ ϕ − ′ ( ( x + δ ) / m ) ≤ ϕ ∞ ′ . (10)
while for m = 0 m = 0 m = 0 the bound holds as an equality since φ 0 \varphi_0 φ 0 is a linear function if φ ∞ ′ < ∞ \varphi_\infty' < \infty φ ∞ ′ < ∞ .
Given α , μ ∈ M + ( X ) \alpha,\mu\in\mathcal{M}_+(\mathcal{X}) α , μ ∈ M + ( X ) two positive measures, let α = ρ μ + α ⊥ \alpha= \rho\mu+ \alpha^\perp α = ρ μ + α ⊥ be the unique Lebesgue decomposition of α \alpha α with respect to μ \mu μ . The Csiszár divergence induced by φ \varphi φ is
D φ ( α ∣ μ ) = ∫ X φ ( ρ ) d μ + φ ∞ ′ α ⊥ ( X ) , (11) \mathrm{D}_{\varphi}(\alpha\mid\mu) = \int_{\mathcal{X}} \varphi(\rho)\,\mathrm{d}\mu+ \varphi_\infty' \alpha^\perp(\mathcal{X}), \tag*{(11)} D φ ( α ∣ μ ) = ∫ X φ ( ρ ) d μ + φ ∞ ′ α ⊥ ( X ) , (11)
where we adopted the convention + ∞ ⋅ 0 = 0 +\infty\cdot0 = 0 + ∞ ⋅ 0 = 0 . Then, given μ ∈ M + ( X ) \mu\in\mathcal{M}_+(\mathcal{X}) μ ∈ M + ( X ) and ν ∈ M + ( Y ) \nu\in\mathcal{M}_+(\mathcal{Y}) ν ∈ M + ( Y ) , the Unbalanced Optimal Transport (UOT) problem is defined as
v c : = inf π ∈ M + ( X × Y ) U T φ , c ( π ) , U T φ , c ( π ) : = ∫ X × Y c d π + D φ ( π ( 1 ) ∣ μ ) + D φ ( π ( 2 ) ∣ ν ) . (12) v_c := \inf_{\pi\in\mathcal{M}_+(\mathcal{X}\times\mathcal{Y})} \mathrm{UT}_{\varphi,c}(\pi), \qquad\mathrm{UT}_{\varphi,c}(\pi) := \int_{\mathcal{X}\times\mathcal{Y}} c\,\mathrm{d}\pi+ \mathrm{D}_{\varphi}(\pi^{(1)} \mid\mu) + \mathrm{D}_{\varphi}(\pi^{(2)} \mid\nu). \tag*{(12)} v c := π ∈ M + ( X × Y ) inf UT φ , c ( π ) , UT φ , c ( π ) := ∫ X × Y c d π + D φ ( π ( 1 ) ∣ μ ) + D φ ( π ( 2 ) ∣ ν ) . (12)
We recall that, under the stated assumptions, the Unbalanced Optimal Transport problem always admits at least a solution [36 ] .
Proposition 1 (Existence). Under the nondegenerate conditions in Remark 1, v c < + ∞ v_c < +\infty v c < + ∞ and the infimum in (12) is attained.
The UOT Problem between Discrete Positive Measures
We first show that the UOT problem induced by any couple of positive measures with finite support admits a minimizer with finite support. This allows to cast the UOT problem to a finite-dimensional convex problem and to express the optimality conditions via potential vectors. We then exploit the optimality conditions to derive bounds on the support, the individual entries, and the marginals of optimal solutions to the UOT problem.
The Support of the Optimal Solution
In what follows, we set A = supp ( μ ) \mathcal{A} = \operatorname{supp}(\mu) A = supp ( μ ) and B = supp ( ν ) \mathcal{B} = \operatorname{supp}(\nu) B = supp ( ν ) . If φ ∞ ′ = + ∞ \varphi_\infty' = +\infty φ ∞ ′ = + ∞ , finiteness of the divergences forces every plan attaining bounded objective value to be supported on A × B \mathcal{A} \times\mathcal{B} A × B [36 ] . In the next result, we show that something similar happens when φ ∞ ′ < + ∞ \varphi_\infty' < +\infty φ ∞ ′ < + ∞ . Indeed, let us set
y ^ ( x ) ∈ arg min y ∈ Y c ( x , y ) ( x ∈ A ) , x ^ ( y ) ∈ arg min x ∈ X c ( x , y ) ( y ∈ B ) , (13) \hat{y}(x) \in\underset{y \in\mathcal{Y}}{\arg\min}\, c(x,y) \quad(x \in\mathcal{A}), \qquad\hat{x}(y) \in\underset{x \in\mathcal{X}}{\arg\min}\, c(x,y) \quad(y \in\mathcal{B}), \tag*{(13)} y ^ ( x ) ∈ y ∈ Y arg min c ( x , y ) ( x ∈ A ) , x ^ ( y ) ∈ x ∈ X arg min c ( x , y ) ( y ∈ B ) , (13)
and define
K X = A ∪ { x ^ ( y ) : y ∈ B } , K Y = B ∪ { y ^ ( x ) : x ∈ A } . (14) \mathcal{K}_{\mathcal{X}} = \mathcal{A} \cup\{\hat{x}(y) : y \in\mathcal{B}\}, \qquad\mathcal{K}_{\mathcal{Y}} = \mathcal{B} \cup\{\hat{y}(x) : x \in\mathcal{A}\}. \tag*{(14)} K X = A ∪ { x ^ ( y ) : y ∈ B } , K Y = B ∪ { y ^ ( x ) : x ∈ A } . (14)
The selected points exist by compactness and continuity, albeit they might not be unique, in which case we pick an arbitrary minimizer.
Theorem 1. Let c : X × Y → [ 0 , + ∞ [ c:\mathcal{X}\times\mathcal{Y}\to[0,+\infty[ c : X × Y → [ 0 , + ∞ [ be a continuous cost function and ϕ ∈ Γ 0 ( R ) \phi\in\Gamma_0(\mathbb{R}) ϕ ∈ Γ 0 ( R ) be such that 0 < ϕ 0 ′ ′ < + ∞ 0<\phi''_0<+\infty 0 < ϕ 0 ′′ < + ∞ . Then, for any couple of finite positive measures μ ∈ M + ( X ) \mu\in\mathcal{M}_+(\mathcal{X}) μ ∈ M + ( X ) and ν ∈ M + ( Y ) \nu\in\mathcal{M}_+(\mathcal{Y}) ν ∈ M + ( Y ) , every solution to the UOT, namely π \pi π , satisfies
π ( A c × B c ) = 0 with A = supp ( μ ) and B = supp ( ν ) . (15) \pi(\mathcal{A}^{\mathrm{c}}\times\mathcal{B}^{\mathrm{c}})=0 \qquad\text{with }\mathcal{A}=\operatorname{supp}(\mu)\quad\text{and}\quad\mathcal{B}=\operatorname{supp}(\nu). \tag*{(15)} π ( A c × B c ) = 0 with A = supp ( μ ) and B = supp ( ν ) . (15) Moreover, there exists at least an optimal solution to the UOT problem, namely π ′ \pi' π ′ , such that
supp ( π ′ ) ⊂ K X × K Y , (16) \operatorname{supp}(\pi')\subset\mathcal{K}_{\mathcal{X}}\times\mathcal{K}_{\mathcal{Y}}, \tag*{(16)} supp ( π ′ ) ⊂ K X × K Y , (16) where K X \mathcal{K}_{\mathcal{X}} K X and K Y \mathcal{K}_{\mathcal{Y}} K Y are defined as in (14) .
Proof. First, we show that π ( A c × B c ) = 0 \pi(\mathcal{A}^{\mathrm{c}}\times\mathcal{B}^{\mathrm{c}})=0 π ( A c × B c ) = 0 . Indeed, towards a contradiction, assume that π \pi π is a solution to (12) such that π ( A c × B c ) > 0 \pi(\mathcal{A}^{\mathrm{c}}\times\mathcal{B}^{\mathrm{c}})>0 π ( A c × B c ) > 0 . We then define π ′ = π ∣ A c × B c \pi'=\pi|_{\mathcal{A}^{\mathrm{c}}\times\mathcal{B}^{\mathrm{c}}} π ′ = π ∣ A c × B c . Notice that π ′ \pi' π ′ is a positive measure, thus feasible for (12) . We now show that U T ϕ , c ( π ′ ) < U T ϕ , c ( π ) \mathrm{UT}_{\phi,c}(\pi')<\mathrm{UT}_{\phi,c}(\pi) UT ϕ , c ( π ′ ) < UT ϕ , c ( π ) . First, since π ∣ A c × B c \pi|_{\mathcal{A}^{\mathrm{c}}\times\mathcal{B}^{\mathrm{c}}} π ∣ A c × B c is a positive measure and c c c is non-negative
∫ X × Y c ( x , y ) π ′ ( d x , d y ) ≤ ∫ X × Y c ( x , y ) π ( d x , d y ) . \int_{\mathcal{X}\times\mathcal{Y}}c(x,y)\pi'(dx,dy)\leq\int_{\mathcal{X}\times\mathcal{Y}}c(x,y)\pi(dx,dy). ∫ X × Y c ( x , y ) π ′ ( d x , d y ) ≤ ∫ X × Y c ( x , y ) π ( d x , d y ) .
Moreover, the first and second marginals of π ˉ : = π ∣ A c × B c \bar{\pi}:=\pi|_{\mathcal{A}^{\mathrm{c}}\times\mathcal{B}^{\mathrm{c}}} π ˉ := π ∣ A c × B c are, by construction, singular with respect to μ \mu μ and ν \nu ν , respectively. Thus
π ( 1 ) : = ρ ( μ ) μ + α ⊥ ⇒ ( π ′ ) ( 1 ) = π ( 1 ) − π ˉ ( 1 ) = ρ ( μ ) μ + α ⊥ − π ˉ ( 1 ) \pi^{(1)}:=\rho^{(\mu)}\mu+\alpha_{\perp}\quad\Rightarrow\quad(\pi')^{(1)}=\pi^{(1)}-\bar{\pi}^{(1)}=\rho^{(\mu)}\mu+\alpha_{\perp}-\bar{\pi}^{(1)} π ( 1 ) := ρ ( μ ) μ + α ⊥ ⇒ ( π ′ ) ( 1 ) = π ( 1 ) − π ˉ ( 1 ) = ρ ( μ ) μ + α ⊥ − π ˉ ( 1 )
π ( 2 ) : = ρ ( ν ) ν + β ⊥ ⇒ ( π ′ ) ( 2 ) = π ( 2 ) − π ˉ ( 2 ) = ρ ( ν ) ν + β ⊥ − π ˉ ( 2 ) , \pi^{(2)}:=\rho^{(\nu)}\nu+\beta_{\perp}\quad\Rightarrow\quad(\pi')^{(2)}=\pi^{(2)}-\bar{\pi}^{(2)}=\rho^{(\nu)}\nu+\beta_{\perp}-\bar{\pi}^{(2)}, π ( 2 ) := ρ ( ν ) ν + β ⊥ ⇒ ( π ′ ) ( 2 ) = π ( 2 ) − π ˉ ( 2 ) = ρ ( ν ) ν + β ⊥ − π ˉ ( 2 ) ,
and, by the uniqueness of Lebesgue’s decomposition for measures, we infer α ˉ ⊥ : = α ⊥ − π ˉ ( 1 ) \bar{\alpha}_{\perp}:=\alpha_{\perp}-\bar{\pi}^{(1)} α ˉ ⊥ := α ⊥ − π ˉ ( 1 ) and β ˉ ⊥ : = β ⊥ − π ˉ ( 2 ) \bar{\beta}_{\perp}:=\beta_{\perp}-\bar{\pi}^{(2)} β ˉ ⊥ := β ⊥ − π ˉ ( 2 ) are the singular part of the first and second marginals of π ′ \pi' π ′ with respect to μ \mu μ and ν \nu ν , respectively. Therefore, the definition of D ϕ D_{\phi} D ϕ yields
D ϕ ( ( π ′ ) ( 1 ) ∣ μ ) + D ϕ ( ( π ′ ) ( 2 ) ∣ ν ) = D ϕ ( π ( 1 ) ∣ μ ) + D ϕ ( π ( 2 ) ∣ ν ) − 2 λ ϕ ∞ ′ , (17) D_{\phi}((\pi')^{(1)}\mid\mu)+D_{\phi}((\pi')^{(2)}\mid\nu)=D_{\phi}(\pi^{(1)}\mid\mu)+D_{\phi}(\pi^{(2)}\mid\nu)-2\lambda\phi'_{\infty}, \tag*{(17)} D ϕ (( π ′ ) ( 1 ) ∣ μ ) + D ϕ (( π ′ ) ( 2 ) ∣ ν ) = D ϕ ( π ( 1 ) ∣ μ ) + D ϕ ( π ( 2 ) ∣ ν ) − 2 λ ϕ ∞ ′ , (17)
and hence U T ϕ , c ( π ′ ) < U T ϕ , c ( π ) \mathrm{UT}_{\phi,c}(\pi')<\mathrm{UT}_{\phi,c}(\pi) UT ϕ , c ( π ′ ) < UT ϕ , c ( π ) , which is a contradiction allowing us to conclude (15) .
We now move to the second part of the statement. Without loss of generality, we assume that π ( A c × B c ) = 0 \pi(\mathcal{A}^{\mathrm{c}}\times\mathcal{B}^{\mathrm{c}})=0 π ( A c × B c ) = 0 , which implies
supp ( π ) ⊆ A c × B c . (18) \operatorname{supp}(\pi)\subseteq\mathcal{A}^{\mathrm{c}}\times\mathcal{B}^{\mathrm{c}}. \tag*{(18)} supp ( π ) ⊆ A c × B c . (18)
Given π \pi π a solution to (12) , owing (18) we have
supp ( π ) \ ( K X × K Y ) ⊆ ( S 0 × K Y c ) ∪ ( K X c × T 0 ) , with S 0 = A , T 0 = B . (19) \operatorname{supp}(\pi)\backslash(\mathcal{K}_{\mathcal{X}}\times\mathcal{K}_{\mathcal{Y}})\subseteq(\mathcal{S}_0\times\mathcal{K}_{\mathcal{Y}}^{\mathrm{c}})\cup(\mathcal{K}_{\mathcal{X}}^{\mathrm{c}}\times\mathcal{T}_0),\quad\text{with }\mathcal{S}_0=\mathcal{A},\ \mathcal{T}_0=\mathcal{B}. \tag*{(19)} supp ( π ) \ ( K X × K Y ) ⊆ ( S 0 × K Y c ) ∪ ( K X c × T 0 ) , with S 0 = A , T 0 = B . (19)
We note that the two sets on the right hand side of (19) are disjoints, since K X c \mathcal{K}_{\mathcal{X}}^{\mathrm{c}} K X c is disjoint from A \mathcal{A} A by definition and, similarly, K Y c \mathcal{K}_{\mathcal{Y}}^{\mathrm{c}} K Y c is disjoint from B \mathcal{B} B .
If the left hand side of (19) is empty, we have nothing to prove since supp ( π ) \ ( K X × K Y ) = ∅ \operatorname{supp}(\pi)\backslash(\mathcal{K}_{\mathcal{X}}\times\mathcal{K}_{\mathcal{Y}})=\varnothing supp ( π ) \ ( K X × K Y ) = ∅ implies supp ( π ) ⊆ K X × K Y \operatorname{supp}(\pi)\subseteq\mathcal{K}_{\mathcal{X}}\times\mathcal{K}_{\mathcal{Y}} supp ( π ) ⊆ K X × K Y . Suppose therefore that there exists an element ( x ˉ , y ˉ ) ∈ supp ( π ) \ ( K X × K Y ) (\bar{x},\bar{y})\in\operatorname{supp}(\pi)\backslash(\mathcal{K}_{\mathcal{X}}\times\mathcal{K}_{\mathcal{Y}}) ( x ˉ , y ˉ ) ∈ supp ( π ) \ ( K X × K Y ) and, without loss of generality, that ( x ˉ , y ˉ ) ∈ S 0 × K Y c (\bar{x},\bar{y})\in\mathcal{S}_0\times\mathcal{K}_{\mathcal{Y}}^{\mathrm{c}} ( x ˉ , y ˉ ) ∈ S 0 × K Y c . We will construct a transport plan π ~ \tilde{\pi} π ~ which is still optimal for (12) and satisfies
π ~ ∣ { x ˉ } × Y c = π ∣ { x ˉ } × Y c and supp ( π ~ ∣ { x ˉ } × Y ) ⊆ K X × K Y , (20) \tilde{\pi}|_{\{ \bar{x}\}\times\mathcal{Y}}^{\mathrm{c}}=\pi|_{\{\bar{x}\}\times\mathcal{Y}}^{\mathrm{c}}\quad\text{and}\quad\operatorname{supp}(\tilde{\pi}|_{\{\bar{x}\}\times\mathcal{Y}})\subseteq\mathcal{K}_{\mathcal{X}}\times\mathcal{K}_{\mathcal{Y}}, \tag*{(20)} π ~ ∣ { x ˉ } × Y c = π ∣ { x ˉ } × Y c and supp ( π ~ ∣ { x ˉ } × Y ) ⊆ K X × K Y , (20)
so that
supp ( π ~ ) \ ( K X × K Y ) ⊆ ( S 1 × K Y c ) ∪ ( K X c × T 0 ) , with S 1 = S 0 \ { x ˉ } . (21) \operatorname{supp}(\tilde{\pi})\backslash(\mathcal{K}_{\mathcal{X}}\times\mathcal{K}_{\mathcal{Y}})\subseteq(\mathcal{S}_1\times\mathcal{K}_{\mathcal{Y}}^{\mathrm{c}})\cup(\mathcal{K}_{\mathcal{X}}^{\mathrm{c}}\times\mathcal{T}_0),\quad\text{with }\mathcal{S}_1=\mathcal{S}_0\backslash\{\bar{x}\}. \tag*{(21)} supp ( π ~ ) \ ( K X × K Y ) ⊆ ( S 1 × K Y c ) ∪ ( K X c × T 0 ) , with S 1 = S 0 \ { x ˉ } . (21)
Similarly, if ( x ˉ , y ˉ ) ∈ K X c × T 0 (\bar{x},\bar{y}) \in\mathcal{K}_{\mathcal{X}}^{c} \times\mathcal{T}_{0} ( x ˉ , y ˉ ) ∈ K X c × T 0 , then there exists an optimal transport plan π ˉ \bar{\pi} π ˉ such that supp ( π ˉ ) \ ( K X × K Y ) ⊆ ( S 0 × K Y c ) ∪ ( K X c × T 1 ) \operatorname{supp}(\bar{\pi})\backslash(\mathcal{K}_{\mathcal{X}} \times\mathcal{K}_{\mathcal{Y}}) \subseteq(\mathcal{S}_{0} \times\mathcal{K}_{\mathcal{Y}}^{c}) \cup(\mathcal{K}_{\mathcal{X}}^{c} \times\mathcal{T}_{1}) supp ( π ˉ ) \ ( K X × K Y ) ⊆ ( S 0 × K Y c ) ∪ ( K X c × T 1 ) with T 1 = T 0 \ { y ˉ } \mathcal{T}_{1} = \mathcal{T}_{0}\backslash\{\bar{y}\} T 1 = T 0 \ { y ˉ } .
If starting from a solution whose support satisfies (3.7) we are able to build a solution that satisfies (3.9), we can iteratively apply this construction. Since at each iteration we reduce the cardinality of either S 0 \mathcal{S}_{0} S 0 or T 0 \mathcal{T}_{0} T 0 by one, within a finite number of iteration, we obtain a solution to (2.11), namely π ˉ \bar{\pi} π ˉ , such that supp ( π ˉ ) ⊆ K X × K Y \operatorname{supp}(\bar{\pi}) \subseteq\mathcal{K}_{\mathcal{X}} \times\mathcal{K}_{\mathcal{Y}} supp ( π ˉ ) ⊆ K X × K Y , proving the statement.
In order to complete the proof, we have to prove that if π \pi π is optimal, satisfies
supp ( π ) \ ( K X × K Y ) ⊆ ( S × K Y c ) ∪ ( K X c × T ) , with S ⊆ supp ( μ ) , T ⊆ supp ( ν ) , (22) \operatorname{supp}(\pi)\backslash(\mathcal{K}_{\mathcal{X}} \times\mathcal{K}_{\mathcal{Y}}) \subseteq(\mathcal{S} \times\mathcal{K}_{\mathcal{Y}}^{c}) \cup(\mathcal{K}_{\mathcal{X}}^{c} \times\mathcal{T}), \quad\text{with } \mathcal{S} \subseteq\operatorname{supp}(\mu), \quad\mathcal{T} \subseteq\operatorname{supp}(\nu), \tag*{(22)} supp ( π ) \ ( K X × K Y ) ⊆ ( S × K Y c ) ∪ ( K X c × T ) , with S ⊆ supp ( μ ) , T ⊆ supp ( ν ) , (22)
and ( x ˉ , y ˉ ) ∈ S × K Y c (\bar{x},\bar{y}) \in\mathcal{S} \times\mathcal{K}_{\mathcal{Y}}^{c} ( x ˉ , y ˉ ) ∈ S × K Y c , then there exist an optimal π ˉ \bar{\pi} π ˉ that satisfies (3.8). By disintegrating π \pi π with respect to π ( 1 ) \pi^{(1)} π ( 1 ) , we find a measurable family of measures, namely { ρ ( ⋅ ∣ x ) } x ∈ X \{\rho(\cdot|x)\}_{x\in\mathcal{X}} { ρ ( ⋅ ∣ x ) } x ∈ X , such that for every pair of measurable sets X ⊂ X X \subset\mathcal{X} X ⊂ X and Y ⊂ Y Y \subset\mathcal{Y} Y ⊂ Y , we get
π ( X × Y ) = ∫ X ρ ( Y ∣ x ) π ( 1 ) ( d x ) = ∫ X ρ ( Y ∣ x ) f ( x ) μ ( d x ) + ∫ X ρ ( Y ∣ x ) π ⊥ , μ ( 1 ) ( d x ) \pi(X \times Y) = \int_{X} \rho(Y|x)\pi^{(1)}(dx) = \int_{X} \rho(Y|x)f(x)\mu(dx) + \int_{X} \rho(Y|x)\pi_{\perp,\mu}^{(1)}(dx) π ( X × Y ) = ∫ X ρ ( Y ∣ x ) π ( 1 ) ( d x ) = ∫ X ρ ( Y ∣ x ) f ( x ) μ ( d x ) + ∫ X ρ ( Y ∣ x ) π ⊥ , μ ( 1 ) ( d x )
where in the last equality we used the standard Lebesgue’s decomposition of the positive measures π ( 1 ) = f μ + π ⊥ , μ ( 1 ) \pi^{(1)} = f\mu+ \pi_{\perp,\mu}^{(1)} π ( 1 ) = f μ + π ⊥ , μ ( 1 ) . For every Y ⊆ Y Y \subseteq\mathcal{Y} Y ⊆ Y , we define
ρ ~ ( Y ∣ x ) = { ρ ( B c ∣ x ˉ ) δ y ~ ( x ˉ ) ( Y ) + ρ ( B ∩ Y ∣ x ˉ ) if x = x ˉ ; ρ ( Y ∣ x ) otherwise, (23) \widetilde{\rho}(Y|x) =
\begin{cases}
\rho(\mathcal{B}^{c}|\bar{x})\delta_{\widetilde{y}(\bar{x})}(Y) + \rho(\mathcal{B}\cap Y|\bar{x}) & \text{if } x = \bar{x};\\
\rho(Y|x) & \text{otherwise,}
\end{cases} \tag*{(23)} ρ ( Y ∣ x ) = { ρ ( B c ∣ x ˉ ) δ y ( x ˉ ) ( Y ) + ρ ( B ∩ Y ∣ x ˉ ) ρ ( Y ∣ x ) if x = x ˉ ; otherwise, (23)
where y ~ ( x ˉ ) ∈ argmin y ∈ Y c ( x ˉ , y ) \widetilde{y}(\bar{x}) \in\operatorname*{argmin}_{y\in\mathcal{Y}} c(\bar{x},y) y ( x ˉ ) ∈ argmin y ∈ Y c ( x ˉ , y ) and set π ~ = ρ ~ ⊗ π ( 1 ) \widetilde{\pi} = \widetilde{\rho} \otimes\pi^{(1)} π = ρ ⊗ π ( 1 ) .
First, we show that π ~ ( 1 ) = π ( 1 ) \widetilde{\pi}^{(1)} = \pi^{(1)} π ( 1 ) = π ( 1 ) . Indeed, given a measurable set X ⊆ X X \subseteq\mathcal{X} X ⊆ X , we have
π ~ ( 1 ) ( X ) = π ~ ( X × Y ) = ∫ X ρ ~ ( Y ∣ x ) π ( 1 ) ( d x ) = π ( 1 ) ( X ) , \widetilde{\pi}^{(1)}(X) = \widetilde{\pi}(X \times\mathcal{Y}) = \int_{X} \widetilde{\rho}(\mathcal{Y}|x)\pi^{(1)}(dx) = \pi^{(1)}(X), π ( 1 ) ( X ) = π ( X × Y ) = ∫ X ρ ( Y ∣ x ) π ( 1 ) ( d x ) = π ( 1 ) ( X ) ,
since ρ ~ ( Y ∣ x ) = ρ ( Y ∣ x ) = 1 \widetilde{\rho}(\mathcal{Y}|x) = \rho(\mathcal{Y}|x) = 1 ρ ( Y ∣ x ) = ρ ( Y ∣ x ) = 1 for every x ∈ X x \in\mathcal{X} x ∈ X . Notice that π ~ ( 1 ) = π ( 1 ) \widetilde{\pi}^{(1)} = \pi^{(1)} π ( 1 ) = π ( 1 ) implies D ϕ ( π ~ ( 1 ) ∣ μ ) = D ϕ ( π ( 1 ) ∣ μ ) D_{\phi}(\widetilde{\pi}^{(1)}|\mu) = D_{\phi}(\pi^{(1)}|\mu) D ϕ ( π ( 1 ) ∣ μ ) = D ϕ ( π ( 1 ) ∣ μ ) and π ~ ( X × Y ) = π ( X × Y ) \widetilde{\pi}(\mathcal{X} \times\mathcal{Y}) = \pi(\mathcal{X} \times\mathcal{Y}) π ( X × Y ) = π ( X × Y ) .
By construction, π ~ \widetilde{\pi} π satisfies (3.8). Indeed, given C ⊆ { x ˉ } c × Y C \subseteq\{\bar{x}\}^{c} \times\mathcal{Y} C ⊆ { x ˉ } c × Y , let χ C : X × Y → R \chi_{C}:\mathcal{X} \times\mathcal{Y} \to\mathbb{R} χ C : X × Y → R denote the function that takes value 1 when ( x , y ) ∈ C (x,y) \in C ( x , y ) ∈ C and 0 otherwise, then
π ~ ( C ) = ∫ X × Y χ C ( x , y ) π ~ ( d x , d y ) = ∫ X \ { x ˉ } ∫ Y χ C ( x , y ) ρ ~ ( d y ∣ x ) π ~ ( 1 ) ( d x ) = ∫ X \ { x ˉ } ∫ Y χ C ( x , y ) ρ ( d y ∣ x ) π ( 1 ) ( d x ) = ∫ X ∫ Y χ C ( x , y ) ρ ( d y ∣ x ) π ( 1 ) ( d x ) = π ( C ) , \begin{aligned}
\widetilde{\pi}(C) = \int_{\mathcal{X}\times\mathcal{Y}} \chi_{C}(x,y)\widetilde{\pi}(dx,dy) = \int_{\mathcal{X}\backslash\{\bar{x}\}} \int_{\mathcal{Y}} \chi_{C}(x,y)\widetilde{\rho}(dy|x)\widetilde{\pi}^{(1)}(dx) \\
&= \int_{\mathcal{X}\backslash\{\bar{x}\}} \int_{\mathcal{Y}} \chi_{C}(x,y)\rho(dy|x)\pi^{(1)}(dx) = \int_{\mathcal{X}} \int_{\mathcal{Y}} \chi_{C}(x,y)\rho(dy|x)\pi^{(1)}(dx) = \pi(C),
\end{aligned} π ( C ) = ∫ X × Y χ C ( x , y ) π ( d x , d y ) = ∫ X \ { x ˉ } ∫ Y χ C ( x , y ) ρ ( d y ∣ x ) π ( 1 ) ( d x ) = ∫ X \ { x ˉ } ∫ Y χ C ( x , y ) ρ ( d y ∣ x ) π ( 1 ) ( d x ) = ∫ X ∫ Y χ C ( x , y ) ρ ( d y ∣ x ) π ( 1 ) ( d x ) = π ( C ) ,
since C ⊆ { x ˉ } c × Y C \subseteq\{\bar{x}\}^{c} \times\mathcal{Y} C ⊆ { x ˉ } c × Y and ρ ~ ( ⋅ ∣ x ) = ρ ( ⋅ ∣ x ) \widetilde{\rho}(\cdot|x) = \rho(\cdot|x) ρ ( ⋅ ∣ x ) = ρ ( ⋅ ∣ x ) for every x ≠ x ˉ x \ne\bar{x} x = x ˉ . It then follows that π ~ ∣ { x ˉ } × Y c = π ∣ { x ˉ } × Y c \widetilde{\pi}|_{\{\bar{x}\}\times\mathcal{Y}}^{c} = \pi|_{\{\bar{x}\}\times\mathcal{Y}}^{c} π ∣ { x ˉ } × Y c = π ∣ { x ˉ } × Y c . Moreover, supp ( π ~ ∣ { x ˉ } × Y ) ⊆ K X × K Y \operatorname{supp}(\widetilde{\pi}|_{\{\bar{x}\}\times\mathcal{Y}}) \subseteq\mathcal{K}_{\mathcal{X}} \times\mathcal{K}_{\mathcal{Y}} supp ( π ∣ { x ˉ } × Y ) ⊆ K X × K Y since K X × K Y \mathcal{K}_{\mathcal{X}} \times\mathcal{K}_{\mathcal{Y}} K X × K Y is closed and
π ~ ∣ { x ˉ } × Y ( ( K X × K Y ) c ) ≤ π ~ ( ( { x ˉ } × Y ) ∩ ( K X c × Y ) ) + π ~ ( ( { x ˉ } × Y ) ∩ ( X × K Y c ) ) = π ~ ( { x ˉ } × K Y c ) = ∫ { x ˉ } ρ ~ ( K Y c ∣ x ) π ( 1 ) ( d x ) = 0 , \begin{aligned}
\widetilde{\pi}|_{\{\bar{x}\}\times\mathcal{Y}}\left((\mathcal{K}_{\mathcal{X}} \times\mathcal{K}_{\mathcal{Y}})^{c}\right) \leq\widetilde{\pi}\left((\{\bar{x}\} \times\mathcal{Y}) \cap(\mathcal{K}_{\mathcal{X}}^{c} \times\mathcal{Y})\right) + \widetilde{\pi}\left((\{\bar{x}\} \times\mathcal{Y}) \cap(\mathcal{X} \times\mathcal{K}_{\mathcal{Y}}^{c})\right) \\
&= \widetilde{\pi}(\{\bar{x}\} \times\mathcal{K}_{\mathcal{Y}}^{c}) = \int_{\{\bar{x}\}} \widetilde{\rho}(\mathcal{K}_{\mathcal{Y}}^{c}|x)\pi^{(1)}(dx) = 0,
\end{aligned} π ∣ { x ˉ } × Y ( ( K X × K Y ) c ) ≤ π ( ({ x ˉ } × Y ) ∩ ( K X c × Y ) ) + π ( ({ x ˉ } × Y ) ∩ ( X × K Y c ) ) = π ({ x ˉ } × K Y c ) = ∫ { x ˉ } ρ ( K Y c ∣ x ) π ( 1 ) ( d x ) = 0 ,
where in the second last equality we used the fact that K Y c \mathcal{K}_{\mathcal{Y}}^{c} K Y c is disjoint from B ∪ { y ~ ( x ) : x ∈ A } \mathcal{B} \cup\{\widetilde{y}(x):x\in\mathcal{A}\} B ∪ { y ( x ) : x ∈ A } and in the last equality we used the definition of ρ ~ \widetilde{\rho} ρ .
We now consider π ~ ( 2 ) \widetilde{\pi}^{(2)} π ( 2 ) . Given a measurable Y ⊆ Y Y \subseteq\mathcal{Y} Y ⊆ Y , we have
π ~ ( 2 ) ( Y ) = ∫ X ρ ~ ( Y ∣ x ) π ( 1 ) ( d x ) = ∫ X ρ ~ ( Y ∣ x ) f ( x ) μ ( d x ) + ∫ X ρ ~ ( Y ∣ x ) π ⊥ , μ ( 1 ) ( d x ) = f x ˉ μ x ˉ ( ρ ( B c ∣ x ˉ ) δ y ~ ( x ˉ ) ( Y ) + ρ ( Y ∩ B ∣ x ˉ ) ) + ∑ x ≠ x ˉ , x ∈ A ρ ( Y ∣ x ) f x μ x + ∫ X ρ ( Y ∣ x ) π ⊥ , μ ( 1 ) ( d x ) , (24) \begin{aligned}
\widetilde{\pi}^{(2)}(Y) = \int_{\mathcal{X}} \widetilde{\rho}(Y|x)\pi^{(1)}(dx) = \int_{\mathcal{X}} \widetilde{\rho}(Y|x)f(x)\mu(dx) + \int_{\mathcal{X}} \widetilde{\rho}(Y|x)\pi_{\perp,\mu}^{(1)}(dx) \\
&= f_{\bar{x}}\mu_{\bar{x}}\left(\rho(\mathcal{B}^{c}|\bar{x})\delta_{\widetilde{y}(\bar{x})}(Y) + \rho(Y\cap\mathcal{B}|\bar{x})\right)
+ \sum_{\substack{x\ne\bar{x},\ x\in\mathcal{A}}} \rho(Y|x)f_x\mu_x
+ \int_{\mathcal{X}} \rho(Y|x)\pi_{\perp,\mu}^{(1)}(dx), \tag*{(24)}
\end{aligned} π ( 2 ) ( Y ) = ∫ X ρ ( Y ∣ x ) π ( 1 ) ( d x ) = ∫ X ρ ( Y ∣ x ) f ( x ) μ ( d x ) + ∫ X ρ ( Y ∣ x ) π ⊥ , μ ( 1 ) ( d x ) = f x ˉ μ x ˉ ( ρ ( B c ∣ x ˉ ) δ y ( x ˉ ) ( Y ) + ρ ( Y ∩ B ∣ x ˉ ) ) + x = x ˉ , x ∈ A ∑ ρ ( Y ∣ x ) f x μ x + ∫ X ρ ( Y ∣ x ) π ⊥ , μ ( 1 ) ( d x ) , (24)
where we have used the definition of ρ ~ \tilde{\rho} ρ ~ . Since ν \nu ν is discrete
π ~ ( 2 ) = π ~ ν ( 2 ) + π ~ ⊥ , ν ( 2 ) , with π ~ ν ( 2 ) = ∑ y ∈ B π ~ ( 2 ) ( { y } ) ν y ν y d y , ν y = ν ( { y } ) , (25) \tilde{\pi}^{(2)}=\tilde{\pi}_{\nu}^{(2)}+\tilde{\pi}_{\perp,\nu}^{(2)},\qquad\text{with}\quad\tilde{\pi}_{\nu}^{(2)}=\sum_{y\in\mathcal{B}}\frac{\tilde{\pi}^{(2)}(\{y\})}{\nu_y}\nu_y\,\mathrm{d}y,\quad\nu_y=\nu(\{y\}), \tag*{(25)} π ~ ( 2 ) = π ~ ν ( 2 ) + π ~ ⊥ , ν ( 2 ) , with π ~ ν ( 2 ) = y ∈ B ∑ ν y π ~ ( 2 ) ({ y }) ν y d y , ν y = ν ({ y }) , (25)
and π ~ ⊥ , ν ( 2 ) ( { y } ) = 0 \tilde{\pi}_{\perp,\nu}^{(2)}(\{y\})=0 π ~ ⊥ , ν ( 2 ) ({ y }) = 0 for every y ∈ B y\in\mathcal{B} y ∈ B .
To conclude, we show U T ϕ , c ( π ~ ) ≤ U T ϕ , c ( π ) \mathrm{UT}_{\phi,c}(\tilde{\pi})\leq\mathrm{UT}_{\phi,c}(\pi) UT ϕ , c ( π ~ ) ≤ UT ϕ , c ( π ) by proving that
∫ X × Y c ( x , y ) π ~ ( d x , d y ) ≤ ∫ X × Y c ( x , y ) π ( d x , d y ) (26) \int_{\mathcal{X}\times\mathcal{Y}}c(x,y)\tilde{\pi}(\mathrm{d}x,\mathrm{d}y)\leq\int_{\mathcal{X}\times\mathcal{Y}}c(x,y)\pi(\mathrm{d}x,\mathrm{d}y) \tag*{(26)} ∫ X × Y c ( x , y ) π ~ ( d x , d y ) ≤ ∫ X × Y c ( x , y ) π ( d x , d y ) (26)
and D ϕ ( π ~ ( 2 ) ∣ ν ) ≤ D ϕ ( π ( 2 ) ∣ ν ) D_{\phi}(\tilde{\pi}^{(2)}\mathbin{|}\nu)\leq D_{\phi}(\pi^{(2)}\mathbin{|}\nu) D ϕ ( π ~ ( 2 ) ∣ ν ) ≤ D ϕ ( π ( 2 ) ∣ ν ) since we have already proven that D ϕ ( π ~ ( 1 ) ∣ μ ) ≤ D ϕ ( π ( 1 ) ∣ μ ) D_{\phi}(\tilde{\pi}^{(1)}\mathbin{|}\mu)\leq D_{\phi}(\pi^{(1)}\mathbin{|}\mu) D ϕ ( π ~ ( 1 ) ∣ μ ) ≤ D ϕ ( π ( 1 ) ∣ μ ) .
Let us start with (3.14). By definition,
∫ X × Y c ( x , y ) π ( d x , d y ) = ∫ X ∖ { x ˉ } ( ∫ Y c ( x , y ) ρ ( d y ∣ x ) ) π ( 1 ) ( d x ) + π ( 1 ) ( { x ˉ } ) ∫ Y c ( x ˉ , y ) ρ ( d y ∣ x ˉ ) . \int_{\mathcal{X}\times\mathcal{Y}}c(x,y)\pi(\mathrm{d}x,\mathrm{d}y)=\int_{\mathcal{X}\setminus\{\bar{x}\}}\left(\int_{\mathcal{Y}}c(x,y)\rho(\mathrm{d}y\mathbin{|}x)\right)\pi^{(1)}(\mathrm{d}x)+\pi^{(1)}(\{\bar{x}\})\int_{\mathcal{Y}}c(\bar{x},y)\rho(\mathrm{d}y\mathbin{|}\bar{x}). ∫ X × Y c ( x , y ) π ( d x , d y ) = ∫ X ∖ { x ˉ } ( ∫ Y c ( x , y ) ρ ( d y ∣ x ) ) π ( 1 ) ( d x ) + π ( 1 ) ({ x ˉ }) ∫ Y c ( x ˉ , y ) ρ ( d y ∣ x ˉ ) .
Notice that
∫ Y c ( x ˉ , y ) ρ ( d y ∣ x ˉ ) ≥ c ( x ˉ , y ^ ( x ˉ ) ) ρ ( B c ∣ x ˉ ) + ∑ y ∈ B c ( x ˉ , y ) ρ ( { y } ∣ x ˉ ) = ∫ Y c ( x ˉ , y ) ρ ~ ( d y ∣ x ˉ ) . \int_{\mathcal{Y}}c(\bar{x},y)\rho(\mathrm{d}y\mathbin{|}\bar{x})\geq c(\bar{x},\hat{y}(\bar{x}))\rho(\mathcal{B}^{c}\mathbin{|}\bar{x})+\sum_{y\in\mathcal{B}}c(\bar{x},y)\rho(\{y\}\mathbin{|}\bar{x})=\int_{\mathcal{Y}}c(\bar{x},y)\tilde{\rho}(\mathrm{d}y\mathbin{|}\bar{x}). ∫ Y c ( x ˉ , y ) ρ ( d y ∣ x ˉ ) ≥ c ( x ˉ , y ^ ( x ˉ )) ρ ( B c ∣ x ˉ ) + y ∈ B ∑ c ( x ˉ , y ) ρ ({ y } ∣ x ˉ ) = ∫ Y c ( x ˉ , y ) ρ ~ ( d y ∣ x ˉ ) .
To conclude, we recall that ρ ~ ( d y ∣ x ) = ρ ( d y ∣ x ) \tilde{\rho}(\mathrm{d}y\mathbin{|}x)=\rho(\mathrm{d}y\mathbin{|}x) ρ ~ ( d y ∣ x ) = ρ ( d y ∣ x ) for every x ≠ x ˉ x\neq\bar{x} x = x ˉ , hence (3.14) follows.
Let us now consider D ϕ ( π ~ ( 2 ) ∣ ν ) ≤ D ϕ ( π ( 2 ) ∣ ν ) D_{\phi}(\tilde{\pi}^{(2)}\mathbin{|}\nu)\leq D_{\phi}(\pi^{(2)}\mathbin{|}\nu) D ϕ ( π ~ ( 2 ) ∣ ν ) ≤ D ϕ ( π ( 2 ) ∣ ν ) . We split our discussion in two parts, depending on whether y ^ ( x ˉ ) ∈ B \hat{y}(\bar{x})\in\mathcal{B} y ^ ( x ˉ ) ∈ B or y ^ ( x ˉ ) ∉ B \hat{y}(\bar{x})\notin\mathcal{B} y ^ ( x ˉ ) ∈ / B .
Let us first consider the case in which y ^ ( x ˉ ) ∉ B \hat{y}(\bar{x})\notin\mathcal{B} y ^ ( x ˉ ) ∈ / B . In this case, for every y ∈ B y\in\mathcal{B} y ∈ B we have
π ~ ( 2 ) ( { y } ) = ∑ x ∈ A ρ ( { y } ∣ x ) f x μ x + ∫ X ρ ( { y } ∣ x ) π ⊥ , μ ( 1 ) ( d x ) = π ( 2 ) ( { y } ) , (27) \tilde{\pi}^{(2)}(\{y\})=\sum_{x\in\mathcal{A}}\rho(\{y\}\mathbin{|}x)f_x\mu_x+\int_{\mathcal{X}}\rho(\{y\}\mathbin{|}x)\pi_{\perp,\mu}^{(1)}(\mathrm{d}x)=\pi^{(2)}(\{y\}), \tag*{(27)} π ~ ( 2 ) ({ y }) = x ∈ A ∑ ρ ({ y } ∣ x ) f x μ x + ∫ X ρ ({ y } ∣ x ) π ⊥ , μ ( 1 ) ( d x ) = π ( 2 ) ({ y }) , (27)
hence π ~ ( 2 ) ( { y } ) = π ( 2 ) ( { y } ) \tilde{\pi}^{(2)}(\{y\})=\pi^{(2)}(\{y\}) π ~ ( 2 ) ({ y }) = π ( 2 ) ({ y }) and thus π ~ ( 2 ) \tilde{\pi}^{(2)} π ~ ( 2 ) and π ( 2 ) \pi^{(2)} π ( 2 ) have the same absolute continuous part with respect to ν \nu ν . Concerning the singular parts of π ~ ( 2 ) \tilde{\pi}^{(2)} π ~ ( 2 ) and π ( 2 ) \pi^{(2)} π ( 2 ) , we note that by (3.12), the following holds
π ~ ( 2 ) ( B c ) = ρ ( B c ∣ x ˉ ) f x ˉ μ x ˉ + ∑ x ≠ x ˉ x ∈ A ρ ( B c ∣ x ) f x μ x + ∫ X ρ ( B c ∣ x ) π ⊥ , μ ( 1 ) ( d x ) = ∑ x ∈ A ρ ( B c ∣ x ) f x μ x + ∫ X ρ ( B c ∣ x ) π ⊥ , μ ( 1 ) ( d x ) = π ( 2 ) ( B c ) . \begin{aligned}
\tilde{\pi}^{(2)}(\mathcal{B}^{c})&=\rho(\mathcal{B}^{c}\mathbin{|}\bar{x})f_{\bar{x}}\mu_{\bar{x}}+\sum_{\substack{x\neq\bar{x}\\x\in\mathcal{A}}}\rho(\mathcal{B}^{c}\mathbin{|}x)f_x\mu_x+\int_{\mathcal{X}}\rho(\mathcal{B}^{c}\mathbin{|}x)\pi_{\perp,\mu}^{(1)}(\mathrm{d}x)\\
&=\sum_{x\in\mathcal{A}}\rho(\mathcal{B}^{c}\mathbin{|}x)f_x\mu_x+\int_{\mathcal{X}}\rho(\mathcal{B}^{c}\mathbin{|}x)\pi_{\perp,\mu}^{(1)}(\mathrm{d}x)=\pi^{(2)}(\mathcal{B}^{c}).
\end{aligned} π ~ ( 2 ) ( B c ) = ρ ( B c ∣ x ˉ ) f x ˉ μ x ˉ + x = x ˉ x ∈ A ∑ ρ ( B c ∣ x ) f x μ x + ∫ X ρ ( B c ∣ x ) π ⊥ , μ ( 1 ) ( d x ) = x ∈ A ∑ ρ ( B c ∣ x ) f x μ x + ∫ X ρ ( B c ∣ x ) π ⊥ , μ ( 1 ) ( d x ) = π ( 2 ) ( B c ) .
Therefore π ~ ⊥ , ν ( 2 ) ( ν ) = π ⊥ , ν ( 2 ) ( ν ) \tilde{\pi}_{\perp,\nu}^{(2)}(\nu)=\pi_{\perp,\nu}^{(2)}(\nu) π ~ ⊥ , ν ( 2 ) ( ν ) = π ⊥ , ν ( 2 ) ( ν ) , and hence, in the end, D ϕ ( π ~ ( 2 ) ∣ ν ) = D ϕ ( π ( 2 ) ∣ ν ) D_{\phi}(\tilde{\pi}^{(2)}\mathbin{|}\nu)=D_{\phi}(\pi^{(2)}\mathbin{|}\nu) D ϕ ( π ~ ( 2 ) ∣ ν ) = D ϕ ( π ( 2 ) ∣ ν ) .
Let us now consider the case in which y ^ ( x ˉ ) ∈ B \hat{y}(\bar{x})\in\mathcal{B} y ^ ( x ˉ ) ∈ B , so that y ^ ( x ˉ ) = y ˉ \hat{y}(\bar{x})=\bar{y} y ^ ( x ˉ ) = y ˉ for some y ˉ ∈ B \bar{y}\in\mathcal{B} y ˉ ∈ B . For every y ≠ y ^ ( x ˉ ) y\neq\hat{y}(\bar{x}) y = y ^ ( x ˉ ) equation (3.15) still holds. If y = y ˉ y=\bar{y} y = y ˉ
π ~ ( 2 ) ( { y ˉ } ) = ( ρ ( B c ∣ x ˉ ) + ρ ( { y ˉ } ∣ x ˉ ) ) f x ˉ μ x ˉ + ∑ x ≠ x ˉ ρ ( { y ˉ } ∣ x ) f x μ x + ∫ X ρ ( { y ˉ } ∣ x ) π ⊥ , μ ( 1 ) ( d x ) = π ( 2 ) ( { y ˉ } ) + ρ ( B c ∣ x ˉ ) f x ˉ μ x ˉ . \begin{aligned}
\tilde{\pi}^{(2)}(\{\bar{y}\})=\left(\rho(\mathcal{B}^{c}\mathbin{|}\bar{x})+\rho(\{\bar{y}\}\mathbin{|}\bar{x})\right)f_{\bar{x}}\mu_{\bar{x}}+\sum_{x\neq\bar{x}}\rho(\{\bar{y}\}\mathbin{|}x)f_x\mu_x+\int_{\mathcal{X}}\rho(\{\bar{y}\}\mathbin{|}x)\pi_{\perp,\mu}^{(1)}(\mathrm{d}x) \\
&=\pi^{(2)}(\{\bar{y}\})+\rho(\mathcal{B}^{c}\mathbin{|}\bar{x})f_{\bar{x}}\mu_{\bar{x}}.
\end{aligned} π ~ ( 2 ) ({ y ˉ }) = ( ρ ( B c ∣ x ˉ ) + ρ ({ y ˉ } ∣ x ˉ ) ) f x ˉ μ x ˉ + x = x ˉ ∑ ρ ({ y ˉ } ∣ x ) f x μ x + ∫ X ρ ({ y ˉ } ∣ x ) π ⊥ , μ ( 1 ) ( d x ) = π ( 2 ) ({ y ˉ }) + ρ ( B c ∣ x ˉ ) f x ˉ μ x ˉ .
Let us set Δ ˉ = ρ ( B c ∣ x ˉ ) f x ˉ μ x ˉ \bar{\Delta}=\rho(\mathcal{B}^{c}\mathbin{|}\bar{x})f_{\bar{x}}\mu_{\bar{x}} Δ ˉ = ρ ( B c ∣ x ˉ ) f x ˉ μ x ˉ . Since π ~ ( X × Y ) = π ( X × Y ) \tilde{\pi}(\mathcal{X}\times\mathcal{Y})=\pi(\mathcal{X}\times\mathcal{Y}) π ~ ( X × Y ) = π ( X × Y ) , we have
π ~ ( 2 ) ( B c ) = π ( 2 ) ( B c ) − Δ ˉ . (28) \tilde{\pi}^{(2)}(\mathcal{B}^{c})=\pi^{(2)}(\mathcal{B}^{c})-\bar{\Delta}. \tag*{(28)} π ~ ( 2 ) ( B c ) = π ( 2 ) ( B c ) − Δ ˉ . (28)
Lastly, if Δ ˉ = 0 \bar{\Delta}=0 Δ ˉ = 0 , we have π ~ ( 2 ) = π ( 2 ) \tilde{\pi}^{(2)}=\pi^{(2)} π ~ ( 2 ) = π ( 2 ) and thus D ϕ ( π ~ ( 2 ) ∣ ν ) = D ϕ ( π ( 2 ) ∣ ν ) D_{\phi}(\tilde{\pi}^{(2)}\mathbin{|}\nu)=D_{\phi}(\pi^{(2)}\mathbin{|}\nu) D ϕ ( π ~ ( 2 ) ∣ ν ) = D ϕ ( π ( 2 ) ∣ ν ) . If Δ ˉ ≠ 0 \bar{\Delta}\neq0 Δ ˉ = 0 , from (2.9) we infer
D ϕ ( π ~ ( 2 ) ∣ ν ) − D ϕ ( π ( 2 ) ∣ ν ) = ν y ˉ ( ϕ ( π y ˉ ( 2 ) + Δ ˉ ν y ˉ ) − ϕ ( π y ˉ ( 2 ) ν y ˉ ) ) − ϕ ∞ ′ Δ ˉ ≤ 0. D_{\phi}(\tilde{\pi}^{(2)}\mathbin{|}\nu)-D_{\phi}(\pi^{(2)}\mathbin{|}\nu)=\nu_{\bar{y}}\left(\phi\left(\frac{\pi_{\bar{y}}^{(2)}+\bar{\Delta}}{\nu_{\bar{y}}}\right)-\phi\left(\frac{\pi_{\bar{y}}^{(2)}}{\nu_{\bar{y}}}\right)\right)-\phi'_{\infty}\bar{\Delta}\leq0. D ϕ ( π ~ ( 2 ) ∣ ν ) − D ϕ ( π ( 2 ) ∣ ν ) = ν y ˉ ( ϕ ( ν y ˉ π y ˉ ( 2 ) + Δ ˉ ) − ϕ ( ν y ˉ π y ˉ ( 2 ) ) ) − ϕ ∞ ′ Δ ˉ ≤ 0.
In both cases, we have D ϕ ( π ~ ( 2 ) ∣ ν ) ≤ D ϕ ( π ( 2 ) ∣ ν ) D_{\phi}(\tilde{\pi}^{(2)}\mathbin{|}\nu)\leq D_{\phi}(\pi^{(2)}\mathbin{|}\nu) D ϕ ( π ~ ( 2 ) ∣ ν ) ≤ D ϕ ( π ( 2 ) ∣ ν ) , which concludes the proof.
Remark 2 (Existence versus all optimizers). If ϕ ∞ ′ < + ∞ \phi'_{\infty} < +\infty ϕ ∞ ′ < + ∞ , Theorem 1 ensures the existence of a discrete optimal solution, not discreteness of every optimal solution. If all the minima in (13) are attained at a unique point, the proof of Theorem 1 shows that every optimal plan is supported on K X × K Y \mathcal{K}_{\mathcal{X}} \times\mathcal{K}_{\mathcal{Y}} K X × K Y . In particular, if X = Y \mathcal{X} = \mathcal{Y} X = Y and c ( x , y ) = 0 c(x,y) = 0 c ( x , y ) = 0 if and only if x = y x = y x = y , then K X = K Y = A ∪ B \mathcal{K}_{\mathcal{X}} = \mathcal{K}_{\mathcal{Y}} = \mathcal{A} \cup\mathcal{B} K X = K Y = A ∪ B and every optimal solution is supported on ( A ∪ B ) × ( A ∪ B ) (\mathcal{A} \cup\mathcal{B}) \times(\mathcal{A} \cup\mathcal{B}) ( A ∪ B ) × ( A ∪ B ) .
Owing to Theorem 1, given two discrete positive measures μ ∈ M + ( X ) \mu\in\mathcal{M}_{+}(\mathcal{X}) μ ∈ M + ( X ) and ν ∈ M + ( Y ) \nu\in\mathcal{M}_{+}(\mathcal{Y}) ν ∈ M + ( Y ) , there exists a finite set of points { x 1 , … , x n } ⊆ X \{x_1,\ldots,x_n\} \subseteq\mathcal{X} { x 1 , … , x n } ⊆ X and { y 1 , … , y m } ⊆ Y \{y_1,\ldots,y_m\} \subseteq\mathcal{Y} { y 1 , … , y m } ⊆ Y such that there exist a solution to (12) for which it holds
supp ( π ) ⊆ { x 1 , … , x n } × { y 1 , … , y m } . \operatorname{supp}(\pi) \subseteq\{x_1,\ldots,x_n\} \times\{y_1,\ldots,y_m\}. supp ( π ) ⊆ { x 1 , … , x n } × { y 1 , … , y m } .
In particular, it is possible to formulate the UOT problem as a minimization problem over the set of n × m n \times m n × m matrices with positive entries, that is
v c = min π ∈ R + n × m F c ( π ) , F c ( π ) = ⟨ c , π ⟩ + ∑ i = 1 n φ μ i ( ( P 1 π ) i ) + ∑ j = 1 m φ ν j ( ( P 2 π ) j ) . (29) v_c = \min_{\pi\in\mathbb{R}_{+}^{n \times m}} F_c(\pi), \qquad F_c(\pi) = \langle c,\pi\rangle+ \sum_{i=1}^{n}\varphi_{\mu_i}\big((P_1\pi)_i\big) + \sum_{j=1}^{m}\varphi_{\nu_j}\big((P_2\pi)_j\big). \tag*{(29)} v c = π ∈ R + n × m min F c ( π ) , F c ( π ) = ⟨ c , π ⟩ + i = 1 ∑ n φ μ i ( ( P 1 π ) i ) + j = 1 ∑ m φ ν j ( ( P 2 π ) j ) . (29)
where P 1 : R n × m → R n P_1:\mathbb{R}^{n \times m} \to\mathbb{R}^{n} P 1 : R n × m → R n and P 2 : R n × m → R m P_2:\mathbb{R}^{n \times m} \to\mathbb{R}^{m} P 2 : R n × m → R m are the linear maps corresponding to the operations of push-forward of the projections onto X \mathcal{X} X and Y \mathcal{Y} Y , respectively, that is
P 1 π = ( ∑ j = 1 m π i j ) 1 ≤ i ≤ n and P 2 π = ( ∑ i = 1 n π i j ) 1 ≤ j ≤ m . P_1\pi= \left(\sum_{j=1}^{m}\pi_{ij}\right)_{1 \leq i \leq n}
\qquad\text{and}\qquad
P_2\pi= \left(\sum_{i=1}^{n}\pi_{ij}\right)_{1 \leq j \leq m}. P 1 π = ( j = 1 ∑ m π ij ) 1 ≤ i ≤ n and P 2 π = ( i = 1 ∑ n π ij ) 1 ≤ j ≤ m .
and φ m \varphi_m φ m is the perspective function defined as in (7) . Even if the problem can be formulated via matrices, we will refer to any π \pi π feasible for (29) as a transportation plan. With a slight abuse of notation, we will refer to π i j \pi_{ij} π ij as the entry of the transportation plan. Notice that in (29) all reference weights { μ i } i ∈ [ n ] \{\mu_i\}_{i\in[n]} { μ i } i ∈ [ n ] and { ν j } j ∈ [ m ] \{\nu_j\}_{j\in[m]} { ν j } j ∈ [ m ] are strictly positive if ϕ ∞ ′ = + ∞ \phi'_{\infty} = +\infty ϕ ∞ ′ = + ∞ , otherwise, if ϕ ∞ ′ < + ∞ \phi'_{\infty} < +\infty ϕ ∞ ′ < + ∞ , they can be equal to zero.
The formulation presented in (29) , allows us to express the optimality conditions as follows.
Theorem 2. A transportation plan π ≥ 0 \pi\geq0 π ≥ 0 is optimal in (29) if and only if there exist a ∈ R n a \in\mathbb{R}^{n} a ∈ R n and b ∈ R m b \in\mathbb{R}^{m} b ∈ R m such that
a i ∈ ∂ φ μ i ( ( P 1 π ) i ) , b j ∈ ∂ φ ν j ( ( P 2 π ) j ) (30) a_i \in\partial\varphi_{\mu_i}\big((P_1\pi)_i\big), \qquad b_j \in\partial\varphi_{\nu_j}\big((P_2\pi)_j\big) \tag*{(30)} a i ∈ ∂ φ μ i ( ( P 1 π ) i ) , b j ∈ ∂ φ ν j ( ( P 2 π ) j ) (30) and, for every i ∈ [ n ] , j ∈ [ m ] i \in[n], j \in[m] i ∈ [ n ] , j ∈ [ m ] , it holds
c i j + a i + b j ≥ 0 , π i j ( c i j + a i + b j ) = 0. (31) c_{ij} + a_i + b_j \geq0, \qquad\pi_{ij}(c_{ij} + a_i + b_j) = 0. \tag*{(31)} c ij + a i + b j ≥ 0 , π ij ( c ij + a i + b j ) = 0. (31)
Proof. Put C = R + n × m C = \mathbb{R}_{+}^{n \times m} C = R + n × m and J ( π ) = ⟨ c , π ⟩ + H ( L π ) + ι C ( π ) J(\pi) = \langle c,\pi\rangle+ H(L\pi) + \iota_C(\pi) J ( π ) = ⟨ c , π ⟩ + H ( L π ) + ι C ( π ) , where
L π = ( P 1 π , P 2 π ) , H ( r , s ) = ∑ i ∈ [ n ] φ μ i ( r i ) + ∑ j ∈ [ m ] φ ν j ( s j ) . L\pi= (P_1\pi, P_2\pi), \qquad H(r,s) = \sum_{i\in[n]}\varphi_{\mu_i}(r_i) + \sum_{j\in[m]}\varphi_{\nu_j}(s_j). L π = ( P 1 π , P 2 π ) , H ( r , s ) = i ∈ [ n ] ∑ φ μ i ( r i ) + j ∈ [ m ] ∑ φ ν j ( s j ) .
For π 0 = 1 n 1 m T \pi^0 = \mathbf{1}_n\mathbf{1}_m^{\mathsf{T}} π 0 = 1 n 1 m T ,
π 0 ∈ int C , L π 0 = ( m 1 n , n 1 m ) ∈ int ( dom H ) . \pi^0 \in\operatorname{int} C, \qquad L\pi^0 = (m\mathbf{1}_n,n\mathbf{1}_m) \in\operatorname{int}(\operatorname{dom} H). π 0 ∈ int C , L π 0 = ( m 1 n , n 1 m ) ∈ int ( dom H ) .
Thus H H H is continuous at L π 0 L\pi^0 L π 0 and H ∘ L H \circ L H ∘ L is continuous at π 0 ∈ C \pi^0 \in C π 0 ∈ C . The convex chain and sum rules give
∂ J ( π ) = c + L ∗ ∂ H ( L π ) + N C ( π ) , \partial J(\pi) = c + L^* \partial H(L\pi) + N_C(\pi), ∂ J ( π ) = c + L ∗ ∂ H ( L π ) + N C ( π ) ,
∂ H ( r , s ) = ∏ i ∂ φ μ i ( r i ) × ∏ j ∂ φ ν j ( s j ) , \partial H(r,s) = \prod_i \partial\varphi_{\mu_i}(r_i) \times\prod_j \partial\varphi_{\nu_j}(s_j), ∂ H ( r , s ) = i ∏ ∂ φ μ i ( r i ) × j ∏ ∂ φ ν j ( s j ) ,
( L ∗ ( a , b ) ) i j = a i + b j , N C ( π ) = { z : z i j ≤ 0 , z i j π i j = 0 for all i , j } . \big(L^*(a,b)\big)_{ij} = a_i + b_j, \qquad N_C(\pi) = \{z : z_{ij} \leq0,\ z_{ij}\pi_{ij} = 0\text{ for all }i,j\}. ( L ∗ ( a , b ) ) ij = a i + b j , N C ( π ) = { z : z ij ≤ 0 , z ij π ij = 0 for all i , j } .
Consequently,
π ∈ argmin J ⟺ 0 ∈ ∂ J ( π ) \pi\in\operatorname*{argmin} J \Longleftrightarrow0 \in\partial J(\pi) π ∈ argmin J ⟺ 0 ∈ ∂ J ( π )
⟺ { there exist a i ∈ ∂ φ μ i ( ( P 1 π ) i ) , b j ∈ ∂ φ ν j ( ( P 2 π ) j ) , − ( c i j + a i + b j ) ≤ 0 , π i j ( c i j + a i + b j ) = 0 for all i , j . \Longleftrightarrow
\begin{cases}
\text{there exist } a_i \in\partial\varphi_{\mu_i}((P_1\pi)_i),\quad b_j \in\partial\varphi_{\nu_j}((P_2\pi)_j),\\
-(c_{ij}+a_i+b_j)\leq0,\quad\pi_{ij}(c_{ij}+a_i+b_j)=0\quad\text{for all }i,j.
\end{cases} ⟺ { there exist a i ∈ ∂ φ μ i (( P 1 π ) i ) , b j ∈ ∂ φ ν j (( P 2 π ) j ) , − ( c ij + a i + b j ) ≤ 0 , π ij ( c ij + a i + b j ) = 0 for all i , j .
In particular, these conditions imply, for every γ ≥ 0 \gamma\geq0 γ ≥ 0 ,
F c ( γ ) − F c ( π ) ≥ ⟨ c , γ − π ⟩ + ⟨ a , P 1 γ − P 1 π ⟩ + ⟨ b , P 2 γ − P 2 π ⟩ = ∑ i , j ( c i j + a i + b j ) ( γ i j − π i j ) = ∑ i , j ( c i j + a i + b j ) γ i j ≥ 0. \begin{aligned}
F_c(\gamma)-F_c(\pi)\geq\langle c,\gamma-\pi\rangle+\langle a,P_1\gamma-P_1\pi\rangle+\langle b,P_2\gamma-P_2\pi\rangle\\
&=\sum_{i,j}(c_{ij}+a_i+b_j)(\gamma_{ij}-\pi_{ij}) \\
&=\sum_{i,j}(c_{ij}+a_i+b_j)\gamma_{ij}\geq0.
\end{aligned} F c ( γ ) − F c ( π ) ≥ ⟨ c , γ − π ⟩ + ⟨ a , P 1 γ − P 1 π ⟩ + ⟨ b , P 2 γ − P 2 π ⟩ = i , j ∑ ( c ij + a i + b j ) ( γ ij − π ij ) = i , j ∑ ( c ij + a i + b j ) γ ij ≥ 0.
Solution Entry-wise Bounds and Elimination of Expensive Arcs
In this section, we provide an upper and lower bound on each entry of a solution to the problem in (3.17). For q ≥ 0 q\geq0 q ≥ 0 , let us define the following auxiliary function
T i j ( q ) : = max argmin t ≥ 0 { q t + φ μ i ( t ) + φ ν j ( t ) } ( i ∈ [ n ] , j ∈ [ m ] ) . (32) T_{ij}(q):=\max\operatorname*{argmin}_{t\geq0}\{qt+\varphi_{\mu_i}(t)+\varphi_{\nu_j}(t)\}\qquad(i\in[n],\ j\in[m]). \tag*{(32)} T ij ( q ) := max t ≥ 0 argmin { q t + φ μ i ( t ) + φ ν j ( t )} ( i ∈ [ n ] , j ∈ [ m ]) . (32)
Under our assumptions, the scalar objective in (3.20) is proper, convex, and lower semicontinuous in t t t . Moreover, it is coercive since it grows at infinity with strictly positive asymptotic slope q + 2 φ ∞ ′ q+2\varphi'_{\infty} q + 2 φ ∞ ′ . The argmin set defining T i j T_{ij} T ij is then non empty and closed, thus T i j ( q ) T_{ij}(q) T ij ( q ) is always finite and well defined. Notice that if μ i = ν j = 0 \mu_i=\nu_j=0 μ i = ν j = 0 , we have T i j ( q ) = 0 T_{ij}(q)=0 T ij ( q ) = 0 for every q ≥ 0 q\geq0 q ≥ 0 .
Theorem 3. The function T i j T_{ij} T ij is nonincreasing. Moreover, every optimizer of (3.17) satisfies
0 ≤ π i j ≤ T i j ( c i j ) ( i ∈ [ n ] , j ∈ [ m ] ) . (33) 0\leq\pi_{ij}\leq T_{ij}(c_{ij})\qquad(i\in[n],\ j\in[m]). \tag*{(33)} 0 ≤ π ij ≤ T ij ( c ij ) ( i ∈ [ n ] , j ∈ [ m ]) . (33)
Proof. First, we show that
T i j ( q ) = sup ( { t > 0 : q + d i + ( t ) + e j + ( t ) ≤ 0 } ∪ { 0 } ) , (34) T_{ij}(q)=\sup\left(\{t>0:q+d_i^+(t)+e_j^+(t)\leq0\}\cup\{0\}\right), \tag*{(34)} T ij ( q ) = sup ( { t > 0 : q + d i + ( t ) + e j + ( t ) ≤ 0 } ∪ { 0 } ) , (34)
where
d i + ( t ) = { φ μ i + ( t / μ i ) , μ i > 0 , φ ∞ + , μ i = 0 , e j + ( t ) = { φ ν j + ( t / ν j ) , ν j > 0 , φ ∞ + , ν j = 0. (35) d_i^+(t)=
\begin{cases}
\varphi_{\mu_i}^+(t/\mu_i),&\mu_i>0,\\
\varphi_{\infty}^+,&\mu_i=0,
\end{cases}
\qquad
e_j^+(t)=
\begin{cases}
\varphi_{\nu_j}^+(t/\nu_j),&\nu_j>0,\\
\varphi_{\infty}^+,&\nu_j=0.
\end{cases} \tag*{(35)} d i + ( t ) = { φ μ i + ( t / μ i ) , φ ∞ + , μ i > 0 , μ i = 0 , e j + ( t ) = { φ ν j + ( t / ν j ) , φ ∞ + , ν j > 0 , ν j = 0. (35)
Indeed, for the sake of simplicity, let us set g q ( t ) = q t + φ μ i ( t ) + φ ν j ( t ) g_q(t)=qt+\varphi_{\mu_i}(t)+\varphi_{\nu_j}(t) g q ( t ) = q t + φ μ i ( t ) + φ ν j ( t ) and
T = T i j ( q ) = max argmin t ≥ 0 g q ( t ) . (36) T=T_{ij}(q)=\max\operatorname*{argmin}_{t\geq0}g_q(t). \tag*{(36)} T = T ij ( q ) = max t ≥ 0 argmin g q ( t ) . (36)
For t > 0 t>0 t > 0 , we have g q , + ′ ( t ) = q + d i + ( t ) + e j + ( t ) g'_{q,+}(t)=q+d_i^+(t)+e_j^+(t) g q , + ′ ( t ) = q + d i + ( t ) + e j + ( t ) . Convexity gives
0 < t < T ⟹ g q , + ′ ( t ) ≤ g q ( T ) − g q ( t ) T − t ≤ 0 , 0<t<T\quad\Longrightarrow\quad g'_{q,+}(t)\leq\frac{g_q(T)-g_q(t)}{T-t}\leq0, 0 < t < T ⟹ g q , + ′ ( t ) ≤ T − t g q ( T ) − g q ( t ) ≤ 0 ,
t > T ⟹ 0 < g q ( t ) − g q ( T ) t − T ≤ g q , − ′ ( t ) ≤ g q , + ′ ( t ) , (37) t>T\quad\Longrightarrow\quad0<\frac{g_q(t)-g_q(T)}{t-T}\leq g'_{q,-}(t)\leq g'_{q,+}(t), \tag*{(37)} t > T ⟹ 0 < t − T g q ( t ) − g q ( T ) ≤ g q , − ′ ( t ) ≤ g q , + ′ ( t ) , (37)
hence, if T > 0 T>0 T > 0 , we have
] 0 , T ] ⊆ { t > 0 : g q , + ′ ( t ) ≤ 0 } ⊆ ] 0 , T ] ]0,T]\subseteq\{t>0:g'_{q,+}(t)\leq0\}\subseteq]0,T] ] 0 , T ] ⊆ { t > 0 : g q , + ′ ( t ) ≤ 0 } ⊆ ] 0 , T ]
and { t > 0 : g q , + ′ ( t ) ≤ 0 } = ∅ \{t > 0 : g'_{q,+}(t) \le0\} = \varnothing { t > 0 : g q , + ′ ( t ) ≤ 0 } = ∅ if T = 0 T = 0 T = 0 , proving (34) . To conclude that T i j T_{ij} T ij is monotone, we notice that, owing to the representation of T i j T_{ij} T ij in (34) , if q 1 ≤ q 2 q_1 \le q_2 q 1 ≤ q 2 then
g q 1 , + ′ ( t ) ≤ g q 2 , + ′ ( t ) ( t ≥ 0 ) ⟹ T i j ( q 2 ) ≤ T i j ( q 1 ) . g'_{q_1,+}(t) \le g'_{q_2,+}(t) \quad(t \ge0) \quad\Longrightarrow\quad T_{ij}(q_2) \le T_{ij}(q_1). g q 1 , + ′ ( t ) ≤ g q 2 , + ′ ( t ) ( t ≥ 0 ) ⟹ T ij ( q 2 ) ≤ T ij ( q 1 ) .
We now prove the last part of the statement. Let π \pi π be optimal. If π i j = 0 \pi_{ij} = 0 π ij = 0 , then (33) trivially holds. Otherwise, let us set s = π i j > 0 s = \pi_{ij} > 0 s = π ij > 0 and define d ˉ i \bar d_i d ˉ i , e ˉ j \bar e_j e ˉ j as in (35) by replacing the right derivatives with left derivatives for every t > 0 t > 0 t > 0 . Then, we have g q , − ′ ( t ) = q + d ˉ i ( t ) + e ˉ j ( t ) g'_{q,-}(t) = q + \bar d_i(t) + \bar e_j(t) g q , − ′ ( t ) = q + d ˉ i ( t ) + e ˉ j ( t ) .
Toward a contradiction, assume that s > T i j ( c i j ) s > T_{ij}(c_{ij}) s > T ij ( c ij ) . By (37) , we have 0 < g c i j , − ′ ( s ) 0 < g'_{c_{ij},-}(s) 0 < g c ij , − ′ ( s ) . From Theorem 2, we find a couple of potentials a i a_i a i and b j b_j b j for which it holds
a i ≥ d ˉ i ( ( P 1 π ) i ) ≥ d ˉ i ( π i j ) = d ˉ i ( s ) , a_i \ge\bar d_i((P_1\pi)_i) \ge\bar d_i(\pi_{ij}) = \bar d_i(s), a i ≥ d ˉ i (( P 1 π ) i ) ≥ d ˉ i ( π ij ) = d ˉ i ( s ) ,
b j ≥ e ˉ j ( ( P 2 π ) j ) ≥ e ˉ j ( π i j ) = e ˉ j ( s ) . b_j \ge\bar e_j((P_2\pi)_j) \ge\bar e_j(\pi_{ij}) = \bar e_j(s). b j ≥ e ˉ j (( P 2 π ) j ) ≥ e ˉ j ( π ij ) = e ˉ j ( s ) .
Since π i j > 0 \pi_{ij} > 0 π ij > 0 , we infer that
0 = c i j + a i + b j ≥ c i j + d ˉ i ( s ) + e ˉ j ( s ) = g c i j , − ′ ( s ) , (38) 0 = c_{ij} + a_i + b_j \ge c_{ij} + \bar d_i(s) + \bar e_j(s) = g'_{c_{ij},-}(s), \tag*{(38)} 0 = c ij + a i + b j ≥ c ij + d ˉ i ( s ) + e ˉ j ( s ) = g c ij , − ′ ( s ) , (38)
which contradicts 0 < g c i j , − ′ ( s ) 0 < g'_{c_{ij},-}(s) 0 < g c ij , − ′ ( s ) , hence (33) holds. □
Remark 3 (Smooth penalties). If μ i , ν j > 0 \mu_i,\nu_j > 0 μ i , ν j > 0 and ϕ \phi ϕ is differentiable, we have that T i j ( q ) = t > 0 T_{ij}(q) = t > 0 T ij ( q ) = t > 0 if and only if
q + ϕ ′ ( t / μ i ) + ϕ ′ ( t / ν j ) = 0 , q + \phi'(t/\mu_i) + \phi'(t/\nu_j) = 0, q + ϕ ′ ( t / μ i ) + ϕ ′ ( t / ν j ) = 0 ,
otherwise T i j ( q ) = 0 T_{ij}(q) = 0 T ij ( q ) = 0 . Notice however that no differentiability is needed in Theorem 3. For reference, if ϕ ( t ) = t log t − t + 1 \phi(t) = t \log t - t + 1 ϕ ( t ) = t log t − t + 1 , then T i j ( q ) = μ i ν j exp ( − q / 2 ) T_{ij}(q) = \sqrt{\mu_i\nu_j}\exp(-q/2) T ij ( q ) = μ i ν j exp ( − q /2 ) .
Notice that if T i j ( c i j ) = 0 T_{ij}(c_{ij}) = 0 T ij ( c ij ) = 0 for some of the costs c i j c_{ij} c ij , Theorem 3 establishes that π i j = 0 \pi_{ij} = 0 π ij = 0 for every solution to (29) , allowing us to remove the variable from the problem. In the next theorem, we give a characterization of this phenomenon. In what follows, we set
c i , min : = min j ∈ [ m ] c i j and c min , j : = min i ∈ [ n ] c i j . (39) c_{i,\min} := \min_{j \in[m]} c_{ij}
\qquad\text{and}\qquad
c_{\min,j} := \min_{i \in[n]} c_{ij}. \tag*{(39)} c i , m i n := j ∈ [ m ] min c ij and c m i n , j := i ∈ [ n ] min c ij . (39)
Theorem 4 (Support bounds). Every optimal transportation plan has π i j = 0 \pi_{ij} = 0 π ij = 0 if μ i = ν j = 0 \mu_i = \nu_j = 0 μ i = ν j = 0 . Moreover, the following implications hold for every optimizer
(i) If ϕ 0 ′ > − ∞ \phi'_0 > -\infty ϕ 0 ′ > − ∞ , then
c i j > − 2 ϕ 0 ′ ⟹ π i j = 0. (40) c_{ij} > -2\phi'_0 \quad\Longrightarrow\quad\pi_{ij} = 0. \tag*{(40)} c ij > − 2 ϕ 0 ′ ⟹ π ij = 0. (40) (ii) If ϕ ∞ ′ < + ∞ \phi'_\infty< +\infty ϕ ∞ ′ < + ∞ , then
c i j > c i , min + c min , j + 2 ϕ ∞ ′ ⟹ π i j = 0. (41) c_{ij} > c_{i,\min} + c_{\min,j} + 2\phi'_\infty
\quad\Longrightarrow\quad\pi_{ij} = 0. \tag*{(41)} c ij > c i , m i n + c m i n , j + 2 ϕ ∞ ′ ⟹ π ij = 0. (41)
Proof. Without loss of generality, let π i j > 0 \pi_{ij} > 0 π ij > 0 , then ( P 1 π ) i , ( P 2 π ) j > 0 (P_1\pi)_i,(P_2\pi)_j > 0 ( P 1 π ) i , ( P 2 π ) j > 0 . Owing to Theorem 2, we find two potentials, namely a i a_i a i and b j b_j b j , such that
c i j + a i + b j = 0 ⟹ c i j = − a i − b j . c_{ij} + a_i + b_j = 0 \quad\Longrightarrow\quad c_{ij} = -a_i - b_j. c ij + a i + b j = 0 ⟹ c ij = − a i − b j .
If μ i = ν j = 0 \mu_i = \nu_j = 0 μ i = ν j = 0 , the result follows from Theorem 1.
Let us now consider (i). Owing to Theorem 2, we have that a i ∈ ∂ ϕ μ i ( ( P 1 π ) i ) a_i \in\partial\phi_{\mu_i}((P_1\pi)_i) a i ∈ ∂ ϕ μ i (( P 1 π ) i ) and b j ∈ ∂ ϕ ν j ( ( P 2 π ) j ) b_j \in\partial\phi_{\nu_j}((P_2\pi)_j) b j ∈ ∂ ϕ ν j (( P 2 π ) j ) . If π i j > 0 \pi_{ij} > 0 π ij > 0 , combining (6) and (8) with c i j + a i + b j = 0 c_{ij} + a_i + b_j = 0 c ij + a i + b j = 0 gives
a i ≥ ϕ 0 ′ , b j ≥ ϕ 0 ′ , c i j = − a i − b j ≤ − 2 ϕ 0 ′ a_i \ge\phi'_0,\qquad b_j \ge\phi'_0,\qquad c_{ij} = -a_i-b_j \le-2\phi'_0 a i ≥ ϕ 0 ′ , b j ≥ ϕ 0 ′ , c ij = − a i − b j ≤ − 2 ϕ 0 ′
which concludes the proof of (i).
Let us now consider (ii). Owing again to Theorem 2, we have a l ∈ ∂ φ μ ( ( P 1 π ) l ) a_l \in\partial\varphi_\mu((P_1\pi)_l) a l ∈ ∂ φ μ (( P 1 π ) l ) , thus a l ≤ φ ∞ ′ a_l \le\varphi'_\infty a l ≤ φ ∞ ′ for every l ∈ [ n ] l \in[n] l ∈ [ n ] and, likewise, b k ≤ φ ∞ ′ b_k \le\varphi'_\infty b k ≤ φ ∞ ′ for every k ∈ [ m ] k \in[m] k ∈ [ m ] . Choose j i ∈ [ m ] j_i \in[m] j i ∈ [ m ] such that c i j i = c i , min c_{ij_i} = c_{i,\min} c i j i = c i , m i n and i j ∈ [ n ] i_j \in[n] i j ∈ [ n ] such that c i j j = c min , j c_{i_jj} = c_{\min,j} c i j j = c m i n , j . Then the optimality conditions yield
0 ≤ c i , min + a i + b j i ⟹ a i ≥ − c i , min − b j i ≥ − c i , min − φ ∞ ′ , 0 ≤ c min , j + a i j + b j ⟹ b j ≥ − c min , j − a i j ≥ − c min , j − φ ∞ ′ . \begin{aligned}
0 &\le c_{i,\min} + a_i + b_{j_i} &&\Longrightarrow&& a_i \ge-c_{i,\min} - b_{j_i} \ge-c_{i,\min} - \varphi'_\infty,\\
0 &\le c_{\min,j} + a_{i_j} + b_j &&\Longrightarrow&& b_j \ge-c_{\min,j} - a_{i_j} \ge-c_{\min,j} - \varphi'_\infty.
\end{aligned} 0 0 ≤ c i , m i n + a i + b j i ≤ c m i n , j + a i j + b j ⟹ ⟹ a i ≥ − c i , m i n − b j i ≥ − c i , m i n − φ ∞ ′ , b j ≥ − c m i n , j − a i j ≥ − c m i n , j − φ ∞ ′ .
which, combined together gives c i j = − a i − b j ≤ c i , min + c min , j + 2 φ ∞ ′ c_{ij} = -a_i - b_j \le c_{i,\min} + c_{\min,j} + 2\varphi'_\infty c ij = − a i − b j ≤ c i , m i n + c m i n , j + 2 φ ∞ ′ , which concludes the proof.
Under additional assumptions, the bounds presented in Theorem 4 simplify as follows.
Corollary 1. Under the hypothesis of Theorem 4, if X = Y \mathcal{X} = \mathcal{Y} X = Y , c ( x , x ) = 0 c(x,x) = 0 c ( x , x ) = 0 for all x ∈ X x \in\mathcal{X} x ∈ X , we have
( i ∈ [ n ] , j ∈ [ m ] ) c i j > 2 min { − φ 0 ′ , φ ∞ ′ } ⟹ π i j = 0. (42) (i \in[n],j \in[m]) \qquad c_{ij} > 2\min\{-\varphi'_0,\varphi'_\infty\} \qquad\Longrightarrow\qquad\pi_{ij} = 0. \tag*{(42)} ( i ∈ [ n ] , j ∈ [ m ]) c ij > 2 min { − φ 0 ′ , φ ∞ ′ } ⟹ π ij = 0. (42) Proof. It follows from Theorem 4 since c i , min = c min , j = 0 c_{i,\min} = c_{\min,j} = 0 c i , m i n = c m i n , j = 0 for every i ∈ [ n ] i \in[n] i ∈ [ n ] and every j ∈ [ m ] j \in[m] j ∈ [ m ] .
It is important to notice that the support bounds presented in Corollary 1 and in Theorem 4 do not depend on the input measures μ \mu μ and ν \nu ν , but only on φ ∞ ′ \varphi'_\infty φ ∞ ′ , φ 0 ′ \varphi'_0 φ 0 ′ , and c c c .
The second consequence of the bounds presented in Theorem 3 is that there exists a solution to (3.17) whose total mass, intended as the sum of all the values π i j \pi_{ij} π ij , is bounded.
Proposition 2. For every optimal transport plan π \pi π for problem (3.17), there is an optimal transport plan, namely π ^ \hat{\pi} π ^ , such that π ^ ≤ π \hat{\pi} \le\pi π ^ ≤ π componentwise and, for every couple of indices ( i , j ) ∈ [ n ] × [ m ] (i,j) \in[n] \times[m] ( i , j ) ∈ [ n ] × [ m ] , it holds
π ^ i j > 0 ⟹ ( P 1 π ) i ≤ μ i or ( P 2 π ) j ≤ ν j . (43) \hat{\pi}_{ij} > 0 \qquad\Longrightarrow\qquad(P_1\pi)_i \le\mu_i \text{ or } (P_2\pi)_j \le\nu_j. \tag*{(43)} π ^ ij > 0 ⟹ ( P 1 π ) i ≤ μ i or ( P 2 π ) j ≤ ν j . (43) In particular, we have that π ^ i j ≤ max { μ i , ν j } \hat{\pi}_{ij} \le\max\{\mu_i,\nu_j\} π ^ ij ≤ max { μ i , ν j } , therefore
∥ π ^ ∥ 1 ≤ B 0 ≔ ∑ i = 1 n ∑ j = 1 m max { μ i , ν j } . (44) \|\hat{\pi}\|_1 \le B_0 \coloneqq\sum_{i=1}^{n}\sum_{j=1}^{m}\max\{\mu_i,\nu_j\}. \tag*{(44)} ∥ π ^ ∥ 1 ≤ B 0 : = i = 1 ∑ n j = 1 ∑ m max { μ i , ν j } . (44) Proof. Let π \pi π be the solution to (3.17). If π i j = 0 \pi_{ij} = 0 π ij = 0 for every i ∈ [ n ] , j ∈ [ m ] i \in[n],j \in[m] i ∈ [ n ] , j ∈ [ m ] , we have nothing to prove. Assume then that i ∈ [ n ] i \in[n] i ∈ [ n ] and j ∈ [ m ] j \in[m] j ∈ [ m ] are such that π i j > 0 \pi_{ij} > 0 π ij > 0 and both the following conditions hold
( P 1 π ) i > μ i and ( P 2 π ) j > ν j . (45) (P_1\pi)_i > \mu_i \qquad\text{and}\qquad(P_2\pi)_j > \nu_j. \tag*{(45)} ( P 1 π ) i > μ i and ( P 2 π ) j > ν j . (45) Then, set ε = min { π i j , ( P 1 π ) i − μ i , ( P 2 π ) j − ν j } > 0 \varepsilon= \min\{\pi_{ij},(P_1\pi)_i-\mu_i,(P_2\pi)_j-\nu_j\} > 0 ε = min { π ij , ( P 1 π ) i − μ i , ( P 2 π ) j − ν j } > 0 . We have (i) π i j − ε ≥ 0 \pi_{ij}-\varepsilon\ge0 π ij − ε ≥ 0 , and (ii) ( P 1 π ) i − ε ≥ μ i (P_1\pi)_i-\varepsilon\ge\mu_i ( P 1 π ) i − ε ≥ μ i and ( P 2 π ) j − ε ≥ ν j (P_2\pi)_j-\varepsilon\ge\nu_j ( P 2 π ) j − ε ≥ ν j .
Then, π ′ = π − ε δ ( i , j ) \pi' = \pi-\varepsilon\delta_{(i,j)} π ′ = π − ε δ ( i , j ) is feasible for problem (3.17). Moreover, since c i j ≥ 0 c_{ij} \ge0 c ij ≥ 0 and owing to the assumptions on φ \varphi φ , we have that U φ , c ( π ′ ) ≤ U φ , c ( π ) U_{\varphi,c}(\pi') \le U_{\varphi,c}(\pi) U φ , c ( π ′ ) ≤ U φ , c ( π ) , hence π ′ \pi' π ′ is also optimal. By iterating the process, we build a solution to problem (3.17), such that either ( P 1 π ) i ≤ μ i (P_1\pi)_i \le\mu_i ( P 1 π ) i ≤ μ i or ( P 2 π ) j ≤ ν j (P_2\pi)_j \le\nu_j ( P 2 π ) j ≤ ν j hold.
Let us assume that π \pi π is a solution to (3.17) such that (3.31) holds, then
π i j > 0 ⟹ π i j ≤ ( P 1 π ) i ≤ μ i or π i j ≤ ( P 2 π ) j ≤ ν j . \pi_{ij} > 0 \qquad\Longrightarrow\qquad\pi_{ij} \le(P_1\pi)_i \le\mu_i \quad\text{or}\quad\pi_{ij} \le(P_2\pi)_j \le\nu_j. π ij > 0 ⟹ π ij ≤ ( P 1 π ) i ≤ μ i or π ij ≤ ( P 2 π ) j ≤ ν j . Thus π i j ≤ max { μ i , ν j } \pi_{ij} \le\max\{\mu_i,\nu_j\} π ij ≤ max { μ i , ν j } and hence (3.32).
In particular, problem (3.17) can be cast as a minimization problem over a scaled generalized simplex in R n × m \mathbb{R}^{n\times m} R n × m , indeed
v c = min π ∈ Λ B 0 F c ( π ) with Λ B 0 : = { π ≥ 0 : ∥ π ∥ 1 ≤ B 0 } . (46) v_c=\min_{\pi\in\Lambda_{B_0}} F_c(\pi)\qquad\text{with}\qquad\Lambda_{B_0}:=\{\pi\geq0:\|\pi\|_1\leq B_0\}. \tag*{(46)} v c = π ∈ Λ B 0 min F c ( π ) with Λ B 0 := { π ≥ 0 : ∥ π ∥ 1 ≤ B 0 } . (46)
Notice that the set Λ B 0 \Lambda_{B_0} Λ B 0 depends only on μ \mu μ and ν \nu ν and not on D ϕ D_\phi D ϕ and c c c . To conclude, we infer a lower bound on the marginals of any optimal solution. First, we define the lower generalized inverse
I ϕ ( q ) : = inf { t ≥ 0 : ϕ + ′ ( t ) ≥ q } , q ∈ R , (47) I_\phi(q):=\inf\{t\geq0:\phi_+'(t)\geq q\},\qquad q\in\mathbb{R}, \tag*{(47)} I ϕ ( q ) := inf { t ≥ 0 : ϕ + ′ ( t ) ≥ q } , q ∈ R , (47)
with inf ∅ = + ∞ \inf\varnothing=+\infty inf ∅ = + ∞ . Notice that if q < 0 q<0 q < 0 , the set defining I ϕ ( q ) I_\phi(q) I ϕ ( q ) is always nonempty as ϕ + ′ ( 1 ) ≥ 0 \phi_+'(1)\geq0 ϕ + ′ ( 1 ) ≥ 0 .
Theorem 5. Let U i , V j U_i,V_j U i , V j be finite, strictly positive numbers such that every π ∈ Λ B 0 \pi\in\Lambda_{B_0} π ∈ Λ B 0 solution to (3.34) satisfies ( P 1 π ) i ≤ U i (P_1\pi)_i\leq U_i ( P 1 π ) i ≤ U i and ( P 2 π ) j ≤ V j (P_2\pi)_j\leq V_j ( P 2 π ) j ≤ V j .1 Define
A i : = { ϕ + ′ ( U i / μ i ) , μ i > 0 , ϕ 0 ′ , μ i = 0 , B j : = { ϕ + ′ ( V j / ν j ) , ν j > 0 , ϕ ∞ ′ , ν j = 0. (48) A_i:=\begin{cases}
\phi_+'\left(U_i/\mu_i\right), & \mu_i>0,\\
\phi'_0, & \mu_i=0,
\end{cases}
\qquad
B_j:=\begin{cases}
\phi_+'\left(V_j/\nu_j\right), & \nu_j>0,\\
\phi'_\infty, & \nu_j=0.
\end{cases} \tag*{(48)} A i := { ϕ + ′ ( U i / μ i ) , ϕ 0 ′ , μ i > 0 , μ i = 0 , B j := { ϕ + ′ ( V j / ν j ) , ϕ ∞ ′ , ν j > 0 , ν j = 0. (48) Then every optimizer π \pi π of (3.17), that belongs to Λ B 0 \Lambda_{B_0} Λ B 0 , satisfies
( P 1 π ) i ≥ μ i max j ∈ [ m ] I ϕ ( − c i j − B j ) if μ i > 0 , (49) (P_1\pi)_i\geq\mu_i\max_{j\in[m]}I_\phi(-c_{ij}-B_j)\qquad\text{if }\mu_i>0, \tag*{(49)} ( P 1 π ) i ≥ μ i j ∈ [ m ] max I ϕ ( − c ij − B j ) if μ i > 0 , (49) ( P 2 π ) j ≥ ν j max i ∈ [ n ] I ϕ ( − c i j − A i ) if ν j > 0. (50) (P_2\pi)_j\geq\nu_j\max_{i\in[n]}I_\phi(-c_{ij}-A_i)\qquad\text{if }\nu_j>0. \tag*{(50)} ( P 2 π ) j ≥ ν j i ∈ [ n ] max I ϕ ( − c ij − A i ) if ν j > 0. (50) If ϕ 0 ′ = − ∞ \phi'_0=-\infty ϕ 0 ′ = − ∞ , these lower bounds are strictly positive. In particular, there exists a common ε > 0 \varepsilon>0 ε > 0 for which if μ i > 0 \mu_i>0 μ i > 0 then ( P 1 π ) i ≥ ε (P_1\pi)_i\geq\varepsilon ( P 1 π ) i ≥ ε and if ν j > 0 \nu_j>0 ν j > 0 then ( P 2 π ) j ≥ ε (P_2\pi)_j\geq\varepsilon ( P 2 π ) j ≥ ε for every optimal π ∈ Λ B 0 \pi\in\Lambda_{B_0} π ∈ Λ B 0 .
Proof. Fix an arbitrary optimizer π \pi π in Λ B 0 \Lambda_{B_0} Λ B 0 . Write r = P 1 π r=P_1\pi r = P 1 π , s = P 2 π s=P_2\pi s = P 2 π , and choose potentials a , b a,b a , b for π \pi π as in Theorem 2. Since r i ≤ U i r_i\leq U_i r i ≤ U i and s j ≤ V j s_j\leq V_j s j ≤ V j , we have
a i ≤ A i , and b j ≤ B j . a_i\leq A_i,\qquad\text{and}\qquad b_j\leq B_j. a i ≤ A i , and b j ≤ B j . Thus, given i ∈ [ n ] i\in[n] i ∈ [ n ] such that μ i > 0 \mu_i>0 μ i > 0 , for every j ∈ [ m ] j\in[m] j ∈ [ m ] , it holds
ϕ + ′ ( r i / μ i ) ≥ a i ≥ − c i j − b j ≥ − c i j − B j , \phi_+'\left(r_i/\mu_i\right)\geq a_i\geq-c_{ij}-b_j\geq-c_{ij}-B_j, ϕ + ′ ( r i / μ i ) ≥ a i ≥ − c ij − b j ≥ − c ij − B j , and hence
r i / μ i ∈ { t ≥ 0 : ϕ + ′ ( t ) ≥ − c i j − B j } , ⇒ r i ≥ μ i I ϕ ( − c i j − B j ) . r_i/\mu_i\in\{t\geq0:\phi_+'(t)\geq-c_{ij}-B_j\},\qquad\Rightarrow\qquad r_i\geq\mu_i I_\phi(-c_{ij}-B_j). r i / μ i ∈ { t ≥ 0 : ϕ + ′ ( t ) ≥ − c ij − B j } , ⇒ r i ≥ μ i I ϕ ( − c ij − B j ) . so that, by taking the maximum with respect to j ∈ [ m ] j\in[m] j ∈ [ m ] allows us to conclude (3.37). Likewise, given j ∈ [ m ] j\in[m] j ∈ [ m ] such that ν j > 0 \nu_j>0 ν j > 0 , for every i ∈ [ n ] i\in[n] i ∈ [ n ] , it holds
ϕ + ′ ( s j / ν j ) ≥ b j ≥ − c i j − a i ≥ − c i j − A i , ⇒ s j ≥ ν j I ϕ ( − c i j − A i ) , \phi_+'\left(s_j/\nu_j\right)\geq b_j\geq-c_{ij}-a_i\geq-c_{ij}-A_i,\qquad\Rightarrow\qquad s_j\geq\nu_j I_\phi(-c_{ij}-A_i), ϕ + ′ ( s j / ν j ) ≥ b j ≥ − c ij − a i ≥ − c ij − A i , ⇒ s j ≥ ν j I ϕ ( − c ij − A i ) , thus, by taking the maximum with respect to i ∈ [ n ] i\in[n] i ∈ [ n ] , we infer (3.38).
If ϕ 0 ′ = − ∞ \phi'_0=-\infty ϕ 0 ′ = − ∞ , then for each finite q q q ,
lim t → 0 ϕ + ′ ( t ) = − ∞ ⟹ ( ∃ ε q > 0 ) ( ∀ t ∈ [ 0 , ε q ] ) ϕ + ′ ( t ) < q ⟹ I ϕ ( q ) ≥ ε q > 0. \lim_{t\to0}\phi_+'(t)=-\infty\qquad\Longrightarrow\qquad(\exists\varepsilon_q>0)(\forall t\in[0,\varepsilon_q])\ \phi_+'(t)<q\qquad\Longrightarrow\qquad I_\phi(q)\geq\varepsilon_q>0. t → 0 lim ϕ + ′ ( t ) = − ∞ ⟹ ( ∃ ε q > 0 ) ( ∀ t ∈ [ 0 , ε q ]) ϕ + ′ ( t ) < q ⟹ I ϕ ( q ) ≥ ε q > 0. Since the A i , B j , c i j A_i,B_j,c_{ij} A i , B j , c ij are finite, all the asserted lower bounds are strictly positive in this case. The last statement follows from setting ε = ε q ˉ min i ∈ I + , j ∈ J + { μ i , ν j } \varepsilon=\varepsilon_{\bar q}\min_{i\in I_+,j\in J_+}\{\mu_i,\nu_j\} ε = ε q ˉ min i ∈ I + , j ∈ J + { μ i , ν j } with
q ˉ = min i , j { − c i j − A i , − c i j − B j } , \bar q=\min_{i,j}\{-c_{ij}-A_i,-c_{ij}-B_j\}, q ˉ = i , j min { − c ij − A i , − c ij − B j } , and I + = { i ∈ [ n ] : μ i > 0 } I_+=\{i\in[n]:\mu_i>0\} I + = { i ∈ [ n ] : μ i > 0 } and J + = { j ∈ [ m ] : ν j > 0 } J_+=\{j\in[m]:\nu_j>0\} J + = { j ∈ [ m ] : ν j > 0 } .
Remark 4. Notice that Theorem 5 holds for every set of transportation plans π \pi π such that ∥ π ∥ 1 ≤ K B 0 \|\pi\|_1\leq KB_0 ∥ π ∥ 1 ≤ K B 0 with K K K positive constant.
Cost truncation and a compressed formulation
We now focus on UOT problems induced by truncated costs, i.e., costs that are capped at a given threshold R > 0 R>0 R > 0 . This class of problems simplifies the structure of the transportation plan, as we do not need to keep track of how the mass is moved between points whose distance is equal to R R R [40 ] . The problem induced by the truncated cost has a finite-valued minimizer by the same argument used for the original problem as in (12) . Moreover, owing to Theorem 1, there exists at least one solution supported over a discrete set. Given a cost function c c c and a positive threshold R > 0 R>0 R > 0 , we then set
c i j ( R ) : = min { c i j , R } , E R : = { ( i , j ) : c i j < R } , E R c : = ( [ n ] × [ m ] ) ∖ E R , (51) c_{ij}^{(R)} := \min\{c_{ij},R\}, \qquad\mathcal{E}_{R} := \{(i,j):c_{ij}<R\}, \qquad\mathcal{E}_{R}^{c} := ([n]\times[m])\setminus\mathcal{E}_{R}, \tag*{(51)} c ij ( R ) := min { c ij , R } , E R := {( i , j ) : c ij < R } , E R c := ([ n ] × [ m ]) ∖ E R , (51)
and define
v ^ R : = min π ≥ 0 U T ϕ , c ( R ) ( π ) . (52) \widehat{v}_{R} := \min_{\pi\geq0}\mathrm{UT}_{\phi,c^{(R)}}(\pi). \tag*{(52)} v R := π ≥ 0 min UT ϕ , c ( R ) ( π ) . (52)
Notice that
R 1 ≤ R 2 ⟹ c i j ( R 1 ) ≤ c i j ( R 2 ) ( ( i ∈ [ n ] , j ∈ [ m ] ) ) ⟹ v ^ R 1 ≤ v ^ R 2 ≤ v c . (53) R_{1}\leq R_{2}\quad\Longrightarrow\quad c_{ij}^{(R_{1})}\leq c_{ij}^{(R_{2})}\quad((i\in[n],\ j\in[m]))\quad\Longrightarrow\quad\widehat{v}_{R_{1}}\leq\widehat{v}_{R_{2}}\leq v_{c}. \tag*{(53)} R 1 ≤ R 2 ⟹ c ij ( R 1 ) ≤ c ij ( R 2 ) (( i ∈ [ n ] , j ∈ [ m ])) ⟹ v R 1 ≤ v R 2 ≤ v c . (53)
The grids selected for c c c also suffice for its continuous capped cost min { c , R } \min\{c,R\} min { c , R } : every selected minimizer of c ( x , ⋅ ) c(x,\cdot) c ( x , ⋅ ) or c ( ⋅ , y ) c(\cdot,y) c ( ⋅ , y ) is still a minimizer after capping. The problem induced by the truncated cost does not remove the variables in E R c \mathcal{E}_{R}^{c} E R c , but makes them all have the same cost. The following formulation exploits this fact by considering the row and column totals associated with the couples in E R c \mathcal{E}_{R}^{c} E R c instead of their individual entries.
Compression of the capped links
Let η ∈ R + n × m \eta\in\mathbb{R}_{+}^{n\times m} η ∈ R + n × m be supported on E R \mathcal{E}_{R} E R , that is η i j = 0 \eta_{ij}=0 η ij = 0 for every ( i , j ) ∈ E R c (i,j)\in\mathcal{E}_{R}^{c} ( i , j ) ∈ E R c and let u ∈ R + n u\in\mathbb{R}_{+}^{n} u ∈ R + n and w ∈ R + m w\in\mathbb{R}_{+}^{m} w ∈ R + m satisfy ∥ u ∥ 1 = ∥ w ∥ 1 \lVert u\rVert_{1}=\lVert w\rVert_{1} ∥ u ∥ 1 = ∥ w ∥ 1 . We define
G R ( η , u , w ) : = ⟨ c , η ⟩ + R ∥ u ∥ 1 + ∑ i ∈ [ n ] φ μ i ( ( P 1 η ) i + u i ) + ∑ j ∈ [ m ] φ ν j ( ( P 2 η ) j + w j ) . (54) G_{R}(\eta,u,w):=\langle c,\eta\rangle+R\lVert u\rVert_{1}+\sum_{i\in[n]}\varphi_{\mu_i}((P_{1}\eta)_i+u_i)+\sum_{j\in[m]}\varphi_{\nu_j}((P_{2}\eta)_j+w_j). \tag*{(54)} G R ( η , u , w ) := ⟨ c , η ⟩ + R ∥ u ∥ 1 + i ∈ [ n ] ∑ φ μ i (( P 1 η ) i + u i ) + j ∈ [ m ] ∑ φ ν j (( P 2 η ) j + w j ) . (54)
and the following feasible set
F R : = { ( η , u , w ) ∈ R + n × m × R + n × R + m : η i j = 0 on E R c , and ∥ u ∥ 1 = ∥ w ∥ 1 } . (55) \mathcal{F}_{R}:=\{(\eta,u,w)\in\mathbb{R}_{+}^{n\times m}\times\mathbb{R}_{+}^{n}\times\mathbb{R}_{+}^{m}:\eta_{ij}=0\text{ on }\mathcal{E}_{R}^{c},\text{ and }\lVert u\rVert_{1}=\lVert w\rVert_{1}\}. \tag*{(55)} F R := {( η , u , w ) ∈ R + n × m × R + n × R + m : η ij = 0 on E R c , and ∥ u ∥ 1 = ∥ w ∥ 1 } . (55)
Notice that only the entries of η \eta η in E R \mathcal{E}_{R} E R need to be stored, so that the formulation uses ∣ E R ∣ + n + m |\mathcal{E}_{R}|+n+m ∣ E R ∣ + n + m nonnegative variables and one balance equation.
For any full transportation plan π ≥ 0 \pi\geq0 π ≥ 0 , we define a canonical element C ( π ) = ( C η ( π ) , C u ( π ) , C w ( π ) ) ∈ F R C(\pi)=(C_{\eta}(\pi),C_{u}(\pi),C_{w}(\pi))\in\mathcal{F}_{R} C ( π ) = ( C η ( π ) , C u ( π ) , C w ( π )) ∈ F R as follows
C η ( π ) = η i j = { π i j , ( i , j ) ∈ E R , 0 , ( i , j ) ∈ E R c , (56) C_{\eta}(\pi)=\eta_{ij}=
\begin{cases}
\pi_{ij}, & (i,j)\in\mathcal{E}_{R},\\
0, & (i,j)\in\mathcal{E}_{R}^{c},
\end{cases} \tag*{(56)} C η ( π ) = η ij = { π ij , 0 , ( i , j ) ∈ E R , ( i , j ) ∈ E R c , (56)
and
C u ( π ) = u i = ∑ j : ( i , j ) ∈ E R π i j , C w ( π ) = w j = ∑ i : ( i , j ) ∈ E R π i j . (57) C_{u}(\pi)=u_i=\sum_{j:(i,j)\in\mathcal{E}_{R}}\pi_{ij},\qquad C_{w}(\pi)=w_j=\sum_{i:(i,j)\in\mathcal{E}_{R}}\pi_{ij}. \tag*{(57)} C u ( π ) = u i = j : ( i , j ) ∈ E R ∑ π ij , C w ( π ) = w j = i : ( i , j ) ∈ E R ∑ π ij . (57)
It is easy to see that U T ϕ , c ( R ) ( π ) = G R ( C ( π ) ) \mathrm{UT}_{\phi,c^{(R)}}(\pi)=G_{R}(C(\pi)) UT ϕ , c ( R ) ( π ) = G R ( C ( π )) . Conversely, given an element of ( η , u , w ) ∈ F R (\eta,u,w)\in\mathcal{F}_{R} ( η , u , w ) ∈ F R , we define
L ( η , u , w ) = η + ϱ with arbitrary ϱ ∈ Π ( u , w ) = { γ ∈ R + n × m : P 1 γ = u , P 2 γ = w } , (58) \mathrm{L}(\eta,u,w)=\eta+\varrho\quad\text{with arbitrary }\varrho\in\Pi(u,w)=\{\gamma\in\mathbb{R}_{+}^{n\times m}:P_{1}\gamma=u,\ P_{2}\gamma=w\}, \tag*{(58)} L ( η , u , w ) = η + ϱ with arbitrary ϱ ∈ Π ( u , w ) = { γ ∈ R + n × m : P 1 γ = u , P 2 γ = w } , (58)
notice that Π ( u , w ) ≠ ∅ \Pi(u,w) \ne\varnothing Π ( u , w ) = ∅ , hence L is well-defined. However, in general, it holds
G R ( η , u , w ) ≥ UT ϕ , c ( R ) ( L ( η , u , w ) ) . (59) G_R(\eta,u,w) \ge\operatorname{UT}_{\phi,c(R)}(L(\eta,u,w)). \tag*{(59)} G R ( η , u , w ) ≥ UT ϕ , c ( R ) ( L ( η , u , w )) . (59)
In the next lemma, we show that if ( η , u , w ) (\eta,u,w) ( η , u , w ) is optimal, then (4.9) holds as equality regardless of how ϱ \varrho ϱ is selected in (4.8).
Lemma 1. Let ( η ⋆ , u ⋆ , w ⋆ ) ∈ F R (\eta^\star,u^\star,w^\star) \in\mathcal{F}_R ( η ⋆ , u ⋆ , w ⋆ ) ∈ F R be a minimizer of G R G_R G R . Then
( i , j ) ∈ E R ⟹ u i ⋆ w j ⋆ = 0 and G R ( η ⋆ , u ⋆ , w ⋆ ) = UT ϕ , c ( R ) ( L ( η ⋆ , u ⋆ , w ⋆ ) ) . (60) (i,j) \in\mathcal{E}_R \quad\Longrightarrow\quad u_i^\star w_j^\star= 0 \quad\text{and} \quad G_R(\eta^\star,u^\star,w^\star) = \operatorname{UT}_{\phi,c(R)}(L(\eta^\star,u^\star,w^\star)). \tag*{(60)} ( i , j ) ∈ E R ⟹ u i ⋆ w j ⋆ = 0 and G R ( η ⋆ , u ⋆ , w ⋆ ) = UT ϕ , c ( R ) ( L ( η ⋆ , u ⋆ , w ⋆ )) . (60)
Proof. Let ( η ⋆ , u ⋆ , w ⋆ ) ∈ F R (\eta^\star,u^\star,w^\star) \in\mathcal{F}_R ( η ⋆ , u ⋆ , w ⋆ ) ∈ F R be a minimizer of G R G_R G R . If c i j < R c_{ij}<R c ij < R and u i ⋆ w j ⋆ > 0 u_i^\star w_j^\star>0 u i ⋆ w j ⋆ > 0 , define
δ = min { u i ⋆ , w j ⋆ } > 0 , η ′ = η ⋆ + δ e i e j T , u ′ = u ⋆ − δ e i , w ′ = w ⋆ − δ e j , \delta= \min\{u_i^\star,w_j^\star\}>0,\qquad\eta'=\eta^\star+\delta e_i e_j^{\mathsf T},\qquad u'=u^\star-\delta e_i,\qquad w'=w^\star-\delta e_j, δ = min { u i ⋆ , w j ⋆ } > 0 , η ′ = η ⋆ + δ e i e j T , u ′ = u ⋆ − δ e i , w ′ = w ⋆ − δ e j ,
where e i e_i e i denotes the i i i -th vector of the Euclidean canonical base. By construction, ∥ u ′ ∥ 1 = ∥ w ′ ∥ 1 \lVert u'\rVert_1=\lVert w'\rVert_1 ∥ u ′ ∥ 1 = ∥ w ′ ∥ 1 , hence ( η ′ , u ′ , w ′ ) ∈ F R (\eta',u',w')\in\mathcal{F}_R ( η ′ , u ′ , w ′ ) ∈ F R and
P 1 η ′ + u ′ = P 1 η ⋆ + u ⋆ , P 2 η ′ + w ′ = P 2 η ⋆ + w ⋆ , P_1\eta'+u'=P_1\eta^\star+u^\star,\qquad P_2\eta'+w'=P_2\eta^\star+w^\star, P 1 η ′ + u ′ = P 1 η ⋆ + u ⋆ , P 2 η ′ + w ′ = P 2 η ⋆ + w ⋆ ,
G R ( η ′ , u ′ , w ′ ) − G R ( η ⋆ , u ⋆ , w ⋆ ) = c i j δ − R δ < 0 , G_R(\eta',u',w')-G_R(\eta^\star,u^\star,w^\star)=c_{ij}\delta-R\delta<0, G R ( η ′ , u ′ , w ′ ) − G R ( η ⋆ , u ⋆ , w ⋆ ) = c ij δ − R δ < 0 ,
contradicting optimality, therefore u i ⋆ w j ⋆ = 0 u_i^\star w_j^\star=0 u i ⋆ w j ⋆ = 0 on E R \mathcal{E}_R E R . Lastly, given ϱ ∈ Π ( u ⋆ , w ⋆ ) \varrho\in\Pi(u^\star,w^\star) ϱ ∈ Π ( u ⋆ , w ⋆ ) we have
0 ≤ ϱ i j ≤ min { u i ⋆ , w j ⋆ } = 0 ( ( i , j ) ∈ E R ) ⟹ ∑ i , j c i j ( R ) ϱ i j = R ∑ i , j ϱ i j = R ∑ i u i ⋆ , 0\leq\varrho_{ij}\leq\min\{u_i^\star,w_j^\star\}=0\quad((i,j)\in\mathcal{E}_R)
\quad\Longrightarrow\quad
\sum_{i,j}c_{ij}^{(R)}\varrho_{ij}=R\sum_{i,j}\varrho_{ij}=R\sum_i u_i^\star, 0 ≤ ϱ ij ≤ min { u i ⋆ , w j ⋆ } = 0 (( i , j ) ∈ E R ) ⟹ i , j ∑ c ij ( R ) ϱ ij = R i , j ∑ ϱ ij = R i ∑ u i ⋆ ,
proving the second identity in (4.10).
Owing to the latter result, we are able to prove that the UOT problem induced by the truncated cost at threshold R > 0 R>0 R > 0 is the same as minimizing G R G_R G R over the set F R \mathcal{F}_R F R .
Theorem 6. Given two discrete measures μ \mu μ and ν \nu ν , we have that
min π ∈ R + n × m F c ( R ) ( π ) = min ( η , u , w ) ∈ F R G R ( η , u , w ) . (61) \min_{\pi\in\mathbb{R}_+^{n\times m}} F_{c(R)}(\pi)
=
\min_{(\eta,u,w)\in\mathcal{F}_R} G_R(\eta,u,w). \tag*{(61)} π ∈ R + n × m min F c ( R ) ( π ) = ( η , u , w ) ∈ F R min G R ( η , u , w ) . (61) Moreover, the following two statements hold
Given π ⋆ ∈ argmin F c ( R ) \pi^\star\in\operatorname*{argmin} F_{c(R)} π ⋆ ∈ argmin F c ( R ) , then C ( π ⋆ ) ∈ argmin G R C(\pi^\star)\in\operatorname*{argmin}G_R C ( π ⋆ ) ∈ argmin G R and
if ( η ⋆ , u ⋆ , w ⋆ ) ∈ argmin G R (\eta^\star,u^\star,w^\star)\in\operatorname*{argmin}G_R ( η ⋆ , u ⋆ , w ⋆ ) ∈ argmin G R , then L ( η ⋆ , u ⋆ , w ⋆ ) ∈ argmin F c ( R ) L(\eta^\star,u^\star,w^\star)\in\operatorname*{argmin}F_{c(R)} L ( η ⋆ , u ⋆ , w ⋆ ) ∈ argmin F c ( R ) regardless of how ϱ ∈ Π ( u ⋆ , w ⋆ ) \varrho\in\Pi(u^\star,w^\star) ϱ ∈ Π ( u ⋆ , w ⋆ ) , is chosen.
Proof. Put v ~ R = inf F R G R \widetilde{v}_R=\inf_{\mathcal{F}_R}G_R v R = inf F R G R . Given any π ∈ R + n × m \pi\in\mathbb{R}_+^{n\times m} π ∈ R + n × m , let ( η , u , w ) = C ( π ) (\eta,u,w)=C(\pi) ( η , u , w ) = C ( π ) . Then, by definition ∥ u ∥ 1 = ∥ w ∥ 1 \lVert u\rVert_1=\lVert w\rVert_1 ∥ u ∥ 1 = ∥ w ∥ 1 and
( P 1 η ) i + u i = ∑ j : ( i , j ) ∈ E R π i j + ∑ j : ( i , j ) ∈ E R c π i j = ( P 1 π ) i , (62) (P_1\eta)_i+u_i=\sum_{j:(i,j)\in\mathcal{E}_R}\pi_{ij}+\sum_{j:(i,j)\in\mathcal{E}_R^c}\pi_{ij}=(P_1\pi)_i, \tag*{(62)} ( P 1 η ) i + u i = j : ( i , j ) ∈ E R ∑ π ij + j : ( i , j ) ∈ E R c ∑ π ij = ( P 1 π ) i , (62)
likewise, we infer ( P 2 η ) j + w j = ( P 2 π ) j (P_2\eta)_j+w_j=(P_2\pi)_j ( P 2 η ) j + w j = ( P 2 π ) j , thus
∑ E R c i j π i j + ∑ E R c R π i j = ⟨ c ( R ) , π ⟩ ⟹ G R ( C ( π ) ) = F c ( R ) ( π ) , (63) \sum_{\mathcal{E}_R}c_{ij}\pi_{ij}+\sum_{\mathcal{E}_R^c}R\pi_{ij}
=\langle c^{(R)},\pi\rangle
\quad\Longrightarrow\quad
G_R(C(\pi))=F_{c(R)}(\pi), \tag*{(63)} E R ∑ c ij π ij + E R c ∑ R π ij = ⟨ c ( R ) , π ⟩ ⟹ G R ( C ( π )) = F c ( R ) ( π ) , (63)
so that v ~ R ≤ v R \widetilde{v}_R\leq v_R v R ≤ v R . Let us now take any ( η , u , w ) ∈ F R (\eta,u,w)\in\mathcal{F}_R ( η , u , w ) ∈ F R and set π = L ( η , u , w ) \pi=L(\eta,u,w) π = L ( η , u , w ) . Then, by definition of L, we have P 1 π = P 1 η + u P_1\pi=P_1\eta+u P 1 π = P 1 η + u and P 2 π = P 2 η + w P_2\pi=P_2\eta+w P 2 π = P 2 η + w . Therefore
F c ( R ) ( L ( η , u , w ) ) − G R ( η , u , w ) = ⟨ c ( R ) , ϱ ⟩ − R τ = ∑ i , j ( c i j ( R ) − R ) ϱ i j ≤ 0 , (64) F_{c(R)}(L(\eta,u,w))-G_R(\eta,u,w)
=\langle c^{(R)},\varrho\rangle-R\tau
=\sum_{i,j}(c_{ij}^{(R)}-R)\varrho_{ij}\leq0, \tag*{(64)} F c ( R ) ( L ( η , u , w )) − G R ( η , u , w ) = ⟨ c ( R ) , ϱ ⟩ − R τ = i , j ∑ ( c ij ( R ) − R ) ϱ ij ≤ 0 , (64)
so that v R ≤ v ~ R v_R \le\tilde{v}_R v R ≤ v ~ R hence (4.11). To conclude, we need to prove that C \mathrm{C} C sends a minimizer of F c ( R ) F_{c(R)} F c ( R ) into a minimizer of G R G_R G R and, vice-versa, L \mathrm{L} L sends a minimizer of G R G_R G R in a minimizer of F c ( R ) F_{c(R)} F c ( R ) for every possible choice of ϱ ∈ Π ( u , w ) \varrho\in\Pi(u,w) ϱ ∈ Π ( u , w ) .
The first implication comes from the identity on the right hand side of (4.13) and from v ~ R = v R \tilde{v}_R = v_R v ~ R = v R .
For the second implication, let us take ( η ⋆ , u ⋆ , w ⋆ ) (\eta^\star,u^\star,w^\star) ( η ⋆ , u ⋆ , w ⋆ ) such that G R ( η ⋆ , u ⋆ , w ⋆ ) = v ~ R G_R(\eta^\star,u^\star,w^\star)=\tilde{v}_R G R ( η ⋆ , u ⋆ , w ⋆ ) = v ~ R . Owing to (4.14), we have
v R ≤ F c ( R ) ( L ( η ⋆ , u ⋆ , w ⋆ ) ) ≤ G R ( η ⋆ , u ⋆ , w ⋆ ) = v ~ R = v R , v_R \le F_{c(R)}(\mathrm{L}(\eta^\star,u^\star,w^\star)) \le G_R(\eta^\star,u^\star,w^\star)=\tilde{v}_R=v_R, v R ≤ F c ( R ) ( L ( η ⋆ , u ⋆ , w ⋆ )) ≤ G R ( η ⋆ , u ⋆ , w ⋆ ) = v ~ R = v R ,
for every ϱ ∈ Π ( u ⋆ , w ⋆ ) \varrho\in\Pi(u^\star,w^\star) ϱ ∈ Π ( u ⋆ , w ⋆ ) , hence L ( η ⋆ , u ⋆ , w ⋆ ) \mathrm{L}(\eta^\star,u^\star,w^\star) L ( η ⋆ , u ⋆ , w ⋆ ) is a minimizer for F c ( R ) F_{c(R)} F c ( R ) .
Notice that Theorem 6 allows us to identify the set of minimizers of the two problems. As a straightforward consequence, we infer the following.
Corollary 2. Denoted by
P R : = { ( η , u , w ) ∈ F R : ∥ η ∥ 1 ≤ B 0 , ∥ u ∥ 1 ≤ B 0 } , (65) \mathcal{P}_R := \{(\eta,u,w) \in\mathcal{F}_R : \|\eta\|_1 \le B_0,\ \|u\|_1 \le B_0\}, \tag*{(65)} P R := {( η , u , w ) ∈ F R : ∥ η ∥ 1 ≤ B 0 , ∥ u ∥ 1 ≤ B 0 } , (65) we have v R = min ( η , u , w ) ∈ P R G R ( η , u , w ) v_R = \min_{(\eta,u,w)\in\mathcal{P}_R} G_R(\eta,u,w) v R = min ( η , u , w ) ∈ P R G R ( η , u , w ) .
Proof. By Proposition 2, there exists a π ∈ R + n × m \pi\in\mathbb{R}_+^{n \times m} π ∈ R + n × m such that F c ( R ) ( π ) = v R F_{c(R)}(\pi)=v_R F c ( R ) ( π ) = v R and ∥ π ∥ 1 ≤ B 0 \|\pi\|_1 \le B_0 ∥ π ∥ 1 ≤ B 0 . Set ( η , u , w ) = C ( π ) (\eta,u,w)=\mathrm{C}(\pi) ( η , u , w ) = C ( π ) . Owing to Theorem 6, C ( π ) \mathrm{C}(\pi) C ( π ) is optimal and ( η , u , w ) ∈ P R (\eta,u,w) \in\mathcal{P}_R ( η , u , w ) ∈ P R since
∥ η ∥ 1 + τ = ∑ E R π i j + ∑ E R c π i j = ∥ π ∥ 1 ≤ B 0 , \|\eta\|_1+\tau=\sum_{\mathcal{E}_R}\pi_{ij}+\sum_{\mathcal{E}_R^c}\pi_{ij}=\|\pi\|_1\le B_0, ∥ η ∥ 1 + τ = E R ∑ π ij + E R c ∑ π ij = ∥ π ∥ 1 ≤ B 0 , which allows us to conclude the proof.
Error Bounds
In this section, we study the UOT problem induced by a truncated cost function as an approximation term of the untruncated UOT problem. In particular, the final goal of the section is to characterize the error induced by approximating v C v_C v C with v R v_R v R .
Recovering plans for the original cost
As we saw in Section 4.1, given an element ( η , u , w ) ∈ F R (\eta,u,w)\in\mathcal{F}_R ( η , u , w ) ∈ F R . The function L \mathrm{L} L assigns every element of F R \mathcal{F}_R F R to a feasible plan π \pi π for the original problem and maps a minimizer of G R G_R G R to a minimizer of F c ( R ) F_{c(R)} F c ( R ) . We describe two additional recovery procedures that define a feasible point for the original UOT problem as in (3.17) starting from an element of F R \mathcal{F}_R F R . This will allow us to estimate the gap between v C v_C v C and v R v_R v R via three different error bounds.
Deleting the residuals. Given ( η , u , w ) ∈ F R (\eta,u,w)\in\mathcal{F}_R ( η , u , w ) ∈ F R , define
D ( η , u , w ) : = η . (66) \mathrm{D}(\eta,u,w):=\eta. \tag*{(66)} D ( η , u , w ) := η . (66)
So that D builds a feasible plan for the original transportation plan by discarding the residual variables ( u , w ) (u,w) ( u , w ) and keeping only the mass transported at distance smaller than R R R . Notice that if ϕ 0 ′ > − ∞ \phi'_0>-\infty ϕ 0 ′ > − ∞ , we must have ϕ ( 0 ) < + ∞ \phi(0)<+\infty ϕ ( 0 ) < + ∞ , D( η , u , w ) (\eta,u,w) ( η , u , w ) has finite marginal penalties, thus U T ϕ , c ( D ( η , u , w ) ) < + ∞ \mathrm{UT}_{\phi,c}(\mathrm{D}(\eta,u,w))<+\infty UT ϕ , c ( D ( η , u , w )) < + ∞ . Otherwise, U T ϕ , c ( D ( η , u , w ) ) \mathrm{UT}_{\phi,c}(\mathrm{D}(\eta,u,w)) UT ϕ , c ( D ( η , u , w )) could attain unbounded value.
Redirecting the residuals. Given ( η , u , w ) ∈ F R (\eta,u,w) \in\mathcal{F}_R ( η , u , w ) ∈ F R , define
B ( η , u , w ) = η + Q u + Q w , Q u : = ∑ i u i e i e j i T , and Q w : = ∑ j w j e j e j T . (67) \mathrm{B}(\eta,u,w) = \eta+ Q^u + Q^w,\qquad Q^u := \sum_i u_i e_i e_{j_i}^{\mathsf{T}},\qquad\text{and}\qquad Q^w := \sum_j w_j e_j e_{j}^{\mathsf{T}}. \tag*{(67)} B ( η , u , w ) = η + Q u + Q w , Q u := i ∑ u i e i e j i T , and Q w := j ∑ w j e j e j T . (67)
where j i ∈ argmin j ∈ [ m ] c i j j_i \in\operatorname*{argmin}_{j \in[m]} c_{ij} j i ∈ argmin j ∈ [ m ] c ij and l j ∈ argmin i ∈ [ n ] c i j l_j \in\operatorname*{argmin}_{i \in[n]} c_{ij} l j ∈ argmin i ∈ [ n ] c ij . Since both Q u Q^u Q u and Q w Q^w Q w are positive matrices, B ( η , u , w ) \mathrm{B}(\eta,u,w) B ( η , u , w ) is always feasible for (29) . Unlike D, this construction retains all the residuals and maps it to the closest point available to the source. Notice, however, that the marginals of B are different from the ones of L, indeed
P 1 B ( η , u , w ) = P 1 η + u + P 1 Q w , P 2 B ( η , u , w ) = P 2 η + w + P 2 Q u . (68) P_1\mathrm{B}(\eta,u,w)=P_1\eta+u+P_1Q^w,\qquad P_2\mathrm{B}(\eta,u,w)=P_2\eta+w+P_2Q^u. \tag*{(68)} P 1 B ( η , u , w ) = P 1 η + u + P 1 Q w , P 2 B ( η , u , w ) = P 2 η + w + P 2 Q u . (68)
Constructive Bounds
Having introduced our three reconstruction routines L, B, and D, we propose our estimation between v c v_c v c and v R v_R v R . First, we propose an a posteriori bound, which allow to quantify the gap between v R v_R v R and v c , R v_{c,R} v c , R through the solution to the problem on the right hand side of (61) .
Theorem 7. Let ( η , u , w ) ∈ F R (\eta,u,w) \in\mathcal{F}_R ( η , u , w ) ∈ F R be such that v R = G R ( η , u , w ) v_R=G_R(\eta,u,w) v R = G R ( η , u , w ) . Put c max = max i , j c i j c_{\max}=\max_{i,j}c_{ij} c m a x = max i , j c ij and set τ = ∥ u ∥ 1 \tau=\lVert u\rVert_1 τ = ∥ u ∥ 1 . Let us now consider
R L ( η , u , w ) : = ( c max − R ) + + τ , (69) \mathcal{R}_L(\eta,u,w):=(c_{\max}-R)_++\tau, \tag*{(69)} R L ( η , u , w ) := ( c m a x − R ) + + τ , (69) R D ( η , u , w ) : = ( − 2 ϕ 0 ′ − R ) + + τ if ϕ 0 ′ > − ∞ , (70) \mathcal{R}_D(\eta,u,w):=(-2\phi'_0-R)_++\tau\qquad\text{if }\phi'_0>-\infty, \tag*{(70)} R D ( η , u , w ) := ( − 2 ϕ 0 ′ − R ) + + τ if ϕ 0 ′ > − ∞ , (70) R B ( η , u , w ) : = [ ∑ i ( c i , min + ϕ ∞ ′ ) u i + ∑ j ( c min , j + ϕ ∞ ′ ) w j − R τ ] + if ϕ ∞ ′ < + ∞ (71) \mathcal{R}_B(\eta,u,w):=\left[\sum_i(c_{i,\min}+\phi'_\infty)u_i+\sum_j(c_{\min,j}+\phi'_\infty)w_j-R\tau\right]_+ \qquad\text{if }\phi'_\infty<+\infty \tag*{(71)} R B ( η , u , w ) := [ i ∑ ( c i , m i n + ϕ ∞ ′ ) u i + j ∑ ( c m i n , j + ϕ ∞ ′ ) w j − R τ ] + if ϕ ∞ ′ < + ∞ (71) where c min , j c_{\min,j} c m i n , j is the vector containing the minimum entry per each column and c i , min c_{i,\min} c i , m i n is the vector containing the minimum element for each row. Then, for each function X ∈ { L , B , D } X\in\{\mathrm{L},\mathrm{B},\mathrm{D}\} X ∈ { L , B , D } , we have
0 ≤ F c ( X ( η , u , w ) ) − v c ≤ F c ( X ( η , u , w ) ) − v R ≤ R X ( η , u , w ) . (72) 0\leq F_c(X(\eta,u,w))-v_c\leq F_c(X(\eta,u,w))-v_R\leq\mathcal{R}_X(\eta,u,w). \tag*{(72)} 0 ≤ F c ( X ( η , u , w )) − v c ≤ F c ( X ( η , u , w )) − v R ≤ R X ( η , u , w ) . (72) In particular, for any optimal ( η , u , w ) (\eta,u,w) ( η , u , w ) , we have
0 ≤ v c − v R ≤ min { R X ( η , u , w ) : X ∈ { L , B , D } } . (73) 0\leq v_c-v_R\leq\min\{\mathcal{R}_X(\eta,u,w):X\in\{\mathrm{L},\mathrm{B},\mathrm{D}\}\}. \tag*{(73)} 0 ≤ v c − v R ≤ min { R X ( η , u , w ) : X ∈ { L , B , D }} . (73) Proof. Let ( η , u , w ) (\eta,u,w) ( η , u , w ) be such that G R ( η , u , w ) = v R G_R(\eta,u,w)=v_R G R ( η , u , w ) = v R , we then set r = P 1 η r=P_1\eta r = P 1 η and s = P 2 η s=P_2\eta s = P 2 η . For every finite-valued feasible γ \gamma γ for problem (29) , we have
0 ≤ F c ( γ ) − v c ≤ F c ( γ ) − v R and 0 ≤ v c − v R ≤ F c ( γ ) − v R . (74) 0\leq F_c(\gamma)-v_c\leq F_c(\gamma)-v_R\qquad\text{and}\qquad0\leq v_c-v_R\leq F_c(\gamma)-v_R. \tag*{(74)} 0 ≤ F c ( γ ) − v c ≤ F c ( γ ) − v R and 0 ≤ v c − v R ≤ F c ( γ ) − v R . (74) We now prove the bound in (72) one function X ∈ { L , B , D } X\in\{\mathrm{L},\mathrm{B},\mathrm{D}\} X ∈ { L , B , D } at the time. The statement will then follow by taking the minimum over the three valid bounds.
Case one: X = L X=\mathrm{L} X = L . Let ϱ ∈ Π ( u , w ) \varrho\in\Pi(u,w) ϱ ∈ Π ( u , w ) , such that L ( η , w , u ) = η + ϱ \mathrm{L}(\eta,w,u)=\eta+\varrho L ( η , w , u ) = η + ϱ . Then, Lemma 1 gives ϱ ∣ E R = 0 \varrho|_{\mathcal{E}_R}=0 ϱ ∣ E R = 0 and F c ( η + ϱ ) = v R F_c(\eta+\varrho)=v_R F c ( η + ϱ ) = v R . Hence
F c ( η + ϱ ) − v R = F c ( η + ϱ ) − F c R ( η + ϱ ) F_c(\eta+\varrho)-v_R=F_c(\eta+\varrho)-F_{c_R}(\eta+\varrho) F c ( η + ϱ ) − v R = F c ( η + ϱ ) − F c R ( η + ϱ ) = ∑ ( i , j ) ∈ E R c ( c i j − R ) ϱ i j ≤ ( c max − R ) + + ∑ ( i , j ) ∈ E R c ϱ i j = ( c max − R ) + + τ . (75) =\sum_{(i,j)\in\mathcal{E}_R^c}(c_{ij}-R)\varrho_{ij}\leq(c_{\max}-R)_++\sum_{(i,j)\in\mathcal{E}_R^c}\varrho_{ij}=(c_{\max}-R)_++\tau. \tag*{(75)} = ( i , j ) ∈ E R c ∑ ( c ij − R ) ϱ ij ≤ ( c m a x − R ) + + ( i , j ) ∈ E R c ∑ ϱ ij = ( c m a x − R ) + + τ . (75) Notice that (4.25) implies that if τ = 0 \tau=0 τ = 0 , then v c = v R v_c=v_R v c = v R and that
min { R X ( η , u , w ) : X ∈ { L , B , D } } = 0 , \min\{\mathcal{R}_X(\eta,u,w):X\in\{\mathrm{L},\mathrm{B},\mathrm{D}\}\}=0, min { R X ( η , u , w ) : X ∈ { L , B , D }} = 0 ,
so that in what follows, we will assume τ > 0 \tau> 0 τ > 0 .
Case two: X = B \mathbf{X} = \mathbf{B} X = B . Assume that ϕ ∞ ′ < + ∞ \phi'_{\infty} < +\infty ϕ ∞ ′ < + ∞ as if ϕ ∞ ′ = + ∞ \phi'_{\infty} = +\infty ϕ ∞ ′ = + ∞ , we have R B < + ∞ \mathcal{R}_{B} < +\infty R B < + ∞ only if τ = 0 \tau= 0 τ = 0 , in which case the bound has been proven already. Let us set ρ = P 1 Q w \rho= \mathrm{P}_{1}Q^{w} ρ = P 1 Q w and σ = P 2 Q w \sigma= \mathrm{P}_{2}Q^{w} σ = P 2 Q w . Then, owing to (68)
φ μ i ( r i + u i + ρ i ) − φ μ i ( r i + u i ) ≤ ϕ ∞ ′ ρ i , and φ ν j ( s j + w j + σ j ) − φ ν j ( s j + w j ) ≤ ϕ ∞ ′ σ j . \varphi_{\mu_i}(r_i+u_i+\rho_i)-\varphi_{\mu_i}(r_i+u_i) \leq\phi'_{\infty}\rho_i,\qquad\text{and}\qquad\varphi_{\nu_j}(s_j+w_j+\sigma_j)-\varphi_{\nu_j}(s_j+w_j) \leq\phi'_{\infty}\sigma_j. φ μ i ( r i + u i + ρ i ) − φ μ i ( r i + u i ) ≤ ϕ ∞ ′ ρ i , and φ ν j ( s j + w j + σ j ) − φ ν j ( s j + w j ) ≤ ϕ ∞ ′ σ j .
The latter inequalities imply
F c ( B ( η , u , w ) ) − v R = ∑ i c i , min u i + ∑ j c min , j w j − R τ + ∑ i [ φ μ i ( r i + u i + ρ i ) − φ μ i ( r i + u i ) ] + ∑ j [ φ ν j ( s j + w j + σ j ) − φ ν j ( s j + w j ) ] ≤ ∑ i c i , min u i + ∑ j c min , j w j − R τ + ϕ ∞ ′ ∑ i ρ i + ϕ ∞ ′ ∑ j σ j = ∑ i ( c i , min + ϕ ∞ ′ ) u i + ∑ j ( c min , j + ϕ ∞ ′ ) w j − R τ ≤ R B ( η , u , w ) . \begin{aligned}
F_c(\mathrm{B}(\eta,u,w))-v_R
&=\sum_i c_{i,\min}u_i+\sum_j c_{\min,j}w_j-R\tau\\
&\quad+\sum_i\left[\varphi_{\mu_i}(r_i+u_i+\rho_i)-\varphi_{\mu_i}(r_i+u_i)\right]\\
&\quad+\sum_j\left[\varphi_{\nu_j}(s_j+w_j+\sigma_j)-\varphi_{\nu_j}(s_j+w_j)\right]\\
&\leq\sum_i c_{i,\min}u_i+\sum_j c_{\min,j}w_j-R\tau+\phi'_{\infty}\sum_i\rho_i+\phi'_{\infty}\sum_j\sigma_j\\
&=\sum_i(c_{i,\min}+\phi'_{\infty})u_i+\sum_j(c_{\min,j}+\phi'_{\infty})w_j-R\tau\leq\mathcal{R}_{B}(\eta,u,w).
\end{aligned} F c ( B ( η , u , w )) − v R = i ∑ c i , m i n u i + j ∑ c m i n , j w j − R τ + i ∑ [ φ μ i ( r i + u i + ρ i ) − φ μ i ( r i + u i ) ] + j ∑ [ φ ν j ( s j + w j + σ j ) − φ ν j ( s j + w j ) ] ≤ i ∑ c i , m i n u i + j ∑ c m i n , j w j − R τ + ϕ ∞ ′ i ∑ ρ i + ϕ ∞ ′ j ∑ σ j = i ∑ ( c i , m i n + ϕ ∞ ′ ) u i + j ∑ ( c m i n , j + ϕ ∞ ′ ) w j − R τ ≤ R B ( η , u , w ) .
Case three: X = D \mathbf{X} = \mathbf{D} X = D . Suppose ϕ 0 ′ > − ∞ \phi'_0 > -\infty ϕ 0 ′ > − ∞ as otherwise the bound trivially holds or τ = 0 \tau= 0 τ = 0 . We recall that for any m > 0 m > 0 m > 0 , x ≥ 0 x \geq0 x ≥ 0 , and t > 0 t > 0 t > 0 , the convexity of ϕ \phi ϕ gives
φ m ( x + t ) − φ m ( x ) t = ϕ ( ( x + t ) / m ) − ϕ ( x / m ) t / m ≥ ϕ 0 ′ . \frac{\varphi_m(x+t)-\varphi_m(x)}{t}=\frac{\phi((x+t)/m)-\phi(x/m)}{t/m}\geq\phi'_0. t φ m ( x + t ) − φ m ( x ) = t / m ϕ (( x + t ) / m ) − ϕ ( x / m ) ≥ ϕ 0 ′ .
Since φ 0 \varphi_0 φ 0 is linear when ϕ ∞ ′ < + ∞ \phi'_{\infty}<+\infty ϕ ∞ ′ < + ∞ the quotient is ϕ ∞ ′ \phi'_{\infty} ϕ ∞ ′ . Thus, since ϕ 0 ′ ≤ ϕ ∞ ′ \phi'_0\leq\phi'_{\infty} ϕ 0 ′ ≤ ϕ ∞ ′ for every m ≥ 0 m\geq0 m ≥ 0 we have
φ m ( x ) − φ m ( x + t ) ≤ − ϕ 0 ′ t ( x , t ≥ 0 ) . (76) \varphi_m(x)-\varphi_m(x+t)\leq-\phi'_0t\qquad(x,t\geq0). \tag*{(76)} φ m ( x ) − φ m ( x + t ) ≤ − ϕ 0 ′ t ( x , t ≥ 0 ) . (76)
It follows that
F c ( η ) − v R = − R τ + ∑ i [ φ μ i ( r i ) − φ μ i ( r i + u i ) ] + ∑ j [ φ ν j ( s j ) − φ ν j ( s j + w j ) ] ≤ − R τ − ϕ 0 ′ ∑ i u i − ϕ 0 ′ ∑ j w j = ( − 2 ϕ 0 ′ − R ) τ ≤ ( − 2 ϕ 0 ′ − R ) + τ , \begin{aligned}
F_c(\eta)-v_R
&=-R\tau+\sum_i\left[\varphi_{\mu_i}(r_i)-\varphi_{\mu_i}(r_i+u_i)\right]+\sum_j\left[\varphi_{\nu_j}(s_j)-\varphi_{\nu_j}(s_j+w_j)\right]\\
&\leq-R\tau-\phi'_0\sum_i u_i-\phi'_0\sum_j w_j=(-2\phi'_0-R)\tau\leq(-2\phi'_0-R)_+\tau,
\end{aligned} F c ( η ) − v R = − R τ + i ∑ [ φ μ i ( r i ) − φ μ i ( r i + u i ) ] + j ∑ [ φ ν j ( s j ) − φ ν j ( s j + w j ) ] ≤ − R τ − ϕ 0 ′ i ∑ u i − ϕ 0 ′ j ∑ w j = ( − 2 ϕ 0 ′ − R ) τ ≤ ( − 2 ϕ 0 ′ − R ) + τ ,
which concludes the proof.
Notice that the bounds presented in Theorem 7 are a posteriori, meaning that the guarantees is quantifiable only after we have access to the optimal solution ( η , u , w ) (\eta,u,w) ( η , u , w ) . In the remainder of the section, we exploit the properties of the problem to make the bounds a priori, i.e., computable without having access to an optimal solution to the truncated UOT problem. As a direct consequence of the latter Theorem, we infer the following conditions under which the optimal value of the two problems coincide v c = v R v_c=v_R v c = v R and the minima of the truncated problem induce a minimizer for (3.17).
Corollary 3 (Exact recovery). If ( η , u , w ) (\eta,u,w) ( η , u , w ) is optimal for the truncated problem and ∥ u ∥ 1 = 0 \lVert u\rVert_1=0 ∥ u ∥ 1 = 0 , then π = η \pi=\eta π = η is a solution to (3.17) and v R = v c v_R=v_c v R = v c . Moreover, we have
if R > c max R>c_{\max} R > c m a x , then π = η \pi=\eta π = η is a solution to (3.17) and v R = v c v_R=v_c v R = v c ;
if ϕ 0 ′ > − ∞ \phi'_0>-\infty ϕ 0 ′ > − ∞ and R ≥ − 2 ϕ 0 ′ R\geq-2\phi'_0 R ≥ − 2 ϕ 0 ′ , then π = η \pi=\eta π = η is a solution to (3.17) and v R = v c v_R=v_c v R = v c ; and
(iii) if ϕ ∞ ′ < + ∞ \phi^{\prime}_{\infty}<+\infty ϕ ∞ ′ < + ∞ and R ≥ max i c i , min + max j c min , j + 2 ϕ ∞ ′ R\ge\max_i c_{i,\min}+\max_j c_{\min,j}+2\phi^{\prime}_{\infty} R ≥ max i c i , m i n + max j c m i n , j + 2 ϕ ∞ ′ , then π = η + Q u + Q w \pi=\eta+Q^u+Q^w π = η + Q u + Q w is a solution to (3.17) and v R = v c v_R=v_c v R = v c . If c min , j = c i , min = 0 c_{\min,j}=c_{i,\min}=0 c m i n , j = c i , m i n = 0 for every i ∈ [ n ] i\in[n] i ∈ [ n ] and every j ∈ [ m ] j\in[m] j ∈ [ m ] we have R ≥ 2 ϕ ∞ ′ R\ge2\phi^{\prime}_{\infty} R ≥ 2 ϕ ∞ ′ .
Proof. The first part of the statement trivially follows from the fact that R L ( η , u , w ) = 0 \mathcal{R}_{\mathrm{L}}(\eta,u,w)=0 R L ( η , u , w ) = 0 whenever u u u and w w w are both null vectors. For the other assertions, we have that for every ( η , u , w ) (\eta,u,w) ( η , u , w ) it holds
R ≥ c max ⟹ R L ( η , u , w ) = 0 , R ≥ − 2 ϕ 0 ′ ⟹ R D ( η , u , w ) = 0 , R\ge c_{\max}\quad\Longrightarrow\quad\mathcal{R}_{\mathrm{L}}(\eta,u,w)=0,\qquad
R\ge-2\phi^{\prime}_{0}\quad\Longrightarrow\quad\mathcal{R}_{\mathrm{D}}(\eta,u,w)=0, R ≥ c m a x ⟹ R L ( η , u , w ) = 0 , R ≥ − 2 ϕ 0 ′ ⟹ R D ( η , u , w ) = 0 ,
R ≥ max i c i , min + max j c min , j + 2 ϕ ∞ ′ ⟹ R B ( η , u , w ) = 0 R\ge\max_i c_{i,\min}+\max_j c_{\min,j}+2\phi^{\prime}_{\infty}
\quad\Longrightarrow\quad\mathcal{R}_{\mathrm{B}}(\eta,u,w)=0 R ≥ i max c i , m i n + j max c m i n , j + 2 ϕ ∞ ′ ⟹ R B ( η , u , w ) = 0
since
∑ i ( c i , min + ϕ ∞ ′ ) u i + ∑ j ( c min , j + ϕ ∞ ′ ) w j − R τ ≤ ( max i c i , min + max j c min , j + 2 ϕ ∞ ′ − R ) + τ . \sum_i(c_{i,\min}+\phi^{\prime}_{\infty})u_i+\sum_j(c_{\min,j}+\phi^{\prime}_{\infty})w_j-R\tau
\le(\max_i c_{i,\min}+\max_j c_{\min,j}+2\phi^{\prime}_{\infty}-R)+\tau. i ∑ ( c i , m i n + ϕ ∞ ′ ) u i + j ∑ ( c m i n , j + ϕ ∞ ′ ) w j − R τ ≤ ( i max c i , m i n + j max c m i n , j + 2 ϕ ∞ ′ − R ) + τ .
We now show that even when the hypothesis of Corollary 3 do not hold, we are able to give an upper bound on the error between the truncated and untruncated problems. First, we introduce a Lemma that bounds the total mass of the minimizer for the truncated problem.
Lemma 2. Let π \pi π be a transportation plan that minimizes F c ( R ) F_{c(R)} F c ( R ) , then π i j ≤ T i j ( R ) \pi_{ij}\le T_{ij}(R) π ij ≤ T ij ( R ) for every ( i , j ) ∈ E R c (i,j)\in\mathcal{E}^{c}_{R} ( i , j ) ∈ E R c , where T i j T_{ij} T ij is defined as in (3.20). In particular, for every R > 0 R>0 R > 0 , let ( η , u , w ) ∈ argmin G R (\eta,u,w)\in\operatorname*{argmin}G_R ( η , u , w ) ∈ argmin G R , then we have
u i ≤ ∑ j : ( i , j ) ∈ E R c T i j ( R ) , w j ≤ ∑ i : ( i , j ) ∈ E R c T i j ( R ) , τ ≤ A R : = ∑ ( i , j ) ∈ E R c T i j ( R ) . (77) u_i\le\sum_{j:(i,j)\in\mathcal{E}^{c}_{R}}T_{ij}(R),\qquad
w_j\le\sum_{i:(i,j)\in\mathcal{E}^{c}_{R}}T_{ij}(R),\qquad
\tau\le A_R:=\sum_{(i,j)\in\mathcal{E}^{c}_{R}}T_{ij}(R). \tag*{(77)} u i ≤ j : ( i , j ) ∈ E R c ∑ T ij ( R ) , w j ≤ i : ( i , j ) ∈ E R c ∑ T ij ( R ) , τ ≤ A R := ( i , j ) ∈ E R c ∑ T ij ( R ) . (77) Proof. Theorem 3 gives π i j ≤ T i j ( c i j ( R ) ) = T i j ( R ) \pi_{ij}\le T_{ij}(c^{(R)}_{ij})=T_{ij}(R) π ij ≤ T ij ( c ij ( R ) ) = T ij ( R ) for every ( i , j ) ∈ E R c (i,j)\in\mathcal{E}^{c}_{R} ( i , j ) ∈ E R c . Let ( η , u , w ) ∈ F R (\eta,u,w)\in\mathcal{F}_R ( η , u , w ) ∈ F R be a minimizer of G R G_R G R and let π = L ( η , u , w ) = η + ρ \pi=\mathrm{L}(\eta,u,w)=\eta+\rho π = L ( η , u , w ) = η + ρ . Then we have η ∣ E R c = 0 \eta|_{\mathcal{E}^{c}_{R}}=0 η ∣ E R c = 0 , hence
u i = ∑ j : ( i , j ) ∈ E R c ρ i j = ∑ j : ( i , j ) ∈ E R c π i j ≤ ∑ j : ( i , j ) ∈ E R c T i j ( R ) , τ = ∑ ( i , j ) ∈ E R c π i j ≤ ∑ ( i , j ) ∈ E R c T i j ( R ) = A R u_i=\sum_{j:(i,j)\in\mathcal{E}^{c}_{R}}\rho_{ij}
=\sum_{j:(i,j)\in\mathcal{E}^{c}_{R}}\pi_{ij}
\le\sum_{j:(i,j)\in\mathcal{E}^{c}_{R}}T_{ij}(R),\qquad
\tau=\sum_{(i,j)\in\mathcal{E}^{c}_{R}}\pi_{ij}
\le\sum_{(i,j)\in\mathcal{E}^{c}_{R}}T_{ij}(R)=A_R u i = j : ( i , j ) ∈ E R c ∑ ρ ij = j : ( i , j ) ∈ E R c ∑ π ij ≤ j : ( i , j ) ∈ E R c ∑ T ij ( R ) , τ = ( i , j ) ∈ E R c ∑ π ij ≤ ( i , j ) ∈ E R c ∑ T ij ( R ) = A R a similar argument yields the bound on w j w_j w j .
Finally, we present our a priori bounds on the error on the gap between v c v_c v c and v R v_R v R .
Theorem 8. Define A R : = ∑ ( i , j ) ∈ E R c T i j ( R ) A_R:=\sum_{(i,j)\in\mathcal{E}^{c}_{R}}T_{ij}(R) A R := ∑ ( i , j ) ∈ E R c T ij ( R ) , then
R L ( R ) : = ∑ ( i , j ) ∈ E R c ( c i j − R ) T i j ( R ) , (78) \mathcal{R}_{\mathrm{L}}(R):=
\sum_{(i,j)\in\mathcal{E}^{c}_{R}}(c_{ij}-R)T_{ij}(R), \tag*{(78)} R L ( R ) := ( i , j ) ∈ E R c ∑ ( c ij − R ) T ij ( R ) , (78) R D ( R ) : = ( − 2 ϕ 0 ′ − R ) + A R if ϕ 0 ′ > − ∞ , (79) \mathcal{R}_{\mathrm{D}}(R):=(-2\phi^{\prime}_{0}-R)+A_R
\qquad\text{if }\phi^{\prime}_{0}>-\infty, \tag*{(79)} R D ( R ) := ( − 2 ϕ 0 ′ − R ) + A R if ϕ 0 ′ > − ∞ , (79) R B ( R ) : = ∑ ( i , j ) ∈ E R c ( c i , min + c min , j + 2 ϕ ∞ ′ − R ) + T i j ( R ) if ϕ ∞ ′ < + ∞ . (80) \mathcal{R}_{\mathrm{B}}(R):=
\sum_{(i,j)\in\mathcal{E}^{c}_{R}}(c_{i,\min}+c_{\min,j}+2\phi^{\prime}_{\infty}-R)+T_{ij}(R)
\qquad\text{if }\phi^{\prime}_{\infty}<+\infty. \tag*{(80)} R B ( R ) := ( i , j ) ∈ E R c ∑ ( c i , m i n + c m i n , j + 2 ϕ ∞ ′ − R ) + T ij ( R ) if ϕ ∞ ′ < + ∞. (80) In particular, 0 ≤ v c − v R ≤ min { R X ( R ) : X ∈ { L , B , D } } 0\le v_c-v_R\le\min\{\mathcal{R}_{\mathrm{X}}(R):X\in\{\mathrm{L},\mathrm{B},\mathrm{D}\}\} 0 ≤ v c − v R ≤ min { R X ( R ) : X ∈ { L , B , D }} .
Proof. We prove the estimates in (4.28), (4.29), and (4.30) one by one, the last statement of the theorem follows by taking the minimum over the three alternatives. The first two estimates follow by combining the estimates in (4.19) and (4.20) with (4.27).
We then need to prove only (4.30). Notice that given an optimal ( η , u , w ) (\eta,u,w) ( η , u , w ) denoted by L ( η , u , w ) = η + ϱ L(\eta,u,w)=\eta+\varrho L ( η , u , w ) = η + ϱ , we have
[ ∑ i ( c i , min + ϕ ∞ ′ ) u i + ∑ j ( c min , j + ϕ ∞ ′ ) w j − R τ ] + = [ ∑ ( i , j ) ∈ E R c ( c i , min + c min , j + 2 ϕ ∞ ′ − R ) ϱ i j ] + ≤ ∑ ( i , j ) ∈ E R c ( c i , min + c min , j + 2 ϕ ∞ ′ − R ) ϱ i j , \begin{aligned}
\left[\sum_i(c_{i,\min}+\phi'_{\infty})u_i+\sum_j(c_{\min,j}+\phi'_{\infty})w_j-R\tau\right]_+
=
\left[\sum_{(i,j)\in\mathcal{E}^c_R}(c_{i,\min}+c_{\min,j}+2\phi'_{\infty}-R)\varrho_{ij}\right]_+ \\
&\leq\sum_{(i,j)\in\mathcal{E}^c_R}(c_{i,\min}+c_{\min,j}+2\phi'_{\infty}-R)\varrho_{ij},
\end{aligned} [ i ∑ ( c i , m i n + ϕ ∞ ′ ) u i + j ∑ ( c m i n , j + ϕ ∞ ′ ) w j − R τ ] + = ( i , j ) ∈ E R c ∑ ( c i , m i n + c m i n , j + 2 ϕ ∞ ′ − R ) ϱ ij + ≤ ( i , j ) ∈ E R c ∑ ( c i , m i n + c m i n , j + 2 ϕ ∞ ′ − R ) ϱ ij ,
which, combined with ϱ i j ≤ T i j ( R ) \varrho_{ij}\leq T_{ij}(R) ϱ ij ≤ T ij ( R ) for ( i , j ) ∈ E R c (i,j)\in\mathcal{E}^c_R ( i , j ) ∈ E R c concludes the proof. □
UOT Induced by Entropy-like Penalties
In this section, we specialize our discussion to the case in which ϕ \phi ϕ is one of the powerlike entropies considered in [36 ] . Moreover, we assume that X = Y ⊂ R n \mathcal{X}=\mathcal{Y}\subset\mathbb{R}^n X = Y ⊂ R n are discrete and finite and c ( x , y ) = h ( ∥ x − y ∥ ) c(x,y)=h(\lVert x-y\rVert) c ( x , y ) = h (∥ x − y ∥) with h : R + → R + h:\mathbb{R}_+\to\mathbb{R}_+ h : R + → R + a convex, increasing function such that h ( t ) = 0 h(t)=0 h ( t ) = 0 if and only if t = 0 t=0 t = 0 . Notice that, owing to Remark 2, the support of the optimal transportation plan is always contained in ( A ∪ B ) × ( A ∪ B ) (\mathcal{A}\cup\mathcal{B})\times(\mathcal{A}\cup\mathcal{B}) ( A ∪ B ) × ( A ∪ B ) since c i , min = c min , j = 0 c_{i,\min}=c_{\min,j}=0 c i , m i n = c m i n , j = 0 and h ( t ) > 0 h(t)>0 h ( t ) > 0 if t > 0 t>0 t > 0 .
Powerlike Entropies and Their Properties
Given p ∈ R p\in\mathbb{R} p ∈ R , the powerlike entropy ψ p \psi_p ψ p associated with p p p is defined as
ψ p ( t ) = { t − 1 − log t if p = 0 , t log t − t + 1 if p = 1 , 1 p ( p − 1 ) ( t p − p ( t − 1 ) − 1 ) otherwise dom ( ψ p ) = { [ 0 , + ∞ [ if p > 0 , ] 0 , + ∞ [ if p ≤ 0 \psi_p(t)=
\begin{cases}
t-1-\log t & \text{if }p=0,\\
t\log t-t+1 & \text{if }p=1,\\
\frac{1}{p(p-1)}\left(t^p-p(t-1)-1\right) & \text{otherwise}
\end{cases}
\qquad
\operatorname{dom}(\psi_p)=
\begin{cases}
[0,+\infty[ & \text{if }p>0,\\
]0,+\infty[ & \text{if }p\leq0
\end{cases} ψ p ( t ) = ⎩ ⎨ ⎧ t − 1 − log t t log t − t + 1 p ( p − 1 ) 1 ( t p − p ( t − 1 ) − 1 ) if p = 0 , if p = 1 , otherwise dom ( ψ p ) = { [ 0 , + ∞ [ ] 0 , + ∞ [ if p > 0 , if p ≤ 0
Moreover, since every ψ p \psi_p ψ p is differentiable on R + + \mathbb{R}_{++} R ++ , we have
∂ ψ p ( t ) = { { ψ p ′ ( t ) } if t > 0 , ] − ∞ , ψ p , 0 ′ ] if t = 0 and p > 1 , dom ( ∂ ψ p ) = { [ 0 , + ∞ [ if p > 1 , ] 0 , + ∞ [ if p ≤ 1 \partial\psi_p(t)=
\begin{cases}
\{\psi'_p(t)\} & \text{if }t>0,\\
]-\infty,\psi'_{p,0}] & \text{if }t=0\text{ and }p>1,
\end{cases}
\qquad
\operatorname{dom}(\partial\psi_p)=
\begin{cases}
[0,+\infty[ & \text{if }p>1,\\
]0,+\infty[ & \text{if }p\leq1
\end{cases} ∂ ψ p ( t ) = { { ψ p ′ ( t )} ] − ∞ , ψ p , 0 ′ ] if t > 0 , if t = 0 and p > 1 , dom ( ∂ ψ p ) = { [ 0 , + ∞ [ ] 0 , + ∞ [ if p > 1 , if p ≤ 1
where ψ p , 0 ′ = ( ψ p ′ ) + ( 0 ) \psi'_{p,0}=(\psi'_p)_+(0) ψ p , 0 ′ = ( ψ p ′ ) + ( 0 ) is the right derivative of ψ p \psi_p ψ p in 0 0 0 and
( ∀ t > 0 ) ψ p ′ ( t ) = { 1 p − 1 ( t p − 1 − 1 ) if p ∈ R \ { 0 , 1 } , log t if p = 1 , 1 − t − 1 if p = 0 (\forall t>0)\qquad
\psi'_p(t)=
\begin{cases}
\frac{1}{p-1}(t^{p-1}-1) & \text{if }p\in\mathbb{R}\backslash\{0,1\},\\
\log t & \text{if }p=1,\\
1-t^{-1} & \text{if }p=0
\end{cases} ( ∀ t > 0 ) ψ p ′ ( t ) = ⎩ ⎨ ⎧ p − 1 1 ( t p − 1 − 1 ) log t 1 − t − 1 if p ∈ R \ { 0 , 1 } , if p = 1 , if p = 0
Note, in addition that
ψ p , 0 ′ = { − ∞ if p ≤ 1 , 1 1 − p if p > 1 and ψ p , ∞ ′ = { 1 1 − p if p < 1 , + ∞ if p ≥ 1. \psi'_{p,0}=
\begin{cases}
-\infty& \text{if }p\leq1,\\
\frac{1}{1-p} & \text{if }p>1
\end{cases}
\qquad\text{and}\qquad
\psi'_{p,\infty}=
\begin{cases}
\frac{1}{1-p} & \text{if }p<1,\\
+\infty& \text{if }p\geq1.
\end{cases} ψ p , 0 ′ = { − ∞ 1 − p 1 if p ≤ 1 , if p > 1 and ψ p , ∞ ′ = { 1 − p 1 + ∞ if p < 1 , if p ≥ 1.
Proposition 3. The powerlike entropy functions ψ p \psi_p ψ p satisfy the following properties:
(i) For every 0 < L < M 0<L<M 0 < L < M , ψ p ′ \psi'_p ψ p ′ is Lipschitz continuous on [ L , M ] [L,M] [ L , M ] . Moreover, ( ψ p ′ ) + (\psi'_p)_+ ( ψ p ′ ) + is Lipschitz continuous on [ 0 , M ] [0,M] [ 0 , M ] with constant M p − 2 M^{p-2} M p − 2 if p ≥ 2 p\geq2 p ≥ 2 , and ( p − 1 ) (p-1) ( p − 1 ) -Hölder continuous with constant ( p − 1 ) − 1 (p-1)^{-1} ( p − 1 ) − 1 if $1<p<2. (ii) For p < 2 p<2 p < 2 , ψ p \psi_p ψ p is 2 2 2 -uniformly convex on [ L , M ] [L,M] [ L , M ] whenever 0 < L < M 0<L<M 0 < L < M . If p > 2 p>2 p > 2 , ψ p \psi_p ψ p is p p p -uniformly convex on [ L , ∞ ) [L,\infty) [ L , ∞ ) for every L > 0 L>0 L > 0 .
(iii) ψ 2 \psi_2 ψ 2 is quadratic, with Lipschitz derivative, and is 2 2 2 -uniformly convex on [ 0 , ∞ ) [0,\infty) [ 0 , ∞ ) .
In particular, every ψ p \psi_p ψ p has Lipschitz derivative and is uniformly convex on each compact interval [ L , M ] ⊂ ( 0 , ∞ ) [L,M]\subset(0,\infty) [ L , M ] ⊂ ( 0 , ∞ ) .
Proof. The statements follow directly from the formulas for ψ p ′ \psi_p' ψ p ′ and ψ p ′ ′ \psi_p'' ψ p ′′ .
Truncated UOT with Powerlike Entropies
Denoting by D p \mathrm{D}_p D p the Csiszár divergence induced by ψ p \psi_p ψ p and by F p , R \mathcal{F}_{p,R} F p , R the set of feasible triplets ( η , u , w ) (\eta,u,w) ( η , u , w ) as in (4.5), we set
v p : = min π i j ≥ 0 U T p , c ( π ) and v p , R : = min F p , R G R ( η , u , w ) (81) v_p := \min_{\pi_{ij}\geq0}\mathrm{UT}_{p,c}(\pi) \qquad\text{and}\qquad v_{p,R}:=\min_{\mathcal{F}_{p,R}}G_R(\eta,u,w) \tag*{(81)} v p := π ij ≥ 0 min UT p , c ( π ) and v p , R := F p , R min G R ( η , u , w ) (81)
where
U T p , c ( π ) = ⟨ c , π ⟩ + D p ( π ( 1 ) ∣ μ ) + D p ( π ( 2 ) ∣ ν ) \mathrm{UT}_{p,c}(\pi)=\langle c,\pi\rangle+\mathrm{D}_p(\pi^{(1)}\mid\mu)+\mathrm{D}_p(\pi^{(2)}\mid\nu) UT p , c ( π ) = ⟨ c , π ⟩ + D p ( π ( 1 ) ∣ μ ) + D p ( π ( 2 ) ∣ ν )
G R ( η , u , w ) = ⟨ c , η ⟩ + R ∥ u ∥ 1 + D p ( P 1 η + u ∣ μ ) + D p ( P 2 η + w ∣ ν ) . G_R(\eta,u,w)=\langle c,\eta\rangle+R\lVert u\rVert_1+\mathrm{D}_p(\mathrm{P}_1\eta+u\mid\mu)+\mathrm{D}_p(\mathrm{P}_2\eta+w\mid\nu). G R ( η , u , w ) = ⟨ c , η ⟩ + R ∥ u ∥ 1 + D p ( P 1 η + u ∣ μ ) + D p ( P 2 η + w ∣ ν ) .
Following the techniques proposed in [7 ] , it is possible to show that the properties of ψ p \psi_p ψ p readily translate to properties over the objective function G R G_R G R . Denoted by F p , R ∗ \mathcal{F}_{p,R}^* F p , R ∗ the set of minimizers of the truncated problem on the right hand side of (5.1), we have the following.
Theorem 9. Let π \pi π be a solution to the UOT problem induced by D p \mathrm{D}_p D p . Then,
π i j ≤ T i j ( p ) ( c i j ) , where T i j ( p ) ( t ) : = { μ i ν j ( ( μ i p − 1 ) + ( ν j p − 1 ) ) 1 p − 1 ( 2 − ( p − 1 ) t ) + 1 p − 1 if p > 1 , μ i ν j e − t 2 if p = 1 , ( μ i 1 − p ν j 1 − p 2 + ( 1 − p ) t ) 1 1 − p otherwise. (82) \pi_{ij}\leq T_{ij}^{(p)}(c_{ij}), \qquad\text{where}\qquad T_{ij}^{(p)}(t):=
\begin{cases}
\frac{\mu_i\nu_j}{\left((\mu_i^{p-1})+(\nu_j^{p-1})\right)^{\frac{1}{p-1}}}\left(2-(p-1)t\right)_{+}^{\frac{1}{p-1}} & \text{if }p>1,\\[6pt]
\sqrt{\mu_i\nu_j}e^{-\frac{t}{2}} & \text{if }p=1,\\[6pt]
\left(\frac{\mu_i^{1-p}\nu_j^{1-p}}{2+(1-p)t}\right)^{\frac{1}{1-p}} & \text{otherwise.}
\end{cases} \tag*{(82)} π ij ≤ T ij ( p ) ( c ij ) , where T ij ( p ) ( t ) := ⎩ ⎨ ⎧ ( ( μ i p − 1 ) + ( ν j p − 1 ) ) p − 1 1 μ i ν j ( 2 − ( p − 1 ) t ) + p − 1 1 μ i ν j e − 2 t ( 2 + ( 1 − p ) t μ i 1 − p ν j 1 − p ) 1 − p 1 if p > 1 , if p = 1 , otherwise. (82) Moreover, if p ≠ 1 p\ne1 p = 1 , then π i j = 0 \pi_{ij}=0 π ij = 0 if c i j > 2 ∣ 1 − p ∣ c_{ij}>\frac{2}{|1-p|} c ij > ∣1 − p ∣ 2 .
Proof. First, we observe that if π i j = 0 \pi_{ij}=0 π ij = 0 all the estimates are trivial, thus we will tacitly assume that π i j > 0 \pi_{ij}>0 π ij > 0 . Second, we notice that ψ p , 0 ′ = 1 1 − p \psi_{p,0}'=\frac{1}{1-p} ψ p , 0 ′ = 1 − p 1 if p > 1 p>1 p > 1 and ψ p , 0 ′ = − ∞ \psi_{p,0}'=-\infty ψ p , 0 ′ = − ∞ otherwise. Likewise ψ p , ∞ ′ = 1 1 − p \psi_{p,\infty}'=\frac{1}{1-p} ψ p , ∞ ′ = 1 − p 1 if p < 1 p<1 p < 1 and ψ p , ∞ ′ = ∞ \psi_{p,\infty}'=\infty ψ p , ∞ ′ = ∞ otherwise. Thus, owing to Theorem 4, gives (i) π i j = 0 \pi_{ij}=0 π ij = 0 whenever c i j ≥ 2 p − 1 c_{ij}\geq\frac{2}{p-1} c ij ≥ p − 1 2 if p > 1 p>1 p > 1 ; and (ii) π i j = 0 \pi_{ij}=0 π ij = 0 if c i j > 2 1 − p c_{ij}>\frac{2}{1-p} c ij > 1 − p 2 if p < 1 p<1 p < 1 .
Notice that ψ p \psi_p ψ p is differentiable on R + + \mathbb{R}_{++} R ++ and ( ψ p ′ ) + ( t ) = ( ψ p ′ ) − ( t ) = ψ p ′ ( t ) (\psi_p')_+(t)=(\psi_p')_-(t)=\psi_p'(t) ( ψ p ′ ) + ( t ) = ( ψ p ′ ) − ( t ) = ψ p ′ ( t ) with
( ∀ t ∈ R + + ) ψ p ′ ( t ) = { 1 p − 1 ( t p − 1 − 1 ) if p ≠ 1 , log ( t ) otherwise. (83) (\forall t\in\mathbb{R}_{++})\qquad
\psi_p'(t)=
\begin{cases}
\frac{1}{p-1}(t^{p-1}-1) & \text{if }p\ne1,\\
\log(t) & \text{otherwise.}
\end{cases} \tag*{(83)} ( ∀ t ∈ R ++ ) ψ p ′ ( t ) = { p − 1 1 ( t p − 1 − 1 ) log ( t ) if p = 1 , otherwise. (83) Following Theorem 3, we compute the function T i j T_{ij} T ij associated with all the ψ p \psi_p ψ p .
First, we consider the case p > 1 p>1 p > 1 . Since ( ψ p ′ ) ∞ = ∞ (\psi_p')_\infty=\infty ( ψ p ′ ) ∞ = ∞ hence, we assume μ i , ν j > 0 \mu_i,\nu_j>0 μ i , ν j > 0 . A direct computation shows that T i j ( t ) = s T_{ij}(t)=s T ij ( t ) = s if and only if
− t = 1 p − 1 ( s p − 1 μ i p − 1 + s p − 1 ν j p − 1 ) − 2 p − 1 ⟺ s p − 1 = ( μ i ν j ) p − 1 μ i p − 1 + ν j p − 1 ( 2 − ( p − 1 ) t ) (84) -t=\frac{1}{p-1}\left(\frac{s^{p-1}}{\mu_i^{p-1}}+\frac{s^{p-1}}{\nu_j^{p-1}}\right)-\frac{2}{p-1}
\quad\Longleftrightarrow\quad
s^{p-1}=\frac{(\mu_i\nu_j)^{p-1}}{\mu_i^{p-1}+\nu_j^{p-1}}\left(2-(p-1)t\right) \tag*{(84)} − t = p − 1 1 ( μ i p − 1 s p − 1 + ν j p − 1 s p − 1 ) − p − 1 2 ⟺ s p − 1 = μ i p − 1 + ν j p − 1 ( μ i ν j ) p − 1 ( 2 − ( p − 1 ) t ) (84)
which allows us to conclude the first part of the proof. Let us now consider the case p < 1 p < 1 p < 1 . Owing to Corollary 1, we have that π i j = 0 \pi_{ij}=0 π ij = 0 if c i j > 2 1 − p c_{ij} > \frac{2}{1-p} c ij > 1 − p 2 . Let us then consider the case in which c i j ≤ 2 1 − p c_{ij} \le\frac{2}{1-p} c ij ≤ 1 − p 2 . In this case, if μ i , ν j > 0 \mu_i,\nu_j > 0 μ i , ν j > 0 , we have T i j ( t ) = s T_{ij}(t)=s T ij ( t ) = s if and only if
− t = 2 1 − p − 1 1 − p ( μ i 1 − p + ν j 1 − p s 1 − p ) ⟺ s 1 − p = μ i 1 − p + ν j 1 − p ( 2 − ( p − 1 ) t ) . (85) -t = \frac{2}{1-p} - \frac{1}{1-p}\left(\frac{\mu_i^{1-p}+\nu_j^{1-p}}{s^{1-p}}\right) \Longleftrightarrow s^{1-p} = \frac{\mu_i^{1-p}+\nu_j^{1-p}}{(2-(p-1)t)}. \tag*{(85)} − t = 1 − p 2 − 1 − p 1 ( s 1 − p μ i 1 − p + ν j 1 − p ) ⟺ s 1 − p = ( 2 − ( p − 1 ) t ) μ i 1 − p + ν j 1 − p . (85)
Via a similar computation we handle the case in which either μ i = 0 \mu_i=0 μ i = 0 or ν j = 0 \nu_j=0 ν j = 0 . Finally, if p = 1 p=1 p = 1 , we have
− t = log ( s / μ i ) + log ( s / ν j ) ⟺ s = μ i ν j e − t / 2 . (86) -t = \log(s/\mu_i)+\log(s/\nu_j) \Longleftrightarrow s=\sqrt{\mu_i\nu_j}e^{-t/2}. \tag*{(86)} − t = log ( s / μ i ) + log ( s / ν j ) ⟺ s = μ i ν j e − t /2 . (86)
For all the cases, the proof follows by setting t = c i j t=c_{ij} t = c ij . □
The bound on the entries of an optimal transportation plan π i j \pi_{ij} π ij can be easily translated in bounds on the error between the objective v p , R v_{p,R} v p , R and v p v_p v p .
Corollary 4. Given p ∈ R p\in\mathbb{R} p ∈ R , we have that
v p − v p , R ≤ { O ( ( 2 p − 1 − R ) p p − 1 ) if p > 1 , O ( e − R / 2 ) if p = 1 , O ( R − 1 1 − p ) otherwise. (87) v_p-v_{p,R}\le
\begin{cases}
\mathcal{O}\left(\left(\frac{2}{p-1}-R\right)^{\frac{p}{p-1}}\right) & \text{if } p>1,\\
\mathcal{O}(e^{-R/2}) & \text{if } p=1,\\
\mathcal{O}\left(R^{-\frac{1}{1-p}}\right) & \text{otherwise.}
\end{cases} \tag*{(87)} v p − v p , R ≤ ⎩ ⎨ ⎧ O ( ( p − 1 2 − R ) p − 1 p ) O ( e − R /2 ) O ( R − 1 − p 1 ) if p > 1 , if p = 1 , otherwise. (87) Proof. It follows by summing over E R c \mathcal{E}_R^c E R c the bounds presented in Theorem 8 combined with the upper bounds on T i j ( p ) T_{ij}^{(p)} T ij ( p ) presented in Theorem 9. □
Pairwise Frank-Wolfe for UOT
In this section, we study an efficient implementation of the Pairwise Frank-Wolfe (PFW) algorithm for the UOT problem. In what follows, we let P = conv ( V ) ⊂ R d P=\operatorname{conv}(\mathcal{V})\subset\mathbb{R}^d P = conv ( V ) ⊂ R d be a non-empty polytope with finite vertex set V \mathcal{V} V , with cardinality N N N and denote by D D D the diameter of P P P . Let F F F be a proper lower semi-continuous convex objective with X ⋆ : = argmin x ∈ P F ( x ) ≠ ∅ \mathcal{X}^\star:=\operatorname*{argmin}_{x\in P}F(x)\ne\varnothing X ⋆ := argmin x ∈ P F ( x ) = ∅ and finite minimum F ⋆ : = min x ∈ P F ( x ) F^\star:=\min_{x\in P}F(x) F ⋆ := min x ∈ P F ( x ) . Given x 0 ∈ P x_0\in P x 0 ∈ P , assume that
C 0 = { x ∈ P : F ( x ) ≤ F ( x 0 ) } (88) C_0=\{x\in P:F(x)\le F(x_0)\} \tag*{(88)} C 0 = { x ∈ P : F ( x ) ≤ F ( x 0 )} (88)
is contained in an open convex set on which F F F is continuously differentiable. Given x ( k ) ∈ P x^{(k)}\in P x ( k ) ∈ P , we say that ξ k , v ≥ 0 \xi_{k,v}\ge0 ξ k , v ≥ 0 are the coefficients associated with an explicit representation of x ( k ) x^{(k)} x ( k ) if
x ( k ) = ∑ v ∈ V ξ k , v v , ∑ v ξ k , v = 1 , (89) x^{(k)}=\sum_{v\in\mathcal{V}}\xi_{k,v}v,\qquad\sum_v\xi_{k,v}=1, \tag*{(89)} x ( k ) = v ∈ V ∑ ξ k , v v , v ∑ ξ k , v = 1 , (89)
and denote by S k = { v : ξ k , v > 0 } S_k=\{v:\xi_{k,v}>0\} S k = { v : ξ k , v > 0 } the active set of the representation. Given a point x ( k ) x^{(k)} x ( k ) with explicit representation ξ k , v \xi_{k,v} ξ k , v , a PFW iteration is defined by the following two linear minimization oracles (LMOs)
s k ∈ argmin v ∈ V ⟨ ∇ F ( x ( k ) ) , v ⟩ , v k ∈ argmax v ∈ S k ⟨ ∇ F ( x ( k ) ) , v ⟩ , d k = s k − v k , g k = − ⟨ ∇ F ( x ( k ) ) , d k ⟩ , γ k max = ξ k , v k , x ( k + 1 ) = x ( k ) + γ k d k . (90) \begin{aligned}
s_k &\in\operatorname*{argmin}_{v\in\mathcal{V}}\langle\nabla F(x^{(k)}),v\rangle, &
v_k &\in\operatorname*{argmax}_{v\in S_k}\langle\nabla F(x^{(k)}),v\rangle,\\
d_k &= s_k-v_k, &
g_k &= -\langle\nabla F(x^{(k)}),d_k\rangle,\\
\gamma_k^{\max} &= \xi_{k,v_k}, &
x^{(k+1)} &= x^{(k)}+\gamma_k d_k. \tag*{(90)}
\end{aligned} s k d k γ k m a x ∈ v ∈ V argmin ⟨ ∇ F ( x ( k ) ) , v ⟩ , = s k − v k , = ξ k , v k , v k g k x ( k + 1 ) ∈ v ∈ S k argmax ⟨ ∇ F ( x ( k ) ) , v ⟩ , = − ⟨ ∇ F ( x ( k ) ) , d k ⟩ , = x ( k ) + γ k d k . (90)
where 0 < γ k ≤ γ k max 0 < \gamma_k \leq\gamma_k^{\max} 0 < γ k ≤ γ k m a x , with γ k max \gamma_k^{\max} γ k m a x being a feasible maximum stepsize, and g k g_k g k is the pairwise Frank-Wolfe gap in x ( k ) x^{(k)} x ( k ) . At each iteration, if g k = 0 g_k = 0 g k = 0 , the algorithm stops, otherwise s k ≠ v k s_k \ne v_k s k = v k , x ( k + 1 ) ≠ x ( k ) x^{(k+1)} \ne x^{(k)} x ( k + 1 ) = x ( k ) , and the coefficients of x ( k + 1 ) x^{(k+1)} x ( k + 1 ) are updated as
ξ k + 1 , s k = ξ k , s k + γ k , ξ k + 1 , v k = ξ k , v k − γ k , (91) \xi_{k+1,s_k} = \xi_{k,s_k} + \gamma_k,\qquad\xi_{k+1,v_k} = \xi_{k,v_k} - \gamma_k, \tag*{(91)} ξ k + 1 , s k = ξ k , s k + γ k , ξ k + 1 , v k = ξ k , v k − γ k , (91)
with all other coefficients unchanged and the active set is updated accordingly.
The bounded compressed problem and its vertices
Given R > 0 R > 0 R > 0 , let P R \mathcal{P}_R P R be the polytope defined in (65) , where B 0 B_0 B 0 is the mass bound in (44) . We now show that we can factor P R \mathcal{P}_R P R as P η × P S \mathcal{P}_\eta\times\mathcal{P}_S P η × P S where
P η = { η ∈ R + n × m : η i j = 0 on E R c , ∥ η ∥ 1 ≤ B 0 } , \mathcal{P}_\eta= \{\eta\in\mathbb{R}_{+}^{n \times m} : \eta_{ij} = 0 \text{ on } \mathcal{E}_R^c,\ \|\eta\|_1 \leq B_0\}, P η = { η ∈ R + n × m : η ij = 0 on E R c , ∥ η ∥ 1 ≤ B 0 } ,
P S = { ( u , w ) ∈ R + n × R + m : ∥ u ∥ 1 = ∥ w ∥ 1 ≤ B 0 } (92) \mathcal{P}_S = \{(u,w) \in\mathbb{R}_{+}^{n} \times\mathbb{R}_{+}^{m} : \|u\|_1 = \|w\|_1 \leq B_0\} \tag*{(92)} P S = {( u , w ) ∈ R + n × R + m : ∥ u ∥ 1 = ∥ w ∥ 1 ≤ B 0 } (92)
and denote by z = ( η , u , w ) z = (\eta,u,w) z = ( η , u , w ) a generic element of P R \mathcal{P}_R P R . Notice that only the coordinates of η \eta η indexed by E R \mathcal{E}_R E R need to be stored. Given z = ( η , u , w ) z = (\eta,u,w) z = ( η , u , w ) , we set
r ( z ) = P 1 η + u , s ( z ) = P 2 η + w , (93) r(z) = P_1\eta+ u,\qquad s(z) = P_2\eta+ w, \tag*{(93)} r ( z ) = P 1 η + u , s ( z ) = P 2 η + w , (93)
using the marginal operators from (29) . Notice that r ( z ) , s ( z ) ≤ 2 B 0 r(z),s(z) \leq2B_0 r ( z ) , s ( z ) ≤ 2 B 0 coordinate-wise for every z ∈ P R z \in\mathcal{P}_R z ∈ P R . First, we characterize the set of vertices of P R \mathcal{P}_R P R and prove that the problems defining the LMOs in (90) can be solved through a simple search when P = P R P = \mathcal{P}_R P = P R .
Proposition 4. The vertex sets of the two factors in (92) are
V η = { 0 } ∪ { B 0 e i j : ( i , j ) ∈ E R } , V S = { 0 } ∪ { B 0 ( e i , e j ) : i ∈ [ n ] , j ∈ [ m ] } . \mathcal{V}_\eta= \{0\} \cup\{B_0 e_{ij} : (i,j) \in\mathcal{E}_R\},\qquad\mathcal{V}_S = \{0\} \cup\{B_0(e_i,e_j) : i \in[n],\ j \in[m]\}. V η = { 0 } ∪ { B 0 e ij : ( i , j ) ∈ E R } , V S = { 0 } ∪ { B 0 ( e i , e j ) : i ∈ [ n ] , j ∈ [ m ]} . Consequently V R = V η × V S \mathcal{V}_R = \mathcal{V}_\eta\times\mathcal{V}_S V R = V η × V S is the vertex set of P R \mathcal{P}_R P R . In particular, given a linear functional restricted over P R \mathcal{P}_R P R with coefficients ( Q , a , b ) (Q,a,b) ( Q , a , b ) , a minimizing vertex for the LMO is obtained as follows:
(i) Minimize Q i j Q_{ij} Q ij on E R \mathcal{E}_R E R . Select B 0 e i j B_0e_{ij} B 0 e ij if its minimum is negative, and select 0 otherwise. If E R = ∅ \mathcal{E}_R = \varnothing E R = ∅ , select 0.
(ii) Independently minimize a i a_i a i and b j b_j b j . Select B 0 ( e i , e j ) B_0(e_i,e_j) B 0 ( e i , e j ) if a i + b j = min i a i + min j b j < 0 a_i + b_j = \min_i a_i + \min_j b_j < 0 a i + b j = min i a i + min j b j < 0 , and select 0 otherwise.
Ties may be resolved arbitrarily among minimizing vertices.
Proof. The first factor is a scaled simplex with its zero vertex. For ( u , w ) ∈ P S (u,w) \in\mathcal{P}_S ( u , w ) ∈ P S , let t = 1 T u = 1 T w t = \mathbf{1}^{\mathsf{T}}u = \mathbf{1}^{\mathsf{T}}w t = 1 T u = 1 T w . If t > 0 t > 0 t > 0 , choose a nonnegative matrix ρ \rho ρ with row sums u u u and column sums w w w , for example ρ = u w T / t \rho= uw^{\mathsf{T}}/t ρ = u w T / t . Then
( u , w ) = ( 1 − t B 0 ) 0 + ∑ i , j ρ i j B 0 B 0 ( e i , e j ) . (u,w) = \left(1 - \frac{t}{B_0}\right)0 + \sum_{i,j} \frac{\rho_{ij}}{B_0} B_0(e_i,e_j). ( u , w ) = ( 1 − B 0 t ) 0 + i , j ∑ B 0 ρ ij B 0 ( e i , e j ) .
For t = 0 t = 0 t = 0 the point is zero. Each displayed nonzero atom is extreme: its zero coordinates force every member of a convex representation to have the same two-coordinate support, and its total mass B 0 B_0 B 0 forces the same mass. The zero point is extreme by nonnegativity. This proves the second vertex description. Vertices of a product are products of vertices. Linear minimization separates between the factors, and its value at a nonzero residual vertex is B 0 ( a i + b j ) B_0(a_i+b_j) B 0 ( a i + b j ) , proving the oracle formulas. □ \square □
Notice that if P = P R P = \mathcal{P}_R P = P R , Proposition 4 allows us to compute s k s_k s k without enumerating V R \mathcal{V}_R V R . Moreover, any vertex of P P P has at most three nonzero coordinates, and at most one new joint atom is added per iteration.
Convergence Guarantees for the PFW Algorithm
In this section, we layout our standing assumptions on the stepsize implemented in the PFW method and on the regularity of the function F F F . We show that these assumptions ensure the convergence of the PFW and characterize its convergence speed.
First, we state the general set of conditions of F F F that we will assume throughout the remainder of the section.
Assumption 1. Let P ⊂ R n \mathcal{P} \subset\mathbb{R}^{n} P ⊂ R n be a polytope. Given F ∈ Γ 0 ( R n ) F \in\Gamma_{0}(\mathbb{R}^{n}) F ∈ Γ 0 ( R n ) , let X ⋆ : = argmin z ∈ P F ( z ) \mathcal{X}^{\star} := \operatorname*{argmin}_{z \in\mathcal{P}} F(z) X ⋆ := argmin z ∈ P F ( z ) be the set of minimizers of F F F over P \mathcal{P} P . We assume there exists a closed and convex set C \mathcal{C} C such that
X ⋆ ⊂ C ⊂ P \mathcal{X}^{\star} \subset\mathcal{C} \subset\mathcal{P} X ⋆ ⊂ C ⊂ P ;
F F F is differentiable over an open set containing C \mathcal{C} C , has an α \alpha α -Hölder continuous gradient on C \mathcal{C} C with constant H H H ; and
there exists a β ≥ 1 \beta\ge1 β ≥ 1 , q ≥ 2 q \ge2 q ≥ 2 , and κ q > 0 \kappa_q > 0 κ q > 0 for which it holds
κ q q d ( z , X ⋆ ) q ≤ F ( z ⋆ ) − F ( z ) − β ⟨ ∇ F ( z ) , z ⋆ − z ⟩ , (94) \frac{\kappa_q}{q}d(z,\mathcal{X}^{\star})^{q} \le F(z^{\star})-F(z)-\beta\langle\nabla F(z),z^{\star}-z\rangle, \tag*{(94)} q κ q d ( z , X ⋆ ) q ≤ F ( z ⋆ ) − F ( z ) − β ⟨ ∇ F ( z ) , z ⋆ − z ⟩ , (94) where d ( z , X ⋆ ) = min z ⋆ ∈ X ⋆ ∥ z − z ⋆ ∥ d(z,\mathcal{X}^{\star})=\min_{z^{\star}\in\mathcal{X}^{\star}}\|z-z^{\star}\| d ( z , X ⋆ ) = min z ⋆ ∈ X ⋆ ∥ z − z ⋆ ∥ for all z ∈ C z\in\mathcal{C} z ∈ C and all z ⋆ ∈ X ⋆ z^{\star}\in\mathcal{X}^{\star} z ⋆ ∈ X ⋆ .
Step sizes and sufficient decrease
Given x 0 x_0 x 0 , let C 0 \mathcal{C}_0 C 0 be defined as in (88) . Assume that ∇ F \nabla F ∇ F is α \alpha α -Hölder continuous on C 0 \mathcal{C}_0 C 0 , with 0 < α ≤ 1 0 < \alpha\le1 0 < α ≤ 1 and constant H > 0 H > 0 H > 0 and set L = H D 1 + α L = H D^{1+\alpha} L = H D 1 + α . In what follows, we consider step sizes satisfying
γ k ≥ γ ‾ k : = min { γ k max , ς g k 1 / α } , F ( x ( k ) ) − F ( x ( k + 1 ) ) ≥ ϑ γ k g k . (95) \gamma_k \ge\overline{\gamma}_k := \min\{\gamma_k^{\max},\varsigma g_k^{1/\alpha}\},\qquad F(x^{(k)})-F(x^{(k+1)}) \ge\vartheta\gamma_k g_k. \tag*{(95)} γ k ≥ γ k := min { γ k m a x , ς g k 1/ α } , F ( x ( k ) ) − F ( x ( k + 1 ) ) ≥ ϑ γ k g k . (95)
where ς , ϑ ≥ 0 \varsigma,\vartheta\ge0 ς , ϑ ≥ 0 are suitable positive parameters. We say that a PFW iteration is a good iteration if ς g k ≤ γ k max \varsigma g_k \le\gamma_k^{\max} ς g k ≤ γ k m a x , so that non-good iteration takes the full step γ k = γ k max \gamma_k=\gamma_k^{\max} γ k = γ k m a x . The next proposition ensures that the assumptions on the stepsize are satisfied by three of the most common stepsizes used in iterative routines.
Proposition 5. If ∇ F \nabla F ∇ F is α \alpha α -Hölder continuous over C \mathcal{C} C , the following step sizes satisfy (95) .
(i) The model step
γ k m s = min { γ k max , ( g k / L ) 1 / α } , \gamma_k^{\mathrm{ms}}=\min\{\gamma_k^{\max},(g_k/L)^{1/\alpha}\}, γ k ms = min { γ k m a x , ( g k / L ) 1/ α } , satisfies (95) with ς = L − 1 / α \varsigma=L^{-1/\alpha} ς = L − 1/ α and ϑ = α / ( 1 + α ) \vartheta=\alpha/(1+\alpha) ϑ = α / ( 1 + α ) .
(ii) The optimal stepsize γ k o p t \gamma_k^{\mathrm{opt}} γ k opt obtained via exact minimization of F ( x ( k ) + γ d k ) F(x^{(k)}+\gamma d_k) F ( x ( k ) + γ d k ) over [ 0 , γ k max ] [0,\gamma_k^{\max}] [ 0 , γ k m a x ] satisfies (95) with the same constants as the model stepsize.
(iii) The Armijo stepsize, denoted by γ k a r m \gamma_k^{\mathrm{arm}} γ k arm , is defined by backtracking as follows. Given 0 < ρ < 1 0<\rho<1 0 < ρ < 1 and 0 < σ < 1 0<\sigma<1 0 < σ < 1 , start from γ k max \gamma_k^{\max} γ k m a x and successively multiply γ k max \gamma_k^{\max} γ k m a x by ρ \rho ρ until F ( x ( k ) + γ d k ) ≤ F ( x ( k ) ) − σ γ g k F(x^{(k)}+\gamma d_k)\le F(x^{(k)})-\sigma\gamma g_k F ( x ( k ) + γ d k ) ≤ F ( x ( k ) ) − σ γ g k . Then the Armijo rule satisfies (95) with
ς = ρ ( ( 1 + α ) ( 1 − σ ) L ) 1 / α , ϑ = σ . \varsigma=\rho\left(\frac{(1+\alpha)(1-\sigma)}{L}\right)^{1/\alpha},\qquad\vartheta=\sigma. ς = ρ ( L ( 1 + α ) ( 1 − σ ) ) 1/ α , ϑ = σ .
Proof. For the sake of simplicity, let us fix x = x ( k ) x=x^{(k)} x = x ( k ) , d = d k d=d_k d = d k , g = g k > 0 g=g_k>0 g = g k > 0 , and γ max = γ k max \gamma^{\max}=\gamma_k^{\max} γ m a x = γ k m a x . The descent lemma for functions with α \alpha α -Hölder gradient yields
F ( x + t d ) ≤ F ( x ) − t g + L 1 + α t 1 + α , (96) F(x+td) \le F(x)-tg+\frac{L}{1+\alpha}t^{1+\alpha}, \tag*{(96)} F ( x + t d ) ≤ F ( x ) − t g + 1 + α L t 1 + α , (96)
so that for t 0 = min { γ max , ( g / L ) 1 / α } t_0=\min\{\gamma^{\max},(g/L)^{1/\alpha}\} t 0 = min { γ m a x , ( g / L ) 1/ α } we have
F ( x ) − F ( x + γ m s d ) = F ( x ) − F ( x + t 0 d ) ≥ α 1 + α t 0 g F(x)-F(x+\gamma^{\mathrm{ms}}d)=F(x)-F(x+t_0d)\ge\frac{\alpha}{1+\alpha}t_0g F ( x ) − F ( x + γ ms d ) = F ( x ) − F ( x + t 0 d ) ≥ 1 + α α t 0 g
and
⟨ ∇ F ( x + γ d ) , d ⟩ ≤ − g + H γ α ∥ d ∥ 1 + α ≤ − g + L γ α < 0 for 0 ≤ γ < ( g / L ) 1 / α , (97) \langle\nabla F(x+\gamma d),d\rangle\le-g+H\gamma^\alpha\lVert d\rVert^{1+\alpha}\le-g+L\gamma^\alpha<0\qquad\text{for}\qquad0\le\gamma<(g/L)^{1/\alpha}, \tag*{(97)} ⟨ ∇ F ( x + γ d ) , d ⟩ ≤ − g + H γ α ∥ d ∥ 1 + α ≤ − g + L γ α < 0 for 0 ≤ γ < ( g / L ) 1/ α , (97)
which ensures that h k + 1 ≤ h k h_{k+1}\le h_k h k + 1 ≤ h k . Proving the result for γ m s \gamma^{\mathrm{ms}} γ ms . The result on the optimal stepsize follows by the same argument since F ( x ) − F ( x + γ o p t d ) ≥ F ( x ) − F ( x + γ m s d ) F(x)-F(x+\gamma^{\mathrm{opt}}d)\ge F(x)-F(x+\gamma^{\mathrm{ms}}d) F ( x ) − F ( x + γ opt d ) ≥ F ( x ) − F ( x + γ ms d ) and the descent lemma ensures that γ ↦ F ( x + γ d ) \gamma\mapsto F(x+\gamma d) γ ↦ F ( x + γ d ) is decreasing on [ 0 , ( g / L ) 1 / α ] [0,(g/L)^{1/\alpha}] [ 0 , ( g / L ) 1/ α ] .
Lastly, we consider the Armijo stepsize, for which the sufficient decrease condition is trivially satisfied by setting σ = ϑ \sigma=\vartheta σ = ϑ . If the initial trial is accepted, it equals γ max \gamma^{\max} γ m a x , hence both conditions are trivially met. Owing to the descent lemma, we notice that
γ A = ( ( 1 + α ) ( 1 − σ ) g L ) 1 / α ⇒ F ( x ) − F ( x + γ A d ) ≥ γ A σ g . \gamma_A=\left(\frac{(1+\alpha)(1-\sigma)g}{L}\right)^{1/\alpha}\qquad\Rightarrow\qquad F(x)-F(x+\gamma_A d)\ge\gamma_A\sigma g. γ A = ( L ( 1 + α ) ( 1 − σ ) g ) 1/ α ⇒ F ( x ) − F ( x + γ A d ) ≥ γ A σ g .
In particular, it must be that ρ γ A ≤ γ a r m \rho\gamma_A\le\gamma^{\mathrm{arm}} ρ γ A ≤ γ arm , since if γ a r m < ρ γ A \gamma^{\mathrm{arm}}<\rho\gamma_A γ arm < ρ γ A it means that we would have accepted γ ~ a r m = γ a r m / ρ \widetilde{\gamma}^{\mathrm{arm}}=\gamma^{\mathrm{arm}}/\rho γ arm = γ arm / ρ during the backtracking procedure since γ ~ a r m < γ A \widetilde{\gamma}^{\mathrm{arm}}<\gamma_A γ arm < γ A , contradicting the definition of γ a r m \gamma^{\mathrm{arm}} γ arm , allowing us to conclude the proof. □
Convergence Rate Guarantees
We now characterize the convergence speed of the PFW depending on the parameters q q q and α \alpha α defined in Assumption 1. For a finite set V V V and x ∈ conv ( V ) x\in\operatorname{conv}(V) x ∈ conv ( V ) , let S V ( x ) S_V(x) S V ( x ) be the sets S ⊆ V S\subseteq V S ⊆ V for which x x x has an explicit representation via the elements in S S S and the coefficients associated with this representation are all strictly positive coefficients. Define
PdirW ( V , r , x ) : = min S ∈ S V ( x ) max v ∈ S ⟨ r ∥ r ∥ , s − v ⟩ , \operatorname{PdirW}(V,r,x):=\min_{S\in S_V(x)}\max_{v\in S}\left\langle\frac{r}{\lVert r\rVert},s-v\right\rangle, PdirW ( V , r , x ) := S ∈ S V ( x ) min v ∈ S max ⟨ ∥ r ∥ r , s − v ⟩ ,
W : = PWidth ( V ) : = inf Q a face of P , x ∈ Q r ∈ cone ( Q − x ) ∖ { 0 } PdirW ( V ∩ Q , r , x ) . W:=\operatorname{PWidth}(\mathcal{V}):=\inf_{\substack{Q\text{ a face of }P,\ x\in Q\\ r\in\operatorname{cone}(Q-x)\setminus\{0\}}}\operatorname{PdirW}(\mathcal{V}\cap Q,r,x). W := PWidth ( V ) := Q a face of P , x ∈ Q r ∈ cone ( Q − x ) ∖ { 0 } inf PdirW ( V ∩ Q , r , x ) .
This width is strictly positive for a nontrivial polytope, with directions taken in its affine hull. We use the following geometric inequality from [33 Theorem 3] : for every nonoptimal x ∈ P x\in P x ∈ P , every active representation of x x x , and every x ⋆ ∈ X ⋆ x^\star\in X^\star x ⋆ ∈ X ⋆ , we have
⟨ − ∇ F ( x ) , x ⋆ − x ⟩ ≤ ⟨ − ∇ F ( x ) , d ⟩ W ∥ x ⋆ − x ∥ , (98) \langle-\nabla F(x),x^\star-x\rangle\le\frac{\langle-\nabla F(x),d\rangle}{W}\lVert x^\star-x\rVert, \tag*{(98)} ⟨ − ∇ F ( x ) , x ⋆ − x ⟩ ≤ W ⟨ − ∇ F ( x ) , d ⟩ ∥ x ⋆ − x ∥ , (98)
where d d d is the pairwise Frank-Wolfe direction computed at point x x x . We are now ready to state the general convergence result that we will use for the UOT problem. This extends the results for 2-conditioned Lipschitz smooth objective functions, given in [33 ] , to the case in which the objective is q q q -conditioned and Hölder smooth.
Theorem 10. Let F F F be such that its gradient is Hölder continuous on C 0 C_0 C 0 with exponent α \alpha α and constant H H H . Assume that there exist q ≥ 2 q\ge2 q ≥ 2 , β ≥ 1 \beta\ge1 β ≥ 1 , and κ q > 0 \kappa_q>0 κ q > 0 for which (94) holds with C = C 0 \mathcal{C} = C_0 C = C 0 . Moreover, let h k = F ( x ( k ) ) − F ⋆ h_k = F(x^{(k)}) - F^\star h k = F ( x ( k ) ) − F ⋆ be the primal gap, t k t_k t k be the number of good iterations among the first k − 1 k-1 k − 1 , and set
A ⋆ : = W κ q 1 / q β ( q q − 1 ) ( q − 1 ) / q , c ⋆ : = ζ ϑ A ⋆ ( 1 + α ) / α , s : = q − 1 − α q α ≥ 0. (99) A_\star:= \frac{W\kappa_q^{1/q}}{\beta}\left(\frac{q}{q-1}\right)^{(q-1)/q}, \qquad c_\star:= \zeta\vartheta A_\star^{(1+\alpha)/\alpha}, \qquad s := \frac{q-1-\alpha}{q\alpha} \ge0. \tag*{(99)} A ⋆ := β W κ q 1/ q ( q − 1 q ) ( q − 1 ) / q , c ⋆ := ζ ϑ A ⋆ ( 1 + α ) / α , s := q α q − 1 − α ≥ 0. (99)
Then, denoted by x ( k ) x^{(k)} x ( k ) the k k k -th iterate of the PFW algorithm, if h 0 > 0 h_0 > 0 h 0 > 0 and the stepsize satisfies (95) , we have that
if s > 0 s > 0 s > 0 , then h k ≤ ( h 0 − s + s c ⋆ t k ) − 1 / s h_k \le\left(h_0^{-s} + sc_\star t_k\right)^{-1/s} h k ≤ ( h 0 − s + s c ⋆ t k ) − 1/ s ;
if s = 0 s = 0 s = 0 , equivalently q = 2 q = 2 q = 2 and α = 1 \alpha= 1 α = 1 , then h k ≤ h 0 ( 1 − θ ) t k h_k \le h_0(1-\theta)^{t_k} h k ≤ h 0 ( 1 − θ ) t k with θ : = min { c ⋆ , 1 / 2 } \theta:= \min\{c_\star,1/2\} θ := min { c ⋆ , 1/2 } .
Proof. Fix a nonoptimal iterate x x x and choose a nearest optimizer to x x x , namely x ⋆ x^\star x ⋆ . Denoted with h = F ( x ) − F ⋆ h = F(x) - F^\star h = F ( x ) − F ⋆ and Δ = ∥ x − x ⋆ ∥ \Delta= \|x-x^\star\| Δ = ∥ x − x ⋆ ∥ , conditions (6.7) and (6.11) give
h ≤ β q W Δ − κ q q Δ q ≤ sup r ≥ 0 { β q W r − κ q q r q } = q − 1 q κ q − 1 / ( q − 1 ) ( β q W ) q / ( q − 1 ) . (100) h \le\frac{\beta q}{W}\Delta-\frac{\kappa_q}{q}\Delta^q \le\sup_{r\ge0}\left\{\frac{\beta q}{W}r-\frac{\kappa_q}{q}r^q\right\} = \frac{q-1}{q}\kappa_q^{-1/(q-1)}\left(\frac{\beta q}{W}\right)^{q/(q-1)}. \tag*{(100)} h ≤ W β q Δ − q κ q Δ q ≤ r ≥ 0 sup { W β q r − q κ q r q } = q q − 1 κ q − 1/ ( q − 1 ) ( W β q ) q / ( q − 1 ) . (100)
Consequently g ≥ A ⋆ h ( q − 1 ) / q g \ge A_\star h^{(q-1)/q} g ≥ A ⋆ h ( q − 1 ) / q . During a good iteration (95) we have
h k − h k + 1 ≥ ζ ϑ g k ( 1 + α ) / α ≥ ζ ϑ A ⋆ ( 1 + α ) / α h k ( q − 1 ) ( 1 + α ) / ( q α ) ≥ c ⋆ h k 1 + s . (101) h_k-h_{k+1} \ge\zeta\vartheta g_k^{(1+\alpha)/\alpha} \ge\zeta\vartheta A_\star^{(1+\alpha)/\alpha}h_k^{(q-1)(1+\alpha)/(q\alpha)} \ge c_\star h_k^{1+s}. \tag*{(101)} h k − h k + 1 ≥ ζ ϑ g k ( 1 + α ) / α ≥ ζ ϑ A ⋆ ( 1 + α ) / α h k ( q − 1 ) ( 1 + α ) / ( q α ) ≥ c ⋆ h k 1 + s . (101)
For non-good iterations, we have that h k + 1 ≤ h k h_{k+1} \le h_k h k + 1 ≤ h k . If s > 0 s > 0 s > 0 and h k + 1 > 0 h_{k+1} > 0 h k + 1 > 0 , then
h k + 1 − s − h k − s = s ∫ h k + 1 h k r − s − 1 d r ≥ s ( h k − h k + 1 ) h k − s − 1 ≥ s c ⋆ h_{k+1}^{-s}-h_k^{-s}=s\int_{h_{k+1}}^{h_k}r^{-s-1}\,dr\ge s(h_k-h_{k+1})h_k^{-s-1}\ge sc_\star h k + 1 − s − h k − s = s ∫ h k + 1 h k r − s − 1 d r ≥ s ( h k − h k + 1 ) h k − s − 1 ≥ s c ⋆
at each good iteration. Summing over all steps proves the convergence rate for s > 0 s > 0 s > 0 .
For s = 0 s = 0 s = 0 , (6.14) gives h k + 1 ≤ ( 1 − θ ) h k h_{k+1} \le(1-\theta)h_k h k + 1 ≤ ( 1 − θ ) h k at good iterations, which is the standard iteration bound for linear convergence rate. □
To conclude, we show that the convergence result holds when the objective function is the composition between a convex function and a linear function. This extends the quadratic conditioning argument in [33 Appendix F] to arbitrary powers q ≥ 2 q \ge2 q ≥ 2 .
Proposition 6. Suppose F ( x ) = H ( A x ) + ⟨ b , x ⟩ F(x)=\mathcal{H}(Ax)+\langle b,x\rangle F ( x ) = H ( A x ) + ⟨ b , x ⟩ on P P P and that H \mathcal{H} H is continuously differentiable on a neighborhood of A ( C 0 ) A(C_0) A ( C 0 ) . Assume that H \mathcal{H} H is q q q -uniformly convex on A ( C 0 ) A(C_0) A ( C 0 ) with constant μ H > 0 \mu_H>0 μ H > 0 and q ≥ 2 q\ge2 q ≥ 2 , as in (2.1). Then F F F is α \alpha α -uniformly convex on C 0 C_0 C 0 as in (2.2), and there exists a suitable constant κ q \kappa_q κ q for which (6.7) holds with β = 2 \beta=2 β = 2 .
Proof. The strict convexity of H \mathcal{H} H implies that all optimizers have the same image y ⋆ = A x ⋆ y^\star=Ax^\star y ⋆ = A x ⋆ . Moreover, following the argument proposed by [7 ] , we have b ⋆ = ⟨ b , x ⋆ ⟩ b^\star=\langle b,x^\star\rangle b ⋆ = ⟨ b , x ⋆ ⟩ . These two equalities characterize optimality among points of P P P , hence
X ⋆ = { x ∈ P : A x = y ⋆ , ⟨ b , x ⟩ = b ⋆ } . \mathcal{X}^\star=\{x\in P: Ax=y^\star,\ \langle b,x\rangle=b^\star\}. X ⋆ = { x ∈ P : A x = y ⋆ , ⟨ b , x ⟩ = b ⋆ } . Hoffman’s error bound for this consistent system of linear equalities and the inequalities defining P P P gives a finite constant K > 0 K>0 K > 0 such that
dist ( x , X ⋆ ) ≤ K ( ∥ A x − y ⋆ ∥ + ∣ ⟨ b , x ⟩ − b ⋆ ∣ ) ( x ∈ P ) . \operatorname{dist}(x,\mathcal{X}^\star)\le K\left(\|Ax-y^\star\|+|\langle b,x\rangle-b^\star|\right)\qquad(x\in P). dist ( x , X ⋆ ) ≤ K ( ∥ A x − y ⋆ ∥ + ∣ ⟨ b , x ⟩ − b ⋆ ∣ ) ( x ∈ P ) .
Let L H = max y ∈ A ( C ) ∥ ∇ H ( y ) ∥ L_H=\max_{y\in A(C)}\|\nabla H(y)\| L H = max y ∈ A ( C ) ∥∇ H ( y ) ∥ , h ˉ = F ( x 0 ) − F ⋆ \bar h=F(x_0)-F^\star h ˉ = F ( x 0 ) − F ⋆ , u = ∥ A x − y ⋆ ∥ u=\|Ax-y^\star\| u = ∥ A x − y ⋆ ∥ , and h = F ( x ) − F ⋆ h=F(x)-F^\star h = F ( x ) − F ⋆ . Convexity of C C C ensures that H H H is L H L_H L H -Lipschitz on A ( C ) A(C) A ( C ) , whence
∣ ⟨ b , x ⟩ − b ⋆ ∣ ≤ h + L H u , dist ( x , x ⋆ ) q ≤ C H ( ( 1 + L H ) q u q + h ˉ q − 1 h ) , C H : = 2 q − 1 K q . |\langle b,x\rangle-b^\star|\le h+L_Hu,\qquad\operatorname{dist}(x,x^\star)^q\le C_H\bigl((1+L_H)^qu^q+\bar h^{q-1}h\bigr),\qquad C_H:=2^{q-1}K^q. ∣ ⟨ b , x ⟩ − b ⋆ ∣ ≤ h + L H u , dist ( x , x ⋆ ) q ≤ C H ( ( 1 + L H ) q u q + h ˉ q − 1 h ) , C H := 2 q − 1 K q .
At an optimizer, first-order optimality and (2) give h ≥ ( μ H / q ) u q h\ge(\mu_H/q)u^q h ≥ ( μ H / q ) u q . Hence (3) holds with
ν : = [ C H ( ( 1 + L H ) q q μ H + h ˉ q − 1 ) ] − 1 . (103) \nu:=\left[C_H\left((1+L_H)^q\frac{q}{\mu_H}+\bar h^{q-1}\right)\right]^{-1}. \tag*{(103)} ν := [ C H ( ( 1 + L H ) q μ H q + h ˉ q − 1 ) ] − 1 . (103)
For the reverse Bregman divergence, put a x = − ⟨ ∇ F ( x ) , x ⋆ − x ⟩ a_x=-\langle\nabla F(x),x^\star-x\rangle a x = − ⟨ ∇ F ( x ) , x ⋆ − x ⟩ . The linear part cancels and
a x − h = H ( y ⋆ ) − H ( A x ) − ⟨ ∇ H ( A x ) , y ⋆ − A x ⟩ ≥ μ H q u q . a_x-h=\mathcal{H}(y^\star)-\mathcal{H}(Ax)-\langle\nabla\mathcal{H}(Ax),y^\star-Ax\rangle\ge\frac{\mu_H}{q}u^q. a x − h = H ( y ⋆ ) − H ( A x ) − ⟨ ∇ H ( A x ) , y ⋆ − A x ⟩ ≥ q μ H u q .
Therefore − h + 2 a x ≥ h + ( 2 μ H / q ) u q -h+2a_x\ge h+(2\mu_H/q)u^q − h + 2 a x ≥ h + ( 2 μ H / q ) u q . Combining the last two estimates proves (94) with β = 2 \beta=2 β = 2 and, for example,
κ q : = q C H max { h ˉ q − 1 , q ( 1 + L H ) q / ( 2 μ H ) } > 0. (104) \kappa_q:=\frac{q}{C_H\max\{\bar h^{q-1},q(1+L_H)^q/(2\mu_H)\}}>0. \tag*{(104)} κ q := C H max { h ˉ q − 1 , q ( 1 + L H ) q / ( 2 μ H )} q > 0. (104)
Support Identification Property
In this section, we extend the support identification property presented in [11 ] to fit our framework. Since the proof techniques are close to the one used in [11 ] , we only report the main result that will be used in what follows and defer the proof in a dedicated part of the Appendix.
First, we introduce some useful notation. For each i ∈ [ n ] i\in[n] i ∈ [ n ] , we define θ i ( x ) = ⟨ ∇ F ( x ) , e i − x ⟩ \theta_i(x)=\langle\nabla F(x),e_i-x\rangle θ i ( x ) = ⟨ ∇ F ( x ) , e i − x ⟩ . Given x ⋆ ∈ X ⋆ x^\star\in\mathcal{X}^\star x ⋆ ∈ X ⋆ , we denote by I ( x ⋆ ) I(x^\star) I ( x ⋆ ) the extended support of x ⋆ x^\star x ⋆ , i.e., the set of indices i ∈ [ n ] i\in[n] i ∈ [ n ] for which θ i ( x ⋆ ) = 0 \theta_i(x^\star)=0 θ i ( x ⋆ ) = 0 . By definition, the complement of I ( x ⋆ ) I(x^\star) I ( x ⋆ ) , that is I c ( x ⋆ ) = [ n ] ∖ I ( x ⋆ ) I^c(x^\star)=[n]\setminus I(x^\star) I c ( x ⋆ ) = [ n ] ∖ I ( x ⋆ ) , corresponds to the set of indices satisfying θ i ( x ⋆ ) > 0 \theta_i(x^\star)>0 θ i ( x ⋆ ) > 0 .
Given x 0 x_0 x 0 and fixed a stepsize, let x ( k ) x^{(k)} x ( k ) be the k k k -th iteration of the PFW algorithm, then set
O k ( x ⋆ ) : = { i ∈ I c ( x ⋆ ) ∣ x i ( k ) = 0 } and J k ( x ⋆ ) = I c ( x ⋆ ) ∖ O k ( x ⋆ ) . O_k(x^\star):=\{i\in I^c(x^\star)\mid x_i^{(k)}=0\}\qquad\text{and}\qquad J_k(x^\star)=I^c(x^\star)\setminus O_k(x^\star). O k ( x ⋆ ) := { i ∈ I c ( x ⋆ ) ∣ x i ( k ) = 0 } and J k ( x ⋆ ) = I c ( x ⋆ ) ∖ O k ( x ⋆ ) .
Finally, we define δ k = max [ n ] ∖ O k ( x ⋆ ) θ i ( x ⋆ ) = max I ( x ⋆ ) ∪ J k ( x ⋆ ) θ i ( x ⋆ ) \delta_k=\max_{[n]\setminus O_k(x^\star)}\theta_i(x^\star)=\max_{I(x^\star)\cup J_k(x^\star)}\theta_i(x^\star) δ k = max [ n ] ∖ O k ( x ⋆ ) θ i ( x ⋆ ) = max I ( x ⋆ ) ∪ J k ( x ⋆ ) θ i ( x ⋆ ) .
When the element x ⋆ ∈ X ⋆ x^\star\in\mathcal{X}^\star x ⋆ ∈ X ⋆ is clear from the context or is unique, we simplify the notation by omitting its dependence, e.g., I = I ( x ⋆ ) I=I(x^\star) I = I ( x ⋆ ) , I c = I c ( x ⋆ ) I^c=I^c(x^\star) I c = I c ( x ⋆ ) , and so on. For the sake of simplicity, we present the result for the case in which P P P is the n n n dimensional simplex and extend the results to the general case afterwards.
Theorem 11. Let F F F be a continuously differentiable function whose gradient is α \alpha α -Hölder continuous with constant H H H over an ϵ \epsilon ϵ -neighborhood of the set of optimal solutions in Δ n \Delta_n Δ n . For every k k k , let x ( k ) x^{(k)} x ( k ) and d ( k ) d^{(k)} d ( k ) be the k k k -th iterate and the direction found at x ( k ) x^{(k)} x ( k ) by the PFW algorithm. Denoted by γ k \gamma_k γ k the step size at step k k k , assume that for every k k k either
γ k = γ k max or γ k ≥ ( − ⟨ ∇ f ( x ( k ) ) , d ( k ) ⟩ 2 H ) 1 α . \gamma_k=\gamma_k^{\max}\qquad\text{or}\qquad\gamma_k\ge\left(-\frac{\langle\nabla f(x^{(k)}),d^{(k)}\rangle}{2H}\right)^{\frac{1}{\alpha}}. γ k = γ k m a x or γ k ≥ ( − 2 H ⟨ ∇ f ( x ( k ) ) , d ( k ) ⟩ ) α 1 . Finally, given x ⋆ ∈ X ⋆ x^\star\in\mathcal{X}^\star x ⋆ ∈ X ⋆ , set
δ min : = min { θ i ( x ⋆ ) s.t. θ i ( x ⋆ ) > 0 } and δ max : = max { θ i ( x ⋆ ) s.t. θ i ( x ⋆ ) > 0 } . \delta_{\min}:=\min\{\theta_i(x^\star)\text{ s.t. }\theta_i(x^\star)>0\}\qquad\text{and}\qquad\delta_{\max}:=\max\{\theta_i(x^\star)\text{ s.t. }\theta_i(x^\star)>0\}. δ m i n := min { θ i ( x ⋆ ) s.t. θ i ( x ⋆ ) > 0 } and δ m a x := max { θ i ( x ⋆ ) s.t. θ i ( x ⋆ ) > 0 } . Then, if ∥ x ( k ) − x ⋆ ∥ 1 < r ⋆ = min { ϵ , 2 α − 1 α ( δ min 2 H n 1 2 + δ max ) 1 α } \|x^{(k)}-x^\star\|_1<r^\star=\min\left\{\epsilon,2^{\frac{\alpha-1}{\alpha}}\left(\frac{\delta_{\min}}{2Hn^{\frac{1}{2}}+\delta_{\max}}\right)^{\frac{1}{\alpha}}\right\} ∥ x ( k ) − x ⋆ ∥ 1 < r ⋆ = min { ϵ , 2 α α − 1 ( 2 H n 2 1 + δ m a x δ m i n ) α 1 } , we have
∣ J k + 1 ∣ ≤ max { ∣ J k ∣ − 1 , 0 } . |J_{k+1}|\le\max\{|J_k|-1,0\}. ∣ J k + 1 ∣ ≤ max { ∣ J k ∣ − 1 , 0 } .
Remark 6 (Extending the results to the general polytope). The results presented in Theorem 11 and Lemma 4 extend to the case in which P \mathcal{P} P is a general polytope as follows. Given the finite set of vertices of P \mathcal{P} P , namely { v ( i ) } i ∈ [ M ] \{v^{(i)}\}_{i\in[M]} { v ( i ) } i ∈ [ M ] take any linear map A P : Δ M → P A_{\mathcal{P}}:\Delta_M\to\mathcal{P} A P : Δ M → P for which it holds
A P ( w ) = ∑ i = 1 M w i v ( i ) (105) A_{\mathcal{P}}(w)=\sum_{i=1}^{M}w_i v^{(i)} \tag*{(105)} A P ( w ) = i = 1 ∑ M w i v ( i ) (105)
where w ∈ Δ M w\in\Delta_M w ∈ Δ M . Since minimizing F F F over P \mathcal{P} P is equivalent to minimizing F A P = F ∘ A P F_{A_{\mathcal{P}}}=F\circ A_{\mathcal{P}} F A P = F ∘ A P over Δ M \Delta_M Δ M , Theorem 11 applies directly, thereby extending the identification result to the general setting in which P \mathcal{P} P is an arbitrary polytope.
Application to UOT problems induced by entropy like divergences
In this subsection, we apply the results obtained for the PFW algorithm to the UOT problems induced by ϕ = ψ p \phi=\psi_p ϕ = ψ p . We recall that ϕ 0 = ψ p , 0 ′ \phi_0=\psi'_{p,0} ϕ 0 = ψ p , 0 ′ , ϕ ∞ = ψ p , ∞ ′ \phi_\infty=\psi'_{p,\infty} ϕ ∞ = ψ p , ∞ ′ , and
ψ p ′ ( t ) = t p − 1 − 1 p − 1 and ψ p ′ ′ ( t ) = t p − 2 ( t ∈ R + + ) . \psi'_p(t)=\frac{t^{p-1}-1}{p-1}\qquad\text{and}\qquad\psi''_p(t)=t^{p-2}\qquad(t\in\mathbb{R}_{++}). ψ p ′ ( t ) = p − 1 t p − 1 − 1 and ψ p ′′ ( t ) = t p − 2 ( t ∈ R ++ ) .
Via a classic calculus argument, we have the following.
Lemma 3. Let M > 0 M>0 M > 0 and set B p ( y , x ) = ψ p ( y ) − ψ p ( x ) − ψ p ′ ( x ) ( y − x ) B_p(y,x)=\psi_p(y)-\psi_p(x)-\psi'_p(x)(y-x) B p ( y , x ) = ψ p ( y ) − ψ p ( x ) − ψ p ′ ( x ) ( y − x ) , then
(i) If p ≥ 2 p\ge2 p ≥ 2 for x , y ∈ [ 0 , M ] x,y\in[0,M] x , y ∈ [ 0 , M ] , it holds
B p ( y , x ) ≥ ∣ y − x ∣ p p ( p − 1 ) , ∣ ψ p ′ ( y ) − ψ p ′ ( x ) ∣ ≤ M p − 2 ∣ y − x ∣ . B_p(y,x)\ge\frac{|y-x|^p}{p(p-1)},\qquad|\psi'_p(y)-\psi'_p(x)|\le M^{p-2}|y-x|. B p ( y , x ) ≥ p ( p − 1 ) ∣ y − x ∣ p , ∣ ψ p ′ ( y ) − ψ p ′ ( x ) ∣ ≤ M p − 2 ∣ y − x ∣.
(ii) If 1 < p ≤ 2 1<p\le2 1 < p ≤ 2 for x , y ∈ [ 0 , M ] x,y\in[0,M] x , y ∈ [ 0 , M ] , it holds
B p ( y , x ) ≥ M p − 2 2 ∣ y − x ∣ 2 , ∣ ψ p ′ ( y ) − ψ p ′ ( x ) ∣ ≤ ∣ y − x ∣ p − 1 p − 1 . B_p(y,x)\ge\frac{M^{p-2}}{2}|y-x|^2,\qquad|\psi'_p(y)-\psi'_p(x)|\le\frac{|y-x|^{p-1}}{p-1}. B p ( y , x ) ≥ 2 M p − 2 ∣ y − x ∣ 2 , ∣ ψ p ′ ( y ) − ψ p ′ ( x ) ∣ ≤ p − 1 ∣ y − x ∣ p − 1 .
(iii) If p ≤ 1 p\le1 p ≤ 1 and 0 < m 0 < M 0<m_0<M 0 < m 0 < M , the derivative is Lipschitz and the function is strongly convex on [ m 0 , M ] [m_0,M] [ m 0 , M ] , with respective constants m 0 p − 2 m_0^{p-2} m 0 p − 2 and M p − 2 M^{p-2} M p − 2 .
Proof. For p ≥ 2 p\ge2 p ≥ 2 and y ≥ x y\ge x y ≥ x , writing d = y − x d=y-x d = y − x gives
B p ( y , x ) = ∫ 0 d ( d − t ) ( x + t ) p − 2 d t ≥ ∫ 0 d ( d − t ) t p − 2 d t = d p p ( p − 1 ) . B_p(y,x)=\int_0^d(d-t)(x+t)^{p-2}\,dt\ge\int_0^d(d-t)t^{p-2}\,dt=\frac{d^p}{p(p-1)}. B p ( y , x ) = ∫ 0 d ( d − t ) ( x + t ) p − 2 d t ≥ ∫ 0 d ( d − t ) t p − 2 d t = p ( p − 1 ) d p .
A similar argument handles the case in which y < x y<x y < x , concluding the first part of the statement. For 1 < p ≤ 2 1<p\le2 1 < p ≤ 2 , we have ψ p ′ ′ ( t ) ≥ M p − 2 \psi''_p(t)\ge M^{p-2} ψ p ′′ ( t ) ≥ M p − 2 on [ 0 , M ] [0,M] [ 0 , M ] , which proves the quadratic bound by integration. The inequality ∣ a r − b r ∣ ≤ ∣ a − b ∣ r |a^r-b^r|\le|a-b|^r ∣ a r − b r ∣ ≤ ∣ a − b ∣ r for a , b ≥ 0 a,b\ge0 a , b ≥ 0 and 0 < r ≤ 1 0<r\le1 0 < r ≤ 1 follows from ( a + b ) r ≤ a r + b r (a+b)^r\le a^r+b^r ( a + b ) r ≤ a r + b r , and proves the derivative estimate with r = p − 1 r=p-1 r = p − 1 . The last statement of the theorem follows by integrating M p − 2 ≤ ψ p ′ ′ ( t ) ≤ m 0 p − 2 M^{p-2}\le\psi''_p(t)\le m_0^{p-2} M p − 2 ≤ ψ p ′′ ( t ) ≤ m 0 p − 2 .
For p > 1 p>1 p > 1 , we then have that ψ p ′ \psi'_p ψ p ′ is both q = max { 2 , p } q=\max\{2,p\} q = max { 2 , p } conditioned and has min { p − 1 , 1 } \min\{p-1,1\} min { p − 1 , 1 } -Hölder continuous gradient on any set of the form [ 0 , M ] [0,M] [ 0 , M ] . If p ≤ 0 p\le0 p ≤ 0 , the derivative of ψ p \psi_p ψ p is not α \alpha α -Hölder on any set of the form [ 0 , M ] [0,M] [ 0 , M ] . However, since ψ p ′ ( t ) → − ∞ \psi'_p(t)\to-\infty ψ p ′ ( t ) → − ∞ as t ↓ 0 t\downarrow0 t ↓ 0 , every finite initial sub-level set is bounded away from zero at every positive-reference marginal allowing us to bypass problematic points by considering a suitable sub-level set. The remaining case 0 < p ≤ 1 0<p\le1 0 < p ≤ 1 the sub-level set approach does not apply since ψ p \psi_p ψ p has bounded values in 0 but its derivative is not α \alpha α -Hölder on any finite sub-level set. To bypass this limitation, we introduce an additive barrier term to ψ p \psi_p ψ p that preserves the solutions to the induced UOT problem and allows us to handle the resulting entropies as done for p ≤ 0 p\le0 p ≤ 0 .
Proposition 7. Let R > 0 R > 0 R > 0 and 0 < p ≤ 1 0 < p \le1 0 < p ≤ 1 , then there exists a positive value t 0 t_0 t 0 such that every optimizer of G R G_R G R on P R P_R P R satisfies r i / μ i ≥ t 0 r_i/\mu_i \ge t_0 r i / μ i ≥ t 0 if μ i > 0 \mu_i > 0 μ i > 0 and s j / ν j ≥ t 0 s_j/\nu_j \ge t_0 s j / ν j ≥ t 0 if ν j > 0 \nu_j > 0 ν j > 0 . Then, if δ = t 0 / 2 \delta= t_0/2 δ = t 0 /2 and ζ > 0 \zeta> 0 ζ > 0 , we set
b δ ( t ) = { log ( δ / t ) + t / δ − 1 , 0 < t < δ , 0 , t ≥ δ , + ∞ , t ≤ 0 , ψ ~ p = ψ p + ζ b δ . (106) b_\delta(t) =
\begin{cases}
\log(\delta/t) + t/\delta- 1, & 0 < t < \delta,\\
0, & t \ge\delta,\\
+\infty, & t \le0,
\end{cases}
\qquad\widetilde{\psi}_p = \psi_p + \zeta b_\delta. \tag*{(106)} b δ ( t ) = ⎩ ⎨ ⎧ log ( δ / t ) + t / δ − 1 , 0 , + ∞ , 0 < t < δ , t ≥ δ , t ≤ 0 , ψ p = ψ p + ζ b δ . (106) Replacing every positive-reference perspective induced by ψ p \psi_p ψ p with the one induced by ψ ~ p \widetilde{\psi}_p ψ p , we obtain an objective G ~ R ≥ G R \widetilde{G}_R \ge G_R G R ≥ G R with the same minimum and exactly the same minimizers on P R P_R P R . Moreover ψ ~ p \widetilde{\psi}_p ψ p has a Lipschitz gradient and is strongly convex on each finite sub-level set.
Proof. The first part of the proof follows from Theorem 5 with U i = V j = 2 B 0 U_i = V_j = 2B_0 U i = V j = 2 B 0 .
Let us then consider the function b δ b_\delta b δ defined as in (6.19). By definition, b δ b_\delta b δ is nonnegative, convex and C 1 C^1 C 1 on ] 0 , + ∞ [ ]0,+\infty[ ] 0 , + ∞ [ . If t < δ t < \delta t < δ the derivative of b δ b_\delta b δ is − 1 / t + 1 / δ -1/t + 1/\delta − 1/ t + 1/ δ and its second derivative is 1 / t 2 1/t^2 1/ t 2 , so that b δ b_\delta b δ is continuous and has a continuous first derivative.
Since b δ ≥ 0 b_\delta\ge0 b δ ≥ 0 we have ψ ~ p ≥ ψ p \widetilde{\psi}_p \ge\psi_p ψ p ≥ ψ p , thus G ~ R ≥ G R ≥ v R \widetilde{G}_R \ge G_R \ge v_R G R ≥ G R ≥ v R for every ( η , u , w ) ∈ F R (\eta,u,w) \in\mathcal{F}_R ( η , u , w ) ∈ F R . Since b δ ( t ) = 0 b_\delta(t) = 0 b δ ( t ) = 0 for every t ≥ δ t \ge\delta t ≥ δ , we have that G ~ R ( η ⋆ , u ⋆ , w ⋆ ) = G R ( η ⋆ , u ⋆ , w ⋆ ) = v R \widetilde{G}_R(\eta^\star,u^\star,w^\star) = G_R(\eta^\star,u^\star,w^\star) = v_R G R ( η ⋆ , u ⋆ , w ⋆ ) = G R ( η ⋆ , u ⋆ , w ⋆ ) = v R for every ( η ⋆ , u ⋆ , w ⋆ ) ∈ argmin G R (\eta^\star,u^\star,w^\star) \in\operatorname*{argmin} G_R ( η ⋆ , u ⋆ , w ⋆ ) ∈ argmin G R . We then infer that ( η ⋆ , u ⋆ , w ⋆ ) ∈ argmin G ~ R (\eta^\star,u^\star,w^\star) \in\operatorname*{argmin} \widetilde{G}_R ( η ⋆ , u ⋆ , w ⋆ ) ∈ argmin G R . Conversely, a minimizer of G ~ R \widetilde{G}_R G R must also minimize G R G_R G R , since G R ≤ G ~ R G_R \le\widetilde{G}_R G R ≤ G R and min G ~ R = v R \min\widetilde{G}_R = v_R min G R = v R .
To conclude, notice that on any finite sub-level set, every positive-reference marginal is bounded away from zero, while all marginals are at most 2 B 0 2B_0 2 B 0 . On such compact intervals the derivative of ψ ~ p \widetilde{\psi}_p ψ p is Lipschitz and its curvature is bounded below by the positive minimum of ψ p ′ ′ \psi_p'' ψ p ′′ . Scaling the perspectives and summing proves the last assertion, concluding the proof. □ \square □
Proposition 7 allows us to apply the results presented for the generic PFW algorithm to every UOT problem induced by an entropy-like penalty function.
Theorem 12. Given P = P R P = P_R P = P R , for p ∈ R p \in\mathbb{R} p ∈ R consider
F p = { G R , p ≤ 0 or p > 1 , G ~ R , 0 < p ≤ 1 and F p ⋆ = v p , R . (107) F_p =
\begin{cases}
G_R, & p \le0 \text{ or } p > 1,\\
\widetilde{G}_R, & 0 < p \le1
\end{cases}
\qquad\text{and}\qquad
F_p^\star= v_{p,R}. \tag*{(107)} F p = { G R , G R , p ≤ 0 or p > 1 , 0 < p ≤ 1 and F p ⋆ = v p , R . (107) Let x 0 x_0 x 0 be a finite-value initial point and let x ( k ) x^{(k)} x ( k ) be the sequence generated by the PFW algorithm combined with a stepsize as in Proposition 5. Then, the hypotheses of Theorems 10 and 11 hold, and the following rates apply
F p ( x ( k ) ) − v p , R = { O ( t k − p p − 2 ) , p > 2 , O ( ( 1 − θ ) t k ) , p = 2 , O ( t k 2 ( p − 1 ) 2 − p ) , 1 < p < 2 , O ( ( 1 − θ p ) t k ) , p ≤ 1. (108) F_p(x^{(k)}) - v_{p,R} =
\begin{cases}
O\left(t_k^{-\frac{p}{p-2}}\right), & p > 2,\\
O\left((1-\theta)^{t_k}\right), & p = 2,\\
O\left(t_k^{\frac{2(p-1)}{2-p}}\right), & 1 < p < 2,\\
O\left((1-\theta_p)^{t_k}\right), & p \le1.
\end{cases} \tag*{(108)} F p ( x ( k ) ) − v p , R = ⎩ ⎨ ⎧ O ( t k − p − 2 p ) , O ( ( 1 − θ ) t k ) , O ( t k 2 − p 2 ( p − 1 ) ) , O ( ( 1 − θ p ) t k ) , p > 2 , p = 2 , 1 < p < 2 , p ≤ 1. (108) where θ p ∈ ( 0 , 1 ) \theta_p \in(0,1) θ p ∈ ( 0 , 1 ) depends on p ≤ 1 p \le1 p ≤ 1 and t k t_k t k is the number of good steps made up to iterate k k k .
Proof. Let A be the linear function defined as
A ( z ) = A ( η , u , w ) = ( P 1 η + u , P 2 η + w ) . \mathrm{A}(z) = \mathrm{A}(\eta,u,w) = (P_1\eta+ u, P_2\eta+ w). A ( z ) = A ( η , u , w ) = ( P 1 η + u , P 2 η + w ) .
Then, we have F ( z ) = H ( A z ) + ⟨ b , z ⟩ F(z)=\mathcal{H}(Az)+\langle b,z\rangle F ( z ) = H ( A z ) + ⟨ b , z ⟩ , for a suitable vector b b b and H \mathcal{H} H is the sum of perspective functions associated with a positive μ i \mu_i μ i or a positive ν j \nu_j ν j and induced by a suitable q q q -uniform convex function.
From our assumptions, H \mathcal{H} H is not a null function since at least one of μ \mu μ and ν \nu ν is not null and, for a given reference weight m m m , the perspective is φ m ( t ) = m ψ p ( t / m ) \varphi_m(t)=m\psi_p(t/m) φ m ( t ) = m ψ p ( t / m ) . Moreover, for each φ m \varphi_m φ m with m > 0 m>0 m > 0 , we have φ m ′ ′ ( t ) = m 1 − p t p − 2 \varphi_m''(t)=m^{1-p}t^{p-2} φ m ′′ ( t ) = m 1 − p t p − 2 so that Lemma 3 ensures φ m \varphi_m φ m is q = max { 2 , p } q=\max\{2,p\} q = max { 2 , p } conditioned over the set [ 0 , 2 B 0 ] [0,2B_0] [ 0 , 2 B 0 ] if p > 1 p>1 p > 1 . For p ≤ 1 p\leq1 p ≤ 1 , we can restrict our attention to a sub-level set to ensure the objective is 2-conditioned. On the same sets, the function has also Lipschitz continuous gradient if p ≥ 2 p\geq2 p ≥ 2 , Hölder continuous if 1 < p < 2 1<p<2 1 < p < 2 or Lipschitz continuous on the sub-level set.
These properties readily translate to H \mathcal{H} H since it is the sum of the perspectives. Indeed, denoted by d + > 0 d_+>0 d + > 0 the number of positive entries, we have
∥ ∇ H ( y ) − ∇ H ( z ) ∥ 2 ≤ L 0 2 ∑ i = 1 d + ∣ y i − z i ∣ 2 α ≤ L 0 2 d + 1 − α ∥ y − z ∥ 2 α , \lVert\nabla\mathcal{H}(y)-\nabla\mathcal{H}(z)\rVert^2\leq L_0^2\sum_{i=1}^{d_+}|y_i-z_i|^{2\alpha}\leq L_0^2d_+^{1-\alpha}\lVert y-z\rVert^{2\alpha}, ∥ ∇ H ( y ) − ∇ H ( z ) ∥ 2 ≤ L 0 2 i = 1 ∑ d + ∣ y i − z i ∣ 2 α ≤ L 0 2 d + 1 − α ∥ y − z ∥ 2 α ,
where α ∈ ] 0 , 1 ] \alpha\in]0,1] α ∈ ] 0 , 1 ] and L 0 L_0 L 0 is a suitable constant that depends on ψ p \psi_p ψ p . In particular, F F F has Hölder continuous gradient with positive constant L 0 d + ( 1 − α ) / 2 ∥ A ∥ 1 + α L_0d_+^{(1-\alpha)/2}\lVert A\rVert^{1+\alpha} L 0 d + ( 1 − α ) /2 ∥ A ∥ 1 + α .
Also, since H \mathcal{H} H is the sum of independent q q q -uniform convex functions, it is also q q q -uniform convex. Thus Proposition 6 allows us to conclude that F F F satisfies Assumption 1 and thus Theorem 10 is applicable. The convergence rates follow by substituting the appropriate values ( q , α ) (q,\alpha) ( q , α ) in (6.12). □
Remark 7 (Generality beyond the power family). While we presented our results for powerlike entropies, the same approach used by us can be applied to any UOT problem in which the function ϕ \phi ϕ induces a Csiszár divergence that satisfies Assumption 1 and whose composed gradient is α \alpha α -Hölder continuous.
Reconstruction and a computable stopping certificate
In this section, we show how to use the reconstruction functions L, B, D to attain an optimality certificate that allow to relate the minimum of G R G_R G R and F c F_c F c .
Proposition 8 (Certificates for inexact compressed iterates). Let z = ( η , u , w ) ∈ P R z=(\eta,u,w)\in\mathcal{P}_R z = ( η , u , w ) ∈ P R have finite objective value. For each X ∈ { L , D , B } X\in\{\mathrm{L},\mathrm{D},\mathrm{B}\} X ∈ { L , D , B } , we have
F c ( X ( z ) ) − G ~ R ( z ) ≤ R X ( z ) . (109) F_c(\mathrm{X}(z))-\widetilde{G}_R(z)\leq\mathcal{R}_{\mathrm{X}}(z). \tag*{(109)} F c ( X ( z )) − G R ( z ) ≤ R X ( z ) . (109) Moreover,
0 ≤ F c ( X ( z ) ) − v c ≤ G ~ R ( z ) − v R + R X ( z ) ≤ g R F W ( z ) + R X ( z ) , (110) 0\leq F_c(\mathrm{X}(z))-v_c\leq\widetilde{G}_R(z)-v_R+\mathcal{R}_{\mathrm{X}}(z)\leq g_R^{\mathrm{FW}}(z)+\mathcal{R}_{\mathrm{X}}(z), \tag*{(110)} 0 ≤ F c ( X ( z )) − v c ≤ G R ( z ) − v R + R X ( z ) ≤ g R FW ( z ) + R X ( z ) , (110) where v R v_R v R the minimum objective of G ~ R \widetilde{G}_R G R on P R \mathcal{P}_R P R and g R F W ( z ) g_R^{\mathrm{FW}}(z) g R FW ( z ) is the Frank-Wolfe gap of the objective G ~ R \widetilde{G}_R G R at point z z z .
Proof. The inequality in (6.22) follows from the same argument used in Theorem 7 applied to a generic element ( η , u , w ) ∈ F R (\eta,u,w)\in\mathcal{F}_R ( η , u , w ) ∈ F R . Then, since v R ≤ v c v_R\leq v_c v R ≤ v c , we have
F c ( X ( z ) ) − v c ≤ G ~ R ( z ) − v R + R X ( z ) ≤ g R F W ( z ) + R X ( z ) , F_c(\mathrm{X}(z))-v_c\leq\widetilde{G}_R(z)-v_R+\mathcal{R}_{\mathrm{X}}(z)\leq g_R^{\mathrm{FW}}(z)+\mathcal{R}_{\mathrm{X}}(z), F c ( X ( z )) − v c ≤ G R ( z ) − v R + R X ( z ) ≤ g R FW ( z ) + R X ( z ) ,
where in the last inequality we used the standard Frank-Wolfe gap bound. □
Notice that a similar result holds for G ~ R \widetilde{G}_R G R . In the end, we are able to evaluate the available certificates and return the corresponding reconstructed plan as soon as
g R F W ( z k ) + min X ∈ { L , B , D } R X ( z k ) ≤ ε . (111) g_R^{\mathrm{FW}}(z_k)+\min_{X\in\{\mathrm{L,B,D}\}}\mathcal{R}_X(z_k)\le\varepsilon. \tag*{(111)} g R FW ( z k ) + X ∈ { L , B , D } min R X ( z k ) ≤ ε . (111)
This certifies the objective error for the original objective F c F_c F c via quantities that depend on R R R and/or on G ~ R \widetilde{G}_R G R . To conclude, we use these optimality certificates to provide a bound on the number of iterates needed to identify the support of the solution.
Theorem 13. Suppose the hypotheses of Theorem 10 hold for a working objective G ~ R \widetilde{G}_R G R whose minimum is v R v_R v R . Define
E B 0 ( R ) : = B 0 min { [ c max − R ] + , [ − 2 ϕ 0 ′ − R ] + , if ϕ 0 ′ > − ∞ , [ max i c i , min + max j c min , j + 2 ϕ ∞ ′ − R ] + , if ϕ ∞ ′ < + ∞ . (112) E_{B_0}(R):=B_0\min\left\{
\begin{array}{ll}
[c_{\max}-R]_+,\\
[-2\phi'_0-R]_+, & \text{if }\phi'_0>-\infty,\\
[\max_i c_{i,\min}+\max_j c_{\min,j}+2\phi'_\infty-R]_+, & \text{if }\phi'_\infty<+\infty.
\end{array}
\right. \tag*{(112)} E B 0 ( R ) := B 0 min ⎩ ⎨ ⎧ [ c m a x − R ] + , [ − 2 ϕ 0 ′ − R ] + , [ max i c i , m i n + max j c m i n , j + 2 ϕ ∞ ′ − R ] + , if ϕ 0 ′ > − ∞ , if ϕ ∞ ′ < + ∞. (112) take ε > 0 \varepsilon>0 ε > 0 , 0 < ϱ ≤ 1 0<\varrho\le1 0 < ϱ ≤ 1 such that E B 0 ( R ) ≤ ( 1 − ϱ ) ε E_{B_0}(R)\le(1-\varrho)\varepsilon E B 0 ( R ) ≤ ( 1 − ϱ ) ε , and any starting point x 0 x_0 x 0 . If c 0 ≥ h 0 ≥ ϱ ε c_0\ge h_0\ge\varrho\varepsilon c 0 ≥ h 0 ≥ ϱε , we have that after T ( ϱ ε ) T(\varrho\varepsilon) T ( ϱε ) good steps, reconstruction by a method attaining the minimum in (6.25) gives F c ( π ) − v c ≤ ε F_c(\pi)-v_c\le\varepsilon F c ( π ) − v c ≤ ε , where
T ( ϱ ε ) : = { ⌈ ( ϱ ε ) − s − c 0 − s s c ∗ ⌉ , s > 0 , ⌈ log ( c 0 / ϱ ε ) − log ( 1 − θ ) ⌉ , s = 0. (113) T(\varrho\varepsilon):=
\begin{cases}
\left\lceil\frac{(\varrho\varepsilon)^{-s}-c_0^{-s}}{s c_*}\right\rceil, & s>0,\\
\left\lceil\frac{\log(c_0/\varrho\varepsilon)}{-\log(1-\theta)}\right\rceil, & s=0.
\end{cases} \tag*{(113)} T ( ϱε ) := ⎩ ⎨ ⎧ ⌈ s c ∗ ( ϱε ) − s − c 0 − s ⌉ , ⌈ − l o g ( 1 − θ ) l o g ( c 0 / ϱε ) ⌉ , s > 0 , s = 0. (113)
Proof. Since τ ≤ B 0 \tau\le B_0 τ ≤ B 0 , Proposition 8 bounds each error by a term in (6.25). For the redirecting plan use ∑ i c i , min u i ≤ τ max i c i , min \sum_i c_{i,\min}u_i\le\tau\max_i c_{i,\min} ∑ i c i , m i n u i ≤ τ max i c i , m i n and ∑ j c min , j w j ≤ τ max j c min , j \sum_j c_{\min,j}w_j\le\tau\max_j c_{\min,j} ∑ j c m i n , j w j ≤ τ max j c m i n , j . For the selected reconstruction, (6.23) gives F c ( π ˉ ) − v c ≤ h k + E B 0 ( R ) F_c(\bar{\pi})-v_c\le h_k+E_{B_0}(R) F c ( π ˉ ) − v c ≤ h k + E B 0 ( R ) .
If s > 0 s>0 s > 0 , h 0 ≤ c 0 h_0\le c_0 h 0 ≤ c 0 and (6.21) imply h k ≤ ( c 0 − s + s c ∗ t k ) − 1 / s h_k\le(c_0^{-s}+sc_*t_k)^{-1/s} h k ≤ ( c 0 − s + s c ∗ t k ) − 1/ s . Solving for this bound to be at most ϱ ε \varrho\varepsilon ϱε gives the first line of (6.26). Solving c 0 ( 1 − θ ) t k ≤ ϱ ε c_0(1-\theta)^{t_k}\le\varrho\varepsilon c 0 ( 1 − θ ) t k ≤ ϱε gives the second line. The total error is then at most ϱ ε + ( 1 − ϱ ) ε = ε \varrho\varepsilon+(1-\varrho)\varepsilon=\varepsilon ϱε + ( 1 − ϱ ) ε = ε . Remark 5 proves the actual-iteration count. Early termination means h k = 0 h_k=0 h k = 0 and hence also suffices.
Proof of the support identification property
Lemma 4. Given a continuously differentiable function F F F over the simplex Δ n \Delta_n Δ n , let x ⋆ ∈ Δ n x^\star\in\Delta_n x ⋆ ∈ Δ n be a minimizer of F F F . Given h > 0 h > 0 h > 0 , let x ( k ) ∈ Δ n x^{(k)} \in\Delta_n x ( k ) ∈ Δ n be such that ∥ x ⋆ − x ( k ) ∥ 1 ≤ h \lVert x^\star- x^{(k)} \rVert_1 \le h ∥ x ⋆ − x ( k ) ∥ 1 ≤ h , then, if F F F has α \alpha α -Hölder continuous gradient over B h , 1 ( x ⋆ ) ∩ Δ n B_{h,1}(x^\star) \cap\Delta_n B h , 1 ( x ⋆ ) ∩ Δ n , we have
∣ θ i ( x ⋆ ) − θ i ( x ( k ) ) ∣ ≤ h α ( H n 1 2 + δ k h 1 − α 2 ) , (114) \lvert\theta_i(x^\star) - \theta_i(x^{(k)}) \rvert\le h^\alpha\left(H n^{\frac{1}{2}} + \frac{\delta_k h^{1-\alpha}}{2}\right), \tag*{(114)} ∣ θ i ( x ⋆ ) − θ i ( x ( k ) )∣ ≤ h α ( H n 2 1 + 2 δ k h 1 − α ) , (114) where H H H is the Hölder constant associated with the gradient and δ k = max i ∈ [ n ] ∖ O k θ i ( x ⋆ ) \delta_k = \max_{i \in[n] \setminus O_k} \theta_i(x^\star) δ k = max i ∈ [ n ] ∖ O k θ i ( x ⋆ )
Remark 8. Notice that, if α = 1 \alpha= 1 α = 1 , the result is the same as the one presented in [11 ] . Moreover, if h < 1 h < 1 h < 1 , equation (A.1) simplifies into
∣ θ i ( x ⋆ ) − θ i ( x ( k ) ) ∣ ≤ h α ( H n 1 2 + δ k 2 ) , (115) \lvert\theta_i(x^\star) - \theta_i(x^{(k)}) \rvert\le h^\alpha\left(H n^{\frac{1}{2}} + \frac{\delta_k}{2}\right), \tag*{(115)} ∣ θ i ( x ⋆ ) − θ i ( x ( k ) )∣ ≤ h α ( H n 2 1 + 2 δ k ) , (115)
which is equivalent to the condition presented in [11 ] , whose Lipschitz constant is computed with respect to the ∥ ⋅ ∥ 1 \lVert\cdot\rVert_1 ∥ ⋅ ∥ 1 norm.
Proof. By definition of θ i ( x ) \theta_i(x) θ i ( x ) , we have
∣ θ i ( x ⋆ ) − θ i ( x ( k ) ) ∣ = ∣ ⟨ ∇ F ( x ( k ) ) − ∇ F ( x ⋆ ) , e i − x ( k ) ⟩ − ⟨ ∇ F ( x ⋆ ) , x ( k ) − x ⋆ ⟩ ∣ ≤ ∣ ⟨ ∇ F ( x ( k ) ) − ∇ F ( x ⋆ ) , e i − x ( k ) ⟩ ∣ + ∣ ⟨ ∇ F ( x ⋆ ) , x ( k ) − x ⋆ ⟩ ∣ ≤ ∥ ∇ F ( x ( k ) ) − ∇ F ( x ⋆ ) ∥ ∥ e i − x ( k ) ∥ + ∣ ⟨ ∇ F ( x ⋆ ) , x ( k ) − x ⋆ ⟩ ∣ ≤ H ∥ x ⋆ − x ( k ) ∥ α n 1 2 + ∣ ⟨ ∇ F ( x ⋆ ) , x ( k ) − x ⋆ ⟩ ∣ ≤ H n 1 2 h α + ∣ ⟨ ∇ F ( x ⋆ ) , x ( k ) − x ⋆ ⟩ ∣ , \begin{aligned}
|\theta_i(x^\star)-\theta_i(x^{(k)})| &= |\langle\nabla F(x^{(k)})-\nabla F(x^\star),e_i-x^{(k)}\rangle-\langle\nabla F(x^\star),x^{(k)}-x^\star\rangle|\\
&\leq|\langle\nabla F(x^{(k)})-\nabla F(x^\star),e_i-x^{(k)}\rangle|+|\langle\nabla F(x^\star),x^{(k)}-x^\star\rangle|\\
&\leq\lVert\nabla F(x^{(k)})-\nabla F(x^\star)\rVert\lVert e_i-x^{(k)}\rVert+|\langle\nabla F(x^\star),x^{(k)}-x^\star\rangle|\\
&\leq H\lVert x^\star-x^{(k)}\rVert^\alpha n^{\frac{1}{2}}+|\langle\nabla F(x^\star),x^{(k)}-x^\star\rangle|\\
&\leq Hn^{\frac{1}{2}}h^\alpha+|\langle\nabla F(x^\star),x^{(k)}-x^\star\rangle|,
\end{aligned} ∣ θ i ( x ⋆ ) − θ i ( x ( k ) ) ∣ = ∣ ⟨ ∇ F ( x ( k ) ) − ∇ F ( x ⋆ ) , e i − x ( k ) ⟩ − ⟨ ∇ F ( x ⋆ ) , x ( k ) − x ⋆ ⟩ ∣ ≤ ∣ ⟨ ∇ F ( x ( k ) ) − ∇ F ( x ⋆ ) , e i − x ( k ) ⟩ ∣ + ∣ ⟨ ∇ F ( x ⋆ ) , x ( k ) − x ⋆ ⟩ ∣ ≤ ∥ ∇ F ( x ( k ) ) − ∇ F ( x ⋆ )∥ ∥ e i − x ( k ) ∥ + ∣ ⟨ ∇ F ( x ⋆ ) , x ( k ) − x ⋆ ⟩ ∣ ≤ H ∥ x ⋆ − x ( k ) ∥ α n 2 1 + ∣ ⟨ ∇ F ( x ⋆ ) , x ( k ) − x ⋆ ⟩ ∣ ≤ H n 2 1 h α + ∣ ⟨ ∇ F ( x ⋆ ) , x ( k ) − x ⋆ ⟩ ∣ ,
where in the last inequality we have used the hypothesis on ∥ x ⋆ − x ( k ) ∥ 1 \lVert x^\star-x^{(k)}\rVert_1 ∥ x ⋆ − x ( k ) ∥ 1 . We now provide an estimate of the term ∣ ⟨ ∇ F ( x ⋆ ) , x ( k ) − x ⋆ ⟩ ∣ |\langle\nabla F(x^\star),x^{(k)}-x^\star\rangle| ∣ ⟨ ∇ F ( x ⋆ ) , x ( k ) − x ⋆ ⟩ ∣ . Since both x ( k ) x^{(k)} x ( k ) and x ⋆ x^\star x ⋆ belong to Δ n \Delta_n Δ n , we have
0 = ∑ i = 1 n ( x i ( k ) − x i ⋆ ) ⟺ ∑ i = 1 n ( x i ( k ) − x i ⋆ ) + = ∑ i = 1 n ( x i ( k ) − x i ⋆ ) − , (116) 0=\sum_{i=1}^{n}(x_i^{(k)}-x_i^\star)\quad\Longleftrightarrow\quad\sum_{i=1}^{n}(x_i^{(k)}-x_i^\star)_+=\sum_{i=1}^{n}(x_i^{(k)}-x_i^\star)_-, \tag*{(116)} 0 = i = 1 ∑ n ( x i ( k ) − x i ⋆ ) ⟺ i = 1 ∑ n ( x i ( k ) − x i ⋆ ) + = i = 1 ∑ n ( x i ( k ) − x i ⋆ ) − , (116)
where ( t ) + (t)_+ ( t ) + and ( t ) − (t)_- ( t ) − denote the positive and negative part of t t t . In particular, for every ℓ ∈ [ n ] \ell\in[n] ℓ ∈ [ n ] ,
∣ x ℓ ( k ) − x ℓ ⋆ ∣ ≤ ∑ i = 1 n ( x i ( k ) − x i ⋆ ) + = 1 2 ∥ x ( k ) − x ⋆ ∥ 1 ≤ h 2 . (117) |x_\ell^{(k)}-x_\ell^\star|\leq\sum_{i=1}^{n}(x_i^{(k)}-x_i^\star)_+=\frac{1}{2}\lVert x^{(k)}-x^\star\rVert_1\leq\frac{h}{2}. \tag*{(117)} ∣ x ℓ ( k ) − x ℓ ⋆ ∣ ≤ i = 1 ∑ n ( x i ( k ) − x i ⋆ ) + = 2 1 ∥ x ( k ) − x ⋆ ∥ 1 ≤ 2 h . (117)
Finally, since x ⋆ x^\star x ⋆ is optimal, we have ∣ ⟨ ∇ F ( x ⋆ ) , x ( k ) − x ⋆ ⟩ ∣ ≤ ( ∇ F ( x ⋆ ) M − ∇ F ( x ⋆ ) m ) h 2 ≤ δ k h 2 |\langle\nabla F(x^\star),x^{(k)}-x^\star\rangle|\leq(\nabla F(x^\star)_M-\nabla F(x^\star)_m)\frac{h}{2}\leq\delta_k\frac{h}{2} ∣ ⟨ ∇ F ( x ⋆ ) , x ( k ) − x ⋆ ⟩ ∣ ≤ ( ∇ F ( x ⋆ ) M − ∇ F ( x ⋆ ) m ) 2 h ≤ δ k 2 h , where ∇ F ( x ⋆ ) M \nabla F(x^\star)_M ∇ F ( x ⋆ ) M and ∇ F ( x ⋆ ) m \nabla F(x^\star)_m ∇ F ( x ⋆ ) m are the maximum and minimum entries of ∇ F \nabla F ∇ F over [ n ] ∖ O k [n]\setminus O_k [ n ] ∖ O k , respectively. We then conclude
∣ θ i ( x ⋆ ) − θ i ( x ( k ) ) ∣ ≤ H n 1 2 h α + δ k 2 h = h α ( H n 1 2 + δ k h 1 − α 2 ) . (118) |\theta_i(x^\star)-\theta_i(x^{(k)})|\leq Hn^{\frac{1}{2}}h^\alpha+\frac{\delta_k}{2}h=h^\alpha\left(Hn^{\frac{1}{2}}+\frac{\delta_kh^{1-\alpha}}{2}\right). \tag*{(118)} ∣ θ i ( x ⋆ ) − θ i ( x ( k ) ) ∣ ≤ H n 2 1 h α + 2 δ k h = h α ( H n 2 1 + 2 δ k h 1 − α ) . (118)
Theorem 14. Let F F F be a continuously differentiable function whose gradient is α \alpha α -Hölder continuous with constant H H H over an ε \varepsilon ε -neighborhood of the set of optimal solutions in Δ n \Delta_n Δ n . For every k k k , let x ( k ) x^{(k)} x ( k ) and d ( k ) d^{(k)} d ( k ) be the k k k -th iterate and the direction found at x ( k ) x^{(k)} x ( k ) by the PFW algorithm. Denoted by γ k \gamma_k γ k the step size at step k k k , assume that for every k k k either
γ k = γ k max or γ k ≥ ( − ⟨ ∇ f ( x ( k ) ) , d ( k ) ⟩ 2 H ) 1 α . \gamma_k=\gamma_k^{\max}\quad\text{or}\quad\gamma_k\geq\left(-\frac{\langle\nabla f(x^{(k)}),d^{(k)}\rangle}{2H}\right)^{\frac{1}{\alpha}}. γ k = γ k m a x or γ k ≥ ( − 2 H ⟨ ∇ f ( x ( k ) ) , d ( k ) ⟩ ) α 1 . Finally, given x ⋆ ∈ X ⋆ x^\star\in\mathcal{X}^\star x ⋆ ∈ X ⋆ , set
δ min : = min { θ i ( x ⋆ ) s.t. θ i ( x ⋆ ) > 0 } and δ max : = max { θ i ( x ⋆ ) s.t. θ i ( x ⋆ ) > 0 } . \delta_{\min}:=\min\{\theta_i(x^\star)\ \text{s.t.}\ \theta_i(x^\star)>0\}\quad\text{and}\quad\delta_{\max}:=\max\{\theta_i(x^\star)\ \text{s.t.}\ \theta_i(x^\star)>0\}. δ m i n := min { θ i ( x ⋆ ) s.t. θ i ( x ⋆ ) > 0 } and δ m a x := max { θ i ( x ⋆ ) s.t. θ i ( x ⋆ ) > 0 } . Then, if ∥ x ( k ) − x ⋆ ∥ 1 < r ⋆ = min { ε , 2 α − 1 α ( δ min 2 H n 1 2 + δ max ) 1 α } \lVert x^{(k)}-x^\star\rVert_1<r^\star=\min\left\{\varepsilon,2^{\frac{\alpha-1}{\alpha}}\left(\frac{\delta_{\min}}{2Hn^{\frac{1}{2}}+\delta_{\max}}\right)^{\frac{1}{\alpha}}\right\} ∥ x ( k ) − x ⋆ ∥ 1 < r ⋆ = min { ε , 2 α α − 1 ( 2 H n 2 1 + δ m a x δ m i n ) α 1 } , we have
∣ J k + 1 ∣ ≤ max { ∣ J k ∣ − 1 , 0 } . |J_{k+1}|\leq\max\{|J_k|-1,0\}. ∣ J k + 1 ∣ ≤ max { ∣ J k ∣ − 1 , 0 } .
Proof. Let us denote by J k J_k J k the set { i ∈ [ n ] s.t. θ i ( x ⋆ ) > 0 and x i ( k ) > 0 } \{i\in[n]\ \text{s.t.}\ \theta_i(x^\star)>0\ \text{and}\ x_i^{(k)}>0\} { i ∈ [ n ] s.t. θ i ( x ⋆ ) > 0 and x i ( k ) > 0 } . If I c = ∅ I^c=\varnothing I c = ∅ , then θ i ( x ⋆ ) = 0 \theta_i(x^\star)=0 θ i ( x ⋆ ) = 0 for every i ∈ [ n ] i\in[n] i ∈ [ n ] , thus J k = ∅ J_k=\varnothing J k = ∅ for every $k. Let us then assume that I c ≠ ∅ I^c \ne\varnothing I c = ∅ . First, we notice that by definition δ min 2 H n 1 2 + δ max ≤ 1 \frac{\delta_{\min}}{2Hn^{\frac{1}{2}}+\delta_{\max}}\le1 2 H n 2 1 + δ m a x δ m i n ≤ 1 , hence (A.5) holds. In this case δ max ≥ δ k ≥ δ min > 0 \delta_{\max}\ge\delta_k\ge\delta_{\min}>0 δ m a x ≥ δ k ≥ δ m i n > 0 . Owing to Lemma 4, for every i ∈ I c i\in I^c i ∈ I c it holds
θ i ( x ( k ) ) ≥ θ i ( x ⋆ ) − ∥ x ( k ) − x ⋆ ∥ 1 α ( H n 1 2 + δ k 2 ) > θ i ( x ⋆ ) − δ min 2 H n 1 2 + δ max ( H n 1 2 + δ k 2 ) = θ i ( x ⋆ ) − δ min 2 2 H n 1 2 + δ k 2 H n 1 2 + δ max , \begin{aligned}
\theta_i(x^{(k)})\ge\theta_i(x^\star)-\lVert x^{(k)}-x^\star\rVert_1^\alpha\left(Hn^{\frac{1}{2}}+\frac{\delta_k}{2}\right)>\theta_i(x^\star)-\frac{\delta_{\min}}{2Hn^{\frac{1}{2}}+\delta_{\max}}\left(Hn^{\frac{1}{2}}+\frac{\delta_k}{2}\right) \\
&=\theta_i(x^\star)-\frac{\delta_{\min}}{2}\frac{2Hn^{\frac{1}{2}}+\delta_k}{2Hn^{\frac{1}{2}}+\delta_{\max}},
\end{aligned} θ i ( x ( k ) ) ≥ θ i ( x ⋆ ) − ∥ x ( k ) − x ⋆ ∥ 1 α ( H n 2 1 + 2 δ k ) > θ i ( x ⋆ ) − 2 H n 2 1 + δ m a x δ m i n ( H n 2 1 + 2 δ k ) = θ i ( x ⋆ ) − 2 δ m i n 2 H n 2 1 + δ m a x 2 H n 2 1 + δ k ,
since 2 α − 1 α ≤ 1 2^{\frac{\alpha-1}{\alpha}}\le1 2 α α − 1 ≤ 1 for every α ∈ ] 0 , 1 ] \alpha\in]0,1] α ∈ ] 0 , 1 ] . We now split the study into two cases, depending on whether J k = ∅ J_k=\varnothing J k = ∅ or J k ≠ ∅ J_k\ne\varnothing J k = ∅ .
Case 1, J k = ∅ J_k=\varnothing J k = ∅ . Since J k = ∅ J_k=\varnothing J k = ∅ , we have that O k = I c O_k=I^c O k = I c , thus x i ( k ) = 0 x_i^{(k)}=0 x i ( k ) = 0 for every i ∈ I c i\in I^c i ∈ I c and hence δ k = 0 \delta_k=0 δ k = 0 . In this case, for every i ∈ I c i\in I^c i ∈ I c
θ i ( x ( k ) ) ≥ θ i ( x ⋆ ) − H n 1 2 δ min 2 H n 1 2 + δ max ≥ θ i ( x ⋆ ) − H n 1 2 δ min 2 H n 1 2 ≥ δ min − δ min 2 > 0. (119) \theta_i(x^{(k)})\ge\theta_i(x^\star)-\frac{Hn^{\frac{1}{2}}\delta_{\min}}{2Hn^{\frac{1}{2}}+\delta_{\max}}\ge\theta_i(x^\star)-\frac{Hn^{\frac{1}{2}}\delta_{\min}}{2Hn^{\frac{1}{2}}}\ge\delta_{\min}-\frac{\delta_{\min}}{2}>0. \tag*{(119)} θ i ( x ( k ) ) ≥ θ i ( x ⋆ ) − 2 H n 2 1 + δ m a x H n 2 1 δ m i n ≥ θ i ( x ⋆ ) − 2 H n 2 1 H n 2 1 δ m i n ≥ δ m i n − 2 δ m i n > 0. (119)
By definition of the pairwise direction, we then have that any i ∈ I c i\in I^c i ∈ I c cannot be an away step direction as i i i does not belong to the active set of the solution x ( k ) x^{(k)} x ( k ) . Similarly, since θ i ( x ( k ) ) > 0 \theta_i(x^{(k)})>0 θ i ( x ( k ) ) > 0 (see (A.6)), none of the indices i ∈ I c i\in I^c i ∈ I c can be selected as a standard Frank-Wolfe direction. Hence x i ( k + 1 ) = 0 x_i^{(k+1)}=0 x i ( k + 1 ) = 0 for every i ∈ I c i\in I^c i ∈ I c .
Case 2, J k ≠ ∅ J_k\ne\varnothing J k = ∅ . By definition of δ k \delta_k δ k , we have that there exists an index ℓ \ell ℓ for which
θ ℓ ( x ⋆ ) = max j ∈ J k θ j ( x ⋆ ) = δ k . \theta_\ell(x^\star)=\max_{j\in J_k}\theta_j(x^\star)=\delta_k. θ ℓ ( x ⋆ ) = j ∈ J k max θ j ( x ⋆ ) = δ k .
Owing again to Lemma 4
θ ℓ ( x ( k ) ) ≥ θ ℓ ( x ⋆ ) − ∥ x ( k ) − x ⋆ ∥ 1 α ( H n 1 2 + δ k 2 ) > δ k − δ min 2 H n 1 2 + δ max ( H n 1 2 + δ k 2 ) ≥ δ k − δ min 2 2 H n 1 2 + δ k 2 H n 1 2 + δ min ≥ δ k − δ k 2 2 H n 1 2 + δ k 2 H n 1 2 + δ k = δ k 2 , \begin{aligned}
\theta_\ell(x^{(k)})\ge\theta_\ell(x^\star)-\lVert x^{(k)}-x^\star\rVert_1^\alpha\left(Hn^{\frac{1}{2}}+\frac{\delta_k}{2}\right)>\delta_k-\frac{\delta_{\min}}{2Hn^{\frac{1}{2}}+\delta_{\max}}\left(Hn^{\frac{1}{2}}+\frac{\delta_k}{2}\right) \\
&\ge\delta_k-\frac{\delta_{\min}}{2}\frac{2Hn^{\frac{1}{2}}+\delta_k}{2Hn^{\frac{1}{2}}+\delta_{\min}}\ge\delta_k-\frac{\delta_k}{2}\frac{2Hn^{\frac{1}{2}}+\delta_k}{2Hn^{\frac{1}{2}}+\delta_k}=\frac{\delta_k}{2},
\end{aligned} θ ℓ ( x ( k ) ) ≥ θ ℓ ( x ⋆ ) − ∥ x ( k ) − x ⋆ ∥ 1 α ( H n 2 1 + 2 δ k ) > δ k − 2 H n 2 1 + δ m a x δ m i n ( H n 2 1 + 2 δ k ) ≥ δ k − 2 δ m i n 2 H n 2 1 + δ m i n 2 H n 2 1 + δ k ≥ δ k − 2 δ k 2 H n 2 1 + δ k 2 H n 2 1 + δ k = 2 δ k ,
where we have used the fact that the function a ↦ − a 2 H n 1 2 + a a\mapsto-\frac{a}{2Hn^{\frac{1}{2}}+a} a ↦ − 2 H n 2 1 + a a is decreasing on [ 0 , + ∞ [ [0,+\infty[ [ 0 , + ∞ [ and δ k ≥ δ min \delta_k\ge\delta_{\min} δ k ≥ δ m i n . Notice that, as a by-product, we get
δ min 2 2 H n 1 2 + δ k 2 H n 1 2 + δ min ≤ δ k 2 . (120) \frac{\delta_{\min}}{2}\frac{2Hn^{\frac{1}{2}}+\delta_k}{2Hn^{\frac{1}{2}}+\delta_{\min}}\le\frac{\delta_k}{2}. \tag*{(120)} 2 δ m i n 2 H n 2 1 + δ m i n 2 H n 2 1 + δ k ≤ 2 δ k . (120)
We now show that the pairwise direction selected at step k k k , namely d ( k ) = e i − e j d^{(k)}=e_i-e_j d ( k ) = e i − e j , is such that i ∈ I i\in I i ∈ I and j ∈ J k j\in J_k j ∈ J k .
First, we notice that for every i ∈ I c i\in I^c i ∈ I c , we have θ i ( x ( k ) ) > 0 \theta_i(x^{(k)})>0 θ i ( x ( k ) ) > 0 , thus the FW direction must belong to I I I .
Second, we show that j ∈ J k j\in J_k j ∈ J k (or, equivalently, that j ∉ I j\notin I j ∈ / I since the indices of O k O_k O k do not belong to the active set of the solution x ( k ) x^{(k)} x ( k ) ). For every j ∈ I j\in I j ∈ I , we have that θ j ( x ⋆ ) = 0 \theta_j(x^\star)=0 θ j ( x ⋆ ) = 0 , thus
∣ θ j ( x ( k ) ) ∣ = ∣ θ j ( x ( k ) ) − θ j ( x ⋆ ) ∣ ≤ ∥ x ( k ) − x ⋆ ∥ 1 α ( H n 1 2 + δ k 2 ) (121) \lvert\theta_j(x^{(k)})\rvert=\lvert\theta_j(x^{(k)})-\theta_j(x^\star)\rvert\le\lVert x^{(k)}-x^\star\rVert_1^\alpha\left(Hn^{\frac{1}{2}}+\frac{\delta_k}{2}\right) \tag*{(121)} ∣ θ j ( x ( k ) )∣ = ∣ θ j ( x ( k ) ) − θ j ( x ⋆ )∣ ≤ ∥ x ( k ) − x ⋆ ∥ 1 α ( H n 2 1 + 2 δ k ) (121)
< δ min 2 H n 1 2 + δ max ( H n 1 2 + δ k 2 ) ≤ δ min 2 H n 1 2 + δ min ( H n 1 2 + δ k 2 ) ≤ δ k 2 , (122) <\frac{\delta_{\min}}{2Hn^{\frac{1}{2}}+\delta_{\max}}\left(Hn^{\frac{1}{2}}+\frac{\delta_k}{2}\right)\le\frac{\delta_{\min}}{2Hn^{\frac{1}{2}}+\delta_{\min}}\left(Hn^{\frac{1}{2}}+\frac{\delta_k}{2}\right)\le\frac{\delta_k}{2}, \tag*{(122)} < 2 H n 2 1 + δ m a x δ m i n ( H n 2 1 + 2 δ k ) ≤ 2 H n 2 1 + δ m i n δ m i n ( H n 2 1 + 2 δ k ) ≤ 2 δ k , (122)
where in the last inequality, we have used (A.7). We then have that none of the directions in I I I can determine the away step direction, since δ k 2 < θ ℓ ( x ( k ) ) \frac{\delta_k}{2}<\theta_\ell(x^{(k)}) 2 δ k < θ ℓ ( x ( k ) ) .
To conclude, we need to show that at each iteration the algorithm takes a full step, that is γ k = γ k ( max ) \gamma_k = \gamma_k^{(\max)} γ k = γ k ( m a x ) . Toward a contradiction, let us assume that γ k < γ k ( max ) \gamma_k < \gamma_k^{(\max)} γ k < γ k ( m a x ) . By hypothesis,
γ k ≥ ( − ⟨ ∇ F ( x ( k ) ) , e i − e j ⟩ 2 H ) 1 α = ( θ j ( x ( k ) ) − θ i ( x ( k ) ) 2 H ) 1 α \gamma_k \ge\left( \frac{-\langle\nabla F(x^{(k)}), e_i-e_j\rangle}{2H} \right)^{\frac{1}{\alpha}} = \left( \frac{\theta_j(x^{(k)})-\theta_i(x^{(k)})}{2H} \right)^{\frac{1}{\alpha}} γ k ≥ ( 2 H − ⟨ ∇ F ( x ( k ) ) , e i − e j ⟩ ) α 1 = ( 2 H θ j ( x ( k ) ) − θ i ( x ( k ) ) ) α 1
owing to the fact that θ i ( x ( k ) ) ≤ 0 \theta_i(x^{(k)}) \le0 θ i ( x ( k ) ) ≤ 0 , we infer
γ k ≥ ( θ j ( x ( k ) ) 2 H ) 1 α ≥ ( δ min 4 H ) 1 α . \gamma_k \ge\left( \frac{\theta_j(x^{(k)})}{2H} \right)^{\frac{1}{\alpha}} \ge\left( \frac{\delta_{\min}}{4H} \right)^{\frac{1}{\alpha}}. γ k ≥ ( 2 H θ j ( x ( k ) ) ) α 1 ≥ ( 4 H δ m i n ) α 1 .
Since j ∈ I c j \in I^c j ∈ I c , we have x j ∗ = 0 x_j^* = 0 x j ∗ = 0 , thus
x j ( k ) = ( x ( k ) − x ∗ ) j ≤ ∥ x ( k ) − x ∗ ∥ 1 2 ≤ 1 2 1 α ( δ min 2 H n 1 2 + δ max ) 1 α = ( δ min 4 H n 1 2 + 2 δ max ) 1 α . x_j^{(k)} = (x^{(k)} - x^*)_j \le\frac{\lVert x^{(k)} - x^* \rVert_1}{2} \le\frac{1}{2^{\frac{1}{\alpha}}}\left( \frac{\delta_{\min}}{2H n^{\frac{1}{2}}+\delta_{\max}} \right)^{\frac{1}{\alpha}} = \left( \frac{\delta_{\min}}{4H n^{\frac{1}{2}}+2\delta_{\max}} \right)^{\frac{1}{\alpha}}. x j ( k ) = ( x ( k ) − x ∗ ) j ≤ 2 ∥ x ( k ) − x ∗ ∥ 1 ≤ 2 α 1 1 ( 2 H n 2 1 + δ m a x δ m i n ) α 1 = ( 4 H n 2 1 + 2 δ m a x δ m i n ) α 1 .
Therefore, by the algorithm definition
x j ( k + 1 ) = x j ( k ) − γ k ≤ ( δ min 4 H n 1 2 + 2 δ max ) 1 α − ( δ min 4 H ) 1 α < 0 , (123) x_j^{(k+1)} = x_j^{(k)} - \gamma_k \le\left( \frac{\delta_{\min}}{4H n^{\frac{1}{2}}+2\delta_{\max}} \right)^{\frac{1}{\alpha}} - \left( \frac{\delta_{\min}}{4H} \right)^{\frac{1}{\alpha}} < 0, \tag*{(123)} x j ( k + 1 ) = x j ( k ) − γ k ≤ ( 4 H n 2 1 + 2 δ m a x δ m i n ) α 1 − ( 4 H δ m i n ) α 1 < 0 , (123)
which is a contradiction.
From Theorem 11 we infer that, whenever we use a PFW algorithm to solve an optimization problem, the routine identifies the extended support of the solution in a finite number of iterations. The number of iterations depends on two terms: (i) the number of iterations needed to satisfy the condition ∥ x ( k ) − x ∗ ∥ 1 ≤ r ∗ \lVert x^{(k)} - x^* \rVert_1 \le r^* ∥ x ( k ) − x ∗ ∥ 1 ≤ r ∗ , where x ∗ x^* x ∗ is an optimal point, and (ii) the number of steps needed to reduce the cardinality of the set J k + 1 : = { i ∈ [ n ] s.t. θ i ( x ∗ ) > 0 and x i ( k + 1 ) > 0 } J_{k+1} := \{i \in[n] \text{ s.t. } \theta_i(x^*) > 0 \text{ and } x_i^{(k+1)} > 0\} J k + 1 := { i ∈ [ n ] s.t. θ i ( x ∗ ) > 0 and x i ( k + 1 ) > 0 } to zero, which is at most equal to the dimension of the simplex.