Introduction
We consider linear programming (LP) in the following primal-dual form:
min ⟨ c , x ⟩ max ⟨ b , y ⟩ A x = b A T y + s = c x ≥ 0 n s ≥ 0 n \begin{aligned}
\min\quad&\langle c,x\rangle& \max\quad&\langle b,y\rangle\\
&Ax=b && A^{\mathsf T}y+s=c\\
&x\ge0_n && s\ge0_n
\end{aligned} min ⟨ c , x ⟩ A x = b x ≥ 0 n max ⟨ b , y ⟩ A T y + s = c s ≥ 0 n (LP( A , b , c ) (A,b,c) ( A , b , c )
where A ∈ R m × n A\in\mathbb{R}^{m\times n} A ∈ R m × n b ∈ R m b\in\mathbb{R}^{m} b ∈ R m c ∈ R n c\in\mathbb{R}^{n} c ∈ R n A A A m m m
The first polynomial-time LP algorithms were the ellipsoid method by Khachiyan in 1979 [28 ] and interior point methods, introduced by Karmarkar in 1984 [27 ] . However, it remains an outstanding open question to find a strongly polynomial algorithm for linear programming. The question was listed by the Fields medalist Smale as one of the most prominent mathematical challenges for the 21st century [Sma98]. In such an algorithm, only poly ( n ) \operatorname{poly}(n) poly ( n ) § 1.1 for formal definitions).
The notion of strongly polynomial algorithms was first formally introduced by Megiddo [31 ] , under the term ‘genuinely polynomial’; he gave such an algorithm for finding feasible solutions to two-variable-per-inequality (2VPI) linear programs. In a seminal, Fulkerson-prize winning paper [Tar85], Tardos obtained the first strongly polynomial algorithm for minimum-cost circulations. A particularly important technique in this paper was variable fixing : by solving an approximate version of the LP with rounded costs, one can deduce that a certain variable is at the lower or upper capacity bound in an optimal solution. Towards general LP, Tardos [Tar86] extended this approach to obtain a strongly polynomial algorithm for ‘combinatorial LPs’. More precisely, for LP ( A , b , c ) \operatorname{LP}(A,b,c) LP ( A , b , c ) A ∈ Z m × n A\in\mathbb{Z}^{m\times n} A ∈ Z m × n poly ( n , log Δ A ) \operatorname{poly}(n,\log\Delta_A) poly ( n , log Δ A ) Δ A \Delta_A Δ A A A A b b b c c c A A A poly ( n ) \operatorname{poly}(n) poly ( n )
Strongly polynomial algorithms from interior point methods. Interior point methods (IPMs) give the fastest current weakly polynomial algorithms for general LPs, see [12 , 7 , 5 , 26 ] , and for special classes such as minimum-cost circulations [11 , 4 ] and multicommodity flows [9 ] . Somewhat surprisingly, they are also a potent approach in the context of strongly polynomial computability. In recent years, IPMs have emerged as a unifying tool in this context.
The algorithms discussed next fall into the class of primal-dual path-following algorithms. A key concept here is the central path , the algebraic curve formed by minimizers of ⟨ c , x ⟩ − μ ∑ i = 1 n log ( x i ) \langle c,x\rangle-\mu\sum_{i=1}^{n}\log(x_i) ⟨ c , x ⟩ − μ ∑ i = 1 n log ( x i ) { x : A x = b } \{x:Ax=b\} { x : A x = b } μ > 0 \mu>0 μ > 0 § 1.2 for more details). As μ → 0 \mu\to0 μ → 0 μ \mu μ
The main progress measure in IPMs is the decrease in optimality gap and thus these methods are inherently weakly polynomial; finding an exact optimal solution typically requires an additional rounding step. In the influential paper [VY96], Vavasis and Ye introduced Layered Least Squares (LLS) IPMs that terminate with an exact optimal solution in poly ( n , log χ ˉ A ) \operatorname{poly}(n,\log\bar{\chi}_A) poly ( n , log χ ˉ A ) χ ˉ A \bar{\chi}_A χ ˉ A A A A b b b c c c χ ˉ A = O ( n Δ A ) \bar{\chi}_A=O(n\Delta_A) χ ˉ A = O ( n Δ A ) [18 ] .
This led to a line of research on improved combinatorial IPMs [32 , 33 , 34 , 29 ] . The paper by Dadush, Huiberts, Natura, and Végh [DHN^24] revealed that χ ˉ A \bar{\chi}_A χ ˉ A circuit imbalance measure κ A \kappa_A κ A A A A κ A ∗ \kappa_A^* κ A ∗
The above results may raise hopes of finding a strongly polynomial IPM. However, the papers by Allamigeon, Benchimol, Gaubert, and Joswig [ABG^18], see also [3 ZLY24] , yield a surprising negative answer. By analyzing the tropical limits of linear programs, these papers exhibit parametric families of LPs such that for suitably large parameter values, no path-following method can be strongly polynomial. This was first shown for the standard logarithmic barrier [ABG^18], and later for arbitrary self-concordant barriers [3 ] .
IPMs matching straight line complexity. The lower bounds in [1 , 3 ] are ultimately based on the following insight. The trajectory of any path-following IPM is a piecewise linear curve in the neighborhood of the central path; the number of pieces corresponds to the number of iterations. Thus, a lower bound on the number of linear segments of any piecewise linear curve in the neighborhood provides a lower bound on the number of iterations. The above-mentioned papers exhibit families of LPs with exponential lower bounds.
The recent result by Allamigeon, Dadush, Loho, Natura, and Végh [2 ] complements these negative results by a positive algorithmic bound. Namely, their Subspace Layered Least Squares (SLLS) IPM matches the straight line complexity lower bound within a strongly polynomial O ~ ( n 1.5 ) \widetilde{O}(n^{1.5}) O ( n 1.5 ) Theorem 1.6 . At a high level, the algorithm exploits the structural property that whenever there is a long straight line segment in the neighborhood of the central path, the variables exhibit a ‘polarized’ behavior: There exists a partition [ n ] = ( B , N ) [n] = (B, N) [ n ] = ( B , N ) B B B N N N O ~ ( n 1.5 ) \widetilde{O}(n^{1.5}) O ( n 1.5 ) [17 ] improved the iteration complexity of the above algorithm, by using a strongly polynomial implementation of the trust region steps introduced in [29 ] . In particular, they showed that the iteration complexity of the Trust Region IPM is optimal within a constant factor in the ℓ 2 \ell_2 ℓ 2
The SLLS IPM in [2 ] provides a unifying framework for strongly polynomially solvable classes of LPs. One can show that the straight line complexity is bounded polynomially by n n n log ( κ A ∗ ) \log(\kappa_A^*) log ( κ A ∗ ) [10 ] . It is also the key tool for settling the strongly polynomial complexity of two-variable-per-inequality LPs, as discussed next.
Two-variable-per-inequality LPs. We say that the dual program in L P ( A , b , c ) LP(A,b,c) L P ( A , b , c ) two-variable-per-inequality (2VPI) linear program if every row of A T A^{\mathsf{T}} A T
As already mentioned, one of the first strongly polynomial algorithms in the literature was given by Megiddo [31 ] for the 2VPI feasibility problem. As shown in [24 ] , 2VPI primal-dual pairs can be strongly polynomially reduced to solving the minimum-cost generalized flow problem, the classical network flow model where the flow may change while traversing the arcs according to gain factors. The strongly polynomial algorithms for generalized flow maximization [Vé17, OV20] thus imply a strongly polynomial algorithm for finding primal-feasible solutions of LPs with at most two nonzeros per column. However, for the minimum-cost generalized flow problem, and for 2VPI primal-dual pairs, this was only settled recently in the conference paper by the authors [15 ] :
Theorem 1.1. There is a deterministic strongly polynomial algorithm for the minimum-cost generalized flow problem, and for two-variable-per-inequality primal-dual pairs.
A key ingredient in showing Theorem 1.1 is proving strongly polynomial straight-line complexity bounds for minimum-cost generalized flow problems; these bounds are described in the companion paper [16 ] . However, to obtain the claimed strongly polynomial bound, there are significant additional challenges regarding initialization and implementation in the Turing model. The current paper exhibits general techniques to address these challenges.
Summary of contributions. Our contributions, presented in § 1.4 , can be informally summarized as follows:
Initialization: IPMs require a pair of strictly feasible initial primal and dual solutions near the central path, even though a strictly feasible (or even a feasible) solution may not exist. We present a careful initialization scheme that solves linear programs in three stages, and preserves the straight line complexity bounds; see Theorem 1.7.
Implementation in the Turing model: We show that the bit-length of the computations can be controlled in a model using only basic arithmetic operations and comparisons; see Theorem 1.8.
Initialization is always a significant issue when applying IPMs. It is usually done by embedding the LP into an auxiliary system that is strictly feasible for which a suitable starting solution can be easily found and the optimal solutions of the auxiliary LP map to the optimal solutions of the original LP. Such constructions, however, may lose the combinatorial structure of the LP, or rely on numerical estimates based on bit-complexity (see § 1.4.1 ). In contrast, our initialization scheme does not require any a priori numerical estimates. The overall running time in Theorem 1.7 is bounded by the maximum straight line complexity of the original LP and its three simple modifications. For the min-cost generalized flow problem, each of these LPs also describes a generalized flow instance.
Our second contribution addresses a fundamental issue regarding the computational models; we formally introduce the real RAM and Turing models in § 1.1 below. While ‘combinatorial’ algorithms such as Tardos’s results [Tar85, Tar86] or the 2VPI/generalized flow papers [31 Vég17, 36 ] are strongly polynomial in the Turing model of computation, the Vavasis–Ye algorithm [VY96] and all subsequent work on combinatorial IPMs, including the SLLS IPM [2 ] , are presented in the real RAM model. Whenever the straight line complexity is strongly polynomially bounded, the number of arithmetic operations is strongly polynomial. However, in the Turing model one needs to ensure that the algorithm only uses polynomial space. The above algorithms are ‘stepwise strongly polynomial’, i.e., each iteration can be implemented in strongly polynomial time. Still, the bit-length could increase exponentially over the sequence of iterations. This notion is defined in § 1.1 , where we also discuss why simple rounding techniques are not applicable in the strongly polynomial model.
In Theorem 1.8, we introduce a ‘combinatorial’ strongly polynomial rounding step. This is applicable broadly for the combinatorial IPMs above, and can thus turn them, under certain conditions explained below, into strongly polynomial algorithms in the Turing model for the classes where their iteration complexities are strongly polynomial.
Computational models
We now turn to the discussion of the computational models. There are multiple related, yet distinct notions of a strongly polynomial computational model. Smale’s question was posed in the Blum–Shub–Smale (BSS) real model of computation [8 ] . In this model, the input can be given by arbitrary real numbers, and one step may compute a rational function of the previously computed quantities with real coefficients, or make comparisons between two quantities. In the more restrictive real RAM model, one can perform a sequence of elementary arithmetic operations (+ + + − - − × \times × / / / ≥ \ge ≥ polynomial in the real RAM model if the number of elementary arithmetic operations and comparisons is bounded polynomially in the dimension of the input; in the case of LP, this is K = n × m + n + m K = n \times m + n + m K = n × m + n + m
We now turn to the Turing model. Consider a problem where the input is given by K K K ( A , b , c ) (A,b,c) ( A , b , c ) K = 2 ( n × m + n + m ) K = 2(n \times m + n + m) K = 2 ( n × m + n + m ) strongly polynomial in the Turing model (see [20 ] ), if it only performs poly ( K ) \operatorname{poly}(K) poly ( K ) poly ( m , n ) \operatorname{poly}(m,n) poly ( m , n )
There is an ambiguity in the models regarding how divisions can be implemented; see discussion of variants in [20 Section 1.3] . The results of this paper work with the most restricted setting: all numbers are maintained as numerator–denominator pairs, which are never reduced to lowest terms. To make this precise, we distinguish a rational number from the pair that represents it, and define sizes of numbers following [20 ] .
Definition 1.2 (Encoded rationals and sizes of numbers). For an integer k ∈ Z k \in\mathbb{Z} k ∈ Z encoding length of k k k size ( k ) = 1 + ⌈ log 2 ( ∣ k ∣ + 1 ) ⌉ \operatorname{size}(k) = 1 + \lceil\log_{2}(|k| + 1) \rceil size ( k ) = 1 + ⌈ log 2 ( ∣ k ∣ + 1 )⌉
Q ^ : = Z × Z > 0 , val ( p , q ) : = p q , size ( p , q ) : = size ( p ) + size ( q ) , \widehat{\mathbb{Q}} := \mathbb{Z} \times\mathbb{Z}_{>0}, \qquad\operatorname{val}(p,q) := \frac{p}{q}, \qquad\operatorname{size}(p,q) := \operatorname{size}(p) + \operatorname{size}(q), Q := Z × Z > 0 , val ( p , q ) := q p , size ( p , q ) := size ( p ) + size ( q ) , calling the elements of Q ^ \widehat{\mathbb{Q}} Q encoded rationals . The canonical value map val : Q ^ → Q \operatorname{val}: \widehat{\mathbb{Q}} \to\mathbb{Q} val : Q → Q Q \mathbb{Q} Q Q ^ \widehat{\mathbb{Q}} Q ( p , q ) ∼ ( p ′ , q ′ ) (p,q) \sim(p',q') ( p , q ) ∼ ( p ′ , q ′ ) p q ′ = p ′ q pq' = p'q p q ′ = p ′ q k k k ( k , 1 ) (k,1) ( k , 1 ) 2 2 2 Q ^ > 0 : = { ( p , q ) ∈ Q ^ ∣ p > 0 } \widehat{\mathbb{Q}}_{>0} := \{(p,q) \in\widehat{\mathbb{Q}} \mid p > 0\} Q > 0 := {( p , q ) ∈ Q ∣ p > 0 } a 1 , a 2 , … , a k a_1, a_2, \ldots, a_k a 1 , a 2 , … , a k
size ( a 1 , a 2 , … , a k ) = ∑ i = 1 k size ( a i ) \operatorname{size}(a_1,a_2,\ldots,a_k)=\sum_{i=1}^{k}\operatorname{size}(a_i) size ( a 1 , a 2 , … , a k ) = ∑ i = 1 k size ( a i ) α ∈ Q \alpha\in\mathbb{Q} α ∈ Q
size ( α ) : = min { size ( a ) ∣ a ∈ Q ^ , val ( a ) = α } , \operatorname{size}(\alpha) := \min\{\operatorname{size}(a) \mid a \in\widehat{\mathbb{Q}},\ \operatorname{val}(a)=\alpha\}, size ( α ) := min { size ( a ) ∣ a ∈ Q , val ( a ) = α } , the minimum being attained by the lowest-terms representation, and likewise entrywise for rational vectors and matrices, matching the classical notion of encoding length of a rational number.
Every number produced by our algorithms is an encoded rational. For ( p , q ) , ( p ′ , q ′ ) ∈ Q ^ (p,q),(p',q') \in\widehat{\mathbb{Q}} ( p , q ) , ( p ′ , q ′ ) ∈ Q ( p q ′ + p ′ q , q q ′ ) (pq'+p'q,qq') ( p q ′ + p ′ q , q q ′ ) ( p p ′ , q q ′ ) (pp',qq') ( p p ′ , q q ′ ) p q / p ′ q ′ \frac{p}{q}/\frac{p'}{q'} q p / q ′ p ′ ( p q ′ , q p ′ ) (pq',qp') ( p q ′ , q p ′ )
We identify an encoded rational with its value wherever only the value matters: for a ∈ Q ^ n a \in\widehat{\mathbb{Q}}^n a ∈ Q n S ⊆ Q n S \subseteq\mathbb{Q}^n S ⊆ Q n a ∈ S a \in S a ∈ S val ( a ) ∈ S \operatorname{val}(a) \in S val ( a ) ∈ S
While a strongly polynomial algorithm in the Turing model implies a polynomial algorithm in the real RAM model, the converse is not necessarily the case: enforcing PSPACE may be challenging. For example, Gaussian elimination needs to be done carefully to keep the sizes of numbers under control, see [19 ] and [20 Section 1.4] .
The LLS interior point methods [VY96, MMT98, MT03, MT05, DHNV24, ADL+25] are polynomial in the real RAM model2 log ( χ ‾ A ) = poly ( n ) \log(\overline{\chi}_{A})=\operatorname{poly}(n) log ( χ A ) = poly ( n )
Stepwise strongly polynomial algorithms. By an iterative algorithm we mean an algorithm that computes a sequence of points z ( 0 ) , z ( 1 ) , … , z ( T ) ∈ R k z^{(0)},z^{(1)},\ldots,z^{(T)} \in\mathbb{R}^k z ( 0 ) , z ( 1 ) , … , z ( T ) ∈ R k k ∈ N k \in\mathbb{N} k ∈ N stepwise strongly polynomial algorithm we mean an iterative algorithm that, given a rational input z ( t ) ∈ Q k z^{(t)} \in\mathbb{Q}^k z ( t ) ∈ Q k z ( t + 1 ) ∈ Q k z^{(t+1)} \in\mathbb{Q}^k z ( t + 1 ) ∈ Q k
Note that stepwise strong polynomiality does not require that the total number of iterations T T T [35 ] , are stepwise strongly polynomial: computing each step direction amounts to solving a system of linear equations, where the coefficients are constructed from the input ( A , b , c ) (A,b,c) ( A , b , c )
Consequently, even if an iterative algorithm terminates in a strongly polynomial number of iterations, and is stepwise strongly polynomial, it does not follow that the algorithm is strongly polynomial in the Turing model.
The difficulty of rounding in the strongly polynomial model. To implement an IPM in PSPACE, a natural idea is to use additional rounding steps. In the weakly polynomial model, i.e., when running time dependence on the total encoding length is allowed, the bit complexity of the algorithms can be controlled by approximate linear system solves and rounding. Recent work by Ghadiri, Peng, and Vempala [22 ] developed general tools that enable one to keep the bit complexity of recent fast IPMs under control.
However, these techniques are not applicable in the strongly polynomial Turing model, where the number of arithmetic operations and comparisons must be bounded independently of the encoding length. In this model one cannot even approximate log ( x ) \log(x) log ( x ) x x x
As already mentioned, Theorem 1.8 in this paper presents a general, strongly polynomial rounding technique that can be used to convert—under certain technical conditions—any stepwise strongly polynomial interior point method that stays in the ℓ 2 \ell_2 ℓ 2
The central path and the analytic center
We next formally introduce some concepts from the theory of interior point methods needed to define straight-line complexity and to state our results. Given an instance of LP ( A , b , c ) \operatorname{LP}(A,b,c) LP ( A , b , c )
P : = { x ∈ R n ∣ A x = b , x ≥ 0 } , P > 0 : = P ∩ R > 0 n , \mathcal{P} := \{x \in\mathbb{R}^{n} \mid Ax=b,\ x\geq0\}, \qquad\mathcal{P}_{>0}:=\mathcal{P}\cap\mathbb{R}_{>0}^{n}, P := { x ∈ R n ∣ A x = b , x ≥ 0 } , P > 0 := P ∩ R > 0 n , D : = { s ∈ R n ∣ ∃ y ∈ R m , A T y + s = c , s ≥ 0 } , D > 0 : = D ∩ R > 0 n . \mathcal{D} := \{s\in\mathbb{R}^{n}\mid\exists y\in\mathbb{R}^{m},\ A^{\mathsf{T}}y+s=c,\ s\geq0\}, \qquad\mathcal{D}_{>0}:=\mathcal{D}\cap\mathbb{R}_{>0}^{n}. D := { s ∈ R n ∣ ∃ y ∈ R m , A T y + s = c , s ≥ 0 } , D > 0 := D ∩ R > 0 n . denote the primal and dual feasible regions, and the sets of strictly feasible primal and dual solutions, respectively. For ( x , s ) ∈ P × D (x,s)\in\mathcal{P}\times\mathcal{D} ( x , s ) ∈ P × D normalized duality gap is
μ ˉ ( x , s ) : = ⟨ x , s ⟩ n . \bar{\mu}(x,s):=\frac{\langle x,s\rangle}{n}. μ ˉ ( x , s ) := n ⟨ x , s ⟩ . By complementary slackness, ( x , s ) ∈ P × D (x,s)\in\mathcal{P}\times\mathcal{D} ( x , s ) ∈ P × D μ ˉ ( x , s ) = 0 \bar{\mu}(x,s)=0 μ ˉ ( x , s ) = 0
Given an instance of LP ( A , b , c ) \operatorname{LP}(A,b,c) LP ( A , b , c ) P > 0 , D > 0 ≠ ∅ \mathcal{P}_{>0},\mathcal{D}_{>0}\ne\varnothing P > 0 , D > 0 = ∅ μ > 0 \mu>0 μ > 0 ( x ( μ ) , s ( μ ) ) ∈ P × D (x(\mu),s(\mu))\in\mathcal{P}\times\mathcal{D} ( x ( μ ) , s ( μ )) ∈ P × D x i ( μ ) s i ( μ ) = μ x_i(\mu)s_i(\mu)=\mu x i ( μ ) s i ( μ ) = μ i ∈ [ n ] i\in[n] i ∈ [ n ] central path point at μ \mu μ ( x c p ( μ ) , s c p ( μ ) ) (x^{\mathrm{cp}}(\mu),s^{\mathrm{cp}}(\mu)) ( x cp ( μ ) , s cp ( μ )) central path is the algebraic curve formed by the points ( x ( μ ) , s ( μ ) ) (x(\mu),s(\mu)) ( x ( μ ) , s ( μ )) μ ∈ ( 0 , ∞ ) \mu\in(0,\infty) μ ∈ ( 0 , ∞ ) ( x ⋆ , s ⋆ ) ∈ P × D (x^\star,s^\star)\in\mathcal{P}\times\mathcal{D} ( x ⋆ , s ⋆ ) ∈ P × D B ⋆ = supp ( x ⋆ ) B^\star=\operatorname{supp}(x^\star) B ⋆ = supp ( x ⋆ ) N ⋆ = supp ( s ⋆ ) N^\star=\operatorname{supp}(s^\star) N ⋆ = supp ( s ⋆ ) [ n ] [n] [ n ]
Interior point methods maintain iterates in some neighborhood of the central path. We define the ℓ 2 \ell_2 ℓ 2 β ∈ [ 0 , 1 ) \beta\in[0,1) β ∈ [ 0 , 1 )
N 2 ( β ) : = { ( x , s ) ∈ P > 0 × D > 0 : ∥ x ∘ s μ ˉ ( x , s ) − 1 n ∥ 2 ≤ β } . (1) \mathcal{N}^{2}(\beta):=\left\{(x,s)\in\mathcal{P}_{>0}\times\mathcal{D}_{>0}:\left\|\frac{x\circ s}{\bar{\mu}(x,s)}-\mathbf{1}_n\right\|_2\leq\beta\right\}. \tag*{(1)} N 2 ( β ) := { ( x , s ) ∈ P > 0 × D > 0 : μ ˉ ( x , s ) x ∘ s − 1 n 2 ≤ β } . (1) where x ∘ s = ( x 1 s 1 , x 2 s 2 , … , x n s n ) x\circ s=(x_1s_1,x_2s_2,\ldots,x_ns_n) x ∘ s = ( x 1 s 1 , x 2 s 2 , … , x n s n ) x x x s s s N ‾ 2 ( β ) \overline{\mathcal{N}}^{2}(\beta) N 2 ( β ) ℓ 2 \ell_2 ℓ 2 β \beta β 1 / poly ( n ) ≤ β ≤ 1 / 16 1/\operatorname{poly}(n)\leq\beta\leq1/16 1/ poly ( n ) ≤ β ≤ 1/16 size ( β ) = O ( log n ) \operatorname{size}(\beta)=O(\log n) size ( β ) = O ( log n ) log ( 1 / β ) = O ( log n ) \log(1/\beta)=O(\log n) log ( 1/ β ) = O ( log n ) size ( β ) \operatorname{size}(\beta) size ( β ) poly ( n ) \operatorname{poly}(n) poly ( n ) ( A , b , c ) (A,b,c) ( A , b , c ) β \beta β β \beta β
If P > 0 \mathcal{P}_{>0} P > 0 analytic center 3
x ~ : = arg max x ∈ P > 0 ∑ i = 1 n log x i . \widetilde{x}:=\underset{x\in\mathcal{P}_{>0}}{\arg\max}\ \sum_{i=1}^{n}\log x_i. x := x ∈ P > 0 arg max i = 1 ∑ n log x i . By Lagrangian duality, there exists a unique v ~ ∈ im ( A T ) \widetilde{v}\in\operatorname{im}(A^{\mathsf{T}}) v ∈ im ( A T ) x ~ ∘ v ~ = 1 n \widetilde{x}\circ\widetilde{v}=\mathbf{1}_n x ∘ v = 1 n D > 0 \mathcal{D}_{>0} D > 0 ∑ i = 1 n log s i \sum_{i=1}^{n}\log s_i ∑ i = 1 n log s i D > 0 \mathcal{D}_{>0} D > 0 w ~ ∈ ker ( A ) \widetilde{w}\in\ker(A) w ∈ ker ( A ) s ~ ∘ w ~ = 1 n \widetilde{s}\circ\widetilde{w}=\mathbf{1}_n s ∘ w = 1 n P > 0 \mathcal{P}_{>0} P > 0 D > 0 \mathcal{D}_{>0} D > 0
It is well known that the central path limit point ( x ⋆ , s ⋆ ) (x^\star,s^\star) ( x ⋆ , s ⋆ ) x ⋆ x^\star x ⋆ { x B ⋆ ∣ x ∈ P , supp ( x ) ⊆ B ⋆ } \{x_{B^\star}\mid x\in\mathcal{P},\operatorname{supp}(x)\subseteq B^\star\} { x B ⋆ ∣ x ∈ P , supp ( x ) ⊆ B ⋆ } s ⋆ s^\star s ⋆ { s N ⋆ ∣ s ∈ D , supp ( s ) ⊆ N ⋆ } \{s_{N^\star}\mid s\in\mathcal{D},\operatorname{supp}(s)\subseteq N^\star\} { s N ⋆ ∣ s ∈ D , supp ( s ) ⊆ N ⋆ }
Straight line complexity and the Subspace Layered Least Squares Interior Point Method
In this section, we introduce the concept of straight-line complexity, and state the main result on the SLLS IPM from [2 ] . Assume LP ( A , b , c ) \operatorname{LP}(A,b,c) LP ( A , b , c ) v ⋆ v^\star v ⋆ g ≥ 0 g\geq0 g ≥ 0
P g : = { x ∈ P ∣ ⟨ c , x ⟩ ≤ v ⋆ + g } , \mathcal{P}_g:=\{x\in\mathcal{P}\mid\langle c,x\rangle\leq v^\star+g\}, P g := { x ∈ P ∣ ⟨ c , x ⟩ ≤ v ⋆ + g } , D g : = { s ∈ D ∣ ∃ y ∈ R m with A T y + s = c and ⟨ b , y ⟩ ≥ v ⋆ − g } . (2) \mathcal{D}_g:=\{s\in\mathcal{D}\mid\exists y\in\mathbb{R}^{m}\text{ with }A^{\mathsf{T}}y+s=c\text{ and }\langle b,y\rangle\geq v^\star-g\}. \tag*{(2)} D g := { s ∈ D ∣ ∃ y ∈ R m with A T y + s = c and ⟨ b , y ⟩ ≥ v ⋆ − g } . (2) the feasible sublevel sets. They correspond to the sets of the primal and dual feasible points ( x , s ) (x,s) ( x , s ) g g g v ⋆ v^\star v ⋆
Definition 1.3 (Max central path). Assume P ≠ ∅ \mathcal{P} \ne\emptyset P = ∅ D ≠ ∅ \mathcal{D} \ne\emptyset D = ∅ max central path is defined as the parametric curve g ↦ z m ( g ) : = ( x m ( g ) , s m ( g ) ) ∈ ( R ≥ 0 ∪ { ∞ } ) 2 n g \mapsto z^{\mathrm{m}}(g) := (x^{\mathrm{m}}(g),s^{\mathrm{m}}(g)) \in(\mathbb{R}_{\ge0} \cup\{\infty\})^{2n} g ↦ z m ( g ) := ( x m ( g ) , s m ( g )) ∈ ( R ≥ 0 ∪ { ∞ } ) 2 n g ≥ 0 g \ge0 g ≥ 0
x i m ( g ) : = sup { x i ∣ x ∈ P g } and s i m ( g ) : = sup { s i ∣ s ∈ D g } , ∀ i ∈ [ n ] , (3) x_i^{\mathrm{m}}(g) := \sup\{x_i \mid x \in\mathcal{P}_g\} \quad\text{and} \quad s_i^{\mathrm{m}}(g) := \sup\{s_i \mid s \in\mathcal{D}_g\}, \qquad\forall i \in[n], \tag*{(3)} x i m ( g ) := sup { x i ∣ x ∈ P g } and s i m ( g ) := sup { s i ∣ s ∈ D g } , ∀ i ∈ [ n ] , (3) The max central path can be seen as a combinatorial proxy to the central path. In particular, for g = n μ g=n\mu g = n μ x c p ( μ ) ∈ P g x^{\mathrm{cp}}(\mu) \in\mathcal{P}_g x cp ( μ ) ∈ P g s c p ( μ ) ∈ D g s^{\mathrm{cp}}(\mu) \in\mathcal{D}_g s cp ( μ ) ∈ D g P > 0 , D > 0 ≠ ∅ \mathcal{P}_{>0},\mathcal{D}_{>0} \ne\emptyset P > 0 , D > 0 = ∅ x m ( g ) x^{\mathrm{m}}(g) x m ( g ) s m ( g ) s^{\mathrm{m}}(g) s m ( g )
Lemma 1.4 ([2 Lemma 1.2] ). If both primal and dual programs in L P ( A , b , c ) LP(A,b,c) L P ( A , b , c ) μ > 0 \mu> 0 μ > 0 g = n μ g=n\mu g = n μ
x m ( g ) 2 n ≤ x c p ( μ ) ≤ x m ( g ) and s m ( g ) 2 n ≤ s c p ( μ ) ≤ s m ( g ) . \frac{x^{\mathrm{m}}(g)}{2n} \le x^{\mathrm{cp}}(\mu) \le x^{\mathrm{m}}(g) \qquad\text{and} \qquad\frac{s^{\mathrm{m}}(g)}{2n} \le s^{\mathrm{cp}}(\mu) \le s^{\mathrm{m}}(g). 2 n x m ( g ) ≤ x cp ( μ ) ≤ x m ( g ) and 2 n s m ( g ) ≤ s cp ( μ ) ≤ s m ( g ) . For each i ∈ [ n ] i \in[n] i ∈ [ n ] g ↦ x i m ( g ) g \mapsto x_i^{\mathrm{m}}(g) g ↦ x i m ( g ) ⟨ c , x ⟩ \langle c,x\rangle ⟨ c , x ⟩ − x i -x_i − x i 2 n 2^n 2 n [2 Section 4] . We will use the following definition.
Definition 1.5. Let f : R ≥ 0 → R ≥ 0 f:\mathbb{R}_{\ge0} \to\mathbb{R}_{\ge0} f : R ≥ 0 → R ≥ 0 η ∈ [ 0 , 1 ] \eta\in[0,1] η ∈ [ 0 , 1 ] straight line complexity of f f f η \eta η SLC η ( f ) \operatorname{SLC}_{\eta}(f) SLC η ( f ) h : R ≥ 0 → R ≥ 0 h:\mathbb{R}_{\ge0} \to\mathbb{R}_{\ge0} h : R ≥ 0 → R ≥ 0 η f ≤ h ≤ f \eta f \le h \le f η f ≤ h ≤ f 4 f ( x ) = ∞ f(x)=\infty f ( x ) = ∞ x ∈ R ≥ 0 x \in\mathbb{R}_{\ge0} x ∈ R ≥ 0 SLC η ( f ) = 1 \operatorname{SLC}_{\eta}(f)=1 SLC η ( f ) = 1
We will refer to η \eta η SLC parameter . The number of iterations taken by the Subspace Layered Least Squares (SLLS) IPM in [2 ] can be bounded by the sum of straight line complexities of each coordinate of the (primal) max central path. We now state the guarantees of SLLS IPM in [2 ] .
Theorem 1.6 ([2 Theorem 1.4] ). There is an interior-point method that, given an instance of L P ( A , b , c ) LP(A,b,c) L P ( A , b , c ) ( x , s ) ∈ N 2 ( β ) (x,s) \in\mathcal{N}^2(\beta) ( x , s ) ∈ N 2 ( β ) 0 < β ≤ 1 / 6 0 < \beta\le1/6 0 < β ≤ 1/6 ( x o u t , s o u t ) ∈ P × D (x^{\mathrm{out}},s^{\mathrm{out}}) \in\mathcal{P} \times\mathcal{D} ( x out , s out ) ∈ P × D
O ( n β min η ∈ ( 0 , 1 ] log ( n β η ) ∑ i = 1 n SLC η ( x i m ) ) (4) O\left(\frac{\sqrt{n}}{\beta}\min_{\eta\in(0,1]} \log\left(\frac{n}{\beta\eta}\right)\sum_{i=1}^{n}\operatorname{SLC}_{\eta}(x_i^{\mathrm{m}})\right) \tag*{(4)} O ( β n η ∈ ( 0 , 1 ] min log ( β η n ) i = 1 ∑ n SLC η ( x i m ) ) (4) many iterations. Let B = supp ( x o u t ) B=\operatorname{supp}(x^{\mathrm{out}}) B = supp ( x out ) N = supp ( s o u t ) N=\operatorname{supp}(s^{\mathrm{out}}) N = supp ( s out ) B ∪ N = [ n ] B \cup N=[n] B ∪ N = [ n ] ( v o u t , w o u t ) ∈ R 2 n (v^{\mathrm{out}},w^{\mathrm{out}}) \in\mathbb{R}^{2n} ( v out , w out ) ∈ R 2 n
(i) v o u t ∈ im ( A T ) v^{\mathrm{out}} \in\operatorname{im}(A^{\mathsf{T}}) v out ∈ im ( A T ) v B o u t > 0 B v^{\mathrm{out}}_B > 0_B v B out > 0 B A w o u t = 0 m Aw^{\mathrm{out}}=0_m A w out = 0 m w N o u t > 0 N w^{\mathrm{out}}_N > 0_N w N out > 0 N
(ii)
∥ ( x B o u t ∘ v B o u t , s N o u t ∘ w N o u t ) − 1 n ∥ 2 ≤ β . \left\|(x^{\mathrm{out}}_B \circ v^{\mathrm{out}}_B,s^{\mathrm{out}}_N \circ w^{\mathrm{out}}_N)-\mathbf{1}_n\right\|_2 \le\beta. ( x B out ∘ v B out , s N out ∘ w N out ) − 1 n 2 ≤ β . The algorithm can be implemented in the real RAM model, and is stepwise strongly polynomial. Further, ( v o u t , w o u t ) (v^{\mathrm{out}},w^{\mathrm{out}}) ( v out , w out ) N 2 ( β ) \mathcal{N}^2(\beta) N 2 ( β ) 1 − β / ( 2 n ) 1-\beta/(2\sqrt{n}) 1 − β / ( 2 n )
The vectors ( v o u t , w o u t ) (v^{\mathrm{out}},w^{\mathrm{out}}) ( v out , w out ) ( x o u t , s o u t ) (x^{\mathrm{out}},s^{\mathrm{out}}) ( x out , s out ) O ~ ( n 1.5 ) \widetilde{O}(n^{1.5}) O ( n 1.5 ) SLC η ( x i m ) \operatorname{SLC}_{\eta}(x_i^{\mathrm{m}}) SLC η ( x i m ) SLC 1 ( x i m ) ≤ 2 n \operatorname{SLC}_1(x_i^{\mathrm{m}}) \le2^n SLC 1 ( x i m ) ≤ 2 n
We note that the theorem could be equivalently rewritten in terms of the dual straight line complexities (see [2 Lemma 4.5] ). Further, we note that the SLC parameter η \eta η 0 < η < η ′ < 1 0 < \eta< \eta' < 1 0 < η < η ′ < 1 SLC η ′ ( f ) = O ( log ( 1 / η ) / log ( 1 / η ′ ) ) SLC η ( f ) \operatorname{SLC}_{\eta'}(f)=O(\log(1/\eta)/\log(1/\eta'))\operatorname{SLC}_{\eta}(f) SLC η ′ ( f ) = O ( log ( 1/ η ) / log ( 1/ η ′ )) SLC η ( f )
According to Theorem 1.6 , analyzing the number of iterations of SLLS IPM boils down to upper-bounding the straight-line complexities of the variables. Note that this is a purely geometric question about understanding the structure of univariate piecewise linear functions x i m ( g ) x_i^{\mathrm{m}}(g) x i m ( g )
Our contributions
Our first main result states that the running time of solving an arbitrary instance of LP ( A , b , c ) \operatorname{LP}(A,b,c) LP ( A , b , c )
Theorem 1.7. Let A ∈ R m × n A \in\mathbb{R}^{m \times n} A ∈ R m × n b ∈ R m b \in\mathbb{R}^{m} b ∈ R m c ∈ R n c \in\mathbb{R}^{n} c ∈ R n rank ( A ) = m \operatorname{rank}(A)=m rank ( A ) = m LP ( A , b , c ) \operatorname{LP}(A,b,c) LP ( A , b , c )
O ( poly ( n ) ⋅ min η ∈ ( 0 , 1 ] Γ η log ( n η ) ) . O\left(\operatorname{poly}(n) \cdot\min_{\eta\in(0,1]} \Gamma_{\eta} \log\left(\frac{n}{\eta}\right)\right). O ( poly ( n ) ⋅ η ∈ ( 0 , 1 ] min Γ η log ( η n ) ) . where Γ η \Gamma_{\eta} Γ η SLC η ( x i m ) \operatorname{SLC}_{\eta}(x_i^{m}) SLC η ( x i m )
Note that (S-Conic), (S-Dual), and (S-Primal) are always primal and dual feasible. For rational input ( A , b , c ) (A,b,c) ( A , b , c )
We highlight as our second main result a rounding technique that is used in the PSPACE implementation. This step is not specific to our algorithm, and can be used to convert a broad class of IPMs in the real RAM model to the Turing model. Below, size ( A , b , c ) \operatorname{size}(A,b,c) size ( A , b , c )
Theorem 1.8. Assume A ∈ Q ^ m × n A \in\widehat{\mathbb{Q}}^{m \times n} A ∈ Q m × n b ∈ Q ^ m b \in\widehat{\mathbb{Q}}^{m} b ∈ Q m c ∈ Q ^ n c \in\widehat{\mathbb{Q}}^{n} c ∈ Q n rank ( A ) = m \operatorname{rank}(A)=m rank ( A ) = m β ∈ ( 0 , 1 / 16 ] \beta\in(0,1/16] β ∈ ( 0 , 1/16 ] ( x , s ) ∈ N 2 ( β ) ∩ Q ^ 2 n (x,s) \in\mathcal{N}^{2}(\beta) \cap\widehat{\mathbb{Q}}^{2n} ( x , s ) ∈ N 2 ( β ) ∩ Q 2 n ( x ~ , s ~ ) ∈ N 2 ( 3 β ) ∩ Q ^ 2 n (\tilde{x},\tilde{s}) \in\mathcal{N}^{2}(3\beta) \cap\widehat{\mathbb{Q}}^{2n} ( x ~ , s ~ ) ∈ N 2 ( 3 β ) ∩ Q 2 n μ ( x ~ , s ~ ) ≤ ( 1 + β / ( 4 n 2 ) ) μ ( x , s ) \mu(\tilde{x},\tilde{s}) \leq(1+\beta/(4n^{2}))\mu(x,s) μ ( x ~ , s ~ ) ≤ ( 1 + β / ( 4 n 2 )) μ ( x , s ) size ( x ~ , s ~ ) = poly ( n ) size ( A , b , c ) \operatorname{size}(\tilde{x},\tilde{s})=\operatorname{poly}(n)\operatorname{size}(A,b,c) size ( x ~ , s ~ ) = poly ( n ) size ( A , b , c ) μ ( x ~ , s ~ ) = 0 \mu(\tilde{x},\tilde{s})=0 μ ( x ~ , s ~ ) = 0 ( v ~ , w ~ ) ∈ Q ^ 2 n (\tilde{v},\tilde{w}) \in\widehat{\mathbb{Q}}^{2n} ( v ~ , w ~ ) ∈ Q 2 n size ( v ~ , w ~ ) = poly ( n ) size ( A , b , c ) \operatorname{size}(\tilde{v},\tilde{w})=\operatorname{poly}(n)\operatorname{size}(A,b,c) size ( v ~ , w ~ ) = poly ( n ) size ( A , b , c ) ( x ~ , s ~ ) (\tilde{x},\tilde{s}) ( x ~ , s ~ ) ( v ~ , w ~ ) (\tilde{v},\tilde{w}) ( v ~ , w ~ ) 3 β 3\beta 3 β β \beta β
Interestingly, the rounding process may replace the current iterate by a much better solution further down the central path. As shown in Proposition 5.15 , using this rounding step already guarantees termination in at most an exponential number of iterations, thus already achieving the property highlighted in the title “Interior point methods are not worse than Simplex” of the paper [2 ] .
Initialization
The SLLS IPM, and more generally path-following IPMs, require an initial point ( x 0 , s 0 ) ∈ P > 0 × D > 0 (x^{0},s^{0}) \in\mathcal{P}_{>0} \times\mathcal{D}_{>0} ( x 0 , s 0 ) ∈ P > 0 × D > 0 LP ( A , b , c ) \operatorname{LP}(A,b,c) LP ( A , b , c )
A common initialization technique is the homogeneous self-dual formulation [TYM94]. However, writing the self-dual formulation of a generalized flow LP changes the combinatorial properties of the problem, and it is not clear if the straight line complexity admits a similar bound. Note also that already in the simpler case of (standard) network flows, the constraint matrix is totally unimodular, while the combined matrix does not have this property.
The following “big-M M M ( x , x ˉ , x ^ ) ∈ R 3 n (x,\bar{x},\hat{x}) \in\mathbb{R}^{3n} ( x , x ˉ , x ^ ) ∈ R 3 n
min ⟨ c , x ⟩ + M ⟨ 1 n , x ˉ ⟩ s.t. A x − A x ˉ = b , x + x ^ = 2 M 1 n , x , x ˉ , x ^ ≥ 0 n . (5) \min\langle c,x\rangle+ M\langle1_n,\bar{x}\rangle\quad\text{s.t.}\quad Ax-A\bar{x}=b,\quad x+\hat{x}=2M1_n,\quad x,\bar{x},\hat{x}\geq0_n. \tag*{(5)} min ⟨ c , x ⟩ + M ⟨ 1 n , x ˉ ⟩ s.t. A x − A x ˉ = b , x + x ^ = 2 M 1 n , x , x ˉ , x ^ ≥ 0 n . (5) Here, x ˉ \bar{x} x ˉ M M M x ^ \hat{x} x ^ 0 n ≤ x ≤ 2 M 1 n 0_n \leq x \leq2M1_n 0 n ≤ x ≤ 2 M 1 n s ˉ \bar{s} s ˉ x ˉ \bar{x} x ˉ s ^ \hat{s} s ^ x ^ \hat{x} x ^ s s s
Such a system, along with its dual, is easy to initialize for sufficiently large M M M x ˉ \bar{x} x ˉ χ ˉ \bar{\chi} χ ˉ χ ˉ A \bar{\chi}_A χ ˉ A
As long as there exists a pair of primal and dual optimal solutions ( x ⋆ , s ⋆ ) (x^\star,s^\star) ( x ⋆ , s ⋆ ) LP ( A , b , c ) \operatorname{LP}(A,b,c) LP ( A , b , c ) ∥ x ⋆ ∥ ∞ , ∥ s ⋆ ∥ ∞ < M \lVert x^\star\rVert_\infty,\lVert s^\star\rVert_\infty< M ∥ x ⋆ ∥ ∞ , ∥ s ⋆ ∥ ∞ < M M M M M M M χ ˉ A \bar{\chi}_A χ ˉ A log log χ ˉ A \log\log\bar{\chi}_A log log χ ˉ A [20 ] ; however, this is also not possible in the strongly polynomial model.
We present two initialization schemes. Both can be interpreted as multi-stage facial reduction strategies in which carefully fixing or projecting out variables yields an equivalent LP with strictly feasible primal and dual solutions.
The first initialization scheme in § 3 relies on a variant of the big-M M M (5) . In Lemma 3.1 , we observe that in the case that primal and dual strictly feasible solutions to LP ( A , b , c ) \operatorname{LP}(A,b,c) LP ( A , b , c ) M M M LP ( A , b , c ) \operatorname{LP}(A,b,c) LP ( A , b , c ) (5) coincide. Moreover, if only a primal strictly feasible solution is given (but no dual solution is available), then we can still choose M M M (5) are feasible to LP ( A , b , c ) \operatorname{LP}(A,b,c) LP ( A , b , c )
Our multi-stage initialization calls the IPM in Theorem 1.6 for different instantiations of (5) , by gradually deriving bounds on the parameters. In Stage I, we address conic feasibility: we first use an auxiliary system to find maximum support solutions to the conic systems A x = 0 m Ax=0_m A x = 0 m x ≥ 0 n x\geq0_n x ≥ 0 n A T y + s = 0 n A^\mathsf{T}y+s=0_n A T y + s = 0 n s ≥ 0 n s\geq0_n s ≥ 0 n Lemma 3.1 , using the solutions from Stage I. Finally, Stage III uses the primal and dual maximum support solutions to construct a system for optimization. Here, we solve the system obtained by deleting variables that are 0 in every primal solution, and by projecting out variables that are 0 in every dual solution. We obtain numerical parameters from the strictly feasible primal and dual solutions.
In § 4 , we develop a stronger, ‘big-M M M § 3 we only use the maximum support primal and dual optimal solutions from the output of Theorem 1.6 , here we also make use of the analytic center certificates ( v o u t , w o u t ) (v^{\mathrm{out}},w^{\mathrm{out}}) ( v out , w out ) LP ( A , b , c ) \operatorname{LP}(A,b,c) LP ( A , b , c )
The straight line complexity bounds stated in Theorem 1.7 follow because Stage I uses the system (S-Conic), whereas Stage II uses subsystems of those used in § 3 , obtained by deleting and projecting out variables from the systems (S-Primal) and (S-Dual). The Stage III system is obtained similarly from (S-Opt). In Lemma 2.10, we show that deletion and projection of ‘irrelevant’ variables do not increase the straight line complexity.
Implementation in the Turing model
We now give an overview of the strongly polynomial rounding step in Theorem 1.8. Starting from an iterate ( x , s ) ∈ N 2 ( β ) (x,s)\in\mathcal{N}^2(\beta) ( x , s ) ∈ N 2 ( β ) ( x ~ , s ~ ) ∈ N 2 ( 3 β ) (\tilde{x},\tilde{s})\in\mathcal{N}^2(3\beta) ( x ~ , s ~ ) ∈ N 2 ( 3 β )
To argue about the encoding length, we ‘anchor’ the point ( x ~ , s ~ ) (\tilde{x},\tilde{s}) ( x ~ , s ~ ) x x x s s s ( x , s ) (x,s) ( x , s ) ⟨ v , u ⟩ < ⟨ x , s ⟩ / 2 n \langle v,u\rangle< \langle x,s\rangle/2^n ⟨ v , u ⟩ < ⟨ x , s ⟩ / 2 n ⟨ v , u ⟩ > 2 n ⟨ x , s ⟩ \langle v,u\rangle> 2^n\langle x,s\rangle ⟨ v , u ⟩ > 2 n ⟨ x , s ⟩ ( v , u ) (v,u) ( v , u ) Lemma 5.10 , we try to find a value μ ⋆ ≈ ⟨ x , s ⟩ / n \mu^\star\approx\langle x,s\rangle/n μ ⋆ ≈ ⟨ x , s ⟩ / n μ ⋆ \mu^\star μ ⋆ ( v , u ) (v,u) ( v , u ) ⟨ x , s ⟩ / 2 n ≤ ⟨ v , u ⟩ ≤ 2 n ⟨ x , s ⟩ \langle x,s\rangle/2^n \le\langle v,u\rangle\le2^n\langle x,s\rangle ⟨ x , s ⟩ / 2 n ≤ ⟨ v , u ⟩ ≤ 2 n ⟨ x , s ⟩ ( x , s ) (x,s) ( x , s ) ( v , u ) (v,u) ( v , u ) ( x , s ) (x,s) ( x , s ) μ ⋆ \mu^\star μ ⋆
We note that, due to the reliance on the Minkowski–Weyl decomposition, the running time of this rounding procedure can be prohibitive in practice; its role is mainly conceptual, showing that rounding in the strongly polynomial model is possible at all. We present loose running-time bounds in Section A. Developing more efficient rounding methods remains for future research.
Organization. The rest of the paper is structured as follows. § 2 introduces some necessary background. § 3 presents the ‘big-M M M § 4 ; Theorem 1.7 is proved in § 4.4 . § 5 describes the rounding procedure and proves Theorem 1.8. Appendix § 8 gives a self-contained description of the strongly polynomial implementation of the Minkowski–Weyl theorem.
Preliminaries
Notation. We let R > 0 \mathbb{R}_{>0} R > 0 R ≥ 0 \mathbb{R}_{\ge0} R ≥ 0 Z > 0 , Z ≥ 0 , Q > 0 \mathbb{Z}_{>0},\mathbb{Z}_{\ge0},\mathbb{Q}_{>0} Z > 0 , Z ≥ 0 , Q > 0 Q ≥ 0 \mathbb{Q}_{\ge0} Q ≥ 0 n ∈ N n \in\mathbb{N} n ∈ N [ n ] ≔ { 1 , 2 , … , n } [n] \coloneqq\{1,2,\ldots,n\} [ n ] : = { 1 , 2 , … , n } 0 n , 1 n ∈ R n 0_n,1_n \in\mathbb{R}^n 0 n , 1 n ∈ R n x ∈ R n x \in\mathbb{R}^n x ∈ R n supp ( x ) ⊆ [ n ] \operatorname{supp}(x) \subseteq[n] supp ( x ) ⊆ [ n ] α ∈ R \alpha\in\mathbb{R} α ∈ R α + = max { α , 0 } \alpha^+ = \max\{\alpha,0\} α + = max { α , 0 } α − = max { − α , 0 } \alpha^- = \max\{-\alpha,0\} α − = max { − α , 0 } x ∈ R n x \in\mathbb{R}^n x ∈ R n x + x^+ x + x − x^- x −
We let ker ( A ) ⊆ R n \ker(A) \subseteq\mathbb{R}^n ker ( A ) ⊆ R n im ( A ) ⊆ R m \operatorname{im}(A) \subseteq\mathbb{R}^m im ( A ) ⊆ R m A ∈ R m × n A \in\mathbb{R}^{m \times n} A ∈ R m × n x , y ∈ R n x,y \in\mathbb{R}^n x , y ∈ R n ⟨ x , y ⟩ = x T y \langle x,y\rangle= x^\mathsf{T}y ⟨ x , y ⟩ = x T y x , y ∈ R n x,y \in\mathbb{R}^n x , y ∈ R n x ∘ y = ( x 1 y 1 , … , x n y n ) x \circ y = (x_1y_1,\ldots,x_ny_n) x ∘ y = ( x 1 y 1 , … , x n y n ) y ∈ R > 0 n y \in\mathbb{R}^n_{>0} y ∈ R > 0 n x / y = ( x 1 / y 1 , … , x n / y n ) x/y = (x_1/y_1,\ldots,x_n/y_n) x / y = ( x 1 / y 1 , … , x n / y n )
Pseudoinverse. For a matrix M ∈ R m × n M \in\mathbb{R}^{m \times n} M ∈ R m × n M + ∈ R n × m M^+ \in\mathbb{R}^{n \times m} M + ∈ R n × m M + M u = u M^+Mu = u M + M u = u u ∈ im ( M T ) u \in\operatorname{im}(M^\mathsf{T}) u ∈ im ( M T ) ker ( M + ) = ker ( M T ) \ker(M^+) = \ker(M^\mathsf{T}) ker ( M + ) = ker ( M T )
Lemma 2.1. For a matrix M ∈ R m × n M \in\mathbb{R}^{m \times n} M ∈ R m × n M + ∈ R n × m M^+ \in\mathbb{R}^{n \times m} M + ∈ R n × m M M + M = M MM^+M = M M M + M = M M M + M MM^+M M M + M I m − M M + I_m-MM^+ I m − M M + im ( M ) \operatorname{im}(M) im ( M ) ker ( M T ) \ker(M^\mathsf{T}) ker ( M T )
These properties are standard. For computing M + M^+ M + § 1.1 , where numbers cannot be reduced to lowest terms, we use the following lemma, proved in Section A.1. The bounds use the naive cubic bound for matrix multiplication, and we make no attempt to optimize them.
Lemma 2.2 (Strongly polynomial linear algebra). Let M ∈ Q m × n M \in\mathbb{Q}^{m \times n} M ∈ Q m × n m ≤ n m \le n m ≤ n poly ( n ) size ( M ) \operatorname{poly}(n)\operatorname{size}(M) poly ( n ) size ( M ) poly ( n ) size ( M ) \operatorname{poly}(n)\operatorname{size}(M) poly ( n ) size ( M )
(i) A basis of M M M B ⊆ [ n ] B \subseteq[n] B ⊆ [ n ] M B M_B M B M M M O ( n 5 ) O(n^5) O ( n 5 )
(ii) For m = n m=n m = n det M \det M det M adj ( M ) \operatorname{adj}(M) adj ( M ) M M M M − 1 M^{-1} M − 1 O ( n 4 ) O(n^4) O ( n 4 )
(iii) The pseudoinverse M + M^+ M + O ( n 5 ) O(n^5) O ( n 5 )
The same holds for m > n m>n m > n m m m n n n ( M T ) + = ( M + ) T (M^\mathsf{T})^+ = (M^+)^\mathsf{T} ( M T ) + = ( M + ) T
An important use of pseudoinverses will be for projecting out variables from an LP. Consider a partition I ∪ J = [ n ] I \cup J = [n] I ∪ J = [ n ]
Π : = I m − A I A I + , H : = Π A J , h : = Π b . (6) \Pi:= I_m-A_I A_I^+,\qquad H := \Pi A_J,\qquad h := \Pi b. \tag*{(6)} Π := I m − A I A I + , H := Π A J , h := Π b . (6) Here, Π \Pi Π ker ( A I T ) \ker(A_I^\mathsf{T}) ker ( A I T ) im ( A I ) \operatorname{im}(A_I) im ( A I ) A x = b Ax=b A x = b H x J = h Hx_J=h H x J = h Lemma 2.1 .
Fact 2.3. The linear system A x = b A x = b A x = b H x J = h H x_J = h H x J = h x J ∈ R J x_J \in\mathbb{R}^J x J ∈ R J H x J = h H x_J = h H x J = h
x I = A I + ( b − A J x J ) x_I = A_I^+ (b - A_J x_J) x I = A I + ( b − A J x J ) satisfies A J x J + A I x I = b A_J x_J + A_I x_I = b A J x J + A I x I = b
Complementary solutions. Let us recall the Goldman–Tucker theorem on strict complementarity [23 ] asserting that any maximum support primal and dual optimal solutions are strictly complementary.
Theorem 2.4 (Goldman & Tucker). Assume that both the primal and dual LPs in LP ( A , b , c ) \operatorname{LP}(A,b,c) LP ( A , b , c ) B ∪ N = [ n ] B \cup N = [n] B ∪ N = [ n ] x ∈ R ≥ 0 n x \in\mathbb{R}_{\geq0}^n x ∈ R ≥ 0 n ( y , s ) ∈ R m × R ≥ 0 n (y,s) \in\mathbb{R}^m \times\mathbb{R}_{\geq0}^n ( y , s ) ∈ R m × R ≥ 0 n B = supp ( x ) B = \operatorname{supp}(x) B = supp ( x ) N = supp ( s ) N = \operatorname{supp}(s) N = supp ( s )
We derive corollaries for the conic, dual-feasibility, and primal-feasibility settings. The corresponding complementary partitions will play important roles in our multi-stage initialization approaches.
Corollary 2.5. Let A ∈ R m × n A \in\mathbb{R}^{m \times n} A ∈ R m × n b ∈ R m b \in\mathbb{R}^m b ∈ R m c ∈ R n c \in\mathbb{R}^n c ∈ R n
(i) There exists a unique partition B ∪ N = [ n ] B \cup N = [n] B ∪ N = [ n ] x , s ∈ R n x,s \in\mathbb{R}^n x , s ∈ R n y ∈ R m y \in\mathbb{R}^m y ∈ R m
A x = 0 m , x ≥ 0 n , supp ( x ) = B and A T y + s = 0 n , s ≥ 0 n , supp ( s ) = N . A x = 0_m,\ x \geq0_n,\ \operatorname{supp}(x) = B \quad\text{and} \quad A^{\mathsf{T}} y + s = 0_n,\ s \geq0_n,\ \operatorname{supp}(s) = N. A x = 0 m , x ≥ 0 n , supp ( x ) = B and A T y + s = 0 n , s ≥ 0 n , supp ( s ) = N . (ii) Assume that A T y + s = c A^{\mathsf{T}} y + s = c A T y + s = c s ≥ 0 n s \geq0_n s ≥ 0 n B ∪ N = [ n ] B \cup N = [n] B ∪ N = [ n ] x , s ∈ R n x,s \in\mathbb{R}^n x , s ∈ R n y ∈ R m y \in\mathbb{R}^m y ∈ R m
A x = 0 m , x ≥ 0 n , supp ( x ) = B and A T y + s = c , s ≥ 0 n , supp ( s ) = N . A x = 0_m,\ x \geq0_n,\ \operatorname{supp}(x) = B \quad\text{and} \quad A^{\mathsf{T}} y + s = c,\ s \geq0_n,\ \operatorname{supp}(s) = N. A x = 0 m , x ≥ 0 n , supp ( x ) = B and A T y + s = c , s ≥ 0 n , supp ( s ) = N . (iii) Assume that A x = b A x = b A x = b x ≥ 0 n x \geq0_n x ≥ 0 n B ∪ N = [ n ] B \cup N = [n] B ∪ N = [ n ] x , s ∈ R n x,s \in\mathbb{R}^n x , s ∈ R n y ∈ R m y \in\mathbb{R}^m y ∈ R m
A x = b , x ≥ 0 n , supp ( x ) = B and A T y + s = 0 n , s ≥ 0 n , supp ( s ) = N . A x = b,\ x \geq0_n,\ \operatorname{supp}(x) = B \quad\text{and} \quad A^{\mathsf{T}} y + s = 0_n,\ s \geq0_n,\ \operatorname{supp}(s) = N. A x = b , x ≥ 0 n , supp ( x ) = B and A T y + s = 0 n , s ≥ 0 n , supp ( s ) = N . From the interior point literature, initialization (§ 3 and § 4 ) only requires the following lemma that shows that the choice of parameter μ = ⟨ x , s ⟩ / n \mu= \langle x,s\rangle/n μ = ⟨ x , s ⟩ / n ( x , s ) (x,s) ( x , s ) § 5.1 .
Lemma 2.6 ([MT03, Lemma 4.4]). For β ∈ ( 0 , 1 / 4 ] \beta\in(0,1/4] β ∈ ( 0 , 1/4 ] ( x , s ) ∈ P > 0 × D > 0 (x,s) \in\mathcal{P}_{>0} \times\mathcal{D}_{>0} ( x , s ) ∈ P > 0 × D > 0 μ ′ > 0 \mu' > 0 μ ′ > 0 ∥ x ∘ s / μ ′ − 1 n ∥ 2 ≤ β \left\|x \circ s/\mu' - 1_n\right\|_2 \leq\beta ∥ x ∘ s / μ ′ − 1 n ∥ 2 ≤ β μ : = ⟨ x , s ⟩ / n \mu:= \langle x,s\rangle/n μ := ⟨ x , s ⟩ / n
∥ x ∘ s μ − 1 n ∥ 2 ≤ β 1 − β , and ( 1 − β n ) μ ′ ≤ μ ≤ ( 1 + β n ) μ ′ . (7) \left\|\frac{x \circ s}{\mu} - 1_n\right\|_2 \leq\frac{\beta}{1-\beta},
\qquad\text{and} \qquad
\left(1-\frac{\beta}{\sqrt{n}}\right)\mu' \leq\mu\leq\left(1+\frac{\beta}{\sqrt{n}}\right)\mu'. \tag*{(7)} μ x ∘ s − 1 n 2 ≤ 1 − β β , and ( 1 − n β ) μ ′ ≤ μ ≤ ( 1 + n β ) μ ′ . (7) The sublevel sets. Assume P , D ≠ ∅ \mathcal{P},\mathcal{D} \ne\varnothing P , D = ∅ LP ( A , b , c ) \operatorname{LP}(A,b,c) LP ( A , b , c ) ( x ⋆ , y ⋆ , s ⋆ ) (x^\star,y^\star,s^\star) ( x ⋆ , y ⋆ , s ⋆ ) ⟨ c , x ⋆ ⟩ = ⟨ b , y ⋆ ⟩ = v ⋆ \langle c,x^\star\rangle= \langle b,y^\star\rangle= v^\star ⟨ c , x ⋆ ⟩ = ⟨ b , y ⋆ ⟩ = v ⋆ ⟨ x ⋆ , s ⋆ ⟩ = 0 \langle x^\star,s^\star\rangle= 0 ⟨ x ⋆ , s ⋆ ⟩ = 0 x i ⋆ s i ⋆ = 0 x_i^\star s_i^\star= 0 x i ⋆ s i ⋆ = 0 i ∈ [ n ] i \in[n] i ∈ [ n ]
Recall the definitions of the sublevel sets P g , D g \mathcal{P}_g,\mathcal{D}_g P g , D g (2) as the sets of primal and dual solutions with objective value within g g g v ⋆ v^\star v ⋆ ( x , y , s ) (x,y,s) ( x , y , s ) LP ( A , b , c ) \operatorname{LP}(A,b,c) LP ( A , b , c ) ⟨ c , x ⟩ − ⟨ b , y ⟩ = ⟨ x , s ⟩ \langle c,x\rangle-\langle b,y\rangle=\langle x,s\rangle ⟨ c , x ⟩ − ⟨ b , y ⟩ = ⟨ x , s ⟩ ⟨ x , s ⋆ ⟩ = ⟨ c , x ⟩ − v ⋆ \langle x,s^\star\rangle=\langle c,x\rangle-v^\star ⟨ x , s ⋆ ⟩ = ⟨ c , x ⟩ − v ⋆ ⟨ x ⋆ , s ⟩ = v ⋆ − ⟨ b , y ⟩ \langle x^\star,s\rangle=v^\star-\langle b,y\rangle ⟨ x ⋆ , s ⟩ = v ⋆ − ⟨ b , y ⟩ P g \mathcal{P}_g P g D g \mathcal{D}_g D g
P g = { x ∈ P : ⟨ x , s ⋆ ⟩ ≤ g } and D g = { s ∈ D : ⟨ x ⋆ , s ⟩ ≤ g } . \mathcal{P}_g = \{x \in\mathcal{P} : \langle x,s^\star\rangle\leq g\}
\quad\text{and}\quad
\mathcal{D}_g = \{s \in\mathcal{D} : \langle x^\star,s\rangle\leq g\}. P g = { x ∈ P : ⟨ x , s ⋆ ⟩ ≤ g } and D g = { s ∈ D : ⟨ x ⋆ , s ⟩ ≤ g } . These expressions are in fact independent of the choice of optimal solutions ( x ⋆ , y ⋆ , s ⋆ ) (x^\star,y^\star,s^\star) ( x ⋆ , y ⋆ , s ⋆ )
Proposition 2.7. If P > 0 , D > 0 ≠ ∅ \mathcal{P}_{>0},\mathcal{D}_{>0} \ne\varnothing P > 0 , D > 0 = ∅ g , g ′ ≥ 0 g,g' \geq0 g , g ′ ≥ 0 i ∈ [ n ] i \in[n] i ∈ [ n ] x i max ( g ) ⋅ s i max ( g ′ ) ≤ g + g ′ x_i^{\max}(g) \cdot s_i^{\max}(g') \leq g+g' x i m a x ( g ) ⋅ s i m a x ( g ′ ) ≤ g + g ′ P g \mathcal{P}_g P g D g \mathcal{D}_g D g
When P > 0 = ∅ \mathcal{P}_{>0} = \varnothing P > 0 = ∅ D > 0 = ∅ \mathcal{D}_{>0} = \varnothing D > 0 = ∅ P , D ≠ ∅ \mathcal{P},\mathcal{D} \ne\varnothing P , D = ∅ i ∈ [ n ] i \in[n] i ∈ [ n ]
These can be easily characterized as follows.
Lemma 2.8. Assume P , D ≠ ∅ \mathcal{P},\mathcal{D}\ne\varnothing P , D = ∅ i ∈ [ n ] i\in[n] i ∈ [ n ]
(i) If x i m ( g ) = ∞ x_i^{\mathsf m}(g)=\infty x i m ( g ) = ∞ g ∈ R ≥ 0 g\in\mathbb{R}_{\ge0} g ∈ R ≥ 0 x i m ( g ′ ) = ∞ x_i^{\mathsf m}(g')=\infty x i m ( g ′ ) = ∞ g ′ ∈ R ≥ 0 g'\in\mathbb{R}_{\ge0} g ′ ∈ R ≥ 0 x i m ( g ) x_i^{\mathsf m}(g) x i m ( g ) s i m ( g ) = ∞ s_i^{\mathsf m}(g)=\infty s i m ( g ) = ∞ g ∈ R ≥ 0 g\in\mathbb{R}_{\ge0} g ∈ R ≥ 0 s i m ( g ) s_i^{\mathsf m}(g) s i m ( g )
(ii) x i m x_i^{\mathsf m} x i m y ′ ∈ R m y'\in\mathbb{R}^m y ′ ∈ R m s ′ ∈ R ≥ 0 n s'\in\mathbb{R}_{\ge0}^n s ′ ∈ R ≥ 0 n A T y ′ + s ′ = 0 n A^{\mathsf T}y'+s'=0_n A T y ′ + s ′ = 0 n s i ′ > 0 s'_i>0 s i ′ > 0 ⟨ b , y ′ ⟩ = 0 \langle b,y'\rangle=0 ⟨ b , y ′ ⟩ = 0
(iii) s i m s_i^{\mathsf m} s i m x ′ ∈ R ≥ 0 n x'\in\mathbb{R}_{\ge0}^n x ′ ∈ R ≥ 0 n A x ′ = 0 m Ax'=0_m A x ′ = 0 m x i ′ > 0 x'_i>0 x i ′ > 0 ⟨ c , x ′ ⟩ = 0 \langle c,x'\rangle=0 ⟨ c , x ′ ⟩ = 0
Proof. For part (i), x i m ( g ) = ∞ x_i^{\mathsf m}(g)=\infty x i m ( g ) = ∞ z ∈ R ≥ 0 n z\in\mathbb{R}_{\ge0}^n z ∈ R ≥ 0 n A z = 0 Az=0 A z = 0 z i > 0 z_i>0 z i > 0 ⟨ c , z ⟩ ≤ 0 \langle c,z\rangle\le0 ⟨ c , z ⟩ ≤ 0 ⟨ c , z ⟩ = 0 \langle c,z\rangle=0 ⟨ c , z ⟩ = 0 ⟨ c , z ⟩ < 0 \langle c,z\rangle<0 ⟨ c , z ⟩ < 0 D = ∅ \mathcal{D}=\varnothing D = ∅ x ⋆ + α z x^\star+\alpha z x ⋆ + α z α ∈ R ≥ 0 \alpha\in\mathbb{R}_{\ge0} α ∈ R ≥ 0 x i m ( 0 ) = ∞ x_i^{\mathsf m}(0)=\infty x i m ( 0 ) = ∞ x i m ( g ) x_i^{\mathsf m}(g) x i m ( g ) s i m ( g ) s_i^{\mathsf m}(g) s i m ( g )
For part (ii), first assume x i m x_i^{\mathsf m} x i m x i = 0 x_i=0 x i = 0 A x = b Ax=b A x = b x ≥ 0 x\ge0 x ≥ 0 y ′ ∈ R m y'\in\mathbb{R}^m y ′ ∈ R m s ′ ∈ R ≥ 0 n s'\in\mathbb{R}_{\ge0}^n s ′ ∈ R ≥ 0 n A T y ′ + s ′ = 0 n A^{\mathsf T}y'+s'=0_n A T y ′ + s ′ = 0 n s i ′ > 0 s'_i>0 s i ′ > 0 x ′ x' x ′ ⟨ b , y ′ ⟩ = ⟨ A x ′ , y ′ ⟩ = ⟨ x ′ , A T y ′ ⟩ = − ⟨ x ′ , s ′ ⟩ = 0 \langle b,y'\rangle=\langle Ax',y'\rangle=\langle x',A^{\mathsf T}y'\rangle=-\langle x',s'\rangle=0 ⟨ b , y ′ ⟩ = ⟨ A x ′ , y ′ ⟩ = ⟨ x ′ , A T y ′ ⟩ = − ⟨ x ′ , s ′ ⟩ = 0 x ′ x' x ′ s ′ s' s ′ A T y ′ + s ′ = 0 n A^{\mathsf T}y'+s'=0_n A T y ′ + s ′ = 0 n s ′ ≥ 0 n s'\ge0_n s ′ ≥ 0 n s i ′ > 0 s'_i>0 s i ′ > 0 b T y ′ = 0 b^{\mathsf T}y'=0 b T y ′ = 0 x ∈ R n x\in\mathbb{R}^n x ∈ R n A x = b Ax=b A x = b x ≥ 0 x\ge0 x ≥ 0 x i s i ′ ≤ x T s ′ = − x T A T y ′ = b T y ′ = 0 x_i s'_i\le x^{\mathsf T}s'=-x^{\mathsf T}A^{\mathsf T}y'=b^{\mathsf T}y'=0 x i s i ′ ≤ x T s ′ = − x T A T y ′ = b T y ′ = 0 x i = 0 x_i=0 x i = 0 x i m x_i^{\mathsf m} x i m
Note that according to parts (ii) and (iii), x i m x_i^{\mathsf m} x i m s i m s_i^{\mathsf m} s i m s i m s_i^{\mathsf m} s i m x i m x_i^{\mathsf m} x i m
Operations preserving straight-line complexity
The following lemma asserts that column rescalings do not affect the straight line complexity bounds.
Lemma 2.9. Given an instance of L P ( A , b , c ) LP(A,b,c) L P ( A , b , c ) ( A , b , c ) (A,b,c) ( A , b , c ) D ∈ R n × n D\in\mathbb{R}^{n\times n} D ∈ R n × n ( A D , b , D c ) (AD,b,Dc) ( A D , b , D c ) ( x m , s m ) (x^{\mathsf m},s^{\mathsf m}) ( x m , s m ) ( x ˉ m , s ˉ m ) (\bar{x}^{\mathsf m},\bar{s}^{\mathsf m}) ( x ˉ m , s ˉ m ) SLC η ( x i m ) = SLC η ( x ˉ i m ) \operatorname{SLC}_{\eta}(x_i^{\mathsf m})=\operatorname{SLC}_{\eta}(\bar{x}_i^{\mathsf m}) SLC η ( x i m ) = SLC η ( x ˉ i m ) SLC η ( s i m ) = SLC η ( s ˉ i m ) \operatorname{SLC}_{\eta}(s_i^{\mathsf m})=\operatorname{SLC}_{\eta}(\bar{s}_i^{\mathsf m}) SLC η ( s i m ) = SLC η ( s ˉ i m ) η ∈ [ 0 , 1 ] \eta\in[0,1] η ∈ [ 0 , 1 ] i ∈ [ n ] i\in[n] i ∈ [ n ]
Proof. We prove the claim only for the primal max central paths; the proof for the dual case is analogous. Note that there is a one-to-one mapping x ↦ D − 1 x x\mapsto D^{-1}x x ↦ D − 1 x ⟨ c , x ⟩ = ⟨ D c , D − 1 x ⟩ \langle c,x\rangle=\langle Dc,D^{-1}x\rangle ⟨ c , x ⟩ = ⟨ D c , D − 1 x ⟩ x ˉ m ( g ) = D − 1 x m ( g ) \bar{x}^{\mathsf m}(g)=D^{-1}x^{\mathsf m}(g) x ˉ m ( g ) = D − 1 x m ( g ) g ≥ 0 g\ge0 g ≥ 0 x i m x_i^{\mathsf m} x i m x ˉ i m \bar{x}_i^{\mathsf m} x ˉ i m
Our next lemma shows that the straight line complexities of the remaining variables are unchanged under two reductions: deleting the variables whose primal max central paths are identically zero, and projecting out the variables whose dual max central paths are identically zero.
Lemma 2.10. Consider an instance of L P ( A , b , c ) LP(A,b,c) L P ( A , b , c ) P ∪ D ∪ R = [ n ] P\cup D\cup R=[n] P ∪ D ∪ R = [ n ] ( x , s ) (x,s) ( x , s ) L P ( A , b , c ) LP(A,b,c) L P ( A , b , c ) x D = 0 D x_D=0_D x D = 0 D s P = 0 P s_P=0_P s P = 0 P s s s
(i) Consider the system
min ⟨ s P ∪ R , x P ∪ R ⟩ , s.t. A P ∪ R x P ∪ R = b , x P ∪ R ≥ 0 P ∪ R . (8) \min\langle s_{P\cup R},x_{P\cup R}\rangle,\qquad\text{s.t.}\quad A_{P\cup R}x_{P\cup R}=b,\ x_{P\cup R}\ge0_{P\cup R}. \tag*{(8)} min ⟨ s P ∪ R , x P ∪ R ⟩ , s.t. A P ∪ R x P ∪ R = b , x P ∪ R ≥ 0 P ∪ R . (8) Then, for any i ∈ P ∪ R i\in P\cup R i ∈ P ∪ R x i m ( g ) = x ˉ i m ( g ) x_i^{\mathsf m}(g)=\bar{x}_i^{\mathsf m}(g) x i m ( g ) = x ˉ i m ( g ) g ∈ R ≥ 0 g\in\mathbb{R}_{\ge0} g ∈ R ≥ 0 x i m ( g ) x_i^{\mathsf m}(g) x i m ( g ) x ˉ i m ( g ) \bar{x}_i^{\mathsf m}(g) x ˉ i m ( g ) L P ( A , b , c ) LP(A,b,c) L P ( A , b , c )
(ii) Define
Π : = I m − A P A P + , H : = Π A D ∪ R , and h : = Π b , \Pi:=I_m-A_P A_P^+,\qquad H:=\Pi A_{D\cup R},\qquad\text{and}\qquad h:=\Pi b, Π := I m − A P A P + , H := Π A D ∪ R , and h := Π b , and consider the system
min ⟨ s D ∪ R , x D ∪ R ⟩ , s.t. H x D ∪ R = h , x D ∪ R ≥ 0 D ∪ R . (9) \min\langle s_{D\cup R},x_{D\cup R}\rangle,\qquad\text{s.t.}\quad Hx_{D\cup R}=h,\ x_{D\cup R}\ge0_{D\cup R}. \tag*{(9)} min ⟨ s D ∪ R , x D ∪ R ⟩ , s.t. H x D ∪ R = h , x D ∪ R ≥ 0 D ∪ R . (9) Then, for any i ∈ D ∪ R i\in D\cup R i ∈ D ∪ R x i m ( g ) = x ˉ i m ( g ) x_i^{\mathsf m}(g)=\bar{x}_i^{\mathsf m}(g) x i m ( g ) = x ˉ i m ( g ) g ∈ R ≥ 0 g\in\mathbb{R}_{\ge0} g ∈ R ≥ 0 x i m ( g ) x_i^{\mathsf m}(g) x i m ( g ) x ˉ i m ( g ) \bar{x}_i^{\mathsf m}(g) x ˉ i m ( g ) L P ( A , b , c ) LP(A,b,c) L P ( A , b , c )
(iii) Define
Π : = I m − A P A P + , H : = Π A R , and h : = Π b , \Pi:= I_m - A_P A_P^+,\qquad H := \Pi A_R,\qquad\text{and}\qquad h := \Pi b, Π := I m − A P A P + , H := Π A R , and h := Π b , and consider the system
min ⟨ s R , x R ⟩ , s.t. H x R = h , x R ≥ 0 R . (10) \min\langle s_R,x_R\rangle,\qquad\text{s.t.}\qquad Hx_R=h,\ x_R\geq0_R. \tag*{(10)} min ⟨ s R , x R ⟩ , s.t. H x R = h , x R ≥ 0 R . (10) Then, for any i ∈ R i\in R i ∈ R x i m ( g ) = x ˉ i m ( g ) x_i^{\mathrm{m}}(g)=\bar{x}_i^{\mathrm{m}}(g) x i m ( g ) = x ˉ i m ( g ) g ∈ R ≥ 0 g\in\mathbb{R}_{\geq0} g ∈ R ≥ 0 x i m ( g ) x_i^{\mathrm{m}}(g) x i m ( g ) x ˉ i m ( g ) \bar{x}_i^{\mathrm{m}}(g) x ˉ i m ( g )
Proof. Part (i). First, note that in LP(A,b,c) and in LP( A , b , s ) (A,b,s) ( A , b , s ) x i x_i x i (8) arises from LP( A , b , s ) (A,b,s) ( A , b , s ) x ∈ R n x\in\mathbb{R}^n x ∈ R n ( A , b , s ) (A,b,s) ( A , b , s ) x D = 0 D x_D=0_D x D = 0 D x P ∪ R x_{P\cup R} x P ∪ R (8) . This also implies that the two LPs have the same optimum value. The claim follows by the definition of the max central path.
Part (ii). As in Part (i), the max-central paths in LP(A,b,c) and in LP( A , b , s ) (A,b,s) ( A , b , s ) ( A , b , s ) (A,b,s) ( A , b , s ) (9) . We do so by exhibiting maps in both directions between the two feasible regions that preserve the objective value.
By Corollary 2.5(ii), the maximum support partition is B ∪ N = [ n ] B\cup N=[n] B ∪ N = [ n ] N ⊆ D ∪ R N\subseteq D\cup R N ⊆ D ∪ R B ⊇ P B\supseteq P B ⊇ P z ∈ R ≥ 0 n z\in\mathbb{R}_{\geq0}^n z ∈ R ≥ 0 n A z = 0 m Az=0_m A z = 0 m supp ( z ) = B \operatorname{supp}(z)=B supp ( z ) = B ⟨ s , z ⟩ = 0 \langle s,z\rangle=0 ⟨ s , z ⟩ = 0 supp ( s ) ⊆ N \operatorname{supp}(s)\subseteq N supp ( s ) ⊆ N
Let x ∈ R ≥ 0 n x\in\mathbb{R}_{\geq0}^n x ∈ R ≥ 0 n A x = b Ax=b A x = b Π A P = 0 \Pi A_P=0 Π A P = 0 Π \Pi Π A x = b Ax=b A x = b H x D ∪ R = h Hx_{D\cup R}=h H x D ∪ R = h x D ∪ R x_{D\cup R} x D ∪ R (9) , and ⟨ s , x ⟩ = ⟨ s D ∪ R , x D ∪ R ⟩ \langle s,x\rangle=\langle s_{D\cup R},x_{D\cup R}\rangle ⟨ s , x ⟩ = ⟨ s D ∪ R , x D ∪ R ⟩ s P = 0 P s_P=0_P s P = 0 P D ∪ R D\cup R D ∪ R
Conversely, let x ˉ ∈ R ≥ 0 D ∪ R \bar{x}\in\mathbb{R}_{\geq0}^{D\cup R} x ˉ ∈ R ≥ 0 D ∪ R H x ˉ = h H\bar{x}=h H x ˉ = h x x x x D ∪ R = x ˉ x_{D\cup R}=\bar{x} x D ∪ R = x ˉ x P = A P + ( b − A D ∪ R x ˉ ) x_P=A_P^+(b-A_{D\cup R}\bar{x}) x P = A P + ( b − A D ∪ R x ˉ ) A x = b Ax=b A x = b x P ≱ 0 P x_P\not\geq0_P x P ≥ 0 P z P > 0 P z_P>0_P z P > 0 P x + α z x+\alpha z x + α z α > 0 \alpha>0 α > 0 A ( x + α z ) = b A(x+\alpha z)=b A ( x + α z ) = b ⟨ s , z ⟩ = 0 \langle s,z\rangle=0 ⟨ s , z ⟩ = 0 ⟨ s , x + α z ⟩ = ⟨ s , x ⟩ = ⟨ s D ∪ R , x ˉ ⟩ \langle s,x+\alpha z\rangle=\langle s,x\rangle=\langle s_{D\cup R},\bar{x}\rangle ⟨ s , x + α z ⟩ = ⟨ s , x ⟩ = ⟨ s D ∪ R , x ˉ ⟩ x + α z x+\alpha z x + α z ( A , b , s ) (A,b,s) ( A , b , s ) x ˉ \bar{x} x ˉ (9) . Furthermore, x ˉ N = ( x + α z ) N \bar{x}_N=(x+\alpha z)_N x ˉ N = ( x + α z ) N supp ( z ) = B \operatorname{supp}(z)=B supp ( z ) = B N N N
We thus have maps in both directions between the feasible regions of LP( A , b , s ) (A,b,s) ( A , b , s ) (9) that preserve objective values. Hence, the two LPs have the same optimum value, and both maps also preserve the optimality gap. As the first map preserves the coordinates in D ∪ R D\cup R D ∪ R N N N x ˉ i m ( g ) ≥ x i m ( g ) \bar{x}_i^{\mathrm{m}}(g)\geq x_i^{\mathrm{m}}(g) x ˉ i m ( g ) ≥ x i m ( g ) i ∈ D ∪ R i\in D\cup R i ∈ D ∪ R x ˉ i m ( g ) ≤ x i m ( g ) \bar{x}_i^{\mathrm{m}}(g)\leq x_i^{\mathrm{m}}(g) x ˉ i m ( g ) ≤ x i m ( g ) i ∈ N ⊆ D ∪ R i\in N\subseteq D\cup R i ∈ N ⊆ D ∪ R g ∈ R ≥ 0 g\in\mathbb{R}_{\geq0} g ∈ R ≥ 0
Finally, consider a coordinate i ∈ ( D ∪ R ) ∩ B i\in(D\cup R)\cap B i ∈ ( D ∪ R ) ∩ B z i > 0 z_i>0 z i > 0 x ⋆ x^\star x ⋆ x ⋆ + α z x^\star+\alpha z x ⋆ + α z ( x ⋆ + α z ) i (x^\star+\alpha z)_i ( x ⋆ + α z ) i x i m x_i^{\mathrm{m}} x i m x ˉ i m ( g ) ≥ x i m ( g ) \bar{x}_i^{\mathrm{m}}(g)\geq x_i^{\mathrm{m}}(g) x ˉ i m ( g ) ≥ x i m ( g ) x ˉ i m \bar{x}_i^{\mathrm{m}} x ˉ i m
Part (iii). We first use part (i) to delete all primal variables in D D D (8) , s ˉ P = 0 P \bar{s}_P=0_P s ˉ P = 0 P ( y ~ , s ~ ) (\tilde{y},\tilde{s}) ( y ~ , s ~ ) ( y ^ , s ^ ) (\hat{y},\hat{s}) ( y ^ , s ^ ) A T y ^ + s ^ = 0 n A^\mathsf{T}\hat{y}+\hat{s}=0_n A T y ^ + s ^ = 0 n s ^ ≥ 0 n \hat{s}\geq0_n s ^ ≥ 0 n D ⊆ supp ( s ^ ) D\subseteq\operatorname{supp}(\hat{s}) D ⊆ supp ( s ^ ) s ˉ D : = s D − A D T y ~ \bar{s}_D:=s_D-A_D^\mathsf{T}\tilde{y} s ˉ D := s D − A D T y ~ α > 0 \alpha>0 α > 0 s ˉ D + α s ^ D ≥ 0 D \bar{s}_D+\alpha\hat{s}_D\geq0_D s ˉ D + α s ^ D ≥ 0 D s ~ + α s ^ \tilde{s}+\alpha\hat{s} s ~ + α s ^ s ~ P ≥ 0 P \tilde{s}_P\geq0_P s ~ P ≥ 0 P P P P D D D P P P
Multi-stage big-M M M
The extended system
In this section, we describe our first initialization procedure that is based on the following variant of the “big-M M M
min ⟨ c , x ⟩ + M ⟨ 1 n , x ˉ ⟩ s.t. A x − A x ˉ = b x + x ^ = 2 K 1 n x , x ˉ , x ^ ≥ 0 n , max ⟨ b , y ⟩ − 2 K ⟨ 1 n , s ^ ⟩ s.t. A T y + s − s ^ = c − A T y + s ˉ = M 1 n s , s ˉ , s ^ ≥ 0 n . MSys ( A , b , c , M , K ) \begin{aligned}
\min\quad&\langle c,x\rangle+M\langle1_n,\bar{x}\rangle\\
\text{s.t.}\quad&Ax-A\bar{x}=b\\
&x+\hat{x}=2K1_n\\
&x,\bar{x},\hat{x}\geq0_n,
\end{aligned}
\qquad
\begin{aligned}
\max\quad&\langle b,y\rangle-2K\langle1_n,\hat{s}\rangle\\
\text{s.t.}\quad&A^\mathsf{T}y+s-\hat{s}=c\\
&-A^\mathsf{T}y+\bar{s}=M1_n\\
&s,\bar{s},\hat{s}\geq0_n.
\end{aligned}
\qquad\operatorname{MSys}(A,b,c,M,K) min s.t. ⟨ c , x ⟩ + M ⟨ 1 n , x ˉ ⟩ A x − A x ˉ = b x + x ^ = 2 K 1 n x , x ˉ , x ^ ≥ 0 n , max s.t. ⟨ b , y ⟩ − 2 K ⟨ 1 n , s ^ ⟩ A T y + s − s ^ = c − A T y + s ˉ = M 1 n s , s ˉ , s ^ ≥ 0 n . MSys ( A , b , c , M , K ) The auxiliary variables play the following roles. The variable x ˉ \bar{x} x ˉ M M M A ( x − x ˉ ) = b A(x-\bar{x})=b A ( x − x ˉ ) = b x ^ = 2 K 1 n − x \hat{x}=2K1_n-x x ^ = 2 K 1 n − x x ≤ 2 K 1 n x\leq2K1_n x ≤ 2 K 1 n x = K 1 n x=K1_n x = K 1 n Lemma 3.1 (i)). The complementary pairs are ( x , s ) (x,s) ( x , s ) ( x ˉ , s ˉ ) (\bar{x},\bar{s}) ( x ˉ , s ˉ ) ( x ^ , s ^ ) (\hat{x},\hat{s}) ( x ^ , s ^ )
The following lemma shows that for sufficiently large values of K K K M M M ( A , b , c , M , K ) (A,b,c,M,K) ( A , b , c , M , K ) ( A , b , c ) (A,b,c) ( A , b , c ) ( A , b , c , M , K ) (A,b,c,M,K) ( A , b , c , M , K )
Lemma 3.1. Let A ∈ R m × n A \in\mathbb{R}^{m \times n} A ∈ R m × n b ∈ R m b \in\mathbb{R}^{m} b ∈ R m d ∈ R n d \in\mathbb{R}^{n} d ∈ R n A d = b Ad=b A d = b c ∈ R n c \in\mathbb{R}^{n} c ∈ R n β ∈ ( 0 , 1 / 2 ] \beta\in(0,1/2] β ∈ ( 0 , 1/2 ]
K > 4 β ∥ d ∥ 2 and M > 4 β ∥ c ∥ 2 . (11) K > \frac{4}{\beta}\lVert d\rVert_{2} \quad\text{and} \quad M > \frac{4}{\beta}\lVert c\rVert_{2}. \tag*{(11)} K > β 4 ∥ d ∥ 2 and M > β 4 ∥ c ∥ 2 . (11) Then, the following hold.
(i) The vectors
( x , x ˉ , x ^ ) : = ( K 1 n , K 1 n − d , K 1 n ) and (x,\bar{x},\hat{x}) := (K1_n,K1_n-d,K1_n) \quad\text{and} ( x , x ˉ , x ^ ) := ( K 1 n , K 1 n − d , K 1 n ) and ( y , s , s ˉ , s ^ ) : = ( 0 m , M 1 n + c , M 1 n , M 1 n ) (y,s,\bar{s},\hat{s}) := (0_m,M1_n+c,M1_n,M1_n) ( y , s , s ˉ , s ^ ) := ( 0 m , M 1 n + c , M 1 n , M 1 n ) are feasible primal and dual solutions to MSys( A , b , c , M , K ) (A,b,c,M,K) ( A , b , c , M , K ) N 2 ( β ) \mathcal{N}^{2}(\beta) N 2 ( β )
(ii) If c , d > 0 n c,d > 0_n c , d > 0 n
K > ⟨ c , d ⟩ min i ∈ [ n ] c i , and M > ⟨ c , d ⟩ min i ∈ [ n ] d i (12) K > \frac{\langle c,d\rangle}{\min_{i\in[n]}c_i}, \quad\text{and} \quad M > \frac{\langle c,d\rangle}{\min_{i\in[n]}d_i} \tag*{(12)} K > min i ∈ [ n ] c i ⟨ c , d ⟩ , and M > min i ∈ [ n ] d i ⟨ c , d ⟩ (12) hold, then every primal optimal solution ( x , x ˉ , x ^ ) (x,\bar{x},\hat{x}) ( x , x ˉ , x ^ ) ( y , s , s ˉ , s ^ ) (y,s,\bar{s},\hat{s}) ( y , s , s ˉ , s ^ ) ( A , b , c , M , K ) (A,b,c,M,K) ( A , b , c , M , K ) x ˉ = s ^ = 0 n \bar{x}=\hat{s}=0_n x ˉ = s ^ = 0 n x x x ( y , s ) (y,s) ( y , s ) ( A , b , c ) (A,b,c) ( A , b , c )
(iii) If d > 0 n d > 0_n d > 0 n
M > 4 K ∥ c ∥ 1 min i ∈ [ n ] d i , (13) M > \frac{4K\lVert c\rVert_{1}}{\min_{i\in[n]}d_i}, \tag*{(13)} M > min i ∈ [ n ] d i 4 K ∥ c ∥ 1 , (13) holds, then every primal optimal solution ( x , x ˉ , x ^ ) (x,\bar{x},\hat{x}) ( x , x ˉ , x ^ ) ( A , b , c , M , K ) (A,b,c,M,K) ( A , b , c , M , K ) x ˉ = 0 n \bar{x}=0_n x ˉ = 0 n x x x ( A , b , c ) (A,b,c) ( A , b , c )
(iv) If c > 0 n c > 0_n c > 0 n
K > M ∥ d ∥ 1 min i ∈ [ n ] c i , (14) K > \frac{M\lVert d\rVert_{1}}{\min_{i\in[n]}c_i}, \tag*{(14)} K > min i ∈ [ n ] c i M ∥ d ∥ 1 , (14) holds, then every dual optimal solution ( y , s , s ˉ , s ^ ) (y,s,\bar{s},\hat{s}) ( y , s , s ˉ , s ^ ) ( A , b , c , M , K ) (A,b,c,M,K) ( A , b , c , M , K ) s ^ = 0 n \hat{s}=0_n s ^ = 0 n ( y , s ) (y,s) ( y , s ) ( A , b , c ) (A,b,c) ( A , b , c )
Proof. Part (i). Feasibility is immediate by the choice of K K K M M M
∥ ( x , x ˉ , x ^ ) ∘ ( s , s ˉ , s ^ ) K M − 1 3 n ∥ 2 = ∥ ( c M , − d K , 0 n ) ∥ 2 ≤ ∥ c M ∥ 2 + ∥ d K ∥ 2 ≤ β 4 + β 4 = β 2 . \left\lVert\frac{(x,\bar{x},\hat{x})\circ(s,\bar{s},\hat{s})}{KM}-1_{3n}\right\rVert_2
=
\left\lVert\left(\frac{c}{M},-\frac{d}{K},0_n\right)\right\rVert_2
\leq\left\lVert\frac{c}{M}\right\rVert_2+\left\lVert\frac{d}{K}\right\rVert_2
\leq\frac{\beta}{4}+\frac{\beta}{4}=\frac{\beta}{2}. K M ( x , x ˉ , x ^ ) ∘ ( s , s ˉ , s ^ ) − 1 3 n 2 = ( M c , − K d , 0 n ) 2 ≤ M c 2 + K d 2 ≤ 4 β + 4 β = 2 β . Then, ( x , x ˉ , x ^ ) (x,\bar{x},\hat{x}) ( x , x ˉ , x ^ ) ( s , s ˉ , s ^ ) (s,\bar{s},\hat{s}) ( s , s ˉ , s ^ ) N 2 ( β ) \mathcal{N}^{2}(\beta) N 2 ( β ) ( β / 2 ) / ( 1 − β / 2 ) < β (\beta/2)/(1-\beta/2)<\beta ( β /2 ) / ( 1 − β /2 ) < β β ∈ ( 0 , 1 / 2 ] \beta\in(0,1/2] β ∈ ( 0 , 1/2 ]
Part (ii). Since c , d > 0 n c,d>0_n c , d > 0 n ( d , 0 n , 2 K 1 n − d ) (d,0_n,2K1_n-d) ( d , 0 n , 2 K 1 n − d ) ( 0 m , c , M 1 n , 0 n ) (0_m,c,M1_n,0_n) ( 0 m , c , M 1 n , 0 n ) γ : = ⟨ c , d ⟩ \gamma:=\langle c,d\rangle γ := ⟨ c , d ⟩ ( x , x ˉ , x ^ ) (x,\bar{x},\hat{x}) ( x , x ˉ , x ^ ) ( y , s , s ˉ , s ^ ) (y,s,\bar{s},\hat{s}) ( y , s , s ˉ , s ^ ) ( A , b , c , M , K ) (A,b,c,M,K) ( A , b , c , M , K )
⟨ x , s ⟩ = ⟨ x ˉ , s ˉ ⟩ = ⟨ x ^ , s ^ ⟩ = 0. (15) \langle x,s\rangle=\langle\bar{x},\bar{s}\rangle=\langle\hat{x},\hat{s}\rangle=0. \tag*{(15)} ⟨ x , s ⟩ = ⟨ x ˉ , s ˉ ⟩ = ⟨ x ^ , s ^ ⟩ = 0. (15) Let us start by showing that s ^ = 0 n \hat{s}=0_n s ^ = 0 n ( x , x ˉ , x ^ ) (x,\bar{x},\hat{x}) ( x , x ˉ , x ^ ) ( 0 m , c , M 1 n , 0 n ) (0_m,c,M1_n,0_n) ( 0 m , c , M 1 n , 0 n ) γ \gamma γ ⟨ x , c ⟩ ≤ γ \langle x,c\rangle\leq\gamma ⟨ x , c ⟩ ≤ γ K K K
0 ≤ x i ≤ γ c i < K , ∀ i ∈ [ n ] . 0\leq x_i\leq\frac{\gamma}{c_i}<K,\quad\forall i\in[n]. 0 ≤ x i ≤ c i γ < K , ∀ i ∈ [ n ] . Therefore, x ^ i = 2 K − x i > 0 \hat{x}_i=2K-x_i>0 x ^ i = 2 K − x i > 0 s ^ i = 0 \hat{s}_i=0 s ^ i = 0 i ∈ [ n ] i\in[n] i ∈ [ n ] x ˉ = 0 n \bar{x}=0_n x ˉ = 0 n ( d , 0 n , 2 K 1 n − d ) (d,0_n,2K1_n-d) ( d , 0 n , 2 K 1 n − d ) ≤ γ \le\gamma ≤ γ ⟨ d , s ⟩ ≤ γ \langle d,s\rangle\le\gamma ⟨ d , s ⟩ ≤ γ M M M
s i ≤ γ d i < M , ∀ i ∈ [ n ] . s_i \le\frac{\gamma}{d_i}<M,\qquad\forall i\in[n]. s i ≤ d i γ < M , ∀ i ∈ [ n ] . From the dual constraints, for every i ∈ [ n ] i\in[n] i ∈ [ n ]
s ˉ i = [ A T y ] i + M ≥ [ A T y ] i − s ^ i + M = c i − s i + M > c i ≥ 0. \bar{s}_i=[A^{\mathsf T}y]_i+M\ge[A^{\mathsf T}y]_i-\hat{s}_i+M=c_i-s_i+M>c_i\ge0. s ˉ i = [ A T y ] i + M ≥ [ A T y ] i − s ^ i + M = c i − s i + M > c i ≥ 0. Consequently, by complementary slackness, x ˉ i = 0 \bar{x}_i=0 x ˉ i = 0 i ∈ [ n ] i\in[n] i ∈ [ n ] x x x ( y , s ) (y,s) ( y , s ) LP ( A , b , c ) \operatorname{LP}(A,b,c) LP ( A , b , c ) ⟨ x , s ⟩ = 0 \langle x,s\rangle=0 ⟨ x , s ⟩ = 0 (15) . This shows that they are primal and dual optima.
Part (iii). The proof follows similarly to the primal part of the proof above, with the ‘reference’ primal and dual feasible solutions ( d , 0 n , 2 K 1 n − d ) (d,0_n,2K1_n-d) ( d , 0 n , 2 K 1 n − d ) ( 0 m , c + , M 1 n , c − ) (0_m,c^+,M1_n,c^-) ( 0 m , c + , M 1 n , c − ) MSys ( A , b , c , M , K ) \operatorname{MSys}(A,b,c,M,K) MSys ( A , b , c , M , K )
γ : = ⟨ d , c + ⟩ + ⟨ 2 K 1 n − d , c − ⟩ ≤ 2 K ∥ c ∥ 1 . \gamma:=\langle d,c^+\rangle+\langle2K1_n-d,c^-\rangle\le2K\lVert c\rVert_1. γ := ⟨ d , c + ⟩ + ⟨ 2 K 1 n − d , c − ⟩ ≤ 2 K ∥ c ∥ 1 . Then, the gap between ( d , 0 n , 2 K 1 n − d ) (d,0_n,2K1_n-d) ( d , 0 n , 2 K 1 n − d ) ( y , s , s ˉ , s ^ ) (y,s,\bar{s},\hat{s}) ( y , s , s ˉ , s ^ ) ≤ γ \le\gamma ≤ γ M M M
0 ≤ s i ≤ γ d i < M 2 ∀ i ∈ [ n ] . 0\le s_i\le\frac{\gamma}{d_i}<\frac{M}{2}\qquad\forall i\in[n]. 0 ≤ s i ≤ d i γ < 2 M ∀ i ∈ [ n ] . We can now write for every i ∈ [ n ] i\in[n] i ∈ [ n ]
s ˉ i = [ A T y ] i + M ≥ [ A T y ] i − s ^ i + M = c i − s i + M > 0 , \bar{s}_i=[A^{\mathsf T}y]_i+M\ge[A^{\mathsf T}y]_i-\hat{s}_i+M=c_i-s_i+M>0, s ˉ i = [ A T y ] i + M ≥ [ A T y ] i − s ^ i + M = c i − s i + M > 0 , using that s i < M / 2 s_i<M/2 s i < M /2 ∥ c ∥ ∞ ≤ ∥ c ∥ 2 < M / 4 \lVert c\rVert_\infty\le\lVert c\rVert_2<M/4 ∥ c ∥ ∞ ≤ ∥ c ∥ 2 < M /4 ( x , x ˉ , x ^ ) (x,\bar{x},\hat{x}) ( x , x ˉ , x ^ ) x ˉ i = 0 \bar{x}_i=0 x ˉ i = 0 i ∈ [ n ] i\in[n] i ∈ [ n ]
Part (iv). We now use the ‘reference’ solutions ( d + , d − , 2 K 1 n − d + ) (d^+,d^-,2K1_n-d^+) ( d + , d − , 2 K 1 n − d + ) ( 0 m , c , M 1 n , 0 n ) (0_m,c,M1_n,0_n) ( 0 m , c , M 1 n , 0 n )
γ : = ⟨ d + , c ⟩ + ⟨ d − , M 1 n ⟩ ≤ M ∥ d ∥ 1 . \gamma:=\langle d^+,c\rangle+\langle d^-,M1_n\rangle\le M\lVert d\rVert_1. γ := ⟨ d + , c ⟩ + ⟨ d − , M 1 n ⟩ ≤ M ∥ d ∥ 1 . The gap between a primal optimal solution ( x , x ˉ , x ^ ) (x,\bar{x},\hat{x}) ( x , x ˉ , x ^ ) ( 0 m , c , M 1 n , 0 n ) (0_m,c,M1_n,0_n) ( 0 m , c , M 1 n , 0 n ) ≤ γ \le\gamma ≤ γ
⟨ x , c ⟩ ≤ γ ≤ M ∥ d ∥ 1 , \langle x,c\rangle\le\gamma\le M\lVert d\rVert_1, ⟨ x , c ⟩ ≤ γ ≤ M ∥ d ∥ 1 , and by the choice of K K K x i ≤ c i − 1 ⋅ M ∥ d ∥ 1 < K x_i\le c_i^{-1}\cdot M\lVert d\rVert_1<K x i ≤ c i − 1 ⋅ M ∥ d ∥ 1 < K i ∈ [ n ] i\in[n] i ∈ [ n ] x ^ i = 2 K − x i > 0 \hat{x}_i=2K-x_i>0 x ^ i = 2 K − x i > 0 s ^ i = 0 \hat{s}_i=0 s ^ i = 0 i ∈ [ n ] i\in[n] i ∈ [ n ]
The algorithm
The algorithm proceeds in three stages, each refining a partition of [ n ] [n] [ n ] LP ( A , b , c ) \operatorname{LP}(A,b,c) LP ( A , b , c ) ( x ⋆ , y ⋆ , s ⋆ ) (x^\star,y^\star,s^\star) ( x ⋆ , y ⋆ , s ⋆ ) ( x , y , s ) (x,y,s) ( x , y , s ) § 3.1 , the two auxiliary variables of every big-M M M x ˉ \bar{x} x ˉ M M M A ( x − x ˉ ) = b A(x-\bar{x})=b A ( x − x ˉ ) = b x ^ \hat{x} x ^ x ≤ 2 K 1 n x\le2K1_n x ≤ 2 K 1 n s ˉ , s ^ \bar{s},\hat{s} s ˉ , s ^ ( y ~ , s ~ ) (\tilde{y},\tilde{s}) ( y ~ , s ~ ) x ~ \tilde{x} x ~ P c , D c P^{\mathrm c},D^{\mathrm c} P c , D c P p f , D d f P^{\mathrm{pf}},D^{\mathrm{df}} P pf , D df R , I , J R,I,J R , I , J
We frequently shift a vector u ∈ R n u\in\mathbb{R}^n u ∈ R n w ∈ R ≥ 0 n w\in\mathbb{R}_{\ge0}^n w ∈ R ≥ 0 n u i ≥ 0 u_i\ge0 u i ≥ 0 i ∉ supp ( w ) i\notin\operatorname{supp}(w) i ∈ / supp ( w ) α ≥ 0 \alpha\ge0 α ≥ 0 u + α w ≥ 0 n u+\alpha w\ge0_n u + α w ≥ 0 n
α 0 : = max ( { 0 } ∪ { − u i / w i ∣ i ∈ supp ( w ) } ) . \alpha_0:=\max\bigl(\{0\}\cup\{-u_i/w_i\mid i\in\operatorname{supp}(w)\}\bigr). α 0 := max ( { 0 } ∪ { − u i / w i ∣ i ∈ supp ( w )} ) . Whenever we choose α ≥ 0 \alpha\ge0 α ≥ 0 u + α w ≥ 0 n u+\alpha w\ge0_n u + α w ≥ 0 n α : = α 0 \alpha:=\alpha_0 α := α 0 u i + α w i > 0 u_i+\alpha w_i>0 u i + α w i > 0 i ∈ supp ( w ) i\in\operatorname{supp}(w) i ∈ supp ( w ) α : = α 0 + 1 \alpha:=\alpha_0+1 α := α 0 + 1 u i + α w i = ( u i + α 0 w i ) + w i ≥ w i > 0 u_i+\alpha w_i=(u_i+\alpha_0w_i)+w_i\ge w_i>0 u i + α w i = ( u i + α 0 w i ) + w i ≥ w i > 0 i ∈ supp ( w ) i\in\operatorname{supp}(w) i ∈ supp ( w ) α \alpha α O ( size ( u , w ) ) O(\operatorname{size}(u,w)) O ( size ( u , w ))
We assume that the system of linear equations A x = b Ax=b A x = b d ∈ R n d\in\mathbb{R}^n d ∈ R n A d = b Ad=b A d = b Theorem 1.6 , we only use the maximum support primal and dual solutions x o u t x^{\mathrm{out}} x out s o u t s^{\mathrm{out}} s out β ∈ ( 0 , 1 / 6 ] \beta\in(0,1/6] β ∈ ( 0 , 1/6 ]
Stage I: Support partition
Corollary 2.5(i) postulates the existence of a partition P c ∪ D c = [ n ] P^{\mathrm{c}} \cup D^{\mathrm{c}} = [n] P c ∪ D c = [ n ] x c , s c ∈ R n x^{\mathrm{c}},s^{\mathrm{c}} \in\mathbb{R}^{n} x c , s c ∈ R n y c ∈ R m y^{\mathrm{c}} \in\mathbb{R}^{m} y c ∈ R m A x c = 0 m Ax^{\mathrm{c}} = 0_m A x c = 0 m x c ≥ 0 x^{\mathrm{c}} \ge0 x c ≥ 0 s c = − A T y c s^{\mathrm{c}} = -A^{\mathsf{T}}y^{\mathrm{c}} s c = − A T y c s c ≥ 0 s^{\mathrm{c}} \ge0 s c ≥ 0 supp ( x c ) = P c \operatorname{supp}(x^{\mathrm{c}}) = P^{\mathrm{c}} supp ( x c ) = P c supp ( s c ) = D c \operatorname{supp}(s^{\mathrm{c}}) = D^{\mathrm{c}} supp ( s c ) = D c ( x c , s c ) (x^{\mathrm{c}},s^{\mathrm{c}}) ( x c , s c )
We can obtain these by running the IPM in Theorem 1.6 to solve MSys( A , 0 m , 0 n , 1 , 1 ) (A,0_m,0_n,1,1) ( A , 0 m , 0 n , 1 , 1 )
min ⟨ 1 n , x ˉ ⟩ max − 2 ⟨ 1 n , s ^ ⟩ A x − A x ˉ = 0 m A T y + s − s ^ = 0 n x + x ^ = 2 ⋅ 1 n − A T y + s ˉ = 1 n x , x ˉ , x ^ ≥ 0 n , s , s ˉ , s ^ ≥ 0 n . (16) \begin{aligned}
\min\ &\langle1_n,\bar{x}\rangle
&&\max\ -2\langle1_n,\hat{s}\rangle\\
Ax-A\bar{x}&=0_m
&&A^{\mathsf{T}}y+s-\hat{s}=0_n\\
x+\hat{x}&=2\cdot1_n
&&-A^{\mathsf{T}}y+\bar{s}=1_n\\
x,\bar{x},\hat{x}&\ge0_n,
&&s,\bar{s},\hat{s}\ge0_n. \tag*{(16)}
\end{aligned} min A x − A x ˉ x + x ^ x , x ˉ , x ^ ⟨ 1 n , x ˉ ⟩ = 0 m = 2 ⋅ 1 n ≥ 0 n , max − 2 ⟨ 1 n , s ^ ⟩ A T y + s − s ^ = 0 n − A T y + s ˉ = 1 n s , s ˉ , s ^ ≥ 0 n . (16) Clearly, we can initialize this with the solutions ( 1 n , 1 n , 1 n ) (1_n,1_n,1_n) ( 1 n , 1 n , 1 n ) ( 0 m , 1 n , 1 n , 1 n ) (0_m,1_n,1_n,1_n) ( 0 m , 1 n , 1 n , 1 n ) ( x c , x ˉ c , x ^ c ) (x^{\mathrm{c}},\bar{x}^{\mathrm{c}},\hat{x}^{\mathrm{c}}) ( x c , x ˉ c , x ^ c ) ( y c , s c , s ˉ c , s ^ c ) (y^{\mathrm{c}},s^{\mathrm{c}},\bar{s}^{\mathrm{c}},\hat{s}^{\mathrm{c}}) ( y c , s c , s ˉ c , s ^ c ) 0 0 0 x ˉ c = s ^ c = 0 n \bar{x}^{\mathrm{c}}=\hat{s}^{\mathrm{c}}=0_n x ˉ c = s ^ c = 0 n A x c = 0 m Ax^{\mathrm{c}}=0_m A x c = 0 m x c ≥ 0 n x^{\mathrm{c}}\ge0_n x c ≥ 0 n s c = − A T y c s^{\mathrm{c}}=-A^{\mathsf{T}}y^{\mathrm{c}} s c = − A T y c s c ≥ 0 n s^{\mathrm{c}}\ge0_n s c ≥ 0 n P c = supp ( x c ) P^{\mathrm{c}}=\operatorname{supp}(x^{\mathrm{c}}) P c = supp ( x c ) D c = supp ( s c ) D^{\mathrm{c}}=\operatorname{supp}(s^{\mathrm{c}}) D c = supp ( s c ) [ n ] [n] [ n ] Theorem 1.6 .
Stage II: Maximum-support feasibility
In the second stage, we define two subroutines to obtain the solutions asserted in Corollary 2.5(ii) and (iii). The inputs and outputs of these subroutines are specified as follows.
Subroutine 3.2 (PrimalMaxS). PrimalMaxS( A , b ) (A,b) ( A , b ) A ∈ R m × n A\in\mathbb{R}^{m\times n} A ∈ R m × n b ∈ R m b\in\mathbb{R}^{m} b ∈ R m
A feasible solution x p f ∈ R n x^{\mathrm{pf}}\in\mathbb{R}^{n} x pf ∈ R n A x = b Ax=b A x = b x ≥ 0 n x\ge0_n x ≥ 0 n P p f : = supp ( x p f ) P^{\mathrm{pf}}:=\operatorname{supp}(x^{\mathrm{pf}}) P pf := supp ( x pf ) ( y ~ , s ~ ) ∈ R m × R n (\tilde{y},\tilde{s})\in\mathbb{R}^{m}\times\mathbb{R}^{n} ( y ~ , s ~ ) ∈ R m × R n s ~ = − A T y ~ \tilde{s}=-A^{\mathsf{T}}\tilde{y} s ~ = − A T y ~ s ~ ≥ 0 n \tilde{s}\ge0_n s ~ ≥ 0 n supp ( s ~ ) = [ n ] ∖ P p f \operatorname{supp}(\tilde{s})=[n]\setminus P^{\mathrm{pf}} supp ( s ~ ) = [ n ] ∖ P pf
an infeasibility certificate ( y ~ , s ~ ) ∈ R m × R n (\tilde{y},\tilde{s})\in\mathbb{R}^{m}\times\mathbb{R}^{n} ( y ~ , s ~ ) ∈ R m × R n s ~ = − A T y ~ \tilde{s}=-A^{\mathsf{T}}\tilde{y} s ~ = − A T y ~ s ~ ≥ 0 n \tilde{s}\ge0_n s ~ ≥ 0 n ⟨ b , y ~ ⟩ > 0 \langle b,\tilde{y}\rangle>0 ⟨ b , y ~ ⟩ > 0
Subroutine 3.3 (DualMaxS). DualMaxS( A , c ) (A,c) ( A , c ) A ∈ R m × n A\in\mathbb{R}^{m\times n} A ∈ R m × n c ∈ R n c\in\mathbb{R}^{n} c ∈ R n
A feasible solution ( y d f , s d f ) ∈ R m × R n (y^{\mathrm{df}},s^{\mathrm{df}})\in\mathbb{R}^{m}\times\mathbb{R}^{n} ( y df , s df ) ∈ R m × R n A T y + s = c A^{\mathsf{T}}y+s=c A T y + s = c s ≥ 0 n s\ge0_n s ≥ 0 n D d f : = supp ( s d f ) D^{\mathrm{df}}:=\operatorname{supp}(s^{\mathrm{df}}) D df := supp ( s df ) x ~ ∈ R n \tilde{x}\in\mathbb{R}^{n} x ~ ∈ R n A x ~ = 0 m A\tilde{x}=0_m A x ~ = 0 m x ~ ≥ 0 n \tilde{x}\ge0_n x ~ ≥ 0 n supp ( x ~ ) = [ n ] ∖ D d f \operatorname{supp}(\tilde{x})=[n]\setminus D^{\mathrm{df}} supp ( x ~ ) = [ n ] ∖ D df
an infeasibility certificate x ~ ∈ R n \tilde{x}\in\mathbb{R}^{n} x ~ ∈ R n A x ~ = 0 m A\tilde{x}=0_m A x ~ = 0 m x ~ ≥ 0 n \tilde{x}\ge0_n x ~ ≥ 0 n ⟨ c , x ~ ⟩ < 0 \langle c,\tilde{x}\rangle<0 ⟨ c , x ~ ⟩ < 0
We note that if the primal (dual) system is feasible, the output of PrimalMaxS (DualMaxS) satisfies the following additional property.
Lemma 3.4. If A x = b Ax=b A x = b x ≥ 0 n x\ge0_n x ≥ 0 n ( y ~ , s ~ ) (\tilde{y},\tilde{s}) ( y ~ , s ~ ) PrimalMaxS( A , b ) (A,b) ( A , b ) ⟨ b , y ~ ⟩ = 0 \langle b,\tilde{y}\rangle=0 ⟨ b , y ~ ⟩ = 0 A T y + s = c A^{\mathsf{T}}y+s=c A T y + s = c s ≥ 0 n s\ge0_n s ≥ 0 n x ~ \tilde{x} x ~ DualMaxS( A , c ) (A,c) ( A , c ) ⟨ c , x ~ ⟩ = 0 \langle c,\tilde{x}\rangle=0 ⟨ c , x ~ ⟩ = 0
Proof. In case (i), let x p f x^{\mathrm{pf}} x pf P p f P^{\mathrm{pf}} P pf supp ( s ~ ) = [ n ] ∖ P p f \operatorname{supp}(\tilde{s})=[n]\setminus P^{\mathrm{pf}} supp ( s ~ ) = [ n ] ∖ P pf ⟨ b , y ~ ⟩ = ⟨ A x p f , y ~ ⟩ = ⟨ x p f , A T y ~ ⟩ = − ⟨ x p f , s ~ ⟩ = 0 \langle b,\tilde{y}\rangle=\langle Ax^{\mathrm{pf}},\tilde{y}\rangle=\langle x^{\mathrm{pf}},A^{\mathsf{T}}\tilde{y}\rangle=-\langle x^{\mathrm{pf}},\tilde{s}\rangle=0 ⟨ b , y ~ ⟩ = ⟨ A x pf , y ~ ⟩ = ⟨ x pf , A T y ~ ⟩ = − ⟨ x pf , s ~ ⟩ = 0 x p f x^{\mathrm{pf}} x pf s ~ \tilde{s} s ~
Dual feasibility. We start by describing DualMaxS( A , c ) (A,c) ( A , c ) x c x^{\mathrm{c}} x c ( y c , s c ) (y^{\mathrm{c}},s^{\mathrm{c}}) ( y c , s c ) ( P c , D c ) (P^{\mathrm{c}},D^{\mathrm{c}}) ( P c , D c ) DualMaxS( A P c , c P c ) (A_{P^{\mathrm{c}}},c_{P^{\mathrm{c}}}) ( A P c , c P c ) y ∈ R m y\in\mathbb{R}^{m} y ∈ R m s : = c − A T y s:=c-A^{\mathsf{T}}y s := c − A T y s P c ≥ 0 P c s_{P^{\mathrm{c}}}\ge0_{P^{\mathrm{c}}} s P c ≥ 0 P c ( y + α y c , s + α s c ) (y+\alpha y^{\mathrm{c}},s+\alpha s^{\mathrm{c}}) ( y + α y c , s + α s c ) s + α s c ≥ 0 n s+\alpha s^{\mathrm{c}}\ge0_n s + α s c ≥ 0 n supp ( s + α s c ) = supp ( s P c ) ∪ D c \operatorname{supp}(s+\alpha s^{\mathrm{c}})=\operatorname{supp}(s_{P^{\mathrm{c}}})\cup D^{\mathrm{c}} supp ( s + α s c ) = supp ( s P c ) ∪ D c α \alpha α P c ≠ ∅ P^{\mathrm{c}}\ne\varnothing P c = ∅
We solve DualMaxS( A P c , c P c ) (A_{P^{\mathrm{c}}},c_{P^{\mathrm{c}}}) ( A P c , c P c ) MSys( A P c , 0 m , c P c , M , K ) (A_{P^{\mathrm{c}}},0_m,c_{P^{\mathrm{c}}},M,K) ( A P c , 0 m , c P c , M , K ) M M M K K K d d d x P c c > 0 P c x_{P^{\mathrm{c}}}^{\mathrm{c}}>0_{P^{\mathrm{c}}} x P c c > 0 P c
min ⟨ c P c , x P c ⟩ + M ⟨ 1 P c , x ˉ P c ⟩ max − 2 K ⟨ 1 P c , s ^ P c ⟩ A P c x P c − A P c x ˉ P c = 0 m A P c T y + s P c − s ^ P c = c P c x P c + x ^ P c = 2 K ⋅ 1 P c − A P c T y + s ˉ P c = M 1 P c x P c , x ˉ P c , x ^ P c ≥ 0 P c , s P c , s ˉ P c , s ^ P c ≥ 0 P c . (17) \begin{aligned}
\min\ &\langle c_{P^{\mathrm{c}}},x_{P^{\mathrm{c}}}\rangle+M\langle1_{P^{\mathrm{c}}},\bar{x}_{P^{\mathrm{c}}}\rangle
&&\max\ -2K\langle1_{P^{\mathrm{c}}},\hat{s}_{P^{\mathrm{c}}}\rangle\\
A_{P^{\mathrm{c}}}x_{P^{\mathrm{c}}}-A_{P^{\mathrm{c}}}\bar{x}_{P^{\mathrm{c}}}&=0_m
&&A_{P^{\mathrm{c}}}^{\mathsf{T}}y+s_{P^{\mathrm{c}}}-\hat{s}_{P^{\mathrm{c}}}=c_{P^{\mathrm{c}}}\\
x_{P^{\mathrm{c}}}+\hat{x}_{P^{\mathrm{c}}}&=2K\cdot1_{P^{\mathrm{c}}}
&&-A_{P^{\mathrm{c}}}^{\mathsf{T}}y+\bar{s}_{P^{\mathrm{c}}}=M1_{P^{\mathrm{c}}}\\
x_{P^{\mathrm{c}}},\bar{x}_{P^{\mathrm{c}}},\hat{x}_{P^{\mathrm{c}}}&\ge0_{P^{\mathrm{c}}},
&&s_{P^{\mathrm{c}}},\bar{s}_{P^{\mathrm{c}}},\hat{s}_{P^{\mathrm{c}}}\ge0_{P^{\mathrm{c}}}. \tag*{(17)}
\end{aligned} min A P c x P c − A P c x ˉ P c x P c + x ^ P c x P c , x ˉ P c , x ^ P c ⟨ c P c , x P c ⟩ + M ⟨ 1 P c , x ˉ P c ⟩ = 0 m = 2 K ⋅ 1 P c ≥ 0 P c , max − 2 K ⟨ 1 P c , s ^ P c ⟩ A P c T y + s P c − s ^ P c = c P c − A P c T y + s ˉ P c = M 1 P c s P c , s ˉ P c , s ^ P c ≥ 0 P c . (17) This is the instantiation of MSys ( A , b , c , M , K ) \operatorname{MSys}(A,b,c,M,K) MSys ( A , b , c , M , K ) b = 0 m b=0_m b = 0 m c P c c_{P^c} c P c P c P^c P c A P c A_{P^c} A P c A A A Theorem 1.6 . This is because any dual optimal solution of the resulting system can be trivially lifted to a dual optimal solution of the original system by filling y y y
Let ( x P c d f , x ˉ P c , x ^ P c d f ) (x_{P^c}^{\mathrm{df}},\bar{x}_{P^c},\hat{x}_{P^c}^{\mathrm{df}}) ( x P c df , x ˉ P c , x ^ P c df ) ( y ′ , s P c d f , s ˉ P c d f , s ^ P c d f ) (y^{\prime},s_{P^c}^{\mathrm{df}},\bar{s}_{P^c}^{\mathrm{df}},\hat{s}_{P^c}^{\mathrm{df}}) ( y ′ , s P c df , s ˉ P c df , s ^ P c df ) (17) obtained via Theorem 1.6 . According to Lemma 3.1 (iii), we have x ˉ P c = 0 P c \bar{x}_{P^c}=0_{P^c} x ˉ P c = 0 P c ≤ 0 \le0 ≤ 0 ⟨ c P c , x P c d f ⟩ < 0 \langle c_{P^c},x_{P^c}^{\mathrm{df}}\rangle<0 ⟨ c P c , x P c df ⟩ < 0 LP ( A P c , 0 m , c P c ) \operatorname{LP}(A_{P^c},0_m,c_{P^c}) LP ( A P c , 0 m , c P c ) x ~ = ( x P c d f , 0 D c ) \tilde{x}=(x_{P^c}^{\mathrm{df}},0_{D^c}) x ~ = ( x P c df , 0 D c ) LP ( A , b , c ) \operatorname{LP}(A,b,c) LP ( A , b , c ) ⟨ c P c , x P c d f ⟩ = 0 \langle c_{P^c},x_{P^c}^{\mathrm{df}}\rangle=0 ⟨ c P c , x P c df ⟩ = 0 0 0 0 s ^ P c d f = 0 P c \hat{s}_{P^c}^{\mathrm{df}}=0_{P^c} s ^ P c df = 0 P c A P c T y ′ + s P c d f = c P c A_{P^c}^{\mathsf T}y^{\prime}+s_{P^c}^{\mathrm{df}}=c_{P^c} A P c T y ′ + s P c df = c P c s P c d f ≥ 0 P c s_{P^c}^{\mathrm{df}}\geq0_{P^c} s P c df ≥ 0 P c ( y ′ , s P c d f ) (y^{\prime},s_{P^c}^{\mathrm{df}}) ( y ′ , s P c df ) Theorem 1.6 . As noted above, y d f = y ′ + α y c y^{\mathrm{df}}=y^{\prime}+\alpha y^c y df = y ′ + α y c s d f = c − A T y d f s^{\mathrm{df}}=c-A^{\mathsf T}y^{\mathrm{df}} s df = c − A T y df α \alpha α ( y d f , s d f ) (y^{\mathrm{df}},s^{\mathrm{df}}) ( y df , s df ) DualMaxS ( A , c ) \operatorname{DualMaxS}(A,c) DualMaxS ( A , c ) x ~ = ( x P c d f , 0 D c ) \tilde{x}=(x_{P^c}^{\mathrm{df}},0_{D^c}) x ~ = ( x P c df , 0 D c )
Primal feasibility. We now turn to the subroutine PrimalMaxS ( A , b ) \operatorname{PrimalMaxS}(A,b) PrimalMaxS ( A , b ) D c ≠ ∅ D^c\ne\varnothing D c = ∅ d + α x c d+\alpha x^c d + α x c A ( d + α x c ) = b A(d+\alpha x^c)=b A ( d + α x c ) = b d + α x c > 0 n d+\alpha x^c>0_n d + α x c > 0 n α ≥ 0 \alpha\geq0 α ≥ 0 Π \Pi Π ker ( A P c T ) \ker(A_{P^c}^{\mathsf T}) ker ( A P c T ) (6) , that is,
Π : = I m − A P c A P c + , H : = Π A D c , h : = Π b , \Pi:=I_m-A_{P^c}A_{P^c}^{+},\qquad H:=\Pi A_{D^c},\qquad h:=\Pi b, Π := I m − A P c A P c + , H := Π A D c , h := Π b , with the convention that Π : = I m \Pi:=I_m Π := I m P c = ∅ P^c=\varnothing P c = ∅ A x = b Ax=b A x = b H x D c = h Hx_{D^c}=h H x D c = h
From Stage I, we have the conic dual solution ( y c , s c ) (y^c,s^c) ( y c , s c ) s c = − A T y c s^c=-A^{\mathsf T}y^c s c = − A T y c s c ≥ 0 s^c\geq0 s c ≥ 0 supp ( s c ) = D c \operatorname{supp}(s^c)=D^c supp ( s c ) = D c MSys ( H , h , s D c c , M , K ) \operatorname{MSys}(H,h,s_{D^c}^c,M,K) MSys ( H , h , s D c c , M , K ) s D c c > 0 D c s_{D^c}^c>0_{D^c} s D c c > 0 D c M M M K K K Lemma 3.1 :
min ⟨ s D c c , x D c ⟩ + M ⟨ 1 D c , x ˉ D c ⟩ max ⟨ h , y ⟩ − 2 K ⟨ 1 D c , s ^ D c ⟩ s.t. H x D c − H x ˉ D c = h s.t. H T y + s D c − s ^ D c = s D c c x D c + x ^ D c = 2 K 1 D c − H T y + s ˉ D c = M 1 D c x D c , x ˉ D c , x ^ D c ≥ 0 D c s D c , s ˉ D c , s ^ D c ≥ 0 D c . (18) \begin{aligned}
\min\quad&\langle s_{D^c}^c,x_{D^c}\rangle+M\langle1_{D^c},\bar{x}_{D^c}\rangle
&\max\quad&\langle h,y\rangle-2K\langle1_{D^c},\hat{s}_{D^c}\rangle\\
\text{s.t.}\quad&Hx_{D^c}-H\bar{x}_{D^c}=h
&\text{s.t.}\quad&H^{\mathsf T}y+s_{D^c}-\hat{s}_{D^c}=s_{D^c}^c\\
&x_{D^c}+\hat{x}_{D^c}=2K1_{D^c}
&&-H^{\mathsf T}y+\bar{s}_{D^c}=M1_{D^c}\\
&x_{D^c},\bar{x}_{D^c},\hat{x}_{D^c}\geq0_{D^c}
&&s_{D^c},\bar{s}_{D^c},\hat{s}_{D^c}\geq0_{D^c}. \tag*{(18)}
\end{aligned} min s.t. ⟨ s D c c , x D c ⟩ + M ⟨ 1 D c , x ˉ D c ⟩ H x D c − H x ˉ D c = h x D c + x ^ D c = 2 K 1 D c x D c , x ˉ D c , x ^ D c ≥ 0 D c max s.t. ⟨ h , y ⟩ − 2 K ⟨ 1 D c , s ^ D c ⟩ H T y + s D c − s ^ D c = s D c c − H T y + s ˉ D c = M 1 D c s D c , s ˉ D c , s ^ D c ≥ 0 D c . (18) This is the instantiation of MSys ( A , b , c , M , K ) \operatorname{MSys}(A,b,c,M,K) MSys ( A , b , c , M , K ) s D c c s_{D^c}^c s D c c ( H , h ) (H,h) ( H , h ) H H H H H H Theorem 1.6 .
The next lemma shows how we can construct the output for PrimalMaxS ( A , b ) \operatorname{PrimalMaxS}(A,b) PrimalMaxS ( A , b ) M M M D c D^c D c x D c p f x_{D^c}^{\mathrm{pf}} x D c pf x p f x^{\mathrm{pf}} x pf P c P^c P c
Lemma 3.5. Let ( x D c p f , x ˉ D c p f , x ^ D c p f ) (x_{D^c}^{\mathrm{pf}},\bar{x}_{D^c}^{\mathrm{pf}},\hat{x}_{D^c}^{\mathrm{pf}}) ( x D c pf , x ˉ D c pf , x ^ D c pf ) ( y p f , s D c p f , s ˉ D c p f , s ^ D c p f ) (y^{\mathrm{pf}},s_{D^c}^{\mathrm{pf}},\bar{s}_{D^c}^{\mathrm{pf}},\hat{s}_{D^c}^{\mathrm{pf}}) ( y pf , s D c pf , s ˉ D c pf , s ^ D c pf ) Theorem 1.6 . Let ( x c , s c ) (x^c,s^c) ( x c , s c ) A x c = 0 m Ax^c=0_m A x c = 0 m x c ≥ 0 x^c\geq0 x c ≥ 0 s c = − A T y c s^c=-A^{\mathsf T}y^c s c = − A T y c s c ≥ 0 s^c\geq0 s c ≥ 0 supp ( x c ) = P c \operatorname{supp}(x^c)=P^c supp ( x c ) = P c supp ( s c ) = D c \operatorname{supp}(s^c)=D^c supp ( s c ) = D c
z : = A P c + ( b − A D c x D c p f ) . z:=A_{P^c}^{+}(b-A_{D^c}x_{D^c}^{\mathrm{pf}}). z := A P c + ( b − A D c x D c pf ) . Let us pick α ≥ 0 \alpha\geq0 α ≥ 0 z + α x P c c > 0 z+\alpha x_{P^c}^c>0 z + α x P c c > 0
x p f = ( x D c p f , x P c p f ) : = ( x D c p f , z ) + α x c , x^{\mathrm{pf}}=(x_{D^c}^{\mathrm{pf}},x_{P^c}^{\mathrm{pf}}):=(x_{D^c}^{\mathrm{pf}},z)+\alpha x^c, x pf = ( x D c pf , x P c pf ) := ( x D c pf , z ) + α x c , y ~ : = Π y p f + y c , s ~ : = − A T y ~ . \tilde{y}:=\Pi y^{\mathrm{pf}}+y^c,\qquad\tilde{s}:=-A^{\mathsf T}\tilde{y}. y ~ := Π y pf + y c , s ~ := − A T y ~ . Then, s ~ ≥ 0 n \tilde{s}\geq0_n s ~ ≥ 0 n s ~ P c = 0 P c \tilde{s}_{P^c}=0_{P^c} s ~ P c = 0 P c
⟨ b , y ~ ⟩ > 0 \langle b,\tilde{y}\rangle>0 ⟨ b , y ~ ⟩ > 0
⟨ b , y ~ ⟩ = 0 \langle b,\tilde{y}\rangle=0 ⟨ b , y ~ ⟩ = 0 A x p f = b Ax^{\mathrm{pf}}=b A x pf = b x p f ≥ 0 x^{\mathrm{pf}}\geq0 x pf ≥ 0 supp ( x p f ) ∪ supp ( s ~ ) = [ n ] \operatorname{supp}(x^{\mathrm{pf}})\cup\operatorname{supp}(\tilde{s})=[n] supp ( x pf ) ∪ supp ( s ~ ) = [ n ]
Proof. Since s D c c > 0 D c s^{c}_{D^{c}}>0_{D^{c}} s D c c > 0 D c Lemma 3.1 (iv) implies that s ^ D c p f = 0 D c \hat{s}^{\mathrm{pf}}_{D^{c}}=0_{D^{c}} s ^ D c pf = 0 D c H T y p f + s D c p f = s D c c H^{\mathsf{T}}y^{\mathrm{pf}}+s^{\mathrm{pf}}_{D^{c}}=s^{c}_{D^{c}} H T y pf + s D c pf = s D c c H = Π A D c H=\Pi A_{D^{c}} H = Π A D c s D c c = − A D c T y c s^{c}_{D^{c}}=-A_{D^{c}}^{\mathsf{T}}y^{c} s D c c = − A D c T y c A D c T Π y p f + s D c p f = − A D c T y c A_{D^{c}}^{\mathsf{T}}\Pi y^{\mathrm{pf}}+s^{\mathrm{pf}}_{D^{c}}=-A_{D^{c}}^{\mathsf{T}}y^{c} A D c T Π y pf + s D c pf = − A D c T y c
A D c T y ~ + s D c p f = 0 D c , A_{D^{c}}^{\mathsf{T}}\tilde{y}+s^{\mathrm{pf}}_{D^{c}}=0_{D^{c}}, A D c T y ~ + s D c pf = 0 D c , i.e., s ~ D c = s D c p f ≥ 0 D c \tilde{s}_{D^{c}}=s^{\mathrm{pf}}_{D^{c}}\geq0_{D^{c}} s ~ D c = s D c pf ≥ 0 D c Π \Pi Π ker ( A P c T ) \ker(A_{P^{c}}^{\mathsf{T}}) ker ( A P c T ) Lemma 2.1 ), and A P c T y c = 0 P c A_{P^{c}}^{\mathsf{T}}y^{c}=0_{P^{c}} A P c T y c = 0 P c A P c T y ~ = 0 P c A_{P^{c}}^{\mathsf{T}}\tilde{y}=0_{P^{c}} A P c T y ~ = 0 P c s ~ ≥ 0 n \tilde{s}\geq0_{n} s ~ ≥ 0 n s ~ P c = 0 P c \tilde{s}_{P^{c}}=0_{P^{c}} s ~ P c = 0 P c
Let us now consider the dual objective ⟨ h , y ⟩ − 2 K ⟨ 1 D c , s ^ D c ⟩ \langle h,y\rangle-2K\langle1_{D^{c}},\hat{s}_{D^{c}}\rangle ⟨ h , y ⟩ − 2 K ⟨ 1 D c , s ^ D c ⟩ supp ( s c ) = D c \operatorname{supp}(s^{c})=D^{c} supp ( s c ) = D c P c = [ n ] ∖ D c P^{c}=[n]\setminus D^{c} P c = [ n ] ∖ D c y c ∈ ker ( A P c ) y^{c}\in\ker(A_{P^{c}}) y c ∈ ker ( A P c ) Π y c = y c \Pi y^{c}=y^{c} Π y c = y c H T y c = A D c T Π y c = A D c T y c H^{\mathsf{T}}y^{c}=A_{D^{c}}^{\mathsf{T}}\Pi y^{c}=A_{D^{c}}^{\mathsf{T}}y^{c} H T y c = A D c T Π y c = A D c T y c ⟨ h , y c ⟩ = ⟨ Π b , y c ⟩ = ⟨ b , Π y c ⟩ = ⟨ b , y c ⟩ \langle h,y^{c}\rangle=\langle\Pi b,y^{c}\rangle=\langle b,\Pi y^{c}\rangle=\langle b,y^{c}\rangle ⟨ h , y c ⟩ = ⟨ Π b , y c ⟩ = ⟨ b , Π y c ⟩ = ⟨ b , y c ⟩ ( y , s , s ˉ , s ^ ) : = ( − y c , 0 D c , 1 D c − A D c T y c , 0 D c ) (y,s,\bar{s},\hat{s}):=(-y^{c},0_{D^{c}},1_{D^{c}}-A_{D^{c}}^{\mathsf{T}}y^{c},0_{D^{c}}) ( y , s , s ˉ , s ^ ) := ( − y c , 0 D c , 1 D c − A D c T y c , 0 D c ) − ⟨ h , y c ⟩ = − ⟨ b , y c ⟩ -\langle h,y^{c}\rangle=-\langle b,y^{c}\rangle − ⟨ h , y c ⟩ = − ⟨ b , y c ⟩ s ˉ D c = 1 D c − A D c T y c = 1 D c + s D c c ≥ 1 D c > 0 D c \bar{s}_{D^{c}}=1_{D^{c}}-A_{D^{c}}^{\mathsf{T}}y^{c}=1_{D^{c}}+s^{c}_{D^{c}}\geq1_{D^{c}}>0_{D^{c}} s ˉ D c = 1 D c − A D c T y c = 1 D c + s D c c ≥ 1 D c > 0 D c s ^ D c p f = 0 D c \hat{s}^{\mathrm{pf}}_{D^{c}}=0_{D^{c}} s ^ D c pf = 0 D c ⟨ h , y p f ⟩ ≥ − ⟨ b , y c ⟩ \langle h,y^{\mathrm{pf}}\rangle\geq-\langle b,y^{c}\rangle ⟨ h , y pf ⟩ ≥ − ⟨ b , y c ⟩ ⟨ b , y ~ ⟩ = ⟨ b , Π y p f + y c ⟩ = ⟨ h , y p f ⟩ + ⟨ b , y c ⟩ ≥ 0 \langle b,\tilde{y}\rangle=\langle b,\Pi y^{\mathrm{pf}}+y^{c}\rangle=\langle h,y^{\mathrm{pf}}\rangle+\langle b,y^{c}\rangle\geq0 ⟨ b , y ~ ⟩ = ⟨ b , Π y pf + y c ⟩ = ⟨ h , y pf ⟩ + ⟨ b , y c ⟩ ≥ 0
If ⟨ b , y ~ ⟩ > 0 \langle b,\tilde{y}\rangle>0 ⟨ b , y ~ ⟩ > 0 ( y ~ , s ~ ) (\tilde{y},\tilde{s}) ( y ~ , s ~ ) A x = b Ax=b A x = b x ≥ 0 n x\geq0_{n} x ≥ 0 n ( A , b ) (A,b) ( A , b )
The remaining case is when ⟨ b , y ~ ⟩ = 0 \langle b,\tilde{y}\rangle=0 ⟨ b , y ~ ⟩ = 0 ( − y c , 0 D c , 1 D c − A D c T y c , 0 D c ) (-y^{c},0_{D^{c}},1_{D^{c}}-A_{D^{c}}^{\mathsf{T}}y^{c},0_{D^{c}}) ( − y c , 0 D c , 1 D c − A D c T y c , 0 D c ) ( x D c p f , x ˉ D c p f , x ^ D c p f ) (x^{\mathrm{pf}}_{D^{c}},\bar{x}^{\mathrm{pf}}_{D^{c}},\hat{x}^{\mathrm{pf}}_{D^{c}}) ( x D c pf , x ˉ D c pf , x ^ D c pf ) x ˉ D c p f = 0 D c \bar{x}^{\mathrm{pf}}_{D^{c}}=0_{D^{c}} x ˉ D c pf = 0 D c A x p f = b Ax^{\mathrm{pf}}=b A x pf = b x p f ≥ 0 n x^{\mathrm{pf}}\geq0_{n} x pf ≥ 0 n A D c x D c p f + A P c z = b A_{D^{c}}x^{\mathrm{pf}}_{D^{c}}+A_{P^{c}}z=b A D c x D c pf + A P c z = b A x c = 0 m Ax^{c}=0_{m} A x c = 0 m A x p f = b Ax^{\mathrm{pf}}=b A x pf = b x D c p f ≥ 0 D c x^{\mathrm{pf}}_{D^{c}}\geq0_{D^{c}} x D c pf ≥ 0 D c α \alpha α supp ( x c ) = P c \operatorname{supp}(x^{c})=P^{c} supp ( x c ) = P c x P c p f = z + α x P c c > 0 P c x^{\mathrm{pf}}_{P^{c}}=z+\alpha x^{c}_{P^{c}}>0_{P^{c}} x P c pf = z + α x P c c > 0 P c
Finally, supp ( x p f ) ∪ supp ( s ~ ) = [ n ] \operatorname{supp}(x^{\mathrm{pf}})\cup\operatorname{supp}(\tilde{s})=[n] supp ( x pf ) ∪ supp ( s ~ ) = [ n ] supp ( x D c p f ) \operatorname{supp}(x^{\mathrm{pf}}_{D^{c}}) supp ( x D c pf ) supp ( s D c p f ) \operatorname{supp}(s^{\mathrm{pf}}_{D^{c}}) supp ( s D c pf ) D c D^{c} D c s ~ D c = s D c p f \tilde{s}_{D^{c}}=s^{\mathrm{pf}}_{D^{c}} s ~ D c = s D c pf supp ( x p f ) = P c ∪ supp ( x D c p f ) \operatorname{supp}(x^{\mathrm{pf}})=P^{c}\cup\operatorname{supp}(x^{\mathrm{pf}}_{D^{c}}) supp ( x pf ) = P c ∪ supp ( x D c pf )
Stage III: Optimal partition and solution
The overall algorithm first calls PrimalMaxS( A , b ) (A,b) ( A , b ) ( A , c ) (A,c) ( A , c ) x p f x^{\mathrm{pf}} x pf ( A , b ) (A,b) ( A , b ) P p f = supp ( x p f ) P^{\mathrm{pf}}=\operatorname{supp}(x^{\mathrm{pf}}) P pf = supp ( x pf ) ( y ~ , s ~ ) (\tilde{y},\tilde{s}) ( y ~ , s ~ ) ( y d f , s d f ) (y^{\mathrm{df}},s^{\mathrm{df}}) ( y df , s df ) ( A , c ) (A,c) ( A , c ) D d f = supp ( s d f ) D^{\mathrm{df}}=\operatorname{supp}(s^{\mathrm{df}}) D df = supp ( s df ) x ~ \tilde{x} x ~
R : = P p f ∩ D d f , I : = P p f ∖ D d f , J : = D d f ∖ P p f . R:=P^{\mathrm{pf}}\cap D^{\mathrm{df}},\qquad I:=P^{\mathrm{pf}}\setminus D^{\mathrm{df}},\qquad J:=D^{\mathrm{df}}\setminus P^{\mathrm{pf}}. R := P pf ∩ D df , I := P pf ∖ D df , J := D df ∖ P pf . Since P c ⊆ P p f P^{c}\subseteq P^{\mathrm{pf}} P c ⊆ P pf D c ⊆ D d f D^{c}\subseteq D^{\mathrm{df}} D c ⊆ D df [ n ] = P c ∪ D c ⊆ P p f ∪ D d f = R ∪ I ∪ J [n]=P^{c}\cup D^{c}\subseteq P^{\mathrm{pf}}\cup D^{\mathrm{df}}=R\cup I\cup J [ n ] = P c ∪ D c ⊆ P pf ∪ D df = R ∪ I ∪ J supp ( s ~ ) = [ n ] ∖ P p f = J \operatorname{supp}(\tilde{s})=[n]\setminus P^{\mathrm{pf}}=J supp ( s ~ ) = [ n ] ∖ P pf = J supp ( x ~ ) = [ n ] ∖ D d f = I \operatorname{supp}(\tilde{x})=[n]\setminus D^{\mathrm{df}}=I supp ( x ~ ) = [ n ] ∖ D df = I I I I J J J R R R R ≠ ∅ R\neq\varnothing R = ∅ x p f x^{\mathrm{pf}} x pf ( y d f , s d f ) (y^{\mathrm{df}},s^{\mathrm{df}}) ( y df , s df ) ( A , b , c ) (A,b,c) ( A , b , c )
We now remove all coordinates in J J J I I I
Π : = I m − A I A I T , H : = Π A R , h : = Π b , \Pi:=I_{m}-A_{I}A_{I}^{\mathsf{T}},\qquad H:=\Pi A_{R},\qquad h:=\Pi b, Π := I m − A I A I T , H := Π A R , h := Π b , with the convention that Π : = I m \Pi:=I_{m} Π := I m I = ∅ I=\varnothing I = ∅ ( H , h , s R d f , M , K ) (H,h,s^{\mathrm{df}}_{R},M,K) ( H , h , s R df , M , K ) s R d f > 0 R s^{\mathrm{df}}_{R}>0_{R} s R df > 0 R d = x R p f > 0 R d=x^{\mathrm{pf}}_{R}>0_{R} d = x R pf > 0 R
min ⟨ s R d f , x R ⟩ + M ⟨ 1 R , x ˉ R ⟩ max ⟨ h , y ⟩ − 2 K ⟨ 1 R , s ^ R ⟩ H x R − H x ˉ R = h H T y + s R − s ^ R = s R d f x R + x ˉ R = 2 K 1 R − H T y + s ˉ R = M 1 R x R , x ˉ R , x ^ R ≥ 0 R , s R , s ˉ R , s ^ R ≥ 0 R . (19) \begin{aligned}
\min\ &\langle s^{\mathrm{df}}_{R},x_{R}\rangle+M\langle1_{R},\bar{x}_{R}\rangle
&\max\ &\langle h,y\rangle-2K\langle1_{R},\hat{s}_{R}\rangle\\
&Hx_{R}-H\bar{x}_{R}=h
&&H^{\mathsf{T}}y+s_{R}-\hat{s}_{R}=s^{\mathrm{df}}_{R}\\
&x_{R}+\bar{x}_{R}=2K1_{R}
&&-H^{\mathsf{T}}y+\bar{s}_{R}=M1_{R}\\
&x_{R},\bar{x}_{R},\hat{x}_{R}\geq0_{R},
&&s_{R},\bar{s}_{R},\hat{s}_{R}\geq0_{R}. \tag*{(19)}
\end{aligned} min ⟨ s R df , x R ⟩ + M ⟨ 1 R , x ˉ R ⟩ H x R − H x ˉ R = h x R + x ˉ R = 2 K 1 R x R , x ˉ R , x ^ R ≥ 0 R , max ⟨ h , y ⟩ − 2 K ⟨ 1 R , s ^ R ⟩ H T y + s R − s ^ R = s R df − H T y + s ˉ R = M 1 R s R , s ˉ R , s ^ R ≥ 0 R . (19) This is the instantiation of MSys( A , b , c , M , K ) (A,b,c,M,K) ( A , b , c , M , K ) ( H , h ) (H,h) ( H , h ) s R d f s^{\mathrm{df}}_{R} s R df M M M K K K (11) and (12) in Lemma 3.1 . Note again that H H H H H H Theorem 1.6 .
The next lemma, analogously to Lemma 3.5 , shows how to convert these to optimal primal and dual solutions to LP ( A , b , c ) \operatorname{LP}(A,b,c) LP ( A , b , c )
Lemma 3.6. Let ( x R o , x R d f , x ^ R o ) (x_R^{\mathrm{o}},x_R^{\mathrm{df}},\hat{x}_R^{\mathrm{o}}) ( x R o , x R df , x ^ R o ) ( y o , s R o , s R d f , s ^ R o ) (y^{\mathrm{o}},s_R^{\mathrm{o}},s_R^{\mathrm{df}},\hat{s}_R^{\mathrm{o}}) ( y o , s R o , s R df , s ^ R o ) Theorem 1.6 . Let ( y ~ , s ~ ) (\tilde{y},\tilde{s}) ( y ~ , s ~ ) PrimalMaxS ( A , b ) \operatorname{PrimalMaxS}(A,b) PrimalMaxS ( A , b ) x ~ \tilde{x} x ~ DualMaxS ( A , c ) \operatorname{DualMaxS}(A,c) DualMaxS ( A , c )
s ~ = − A T y ~ , ⟨ b , y ~ ⟩ = 0 , s ~ ≥ 0 n , supp ( s ~ ) = J , and \tilde{s}=-A^{\mathsf{T}}\tilde{y},\quad\langle b,\tilde{y}\rangle=0,\quad\tilde{s}\geq0_n,\quad\operatorname{supp}(\tilde{s})=J,\quad\text{and} s ~ = − A T y ~ , ⟨ b , y ~ ⟩ = 0 , s ~ ≥ 0 n , supp ( s ~ ) = J , and A x ~ = 0 m , ⟨ c , x ~ ⟩ = 0 , x ~ ≥ 0 n , supp ( x ~ ) = I . A\tilde{x}=0_m,\quad\langle c,\tilde{x}\rangle=0,\quad\tilde{x}\geq0_n,\quad\operatorname{supp}(\tilde{x})=I. A x ~ = 0 m , ⟨ c , x ~ ⟩ = 0 , x ~ ≥ 0 n , supp ( x ~ ) = I . Let us define
z : = A I + ( b − A R x R o ) , z:=A_I^{+}(b-A_Rx_R^{\mathrm{o}}), z := A I + ( b − A R x R o ) , and pick α , λ ≥ 0 \alpha,\lambda\geq0 α , λ ≥ 0 z + α x ~ I > 0 I z+\alpha\tilde{x}_I>0_I z + α x ~ I > 0 I A J T ( Π y o + y d f + λ y ~ ) < c J A_J^{\mathsf{T}}(\Pi y^{\mathrm{o}}+y^{\mathrm{df}}+\lambda\tilde{y})<c_J A J T ( Π y o + y df + λ y ~ ) < c J
x ⋆ = ( x R ⋆ , x I ⋆ , x J ⋆ ) : = ( x R o , z , 0 J ) + α x ~ , x^\star=(x_R^\star,x_I^\star,x_J^\star):=(x_R^{\mathrm{o}},z,0_J)+\alpha\tilde{x}, x ⋆ = ( x R ⋆ , x I ⋆ , x J ⋆ ) := ( x R o , z , 0 J ) + α x ~ , y ⋆ : = Π y o + y d f + λ y ~ , and s ⋆ : = c − A T y ⋆ . y^\star:=\Pi y^{\mathrm{o}}+y^{\mathrm{df}}+\lambda\tilde{y},\quad\text{and}\quad s^\star:=c-A^{\mathsf{T}}y^\star. y ⋆ := Π y o + y df + λ y ~ , and s ⋆ := c − A T y ⋆ . Then, x ⋆ x^\star x ⋆ ( y ⋆ , s ⋆ ) (y^\star,s^\star) ( y ⋆ , s ⋆ ) LP ( A , b , c ) \operatorname{LP}(A,b,c) LP ( A , b , c )
Proof. Since x R d f , s R d f > 0 R x_R^{\mathrm{df}},s_R^{\mathrm{df}}>0_R x R df , s R df > 0 R x R o x_R^{\mathrm{o}} x R o ( y o , s R o ) (y^{\mathrm{o}},s_R^{\mathrm{o}}) ( y o , s R o ) LP ( H , h , s R d f ) \operatorname{LP}(H,h,s_R^{\mathrm{df}}) LP ( H , h , s R df ) x ⋆ x^\star x ⋆ ( y ⋆ , s ⋆ ) (y^\star,s^\star) ( y ⋆ , s ⋆ ) LP ( A , b , c ) \operatorname{LP}(A,b,c) LP ( A , b , c ) supp ( x ~ ) = I \operatorname{supp}(\tilde{x})=I supp ( x ~ ) = I supp ( s ~ ) = J \operatorname{supp}(\tilde{s})=J supp ( s ~ ) = J α , λ ≥ 0 \alpha,\lambda\geq0 α , λ ≥ 0
On the primal side, the choice of z z z A x ~ = 0 m A\tilde{x}=0_m A x ~ = 0 m A x ⋆ = b Ax^\star=b A x ⋆ = b x ⋆ ≥ 0 n x^\star\geq0_n x ⋆ ≥ 0 n x R o ≥ 0 R x_R^{\mathrm{o}}\geq0_R x R o ≥ 0 R α \alpha α x ⋆ x^\star x ⋆
Turning to the dual side, on the coordinates in R R R
H T y o + s R o = s R d f , A R T y d f + s R d f = c R , and A R T y ~ = 0 R . H^{\mathsf{T}}y^{\mathrm{o}}+s_R^{\mathrm{o}}=s_R^{\mathrm{df}},\quad A_R^{\mathsf{T}}y^{\mathrm{df}}+s_R^{\mathrm{df}}=c_R,\quad\text{and}\quad A_R^{\mathsf{T}}\tilde{y}=0_R. H T y o + s R o = s R df , A R T y df + s R df = c R , and A R T y ~ = 0 R . Since H T y o = A R T Π y o H^{\mathsf{T}}y^{\mathrm{o}}=A_R^{\mathsf{T}}\Pi y^{\mathrm{o}} H T y o = A R T Π y o s R ⋆ = s R o ≥ 0 R s_R^\star=s_R^{\mathrm{o}}\geq0_R s R ⋆ = s R o ≥ 0 R I I I
A I T Π y o = 0 I , A I T y d f = c I , and A I T y ~ = 0 I , A_I^{\mathsf{T}}\Pi y^{\mathrm{o}}=0_I,\quad A_I^{\mathsf{T}}y^{\mathrm{df}}=c_I,\quad\text{and}\quad A_I^{\mathsf{T}}\tilde{y}=0_I, A I T Π y o = 0 I , A I T y df = c I , and A I T y ~ = 0 I , where the first equation holds because Π \Pi Π ker ( A I T ) \ker(A_I^{\mathsf{T}}) ker ( A I T ) s I ⋆ = 0 I s_I^\star=0_I s I ⋆ = 0 I s J ⋆ > 0 J s_J^\star>0_J s J ⋆ > 0 J λ \lambda λ
To show optimality and strict complementarity, we note that supp ( x ⋆ ) = supp ( x R o ) ∪ I \operatorname{supp}(x^\star)=\operatorname{supp}(x_R^{\mathrm{o}})\cup I supp ( x ⋆ ) = supp ( x R o ) ∪ I supp ( s ⋆ ) = supp ( s R o ) ∪ J \operatorname{supp}(s^\star)=\operatorname{supp}(s_R^{\mathrm{o}})\cup J supp ( s ⋆ ) = supp ( s R o ) ∪ J x R o x_R^{\mathrm{o}} x R o s R o s_R^{\mathrm{o}} s R o R R R Theorem 1.6 returns maximum-support optimal solutions to (19); hence supp ( x R o ) \operatorname{supp}(x_R^{\mathrm{o}}) supp ( x R o ) supp ( s R o ) \operatorname{supp}(s_R^{\mathrm{o}}) supp ( s R o ) R R R I , J I,J I , J [ n ] [n] [ n ] supp ( x ⋆ ) \operatorname{supp}(x^\star) supp ( x ⋆ ) supp ( s ⋆ ) \operatorname{supp}(s^\star) supp ( s ⋆ ) x ⋆ x^\star x ⋆ ( y ⋆ , s ⋆ ) (y^\star,s^\star) ( y ⋆ , s ⋆ )
Proof of a weaker version of Theorem 1.7
The proof of Theorem 1.7 will require the stronger form of the initialization process described in § 4 . This also requires the dual certificates provided in Theorem 1.6 ; on the other hand, one can use simpler systems in Stages II and III.
We now explain the slightly weaker result that could be derived based on the algorithm in this section. First, the conic system (16) used in Stage I is the same as (S-Conic), except that the right-hand side is 2 ⋅ 1 n 2\cdot1_n 2 ⋅ 1 n
Consider now (17) solved in Stage II for dual feasibility. This arises from
min ⟨ c , x ⟩ + M ⟨ 1 n , x ˉ ⟩ , A x − A x ˉ = 0 m , x + x ^ = 2 K ⋅ 1 n , x , x ˉ , x ^ ≥ 0 n , \min\langle c,x\rangle+M\langle1_n,\bar{x}\rangle,\quad Ax-A\bar{x}=0_m,\quad x+\hat{x}=2K\cdot1_n,\quad x,\bar{x},\hat{x}\geq0_n, min ⟨ c , x ⟩ + M ⟨ 1 n , x ˉ ⟩ , A x − A x ˉ = 0 m , x + x ^ = 2 K ⋅ 1 n , x , x ˉ , x ^ ≥ 0 n , after deleting the variables corresponding to D c D^c D c
min ⟨ s c , x ⟩ + M ⟨ 1 n , x ˉ ⟩ , A x − A x ˉ = b , x + x ^ = 2 K ⋅ 1 n , x , x ˉ , x ^ ≥ 0 n , (20) \min\langle s^c,x\rangle+M\langle1_n,\bar{x}\rangle,\quad Ax-A\bar{x}=b,\quad x+\hat{x}=2K\cdot1_n,\quad x,\bar{x},\hat{x}\geq0_n, \tag*{(20)} min ⟨ s c , x ⟩ + M ⟨ 1 n , x ˉ ⟩ , A x − A x ˉ = b , x + x ^ = 2 K ⋅ 1 n , x , x ˉ , x ^ ≥ 0 n , (20) @page 20/46 starts_mid=0 ends_mid=1 The proof of Theorem 1.7 will require the stronger form of the initialization process described in § 4 . This also requires the dual certificates provided in Theorem 1.6 ; on the other hand, one can use simpler systems in Stages II and III.
We now explain the slightly weaker result that could be derived based on the algorithm in this section. First, the conic system (16) used in Stage I is the same as (S-Conic), except that the right-hand side is 2 ⋅ 1 n 2\cdot1_n 2 ⋅ 1 n
Consider now (17) solved in Stage II for dual feasibility. This arises from
min ⟨ c , x ⟩ + M ⟨ 1 n , x ˉ ⟩ , A x − A x ˉ = 0 m , x + x ^ = 2 K ⋅ 1 n , x , x ˉ , x ^ ≥ 0 n , \min\langle c,x\rangle+M\langle1_n,\bar{x}\rangle,\quad Ax-A\bar{x}=0_m,\quad x+\hat{x}=2K\cdot1_n,\quad x,\bar{x},\hat{x}\geq0_n, min ⟨ c , x ⟩ + M ⟨ 1 n , x ˉ ⟩ , A x − A x ˉ = 0 m , x + x ^ = 2 K ⋅ 1 n , x , x ˉ , x ^ ≥ 0 n , after deleting the variables corresponding to D c D^c D c
min ⟨ s c , x ⟩ + M ⟨ 1 n , x ˉ ⟩ , A x − A x ˉ = b , x + x ^ = 2 K ⋅ 1 n , x , x ˉ , x ^ ≥ 0 n , \min\langle s^c,x\rangle+M\langle1_n,\bar{x}\rangle,\quad Ax-A\bar{x}=b,\quad x+\hat{x}=2K\cdot1_n,\quad x,\bar{x},\hat{x}\geq0_n, min ⟨ s c , x ⟩ + M ⟨ 1 n , x ˉ ⟩ , A x − A x ˉ = b , x + x ^ = 2 K ⋅ 1 n , x , x ˉ , x ^ ≥ 0 n , after projecting out the variables corresponding to P P P (19) solved for optimization in Stage III arises from
min ⟨ s d f , x ⟩ + M ⟨ 1 n , x ˉ ⟩ , A x − A x ˉ = b , x + x ^ = 2 K ⋅ 1 n , x , x ˉ , x ^ ≥ 0 n , (21) \min\langle s^{\mathrm{df}},x\rangle+ M\langle1_n,\bar{x}\rangle,\quad Ax-A\bar{x}=b,\quad x+\hat{x}=2K\cdot1_n,\quad x,\bar{x},\hat{x}\geq0_n, \tag*{(21)} min ⟨ s df , x ⟩ + M ⟨ 1 n , x ˉ ⟩ , A x − A x ˉ = b , x + x ^ = 2 K ⋅ 1 n , x , x ˉ , x ^ ≥ 0 n , (21) after deleting and projecting out variables.
Hence, one could derive a version of Theorem 1.7 where the running time is bounded in terms of the maximum of the straight line complexities of the systems (S-Opt′), (S-Conic), (S-Dual′), and (S-Primal′), for suitable choices of M M M K K K ( c , M 1 n ) (c,M1_n) ( c , M 1 n ) ( s , 1 n ) (s,1_n) ( s , 1 n ) ( s d f , M 1 n ) (s^{\mathrm{df}},M1_n) ( s df , M 1 n )
However, we note that for many natural classes of LPs, the systems (S-Opt′), (S-Conic), (S-Dual′), and (S-Primal′) would still fall into the same class, and the class is closed under variable deletion and projection. For example, if the starting LP corresponds to a 2VPI primal-dual pair, all obtained systems also belong to this family, after straight-line-complexity-preserving transformations; we omit the details. In particular, for generalized flow problems, variable deletion and projection correspond to deleting and contracting arcs, respectively.
Multi-stage analytic center initialization
We now present the second initialization approach that yields the guarantees in Theorem 1.7. We make use of the dual certificates provided in Theorem 1.6 that show that the optimal solutions returned are near the analytic centers of their optimal faces. We proceed in the same three stages as in § 3.2 . We will use this theorem for different neighborhoods in the three stages, with neighborhood parameters
β c : = 2 − 16 , β f : = 2 − 8 , β o : = 2 − 4 (22) \beta^{\mathrm{c}}:=2^{-16},\qquad\beta^{\mathrm{f}}:=2^{-8},\qquad\beta^{\mathrm{o}}:=2^{-4} \tag*{(22)} β c := 2 − 16 , β f := 2 − 8 , β o := 2 − 4 (22) for the conic, feasibility, and optimization stages, respectively. These parameters leave enough room for the certificates to hold only with 3 β 3\beta 3 β β \beta β d ∈ R n d\in\mathbb{R}^{n} d ∈ R n A d = b Ad=b A d = b d d d size ( d ) = poly ( n ) size ( A , b ) \operatorname{size}(d)=\operatorname{poly}(n)\operatorname{size}(A,b) size ( d ) = poly ( n ) size ( A , b ) O ( n 5 ) O(n^5) O ( n 5 ) d : = A + b d:=A^{+}b d := A + b
Stage I: Conic feasibility
This stage is identical to the algorithm in § 3.2 . We run the IPM in Theorem 1.6 to solve MSys ( A , 0 m , 0 n , 1 , 1 ) \operatorname{MSys}(A,0_m,0_n,1,1) MSys ( A , 0 m , 0 n , 1 , 1 ) (16) ) using the neighborhood parameter β c \beta^{\mathrm{c}} β c
A partition P c ∪ D c = [ n ] P^{\mathrm{c}}\cup D^{\mathrm{c}}=[n] P c ∪ D c = [ n ] ( x c , x ˉ c , x ^ c ) (x^{\mathrm{c}},\bar{x}^{\mathrm{c}},\hat{x}^{\mathrm{c}}) ( x c , x ˉ c , x ^ c ) ( y c , s c , s ˉ c , s ^ c ) (y^{\mathrm{c}},s^{\mathrm{c}},\bar{s}^{\mathrm{c}},\hat{s}^{\mathrm{c}}) ( y c , s c , s ˉ c , s ^ c ) (16) with x ˉ c = s ^ c = 0 n \bar{x}^{\mathrm{c}}=\hat{s}^{\mathrm{c}}=0_n x ˉ c = s ^ c = 0 n supp ( x c ) = P c \operatorname{supp}(x^{\mathrm{c}})=P^{\mathrm{c}} supp ( x c ) = P c supp ( s c ) = D c \operatorname{supp}(s^{\mathrm{c}})=D^{\mathrm{c}} supp ( s c ) = D c
Vectors ( v c , v ^ c ) ∈ R 2 n (v^{\mathrm{c}},\hat{v}^{\mathrm{c}})\in\mathbb{R}^{2n} ( v c , v ^ c ) ∈ R 2 n u c ∈ R m u^{\mathrm{c}}\in\mathbb{R}^{m} u c ∈ R m v c = A T u c + v ^ c v^{\mathrm{c}}=A^{\mathsf T}u^{\mathrm{c}}+\hat{v}^{\mathrm{c}} v c = A T u c + v ^ c
∥ ( x P c c ∘ v P c c , x ^ P c c ∘ v ^ P c c ) − 1 2 ∣ P c ∣ ∥ 2 ≤ β c , (23) \left\|(x^{\mathrm{c}}_{P^{\mathrm{c}}}\circ v^{\mathrm{c}}_{P^{\mathrm{c}}},\hat{x}^{\mathrm{c}}_{P^{\mathrm{c}}}\circ\hat{v}^{\mathrm{c}}_{P^{\mathrm{c}}})-1_{2|P^{\mathrm{c}}|}\right\|_2\leq\beta^{\mathrm{c}}, \tag*{(23)} ( x P c c ∘ v P c c , x ^ P c c ∘ v ^ P c c ) − 1 2∣ P c ∣ 2 ≤ β c , (23) Vectors ( w c , w ˉ c ) ∈ R 2 n (w^{\mathrm{c}},\bar{w}^{\mathrm{c}})\in\mathbb{R}^{2n} ( w c , w ˉ c ) ∈ R 2 n A ( w c − w ˉ c ) = 0 m A(w^{\mathrm{c}}-\bar{w}^{\mathrm{c}})=0_m A ( w c − w ˉ c ) = 0 m
∥ ( s D c c ∘ w D c c , s ˉ D c c ∘ w ˉ D c c ) − 1 2 ∣ D c ∣ ∥ 2 ≤ β c . (24) \left\|(s^{\mathrm{c}}_{D^{\mathrm{c}}}\circ w^{\mathrm{c}}_{D^{\mathrm{c}}},\bar{s}^{\mathrm{c}}_{D^{\mathrm{c}}}\circ\bar{w}^{\mathrm{c}}_{D^{\mathrm{c}}})-1_{2|D^{\mathrm{c}}|}\right\|_2\leq\beta^{\mathrm{c}}. \tag*{(24)} ( s D c c ∘ w D c c , s ˉ D c c ∘ w ˉ D c c ) − 1 2∣ D c ∣ 2 ≤ β c . (24) We want to emphasize that the conditions in (23) and (24) imply that w D c c , w ˉ D c c , v P c c , v ^ P c c w^{\mathrm{c}}_{D^{\mathrm{c}}},\bar{w}^{\mathrm{c}}_{D^{\mathrm{c}}},v^{\mathrm{c}}_{P^{\mathrm{c}}},\hat{v}^{\mathrm{c}}_{P^{\mathrm{c}}} w D c c , w ˉ D c c , v P c c , v ^ P c c
Stage II: Maximum support feasible solutions
In the second stage, we define two modifications of PrimalMaxS and DualMaxS that deliver stronger outputs as follows. Here, P c P^{\mathrm{c}} P c D c D^{\mathrm{c}} D c
Subroutine 4.1 (PrimalMaxSA). PrimalMaxSA( A , b ) (A,b) ( A , b ) A ∈ R m × n A\in\mathbb{R}^{m\times n} A ∈ R m × n b ∈ R m b\in\mathbb{R}^{m} b ∈ R m x p f ∈ R n x^{\mathrm{pf}} \in\mathbb{R}^n x pf ∈ R n A x = b Ax=b A x = b x ≥ 0 n x\geq0_n x ≥ 0 n P p f = supp ( x p f ) P^{\mathrm{pf}}=\operatorname{supp}(x^{\mathrm{pf}}) P pf = supp ( x pf ) ( y ~ , s ~ ) (\widetilde{y},\widetilde{s}) ( y , s ) s ~ = − A T y ~ \widetilde{s}=-A^{\mathsf{T}}\widetilde{y} s = − A T y s ~ ≥ 0 n \widetilde{s}\geq0_n s ≥ 0 n supp ( s ~ ) = [ n ] ∖ P p f \operatorname{supp}(\widetilde{s})=[n]\setminus P^{\mathrm{pf}} supp ( s ) = [ n ] ∖ P pf u p f ∈ R m u^{\mathrm{pf}}\in\mathbb{R}^m u pf ∈ R m v p f = A T u p f ∈ R n v^{\mathrm{pf}}=A^{\mathsf{T}}u^{\mathrm{pf}}\in\mathbb{R}^n v pf = A T u pf ∈ R n v P p f ∩ D c p f = 0 P p f ∩ D c v^{\mathrm{pf}}_{P^{\mathrm{pf}}\cap D^c}=0_{P^{\mathrm{pf}}\cap D^c} v P pf ∩ D c pf = 0 P pf ∩ D c
∥ ( x P p f ∩ D c p f ∘ v P p f ∩ D c p f ) − 1 P p f ∩ D c ∥ 2 ≤ β f . (25) \left\|\left(x^{\mathrm{pf}}_{P^{\mathrm{pf}}\cap D^c}\circ v^{\mathrm{pf}}_{P^{\mathrm{pf}}\cap D^c}\right)-1_{P^{\mathrm{pf}}\cap D^c}\right\|_2\leq\beta^{\mathrm{f}}. \tag*{(25)} ( x P pf ∩ D c pf ∘ v P pf ∩ D c pf ) − 1 P pf ∩ D c 2 ≤ β f . (25) (ii) An infeasibility certificate ( y ~ , s ~ ) ∈ R m × R n (\widetilde{y},\widetilde{s})\in\mathbb{R}^m\times\mathbb{R}^n ( y , s ) ∈ R m × R n s ~ = − A T y ~ \widetilde{s}=-A^{\mathsf{T}}\widetilde{y} s = − A T y s ~ ≥ 0 n \widetilde{s}\geq0_n s ≥ 0 n ⟨ b , y ~ ⟩ > 0 \langle b,\widetilde{y}\rangle>0 ⟨ b , y ⟩ > 0
Subroutine 4.2 (DualMaxSA). DualMaxSA ( A , c ) (A,c) ( A , c ) A ∈ R m × n A\in\mathbb{R}^{m\times n} A ∈ R m × n c ∈ R n c\in\mathbb{R}^n c ∈ R n
(i) A solution ( y d f , s d f ) ∈ R m × R n (y^{\mathrm{df}},s^{\mathrm{df}})\in\mathbb{R}^m\times\mathbb{R}^n ( y df , s df ) ∈ R m × R n s d f s^{\mathrm{df}} s df A T y + s = c A^{\mathsf{T}}y+s=c A T y + s = c s ≥ 0 n s\geq0_n s ≥ 0 n D d f = supp ( s d f ) D^{\mathrm{df}}=\operatorname{supp}(s^{\mathrm{df}}) D df = supp ( s df ) A x ~ = 0 m A\widetilde{x}=0_m A x = 0 m x ~ ≥ 0 n \widetilde{x}\geq0_n x ≥ 0 n supp ( x ~ ) = [ n ] ∖ D d f \operatorname{supp}(\widetilde{x})=[n]\setminus D^{\mathrm{df}} supp ( x ) = [ n ] ∖ D df w d f ∈ R n w^{\mathrm{df}}\in\mathbb{R}^n w df ∈ R n A w d f = 0 m Aw^{\mathrm{df}}=0_m A w df = 0 m w D c d f = 0 D c w^{\mathrm{df}}_{D^c}=0_{D^c} w D c df = 0 D c
∥ ( s D d f ∩ P c d f ∘ w D d f ∩ P c d f ) − 1 D d f ∩ P c ∥ 2 ≤ β f . (26) \left\|\left(s^{\mathrm{df}}_{D^{\mathrm{df}}\cap P^c}\circ w^{\mathrm{df}}_{D^{\mathrm{df}}\cap P^c}\right)-1_{D^{\mathrm{df}}\cap P^c}\right\|_2\leq\beta^{\mathrm{f}}. \tag*{(26)} ( s D df ∩ P c df ∘ w D df ∩ P c df ) − 1 D df ∩ P c 2 ≤ β f . (26) (ii) An infeasibility certificate x ~ ∈ R n \widetilde{x}\in\mathbb{R}^n x ∈ R n A x ~ = 0 m A\widetilde{x}=0_m A x = 0 m x ~ ≥ 0 n \widetilde{x}\geq0_n x ≥ 0 n ⟨ c , x ~ ⟩ < 0 \langle c,\widetilde{x}\rangle<0 ⟨ c , x ⟩ < 0
Dual feasibility We start by describing DualMaxSA ( A , c ) (A,c) ( A , c ) § 3.2 , we can restrict our attention to the subproblem defined by ( A P c , c P c ) (A_{P^c},c_{P^c}) ( A P c , c P c ) (26) . We may assume that P c ≠ ∅ P^c\neq\varnothing P c = ∅ c P c ≠ 0 P c c_{P^c}\neq0_{P^c} c P c = 0 P c D d f = D c D^{\mathrm{df}}=D^c D df = D c ( y d f , s d f ) : = ( α y c , c + α s c ) (y^{\mathrm{df}},s^{\mathrm{df}}):=(\alpha y^c,c+\alpha s^c) ( y df , s df ) := ( α y c , c + α s c ) α ≥ 0 \alpha\geq0 α ≥ 0 s D c d f > 0 D c s^{\mathrm{df}}_{D^c}>0_{D^c} s D c df > 0 D c x ~ : = x c \widetilde{x}:=x^c x := x c w d f : = x c w^{\mathrm{df}}:=x^c w df := x c (26) is vacuously true.
Instead of (17) , we solve a simpler system where the residual variables x ˉ P c \bar{x}_{P^c} x ˉ P c s ˉ P c \bar{s}_{P^c} s ˉ P c M M M
min ⟨ c P c , x P c ⟩ s.t. A P c x P c = 0 m x P c + x ^ P c = 2 ⋅ 1 P c x P c , x ^ P c ≥ 0 P c , max − 2 ⟨ 1 P c , s ^ P c ⟩ s.t. A P c T y + s P c − s ^ P c = c P c s P c , s ^ P c ≥ 0 P c . (27) \begin{aligned}
\min\quad&\langle c_{P^c},x_{P^c}\rangle\\
\text{s.t.}\quad&A_{P^c}x_{P^c}=0_m\\
&x_{P^c}+\widehat{x}_{P^c}=2\cdot1_{P^c}\\
&x_{P^c},\widehat{x}_{P^c}\geq0_{P^c},
\end{aligned}
\qquad
\begin{aligned}
\max\quad&-2\langle1_{P^c},\widehat{s}_{P^c}\rangle\\
\text{s.t.}\quad&A_{P^c}^{\mathsf{T}}y+s_{P^c}-\widehat{s}_{P^c}=c_{P^c}\\
&s_{P^c},\widehat{s}_{P^c}\geq0_{P^c}. \tag*{(27)}
\end{aligned} min s.t. ⟨ c P c , x P c ⟩ A P c x P c = 0 m x P c + x P c = 2 ⋅ 1 P c x P c , x P c ≥ 0 P c , max s.t. − 2 ⟨ 1 P c , s P c ⟩ A P c T y + s P c − s P c = c P c s P c , s P c ≥ 0 P c . (27) We use the primal solution ( x c , x ^ c ) ∈ R 2 n (x^c,\widehat{x}^c)\in\mathbb{R}^{2n} ( x c , x c ) ∈ R 2 n ( v c , v ^ c ) ∈ R 2 n (v^c,\widehat{v}^c)\in\mathbb{R}^{2n} ( v c , v c ) ∈ R 2 n u c ∈ R m u^c\in\mathbb{R}^m u c ∈ R m v c = A T u c + v ^ c v^c=A^{\mathsf{T}}u^c+\widehat{v}^c v c = A T u c + v c (23) to find an initial solution to this system. Namely, we define
μ : = 2 ∥ c P c ∥ 1 β c , \mu:=\frac{2\|c_{P^c}\|_1}{\beta^c}, μ := β c 2∥ c P c ∥ 1 , y : = − μ u c , y:=-\mu u^c, y := − μ u c , ( s P c , s ^ P c ) : = ( c P c + μ v P c c , μ v ^ P c c ) . (s_{P^c},\widehat{s}_{P^c}):=(c_{P^c}+\mu v^c_{P^c},\mu\widehat{v}^c_{P^c}). ( s P c , s P c ) := ( c P c + μ v P c c , μ v P c c ) . Note that μ > 0 \mu>0 μ > 0 ( x P c c , x ^ P c c ) (x^c_{P^c},\widehat{x}^c_{P^c}) ( x P c c , x P c c ) ( y , s P c , s ^ P c ) (y,s_{P^c},\widehat{s}_{P^c}) ( y , s P c , s P c )
∥ ( x P c c , x ^ P c c ) ∘ ( s P c , s ^ P c ) μ − 1 2 ∣ P c ∣ ∥ 2 = ∥ ( x P c c , x ^ P c c ) ∘ ( v P c c , v ^ P c c ) − 1 2 ∣ P c ∣ + ( x P c c , x ^ P c c ) ∘ ( c P c , 0 P c ) μ ∥ 2 ≤ ∥ ( x P c c , x ^ P c c ) ∘ ( v P c c , v ^ P c c ) − 1 2 ∣ P c ∣ ∥ 2 + ∥ x P c c ∘ c P c ∥ 2 μ ≤ β c + β c = 2 β c , (28) \begin{aligned}
\left\|\frac{(x^c_{P^c},\widehat{x}^c_{P^c})\circ(s_{P^c},\widehat{s}_{P^c})}{\mu}-1_{2|P^c|}\right\|_2
=
\left\|(x^c_{P^c},\widehat{x}^c_{P^c})\circ(v^c_{P^c},\widehat{v}^c_{P^c})-1_{2|P^c|}
+\frac{(x^c_{P^c},\widehat{x}^c_{P^c})\circ(c_{P^c},0_{P^c})}{\mu}\right\|_2 \\
&\leq\left\|(x^c_{P^c},\widehat{x}^c_{P^c})\circ(v^c_{P^c},\widehat{v}^c_{P^c})-1_{2|P^c|}\right\|_2
+\frac{\|x^c_{P^c}\circ c_{P^c}\|_2}{\mu} \\
&\leq\beta^c+\beta^c=2\beta^c, \tag*{(28)}
\end{aligned} μ ( x P c c , x P c c ) ∘ ( s P c , s P c ) − 1 2∣ P c ∣ 2 = ( x P c c , x P c c ) ∘ ( v P c c , v P c c ) − 1 2∣ P c ∣ + μ ( x P c c , x P c c ) ∘ ( c P c , 0 P c ) 2 ≤ ( x P c c , x P c c ) ∘ ( v P c c , v P c c ) − 1 2∣ P c ∣ 2 + μ ∥ x P c c ∘ c P c ∥ 2 ≤ β c + β c = 2 β c , (28) where the last inequality uses that ∥ x P c c ∥ ∞ ≤ 2 \|x^c_{P^c}\|_\infty\leq2 ∥ x P c c ∥ ∞ ≤ 2 ∥ x P c c ∘ c P c ∥ 2 ≤ 2 ∥ c P c ∥ 2 \|x^c_{P^c}\circ c_{P^c}\|_2\leq2\|c_{P^c}\|_2 ∥ x P c c ∘ c P c ∥ 2 ≤ 2∥ c P c ∥ 2 β c \beta^c β c β f \beta^f β f
( ( x P c c , x ^ P c c ) , ( s P c , s ^ P c ) ) ∈ N 2 ( 2 β c 1 − 2 β c ) ⊆ N 2 ( β f ) . \left((x^c_{P^c},\widehat{x}^c_{P^c}),(s_{P^c},\widehat{s}_{P^c})\right)\in\mathcal{N}^2\left(\frac{2\beta^c}{1-2\beta^c}\right)\subseteq\mathcal{N}^2(\beta^f). ( ( x P c c , x P c c ) , ( s P c , s P c ) ) ∈ N 2 ( 1 − 2 β c 2 β c ) ⊆ N 2 ( β f ) . Hence, we can run the IPM in Theorem 1.6 for the system (27) using this starting solution and neighborhood parameter β = β f \beta=\beta^f β = β f ( x P c d f , x ^ P c d f ) (x^{\mathrm{df}}_{P^c},\widehat{x}^{\mathrm{df}}_{P^c}) ( x P c df , x P c df ) ( y ′ , s P c d f , s ^ P c d f ) (y',s^{\mathrm{df}}_{P^c},\widehat{s}^{\mathrm{df}}_{P^c}) ( y ′ , s P c df , s P c df ) § 3.2 , if ⟨ c P c , x P c d f ⟩ < 0 \langle c_{P^c},x_{P^c}^{\mathrm{df}}\rangle< 0 ⟨ c P c , x P c df ⟩ < 0 x ~ : = ( x P c d f , 0 D c ) \tilde{x}:=(x_{P^c}^{\mathrm{df}},0_{D^c}) x ~ := ( x P c df , 0 D c ) LP ( A , b , c ) \operatorname{LP}(A,b,c) LP ( A , b , c ) ⟨ c P c , x P c d f ⟩ = 0 \langle c_{P^c},x_{P^c}^{\mathrm{df}}\rangle=0 ⟨ c P c , x P c df ⟩ = 0 s ^ P c d f = 0 P c \hat{s}_{P^c}^{\mathrm{df}}=0_{P^c} s ^ P c df = 0 P c ( y ′ , s P c ′ ) (y',s_{P^c}') ( y ′ , s P c ′ ) LP ( A P c , 0 m , c P c ) \operatorname{LP}(A_{P^c},0_m,c_{P^c}) LP ( A P c , 0 m , c P c ) A P c T y ′ + s P c ′ = c P c A_{P^c}^{\mathsf T}y'+s_{P^c}'=c_{P^c} A P c T y ′ + s P c ′ = c P c s P c ′ ≥ 0 P c s_{P^c}'\geq0_{P^c} s P c ′ ≥ 0 P c D ′ : = supp ( s P c d f ) D':=\operatorname{supp}(s_{P^c}^{\mathrm{df}}) D ′ := supp ( s P c df ) ( y c , s c ) (y^c,s^c) ( y c , s c ) y d f : = y ′ + α y c y^{\mathrm{df}}:=y'+\alpha y^c y df := y ′ + α y c s d f : = c − A T y d f s^{\mathrm{df}}:=c-A^{\mathsf T}y^{\mathrm{df}} s df := c − A T y df α > 0 \alpha>0 α > 0 α \alpha α s D c d f > 0 D c s_{D^c}^{\mathrm{df}}>0_{D^c} s D c df > 0 D c LP ( A , b , c ) \operatorname{LP}(A,b,c) LP ( A , b , c ) s P c d f s_{P^c}^{\mathrm{df}} s P c df A P c T y c = 0 P c A_{P^c}^{\mathsf T}y^c=0_{P^c} A P c T y c = 0 P c D d f : = supp ( s d f ) = D ′ ∪ D c D^{\mathrm{df}}:=\operatorname{supp}(s^{\mathrm{df}})=D'\cup D^c D df := supp ( s df ) = D ′ ∪ D c D d f ∩ P c = D ′ D^{\mathrm{df}}\cap P^c=D' D df ∩ P c = D ′ x ~ : = ( x P c d f , 0 D c ) \tilde{x}:=(x_{P^c}^{\mathrm{df}},0_{D^c}) x ~ := ( x P c df , 0 D c ) A x ~ = 0 m A\tilde{x}=0_m A x ~ = 0 m x ~ ≥ 0 n \tilde{x}\geq0_n x ~ ≥ 0 n supp ( x ~ ) = [ n ] ∖ D d f \operatorname{supp}(\tilde{x})=[n]\setminus D^{\mathrm{df}} supp ( x ~ ) = [ n ] ∖ D df
From Theorem 1.6 , we also obtained ( w ′ , w ^ ′ ) ∈ R 2 ∣ P c ∣ (w',\hat{w}')\in\mathbb{R}^{2|P^c|} ( w ′ , w ^ ′ ) ∈ R 2∣ P c ∣ A P c w ′ = 0 m A_{P^c}w'=0_m A P c w ′ = 0 m w ′ + w ^ ′ = 0 P c w'+\hat{w}'=0_{P^c} w ′ + w ^ ′ = 0 P c
∥ ( s D ′ d f ∘ w D ′ ′ − 1 D ′ ) ∥ 2 ≤ β f , \left\|(s_{D'}^{\mathrm{df}}\circ w_{D'}'-1_{D'})\right\|_2\leq\beta^{\mathrm f}, ( s D ′ df ∘ w D ′ ′ − 1 D ′ ) 2 ≤ β f , and thus, w d f : = ( w ′ , 0 D c ) w^{\mathrm{df}}:=(w',0_{D^c}) w df := ( w ′ , 0 D c ) (26) in the feasibility case, noting that D ′ = D d f ∩ P c D'=D^{\mathrm{df}}\cap P^c D ′ = D df ∩ P c
Primal feasibility. As in § 3.2 , we define
Π : = I m − A P c A P c + , H : = Π A D c , and h : = Π b . \Pi:=I_m-A_{P^c}A_{P^c}^{+},\qquad H:=\Pi A_{D^c},\qquad\text{and}\qquad h:=\Pi b. Π := I m − A P c A P c + , H := Π A D c , and h := Π b . We may assume that D c ≠ ∅ D^c\ne\varnothing D c = ∅ d D c ≠ 0 D c d_{D^c}\ne0_{D^c} d D c = 0 D c P p f = P c P^{\mathrm{pf}}=P^c P pf = P c x p f : = d + α x c x^{\mathrm{pf}}:=d+\alpha x^c x pf := d + α x c α ≥ 0 \alpha\geq0 α ≥ 0 x P c p f > 0 P c x_{P^c}^{\mathrm{pf}}>0_{P^c} x P c pf > 0 P c ( y ~ , s ~ ) : = ( y c , s c ) (\tilde{y},\tilde{s}):=(y^c,s^c) ( y ~ , s ~ ) := ( y c , s c ) u p f : = − y c u^{\mathrm{pf}}:=-y^c u pf := − y c v p f : = A T u p f = s c v^{\mathrm{pf}}:=A^{\mathsf T}u^{\mathrm{pf}}=s^c v pf := A T u pf = s c (25) is vacuously true.
From the system (18), we drop the bound variables x ^ D c \hat{x}_{D^c} x ^ D c s ^ D c \hat{s}_{D^c} s ^ D c K K K
min ⟨ 1 D c , x ˉ D c ⟩ max ⟨ h , y ⟩ s.t. H x D c − H x ˉ D c = h s.t. H T y + s D c = 0 D c x D c , x ˉ D c ≥ 0 D c , − H T y + s ˉ D c = 1 D c s D c , s ˉ D c ≥ 0 D c . (29) \begin{aligned}
\min\quad&\langle1_{D^c},\bar{x}_{D^c}\rangle
&\max\quad&\langle h,y\rangle\\
\text{s.t.}\quad&Hx_{D^c}-H\bar{x}_{D^c}=h
&\text{s.t.}\quad&H^{\mathsf T}y+s_{D^c}=0_{D^c}\\
&x_{D^c},\bar{x}_{D^c}\geq0_{D^c},
&&-H^{\mathsf T}y+\bar{s}_{D^c}=1_{D^c}\\
&&&s_{D^c},\bar{s}_{D^c}\geq0_{D^c}. \tag*{(29)}
\end{aligned} min s.t. ⟨ 1 D c , x ˉ D c ⟩ H x D c − H x ˉ D c = h x D c , x ˉ D c ≥ 0 D c , max s.t. ⟨ h , y ⟩ H T y + s D c = 0 D c − H T y + s ˉ D c = 1 D c s D c , s ˉ D c ≥ 0 D c . (29) We initialize the system using the dual solution ( y c , s c , s ˉ c ) (y^c,s^c,\bar{s}^c) ( y c , s c , s ˉ c ) ( w c , w ˉ c ) (w^c,\bar{w}^c) ( w c , w ˉ c ) (24) . Observe that Π y c = y c \Pi y^c=y^c Π y c = y c y c ∈ ker ( A P c T ) y^c\in\ker(A_{P^c}^{\mathsf T}) y c ∈ ker ( A P c T ) H T y c = A D c T y c H^{\mathsf T}y^c=A_{D^c}^{\mathsf T}y^c H T y c = A D c T y c d ∈ R n d\in\mathbb{R}^n d ∈ R n A d = b Ad=b A d = b H d D c = h Hd_{D^c}=h H d D c = h Π A P c d P c = 0 m \Pi A_{P^c}d_{P^c}=0_m Π A P c d P c = 0 m
μ : = ∥ d D c ∥ 1 β c , \mu:=\frac{\lVert d_{D^c}\rVert_1}{\beta^c}, μ := β c ∥ d D c ∥ 1 , ( x D c , x ˉ D c ) : = ( d D c + μ w D c c , μ w ˉ D c c ) . (x_{D^c},\bar{x}_{D^c}):=(d_{D^c}+\mu w_{D^c}^c,\mu\bar{w}_{D^c}^c). ( x D c , x ˉ D c ) := ( d D c + μ w D c c , μ w ˉ D c c ) . Note that μ > 0 \mu>0 μ > 0
Primal feasibility is verified by observing that x D c , x ˉ D c ≥ 0 D c x_{D^c},\bar{x}_{D^c}\geq0_{D^c} x D c , x ˉ D c ≥ 0 D c H ( x D c − x ˉ D c ) = H d D c + μ H ( w D c c − w ˉ D c c ) = h H(x_{D^c}-\bar{x}_{D^c})=Hd_{D^c}+\mu H(w_{D^c}^c-\bar{w}_{D^c}^c)=h H ( x D c − x ˉ D c ) = H d D c + μ H ( w D c c − w ˉ D c c ) = h Π A D c ( w D c c − w ˉ D c c ) = Π A ( w c − w ˉ c ) = 0 m \Pi A_{D^c}(w_{D^c}^c-\bar{w}_{D^c}^c)=\Pi A(w^c-\bar{w}^c)=0_m Π A D c ( w D c c − w ˉ D c c ) = Π A ( w c − w ˉ c ) = 0 m
( ( x D c , x ˉ D c ) , ( s D c c , s ˉ D c c ) ) ∈ N 2 ( β f ) . \bigl((x_{D^c},\bar{x}_{D^c}),(s_{D^c}^c,\bar{s}_{D^c}^c)\bigr)\in\mathcal{N}^{2}(\beta^{\mathrm f}). ( ( x D c , x ˉ D c ) , ( s D c c , s ˉ D c c ) ) ∈ N 2 ( β f ) . Indeed, in the conic system the optimum has s ^ c = 0 \hat{s}^c=0 s ^ c = 0 s c = − A T y c s^c=-A^{\mathsf T}y^c s c = − A T y c s ˉ c = 1 n + A T y c = 1 n − s c \bar{s}^c=1_n+A^{\mathsf T}y^c=1_n-s^c s ˉ c = 1 n + A T y c = 1 n − s c s ˉ c ≥ 0 \bar{s}^c\geq0 s ˉ c ≥ 0 s c ≤ 1 n s^c\leq1_n s c ≤ 1 n
∥ ( x D c , x ˉ D c ) ∘ ( s D c c , s ˉ D c c ) μ − 1 2 ∣ D c ∣ ∥ 2 = ∥ ( s D c c ∘ w D c c , s ˉ D c c ∘ w ˉ D c c ) − 1 2 ∣ D c ∣ + ( d D c ∘ s D c c μ , 0 D c ) ∥ 2 ≤ β c + β c = 2 β c , \left\|\frac{(x_{D^c},\bar{x}_{D^c})\circ(s_{D^c}^c,\bar{s}_{D^c}^c)}{\mu}-1_{2|D^c|}\right\|_2
=
\left\|(s_{D^c}^c\circ w_{D^c}^c,\bar{s}_{D^c}^c\circ\bar{w}_{D^c}^c)-1_{2|D^c|}+\left(\frac{d_{D^c}\circ s_{D^c}^c}{\mu},0_{D^c}\right)\right\|_2
\leq\beta^c+\beta^c=2\beta^c, μ ( x D c , x ˉ D c ) ∘ ( s D c c , s ˉ D c c ) − 1 2∣ D c ∣ 2 = ( s D c c ∘ w D c c , s ˉ D c c ∘ w ˉ D c c ) − 1 2∣ D c ∣ + ( μ d D c ∘ s D c c , 0 D c ) 2 ≤ β c + β c = 2 β c , where the last inequality uses (24) and the definition of μ \mu μ
( ( x D c , x ˉ D c ) , ( s D c c , s ˉ D c c ) ) ∈ N 2 ( 2 β c 1 − 2 β c ) ⊆ N 2 ( β f ) . (30) \bigl((x_{D^c},\bar{x}_{D^c}),(s_{D^c}^c,\bar{s}_{D^c}^c)\bigr)\in\mathcal{N}^{2}\left(\frac{2\beta^c}{1-2\beta^c}\right)\subseteq\mathcal{N}^{2}(\beta^{\mathrm f}). \tag*{(30)} ( ( x D c , x ˉ D c ) , ( s D c c , s ˉ D c c ) ) ∈ N 2 ( 1 − 2 β c 2 β c ) ⊆ N 2 ( β f ) . (30) The next lemma is the adaptation of Lemma 3.5 , also including the analytic center certificate as in (25) . The proof follows the same lines.
Lemma 4.3. Let ( x D c p f , x ˉ D c p f ) (x^{\mathrm{pf}}_{D^c},\bar{x}^{\mathrm{pf}}_{D^c}) ( x D c pf , x ˉ D c pf ) ( y p f , s D c p f , s ˉ D c p f ) (y^{\mathrm{pf}},s^{\mathrm{pf}}_{D^c},\bar{s}^{\mathrm{pf}}_{D^c}) ( y pf , s D c pf , s ˉ D c pf ) (29) obtained via Theorem 1.6 , along with the dual analytic center certificate u ′ ∈ R m u' \in\mathbb{R}^m u ′ ∈ R m v ′ = H T u ′ v' = H^\mathsf{T}u' v ′ = H T u ′ 5 x c , s c ∈ R n x^c,s^c \in\mathbb{R}^n x c , s c ∈ R n y c ∈ R m y^c \in\mathbb{R}^m y c ∈ R m A x c = 0 m Ax^c=0_m A x c = 0 m x c ≥ 0 n x^c \geq0_n x c ≥ 0 n s c = − A T y c s^c=-A^\mathsf{T}y^c s c = − A T y c s c ≥ 0 n s^c\geq0_n s c ≥ 0 n supp ( x c ) = P c \operatorname{supp}(x^c)=P^c supp ( x c ) = P c supp ( s c ) = D c \operatorname{supp}(s^c)=D^c supp ( s c ) = D c
z : = A P c + ( b − A D c x D c p f ) . z := A_{P^c}^{+}(b-A_{D^c}x^{\mathrm{pf}}_{D^c}). z := A P c + ( b − A D c x D c pf ) . Let us pick α ≥ 0 \alpha\geq0 α ≥ 0 z + α x P c c > 0 z+\alpha x^c_{P^c}>0 z + α x P c c > 0
x p f : = ( x D c p f , x P c p f ) : = ( x D c p f , z ) + α x c , x^{\mathrm{pf}} := (x^{\mathrm{pf}}_{D^c},x^{\mathrm{pf}}_{P^c}) := (x^{\mathrm{pf}}_{D^c},z)+\alpha x^c, x pf := ( x D c pf , x P c pf ) := ( x D c pf , z ) + α x c , y ~ : = Π y p f , and s ~ : = − A T y ~ , \tilde{y}:=\Pi y^{\mathrm{pf}}, \qquad\text{and} \qquad\tilde{s}:=-A^\mathsf{T}\tilde{y}, y ~ := Π y pf , and s ~ := − A T y ~ , u p f : = Π u ′ , and v p f : = A T u p f . u^{\mathrm{pf}}:=\Pi u', \qquad\text{and} \qquad v^{\mathrm{pf}}:=A^\mathsf{T}u^{\mathrm{pf}}. u pf := Π u ′ , and v pf := A T u pf . Then, s ~ ≥ 0 n \tilde{s}\geq0_n s ~ ≥ 0 n s ~ P c = 0 P c \tilde{s}_{P^c}=0_{P^c} s ~ P c = 0 P c
⟨ b , y ~ ⟩ > 0 \langle b,\tilde{y}\rangle>0 ⟨ b , y ~ ⟩ > 0
⟨ b , y ~ ⟩ = 0 \langle b,\tilde{y}\rangle=0 ⟨ b , y ~ ⟩ = 0 A x p f = b Ax^{\mathrm{pf}}=b A x pf = b x p f ≥ 0 x^{\mathrm{pf}}\geq0 x pf ≥ 0
In the latter case, supp ( x p f ) ∪ supp ( s ~ ) = [ n ] \operatorname{supp}(x^{\mathrm{pf}})\cup\operatorname{supp}(\tilde{s})=[n] supp ( x pf ) ∪ supp ( s ~ ) = [ n ] v P c p f = 0 P c v^{\mathrm{pf}}_{P^c}=0_{P^c} v P c pf = 0 P c P ′ : = supp ( x D c p f ) P':=\operatorname{supp}(x^{\mathrm{pf}}_{D^c}) P ′ := supp ( x D c pf )
∥ x P ′ p f ∘ v P ′ p f − 1 P ′ ∥ 2 ≤ β f . \left\|x^{\mathrm{pf}}_{P'}\circ v^{\mathrm{pf}}_{P'}-1_{P'}\right\|_2\leq\beta^{\mathrm{f}}. x P ′ pf ∘ v P ′ pf − 1 P ′ 2 ≤ β f . Proof. The argument parallels that of Lemma 3.5, with the simplification that the cost on x D c x_{D^c} x D c (29) is 0 D c 0_{D^c} 0 D c
Since the cost of x D c x_{D^c} x D c (29) is 0 D c 0_{D^c} 0 D c H T y p f + s D c p f = 0 D c H^\mathsf{T}y^{\mathrm{pf}}+s^{\mathrm{pf}}_{D^c}=0_{D^c} H T y pf + s D c pf = 0 D c H = Π A D c H=\Pi A_{D^c} H = Π A D c Π \Pi Π ker ( A P c T ) \ker(A^\mathsf{T}_{P^c}) ker ( A P c T ) Π T = Π \Pi^\mathsf{T}=\Pi Π T = Π
A D c T y ~ = A D c T Π y p f = H T y p f = − s D c p f , A^\mathsf{T}_{D^c}\tilde{y}=A^\mathsf{T}_{D^c}\Pi y^{\mathrm{pf}}=H^\mathsf{T}y^{\mathrm{pf}}=-s^{\mathrm{pf}}_{D^c}, A D c T y ~ = A D c T Π y pf = H T y pf = − s D c pf , that is, s ~ D c = s D c p f ≥ 0 D c \tilde{s}_{D^c}=s^{\mathrm{pf}}_{D^c}\geq0_{D^c} s ~ D c = s D c pf ≥ 0 D c A P c T Π = ( Π A P c ) T = 0 A^\mathsf{T}_{P^c}\Pi=(\Pi A_{P^c})^\mathsf{T}=0 A P c T Π = ( Π A P c ) T = 0 s ~ P c = − A P c T Π y p f = 0 P c \tilde{s}_{P^c}=-A^\mathsf{T}_{P^c}\Pi y^{\mathrm{pf}}=0_{P^c} s ~ P c = − A P c T Π y pf = 0 P c s ~ ≥ 0 n \tilde{s}\geq0_n s ~ ≥ 0 n s ~ P c = 0 P c \tilde{s}_{P^c}=0_{P^c} s ~ P c = 0 P c
For the dichotomy, note that ⟨ b , y ~ ⟩ = ⟨ b , Π y p f ⟩ = ⟨ Π b , y p f ⟩ = ⟨ h , y p f ⟩ \langle b,\tilde{y}\rangle=\langle b,\Pi y^{\mathrm{pf}}\rangle=\langle\Pi b,y^{\mathrm{pf}}\rangle=\langle h,y^{\mathrm{pf}}\rangle ⟨ b , y ~ ⟩ = ⟨ b , Π y pf ⟩ = ⟨ Π b , y pf ⟩ = ⟨ h , y pf ⟩ (29) , which is nonnegative since the primal minimizes ⟨ 1 D c , x D c ⟩ ≥ 0 \langle1_{D^c},x_{D^c}\rangle\geq0 ⟨ 1 D c , x D c ⟩ ≥ 0 ⟨ b , y ~ ⟩ > 0 \langle b,\tilde{y}\rangle>0 ⟨ b , y ~ ⟩ > 0 ( y ~ , s ~ ) (\tilde{y},\tilde{s}) ( y ~ , s ~ ) A x = b Ax=b A x = b x ≥ 0 n x\geq0_n x ≥ 0 n ( A , b ) (A,b) ( A , b )
In the remaining case ⟨ b , y ~ ⟩ = ⟨ h , y p f ⟩ = 0 \langle b,\tilde{y}\rangle=\langle h,y^{\mathrm{pf}}\rangle=0 ⟨ b , y ~ ⟩ = ⟨ h , y pf ⟩ = 0 0 0 0 x ˉ D c p f = 0 D c \bar{x}^{\mathrm{pf}}_{D^c}=0_{D^c} x ˉ D c pf = 0 D c H x D c p f = h Hx^{\mathrm{pf}}_{D^c}=h H x D c pf = h A D c x D c p f + A P c z = b A_{D^c}x^{\mathrm{pf}}_{D^c}+A_{P^c}z=b A D c x D c pf + A P c z = b A x c = 0 m Ax^c=0_m A x c = 0 m A x p f = b Ax^{\mathrm{pf}}=b A x pf = b x p f ≥ 0 n x^{\mathrm{pf}}\geq0_n x pf ≥ 0 n x D c p f ≥ 0 D c x^{\mathrm{pf}}_{D^c}\geq0_{D^c} x D c pf ≥ 0 D c supp ( x c ) = P c \operatorname{supp}(x^c)=P^c supp ( x c ) = P c α \alpha α x P c p f = z + α x P c c > 0 P c x^{\mathrm{pf}}_{P^c}=z+\alpha x^c_{P^c}>0_{P^c} x P c pf = z + α x P c c > 0 P c supp ( x p f ) ∪ supp ( s ~ ) = [ n ] \operatorname{supp}(x^{\mathrm{pf}})\cup\operatorname{supp}(\tilde{s})=[n] supp ( x pf ) ∪ supp ( s ~ ) = [ n ] (29) is solved to strictly complementary optima, so supp ( x D c p f ) \operatorname{supp}(x^{\mathrm{pf}}_{D^c}) supp ( x D c pf ) supp ( s D c p f ) \operatorname{supp}(s^{\mathrm{pf}}_{D^c}) supp ( s D c pf ) D c D^c D c s ~ D c = s D c p f \tilde{s}_{D^c}=s^{\mathrm{pf}}_{D^c} s ~ D c = s D c pf supp ( x p f ) = P c ∪ supp ( x D c p f ) \operatorname{supp}(x^{\mathrm{pf}})=P^c\cup\operatorname{supp}(x^{\mathrm{pf}}_{D^c}) supp ( x pf ) = P c ∪ supp ( x D c pf )
It remains to verify the analytic center certificate. Recall that u p f = Π u ′ u^{\mathrm{pf}}=\Pi u' u pf = Π u ′ v p f = A T u p f v^{\mathrm{pf}}=A^\mathsf{T}u^{\mathrm{pf}} v pf = A T u pf A P c T Π = 0 A^\mathsf{T}_{P^c}\Pi=0 A P c T Π = 0 v P c p f = A P c T Π u ′ = 0 P c v^{\mathrm{pf}}_{P^c}=A^\mathsf{T}_{P^c}\Pi u'=0_{P^c} v P c pf = A P c T Π u ′ = 0 P c D c D^c D c Π T = Π \Pi^\mathsf{T}=\Pi Π T = Π
v D c p f = A D c T Π u ′ = ( Π A D c ) T u ′ = H T u ′ = v ′ . v^{\mathrm{pf}}_{D^c}=A^\mathsf{T}_{D^c}\Pi u'=(\Pi A_{D^c})^\mathsf{T}u'=H^\mathsf{T}u'=v'. v D c pf = A D c T Π u ′ = ( Π A D c ) T u ′ = H T u ′ = v ′ . By Theorem 1.6 applied to (29) , the certificate v ′ = H T u ′ v'=H^\mathsf{T}u' v ′ = H T u ′ ∥ x P ′ p f ∘ v P ′ ′ − 1 P ′ ∥ 2 ≤ β f \left\|x^{\mathrm{pf}}_{P'}\circ v'_{P'}-1_{P'}\right\|_2\leq\beta^{\mathrm{f}} x P ′ pf ∘ v P ′ ′ − 1 P ′ 2 ≤ β f P ′ = supp ( x D c p f ) P'=\operatorname{supp}(x^{\mathrm{pf}}_{D^c}) P ′ = supp ( x D c pf ) v P ′ p f = v P ′ ′ v^{\mathrm{pf}}_{P'}=v'_{P'} v P ′ pf = v P ′ ′
∥ x P ′ p f ∘ v P ′ p f − 1 P ′ ∥ 2 ≤ β f , \left\|x^{\mathrm{pf}}_{P'}\circ v^{\mathrm{pf}}_{P'}-1_{P'}\right\|_2\leq\beta^{\mathrm{f}}, x P ′ pf ∘ v P ′ pf − 1 P ′ 2 ≤ β f , as required. □
Stage III: Optimization
We start by calling PrimalMaxSA ( A , b ) \operatorname{PrimalMaxSA}(A,b) PrimalMaxSA ( A , b ) DualMaxSA ( A , c ) \operatorname{DualMaxSA}(A,c) DualMaxSA ( A , c ) x p f x^{\mathrm{pf}} x pf PrimalMaxSA ( A , b ) \operatorname{PrimalMaxSA}(A,b) PrimalMaxSA ( A , b ) P p f = supp ( x p f ) P^{\mathrm{pf}}=\operatorname{supp}(x^{\mathrm{pf}}) P pf = supp ( x pf ) ( y ~ , s ~ ) (\tilde{y},\tilde{s}) ( y ~ , s ~ ) ( y d f , s d f ) (y^{\mathrm{df}},s^{\mathrm{df}}) ( y df , s df ) DualMaxSA ( A , c ) \operatorname{DualMaxSA}(A,c) DualMaxSA ( A , c ) D d f = supp ( s d f ) D^{\mathrm{df}}=\operatorname{supp}(s^{\mathrm{df}}) D df = supp ( s df ) x ~ \tilde{x} x ~ ( u p f , v p f ) (u^{\mathrm{pf}},v^{\mathrm{pf}}) ( u pf , v pf ) w d f w^{\mathrm{df}} w df (25) and (26) . We define the index sets
R : = P p f ∩ D d f , I : = P p f ∖ D d f , J : = D d f ∖ P p f , P ′ : = P p f ∩ D c , D ′ : = D d f ∩ P c . R := P^{\mathrm{pf}} \cap D^{\mathrm{df}}, \qquad I := P^{\mathrm{pf}} \setminus D^{\mathrm{df}}, \qquad J := D^{\mathrm{df}} \setminus P^{\mathrm{pf}}, \qquad P' := P^{\mathrm{pf}} \cap D^{c}, \qquad D' := D^{\mathrm{df}} \cap P^{c}. R := P pf ∩ D df , I := P pf ∖ D df , J := D df ∖ P pf , P ′ := P pf ∩ D c , D ′ := D df ∩ P c . Note that P ′ ∩ D ′ = ∅ P' \cap D' = \varnothing P ′ ∩ D ′ = ∅
R = P ′ ∪ D ′ . R = P' \cup D'. R = P ′ ∪ D ′ . We may assume that R ≠ ∅ R \ne\varnothing R = ∅ x p f x^{\mathrm{pf}} x pf ( y d f , s d f ) (y^{\mathrm{df}},s^{\mathrm{df}}) ( y df , s df ) LP ( A , b , c ) \operatorname{LP}(A,b,c) LP ( A , b , c )
Similarly to § 3.2 , we remove all coordinates in J J J I I I
Π : = I m − A I A I T , H : = Π A R , and h : = Π b , \Pi:= I_m - A_I A_I^{\mathsf{T}}, \qquad H := \Pi A_R, \qquad\text{and} \qquad h := \Pi b, Π := I m − A I A I T , H := Π A R , and h := Π b , with the convention that Π : = I m \Pi:= I_m Π := I m I = ∅ I=\varnothing I = ∅
min ⟨ s R d f , x R ⟩ max ⟨ h , y ⟩ H x R = h H T y + s R = s R d f x R ≥ 0 R , s R ≥ 0 R . (31) \begin{aligned}
\min\ &\langle s_R^{\mathrm{df}},x_R\rangle&& \max\ \langle h,y\rangle\\
&Hx_R=h && H^{\mathsf{T}}y+s_R=s_R^{\mathrm{df}}\\
&x_R\ge0_R, && s_R\ge0_R. \tag*{(31)}
\end{aligned} min ⟨ s R df , x R ⟩ H x R = h x R ≥ 0 R , max ⟨ h , y ⟩ H T y + s R = s R df s R ≥ 0 R . (31) We can initialize this system using the outputs of PrimalMaxSA ( A , b ) \operatorname{PrimalMaxSA}(A,b) PrimalMaxSA ( A , b ) DualMaxSA ( A , c ) \operatorname{DualMaxSA}(A,c) DualMaxSA ( A , c )
μ : = ∥ x R p f ∘ s R d f ∥ 1 β f , \mu:= \frac{\lVert x_R^{\mathrm{pf}}\circ s_R^{\mathrm{df}}\rVert_1}{\beta^{\mathrm{f}}}, μ := β f ∥ x R pf ∘ s R df ∥ 1 , x R : = x R p f + μ ( 0 P ′ , w D ′ d f ) x_R := x_R^{\mathrm{pf}}+\mu(0_{P'},w_{D'}^{\mathrm{df}}) x R := x R pf + μ ( 0 P ′ , w D ′ df ) and
y : = − μ u p f , and s R : = s R d f − H T y = s R d f + μ ( v P ′ p f , v D ′ p f ) = s R d f + μ ( v P ′ p f , 0 D ′ ) . y:=-\mu u^{\mathrm{pf}},\qquad\text{and}\qquad s_R:=s_R^{\mathrm{df}}-H^{\mathsf{T}}y=s_R^{\mathrm{df}}+\mu(v_{P'}^{\mathrm{pf}},v_{D'}^{\mathrm{pf}})=s_R^{\mathrm{df}}+\mu(v_{P'}^{\mathrm{pf}},0_{D'}). y := − μ u pf , and s R := s R df − H T y = s R df + μ ( v P ′ pf , v D ′ pf ) = s R df + μ ( v P ′ pf , 0 D ′ ) . Here the first equality uses H T u p f = A R T u p f = ( v P ′ p f , v D ′ p f ) H^{\mathsf{T}}u^{\mathrm{pf}}=A_R^{\mathsf{T}}u^{\mathrm{pf}}=(v_{P'}^{\mathrm{pf}},v_{D'}^{\mathrm{pf}}) H T u pf = A R T u pf = ( v P ′ pf , v D ′ pf ) A I T u p f = v I p f = 0 A_I^{\mathsf{T}}u^{\mathrm{pf}}=v_I^{\mathrm{pf}}=0 A I T u pf = v I pf = 0 I ⊆ P c I\subseteq P^c I ⊆ P c v P c p f = 0 v_{P^c}^{\mathrm{pf}}=0 v P c pf = 0 Π u p f = u p f \Pi u^{\mathrm{pf}}=u^{\mathrm{pf}} Π u pf = u pf H T u p f = A R T Π u p f = A R T u p f H^{\mathsf{T}}u^{\mathrm{pf}}=A_R^{\mathsf{T}}\Pi u^{\mathrm{pf}}=A_R^{\mathsf{T}}u^{\mathrm{pf}} H T u pf = A R T Π u pf = A R T u pf v D ′ p f = 0 D ′ v_{D'}^{\mathrm{pf}}=0_{D'} v D ′ pf = 0 D ′ D ′ ⊆ P c D'\subseteq P^c D ′ ⊆ P c x R x_R x R Π \Pi Π A x p f = b Ax^{\mathrm{pf}}=b A x pf = b H x R p f = h Hx_R^{\mathrm{pf}}=h H x R pf = h x J p f = 0 J x_J^{\mathrm{pf}}=0_J x J pf = 0 J Π \Pi Π A w d f = 0 m Aw^{\mathrm{df}}=0_m A w df = 0 m H D ′ w D ′ d f = 0 m H_{D'}w_{D'}^{\mathrm{df}}=0_m H D ′ w D ′ df = 0 m w D c d f = 0 D c w_{D^c}^{\mathrm{df}}=0_{D^c} w D c df = 0 D c x R p f > 0 R x_R^{\mathrm{pf}}>0_R x R pf > 0 R w D ′ d f > 0 D ′ w_{D'}^{\mathrm{df}}>0_{D'} w D ′ df > 0 D ′ (26) .
These solutions satisfy
∥ x R ∘ s R μ − 1 R ∥ 2 = ∥ x R p f ∘ s R d f μ + ( x P ′ p f ∘ v P ′ p f , 0 D ′ ) − ( 1 P ′ , 0 D ′ ) + ( ( 0 P ′ , w D ′ d f ) ∘ s R d f ) − ( 0 P ′ , 1 D ′ ) ∥ 2 ≤ ∥ x R p f ∘ s R d f μ ∥ 2 + ∥ ( x P ′ p f ∘ v P ′ p f ) − 1 P ′ ∥ 2 + ∥ ( w D ′ d f ∘ s D ′ d f ) − 1 D ′ ∥ 2 ≤ 3 β f . (32) \begin{aligned}
\left\lVert\frac{x_R\circ s_R}{\mu}-1_R\right\rVert_2
=
\left\lVert
\frac{x_R^{\mathrm{pf}}\circ s_R^{\mathrm{df}}}{\mu}
+
(x_{P'}^{\mathrm{pf}}\circ v_{P'}^{\mathrm{pf}},0_{D'})
-
(1_{P'},0_{D'})
+
((0_{P'},w_{D'}^{\mathrm{df}})\circ s_R^{\mathrm{df}})
-
(0_{P'},1_{D'})
\right\rVert_2 \\
&\le
\left\lVert\frac{x_R^{\mathrm{pf}}\circ s_R^{\mathrm{df}}}{\mu}\right\rVert_2
+
\left\lVert(x_{P'}^{\mathrm{pf}}\circ v_{P'}^{\mathrm{pf}})-1_{P'}\right\rVert_2
+
\left\lVert(w_{D'}^{\mathrm{df}}\circ s_{D'}^{\mathrm{df}})-1_{D'}\right\rVert_2
\le3\beta^{\mathrm{f}}. \tag*{(32)}
\end{aligned} μ x R ∘ s R − 1 R 2 = μ x R pf ∘ s R df + ( x P ′ pf ∘ v P ′ pf , 0 D ′ ) − ( 1 P ′ , 0 D ′ ) + (( 0 P ′ , w D ′ df ) ∘ s R df ) − ( 0 P ′ , 1 D ′ ) 2 ≤ μ x R pf ∘ s R df 2 + ( x P ′ pf ∘ v P ′ pf ) − 1 P ′ 2 + ( w D ′ df ∘ s D ′ df ) − 1 D ′ 2 ≤ 3 β f . (32) where the last inequality uses the definition of μ \mu μ (25) and (26) . From Lemma 2.6, and the choice of β f \beta^{\mathrm{f}} β f β o \beta^{\mathrm{o}} β o
( x R , s R ) ∈ N 2 ( 3 β f 1 − 3 β f ) ⊆ N 2 ( β o ) . (33) (x_R,s_R)\in\mathcal{N}^{2}\left(\frac{3\beta^{\mathrm{f}}}{1-3\beta^{\mathrm{f}}}\right)\subseteq\mathcal{N}^{2}(\beta^{\mathrm{o}}). \tag*{(33)} ( x R , s R ) ∈ N 2 ( 1 − 3 β f 3 β f ) ⊆ N 2 ( β o ) . (33) We solve the system (31) with the IPM solver as in Theorem 1.6, for neighborhood parameter β = β o \beta=\beta^{\mathrm{o}} β = β o LP ( A , b , c ) \operatorname{LP}(A,b,c) LP ( A , b , c )
Lemma 4.4. Let x R ∘ x_R^{\circ} x R ∘ ( y ∘ , s R ∘ ) (y^{\circ},s_R^{\circ}) ( y ∘ , s R ∘ ) (31) obtained via Theorem 1.6 . Let ( y ~ , s ~ ) (\tilde{y},\tilde{s}) ( y ~ , s ~ ) PrimalMaxSA( A , b ) (A,b) ( A , b ) x ~ \tilde{x} x ~ DualMaxSA( A , c ) (A,c) ( A , c )
s ~ = − A T y ~ , ⟨ b , y ~ ⟩ = 0 , s ~ ≥ 0 n , supp ( s ~ ) = J , and \tilde{s}=-A^{\mathsf{T}}\tilde{y},\quad\langle b,\tilde{y}\rangle=0,\quad\tilde{s}\geq0_n,\quad\operatorname{supp}(\tilde{s})=J,\quad\text{and} s ~ = − A T y ~ , ⟨ b , y ~ ⟩ = 0 , s ~ ≥ 0 n , supp ( s ~ ) = J , and A x ~ = 0 m , ⟨ c , x ~ ⟩ = 0 , x ~ ≥ 0 , supp ( x ~ ) = I . A\tilde{x}=0_m,\quad\langle c,\tilde{x}\rangle=0,\quad\tilde{x}\geq0,\quad\operatorname{supp}(\tilde{x})=I. A x ~ = 0 m , ⟨ c , x ~ ⟩ = 0 , x ~ ≥ 0 , supp ( x ~ ) = I . Let us define
z : = A I + ( b − A R x R ∘ ) , z:=A_I^{+}(b-A_Rx_R^{\circ}), z := A I + ( b − A R x R ∘ ) , and pick α , λ ≥ 0 \alpha,\lambda\geq0 α , λ ≥ 0 z + α x ~ I > 0 z+\alpha\tilde{x}_I>0 z + α x ~ I > 0 A J T ( Π y ∘ + y d f + λ y ~ ) < c J A_J^{\mathsf{T}}(\Pi y^{\circ}+y^{\mathrm{df}}+\lambda\tilde{y})<c_J A J T ( Π y ∘ + y df + λ y ~ ) < c J
x ⋆ = ( x R ⋆ , x I ⋆ , x J ⋆ ) : = ( x R ∘ , z , 0 J ) + α x ~ , x^{\star}=(x_R^{\star},x_I^{\star},x_J^{\star}):=(x_R^{\circ},z,0_J)+\alpha\tilde{x}, x ⋆ = ( x R ⋆ , x I ⋆ , x J ⋆ ) := ( x R ∘ , z , 0 J ) + α x ~ , y ⋆ : = Π y ∘ + y d f + λ y ~ , and s ⋆ : = c − A T y ⋆ . y^{\star}:=\Pi y^{\circ}+y^{\mathrm{df}}+\lambda\tilde{y},\quad\text{and}\quad s^{\star}:=c-A^{\mathsf{T}}y^{\star}. y ⋆ := Π y ∘ + y df + λ y ~ , and s ⋆ := c − A T y ⋆ . Then, x ⋆ x^{\star} x ⋆ ( y ⋆ , s ⋆ ) (y^{\star},s^{\star}) ( y ⋆ , s ⋆ ) LP ( A , b , c ) \operatorname{LP}(A,b,c) LP ( A , b , c )
Proof. Since (31) is precisely LP ( H , h , s R d f ) \operatorname{LP}(H,h,s_R^{\mathrm{df}}) LP ( H , h , s R df ) x R ∘ x_R^{\circ} x R ∘ ( y ∘ , s R ∘ ) (y^{\circ},s_R^{\circ}) ( y ∘ , s R ∘ ) x ⋆ x^{\star} x ⋆ ( y ⋆ , s ⋆ ) (y^{\star},s^{\star}) ( y ⋆ , s ⋆ ) LP ( A , b , c ) \operatorname{LP}(A,b,c) LP ( A , b , c ) supp ( x ~ ) = I \operatorname{supp}(\tilde{x})=I supp ( x ~ ) = I supp ( s ~ ) = J \operatorname{supp}(\tilde{s})=J supp ( s ~ ) = J α , λ ≥ 0 \alpha,\lambda\geq0 α , λ ≥ 0
On the primal side, Fact 2.3 and A x ~ = 0 m A\tilde{x}=0_m A x ~ = 0 m A x ⋆ = b Ax^{\star}=b A x ⋆ = b x ⋆ ≥ 0 n x^{\star}\geq0_n x ⋆ ≥ 0 n x R ∘ ≥ 0 R x_R^{\circ}\geq0_R x R ∘ ≥ 0 R α \alpha α x ⋆ x^{\star} x ⋆
Turning to the dual side, on the coordinates in R R R
H T y ∘ + s R ∘ = s R d f , A R T y d f + s R d f = c R , and A R T y ~ = 0 R . H^{\mathsf{T}}y^{\circ}+s_R^{\circ}=s_R^{\mathrm{df}},\quad A_R^{\mathsf{T}}y^{\mathrm{df}}+s_R^{\mathrm{df}}=c_R,\quad\text{and}\quad A_R^{\mathsf{T}}\tilde{y}=0_R. H T y ∘ + s R ∘ = s R df , A R T y df + s R df = c R , and A R T y ~ = 0 R . Since H T y ∘ = A R T Π y ∘ H^{\mathsf{T}}y^{\circ}=A_R^{\mathsf{T}}\Pi y^{\circ} H T y ∘ = A R T Π y ∘ s R ⋆ = c R − A R T y ⋆ = s R d f + ( s R ∘ − s R d f ) − λ A R T y ~ = s R ∘ ≥ 0 R s_R^{\star}=c_R-A_R^{\mathsf{T}}y^{\star}=s_R^{\mathrm{df}}+(s_R^{\circ}-s_R^{\mathrm{df}})-\lambda A_R^{\mathsf{T}}\tilde{y}=s_R^{\circ}\geq0_R s R ⋆ = c R − A R T y ⋆ = s R df + ( s R ∘ − s R df ) − λ A R T y ~ = s R ∘ ≥ 0 R I I I
A I T Π y ∘ = 0 I , A I T y d f = c I , and A I T y ~ = 0 I , A_I^{\mathsf{T}}\Pi y^{\circ}=0_I,\quad A_I^{\mathsf{T}}y^{\mathrm{df}}=c_I,\quad\text{and}\quad A_I^{\mathsf{T}}\tilde{y}=0_I, A I T Π y ∘ = 0 I , A I T y df = c I , and A I T y ~ = 0 I , where the first equation holds because Π \Pi Π ker ( A I T ) \ker(A_I^{\mathsf{T}}) ker ( A I T ) s I ⋆ = 0 I s_I^{\star}=0_I s I ⋆ = 0 I
Finally, s J ⋆ > 0 J s_J^{\star}>0_J s J ⋆ > 0 J λ \lambda λ s J ⋆ = c J − A J T y ⋆ = c J − A J T ( Π y ∘ + y d f ) + λ s ~ J s_J^{\star}=c_J-A_J^{\mathsf{T}}y^{\star}=c_J-A_J^{\mathsf{T}}(\Pi y^{\circ}+y^{\mathrm{df}})+\lambda\tilde{s}_J s J ⋆ = c J − A J T y ⋆ = c J − A J T ( Π y ∘ + y df ) + λ s ~ J s ~ J > 0 J \tilde{s}_J>0_J s ~ J > 0 J supp ( s ~ ) = J \operatorname{supp}(\tilde{s})=J supp ( s ~ ) = J λ \lambda λ s J ⋆ > 0 J s_J^{\star}>0_J s J ⋆ > 0 J
To show optimality, we show complementarity. We note that supp ( x ⋆ ) = supp ( x R ∘ ) ∪ I \operatorname{supp}(x^{\star})=\operatorname{supp}(x_R^{\circ})\cup I supp ( x ⋆ ) = supp ( x R ∘ ) ∪ I supp ( s ⋆ ) = supp ( s R ∘ ) ∪ J \operatorname{supp}(s^{\star})=\operatorname{supp}(s_R^{\circ})\cup J supp ( s ⋆ ) = supp ( s R ∘ ) ∪ J x R ∘ x_R^{\circ} x R ∘ s R ∘ s_R^{\circ} s R ∘ (31) . Since (31) is solved to strictly complementary optima, supp ( x R ∘ ) \operatorname{supp}(x_R^{\circ}) supp ( x R ∘ ) supp ( s R ∘ ) \operatorname{supp}(s_R^{\circ}) supp ( s R ∘ ) R R R I ⊆ supp ( x ⋆ ) I\subseteq\operatorname{supp}(x^{\star}) I ⊆ supp ( x ⋆ ) J ⊆ supp ( s ⋆ ) J\subseteq\operatorname{supp}(s^{\star}) J ⊆ supp ( s ⋆ ) supp ( x ⋆ ) ∪ supp ( s ⋆ ) = [ n ] \operatorname{supp}(x^{\star})\cup\operatorname{supp}(s^{\star})=[n] supp ( x ⋆ ) ∪ supp ( s ⋆ ) = [ n ] x ⋆ x^{\star} x ⋆ ( y ⋆ , s ⋆ ) (y^{\star},s^{\star}) ( y ⋆ , s ⋆ )
Proof of Theorem 1.7
We are ready to prove Theorem 1.7, restated here for convenience. The remaining ingredient, Theorem 1.8, will be proved in § 5 .
Theorem 1.7. Let A ∈ R m × n A\in\mathbb{R}^{m\times n} A ∈ R m × n b ∈ R m b\in\mathbb{R}^{m} b ∈ R m c ∈ R n c\in\mathbb{R}^{n} c ∈ R n rank ( A ) = m \operatorname{rank}(A)=m rank ( A ) = m LP ( A , b , c ) \operatorname{LP}(A,b,c) LP ( A , b , c )
O ( poly ( n ) ⋅ min η ∈ ( 0 , 1 ] Γ η log ( n η ) ) , O\left(\operatorname{poly}(n)\cdot\min_{\eta\in(0,1]}\Gamma_{\eta}\log\left(\frac{n}{\eta}\right)\right), O ( poly ( n ) ⋅ η ∈ ( 0 , 1 ] min Γ η log ( η n ) ) , where Γ η \Gamma_{\eta} Γ η SLC η ( x i m ) \operatorname{SLC}_{\eta}(x_i^{\mathrm{m}}) SLC η ( x i m )
min ⟨ c , x ⟩ , A x = b , x ≥ 0 n (S-Opt) \min\langle c,x\rangle,\quad Ax=b,\quad x\geq0_n \qquad\text{(S-Opt)} min ⟨ c , x ⟩ , A x = b , x ≥ 0 n (S-Opt) min ⟨ 1 n , x ˉ ⟩ , A x − A x ˉ = 0 m , x + x ^ = 1 n , x , x ˉ , x ^ ≥ 0 n (S-Conic) \min\langle1_n,\bar{x}\rangle,\quad Ax-A\bar{x}=0_m,\quad x+\hat{x}=1_n,\quad x,\bar{x},\hat{x}\geq0_n \qquad\text{(S-Conic)} min ⟨ 1 n , x ˉ ⟩ , A x − A x ˉ = 0 m , x + x ^ = 1 n , x , x ˉ , x ^ ≥ 0 n (S-Conic) min ⟨ c , x ⟩ , A x = 0 m , x + x ^ = 1 n , x , x ^ ≥ 0 n (S-Dual) \min\langle c,x\rangle,\quad Ax=0_m,\quad x+\hat{x}=1_n,\quad x,\hat{x}\geq0_n \qquad\text{(S-Dual)} min ⟨ c , x ⟩ , A x = 0 m , x + x ^ = 1 n , x , x ^ ≥ 0 n (S-Dual) min ⟨ 1 n , x ˉ ⟩ , A x − A x ˉ = b , x , x ˉ ≥ 0 n (S-Primal) \min\langle1_n,\bar{x}\rangle,\quad Ax-A\bar{x}=b,\quad x,\bar{x}\geq0_n \qquad\text{(S-Primal)} min ⟨ 1 n , x ˉ ⟩ , A x − A x ˉ = b , x , x ˉ ≥ 0 n (S-Primal) Note that (S-Conic), (S-Dual), and (S-Primal) are always primal and dual feasible. For rational input ( A , b , c ) (A,b,c) ( A , b , c )
Proof. We use the three-stage initialization as described in this section. Namely, we call the IPM in Theorem 1.6 on the four LPs (16) , (27) , (29) , and (31) . Note that the matrices A P c A_{P^c} A P c H H H A A A Theorem 1.6 . This is because any dual optimal solution of the resulting system can be trivially lifted to a dual optimal solution of the original system by filling y y y
As already noted in § 3.3 , the straight line complexity of the variables in (16) are the same as those for (S-Conic). Next, consider the system (27) solved for dual feasibility. To obtain this system from (S-Dual), we first replace the right-hand side vector 1 n \mathbf{1}_n 1 n 2 ⋅ 1 n 2 \cdot\mathbf{1}_n 2 ⋅ 1 n ( x , x ^ ) (x,\hat{x}) ( x , x ^ ) x D c = 0 D c x_{D^c} = 0_{D^c} x D c = 0 D c x D c x_{D^c} x D c ( x P c , x ^ ) (x_{P^c},\hat{x}) ( x P c , x ^ ) x ^ D c \hat{x}_{D^c} x ^ D c ( x P c , x ^ P c ) (x_{P^c},\hat{x}_{P^c}) ( x P c , x ^ P c ) (27) without affecting the max central path of ( x P c , x ^ P c ) (x_{P^c},\hat{x}_{P^c}) ( x P c , x ^ P c )
Similarly, we show the system (29) solved for primal feasibility is obtained from (S-Primal). Since every dual feasible solution satisfies s P c = 0 P c s_{P^c} = 0_{P^c} s P c = 0 P c x P c x_{P^c} x P c ( x D c , x ˉ ) (x_{D^c},\bar{x}) ( x D c , x ˉ ) Π ( − A P c ) = 0 \Pi(-A_{P^c}) = 0 Π ( − A P c ) = 0 x ˉ P c \bar{x}_{P^c} x ˉ P c (29) without affecting the max central path of ( x D c , x ˉ D c ) (x_{D^c},\bar{x}_{D^c}) ( x D c , x ˉ D c )
Finally, we show how the system (31) solved in the optimization stage is obtained from (S-Opt). Recall that I = [ n ] ∖ D d f I = [n] \setminus D^{\mathrm{df}} I = [ n ] ∖ D df J = [ n ] ∖ P p f J = [n] \setminus P^{\mathrm{pf}} J = [ n ] ∖ P pf x J = 0 J x_J = 0_J x J = 0 J s I = 0 I s_I = 0_I s I = 0 I (31) have the same straight line complexities as in LP ( A , b , c ) \operatorname{LP}(A,b,c) LP ( A , b , c )
We now give a PSPACE implementation of the SLLS IPM. We use that the IPM is stepwise strongly polynomial, stays in the N 2 ( β ) \mathcal{N}^{2}(\beta) N 2 ( β ) 1 − β / ( 2 n ) 1 - \beta/(2\sqrt{n}) 1 − β / ( 2 n ) Theorem 1.6 . For each iterate ( x , s ) (x,s) ( x , s ) ( x ~ , s ~ ) ∈ N 2 ( 3 β ) (\tilde{x},\tilde{s}) \in\mathcal{N}^{2}(3\beta) ( x ~ , s ~ ) ∈ N 2 ( 3 β ) μ ~ ( x ~ , s ~ ) ≤ ( 1 + β / ( 4 n 2 ) ) μ ( x , s ) \tilde{\mu}(\tilde{x},\tilde{s}) \le(1 + \beta/(4n^{2}))\mu(x,s) μ ~ ( x ~ , s ~ ) ≤ ( 1 + β / ( 4 n 2 )) μ ( x , s ) size ( x ~ , s ~ ) \operatorname{size}(\tilde{x},\tilde{s}) size ( x ~ , s ~ )
( 1 − β 2 n ) ( 1 + β 4 n 2 ) ≤ 1 − β 2 n + β 4 n 2 ≤ 1 − β 4 n . (34) \left(1-\frac{\beta}{2\sqrt{n}}\right)\left(1+\frac{\beta}{4n^{2}}\right) \le1-\frac{\beta}{2\sqrt{n}}+\frac{\beta}{4n^{2}} \le1-\frac{\beta}{4\sqrt{n}}. \tag*{(34)} ( 1 − 2 n β ) ( 1 + 4 n 2 β ) ≤ 1 − 2 n β + 4 n 2 β ≤ 1 − 4 n β . (34) After each such rounding, unless μ ~ ( x ~ , s ~ ) = 0 \tilde{\mu}(\tilde{x},\tilde{s}) = 0 μ ~ ( x ~ , s ~ ) = 0 N 2 ( β ) \mathcal{N}^{2}(\beta) N 2 ( β ) μ ~ ( x ~ , s ~ ) = 0 \tilde{\mu}(\tilde{x},\tilde{s}) = 0 μ ~ ( x ~ , s ~ ) = 0 ( x ~ , s ~ ) (\tilde{x},\tilde{s}) ( x ~ , s ~ ) ( x ~ , s ~ ) (\tilde{x},\tilde{s}) ( x ~ , s ~ ) ( v ~ , w ~ ) (\tilde{v},\tilde{w}) ( v ~ , w ~ ) Theorem 1.6 with 3 β 3\beta 3 β β \beta β Theorem 1.6 , with certificates satisfying the output conditions for β \beta β (23) , (24) , (25) and (26) hold with 3 β c 3\beta^{\mathrm{c}} 3 β c 3 β f 3\beta^{\mathrm{f}} 3 β f β c \beta^{\mathrm{c}} β c β f \beta^{\mathrm{f}} β f § 4 remains valid with these weaker bounds. The bounds ∥ x P c ∥ ∞ ≤ 2 \lVert x_{P^c}\rVert_{\infty} \le2 ∥ x P c ∥ ∞ ≤ 2 s c ≤ 1 n s^c \le\mathbf{1}_n s c ≤ 1 n (16) . In both initializations of Stage II, the bound 2 β c 2\beta^{\mathrm{c}} 2 β c 3 β c + β c = 4 β c 3\beta^{\mathrm{c}}+\beta^{\mathrm{c}} = 4\beta^{\mathrm{c}} 3 β c + β c = 4 β c (22) we have 4 β c / ( 1 − 4 β c ) ≤ β f 4\beta^{\mathrm{c}}/(1-4\beta^{\mathrm{c}}) \le\beta^{\mathrm{f}} 4 β c / ( 1 − 4 β c ) ≤ β f N 2 ( β f ) \mathcal{N}^{2}(\beta^{\mathrm{f}}) N 2 ( β f ) Theorem 1.6 through unchanged, so (25) holds with 3 β f 3\beta^{\mathrm{f}} 3 β f 3 β f 3\beta^{\mathrm{f}} 3 β f β f + 3 β f + 3 β f = 7 β f \beta^{\mathrm{f}}+3\beta^{\mathrm{f}}+3\beta^{\mathrm{f}} = 7\beta^{\mathrm{f}} β f + 3 β f + 3 β f = 7 β f 7 β f / ( 1 − 7 β f ) < 2 − 5 < β o 7\beta^{\mathrm{f}}/(1-7\beta^{\mathrm{f}}) < 2^{-5} < \beta^{\mathrm{o}} 7 β f / ( 1 − 7 β f ) < 2 − 5 < β o β o = 1 / 16 \beta^{\mathrm{o}} = 1/16 β o = 1/16
We still need to bound the encoding lengths across the four IPM calls. For a call on a system with data ( A ′ , b ′ , c ′ ) (A',b',c') ( A ′ , b ′ , c ′ ) poly ( n ) size ( A ′ , b ′ , c ′ ) \operatorname{poly}(n)\operatorname{size}(A',b',c') poly ( n ) size ( A ′ , b ′ , c ′ ) poly ( n ) size ( A ′ , b ′ , c ′ ) \operatorname{poly}(n)\operatorname{size}(A',b',c') poly ( n ) size ( A ′ , b ′ , c ′ ) ( A ′ , b ′ , c ′ ) (A',b',c') ( A ′ , b ′ , c ′ ) ( A , b , c ) (A,b,c) ( A , b , c ) § 4 outside the IPM calls are strongly polynomial in the Turing model. This includes the choice of shift scalars such as α \alpha α λ \lambda λ poly ( n ) size ( A , b , c ) \operatorname{poly}(n)\operatorname{size}(A,b,c) poly ( n ) size ( A , b , c )
What remains to be argued is that the SLLS IPM with rounding still satisfies the same iteration bound as Theorem 1.6 . Note that the rounding step pushes each iterate only slightly up the path, which intuitively should not matter.
To formally justify this, we will explain why the analysis used to prove Theorem 1.6 applies to the iterate sequence produced by the rounded algorithm. As it is relatively direct, we simply highlight which statements of the proof of Theorem 1.6 in [2 ] need to be verified for the rounded IPM.
At a high level, the authors of [2 ] decompose the central path into a number of η \eta η O ( n / β ln ( n / ( η β ) ) ) O(\sqrt{n}/\beta\ln(n/(\eta\beta))) O ( n / β ln ( n / ( η β ))) z 0 , … , z T ∈ N 2 ( β ) z^0,\ldots,z^T \in\mathcal{N}^2(\beta) z 0 , … , z T ∈ N 2 ( β ) [2 Lemma 7.5] , which states that for D : = C i t n / β ln ( n / ( η β ) ) D := C_{\mathrm{it}}\sqrt{n}/\beta\ln(n/(\eta\beta)) D := C it n / β ln ( n / ( η β )) C i t ≥ 1 C_{\mathrm{it}} \ge1 C it ≥ 1 z i , z j z^i,z^j z i , z j j ≥ D + i j \ge D+i j ≥ D + i z j z^j z j z i z^i z i z j z^j z j [2 Section 8.2] , it is shown that any sequence of iterates that satisfies the above bound also satisfies the SLC bound of Theorem 1.6 . Thus, it suffices to verify the lemma. Lemma 7.5 relies only on the following three properties: that the iterates stay in the N 2 ( β ) \mathcal{N}^2(\beta) N 2 ( β ) 1 − Ω ( 1 / n ) 1-\Omega(1/\sqrt{n}) 1 − Ω ( 1/ n ) (34) ), and the iterates return to N 2 ( β ) \mathcal{N}^2(\beta) N 2 ( β ) 2 C i t 2C_{\mathrm{it}} 2 C it
Alternatively, one could use the ℓ 2 \ell_2 ℓ 2 [17 ] . Theorem 1.3 in this paper asserts that, for any μ 0 ≥ μ 1 ≥ 0 \mu_0 \ge\mu_1 \ge0 μ 0 ≥ μ 1 ≥ 0 z 0 ∈ N 2 ( β ) z_0 \in\mathcal{N}^2(\beta) z 0 ∈ N 2 ( β ) μ ( z 0 ) ∈ [ μ 1 , μ 0 ] \mu(z_0) \in[\mu_1,\mu_0] μ ( z 0 ) ∈ [ μ 1 , μ 0 ] z 1 z_1 z 1 μ ( z 1 ) ≤ μ 1 \mu(z_1) \le\mu_1 μ ( z 1 ) ≤ μ 1 O ( SLC ( N 2 ( β ) , μ 1 , μ 0 ) ) O(\operatorname{SLC}(\mathcal{N}^2(\beta),\mu_1,\mu_0)) O ( SLC ( N 2 ( β ) , μ 1 , μ 0 ))
A strongly polynomial rounding procedure
In this section, we prove the main rounding statement Theorem 1.8 used for the PSPACE implementation in Theorem 1.7. Throughout, all numbers are encoded rationals in the sense of Definition 1.2 , and we use the conventions of § 1.1 for how to compute with them.
Addition and multiplication of encoded rationals are associative and commutative: for ( p i , q i ) ∈ Q ^ (p_i,q_i) \in\widehat{\mathbb{Q}} ( p i , q i ) ∈ Q i ∈ [ k ] i \in[k] i ∈ [ k ]
∑ i ( p i , q i ) = ( ∑ i = 1 k p i ∏ j = 1 j ≠ i k q j , ∏ i = 1 k q i ) , ∏ i = 1 k ( p i , q i ) = ( ∏ i = 1 k p i , ∏ i = 1 k q i ) , (35) \sum_i (p_i,q_i)=\left(\sum_{i=1}^{k}p_i\prod_{\substack{j=1\\j\ne i}}^{k}q_j,\prod_{i=1}^{k}q_i\right),\qquad\prod_{i=1}^{k}(p_i,q_i)=\left(\prod_{i=1}^{k}p_i,\prod_{i=1}^{k}q_i\right), \tag*{(35)} i ∑ ( p i , q i ) = i = 1 ∑ k p i j = 1 j = i ∏ k q j , i = 1 ∏ k q i , i = 1 ∏ k ( p i , q i ) = ( i = 1 ∏ k p i , i = 1 ∏ k q i ) , (35) independently of the order of the terms and of the bracketing. Hence, iterated sums and products are well-defined elements of Q ^ \widehat{\mathbb{Q}} Q
Distributivity, however, does not hold. For instance,
( ( 1 , 1 ) + ( 1 , 1 ) ) ⋅ ( 1 , 2 ) = ( 2 , 1 ) ⋅ ( 1 , 2 ) = ( 2 , 2 ) , ((1,1)+(1,1))\cdot(1,2)=(2,1)\cdot(1,2)=(2,2), (( 1 , 1 ) + ( 1 , 1 )) ⋅ ( 1 , 2 ) = ( 2 , 1 ) ⋅ ( 1 , 2 ) = ( 2 , 2 ) , whereas
( 1 , 1 ) ⋅ ( 1 , 2 ) + ( 1 , 1 ) ⋅ ( 1 , 2 ) = ( 1 , 2 ) + ( 1 , 2 ) = ( 4 , 4 ) , (1,1)\cdot(1,2)+(1,1)\cdot(1,2)=(1,2)+(1,2)=(4,4), ( 1 , 1 ) ⋅ ( 1 , 2 ) + ( 1 , 1 ) ⋅ ( 1 , 2 ) = ( 1 , 2 ) + ( 1 , 2 ) = ( 4 , 4 ) , two different representations of the same value 1. Thus, the size of a computed number depends on how it is computed, and our size bounds always refer to the specific computations performed by our algorithms.
We restate Theorem 1.8 for convenience.
Theorem 1.8. Assume A ∈ Q ^ m × n A \in\widehat{\mathbb{Q}}^{m\times n} A ∈ Q m × n b ∈ Q ^ m b \in\widehat{\mathbb{Q}}^m b ∈ Q m c ∈ Q ^ n c \in\widehat{\mathbb{Q}}^n c ∈ Q n rank ( A ) = m \operatorname{rank}(A)=m rank ( A ) = m β ∈ ( 0 , 1 / 16 ] \beta\in(0,1/16] β ∈ ( 0 , 1/16 ] ( x , s ) ∈ N 2 ( β ) ∩ Q 2 n (x,s) \in\mathcal{N}^2(\beta)\cap\mathbb{Q}^{2n} ( x , s ) ∈ N 2 ( β ) ∩ Q 2 n ( x ~ , s ~ ) ∈ N 2 ( 3 β ) ∩ Q 2 n (\widetilde{x},\widetilde{s})\in\mathcal{N}^2(3\beta)\cap\mathbb{Q}^{2n} ( x , s ) ∈ N 2 ( 3 β ) ∩ Q 2 n μ ‾ ( x ~ , s ~ ) ≤ ( 1 + β / ( 4 n 2 ) ) μ ‾ ( x , s ) \overline{\mu}(\widetilde{x},\widetilde{s})\le(1+\beta/(4n^2))\overline{\mu}(x,s) μ ( x , s ) ≤ ( 1 + β / ( 4 n 2 )) μ ( x , s ) size ( x ~ , s ~ ) = poly ( n ) size ( A , b , c ) \operatorname{size}(\widetilde{x},\widetilde{s})=\operatorname{poly}(n)\operatorname{size}(A,b,c) size ( x , s ) = poly ( n ) size ( A , b , c ) μ ‾ ( x , s ) = 0 \overline{\mu}(x,s)=0 μ ( x , s ) = 0 ( v ~ , w ~ ) ∈ Q 2 n (\widetilde{v},\widetilde{w})\in\mathbb{Q}^{2n} ( v , w ) ∈ Q 2 n size ( v ~ , w ~ ) = poly ( n ) size ( A , b , c ) \operatorname{size}(\widetilde{v},\widetilde{w})=\operatorname{poly}(n)\operatorname{size}(A,b,c) size ( v , w ) = poly ( n ) size ( A , b , c ) ( x ~ , s ~ ) (\widetilde{x},\widetilde{s}) ( x , s ) ( v ~ , w ~ ) (\widetilde{v},\widetilde{w}) ( v , w ) Theorem 1.6 with the choice of β \beta β
Theorem 1.8 is not specific to the SLLS IPM. As indicated in the introduction, the rounding step can be interleaved with the iterations of any log-barrier path-following IPM that satisfies the following two requirements. First, the method maintains rational iterates in the ℓ 2 \ell_2 ℓ 2 N 2 ( β ) \mathcal{N}^2(\beta) N 2 ( β ) β ≤ 1 / 16 \beta\le1/16 β ≤ 1/16 ( x , s ) ∈ Q 2 n (x,s)\in\mathbb{Q}^{2n} ( x , s ) ∈ Q 2 n poly ( n ) \operatorname{poly}(n) poly ( n ) poly ( n ) size ( A , b , c , x , s ) \operatorname{poly}(n)\operatorname{size}(A,b,c,x,s) poly ( n ) size ( A , b , c , x , s ) N ‾ 2 ( 3 β ) \overline{\mathcal{N}}^2(3\beta) N 2 ( 3 β ) ( 1 + β / ( 4 n 2 ) ) (1+\beta/(4n^2)) ( 1 + β / ( 4 n 2 )) poly ( n ) \operatorname{poly}(n) poly ( n ) N 2 ( β ) \mathcal{N}^2(\beta) N 2 ( β ) size ( A , b , c ) \operatorname{size}(A,b,c) size ( A , b , c ) L = size ( A , b , c ) L = \operatorname{size}(A,b,c) L = size ( A , b , c ) [22 ] . It is interesting to note that the new iterate ( x ~ , s ~ ) (\tilde{x},\tilde{s}) ( x ~ , s ~ ) ( x , s ) (x,s) ( x , s ) poly ( n ) \operatorname{poly}(n) poly ( n ) P × D \mathcal{P} \times\mathcal{D} P × D 2 n ⋅ 2 n = 2 2 n 2^n \cdot2^n = 2^{2n} 2 n ⋅ 2 n = 2 2 n
We use the following simple bounds from [20 ] . As in [20 ] , these bound the sizes of values, that is, of lowest-terms representations. Remark 5.2 below discusses which of them hold for encoded rationals.
Lemma 5.1. The following bounds hold:
(i) For α 1 , … , α k ∈ Q \alpha_1,\ldots,\alpha_k \in\mathbb{Q} α 1 , … , α k ∈ Q size ( ∑ i = 1 k α i ) ≤ 2 ∑ i = 1 k size ( α i ) \operatorname{size}\left(\sum_{i=1}^{k}\alpha_i\right) \le2\sum_{i=1}^{k}\operatorname{size}(\alpha_i) size ( ∑ i = 1 k α i ) ≤ 2 ∑ i = 1 k size ( α i ) size ( ∏ i = 1 k α i ) ≤ ∑ i = 1 k size ( α i ) \operatorname{size}\left(\prod_{i=1}^{k}\alpha_i\right) \le\sum_{i=1}^{k}\operatorname{size}(\alpha_i) size ( ∏ i = 1 k α i ) ≤ ∑ i = 1 k size ( α i )
(ii) For an invertible matrix M ∈ Q n × n M \in\mathbb{Q}^{n \times n} M ∈ Q n × n size ( M − 1 ) ≤ 4 n 2 size ( M ) \operatorname{size}(M^{-1}) \le4n^2\operatorname{size}(M) size ( M − 1 ) ≤ 4 n 2 size ( M )
(iii) For A ∈ Q m × n A \in\mathbb{Q}^{m \times n} A ∈ Q m × n b ∈ Q m b \in\mathbb{Q}^m b ∈ Q m c ∈ Q n c \in\mathbb{Q}^n c ∈ Q n P = { x ∈ R n ∣ A x = b , x ≥ 0 n } \mathcal{P} = \{x \in\mathbb{R}^n \mid Ax = b,\ x \ge0_n\} P = { x ∈ R n ∣ A x = b , x ≥ 0 n } D = { s ∈ R n ∣ ∃ y ∈ R m , A T y + s = c , s ≥ 0 n } \mathcal{D} = \{s \in\mathbb{R}^n \mid\exists y \in\mathbb{R}^m,\ A^{\mathsf T}y+s=c,\ s \ge0_n\} D = { s ∈ R n ∣ ∃ y ∈ R m , A T y + s = c , s ≥ 0 n } P \mathcal{P} P O ( n 3 size ( A , b ) ) O(n^3\operatorname{size}(A,b)) O ( n 3 size ( A , b )) P \mathcal{P} P O ( n 3 size ( A , b ) ) O(n^3\operatorname{size}(A,b)) O ( n 3 size ( A , b )) D \mathcal{D} D size ( A , c ) \operatorname{size}(A,c) size ( A , c ) size ( A , b ) \operatorname{size}(A,b) size ( A , b )
Remark 5.2 (The same bounds on \widehat{\mathbb{Q}}). Part (i) also holds for encoded rationals. By (35) , for (p_1,q_1),\ldots,(p_k,q_k) \in \widehat{\mathbb{Q}}$ we have
size ( ∑ i = 1 k ( p i , q i ) ) ≤ 2 ∑ i = 1 k size ( p i , q i ) , size ( ∏ i = 1 k ( p i , q i ) ) ≤ ∑ i = 1 k size ( p i , q i ) , \operatorname{size}\left(\sum_{i=1}^{k}(p_i,q_i)\right) \le2\sum_{i=1}^{k}\operatorname{size}(p_i,q_i), \qquad\operatorname{size}\left(\prod_{i=1}^{k}(p_i,q_i)\right) \le\sum_{i=1}^{k}\operatorname{size}(p_i,q_i), size ( i = 1 ∑ k ( p i , q i ) ) ≤ 2 i = 1 ∑ k size ( p i , q i ) , size ( i = 1 ∏ k ( p i , q i ) ) ≤ i = 1 ∑ k size ( p i , q i ) , and likewise entrywise for sums and products of encoded vectors and matrices.
Parts (ii) and (iii), however, bound the lowest-terms representations of M − 1 M^{-1} M − 1 P \mathcal{P} P M M M M ~ \widetilde{M} M val ( M ~ ) = M − 1 \operatorname{val}(\widetilde{M}) = M^{-1} val ( M ) = M − 1 A A A b b b x ∈ P x \in\mathcal{P} x ∈ P v ~ , r ~ ∈ Q ^ n \tilde{v},\tilde{r} \in\widehat{\mathbb{Q}}^n v ~ , r ~ ∈ Q n val ( v ~ ) = v \operatorname{val}(\tilde{v}) = v val ( v ~ ) = v val ( r ~ ) = r \operatorname{val}(\tilde{r}) = r val ( r ~ ) = r v v v r r r P \mathcal{P} P
We next give a simple rounding step for rational numbers. Suppose that, given α ∈ Q > 0 \alpha\in\mathbb{Q}_{>0} α ∈ Q > 0 r ∈ Q > 0 r \in\mathbb{Q}_{>0} r ∈ Q > 0 α \alpha α 2 O ( n ) 2^{O(n)} 2 O ( n ) α \alpha α r r r α \alpha α r r r
Lemma 5.3 (Anchoring numbers). Let α , r ∈ Q ^ > 0 \alpha,r \in\widehat{\mathbb{Q}}_{>0} α , r ∈ Q > 0 n , K ∈ Z ≥ 0 n,K \in\mathbb{Z}_{\ge0} n , K ∈ Z ≥ 0 r / 2 n ≤ α ≤ 2 n r r/2^n \le\alpha\le2^n r r / 2 n ≤ α ≤ 2 n r O ( log ( n + K ) ) O(\log(n+K)) O ( log ( n + K )) α ^ ∈ Q \hat{\alpha} \in\mathbb{Q} α ^ ∈ Q α ≤ α ^ ≤ ( 1 + 1 / K ) α \alpha\le\hat{\alpha} \le(1+1/K)\alpha α ≤ α ^ ≤ ( 1 + 1/ K ) α size ( α ^ ) ≤ size ( r ) + O ( n + log K ) \operatorname{size}(\hat{\alpha}) \le\operatorname{size}(r)+O(n+\log K) size ( α ^ ) ≤ size ( r ) + O ( n + log K )
Proof. Let m m m n n n O ( log ( m ) ) = O ( log ( n ) ) O(\log(m)) = O(\log(n)) O ( log ( m )) = O ( log ( n )) 2 0 , 2 1 , 2 2 , … , 2 m 2^0,2^1,2^2,\ldots,2^m 2 0 , 2 1 , 2 2 , … , 2 m O ( log n ) O(\log n) O ( log n ) k ∈ [ − m , m ] k \in[-m,m] k ∈ [ − m , m ] 2 k r ≤ α ≤ 2 k + 1 r 2^k r \le\alpha\le2^{k+1}r 2 k r ≤ α ≤ 2 k + 1 r 2 k ′ r 2^{k'}r 2 k ′ r k ′ k' k ′ k ′ = 0 k'=0 k ′ = 0 2 2 j 2^{2^j} 2 2 j α \alpha α 2 2 j 2^{2^j} 2 2 j r r r [ − m , m ] [-m,m] [ − m , m ] 2 k r 2^k r 2 k r O ( log log K ) O(\log\log K) O ( log log K ) q ∈ [ 0 , 2 ⌊ log 2 K ⌋ ] q \in[0,2\lfloor\log_2 K\rfloor] q ∈ [ 0 , 2 ⌊ log 2 K ⌋] 2 q / 2 ≤ K ≤ 2 q 2^{q/2} \le K \le2^q 2 q /2 ≤ K ≤ 2 q O ( q ) = O ( log K ) O(q) = O(\log K) O ( q ) = O ( log K ) p ∈ [ 0 , 2 q ] p \in[0,2^q] p ∈ [ 0 , 2 q ] α ≤ r 2 k ( 1 + p / 2 q ) ≤ ( 1 + 1 / K ) α \alpha\le r2^k(1+p/2^q) \le(1+1/K)\alpha α ≤ r 2 k ( 1 + p / 2 q ) ≤ ( 1 + 1/ K ) α α ^ : = r 2 k ( 1 + p / 2 q ) \hat{\alpha} := r2^k(1+p/2^q) α ^ := r 2 k ( 1 + p / 2 q ) □ \square □
We next state the algorithmic version of the Minkowski–Weyl decomposition theorem. The proof follows the standard lines of finding first a vertex v ( 1 ) v^{(1)} v ( 1 ) x x x v ( 1 ) v^{(1)} v ( 1 ) x x x [20 ] . We need explicit bounds on the number of arithmetic operations and on the encoding lengths of all numbers computed, in the model of § 1.1 . We give a self-contained proof for polyhedra in the standard form A x = b Ax=b A x = b x ≥ 0 n x\ge0_n x ≥ 0 n
Lemma 5.4 (Minkowski–Weyl decomposition). For A ∈ Q m × n A\in\mathbb{Q}^{m\times n} A ∈ Q m × n m ≤ n m\le n m ≤ n b ∈ Q m b\in\mathbb{Q}^m b ∈ Q m P = { x ∈ R n ∣ A x = b , x ≥ 0 n } \mathcal{P}=\{x\in\mathbb{R}^n\mid Ax=b,x\ge0_n\} P = { x ∈ R n ∣ A x = b , x ≥ 0 n } x ∈ P ∩ Q n x\in\mathcal{P}\cap\mathbb{Q}^n x ∈ P ∩ Q n v ( 1 ) , … , v ( k ) v^{(1)},\ldots,v^{(k)} v ( 1 ) , … , v ( k ) r ( 1 ) , … , r ( ℓ ) r^{(1)},\ldots,r^{(\ell)} r ( 1 ) , … , r ( ℓ ) λ 1 , … , λ k ≥ 0 \lambda_1,\ldots,\lambda_k\ge0 λ 1 , … , λ k ≥ 0 η 1 , … , η ℓ ≥ 0 \eta_1,\ldots,\eta_\ell\ge0 η 1 , … , η ℓ ≥ 0 ∑ i = 1 k λ i = 1 \sum_{i=1}^{k}\lambda_i=1 ∑ i = 1 k λ i = 1 k + ℓ ≤ n + 1 k+\ell\le n+1 k + ℓ ≤ n + 1
x = ∑ i = 1 k λ i v ( i ) + ∑ j = 1 ℓ η j r ( j ) . x=\sum_{i=1}^{k}\lambda_i v^{(i)}+\sum_{j=1}^{\ell}\eta_j r^{(j)}. x = i = 1 ∑ k λ i v ( i ) + j = 1 ∑ ℓ η j r ( j ) . The algorithm performs O ( n 7 ) O(n^7) O ( n 7 ) poly ( n ) size ( A , b , x ) \operatorname{poly}(n)\operatorname{size}(A,b,x) poly ( n ) size ( A , b , x ) poly ( n ) size ( A , b ) \operatorname{poly}(n)\operatorname{size}(A,b) poly ( n ) size ( A , b ) poly ( n ) size ( A , b , x ) \operatorname{poly}(n)\operatorname{size}(A,b,x) poly ( n ) size ( A , b , x )
Preliminaries on interior point methods
Before proving Theorem 1.8 in § 5.2 , we summarize some simple facts needed from the interior-point literature. The following simple formula allows us to express gaps between different primal and dual solutions.
Proposition 5.5. Given x , x ′ x,x' x , x ′ x − x ′ ∈ ker ( A ) x-x'\in\ker(A) x − x ′ ∈ ker ( A ) s , s ′ s,s' s , s ′ s − s ′ ∈ im ( A T ) s-s'\in\operatorname{im}(A^{\mathsf{T}}) s − s ′ ∈ im ( A T )
⟨ x , s ⟩ + ⟨ x ′ , s ′ ⟩ = ⟨ x , s ′ ⟩ + ⟨ x ′ , s ⟩ . \langle x,s\rangle+\langle x',s'\rangle=\langle x,s'\rangle+\langle x',s\rangle. ⟨ x , s ⟩ + ⟨ x ′ , s ′ ⟩ = ⟨ x , s ′ ⟩ + ⟨ x ′ , s ⟩ . In particular, if ⟨ x ′ , s ′ ⟩ = 0 \langle x',s'\rangle=0 ⟨ x ′ , s ′ ⟩ = 0
⟨ x , s ⟩ = ⟨ x , s ′ ⟩ + ⟨ x ′ , s ⟩ . \langle x,s\rangle=\langle x,s'\rangle+\langle x',s\rangle. ⟨ x , s ⟩ = ⟨ x , s ′ ⟩ + ⟨ x ′ , s ⟩ . Proof. This follows since x − x ′ ∈ ker ( A ) x-x'\in\ker(A) x − x ′ ∈ ker ( A ) s − s ′ ∈ im ( A T ) s-s'\in\operatorname{im}(A^{\mathsf{T}}) s − s ′ ∈ im ( A T ) □ \square □
A key property of the central path is ‘near monotonicity’, formulated in the following lemma, see [VY96, Lemma 16].
Lemma 5.6. For the central path point z c p ( μ ) = ( x c p ( μ ) , s c p ( μ ) ) z^{\mathrm{cp}}(\mu)=(x^{\mathrm{cp}}(\mu),s^{\mathrm{cp}}(\mu)) z cp ( μ ) = ( x cp ( μ ) , s cp ( μ )) z = ( x , s ) ∈ P × D z=(x,s)\in\mathcal{P}\times\mathcal{D} z = ( x , s ) ∈ P × D
z ≤ n ( 1 + μ ˉ ( z ) μ ) z c p ( μ ) . z\le n\left(1+\frac{\bar{\mu}(z)}{\mu}\right)z^{\mathrm{cp}}(\mu). z ≤ n ( 1 + μ μ ˉ ( z ) ) z cp ( μ ) . Moreover, if μ ˉ ( z ) ≤ μ \bar{\mu}(z)\le\mu μ ˉ ( z ) ≤ μ z ≤ 2 n z c p ( μ ) z\le2nz^{\mathrm{cp}}(\mu) z ≤ 2 n z cp ( μ )
We formulate a simple linearity property of the duality gap, see e.g. [2 Proposition 2.29] .
Proposition 5.7 (Linearity of duality gap). Given x ( 1 ) , … , x ( k ) ∈ P x^{(1)},\ldots,x^{(k)}\in\mathcal{P} x ( 1 ) , … , x ( k ) ∈ P s ( 1 ) , … , s ( k ) ∈ D s^{(1)},\ldots,s^{(k)}\in\mathcal{D} s ( 1 ) , … , s ( k ) ∈ D λ ∈ R ≥ 0 k \lambda\in\mathbb{R}_{\ge0}^{k} λ ∈ R ≥ 0 k ∑ i = 1 k λ i = 1 \sum_{i=1}^{k}\lambda_i=1 ∑ i = 1 k λ i = 1
μ ˉ ( ∑ i = 1 k λ i x ( i ) , ∑ i = 1 k λ i s ( i ) ) = ∑ i = 1 k λ i μ ˉ ( x ( i ) , s ( i ) ) . \bar{\mu}\left(\sum_{i=1}^{k}\lambda_i x^{(i)},\sum_{i=1}^{k}\lambda_i s^{(i)}\right)=\sum_{i=1}^{k}\lambda_i\bar{\mu}(x^{(i)},s^{(i)}). μ ˉ ( i = 1 ∑ k λ i x ( i ) , i = 1 ∑ k λ i s ( i ) ) = i = 1 ∑ k λ i μ ˉ ( x ( i ) , s ( i ) ) . We use the following proximity properties. For the first one, see e.g., [21 Lemma 5.4] and [33 Proposition 2.1] .
Proposition 5.8. Let z = ( x , s ) ∈ N 2 ( β ) z=(x,s)\in\mathcal{N}^{2}(\beta) z = ( x , s ) ∈ N 2 ( β ) β ∈ ( 0 , 1 / 4 ] \beta\in(0,1/4] β ∈ ( 0 , 1/4 ] μ = μ ˉ ( z ) \mu=\bar{\mu}(z) μ = μ ˉ ( z ) z c p ( μ ) = ( x c p ( μ ) , s c p ( μ ) ) z^{\mathrm{cp}}(\mu)=(x^{\mathrm{cp}}(\mu),s^{\mathrm{cp}}(\mu)) z cp ( μ ) = ( x cp ( μ ) , s cp ( μ )) i ∈ [ n ] i\in[n] i ∈ [ n ]
x i 1 + 2 β ≤ 1 − 2 β 1 − β x i ≤ x i c p ( μ ) ≤ x i 1 − β , and \frac{x_i}{1+2\beta}\le\frac{1-2\beta}{1-\beta}x_i\le x_i^{\mathrm{cp}}(\mu)\le\frac{x_i}{1-\beta},\qquad\text{and} 1 + 2 β x i ≤ 1 − β 1 − 2 β x i ≤ x i cp ( μ ) ≤ 1 − β x i , and s i 1 + 2 β ≤ 1 − 2 β 1 − β s i ≤ s i c p ( μ ) ≤ s i 1 − β . \frac{s_i}{1+2\beta}\le\frac{1-2\beta}{1-\beta}s_i\le s_i^{\mathrm{cp}}(\mu)\le\frac{s_i}{1-\beta}. 1 + 2 β s i ≤ 1 − β 1 − 2 β s i ≤ s i cp ( μ ) ≤ 1 − β s i . The next lemma formalizes the well-known property of standard corrector steps, see e.g., [35 ] .
Lemma 5.9 (Corrector step). Let A ∈ Q ^ m × n A \in\widehat{\mathbb{Q}}^{m \times n} A ∈ Q m × n β ∈ ( 0 , 1 / 4 ] \beta\in(0,1/4] β ∈ ( 0 , 1/4 ] ( x , s ) ∈ N 2 ( 2 β ) ∩ Q ^ 2 n (x,s) \in\mathcal{N}^{2}(2\beta) \cap\widehat{\mathbb{Q}}^{2n} ( x , s ) ∈ N 2 ( 2 β ) ∩ Q 2 n ( x ′ , s ′ ) ∈ N 2 ( β ) ∩ Q ^ 2 n (x',s') \in\mathcal{N}^{2}(\beta) \cap\widehat{\mathbb{Q}}^{2n} ( x ′ , s ′ ) ∈ N 2 ( β ) ∩ Q 2 n μ ‾ ( x ′ , s ′ ) = μ ‾ ( x , s ) \overline{\mu}(x',s') = \overline{\mu}(x,s) μ ( x ′ , s ′ ) = μ ( x , s ) size ( x ′ , s ′ ) = poly ( n ) size ( A , x , s ) \operatorname{size}(x',s') = \operatorname{poly}(n)\operatorname{size}(A,x,s) size ( x ′ , s ′ ) = poly ( n ) size ( A , x , s ) 2 n + m 2n+m 2 n + m A A A x x x s s s
The rounding procedure
We prove Theorem 1.8 in two steps. First, we show that every iterate ( x , s ) (x,s) ( x , s ) ( x ^ , s ^ ) (\hat{x},\hat{s}) ( x ^ , s ^ )
Lemma 5.10. Let A ∈ Q ^ m × n A \in\widehat{\mathbb{Q}}^{m \times n} A ∈ Q m × n b ∈ Q ^ m b \in\widehat{\mathbb{Q}}^{m} b ∈ Q m c ∈ Q ^ n c \in\widehat{\mathbb{Q}}^{n} c ∈ Q n β ∈ ( 0 , 1 / 16 ] \beta\in(0,1/16] β ∈ ( 0 , 1/16 ] § 1.2 , and let ( x , s ) ∈ N 2 ( β ) ∩ Q ^ 2 n (x,s) \in\mathcal{N}^{2}(\beta) \cap\widehat{\mathbb{Q}}^{2n} ( x , s ) ∈ N 2 ( β ) ∩ Q 2 n ( x ^ , s ^ ) ∈ N ‾ 2 ( 2 β ) ∩ Q ^ 2 n (\hat{x},\hat{s}) \in\overline{\mathcal{N}}^{2}(2\beta) \cap\widehat{\mathbb{Q}}^{2n} ( x ^ , s ^ ) ∈ N 2 ( 2 β ) ∩ Q 2 n μ ‾ ( x ^ , s ^ ) ≤ μ ‾ ( x , s ) \overline{\mu}(\hat{x},\hat{s}) \le\overline{\mu}(x,s) μ ( x ^ , s ^ ) ≤ μ ( x , s ) μ ⋆ ∈ Q ^ \mu^{\star} \in\widehat{\mathbb{Q}} μ ⋆ ∈ Q μ ‾ ( x ^ , s ^ ) ≤ μ ⋆ ≤ ( 1 + β / ( 4 n 2 ) ) μ ‾ ( x ^ , s ^ ) \overline{\mu}(\hat{x},\hat{s}) \le\mu^{\star} \le(1+\beta/(4n^{2}))\overline{\mu}(\hat{x},\hat{s}) μ ( x ^ , s ^ ) ≤ μ ⋆ ≤ ( 1 + β / ( 4 n 2 )) μ ( x ^ , s ^ ) size ( μ ⋆ ) = poly ( n ) size ( A , b , c ) \operatorname{size}(\mu^{\star}) = \operatorname{poly}(n)\operatorname{size}(A,b,c) size ( μ ⋆ ) = poly ( n ) size ( A , b , c )
(i) μ ⋆ > 0 \mu^{\star} > 0 μ ⋆ > 0 ( x ^ , s ^ ) ∈ N 2 ( 2 β ) (\hat{x},\hat{s}) \in\mathcal{N}^{2}(2\beta) ( x ^ , s ^ ) ∈ N 2 ( 2 β )
(ii) μ ⋆ = 0 \mu^{\star} = 0 μ ⋆ = 0 ( x ^ , s ^ ) (\hat{x},\hat{s}) ( x ^ , s ^ ) ( v ^ , w ^ ) ∈ Q ^ 2 n (\hat{v},\hat{w}) \in\widehat{\mathbb{Q}}^{2n} ( v ^ , w ^ ) ∈ Q 2 n B : = supp ( x ^ ) B := \operatorname{supp}(\hat{x}) B := supp ( x ^ ) N : = supp ( s ^ ) N := \operatorname{supp}(\hat{s}) N := supp ( s ^ ) [ n ] [n] [ n ] v ^ ∈ im ( A T ) \hat{v} \in\operatorname{im}(A^{\mathsf T}) v ^ ∈ im ( A T ) v ^ B > 0 B \hat{v}_{B} > 0_{B} v ^ B > 0 B A w ^ = 0 m A\hat{w} = 0_{m} A w ^ = 0 m w ^ N > 0 N \hat{w}_{N} > 0_{N} w ^ N > 0 N ∥ ( x ^ B ∘ v ^ B , s ^ N ∘ w ^ N ) − 1 n ∥ 2 ≤ 2 β \lVert(\hat{x}_{B} \circ\hat{v}_{B},\hat{s}_{N} \circ\hat{w}_{N})-\mathbf{1}_{n}\rVert_{2} \le2\beta ∥( x ^ B ∘ v ^ B , s ^ N ∘ w ^ N ) − 1 n ∥ 2 ≤ 2 β ( x ^ , s ^ ) (\hat{x},\hat{s}) ( x ^ , s ^ ) ( v ^ , w ^ ) (\hat{v},\hat{w}) ( v ^ , w ^ ) Theorem 1.6 with 2 β 2\beta 2 β β \beta β
Proof. Let μ = μ ‾ ( x , s ) \mu=\overline{\mu}(x,s) μ = μ ( x , s ) ⟨ x , s ⟩ = n μ \langle x,s\rangle=n\mu ⟨ x , s ⟩ = n μ ( x , s ) ∈ P × D (x,s)\in\mathcal{P}\times\mathcal{D} ( x , s ) ∈ P × D P × D \mathcal{P}\times\mathcal{D} P × D ( v ( i ) , u ( i ) ) (v^{(i)},u^{(i)}) ( v ( i ) , u ( i ) ) v ( i ) v^{(i)} v ( i ) P \mathcal{P} P u ( i ) u^{(i)} u ( i ) D \mathcal{D} D P × D \mathcal{P}\times\mathcal{D} P × D ( r ( j ) , 0 ) (r^{(j)},0) ( r ( j ) , 0 ) ( 0 , t ( j ) ) (0,t^{(j)}) ( 0 , t ( j ) ) r ( j ) r^{(j)} r ( j ) P \mathcal{P} P t ( j ) t^{(j)} t ( j ) D \mathcal{D} D 2 n + 1 2n+1 2 n + 1
To apply Lemma 5.4 we need P × D \mathcal{P}\times\mathcal{D} P × D D \mathcal{D} D y y y
B : = det ( A A T ) I n − A T adj ( A A T ) A ∈ Q n × n . B := \det(AA^{\mathsf T})I_n-A^{\mathsf T}\operatorname{adj}(AA^{\mathsf T})A \in\mathbb{Q}^{n\times n}. B := det ( A A T ) I n − A T adj ( A A T ) A ∈ Q n × n . As A A A A A T AA^{\mathsf T} A A T B = det ( A A T ) ( I n − A T ( A A T ) − 1 A ) B=\det(AA^{\mathsf T})(I_n-A^{\mathsf T}(AA^{\mathsf T})^{-1}A) B = det ( A A T ) ( I n − A T ( A A T ) − 1 A ) ker ( A ) \ker(A) ker ( A ) ker ( B ) = ker ( A ) ⊥ = im ( A T ) \ker(B)=\ker(A)^{\perp}=\operatorname{im}(A^{\mathsf T}) ker ( B ) = ker ( A ) ⊥ = im ( A T ) s ∈ D s\in\mathcal{D} s ∈ D B ( c − s ) = 0 n B(c-s)=0_n B ( c − s ) = 0 n s ≥ 0 n s\geq0_n s ≥ 0 n
P × D = { ( x , s ) ∈ R 2 n | ( A 0 m × n 0 n × n B ) ( x s ) = ( b B c ) , ( x , s ) ≥ 0 2 n } . \mathcal{P}\times\mathcal{D}
=
\left\{(x,s)\in\mathbb{R}^{2n}\ \middle|\
\begin{pmatrix}
A & 0_{m\times n}\\
0_{n\times n} & B
\end{pmatrix}
\binom{x}{s}
=
\binom{b}{Bc},\ (x,s)\geq0_{2n}
\right\}. P × D = { ( x , s ) ∈ R 2 n ( A 0 n × n 0 m × n B ) ( s x ) = ( B c b ) , ( x , s ) ≥ 0 2 n } . This describes the same set, so its vertices and extreme rays are exactly those of P × D \mathcal{P}\times\mathcal{D} P × D rank ( B ) = n − m \operatorname{rank}(B)=n-m rank ( B ) = n − m Lemma 5.4 . The matrix B B B det ( A A T ) \det(AA^{\mathsf T}) det ( A A T ) adj ( A A T ) \operatorname{adj}(AA^{\mathsf T}) adj ( A A T ) Lemma 2.2 (ii), and matrix products, in O ( n 4 ) O(n^{4}) O ( n 4 ) size ( A A T ) = poly ( n ) size ( A ) \operatorname{size}(AA^{\mathsf T})=\operatorname{poly}(n)\operatorname{size}(A) size ( A A T ) = poly ( n ) size ( A ) Lemma 2.2 and Remark 5.2 give size ( B ) = poly ( n ) size ( A ) \operatorname{size}(B)=\operatorname{poly}(n)\operatorname{size}(A) size ( B ) = poly ( n ) size ( A ) size ( B c ) = poly ( n ) size ( A , c ) \operatorname{size}(Bc)=\operatorname{poly}(n)\operatorname{size}(A,c) size ( B c ) = poly ( n ) size ( A , c ) m + n ≤ 2 n m+n\leq2n m + n ≤ 2 n 2 n 2n 2 n Lemma 5.4 bounds this decomposition by O ( n 7 ) O(n^{7}) O ( n 7 ) poly ( n ) ( size ( A , b , c ) + size ( x , s ) ) \operatorname{poly}(n)(\operatorname{size}(A,b,c)+\operatorname{size}(x,s)) poly ( n ) ( size ( A , b , c ) + size ( x , s )) size ( v ( i ) , u ( i ) ) = poly ( n ) size ( A , b , c ) \operatorname{size}(v^{(i)},u^{(i)})=\operatorname{poly}(n)\operatorname{size}(A,b,c) size ( v ( i ) , u ( i ) ) = poly ( n ) size ( A , b , c )
( x , s ) = ∑ i = 1 k λ i ( v ( i ) , u ( i ) ) + ( r , t ) , (x,s)=\sum_{i=1}^{k}\lambda_i(v^{(i)},u^{(i)})+(r,t), ( x , s ) = i = 1 ∑ k λ i ( v ( i ) , u ( i ) ) + ( r , t ) , where λ ≥ 0 \lambda\geq0 λ ≥ 0 ∑ i = 1 k λ i = 1 \sum_{i=1}^{k}\lambda_i=1 ∑ i = 1 k λ i = 1 ( r , t ) (r,t) ( r , t ) P × D \mathcal{P}\times\mathcal{D} P × D
I − = { i ∈ [ k ] : μ ‾ ( v ( i ) , u ( i ) ) ≤ μ } , I + = { i ∈ [ k ] : μ ‾ ( v ( i ) , u ( i ) ) > μ } , (36) I^{-}=\{i\in[k]:\overline{\mu}(v^{(i)},u^{(i)})\leq\mu\},\qquad
I^{+}=\{i\in[k]:\overline{\mu}(v^{(i)},u^{(i)})>\mu\}, \tag*{(36)} I − = { i ∈ [ k ] : μ ( v ( i ) , u ( i ) ) ≤ μ } , I + = { i ∈ [ k ] : μ ( v ( i ) , u ( i ) ) > μ } , (36) We note that I − ≠ ∅ I^{-} \ne\emptyset I − = ∅ I + I^{+} I + ( v ˉ , u ˉ ) : = ∑ i = 1 k λ i ( v ( i ) , u ( i ) ) (\bar{v},\bar{u}) := \sum_{i=1}^{k} \lambda_i(v^{(i)},u^{(i)}) ( v ˉ , u ˉ ) := ∑ i = 1 k λ i ( v ( i ) , u ( i ) ) A r = 0 m Ar=0_m A r = 0 m t ∈ im ( A T ) t \in\operatorname{im}(A^{\mathsf{T}}) t ∈ im ( A T ) ⟨ r , t ⟩ = 0 \langle r,t\rangle=0 ⟨ r , t ⟩ = 0 Proposition 5.7 give
n μ = ⟨ v ˉ + r , u ˉ + t ⟩ = ⟨ v ˉ , u ˉ ⟩ + ⟨ v ˉ , t ⟩ + ⟨ r , u ˉ ⟩ ≥ n μ ( v ˉ , u ˉ ) = n ∑ i = 1 k λ i μ ( v ( i ) , u ( i ) ) , n\mu=\langle\bar{v}+r,\bar{u}+t\rangle=\langle\bar{v},\bar{u}\rangle+\langle\bar{v},t\rangle+\langle r,\bar{u}\rangle\ge n\mu(\bar{v},\bar{u})=n\sum_{i=1}^{k}\lambda_i\mu(v^{(i)},u^{(i)}), n μ = ⟨ v ˉ + r , u ˉ + t ⟩ = ⟨ v ˉ , u ˉ ⟩ + ⟨ v ˉ , t ⟩ + ⟨ r , u ˉ ⟩ ≥ n μ ( v ˉ , u ˉ ) = n i = 1 ∑ k λ i μ ( v ( i ) , u ( i ) ) , and hence μ ( v ( i ) , u ( i ) ) ≤ μ \mu(v^{(i)},u^{(i)})\le\mu μ ( v ( i ) , u ( i ) ) ≤ μ i i i ( v ( i ) , u ( i ) ) (v^{(i)},u^{(i)}) ( v ( i ) , u ( i ) ) size ( v ( i ) , u ( i ) ) = poly ( n ) size ( A , b , c ) \operatorname{size}(v^{(i)},u^{(i)})=\operatorname{poly}(n)\operatorname{size}(A,b,c) size ( v ( i ) , u ( i ) ) = poly ( n ) size ( A , b , c ) Lemma 5.4 , and μ ( v ( i ) , u ( i ) ) \mu(v^{(i)},u^{(i)}) μ ( v ( i ) , u ( i ) ) O ( n ) O(n) O ( n ) size ( μ ( v ( i ) , u ( i ) ) ) = poly ( n ) size ( A , b , c ) \operatorname{size}(\mu(v^{(i)},u^{(i)}))=\operatorname{poly}(n)\operatorname{size}(A,b,c) size ( μ ( v ( i ) , u ( i ) )) = poly ( n ) size ( A , b , c )
δ : = β 16 n 2 . (37) \delta:=\frac{\beta}{16n^{2}}. \tag*{(37)} δ := 16 n 2 β . (37) If there is an index i ∈ [ k ] i\in[k] i ∈ [ k ] μ ( v ( i ) , u ( i ) ) ∈ [ μ δ 2 / ( 4 n ) , 4 n μ / δ 2 ] \mu(v^{(i)},u^{(i)})\in[\mu\delta^{2}/(4n),4n\mu/\delta^{2}] μ ( v ( i ) , u ( i ) ) ∈ [ μ δ 2 / ( 4 n ) , 4 n μ / δ 2 ] α : = μ \alpha:=\mu α := μ r : = μ ( v ( i ) , u ( i ) ) r:=\mu(v^{(i)},u^{(i)}) r := μ ( v ( i ) , u ( i ) ) K K K K ≥ 1 / δ K\ge1/\delta K ≥ 1/ δ O ( log n ) O(\log n) O ( log n ) N N N n n n N N N 2 N ≥ 4 n / δ 2 2^{N}\ge4n/\delta^{2} 2 N ≥ 4 n / δ 2 N = O ( log n ) N=O(\log n) N = O ( log n ) N N N O ( log log n ) O(\log\log n) O ( log log n ) r / 2 N ≤ α ≤ 2 N r r/2^{N}\le\alpha\le2^{N}r r / 2 N ≤ α ≤ 2 N r i i i μ ⋆ ∈ Q + \mu^\star\in\mathbb{Q}_{+} μ ⋆ ∈ Q + μ ≤ μ ⋆ ≤ ( 1 + δ ) μ \mu\le\mu^\star\le(1+\delta)\mu μ ≤ μ ⋆ ≤ ( 1 + δ ) μ O ( log ( N + K ) ) = O ( log n ) O(\log(N+K))=O(\log n) O ( log ( N + K )) = O ( log n ) size ( μ ⋆ ) ≤ size ( r ) + O ( N + log K ) = poly ( n ) size ( A , b , c ) \operatorname{size}(\mu^\star)\le\operatorname{size}(r)+O(N+\log K)=\operatorname{poly}(n)\operatorname{size}(A,b,c) size ( μ ⋆ ) ≤ size ( r ) + O ( N + log K ) = poly ( n ) size ( A , b , c ) size ( μ ( v ( i ) , u ( i ) ) ) \operatorname{size}(\mu(v^{(i)},u^{(i)})) size ( μ ( v ( i ) , u ( i ) )) ( x ^ , s ^ ) = ( x , s ) (\hat{x},\hat{s})=(x,s) ( x ^ , s ^ ) = ( x , s ) μ ⋆ \mu^\star μ ⋆ μ ⋆ ≥ μ > 0 \mu^\star\ge\mu>0 μ ⋆ ≥ μ > 0 ( x , s ) ∈ N 2 ( β ) ⊆ N 2 ( 2 β ) (x,s)\in\mathcal{N}^{2}(\beta)\subseteq\mathcal{N}^{2}(2\beta) ( x , s ) ∈ N 2 ( β ) ⊆ N 2 ( 2 β )
For the rest of the proof, we can assume that
μ ( v ( i ) , u ( i ) ) < δ 2 μ 4 n , ∀ i ∈ I − and μ ( v ( j ) , u ( j ) ) > 4 n μ δ 2 , ∀ j ∈ I + . (38) \mu(v^{(i)},u^{(i)})<\frac{\delta^{2}\mu}{4n},\quad\forall i\in I^{-}
\qquad\text{and}\qquad
\mu(v^{(j)},u^{(j)})>\frac{4n\mu}{\delta^{2}},\quad\forall j\in I^{+}. \tag*{(38)} μ ( v ( i ) , u ( i ) ) < 4 n δ 2 μ , ∀ i ∈ I − and μ ( v ( j ) , u ( j ) ) > δ 2 4 n μ , ∀ j ∈ I + . (38) Let us define
α : = ∑ i ∈ I + λ i , \alpha:=\sum_{i\in I^{+}}\lambda_i, α := i ∈ I + ∑ λ i , ( x − , s − ) : = 1 1 − α ∑ i ∈ I − λ i ( v ( i ) , u ( i ) ) , μ − : = μ ( x − , s − ) (39) (x^{-},s^{-}):=\frac{1}{1-\alpha}\sum_{i\in I^{-}}\lambda_i(v^{(i)},u^{(i)}),\qquad
\mu^{-}:=\mu(x^{-},s^{-}) \tag*{(39)} ( x − , s − ) := 1 − α 1 i ∈ I − ∑ λ i ( v ( i ) , u ( i ) ) , μ − := μ ( x − , s − ) (39) Note that α < 1 \alpha<1 α < 1 I − ≠ ∅ I^{-}\ne\emptyset I − = ∅
Claim 5.11. ( x − , s − ) ≤ ( 1 + δ ) ( x , s ) (x^{-},s^{-})\le(1+\delta)(x,s) ( x − , s − ) ≤ ( 1 + δ ) ( x , s ) i ∈ [ n ] i\in[n] i ∈ [ n ] x i − ≥ ( 1 − δ ) x i x^{-}_{i}\ge(1-\delta)x_i x i − ≥ ( 1 − δ ) x i s i − ≥ ( 1 − δ ) s i s^{-}_{i}\ge(1-\delta)s_i s i − ≥ ( 1 − δ ) s i
Proof. We distinguish two cases according to whether α = 0 \alpha=0 α = 0
If α = 0 \alpha=0 α = 0 ( x , s ) − ( x − , s − ) = ( r , t ) ≥ ( 0 n , 0 n ) (x,s)-(x^{-},s^{-})=(r,t)\ge(0_n,0_n) ( x , s ) − ( x − , s − ) = ( r , t ) ≥ ( 0 n , 0 n ) x i ≥ x i − x_i\ge x^{-}_i x i ≥ x i − s i ≥ s i − s_i\ge s^{-}_i s i ≥ s i − i ∈ [ n ] i\in[n] i ∈ [ n ] r r r t t t ⟨ r , t ⟩ = 0 \langle r,t\rangle=0 ⟨ r , t ⟩ = 0 i ∈ [ n ] i\in[n] i ∈ [ n ] r i = 0 r_i=0 r i = 0 t i = 0 t_i=0 t i = 0 x i − = x i x^{-}_i=x_i x i − = x i s i − = s i s^{-}_i=s_i s i − = s i
If α > 0 \alpha>0 α > 0
( x + , s + ) : = 1 α ( ∑ i ∈ I + λ i ( v ( i ) , u ( i ) ) + ( r , t ) ) , μ + : = μ ( x + , s + ) . (40) (x^{+},s^{+}):=\frac{1}{\alpha}\left(\sum_{i\in I^{+}}\lambda_i(v^{(i)},u^{(i)})+(r,t)\right),\qquad
\mu^{+}:=\mu(x^{+},s^{+}). \tag*{(40)} ( x + , s + ) := α 1 ( i ∈ I + ∑ λ i ( v ( i ) , u ( i ) ) + ( r , t ) ) , μ + := μ ( x + , s + ) . (40) Thus, ( x − , s − ) , ( x + , s + ) ∈ P × D (x^{-},s^{-}),(x^{+},s^{+})\in\mathcal{P}\times\mathcal{D} ( x − , s − ) , ( x + , s + ) ∈ P × D ( x , s ) = ( 1 − α ) ( x − , s − ) + α ( x + , s + ) (x,s)=(1-\alpha)(x^{-},s^{-})+\alpha(x^{+},s^{+}) ( x , s ) = ( 1 − α ) ( x − , s − ) + α ( x + , s + ) μ − ≤ max i ∈ I − μ ( v ( i ) , u ( i ) ) \mu^{-}\le\max_{i\in I^{-}}\mu(v^{(i)},u^{(i)}) μ − ≤ max i ∈ I − μ ( v ( i ) , u ( i ) ) μ + ≥ min i ∈ I + μ ( v ( i ) , u ( i ) ) \mu^{+}\ge\min_{i\in I^{+}}\mu(v^{(i)},u^{(i)}) μ + ≥ min i ∈ I + μ ( v ( i ) , u ( i ) )
μ − < δ 2 μ 4 n and μ + > 4 n μ δ 2 . (41) \mu^{-}<\frac{\delta^{2}\mu}{4n}\qquad\text{and}\qquad\mu^{+}>\frac{4n\mu}{\delta^{2}}. \tag*{(41)} μ − < 4 n δ 2 μ and μ + > δ 2 4 n μ . (41) By the linearity of the duality gap (Proposition 5.7 ), we have μ = ( 1 − α ) μ − + α μ + \mu=(1-\alpha)\mu^{-}+\alpha\mu^{+} μ = ( 1 − α ) μ − + α μ + α ≤ μ / μ + \alpha\le\mu/\mu^{+} α ≤ μ / μ +
α ≤ δ 2 4 n . (42) \alpha\le\frac{\delta^{2}}{4n}. \tag*{(42)} α ≤ 4 n δ 2 . (42) First, assume for a contradiction that x i − ≥ ( 1 + δ ) x i x^{-}_i\ge(1+\delta)x_i x i − ≥ ( 1 + δ ) x i i ∈ [ n ] i\in[n] i ∈ [ n ] s i − ≥ ( 1 + δ ) s i s^{-}_i\ge(1+\delta)s_i s i − ≥ ( 1 + δ ) s i ( 1 − α ) x i − + α x i + = x i (1-\alpha)x^{-}_i+\alpha x^{+}_i=x_i ( 1 − α ) x i − + α x i + = x i x i − x^{-}_i x i − x i + ≥ 0 x^{+}_i\ge0 x i + ≥ 0 1 − α ≤ 1 / ( 1 + δ ) 1-\alpha\le1/(1+\delta) 1 − α ≤ 1/ ( 1 + δ ) α ≥ δ / ( 1 + δ ) \alpha\ge\delta/(1+\delta) α ≥ δ / ( 1 + δ )
Next, assume for a contradiction that x i − ≤ ( 1 − δ ) x i x^{-}_i\le(1-\delta)x_i x i − ≤ ( 1 − δ ) x i s i − ≤ ( 1 − δ ) s i s^{-}_i\le(1-\delta)s_i s i − ≤ ( 1 − δ ) s i i ∈ [ n ] i\in[n] i ∈ [ n ] ( 1 − α ) x i − + α x i + = x i (1-\alpha)x^{-}_i+\alpha x^{+}_i=x_i ( 1 − α ) x i − + α x i + = x i ( 1 − α ) s i − + α s i + = s i (1-\alpha)s^{-}_i+\alpha s^{+}_i=s_i ( 1 − α ) s i − + α s i + = s i x i − x^{-}_i x i − s i − s^{-}_i s i − α x i + ≥ δ x i \alpha x^{+}_i\ge\delta x_i α x i + ≥ δ x i α s i + ≥ δ s i \alpha s^{+}_i\ge\delta s_i α s i + ≥ δ s i
δ 2 x i s i ≤ α 2 x i + s i + ≤ α 2 ⟨ x + , s + ⟩ = n α 2 μ + ≤ n α μ , \delta^{2}x_i s_i\le\alpha^{2}x^{+}_i s^{+}_i\le\alpha^{2}\langle x^{+},s^{+}\rangle=n\alpha^{2}\mu^{+}\le n\alpha\mu, δ 2 x i s i ≤ α 2 x i + s i + ≤ α 2 ⟨ x + , s + ⟩ = n α 2 μ + ≤ n α μ , where the final inequality uses α μ + ≤ μ \alpha\mu^{+} \le\mu α μ + ≤ μ ( x , s ) ∈ N 2 ( β ) (x,s) \in\mathcal{N}^{2}(\beta) ( x , s ) ∈ N 2 ( β ) x i s i ≥ ( 1 − β ) μ x_i s_i \ge(1-\beta)\mu x i s i ≥ ( 1 − β ) μ
δ 2 ( 1 − β ) ≤ n α , \delta^{2}(1-\beta) \le n\alpha, δ 2 ( 1 − β ) ≤ n α , in contradiction with (42) . ■
Claim 5.12. For any ϱ ∈ ( 8 n 2 μ − / ( β μ ) , 1 ] \varrho\in(8n^{2}\mu^{-}/(\beta\mu),1] ϱ ∈ ( 8 n 2 μ − / ( β μ ) , 1 ]
( x ^ , s ^ ) : = ( 1 − ϱ ) ( x − , s − ) + ϱ ( x , s ) ∈ N 2 ( 2 β ) (\hat{x},\hat{s}) := (1-\varrho)(x^{-},s^{-})+\varrho(x,s) \in\mathcal{N}^{2}(2\beta) ( x ^ , s ^ ) := ( 1 − ϱ ) ( x − , s − ) + ϱ ( x , s ) ∈ N 2 ( 2 β )
Proof. Let us define
B : = { i ∈ [ n ] : x i − ≥ ( 1 − δ ) x i } and N : = { i ∈ [ n ] : s i − ≥ ( 1 − δ ) s i } . B := \{i \in[n] : x^{-}_{i} \ge(1-\delta)x_i\}
\qquad\text{and}\qquad
N := \{i \in[n] : s^{-}_{i} \ge(1-\delta)s_i\}. B := { i ∈ [ n ] : x i − ≥ ( 1 − δ ) x i } and N := { i ∈ [ n ] : s i − ≥ ( 1 − δ ) s i } . By Claim 5.11, B ∪ N = [ n ] B \cup N = [n] B ∪ N = [ n ] B ∩ N = ∅ B \cap N = \varnothing B ∩ N = ∅ (41) and x i s i ≥ ( 1 − β ) μ x_i s_i \ge(1-\beta)\mu x i s i ≥ ( 1 − β ) μ i ∈ [ n ] i \in[n] i ∈ [ n ] ( x , s ) ∈ N 2 ( β ) (x,s) \in\mathcal{N}^{2}(\beta) ( x , s ) ∈ N 2 ( β ) (41) imply that for all i ∈ B i \in B i ∈ B
s i − ≤ n μ − x i − ≤ n μ − ( 1 − δ ) x i ≤ n μ − ( 1 − δ ) ( 1 − β ) μ ⋅ s i ≤ 2 n μ − μ ⋅ s i ≤ β 4 n ⋅ ϱ s i , (43) s^{-}_{i} \le\frac{n\mu^{-}}{x^{-}_{i}}
\le\frac{n\mu^{-}}{(1-\delta)x_i}
\le\frac{n\mu^{-}}{(1-\delta)(1-\beta)\mu}\cdot s_i
\le\frac{2n\mu^{-}}{\mu}\cdot s_i
\le\frac{\beta}{4n}\cdot\varrho s_i, \tag*{(43)} s i − ≤ x i − n μ − ≤ ( 1 − δ ) x i n μ − ≤ ( 1 − δ ) ( 1 − β ) μ n μ − ⋅ s i ≤ μ 2 n μ − ⋅ s i ≤ 4 n β ⋅ ϱ s i , (43) by the lower bound on ϱ \varrho ϱ
s ^ i = ( 1 − ϱ ) s i − + ϱ s i ∈ [ 1 , 1 + β 4 n ] ϱ s i . (44) \hat{s}_i=(1-\varrho)s^{-}_i+\varrho s_i
\in\left[1,1+\frac{\beta}{4n}\right]\varrho s_i. \tag*{(44)} s ^ i = ( 1 − ϱ ) s i − + ϱ s i ∈ [ 1 , 1 + 4 n β ] ϱ s i . (44) For i ∈ B i \in B i ∈ B x i − ∈ [ 1 − δ , 1 + δ ] x i x^{-}_i \in[1-\delta,1+\delta]x_i x i − ∈ [ 1 − δ , 1 + δ ] x i x ^ i ∈ [ 1 − δ , 1 + δ ] x i \hat{x}_i \in[1-\delta,1+\delta]x_i x ^ i ∈ [ 1 − δ , 1 + δ ] x i δ \delta δ i ∈ N i \in N i ∈ N
x ^ i s ^ i ∈ [ 1 − δ , 1 + β 2 n ] ϱ x i s i , ∀ i ∈ [ n ] . (45) \hat{x}_i\hat{s}_i \in\left[1-\delta,1+\frac{\beta}{2n}\right]\varrho x_i s_i,\qquad\forall i \in[n]. \tag*{(45)} x ^ i s ^ i ∈ [ 1 − δ , 1 + 2 n β ] ϱ x i s i , ∀ i ∈ [ n ] . (45) Therefore,
∥ x ^ ∘ s ^ ϱ μ − 1 n ∥ 2 = ∥ x ^ ∘ s ^ − ϱ x ∘ s ϱ μ + x ∘ s μ − 1 n ∥ 2 ≤ ∥ x ^ ∘ s ^ − ϱ x ∘ s ϱ μ ∥ 2 + ∥ x ∘ s μ − 1 n ∥ 2 ≤ β 2 n ⋅ ∥ x ∘ s μ ∥ 2 + β ≤ n ⋅ β ⋅ ( 1 + β ) 2 n + β ≤ ( 1 + 1 / 16 2 + 1 ) β = 49 32 β . \begin{aligned}
\left\|\frac{\hat{x}\circ\hat{s}}{\varrho\mu}-1_n\right\|_2
=
\left\|\frac{\hat{x}\circ\hat{s}-\varrho x\circ s}{\varrho\mu}
+\frac{x\circ s}{\mu}-1_n\right\|_2 \\
&\le
\left\|\frac{\hat{x}\circ\hat{s}-\varrho x\circ s}{\varrho\mu}\right\|_2
+
\left\|\frac{x\circ s}{\mu}-1_n\right\|_2 \\
&\le\frac{\beta}{2n}\cdot\left\|\frac{x\circ s}{\mu}\right\|_2+\beta\\
&\le\sqrt{n}\cdot\frac{\beta\cdot(1+\beta)}{2n}+\beta
\le\left(\frac{1+1/16}{2}+1\right)\beta
=\frac{49}{32}\beta.
\end{aligned} ϱ μ x ^ ∘ s ^ − 1 n 2 = ϱ μ x ^ ∘ s ^ − ϱ x ∘ s + μ x ∘ s − 1 n 2 ≤ ϱ μ x ^ ∘ s ^ − ϱ x ∘ s 2 + μ x ∘ s − 1 n 2 ≤ 2 n β ⋅ μ x ∘ s 2 + β ≤ n ⋅ 2 n β ⋅ ( 1 + β ) + β ≤ ( 2 1 + 1/16 + 1 ) β = 32 49 β . Here, we used that δ ≤ β / ( 2 n ) \delta\le\beta/(2n) δ ≤ β / ( 2 n ) β ∈ ( 0 , 1 / 16 ] \beta\in(0,1/16] β ∈ ( 0 , 1/16 ] ( x ^ , s ^ ) ∈ N 2 ( 32 β 1 − 32 β 32 ) ⊆ N 2 ( 2 β ) (\hat{x},\hat{s}) \in\mathcal{N}^{2}\left(\frac{32\beta}{1-\frac{32\beta}{32}}\right) \subseteq\mathcal{N}^{2}(2\beta) ( x ^ , s ^ ) ∈ N 2 ( 1 − 32 32 β 32 β ) ⊆ N 2 ( 2 β )
Let us now pick i ∈ I − i \in I^{-} i ∈ I − μ ‾ ( v ( i ) , u ( i ) ) \overline{\mu}(v^{(i)},u^{(i)}) μ ( v ( i ) , u ( i ) ) Proposition 5.7 , μ − \mu^{-} μ − I − I^{-} I −
0 ≤ μ − ≤ μ ‾ ( v ( i ) , u ( i ) ) . (46) 0 \le\mu^{-} \le\overline{\mu}(v^{(i)},u^{(i)}). \tag*{(46)} 0 ≤ μ − ≤ μ ( v ( i ) , u ( i ) ) . (46) We distinguish the cases μ ‾ ( v ( i ) , u ( i ) ) > 0 \overline{\mu}(v^{(i)},u^{(i)})>0 μ ( v ( i ) , u ( i ) ) > 0 μ ‾ ( v ( i ) , u ( i ) ) = 0 \overline{\mu}(v^{(i)},u^{(i)})=0 μ ( v ( i ) , u ( i ) ) = 0
Case μ ‾ ( v ( i ) , u ( i ) ) > 0 \overline{\mu}(v^{(i)},u^{(i)})>0 μ ( v ( i ) , u ( i ) ) > 0 We let
μ ⋆ : = 9 n 2 β ⋅ μ ‾ ( v ( i ) , u ( i ) ) . (47) \mu^{\star}:=\frac{9n^{2}}{\beta}\cdot\overline{\mu}(v^{(i)},u^{(i)}). \tag*{(47)} μ ⋆ := β 9 n 2 ⋅ μ ( v ( i ) , u ( i ) ) . (47) Then, size ( μ ⋆ ) = poly ( n ) size ( A , b , c ) \operatorname{size}(\mu^{\star})=\operatorname{poly}(n)\operatorname{size}(A,b,c) size ( μ ⋆ ) = poly ( n ) size ( A , b , c ) size ( μ ‾ ( v ( i ) , u ( i ) ) ) \operatorname{size}(\overline{\mu}(v^{(i)},u^{(i)})) size ( μ ( v ( i ) , u ( i ) )) 9 n 2 / β 9n^{2}/\beta 9 n 2 / β O ( log n ) O(\log n) O ( log n ) (38) and (34),
μ ⋆ < 9 n 2 β ⋅ δ 2 μ 4 n = 9 β 1024 n 3 ⋅ μ < μ . \mu^{\star}<\frac{9n^{2}}{\beta}\cdot\frac{\delta^{2}\mu}{4n}
=\frac{9\beta}{1024n^{3}}\cdot\mu<\mu. μ ⋆ < β 9 n 2 ⋅ 4 n δ 2 μ = 1024 n 3 9 β ⋅ μ < μ . Moreover, (46) gives μ − ≤ μ ( v ( i ) , u ( i ) ) = β μ ⋆ / ( 9 n 2 ) < μ ⋆ \mu^{-} \le\mu(v^{(i)},u^{(i)}) = \beta\mu^{\star}/(9n^{2}) < \mu^{\star} μ − ≤ μ ( v ( i ) , u ( i ) ) = β μ ⋆ / ( 9 n 2 ) < μ ⋆ ϱ \varrho ϱ ( 1 − ϱ ) μ − + ϱ μ = μ ⋆ (1-\varrho)\mu^{-}+\varrho\mu=\mu^{\star} ( 1 − ϱ ) μ − + ϱ μ = μ ⋆ ϱ = ( μ ⋆ − μ − ) / ( μ − μ − ) \varrho=(\mu^{\star}-\mu^{-})/(\mu-\mu^{-}) ϱ = ( μ ⋆ − μ − ) / ( μ − μ − ) μ − < μ ⋆ < μ \mu^{-}<\mu^{\star}<\mu μ − < μ ⋆ < μ ϱ ∈ ( 0 , 1 ) \varrho\in(0,1) ϱ ∈ ( 0 , 1 )
ϱ > μ ⋆ − μ − μ ≥ ( 9 n 2 β − 1 ) μ − μ ≥ 8 n 2 μ − β μ . \varrho> \frac{\mu^{\star}-\mu^{-}}{\mu} \ge\left(\frac{9n^{2}}{\beta}-1\right)\frac{\mu^{-}}{\mu} \ge\frac{8n^{2}\mu^{-}}{\beta\mu}. ϱ > μ μ ⋆ − μ − ≥ ( β 9 n 2 − 1 ) μ μ − ≥ β μ 8 n 2 μ − . using β ≤ 1 ≤ n 2 \beta\le1\le n^{2} β ≤ 1 ≤ n 2 Claim 5.12 . For ( x ^ , s ^ ) = ( 1 − ϱ ) ( x − , s − ) + ϱ ( x , s ) (\hat{x},\hat{s})=(1-\varrho)(x^{-},s^{-})+\varrho(x,s) ( x ^ , s ^ ) = ( 1 − ϱ ) ( x − , s − ) + ϱ ( x , s ) Proposition 5.7 we have μ ( x ^ , s ^ ) = μ ⋆ \mu(\hat{x},\hat{s})=\mu^{\star} μ ( x ^ , s ^ ) = μ ⋆ ( x ^ , s ^ ) ∈ N 2 ( 2 β ) (\hat{x},\hat{s})\in\mathcal{N}^{2}(2\beta) ( x ^ , s ^ ) ∈ N 2 ( 2 β ) Claim 5.12 . Since μ ⋆ > 0 \mu^{\star}>0 μ ⋆ > 0 ( x ^ , s ^ ) (\hat{x},\hat{s}) ( x ^ , s ^ )
Case μ ( v ( i ) , u ( i ) ) = 0 \mu(v^{(i)},u^{(i)})=0 μ ( v ( i ) , u ( i ) ) = 0 μ − = 0 \mu^{-}=0 μ − = 0 (46) . We output ( x ^ , s ^ ) : = ( x − , s − ) (\hat{x},\hat{s}):=(x^{-},s^{-}) ( x ^ , s ^ ) := ( x − , s − ) μ ⋆ : = 0 \mu^{\star}:=0 μ ⋆ := 0
v ^ : = s − s ^ μ , w ^ : = x − x ^ μ . \hat{v}:=\frac{s-\hat{s}}{\mu},\qquad\hat{w}:=\frac{x-\hat{x}}{\mu}. v ^ := μ s − s ^ , w ^ := μ x − x ^ . Then μ ( x ^ , s ^ ) = 0 ≤ μ ( x , s ) \mu(\hat{x},\hat{s})=0\le\mu(x,s) μ ( x ^ , s ^ ) = 0 ≤ μ ( x , s ) μ ( x ^ , s ^ ) ≤ μ ⋆ ≤ ( 1 + β / ( 4 n 2 ) ) μ ( x ^ , s ^ ) \mu(\hat{x},\hat{s})\le\mu^{\star}\le(1+\beta/(4n^{2}))\mu(\hat{x},\hat{s}) μ ( x ^ , s ^ ) ≤ μ ⋆ ≤ ( 1 + β / ( 4 n 2 )) μ ( x ^ , s ^ ) size ( μ ⋆ ) = O ( 1 ) \operatorname{size}(\mu^{\star})=O(1) size ( μ ⋆ ) = O ( 1 ) ( x ^ , s ^ , v ^ , w ^ ) (\hat{x},\hat{s},\hat{v},\hat{w}) ( x ^ , s ^ , v ^ , w ^ ) μ − = 0 \mu^{-}=0 μ − = 0 Claim 5.12 applies to every ϱ ∈ ( 0 , 1 ] \varrho\in(0,1] ϱ ∈ ( 0 , 1 ] ( x ^ , s ^ ) + ϱ μ ( w ^ , v ^ ) = ( 1 − ϱ ) ( x − , s − ) + ϱ ( x , s ) ∈ N 2 ( 2 β ) (\hat{x},\hat{s})+\varrho\mu(\hat{w},\hat{v})=(1-\varrho)(x^{-},s^{-})+\varrho(x,s)\in\mathcal{N}^{2}(2\beta) ( x ^ , s ^ ) + ϱ μ ( w ^ , v ^ ) = ( 1 − ϱ ) ( x − , s − ) + ϱ ( x , s ) ∈ N 2 ( 2 β ) ϱ ∈ ( 0 , 1 ] \varrho\in(0,1] ϱ ∈ ( 0 , 1 ] ϱ ↓ 0 \varrho\downarrow0 ϱ ↓ 0 ( x ^ , s ^ ) ∈ N 2 ( 2 β ) (\hat{x},\hat{s})\in\mathcal{N}^{2}(2\beta) ( x ^ , s ^ ) ∈ N 2 ( 2 β )
It remains to verify the conditions on ( v ^ , w ^ ) (\hat{v},\hat{w}) ( v ^ , w ^ ) x , x ^ ∈ P x,\hat{x}\in\mathcal{P} x , x ^ ∈ P s , s ^ ∈ D s,\hat{s}\in\mathcal{D} s , s ^ ∈ D A w ^ = 0 m A\hat{w}=0_{m} A w ^ = 0 m v ^ ∈ im ( A T ) \hat{v}\in\operatorname{im}(A^{\mathsf{T}}) v ^ ∈ im ( A T ) j ∈ [ n ] j\in[n] j ∈ [ n ] x ^ j ≥ ( 1 − δ ) x j > 0 \hat{x}_{j}\ge(1-\delta)x_{j}>0 x ^ j ≥ ( 1 − δ ) x j > 0 s ^ j ≥ ( 1 − δ ) s j > 0 \hat{s}_{j}\ge(1-\delta)s_{j}>0 s ^ j ≥ ( 1 − δ ) s j > 0 B ∪ N = [ n ] B\cup N=[n] B ∪ N = [ n ] B ∩ N = ∅ B\cap N=\varnothing B ∩ N = ∅ ⟨ x ^ , s ^ ⟩ = n μ − = 0 \langle\hat{x},\hat{s}\rangle=n\mu^{-}=0 ⟨ x ^ , s ^ ⟩ = n μ − = 0 x ^ , s ^ ≥ 0 n \hat{x},\hat{s}\ge0_{n} x ^ , s ^ ≥ 0 n j ∈ B j\in B j ∈ B s ^ j = 0 < ( 1 − δ ) s j \hat{s}_{j}=0<(1-\delta)s_{j} s ^ j = 0 < ( 1 − δ ) s j x ^ j ∈ [ 1 − δ , 1 + δ ] x j \hat{x}_{j}\in[1-\delta,1+\delta]x_{j} x ^ j ∈ [ 1 − δ , 1 + δ ] x j v ^ j = s j / μ > 0 \hat{v}_{j}=s_{j}/\mu>0 v ^ j = s j / μ > 0 x ^ j v ^ j = ( x ^ j / x j ) ⋅ x j s j / μ \hat{x}_{j}\hat{v}_{j}=(\hat{x}_{j}/x_{j})\cdot x_{j}s_{j}/\mu x ^ j v ^ j = ( x ^ j / x j ) ⋅ x j s j / μ j ∈ N j\in N j ∈ N s ^ j ∈ [ 1 − δ , 1 + δ ] s j \hat{s}_{j}\in[1-\delta,1+\delta]s_{j} s ^ j ∈ [ 1 − δ , 1 + δ ] s j w ^ j = x j / μ > 0 \hat{w}_{j}=x_{j}/\mu>0 w ^ j = x j / μ > 0 s ^ j w ^ j = ( s ^ j / s j ) ⋅ x j s j / μ \hat{s}_{j}\hat{w}_{j}=(\hat{s}_{j}/s_{j})\cdot x_{j}s_{j}/\mu s ^ j w ^ j = ( s ^ j / s j ) ⋅ x j s j / μ ( x ^ B ∘ v ^ B , s ^ N ∘ w ^ N ) (\hat{x}_{B}\circ\hat{v}_{B},\hat{s}_{N}\circ\hat{w}_{N}) ( x ^ B ∘ v ^ B , s ^ N ∘ w ^ N ) [ n ] [n] [ n ] p : = x ∘ s / μ p:=x\circ s/\mu p := x ∘ s / μ ( x ^ B ∘ v ^ B , s ^ N ∘ w ^ N ) = σ ∘ p (\hat{x}_{B}\circ\hat{v}_{B},\hat{s}_{N}\circ\hat{w}_{N})=\sigma\circ p ( x ^ B ∘ v ^ B , s ^ N ∘ w ^ N ) = σ ∘ p σ ∈ R n \sigma\in\mathbb{R}^{n} σ ∈ R n ∥ σ − 1 n ∥ ∞ ≤ δ \|\sigma-1_{n}\|_{\infty}\le\delta ∥ σ − 1 n ∥ ∞ ≤ δ ( x , s ) ∈ N 2 ( β ) (x,s)\in\mathcal{N}^{2}(\beta) ( x , s ) ∈ N 2 ( β ) ∥ p − 1 n ∥ 2 ≤ β \|p-1_{n}\|_{2}\le\beta ∥ p − 1 n ∥ 2 ≤ β ∥ p ∥ 2 ≤ n + β ≤ 2 n \|p\|_{2}\le\sqrt{n}+\beta\le2\sqrt{n} ∥ p ∥ 2 ≤ n + β ≤ 2 n
∥ σ ∘ p − 1 n ∥ 2 ≤ ∥ ( σ − 1 n ) ∘ p ∥ 2 + ∥ p − 1 n ∥ 2 ≤ 2 n δ + β ≤ 2 β \|\sigma\circ p-1_{n}\|_{2}\le\|(\sigma-1_{n})\circ p\|_{2}+\|p-1_{n}\|_{2}\le2\sqrt{n}\delta+\beta\le2\beta ∥ σ ∘ p − 1 n ∥ 2 ≤ ∥ ( σ − 1 n ) ∘ p ∥ 2 + ∥ p − 1 n ∥ 2 ≤ 2 n δ + β ≤ 2 β by the definition of δ \delta δ □ \square □
Remark 5.13. The path along which we ‘shoot down’ in the above proof exhibits a ‘polarized’ structure: primal variables in B B B μ \mu μ N N N N N N B B B [2 ] .
We need one more lemma to complete the proof of Theorem 1.8.
Lemma 5.14. Let D ∈ Q m × n D\in\mathbb{Q}^{m\times n} D ∈ Q m × n m ≤ n m\le n m ≤ n h ∈ Q m h\in\mathbb{Q}^{m} h ∈ Q m Q = { x ∈ R n ∣ D x = h , x ≥ 0 n } ⊂ R n Q=\{x\in\mathbb{R}^{n}\mid Dx=h,x\ge0_{n}\}\subset\mathbb{R}^{n} Q = { x ∈ R n ∣ D x = h , x ≥ 0 n } ⊂ R n J ⊆ [ n ] J\subseteq[n] J ⊆ [ n ] y ∈ Q ∩ Q n y\in Q\cap\mathbb{Q}^{n} y ∈ Q ∩ Q n y j > 0 y_{j}>0 y j > 0 j ∈ J j\in J j ∈ J R , K ∈ Z > 0 R,K\in\mathbb{Z}_{>0} R , K ∈ Z > 0 R ≥ max x ∈ Q ∥ x J / y J ∥ ∞ R\ge\max_{x\in Q}\|x_{J}/y_{J}\|_{\infty} R ≥ max x ∈ Q ∥ x J / y J ∥ ∞ z ∈ Q ∩ Q ^ n z\in Q\cap\widehat{\mathbb{Q}}^{n} z ∈ Q ∩ Q n ∥ ( z J − y J ) / y J ∥ 2 ≤ 1 / K \|(z_{J}-y_{J})/y_{J}\|_{2}\le1/K ∥ ( z J − y J ) / y J ∥ 2 ≤ 1/ K size ( z ) ≤ poly ( n ) size ( D , h ) + O ~ ( n 2 log ( K + R ) ) \operatorname{size}(z)\le\operatorname{poly}(n)\operatorname{size}(D,h)+\widetilde{O}(n^{2}\log(K+R)) size ( z ) ≤ poly ( n ) size ( D , h ) + O ( n 2 log ( K + R )) n n n log ( K + R ) \log(K+R) log ( K + R ) poly ( n ) size ( D , h , y ) + O ~ ( n 2 log ( K + R ) ) \operatorname{poly}(n)\operatorname{size}(D,h,y)+\widetilde{O}(n^{2}\log(K+R)) poly ( n ) size ( D , h , y ) + O ( n 2 log ( K + R ))
Proof. First, we apply Lemma 5.4 to obtain a Minkowski–Weyl decomposition y = ∑ i = 1 k λ i v ( i ) y=\sum_{i=1}^{k}\lambda_{i}v^{(i)} y = ∑ i = 1 k λ i v ( i ) ∑ i = 1 k λ i = 1 \sum_{i=1}^{k}\lambda_{i}=1 ∑ i = 1 k λ i = 1 λ ≥ 0 \lambda\ge0 λ ≥ 0 k ≤ n + 1 k\le n+1 k ≤ n + 1 v ( i ) v^{(i)} v ( i ) Q Q Q Lemma 5.4 , this single decomposition costs O ( n 2 ) O(n^{2}) O ( n 2 ) poly ( n ) ( size ( D , h , y ) ) \operatorname{poly}(n)(\operatorname{size}(D,h,y)) poly ( n ) ( size ( D , h , y )) v ( i ) v^{(i)} v ( i ) poly ( n ) size ( D , h ) \operatorname{poly}(n)\operatorname{size}(D,h) poly ( n ) size ( D , h ) δ : = 1 / ( 4 n 2 R K ) \delta:=1/(4n^{2}R K) δ := 1/ ( 4 n 2 R K ) λ i \lambda_{i} λ i λ ~ i \widetilde{\lambda}_{i} λ i λ ~ i \widetilde{\lambda}_{i} λ i δ \delta δ ∑ i = 1 k λ ~ i = 1 \sum_{i=1}^{k}\widetilde{\lambda}_{i}=1 ∑ i = 1 k λ i = 1 λ ~ i ≥ 0 \widetilde{\lambda}_{i}\ge0 λ i ≥ 0 ∥ λ ~ − λ ∥ ∞ ≤ δ \|\widetilde{\lambda}-\lambda\|_{\infty}\le\delta ∥ λ − λ ∥ ∞ ≤ δ λ i \lambda_{i} λ i δ \delta δ O ( n log ( 1 / δ ) ) = O ( n log ( n + K + R ) ) O(n\log(1/\delta))=O(n\log(n+K+R)) O ( n log ( 1/ δ )) = O ( n log ( n + K + R )) λ i ′ \lambda'_{i} λ i ′ 1 − ∑ i = 1 k λ i ′ = ℓ δ 1-\sum_{i=1}^{k}\lambda'_{i}=\ell\delta 1 − ∑ i = 1 k λ i ′ = ℓ δ ℓ ≤ n \ell\le n ℓ ≤ n ℓ \ell ℓ λ ~ i = λ i ′ + δ \widetilde{\lambda}_{i}=\lambda'_{i}+\delta λ i = λ i ′ + δ λ ~ i = λ i ′ \widetilde{\lambda}_{i}=\lambda'_{i} λ i = λ i ′
We claim that z : = ∑ i = 1 k λ ~ i v ( i ) z:=\sum_{i=1}^{k}\widetilde{\lambda}_{i}v^{(i)} z := ∑ i = 1 k λ i v ( i )
∥ z J − y J y J ∥ 2 ≤ ∑ i = 1 k ∣ λ i − λ ~ i ∣ ∥ v J ( i ) y J ∥ 2 ≤ 2 R n 1.5 δ < 1 K . (48) \left\|\frac{z_{J}-y_{J}}{y_{J}}\right\|_{2}\le\sum_{i=1}^{k}|\lambda_{i}-\widetilde{\lambda}_{i}|\left\|\frac{v^{(i)}_{J}}{y_{J}}\right\|_{2}\le2Rn^{1.5}\delta<\frac{1}{K}. \tag*{(48)} y J z J − y J 2 ≤ i = 1 ∑ k ∣ λ i − λ i ∣ y J v J ( i ) 2 ≤ 2 R n 1.5 δ < K 1 . (48) since each ∣ λ i − λ ~ i ∣ ≤ δ |\lambda_i-\widetilde{\lambda}_i|\le\delta ∣ λ i − λ i ∣ ≤ δ ∥ v J ( i ) / y J ∥ 2 ≤ ∣ J ∣ ∥ v J ( i ) / y J ∥ ∞ ≤ n R \left\|v_J^{(i)}/y_J\right\|_2\le\sqrt{|J|}\left\|v_J^{(i)}/y_J\right\|_\infty\le\sqrt{n}R v J ( i ) / y J 2 ≤ ∣ J ∣ v J ( i ) / y J ∞ ≤ n R R R R z z z v ( i ) v^{(i)} v ( i ) Lemma 5.4 , size bounded by poly ( n ) size ( D , h ) \operatorname{poly}(n)\operatorname{size}(D,h) poly ( n ) size ( D , h ) λ ~ i \widetilde{\lambda}_i λ i 1 / δ 2 = ( 4 n 2 R K ) 2 1/\delta^2=(4n^2RK)^2 1/ δ 2 = ( 4 n 2 R K ) 2 size ( λ ~ i ) = O ( log ( n R K ) ) \operatorname{size}(\widetilde{\lambda}_i)=O(\log(nRK)) size ( λ i ) = O ( log ( n R K )) z j z_j z j k ≤ n + 1 k\le n+1 k ≤ n + 1 λ ~ i v j ( i ) \widetilde{\lambda}_iv_j^{(i)} λ i v j ( i ) size ( z j ) ≤ 2 ∑ i = 1 k ( size ( λ ~ i ) + size ( v j ( i ) ) ) ≤ 2 ∑ i = 1 k size ( v j ( i ) ) + O ( n log ( n R K ) ) \operatorname{size}(z_j)\le2\sum_{i=1}^k(\operatorname{size}(\widetilde{\lambda}_i)+\operatorname{size}(v_j^{(i)}))\le2\sum_{i=1}^k\operatorname{size}(v_j^{(i)})+O(n\log(nRK)) size ( z j ) ≤ 2 ∑ i = 1 k ( size ( λ i ) + size ( v j ( i ) )) ≤ 2 ∑ i = 1 k size ( v j ( i ) ) + O ( n log ( n R K )) j ∈ [ n ] j\in[n] j ∈ [ n ] size ( z ) ≤ 2 ∑ i = 1 k size ( v ( i ) ) + O ( n 2 log ( n R K ) ) = poly ( n ) size ( D , h ) + O ~ ( n 2 log ( K + R ) ) \operatorname{size}(z)\le2\sum_{i=1}^k\operatorname{size}(v^{(i)})+O(n^2\log(nRK))=\operatorname{poly}(n)\operatorname{size}(D,h)+\widetilde{O}(n^2\log(K+R)) size ( z ) ≤ 2 ∑ i = 1 k size ( v ( i ) ) + O ( n 2 log ( n R K )) = poly ( n ) size ( D , h ) + O ( n 2 log ( K + R )) O ( n 2 + n log ( K + R ) ) O(n^2+n\log(K+R)) O ( n 2 + n log ( K + R ))
Proof of Theorem 1.8. Given ( x , s ) ∈ N 2 ( β ) ∩ Q 2 n (x,s)\in\mathcal{N}^2(\beta)\cap\mathbb{Q}^{2n} ( x , s ) ∈ N 2 ( β ) ∩ Q 2 n Lemma 5.10 to obtain ( x ^ , s ^ ) ∈ N 2 ( 2 β ) (\widehat{x},\widehat{s})\in\mathcal{N}^2(2\beta) ( x , s ) ∈ N 2 ( 2 β ) μ ⋆ \mu^\star μ ⋆ μ ^ ≤ μ ⋆ ≤ ( 1 + β / ( 4 n 2 ) ) μ ^ \widehat{\mu}\le\mu^\star\le(1+\beta/(4n^2))\widehat{\mu} μ ≤ μ ⋆ ≤ ( 1 + β / ( 4 n 2 )) μ μ ^ = μ ( x ^ , s ^ ) \widehat{\mu}=\mu(\widehat{x},\widehat{s}) μ = μ ( x , s ) size ( μ ⋆ ) = poly ( n ) size ( A , b , c ) \operatorname{size}(\mu^\star)=\operatorname{poly}(n)\operatorname{size}(A,b,c) size ( μ ⋆ ) = poly ( n ) size ( A , b , c ) d ∈ Q n d\in\mathbb{Q}^n d ∈ Q n A d = b Ad=b A d = b d : = A + b d:=A^+b d := A + b O ( n 5 ) O(n^5) O ( n 5 ) Lemma 2.2 (iii); as rank ( A ) = m \operatorname{rank}(A)=m rank ( A ) = m A A + = I m AA^+=I_m A A + = I m A d = b Ad=b A d = b size ( d ) = poly ( n ) size ( A , b ) \operatorname{size}(d)=\operatorname{poly}(n)\operatorname{size}(A,b) size ( d ) = poly ( n ) size ( A , b ) Lemma 2.2 (iii) and Remark 5.2. Note that Lemma 5.1(ii) does not apply here, since d d d d d d
We use Lemma 5.14 on the primal-dual polyhedron, which we write in standard form as in the proof of Lemma 5.10 : we describe D \mathcal{D} D y y y B ∈ Q n × n B\in\mathbb{Q}^{n\times n} B ∈ Q n × n ker ( B ) = im ( A ⊤ ) \ker(B)=\operatorname{im}(A^\top) ker ( B ) = im ( A ⊤ ) σ \sigma σ
Q : = { ( x ′ , s ′ , σ ) ∈ R n + n + 1 ∣ A x ′ = b , B s ′ = B c , ⟨ c , x ′ ⟩ + ⟨ d , s ′ ⟩ + σ = ⟨ c , d ⟩ + n μ ⋆ , ( x ′ , s ′ , σ ) ≥ 0 2 n + 1 } . Q:=\left\{(x',s',\sigma)\in\mathbb{R}^{n+n+1}\mid Ax'=b,\ Bs'=Bc,\ \langle c,x'\rangle+\langle d,s'\rangle+\sigma=\langle c,d\rangle+n\mu^\star,\ (x',s',\sigma)\ge0_{2n+1}\right\}. Q := { ( x ′ , s ′ , σ ) ∈ R n + n + 1 ∣ A x ′ = b , B s ′ = B c , ⟨ c , x ′ ⟩ + ⟨ d , s ′ ⟩ + σ = ⟨ c , d ⟩ + n μ ⋆ , ( x ′ , s ′ , σ ) ≥ 0 2 n + 1 } . that is, the set of primal and dual solutions with gap at most μ ⋆ \mu^\star μ ⋆ σ \sigma σ ⟨ x ′ , s ′ ⟩ ≤ n μ ⋆ \langle x',s'\rangle\le n\mu^\star ⟨ x ′ , s ′ ⟩ ≤ n μ ⋆ Proposition 5.5 , which applies since B ( c − s ′ ) = 0 n B(c-s')=0_n B ( c − s ′ ) = 0 n c − s ′ ∈ im ( A ⊤ ) c-s'\in\operatorname{im}(A^\top) c − s ′ ∈ im ( A ⊤ ) ⟨ c , x ′ ⟩ + ⟨ d , s ′ ⟩ = ⟨ c , d ⟩ + ⟨ x ′ , s ′ ⟩ \langle c,x'\rangle+\langle d,s'\rangle=\langle c,d\rangle+\langle x',s'\rangle ⟨ c , x ′ ⟩ + ⟨ d , s ′ ⟩ = ⟨ c , d ⟩ + ⟨ x ′ , s ′ ⟩ Q Q Q x ′ x' x ′ s ′ s' s ′ Proposition 5.5 , using that ⟨ c , x ′ ⟩ + ⟨ d , s ′ ⟩ = ⟨ c , d ⟩ + ⟨ x ′ , s ′ ⟩ + ⟨ x , s ⟩ − ⟨ x , s ⟩ = ⟨ c , d ⟩ + ⟨ x , s ′ ⟩ + ⟨ x ′ , s ⟩ − ⟨ x , s ⟩ \langle c,x'\rangle+\langle d,s'\rangle=\langle c,d\rangle+\langle x',s'\rangle+\langle x,s\rangle-\langle x,s\rangle=\langle c,d\rangle+\langle x,s'\rangle+\langle x',s\rangle-\langle x,s\rangle ⟨ c , x ′ ⟩ + ⟨ d , s ′ ⟩ = ⟨ c , d ⟩ + ⟨ x ′ , s ′ ⟩ + ⟨ x , s ⟩ − ⟨ x , s ⟩ = ⟨ c , d ⟩ + ⟨ x , s ′ ⟩ + ⟨ x ′ , s ⟩ − ⟨ x , s ⟩ x , s > 0 n x,s>0_n x , s > 0 n 0 ≤ σ ≤ ⟨ c , d ⟩ + n μ ⋆ − ⟨ c , x ′ ⟩ − ⟨ d , s ′ ⟩ 0\le\sigma\le\langle c,d\rangle+n\mu^\star-\langle c,x'\rangle-\langle d,s'\rangle 0 ≤ σ ≤ ⟨ c , d ⟩ + n μ ⋆ − ⟨ c , x ′ ⟩ − ⟨ d , s ′ ⟩ x ′ x' x ′ s ′ s' s ′ Q Q Q
We will apply Lemma 5.14 to the polytope Q Q Q K K K K ≥ 8 n / β K\ge8n/\beta K ≥ 8 n / β ( x ^ , s ^ , σ ^ ) ∈ Q ∩ Q 2 n + 1 (\widehat{x},\widehat{s},\widehat{\sigma})\in Q\cap\mathbb{Q}^{2n+1} ( x , s , σ ) ∈ Q ∩ Q 2 n + 1 σ ^ : = ⟨ c , d ⟩ + n μ ⋆ − ⟨ c , x ^ ⟩ − ⟨ d , s ^ ⟩ = n ( μ ⋆ − μ ^ ) ≥ 0 \widehat{\sigma}:=\langle c,d\rangle+n\mu^\star-\langle c,\widehat{x}\rangle-\langle d,\widehat{s}\rangle=n(\mu^\star-\widehat{\mu})\ge0 σ := ⟨ c , d ⟩ + n μ ⋆ − ⟨ c , x ⟩ − ⟨ d , s ⟩ = n ( μ ⋆ − μ ) ≥ 0 Proposition 5.5 . As the index set we take
J : = { j ∈ [ n ] ∣ x ^ j > 0 } ∪ { n + j ∣ j ∈ [ n ] , s ^ j > 0 } ⊆ [ 2 n + 1 ] . J:=\{j\in[n]\mid\widehat{x}_j>0\}\cup\{n+j\mid j\in[n],\ \widehat{s}_j>0\}\subseteq[2n+1]. J := { j ∈ [ n ] ∣ x j > 0 } ∪ { n + j ∣ j ∈ [ n ] , s j > 0 } ⊆ [ 2 n + 1 ] . that is, the x ′ x' x ′ s ′ s' s ′ ( x ^ , s ^ ) (\widehat{x},\widehat{s}) ( x , s ) R R R D , h D,h D , h Q Q Q A , B , b , c , d A,B,b,c,d A , B , b , c , d μ ⋆ \mu^\star μ ⋆ Lemma 5.10 , B B B O ( n 4 ) O(n^4) O ( n 4 ) size ( B ) = poly ( n ) size ( A ) \operatorname{size}(B)=\operatorname{poly}(n)\operatorname{size}(A) size ( B ) = poly ( n ) size ( A ) size ( B c ) = poly ( n ) size ( A , c ) \operatorname{size}(Bc)=\operatorname{poly}(n)\operatorname{size}(A,c) size ( B c ) = poly ( n ) size ( A , c ) Q Q Q O ( size ( A , B , b , B c , c , d , μ ⋆ ) ) = poly ( n ) size ( A , b , c ) O(\operatorname{size}(A,B,b,Bc,c,d,\mu^\star))=\operatorname{poly}(n)\operatorname{size}(A,b,c) O ( size ( A , B , b , B c , c , d , μ ⋆ )) = poly ( n ) size ( A , b , c )
We distinguish the cases μ ⋆ > 0 \mu^\star>0 μ ⋆ > 0 μ ⋆ = 0 \mu^\star=0 μ ⋆ = 0 Lemma 5.10 .
Case μ ⋆ > 0 \mu^\star>0 μ ⋆ > 0 Then ( x ^ , s ^ ) ∈ N 2 ( 2 β ) (\widehat{x},\widehat{s})\in\mathcal{N}^2(2\beta) ( x , s ) ∈ N 2 ( 2 β ) Lemma 5.10 (i); in particular x ^ , s ^ > 0 n \widehat{x},\widehat{s}>0_n x , s > 0 n J J J x ′ x' x ′ s ′ s' s ′ ( x , s , σ ) ∈ Q (x,s,\sigma)\in Q ( x , s , σ ) ∈ Q μ ( x , s ) ≤ μ ⋆ ≤ ( 1 + β / ( 4 n 2 ) ) μ ^ \mu(x,s)\le\mu^\star\le(1+\beta/(4n^2))\widehat{\mu} μ ( x , s ) ≤ μ ⋆ ≤ ( 1 + β / ( 4 n 2 )) μ Lemma 5.6 with μ = μ ^ \mu=\widehat{\mu} μ = μ Proposition 5.8 with 2 β 2\beta 2 β β \beta β
⟨ x , s ⟩ ≤ n ( 2 + β 4 n 2 ) ⟨ x c p ( μ ^ ) , s c p ( μ ^ ) ⟩ ≤ n 1 − 2 β ( 2 + β 4 n 2 ) ⟨ x ^ , s ^ ⟩ ≤ 3 n ⟨ x ^ , s ^ ⟩ , \langle x,s\rangle\le n\left(2+\frac{\beta}{4n^2}\right)\langle x^{\mathrm{cp}}(\widehat{\mu}),s^{\mathrm{cp}}(\widehat{\mu})\rangle\le\frac{n}{1-2\beta}\left(2+\frac{\beta}{4n^2}\right)\langle\widehat{x},\widehat{s}\rangle\le3n\langle\widehat{x},\widehat{s}\rangle, ⟨ x , s ⟩ ≤ n ( 2 + 4 n 2 β ) ⟨ x cp ( μ ) , s cp ( μ )⟩ ≤ 1 − 2 β n ( 2 + 4 n 2 β ) ⟨ x , s ⟩ ≤ 3 n ⟨ x , s ⟩ , using β ≤ 1 / 16 \beta\le1/16 β ≤ 1/16 R : = 3 n R:=3n R := 3 n R R R ( x ′ , s ′ ) (x',s') ( x ′ , s ′ ) size ( x ′ , s ′ ) = poly ( n ) size ( A , b , c ) \operatorname{size}(x',s')=\operatorname{poly}(n)\operatorname{size}(A,b,c) size ( x ′ , s ′ ) = poly ( n ) size ( A , b , c )
∥ x ′ x ^ − 1 n ∥ 2 + ∥ s ′ s ^ − 1 n ∥ 2 ≤ 2 ( ∥ x ′ x ^ − 1 n ∥ 2 2 + ∥ s ′ s ^ − 1 n ∥ 2 2 ) 1 / 2 ≤ 2 K ≤ β 4 n . (49) \left\|\frac{x'}{\widehat{x}}-1_n\right\|_2+\left\|\frac{s'}{\widehat{s}}-1_n\right\|_2\le\sqrt{2}\left(\left\|\frac{x'}{\widehat{x}}-1_n\right\|_2^2+\left\|\frac{s'}{\widehat{s}}-1_n\right\|_2^2\right)^{1/2}\le\frac{\sqrt{2}}{K}\le\frac{\beta}{4n}. \tag*{(49)} x x ′ − 1 n 2 + s s ′ − 1 n 2 ≤ 2 ( x x ′ − 1 n 2 2 + s s ′ − 1 n 2 2 ) 1/2 ≤ K 2 ≤ 4 n β . (49) For μ ′ : = μ ( x ′ , s ′ ) \mu':=\mu(x',s') μ ′ := μ ( x ′ , s ′ ) μ ′ / μ ^ \mu'/\widehat{\mu} μ ′ / μ
( 1 + β 4 n 2 ) μ ^ ≥ μ ⋆ ≥ μ ′ = 1 n ∑ i = 1 n x i ′ s i ′ ≥ ( 1 − 1 / K ) 2 n ∑ i = 1 n x ^ i s ^ i ≥ ( 1 − β 4 n ) μ ^ . (50) \left(1+\frac{\beta}{4n^2}\right)\widehat{\mu}\ge\mu^\star\ge\mu'=\frac{1}{n}\sum_{i=1}^n x_i's_i'\ge\frac{(1-1/K)^2}{n}\sum_{i=1}^n\widehat{x}_i\widehat{s}_i\ge\left(1-\frac{\beta}{4n}\right)\widehat{\mu}. \tag*{(50)} ( 1 + 4 n 2 β ) μ ≥ μ ⋆ ≥ μ ′ = n 1 i = 1 ∑ n x i ′ s i ′ ≥ n ( 1 − 1/ K ) 2 i = 1 ∑ n x i s i ≥ ( 1 − 4 n β ) μ . (50) We are ready to show that ( x ′ , s ′ ) ∈ N 2 ( 3 β ) (x',s') \in\mathcal{N}^{2}(3\beta) ( x ′ , s ′ ) ∈ N 2 ( 3 β )
∥ x ′ ∘ s ′ μ ′ − 1 n ∥ 2 = ∥ ( x ′ x ^ − 1 n + 1 n ) ∘ ( s ′ s ^ − 1 n + 1 n ) ∘ x ^ ∘ s ^ μ ′ − 1 n ∥ 2 ≤ ( ∥ x ′ x ^ − 1 n ∥ 2 + ∥ s ′ s ^ − 1 n ∥ 2 + ∥ x ′ x ^ − 1 n ∥ 2 ∥ s ′ s ^ − 1 n ∥ 2 ) ∥ x ^ ∘ s ^ μ ′ ∥ ∞ + ∥ x ^ ∘ s ^ μ ′ − 1 n ∥ 2 (51) \begin{aligned}
\left\|\frac{x'\circ s'}{\mu'}-1_n\right\|_2
=
\left\|\left(\frac{x'}{\hat{x}}-1_n+1_n\right)\circ\left(\frac{s'}{\hat{s}}-1_n+1_n\right)\circ\frac{\hat{x}\circ\hat{s}}{\mu'}-1_n\right\|_2 \\
&\leq
\left(\left\|\frac{x'}{\hat{x}}-1_n\right\|_2+
\left\|\frac{s'}{\hat{s}}-1_n\right\|_2+
\left\|\frac{x'}{\hat{x}}-1_n\right\|_2
\left\|\frac{s'}{\hat{s}}-1_n\right\|_2\right)
\left\|\frac{\hat{x}\circ\hat{s}}{\mu'}\right\|_\infty+
\left\|\frac{\hat{x}\circ\hat{s}}{\mu'}-1_n\right\|_2 \tag*{(51)}
\end{aligned} μ ′ x ′ ∘ s ′ − 1 n 2 = ( x ^ x ′ − 1 n + 1 n ) ∘ ( s ^ s ′ − 1 n + 1 n ) ∘ μ ′ x ^ ∘ s ^ − 1 n 2 ≤ ( x ^ x ′ − 1 n 2 + s ^ s ′ − 1 n 2 + x ^ x ′ − 1 n 2 s ^ s ′ − 1 n 2 ) μ ′ x ^ ∘ s ^ ∞ + μ ′ x ^ ∘ s ^ − 1 n 2 (51) Using (49) , and bringing out a factor μ ^ / μ ′ \hat{\mu}/\mu' μ ^ / μ ′
∥ x ′ ∘ s ′ μ ′ − 1 n ∥ 2 ≤ μ ^ μ ′ ( β 4 n + ( β 4 n ) 2 ) ∥ x ^ ∘ s ^ μ ^ ∥ ∞ + μ ^ μ ′ ∥ x ^ ∘ s ^ μ ^ − μ ′ μ ^ 1 n ∥ 2 ≤ μ ^ μ ′ ( β 4 n ( 1 + β 4 n ) ( 1 + 2 β ) + ∥ x ^ ∘ s ^ μ ^ − 1 n ∥ 2 + ∥ μ ′ μ ^ 1 n − 1 n ∥ 2 ) ≤ ( 1 − β 4 n ) − 2 ( β 4 n ( 1 + β 4 n ) ( 1 + 2 β ) + 2 β + n β 2 n ) = ( 1 − β 4 n ) − 2 ( ( 1 + β 4 n ) ( 1 + 2 β ) 4 n + 2 + 1 2 n ) β ≤ ( 1 − 1 64 ) − 2 ( 1 4 ⋅ 65 64 ⋅ 9 8 + 2 + 1 2 ) β \begin{aligned}
\left\|\frac{x'\circ s'}{\mu'}-1_n\right\|_2
\leq
\frac{\hat{\mu}}{\mu'}
\left(\frac{\beta}{4n}+\left(\frac{\beta}{4n}\right)^2\right)
\left\|\frac{\hat{x}\circ\hat{s}}{\hat{\mu}}\right\|_\infty
+
\frac{\hat{\mu}}{\mu'}
\left\|\frac{\hat{x}\circ\hat{s}}{\hat{\mu}}-\frac{\mu'}{\hat{\mu}}1_n\right\|_2 \\
&\leq
\frac{\hat{\mu}}{\mu'}
\left(\frac{\beta}{4n}\left(1+\frac{\beta}{4n}\right)(1+2\beta)
+
\left\|\frac{\hat{x}\circ\hat{s}}{\hat{\mu}}-1_n\right\|_2
+
\left\|\frac{\mu'}{\hat{\mu}}1_n-1_n\right\|_2\right) \\
&\leq
\left(1-\frac{\beta}{4n}\right)^{-2}
\left(\frac{\beta}{4n}\left(1+\frac{\beta}{4n}\right)(1+2\beta)
+2\beta+\sqrt{n}\frac{\beta}{2n}\right) \\
&=
\left(1-\frac{\beta}{4n}\right)^{-2}
\left(\frac{\left(1+\frac{\beta}{4n}\right)(1+2\beta)}{4n}
+2+\frac{1}{2\sqrt{n}}\right)\beta\\
&\leq
\left(1-\frac{1}{64}\right)^{-2}
\left(\frac{1}{4}\cdot\frac{65}{64}\cdot\frac{9}{8}+2+\frac{1}{2}\right)\beta
\end{aligned} μ ′ x ′ ∘ s ′ − 1 n 2 ≤ μ ′ μ ^ ( 4 n β + ( 4 n β ) 2 ) μ ^ x ^ ∘ s ^ ∞ + μ ′ μ ^ μ ^ x ^ ∘ s ^ − μ ^ μ ′ 1 n 2 ≤ μ ′ μ ^ ( 4 n β ( 1 + 4 n β ) ( 1 + 2 β ) + μ ^ x ^ ∘ s ^ − 1 n 2 + μ ^ μ ′ 1 n − 1 n 2 ) ≤ ( 1 − 4 n β ) − 2 ( 4 n β ( 1 + 4 n β ) ( 1 + 2 β ) + 2 β + n 2 n β ) = ( 1 − 4 n β ) − 2 4 n ( 1 + 4 n β ) ( 1 + 2 β ) + 2 + 2 n 1 β ≤ ( 1 − 64 1 ) − 2 ( 4 1 ⋅ 64 65 ⋅ 8 9 + 2 + 2 1 ) β ≤ 3 β . (52) \leq3\beta. \tag*{(52)} ≤ 3 β . (52) Here, the first inequality used (49) , the second used that ( x ^ , s ^ ) ∈ N 2 ( 2 β ) (\hat{x},\hat{s})\in\mathcal{N}^{2}(2\beta) ( x ^ , s ^ ) ∈ N 2 ( 2 β ) ( x ^ , s ^ ) ∈ N 2 ( 2 β ) (\hat{x},\hat{s})\in\mathcal{N}^{2}(2\beta) ( x ^ , s ^ ) ∈ N 2 ( 2 β ) (50) on μ ′ / μ ^ \mu'/\hat{\mu} μ ′ / μ ^ ( x ′ , s ′ ) ∈ N 2 ( 3 β ) (x',s')\in\mathcal{N}^{2}(3\beta) ( x ′ , s ′ ) ∈ N 2 ( 3 β )
We set ( x ~ , s ~ ) : = ( x ′ , s ′ ) ∈ Q ^ 2 n (\tilde{x},\tilde{s}):=(x',s')\in\widehat{\mathcal{Q}}^{2n} ( x ~ , s ~ ) := ( x ′ , s ′ ) ∈ Q 2 n poly ( n ) size ( A , b , c ) \operatorname{poly}(n)\operatorname{size}(A,b,c) poly ( n ) size ( A , b , c ) (50) and μ ˉ ( x ^ , s ^ ) ≤ μ ˉ ( x , s ) \bar{\mu}(\hat{x},\hat{s})\leq\bar{\mu}(x,s) μ ˉ ( x ^ , s ^ ) ≤ μ ˉ ( x , s )
μ ˉ ( x ~ , s ~ ) = μ ′ ≤ μ ⋆ ≤ ( 1 + β 4 n 2 ) μ ^ ≤ ( 1 + β 4 n 2 ) μ ˉ ( x , s ) . \bar{\mu}(\tilde{x},\tilde{s})=\mu'\leq\mu^\star\leq\left(1+\frac{\beta}{4n^2}\right)\hat{\mu}\leq\left(1+\frac{\beta}{4n^2}\right)\bar{\mu}(x,s). μ ˉ ( x ~ , s ~ ) = μ ′ ≤ μ ⋆ ≤ ( 1 + 4 n 2 β ) μ ^ ≤ ( 1 + 4 n 2 β ) μ ˉ ( x , s ) . This proves the theorem in this case.
Case μ ⋆ = 0 \mu^\star=0 μ ⋆ = 0 Then μ ˉ ( x ^ , s ^ ) = 0 \bar{\mu}(\hat{x},\hat{s})=0 μ ˉ ( x ^ , s ^ ) = 0 ( x ^ , s ^ ) (\hat{x},\hat{s}) ( x ^ , s ^ ) Q \mathcal{Q} Q ( x ′ , s ′ , 0 ) (x',s',0) ( x ′ , s ′ , 0 ) ( x ′ , s ′ ) (x',s') ( x ′ , s ′ ) σ = − ⟨ x ′ , s ′ ⟩ \sigma=-\langle x',s'\rangle σ = − ⟨ x ′ , s ′ ⟩ Q \mathcal{Q} Q ( v ^ , w ^ ) (\hat{v},\hat{w}) ( v ^ , w ^ ) B : = supp ( x ^ ) B:=\operatorname{supp}(\hat{x}) B := supp ( x ^ ) N : = supp ( s ^ ) N:=\operatorname{supp}(\hat{s}) N := supp ( s ^ ) [ n ] [n] [ n ] v ^ ∈ im ( A T ) \hat{v}\in\operatorname{im}(A^\mathsf{T}) v ^ ∈ im ( A T ) v ^ B > 0 B \hat{v}_B>0_B v ^ B > 0 B w ^ ∈ ker ( A ) \hat{w}\in\operatorname{ker}(A) w ^ ∈ ker ( A ) w ^ N > 0 N \hat{w}_N>0_N w ^ N > 0 N q ^ : = ( x ^ B ∘ v ^ B , s ^ N ∘ w ^ N ) \hat{q}:=(\hat{x}_B\circ\hat{v}_B,\hat{s}_N\circ\hat{w}_N) q ^ := ( x ^ B ∘ v ^ B , s ^ N ∘ w ^ N ) ∥ q ^ − 1 n ∥ 2 ≤ 2 β \|\hat{q}-1_n\|_2\leq2\beta ∥ q ^ − 1 n ∥ 2 ≤ 2 β J J J x ′ x' x ′ B B B s ′ s' s ′ N N N
We first show, using the certificates ( v ^ , w ^ ) (\hat{v},\hat{w}) ( v ^ , w ^ ) ( x ′ , s ′ ) (x',s') ( x ′ , s ′ ) ( x ′ , s ′ ) ≤ 2 n ( x ^ , s ^ ) (x',s')\leq2n(\hat{x},\hat{s}) ( x ′ , s ′ ) ≤ 2 n ( x ^ , s ^ ) ⟨ x ′ , s ^ ⟩ + ⟨ x ^ , s ′ ⟩ = ⟨ x ′ , s ′ ⟩ + ⟨ x ^ , s ^ ⟩ = 0 \langle x',\hat{s}\rangle+\langle\hat{x},s'\rangle=\langle x',s'\rangle+\langle\hat{x},\hat{s}\rangle=0 ⟨ x ′ , s ^ ⟩ + ⟨ x ^ , s ′ ⟩ = ⟨ x ′ , s ′ ⟩ + ⟨ x ^ , s ^ ⟩ = 0 x N ′ = 0 N x'_N=0_N x N ′ = 0 N s B ′ = 0 B s'_B=0_B s B ′ = 0 B x ′ − x ^ ∈ ker ( A ) x'-\hat{x}\in\operatorname{ker}(A) x ′ − x ^ ∈ ker ( A ) v ^ ∈ im ( A T ) \hat{v}\in\operatorname{im}(A^\mathsf{T}) v ^ ∈ im ( A T ) x ′ x' x ′ x ^ \hat{x} x ^ N N N j ∈ B j\in B j ∈ B
x j ′ v ^ j ≤ ⟨ x B ′ , v ^ B ⟩ = ⟨ x ^ B , v ^ B ⟩ ≤ ∥ q ^ ∥ 1 ≤ n + 2 β n . x'_j\hat{v}_j\leq\langle x'_B,\hat{v}_B\rangle=\langle\hat{x}_B,\hat{v}_B\rangle\leq\|\hat{q}\|_1\leq n+2\beta\sqrt{n}. x j ′ v ^ j ≤ ⟨ x B ′ , v ^ B ⟩ = ⟨ x ^ B , v ^ B ⟩ ≤ ∥ q ^ ∥ 1 ≤ n + 2 β n . As x ^ j v ^ j = q ^ j ≥ 1 − 2 β \hat{x}_j\hat{v}_j=\hat{q}_j\geq1-2\beta x ^ j v ^ j = q ^ j ≥ 1 − 2 β x j ′ ≤ n + 2 β n 1 − 2 β x ^ j ≤ 2 n x ^ j x'_j\leq\frac{n+2\beta\sqrt{n}}{1-2\beta}\hat{x}_j\leq2n\hat{x}_j x j ′ ≤ 1 − 2 β n + 2 β n x ^ j ≤ 2 n x ^ j β ≤ 1 / 16 \beta\leq1/16 β ≤ 1/16 s N ′ ≤ 2 n s ^ N s'_N\leq2n\hat{s}_N s N ′ ≤ 2 n s ^ N ⟨ w ^ , s ′ − s ^ ⟩ = 0 \langle\hat{w},s'-\hat{s}\rangle=0 ⟨ w ^ , s ′ − s ^ ⟩ = 0
We apply Lemma 5.14 with R = 2 n R=2n R = 2 n ( x ′ , s ′ , σ ′ ) ∈ Q ∩ Q ^ 2 n + 1 (x',s',\sigma')\in\mathcal{Q}\cap\widehat{\mathcal{Q}}^{2n+1} ( x ′ , s ′ , σ ′ ) ∈ Q ∩ Q 2 n + 1 size ( x ′ , s ′ ) = poly ( n ) size ( A , b , c ) \operatorname{size}(x',s')=\operatorname{poly}(n)\operatorname{size}(A,b,c) size ( x ′ , s ′ ) = poly ( n ) size ( A , b , c ) ∥ r − 1 n ∥ 2 ≤ β / ( 8 n ) \|r-1_n\|_2\leq\beta/(8n) ∥ r − 1 n ∥ 2 ≤ β / ( 8 n ) r : = ( x B ′ / x ^ B , s N ′ / s ^ N ) r:=(x'_B/\hat{x}_B,s'_N/\hat{s}_N) r := ( x B ′ / x ^ B , s N ′ / s ^ N ) x B ′ > 0 B x'_B>0_B x B ′ > 0 B s N ′ > 0 N s'_N>0_N s N ′ > 0 N v ∈ im ( A T ) v\in\operatorname{im}(A^\mathsf{T}) v ∈ im ( A T ) ∥ x B ′ ∘ v B − 1 B ∥ 2 \|x'_B\circ v_B-1_B\|_2 ∥ x B ′ ∘ v B − 1 B ∥ 2 w ∈ ker ( A ) w\in\operatorname{ker}(A) w ∈ ker ( A ) ∥ s N ′ ∘ w N − 1 N ∥ 2 \|s'_N\circ w_N-1_N\|_2 ∥ s N ′ ∘ w N − 1 N ∥ 2 v = A T M − 1 1 B v=A^\mathsf{T}M^{-1}1_B v = A T M − 1 1 B M : = Diag ( x B ′ ) A B T M:=\operatorname{Diag}(x'_B)A^\mathsf{T}_B M := Diag ( x B ′ ) A B T w = Π ( M ′ ) − 1 1 N w=\Pi(M')^{-1}1_N w = Π ( M ′ ) − 1 1 N Π : = I n − A T ( A A T ) + A \Pi:=I_n-A^\mathsf{T}(AA^\mathsf{T})^+A Π := I n − A T ( A A T ) + A ker ( A ) \operatorname{ker}(A) ker ( A ) M ′ : = Diag ( s N ′ ) Π N , ∙ M':=\operatorname{Diag}(s'_N)\Pi_{N,\bullet} M ′ := Diag ( s N ′ ) Π N , ∙ v v v w w w poly ( n ) size ( A , b , c ) \operatorname{poly}(n)\operatorname{size}(A,b,c) poly ( n ) size ( A , b , c ) v ^ \hat{v} v ^ w ^ \hat{w} w ^ ( x B ′ ∘ v ^ B , s N ′ ∘ w ^ N ) = r ∘ q ^ (x'_B\circ\hat{v}_B,s'_N\circ\hat{w}_N)=r\circ\hat{q} ( x B ′ ∘ v ^ B , s N ′ ∘ w ^ N ) = r ∘ q ^ v v v w w w q : = ( x B ′ ∘ v B , s N ′ ∘ w N ) q:=(x'_B\circ v_B,s'_N\circ w_N) q := ( x B ′ ∘ v B , s N ′ ∘ w N )
∥ q − 1 n ∥ 2 ≤ ∥ r ∘ q ^ − 1 n ∥ 2 ≤ ∥ q ^ ∥ ∞ ∥ r − 1 n ∥ 2 + ∥ q ^ − 1 n ∥ 2 ≤ ( 1 + 2 β ) β 8 n + 2 β ≤ 137 64 β = : θ . \|q-1_n\|_2\leq\|r\circ\hat{q}-1_n\|_2\leq\|\hat{q}\|_\infty\|r-1_n\|_2+\|\hat{q}-1_n\|_2\leq(1+2\beta)\frac{\beta}{8n}+2\beta\leq\frac{137}{64}\beta=:\theta. ∥ q − 1 n ∥ 2 ≤ ∥ r ∘ q ^ − 1 n ∥ 2 ≤ ∥ q ^ ∥ ∞ ∥ r − 1 n ∥ 2 + ∥ q ^ − 1 n ∥ 2 ≤ ( 1 + 2 β ) 8 n β + 2 β ≤ 64 137 β =: θ . As θ < 1 \theta< 1 θ < 1 q > 0 N q > 0_N q > 0 N v B > 0 B v_B > 0_B v B > 0 B w N > 0 N w_N > 0_N w N > 0 N
We set ( x ~ , s ~ ) : = ( x ′ , s ′ ) (\widetilde{x},\widetilde{s}) := (x',s') ( x , s ) := ( x ′ , s ′ ) μ ‾ ( x ~ , s ~ ) = 0 ≤ ( 1 + β / ( 4 n 2 ) ) μ ‾ ( x , s ) \overline{\mu}(\widetilde{x},\widetilde{s}) = 0 \le(1+\beta/(4n^2))\overline{\mu}(x,s) μ ( x , s ) = 0 ≤ ( 1 + β / ( 4 n 2 )) μ ( x , s ) ( x ′ , s ′ ) ∈ N 2 ( 3 β ) (x',s') \in\mathcal{N}^2(3\beta) ( x ′ , s ′ ) ∈ N 2 ( 3 β ) t > 0 t > 0 t > 0 z ( t ) : = ( x ′ + t w , s ′ + t v ) z(t) := (x' + tw,s' + tv) z ( t ) := ( x ′ + tw , s ′ + t v ) w ∈ ker ( A ) w \in\ker(A) w ∈ ker ( A ) v ∈ im ( A ⊤ ) v \in\operatorname{im}(A^\top) v ∈ im ( A ⊤ ) z ( t ) z(t) z ( t ) P × D \mathcal{P} \times\mathcal{D} P × D x N ′ = 0 N x'_N = 0_N x N ′ = 0 N s B ′ = 0 B s'_B = 0_B s B ′ = 0 B x B ′ , v B , s N ′ , w N > 0 x'_B,v_B,s'_N,w_N > 0 x B ′ , v B , s N ′ , w N > 0 z ( t ) > 0 2 n z(t) > 0_{2n} z ( t ) > 0 2 n t > 0 t > 0 t > 0 ( x ′ + t w ) ∘ ( s ′ + t v ) = t ( q + t w ∘ v ) (x' + tw) \circ(s' + tv) = t(q + t\,w \circ v) ( x ′ + tw ) ∘ ( s ′ + t v ) = t ( q + t w ∘ v ) μ ′ = t \mu' = t μ ′ = t θ t : = θ + t ∥ w ∘ v ∥ 2 ≤ 1 / 4 \theta_t := \theta+ t\lVert w \circ v\rVert_2 \le1/4 θ t := θ + t ∥ w ∘ v ∥ 2 ≤ 1/4 z ( t ) ∈ N 2 ( θ t / ( 1 − θ t ) ) z(t) \in\mathcal{N}^2(\theta_t/(1-\theta_t)) z ( t ) ∈ N 2 ( θ t / ( 1 − θ t )) t > 0 t > 0 t > 0 θ / ( 1 − θ ) ≤ 137 64 β / ( 1 − 137 1024 ) < 3 β \theta/(1-\theta) \le\frac{137}{64}\beta/(1-\frac{137}{1024}) < 3\beta θ / ( 1 − θ ) ≤ 64 137 β / ( 1 − 1024 137 ) < 3 β z ( t ) ∈ N 2 ( 3 β ) z(t) \in\mathcal{N}^2(3\beta) z ( t ) ∈ N 2 ( 3 β ) t > 0 t > 0 t > 0 t ↓ 0 t \downarrow0 t ↓ 0 ( x ~ , s ~ ) ∈ N 2 ( 3 β ) (\widetilde{x},\widetilde{s}) \in\mathcal{N}^2(3\beta) ( x , s ) ∈ N 2 ( 3 β ) ( x ′ , s ′ , v , w ) (x',s',v,w) ( x ′ , s ′ , v , w ) Theorem 1.6 with θ \theta θ β \beta β poly ( n ) size ( A , b , c ) \operatorname{poly}(n)\operatorname{size}(A,b,c) poly ( n ) size ( A , b , c ) θ = 137 64 β ≤ 3 β \theta= \frac{137}{64}\beta\le3\beta θ = 64 137 β ≤ 3 β ( v ~ , w ~ ) : = ( v , w ) (\widetilde{v},\widetilde{w}) := (v,w) ( v , w ) := ( v , w ) μ ‾ ( x ~ , s ~ ) = 0 \overline{\mu}(\widetilde{x},\widetilde{s}) = 0 μ ( x , s ) = 0
Finally, we show that using the rounding step in Theorem 1.8 in combination with any path-following IPM already guarantees termination in an exponential number of steps. We state this formally for methods in the ℓ 2 \ell_2 ℓ 2 F ( x , s ) F(x,s) F ( x , s ) Lemma 5.10 and Lemma 5.14.
Proposition 5.15. Let β ∈ ( 1 / poly ( n ) , 1 / 16 ] \beta\in(1/\operatorname{poly}(n),1/16] β ∈ ( 1/ poly ( n ) , 1/16 ] ( x , s ) ∈ N 2 ( 3 β ) (x,s) \in\mathcal{N}^2(3\beta) ( x , s ) ∈ N 2 ( 3 β ) F ( x , s ) = ( x ′ , s ′ ) ∈ N 2 ( β ) F(x,s) = (x',s') \in\mathcal{N}^2(\beta) F ( x , s ) = ( x ′ , s ′ ) ∈ N 2 ( β ) μ ‾ ( x ′ , s ′ ) ≤ ( 1 − 2 β / n 2 ) μ ‾ ( x , s ) \overline{\mu}(x',s') \le(1-2\beta/n^2)\overline{\mu}(x,s) μ ( x ′ , s ′ ) ≤ ( 1 − 2 β / n 2 ) μ ( x , s ) ( x 0 , s 0 ) ∈ N 2 ( β ) (x^0,s^0) \in\mathcal{N}^2(\beta) ( x 0 , s 0 ) ∈ N 2 ( β ) F F F 2 2 n poly ( n ) 2^{2n}\operatorname{poly}(n) 2 2 n poly ( n ) F F F
Proof. Let H ⊆ R ≥ 0 ∪ { ∞ } H \subseteq\mathbb{R}_{\ge0} \cup\{\infty\} H ⊆ R ≥ 0 ∪ { ∞ } μ ‾ ( v , u ) \overline{\mu}(v,u) μ ( v , u ) v v v P \mathcal{P} P u u u D \mathcal{D} D ∞ \infty ∞ ∣ H ∣ ≤ 2 2 n + 1 |H| \le2^{2n}+1 ∣ H ∣ ≤ 2 2 n + 1 H = { 0 = h 0 < h 1 < h 2 < … < h T < h T + 1 = ∞ } H = \{0 = h_0 < h_1 < h_2 < \ldots< h_T < h_{T+1} = \infty\} H = { 0 = h 0 < h 1 < h 2 < … < h T < h T + 1 = ∞ } F F F N 2 ( 3 β ) \mathcal{N}^2(3\beta) N 2 ( 3 β ) ( 1 + β / ( 4 n 2 ) ) ( 1 − 2 β / n 2 ) ≤ 1 − β / n 2 (1+\beta/(4n^2))(1-2\beta/n^2) \le1-\beta/n^2 ( 1 + β / ( 4 n 2 )) ( 1 − 2 β / n 2 ) ≤ 1 − β / n 2 β > 1 / poly ( n ) \beta> 1/\operatorname{poly}(n) β > 1/ poly ( n ) poly ( n ) \operatorname{poly}(n) poly ( n ) poly ( n ) \operatorname{poly}(n) poly ( n )
We claim that if the current point ( x , s ) (x,s) ( x , s ) μ ‾ ( x , s ) ≤ h t \overline{\mu}(x,s) \le h_t μ ( x , s ) ≤ h t t ∈ [ T + 1 ] t \in[T+1] t ∈ [ T + 1 ] poly ( n ) \operatorname{poly}(n) poly ( n ) ( x ˉ , s ˉ ) (\bar{x},\bar{s}) ( x ˉ , s ˉ ) μ ‾ ( x ˉ , s ˉ ) ≤ h t − 1 \overline{\mu}(\bar{x},\bar{s}) \le h_{t-1} μ ( x ˉ , s ˉ ) ≤ h t − 1 h 0 = 0 h_0 = 0 h 0 = 0 t = T + 1 , T , … , 1 t = T+1,T,\ldots,1 t = T + 1 , T , … , 1 F F F poly ( n ) \operatorname{poly}(n) poly ( n ) 2 2 n poly ( n ) 2^{2n}\operatorname{poly}(n) 2 2 n poly ( n )
Let δ = β / ( 16 n 2 ) \delta= \beta/(16n^2) δ = β / ( 16 n 2 ) poly ( n ) \operatorname{poly}(n) poly ( n ) ( x ′ , s ′ ) (x',s') ( x ′ , s ′ ) μ ‾ ( x ′ , s ′ ) < h t δ 2 / ( 4 n ) \overline{\mu}(x',s') < h_t\delta^2/(4n) μ ( x ′ , s ′ ) < h t δ 2 / ( 4 n ) μ ‾ ( x ′ , s ′ ) ≤ h t − 1 ⋅ 4 n / δ 2 \overline{\mu}(x',s') \le h_{t-1}\cdot4n/\delta^2 μ ( x ′ , s ′ ) ≤ h t − 1 ⋅ 4 n / δ 2 poly ( n ) \operatorname{poly}(n) poly ( n ) h t − 1 h_{t-1} h t − 1 t = 1 t=1 t = 1 h 0 = 0 h_0=0 h 0 = 0
Otherwise, μ ‾ : = μ ‾ ( x ′ , s ′ ) \overline{\mu} := \overline{\mu}(x',s') μ := μ ( x ′ , s ′ ) h t − 1 ⋅ 4 n / δ 2 h_{t-1}\cdot4n/\delta^2 h t − 1 ⋅ 4 n / δ 2 h t δ 2 / ( 4 n ) h_t\delta^2/(4n) h t δ 2 / ( 4 n ) H H H [ μ δ 2 / ( 4 n ) , 4 n μ / δ 2 ] [\mu\delta^2/(4n),4n\mu/\delta^2] [ μ δ 2 / ( 4 n ) , 4 n μ / δ 2 ] Lemma 5.10 have gaps in H H H (38) , and all pairs in I − I^- I − h t − 1 h_{t-1} h t − 1 g ≤ h t − 1 g \le h_{t-1} g ≤ h t − 1 g = 0 g = 0 g = 0 t = 1 t=1 t = 1 μ ∗ = 0 \mu^* = 0 μ ∗ = 0 μ ∗ = 0 \mu^* = 0 μ ∗ = 0 g > 0 g > 0 g > 0 μ ∗ = 9 n 2 β g ≤ 9 n 2 β h t − 1 \mu^* = \frac{9n^2}{\beta}g \le\frac{9n^2}{\beta}h_{t-1} μ ∗ = β 9 n 2 g ≤ β 9 n 2 h t − 1 poly ( n ) \operatorname{poly}(n) poly ( n ) h t − 1 h_{t-1} h t − 1
Acknowledgments
This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant agreement nos. 757481-ScaleOpt and 805241-QIP), and from the Dutch Science Foundation (NWO) under the NWO Talent Programme (grant 016.Vidi.189.087). Part of this work was done during the ICERM programme ”Combinatorics and Optimization” and the Simons Institute programme ”Data Structures and Optimization for Fast Algorithms”.
AI Disclosure
The mathematical ideas of the paper were obtained without using AI models. OpenAI’s GPT-6 Astra and Anthropic’s Claude Opus 5.5 were used in editing and proofreading the manuscript, leading to several corrections and clarifications.
Minkowski–Weyl decomposition in standard form
In this appendix we prove Lemmas 2.2 and 5.4 , including the running time and space bounds used by the rounding procedure of § 5 . Results of this kind appear throughout the literature; we nonetheless carry the computations out in full, for two reasons. First, the standard references work in a different computational model: the algorithmic Carathéodory theorem [20 Theorem 6.5.11] for polytopes, and its Minkowski–Weyl extension to pointed polyhedra [20 Corollary 6.5.13] , are stated for a polyhedron given by a strong optimization oracle and are oracle-polynomial, whereas our polyhedron is given explicitly as A x = b Ax=b A x = b x ≥ 0 n x\geq0_n x ≥ 0 n [20 Section 1.3] , which is more restrictive than the model used in [20 ] itself.
We make no attempt to optimize the running time in what follows. In particular, we abstract the basic linear algebra away into the cost of implementing a handful of subroutines, and we make no use of dynamic data structures that could amortize the computations. This is partly for clarity of presentation and partly because many of the implementations of these subroutines in the literature do not work in our strongly polynomial model. The algorithm is otherwise the classical one, and the encoding-length argument is that of [20 Theorem 6.5.11] , carried out explicitly in standard form.
The cost of strongly polynomial linear algebra
Definition A.1 (Linear-algebra tasks and their cost). Each of the quantities below denotes the number of arithmetic operations and comparisons on rationals performed by an algorithm for the stated task that is strongly polynomial in the Turing model in the sense of § 1.1 . The input matrix is M ∈ Q ^ m × n M \in\widehat{\mathbb{Q}}^{m\times n} M ∈ Q m × n m ≤ n m\leq n m ≤ n M ∈ Q ^ r × r M \in\widehat{\mathbb{Q}}^{r\times r} M ∈ Q r × r r ≤ n r\leq n r ≤ n
T m u l ( n ) \mathcal{T}_{\mathrm{mul}}(n) T mul ( n ) M N MN M N M ∈ Q ^ m × n M \in\widehat{\mathbb{Q}}^{m\times n} M ∈ Q m × n N ∈ Q ^ n × r N \in\widehat{\mathbb{Q}}^{n\times r} N ∈ Q n × r m , r ≤ n m,r\leq n m , r ≤ n
T d e t ( n ) \mathcal{T}_{\mathrm{det}}(n) T det ( n ) det M \det M det M M ∈ Q ^ r × r M \in\widehat{\mathbb{Q}}^{r\times r} M ∈ Q r × r r ≤ n r\leq n r ≤ n
T a d j ( n ) \mathcal{T}_{\mathrm{adj}}(n) T adj ( n ) adj ( M ) \operatorname{adj}(M) adj ( M ) M ∈ Q ^ r × r M \in\widehat{\mathbb{Q}}^{r\times r} M ∈ Q r × r r ≤ n r\leq n r ≤ n
(iv) T i n v ( n ) \mathcal{T}_{\mathrm{inv}}(n) T inv ( n ) M − 1 ∈ Q ^ r × r M^{-1} \in\widehat{\mathbb{Q}}^{r \times r} M − 1 ∈ Q r × r M ∈ Q ^ r × r M \in\widehat{\mathbb{Q}}^{r \times r} M ∈ Q r × r r ≤ n r \le n r ≤ n
(v) T b a s i s ( n ) \mathcal{T}_{\mathrm{basis}}(n) T basis ( n ) M M M B ⊆ [ n ] B \subseteq[n] B ⊆ [ n ] { M j } j ∈ B \{M_j\}_{j \in B} { M j } j ∈ B M M M
(vi) T c i r c ( n ) \mathcal{T}_{\mathrm{circ}}(n) T circ ( n ) M M M g ∈ Q ^ n ∖ { 0 n } g \in\widehat{\mathbb{Q}}^n \setminus\{0_n\} g ∈ Q n ∖ { 0 n } M g = 0 m Mg = 0_m M g = 0 m supp ( g ) \operatorname{supp}(g) supp ( g ) M M M M M M
We will give explicit upper bounds for (i), (ii) and (iii), and reduce the remaining tasks (iv)–(vi) to these three. Two further tasks, basis crashing and the Minkowski–Weyl decomposition itself, are introduced with the same counting convention in Definitions A.9 and A.13.
Before doing so, we recall that all numbers are encoded rationals as in Definition 1.2, never reduced to lowest terms. Every size bound in this appendix refers to the computed representation.
We begin with the standard cubic bound for matrix multiplication, from which all other running time bounds below are derived. We work with this obvious bound and do not invoke fast matrix multiplication results, as the strongly polynomial implementability of some of these algorithms would require more involved technical arguments.
Proposition A.2 (Matrix multiplication). T m u l ( n ) = O ( n 3 ) \mathcal{T}_{\mathrm{mul}}(n) = O(n^3) T mul ( n ) = O ( n 3 )
The determinant and the adjugate below are computed by the method of Le Verrier [30 ] , also known as the Faddeev–Le Verrier method (see also [25 ] ). The method presented is again not as fast as possible, but it is sufficient for our purposes.
Proposition A.3 (Le Verrier’s identities). Let M ∈ Z n × n M \in\mathbb{Z}^{n \times n} M ∈ Z n × n χ M ( λ ) = det ( λ I n − M ) = ∑ i = 0 n c i λ i \chi_M(\lambda) = \det(\lambda I_n - M) = \sum_{i=0}^{n} c_i \lambda^i χ M ( λ ) = det ( λ I n − M ) = ∑ i = 0 n c i λ i M M M s j : = tr ( M j ) s_j := \operatorname{tr}(M^j) s j := tr ( M j ) c n = 1 c_n = 1 c n = 1
( n − i ) c i = − ∑ j = 1 n − i c i + j s j ( 0 ≤ i ≤ n − 1 ) , (53) (n-i)c_i = -\sum_{j=1}^{n-i} c_{i+j}s_j \qquad(0 \le i \le n-1), \tag*{(53)} ( n − i ) c i = − j = 1 ∑ n − i c i + j s j ( 0 ≤ i ≤ n − 1 ) , (53) adj ( M ) = ( − 1 ) n + 1 ∑ i = 1 n c i M i − 1 , (54) \operatorname{adj}(M) = (-1)^{n+1}\sum_{i=1}^{n} c_i M^{i-1}, \tag*{(54)} adj ( M ) = ( − 1 ) n + 1 i = 1 ∑ n c i M i − 1 , (54) and det M = ( − 1 ) n c 0 \det M = (-1)^n c_0 det M = ( − 1 ) n c 0
Identity (53) is Newton’s identity in the Faddeev–Le Verrier form [13 Algorithm 2.2.7] , [25 ] ; (54) follows from the Cayley–Hamilton theorem.
To compute the determinant and the adjugate in our computational model we require rescalings of the c i c_i c i
Corollary A.4.
T d e t ( n ) , T a d j ( n ) = O ( n T m u l ( n ) ) . (55) \mathcal{T}_{\mathrm{det}}(n),\mathcal{T}_{\mathrm{adj}}(n) = O\left(n\mathcal{T}_{\mathrm{mul}}(n)\right). \tag*{(55)} T det ( n ) , T adj ( n ) = O ( n T mul ( n ) ) . (55) Proof. Let M ∈ Q ^ r × r M \in\widehat{\mathbb{Q}}^{r \times r} M ∈ Q r × r r ≤ n r \le n r ≤ n M M M M j M^j M j
We multiply M M M O ( r 2 ) O(r^2) O ( r 2 ) M i j = ( p i j , q i j ) M_{ij}=(p_{ij},q_{ij}) M ij = ( p ij , q ij ) p i j ∈ Z p_{ij}\in\mathbb{Z} p ij ∈ Z q i j ∈ Z > 0 q_{ij}\in\mathbb{Z}_{>0} q ij ∈ Z > 0 Δ : = ∏ k , l q k l \Delta:= \prod_{k,l} q_{kl} Δ := ∏ k , l q k l ( Δ M ) i j (\Delta M)_{ij} ( Δ M ) ij p i j ⋅ Δ / q i j p_{ij}\cdot\Delta/q_{ij} p ij ⋅ Δ/ q ij Δ / q i j \Delta/q_{ij} Δ/ q ij ( Δ M ) i j : = p i j Δ − i j (\Delta M)_{ij}:=p_{ij}\Delta_{-ij} ( Δ M ) ij := p ij Δ − ij Δ − i j : = ∏ ( k , l ) ≠ ( i , j ) q k l \Delta_{-ij}:=\prod_{(k,l)\ne(i,j)}q_{kl} Δ − ij := ∏ ( k , l ) = ( i , j ) q k l Δ − i j \Delta_{-ij} Δ − ij O ( r 4 ) O(r^4) O ( r 4 ) r 2 r^2 r 2 q k l q_{kl} q k l O ( r 2 ) O(r^2) O ( r 2 ) Δ − i j \Delta_{-ij} Δ − ij ( i , j ) (i,j) ( i , j ) Δ \Delta Δ Δ M \Delta M Δ M O ( size ( M ) ) O(\operatorname{size}(M)) O ( size ( M )) ( Δ M ) = O ( r 2 size ( M ) ) (\Delta M)=O(r^2\operatorname{size}(M)) ( Δ M ) = O ( r 2 size ( M ))
The outputs for M M M det M = det ( Δ M ) / Δ r \det M=\det(\Delta M)/\Delta^r det M = det ( Δ M ) / Δ r adj ( M ) = adj ( Δ M ) / Δ r − 1 \operatorname{adj}(M)=\operatorname{adj}(\Delta M)/\Delta^{r-1} adj ( M ) = adj ( Δ M ) / Δ r − 1 O ( r ) O(r) O ( r ) Δ \Delta Δ poly ( n ) size ( Δ M ) \operatorname{poly}(n)\operatorname{size}(\Delta M) poly ( n ) size ( Δ M ) Δ M \Delta M Δ M poly ( n ) size ( M ) \operatorname{poly}(n)\operatorname{size}(M) poly ( n ) size ( M ) O ( n 2 ) O(n^2) O ( n 2 ) M ∈ Z r × r M\in\mathbb{Z}^{r\times r} M ∈ Z r × r
As all costs in Definition A.1 are monotone in n n n r = n r=n r = n c i c_i c i C i : = ( n − i ) ! ⋅ c i C_i:=(n-i)!\cdot c_i C i := ( n − i )! ⋅ c i D ∗ : = ( n − 1 ) ! ⋅ adj ( M ) D^*:=(n-1)!\cdot\operatorname{adj}(M) D ∗ := ( n − 1 )! ⋅ adj ( M ) C n = 1 C_n=1 C n = 1 (53) by ( n − i − 1 ) ! (n-i-1)! ( n − i − 1 )! (54) by ( n − 1 ) ! (n-1)! ( n − 1 )! c k = C k / ( n − k ) ! c_k=C_k/(n-k)! c k = C k / ( n − k )!
C i = − ∑ j = 1 n − i ( n − i − 1 ) ! ( n − i − j ) ! C i + j s j ( 0 ≤ i ≤ n − 1 ) , (56) C_i=-\sum_{j=1}^{n-i}\frac{(n-i-1)!}{(n-i-j)!}C_{i+j}s_j \qquad(0\leq i\leq n-1), \tag*{(56)} C i = − j = 1 ∑ n − i ( n − i − j )! ( n − i − 1 )! C i + j s j ( 0 ≤ i ≤ n − 1 ) , (56) D ⋆ = ( − 1 ) n + 1 ∑ i = 1 n ( n − 1 ) ! ( n − i ) ! C i M i − 1 , (57) D^\star=(-1)^{n+1}\sum_{i=1}^{n}\frac{(n-1)!}{(n-i)!}C_iM^{i-1}, \tag*{(57)} D ⋆ = ( − 1 ) n + 1 i = 1 ∑ n ( n − i )! ( n − 1 )! C i M i − 1 , (57) in which every coefficient is a falling factorial, hence an integer of size polynomial in n n n a ! / ( a − j ) ! = a ( a − 1 ) ⋯ ( a − j + 1 ) a!/(a-j)!=a(a-1)\cdots(a-j+1) a ! / ( a − j )! = a ( a − 1 ) ⋯ ( a − j + 1 ) 0 ≤ j ≤ a ≤ n − 1 0\leq j\leq a\leq n-1 0 ≤ j ≤ a ≤ n − 1 O ( n 2 ) O(n^2) O ( n 2 )
The powers M 2 , … , M n M^2,\ldots,M^n M 2 , … , M n s j s_j s j O ( n ) O(n) O ( n ) O ( n 2 ) O(n^2) O ( n 2 ) ∥ M n ∥ ∞ ≤ n n ∥ M ∥ ∞ \lVert M^n\rVert_\infty\leq n^n\lVert M\rVert_\infty ∥ M n ∥ ∞ ≤ n n ∥ M ∥ ∞ size ( M n ) = poly ( n , size ( M ) ) \operatorname{size}(M^n)=\operatorname{poly}(n,\operatorname{size}(M)) size ( M n ) = poly ( n , size ( M )) size ( s j ) = poly ( n , size ( M ) ) \operatorname{size}(s_j)=\operatorname{poly}(n,\operatorname{size}(M)) size ( s j ) = poly ( n , size ( M ))
Evaluating (56) for i = n − 1 , … , 0 i=n-1,\ldots,0 i = n − 1 , … , 0 O ( n 2 ) O(n^2) O ( n 2 ) (57) a further O ( n 3 ) O(n^3) O ( n 3 ) det M \det M det M ( ( − 1 ) n C 0 , n ! ) ((-1)^nC_0,n!) (( − 1 ) n C 0 , n !) adj ( M ) \operatorname{adj}(M) adj ( M ) ( D ⋆ , ( n − 1 ) ! ) (D^\star,(n-1)!) ( D ⋆ , ( n − 1 )!)
As T m u l ( n ) ≥ n 2 \mathcal{T}_{\mathrm{mul}}(n)\geq n^2 T mul ( n ) ≥ n 2
Note that for integral M M M det M \det M det M ( ( − 1 ) n C 0 , n ! ) ((-1)^nC_0,n!) (( − 1 ) n C 0 , n !) n ! n! n !
We now reduce T i n v \mathcal{T}_{\mathrm{inv}} T inv T b a s i s \mathcal{T}_{\mathrm{basis}} T basis T c i r c \mathcal{T}_{\mathrm{circ}} T circ T m u l \mathcal{T}_{\mathrm{mul}} T mul T d e t \mathcal{T}_{\mathrm{det}} T det T a d j \mathcal{T}_{\mathrm{adj}} T adj § 1.1 .
Corollary A.5 (Matrix inversion). T i n v ( n ) = O ( T a d j ( n ) + T d e t ( n ) ) \mathcal{T}_{\mathrm{inv}}(n)=O(\mathcal{T}_{\mathrm{adj}}(n)+\mathcal{T}_{\mathrm{det}}(n)) T inv ( n ) = O ( T adj ( n ) + T det ( n ))
Proof. For invertible M ∈ Q r × r M\in\mathbb{Q}^{r\times r} M ∈ Q r × r r ≤ n r\leq n r ≤ n M − 1 = adj ( M ) / det M M^{-1}=\operatorname{adj}(M)/\det M M − 1 = adj ( M ) / det M
A system M z = h Mz=h M z = h M M M z = M − 1 h z=M^{-1}h z = M − 1 h T i n v ( n ) + O ( n 2 ) \mathcal{T}_{\mathrm{inv}}(n)+O(n^2) T inv ( n ) + O ( n 2 )
Lemma A.6 (Basis computation). T b a s i s ( n ) = O ( n ( T m u l ( n ) + T d e t ( n ) ) ) \mathcal{T}_{\mathrm{basis}}(n)=O(n(\mathcal{T}_{\mathrm{mul}}(n)+\mathcal{T}_{\mathrm{det}}(n))) T basis ( n ) = O ( n ( T mul ( n ) + T det ( n )))
Proof. Let M ∈ Q m × n M\in\mathbb{Q}^{m\times n} M ∈ Q m × n m ≤ n m\leq n m ≤ n N ∈ R m × r N\in\mathbb{R}^{m\times r} N ∈ R m × r N T N N^{\mathsf T}N N T N N N N B : = ∅ B:=\varnothing B := ∅ j = 1 , … , n j=1,\ldots,n j = 1 , … , n j j j B B B
det ( M B ∪ { j } T M B ∪ { j } ) ≠ 0 , \det\left(M_{B\cup\{j\}}^{\mathsf T}M_{B\cup\{j\}}\right)\ne0, det ( M B ∪ { j } T M B ∪ { j } ) = 0 , which by the above happens exactly when M j M_j M j M B M_B M B n n n ∣ B ∣ + 1 ≤ m + 1 |B|+1\leq m+1 ∣ B ∣ + 1 ≤ m + 1 poly ( n ) size ( M ) \operatorname{poly}(n)\operatorname{size}(M) poly ( n ) size ( M )
Let us now consider the computation of circuits.
Lemma A.7 (Circuits).
T c i r c ( n ) = O ( T b a s i s ( n ) + T m u l ( n ) + T i n v ( n ) ) . (58) \mathcal{T}_{\mathrm{circ}}(n)=O\left(\mathcal{T}_{\mathrm{basis}}(n)+\mathcal{T}_{\mathrm{mul}}(n)+\mathcal{T}_{\mathrm{inv}}(n)\right). \tag*{(58)} T circ ( n ) = O ( T basis ( n ) + T mul ( n ) + T inv ( n ) ) . (58) Proof. One call to Lemma A.6 returns a basis B ⊆ [ n ] B\subseteq[n] B ⊆ [ n ] M M M B = [ n ] B=[n] B = [ n ] M M M t ∈ [ n ] ∖ B t\in[n]\setminus B t ∈ [ n ] ∖ B B = ∅ B=\varnothing B = ∅ M = 0 M=0 M = 0 { t } \{t\} { t } g = − e t g=-e_t g = − e t g t = − ( M B T M B ) − 1 M B T M t g_t=-\left(M_B^{\mathsf T}M_B\right)^{-1}M_B^{\mathsf T}M_t g t = − ( M B T M B ) − 1 M B T M t g [ n ] ∖ ( B ∪ { t } ) = 0 g_{[n]\setminus(B\cup\{t\})}=0 g [ n ] ∖ ( B ∪ { t }) = 0 t t t B B B T i n v ( n ) \mathcal{T}_{\mathrm{inv}}(n) T inv ( n ) T m u l ( n ) \mathcal{T}_{\mathrm{mul}}(n) T mul ( n ) O ( n 2 ) O(n^2) O ( n 2 ) T m u l ( n ) ≥ n 2 \mathcal{T}_{\mathrm{mul}}(n)\geq n^2 T mul ( n ) ≥ n 2
Pseudoinverses. We can apply these primitives to compute the Moore–Penrose pseudoinverse. As in Definition A.1, we assume m ≤ n m \le n m ≤ n m > n m > n m > n M T M^{\mathsf{T}} M T ( M T ) + = ( M + ) T (M^{\mathsf{T}})^{+} = (M^{+})^{\mathsf{T}} ( M T ) + = ( M + ) T m m m n n n
Lemma A.8 (Pseudoinverse computation). Let M ∈ Q ^ m × n M \in\widehat{\mathbb{Q}}^{m \times n} M ∈ Q m × n m ≤ n m \le n m ≤ n M + ∈ Q ^ n × m M^{+} \in\widehat{\mathbb{Q}}^{n \times m} M + ∈ Q n × m
O ( T b a s i s ( n ) + T i n v ( n ) + T m u l ( n ) ) (59) O\bigl(\mathcal{T}_{\mathrm{basis}}(n) + \mathcal{T}_{\mathrm{inv}}(n) + \mathcal{T}_{\mathrm{mul}}(n)\bigr) \tag*{(59)} O ( T basis ( n ) + T inv ( n ) + T mul ( n ) ) (59) arithmetic operations and comparisons, every number occurring having size poly ( n ) size ( M ) \operatorname{poly}(n)\operatorname{size}(M) poly ( n ) size ( M )
Proof. For rank ( M ) = 0 \operatorname{rank}(M) = 0 rank ( M ) = 0 rank ( M ) ≥ 1 \operatorname{rank}(M) \ge1 rank ( M ) ≥ 1 M M M B ⊆ [ n ] B \subseteq[n] B ⊆ [ n ] F : = M B ∈ Q ^ m × r F := M_B \in\widehat{\mathbb{Q}}^{m \times r} F := M B ∈ Q m × r r = rank ( M ) ∈ { 1 , … , m } r = \operatorname{rank}(M) \in\{1,\ldots,m\} r = rank ( M ) ∈ { 1 , … , m } F F F F T F F^{\mathsf{T}}F F T F G : = ( F T F ) − 1 F T M ∈ Q ^ r × n G := (F^{\mathsf{T}}F)^{-1}F^{\mathsf{T}}M \in\widehat{\mathbb{Q}}^{r \times n} G := ( F T F ) − 1 F T M ∈ Q r × n F ( F T F ) − 1 F T F(F^{\mathsf{T}}F)^{-1}F^{\mathsf{T}} F ( F T F ) − 1 F T im ( F ) = im ( M ) \operatorname{im}(F) = \operatorname{im}(M) im ( F ) = im ( M ) M M M F G = M FG = M F G = M G B = ( F T F ) − 1 F T F = I r G_B = (F^{\mathsf{T}}F)^{-1}F^{\mathsf{T}}F = I_r G B = ( F T F ) − 1 F T F = I r G G G M = F G M = FG M = F G M + = G + F + M^{+} = G^{+}F^{+} M + = G + F + F + = ( F T F ) − 1 F T F^{+} = (F^{\mathsf{T}}F)^{-1}F^{\mathsf{T}} F + = ( F T F ) − 1 F T G + = G T ( G G T ) − 1 G^{+} = G^{\mathsf{T}}(GG^{\mathsf{T}})^{-1} G + = G T ( G G T ) − 1
Beyond the single basis computation, this is a constant number of matrix products together with the two inversions of F T F F^{\mathsf{T}}F F T F G G T GG^{\mathsf{T}} G G T r ≤ n r \le n r ≤ n
Basis crashing
The first of the two polyhedral tasks is basis crashing , a term coined by Beling and Megiddo [6 ] .
Definition A.9 (Basis crashing). T c r ( n ) \mathcal{T}_{\mathrm{cr}}(n) T cr ( n ) A ∈ Q ^ m × n A \in\widehat{\mathbb{Q}}^{m \times n} A ∈ Q m × n m ≤ n m \le n m ≤ n b ∈ Q ^ m b \in\widehat{\mathbb{Q}}^m b ∈ Q m y ∈ P ∩ Q ^ n y \in\mathcal{P} \cap\widehat{\mathbb{Q}}^n y ∈ P ∩ Q n P : = { x ∈ R n ∣ A x = b , x ≥ 0 n } \mathcal{P} := \{x \in\mathbb{R}^n \mid Ax = b,\ x \ge0_n\} P := { x ∈ R n ∣ A x = b , x ≥ 0 n } v v v P \mathcal{P} P supp ( v ) ⊆ supp ( y ) \operatorname{supp}(v) \subseteq\operatorname{supp}(y) supp ( v ) ⊆ supp ( y )
In this section we use the linear algebra subroutines introduced in the previous section to find vertices v v v y y y y y y g g g supp ( g ) ⊆ supp ( y ) \operatorname{supp}(g) \subseteq\operatorname{supp}(y) supp ( g ) ⊆ supp ( y ) [20 Theorem 6.5.11] , we avoid this by computing every iterate directly from y y y
Lemma A.10 (Basis crashing).
T c r ( n ) = O ( n ( T c i r c ( n ) + T i n v ( n ) + T m u l ( n ) ) ) , (60) \mathcal{T}_{\mathrm{cr}}(n) = O\bigl(n(\mathcal{T}_{\mathrm{circ}}(n) + \mathcal{T}_{\mathrm{inv}}(n) + \mathcal{T}_{\mathrm{mul}}(n))\bigr), \tag*{(60)} T cr ( n ) = O ( n ( T circ ( n ) + T inv ( n ) + T mul ( n )) ) , (60) and with A A A b b b y y y poly ( n ) ( size ( A , b ) + size ( y ) ) \operatorname{poly}(n)(\operatorname{size}(A,b) + \operatorname{size}(y)) poly ( n ) ( size ( A , b ) + size ( y )) v v v size ( v ) = poly ( n ) size ( A , b ) \operatorname{size}(v) = \operatorname{poly}(n)\operatorname{size}(A,b) size ( v ) = poly ( n ) size ( A , b )
Proof. Set y ( 1 ) : = y y^{(1)} := y y ( 1 ) := y S 1 : = supp ( y ( 1 ) ) S_1 := \operatorname{supp}(y^{(1)}) S 1 := supp ( y ( 1 ) ) i = 1 , 2 , … i = 1,2,\ldots i = 1 , 2 , … i ≥ 2 i \ge2 i ≥ 2 S i : = supp ( y ( i ) ) S_i := \operatorname{supp}(y^{(i)}) S i := supp ( y ( i ) ) S i ⊆ S i − 1 S_i \subseteq S_{i-1} S i ⊆ S i − 1 A y ( i ) = b Ay^{(i)} = b A y ( i ) = b A S i A_{S_i} A S i A S i A_{S_i} A S i v : = y ( i ) v := y^{(i)} v := y ( i ) size ( A , b ) \operatorname{size}(A,b) size ( A , b ) v v v A x = b Ax = b A x = b S i S_i S i A S i v S i = b A_{S_i}v_{S_i} = b A S i v S i = b v S i v_{S_i} v S i v [ n ] ∖ S i = 0 v_{[n]\setminus S_i} = 0 v [ n ] ∖ S i = 0 A S i A_{S_i} A S i S i S_i S i g ( i ) ∈ ker ( A ) ∖ { 0 n } g^{(i)} \in\ker(A) \setminus\{0_n\} g ( i ) ∈ ker ( A ) ∖ { 0 n } supp ( g ( i ) ) ⊆ S i \operatorname{supp}(g^{(i)}) \subseteq S_i supp ( g ( i ) ) ⊆ S i g ( i ) g^{(i)} g ( i ) − g ( i ) -g^{(i)} − g ( i ) y j ( i ) > 0 y^{(i)}_j > 0 y j ( i ) > 0 j ∈ S i ⊇ supp ( g ( i ) ) j \in S_i \supseteq\operatorname{supp}(g^{(i)}) j ∈ S i ⊇ supp ( g ( i ) ) τ i \tau_i τ i y ( i + 1 ) : = y ( i ) + τ i g ( i ) y^{(i+1)} := y^{(i)} + \tau_i g^{(i)} y ( i + 1 ) := y ( i ) + τ i g ( i ) S i + 1 : = supp ( y ( i + 1 ) ) S_{i+1} := \operatorname{supp}(y^{(i+1)}) S i + 1 := supp ( y ( i + 1 ) ) S i + 1 ⊊ S i S_{i+1} \subsetneq S_i S i + 1 ⊊ S i
We do not compute y ( i + 1 ) y^{(i+1)} y ( i + 1 ) y ( i ) y^{(i)} y ( i ) y y y G G G g ( 1 ) , … , g ( i ) g^{(1)},\ldots,g^{(i)} g ( 1 ) , … , g ( i ) R : = S 1 ∖ S i + 1 R := S_1 \setminus S_{i+1} R := S 1 ∖ S i + 1 y ( i + 1 ) = y + G τ y^{(i+1)} = y + G\tau y ( i + 1 ) = y + G τ τ = ( τ 1 , … , τ i ) \tau= (\tau_1,\ldots,\tau_i) τ = ( τ 1 , … , τ i ) y R ( i + 1 ) = 0 y^{(i+1)}_R = 0 y R ( i + 1 ) = 0 τ \tau τ ( G ∣ R ) τ = − y R (G\vert_R)\tau= -y_R ( G ∣ R ) τ = − y R t ≤ i t \le i t ≤ i p t ∈ S t ∖ S t + 1 ⊆ R p_t \in S_t \setminus S_{t+1} \subseteq R p t ∈ S t ∖ S t + 1 ⊆ R g p t ( t ) = − y p t ( t ) / τ t ≠ 0 g^{(t)}_{p_t} = -y^{(t)}_{p_t}/\tau_t \ne0 g p t ( t ) = − y p t ( t ) / τ t = 0 g p t ( u ) = 0 g^{(u)}_{p_t} = 0 g p t ( u ) = 0 u > t u > t u > t supp ( g ( u ) ) ⊆ S u ⊆ S t + 1 \operatorname{supp}(g^{(u)}) \subseteq S_u \subseteq S_{t+1} supp ( g ( u ) ) ⊆ S u ⊆ S t + 1 p 1 , … , p i p_1,\ldots,p_i p 1 , … , p i G G G G R G_R G R α = − ( G R T G R ) − 1 G R T y R \alpha= -(G_R^{\mathsf{T}}G_R)^{-1}G_R^{\mathsf{T}}y_R α = − ( G R T G R ) − 1 G R T y R α \alpha α T i n v ( n ) \mathcal{T}_{\mathrm{inv}}(n) T inv ( n ) T m u l ( n ) \mathcal{T}_{\mathrm{mul}}(n) T mul ( n ) O ( n 2 ) O(n^2) O ( n 2 ) size ( α ) = poly ( n ) size ( G , y ) = poly ( n ) size ( A , y ) \operatorname{size}(\alpha)=\operatorname{poly}(n)\operatorname{size}(G,y)=\operatorname{poly}(n)\operatorname{size}(A,y) size ( α ) = poly ( n ) size ( G , y ) = poly ( n ) size ( A , y ) size ( y ( i + 1 ) ) = poly ( n ) ( size ( A , y ) ) \operatorname{size}(y^{(i+1)})=\operatorname{poly}(n)(\operatorname{size}(A,y)) size ( y ( i + 1 ) ) = poly ( n ) ( size ( A , y ))
Since S i + 1 ⊊ S i S_{i+1}\subsetneq S_i S i + 1 ⊊ S i n n n T c i r c ( n ) \mathcal{T}_{\mathrm{circ}}(n) T circ ( n ) O ( n ) O(n) O ( n ) T i n v ( n ) \mathcal{T}_{\mathrm{inv}}(n) T inv ( n ) T m u l ( n ) \mathcal{T}_{\mathrm{mul}}(n) T mul ( n ) y ( i + 1 ) y^{(i+1)} y ( i + 1 ) A S i A_{S_i} A S i v S i = ( A S i T A S i ) − 1 A S i T b v_{S_i}=(A_{S_i}^{\mathsf T}A_{S_i})^{-1}A_{S_i}^{\mathsf T}b v S i = ( A S i T A S i ) − 1 A S i T b T i n v ( n ) \mathcal{T}_{\mathrm{inv}}(n) T inv ( n ) T m u l ( n ) \mathcal{T}_{\mathrm{mul}}(n) T mul ( n ) size ( v ) = poly ( n ) size ( A , b ) \operatorname{size}(v)=\operatorname{poly}(n)\operatorname{size}(A,b) size ( v ) = poly ( n ) size ( A , b ) (60) .
Polytopes
We are ready to compute Carathéodory decompositions.
Lemma A.11 (Carathéodory decomposition in standard form). Let A ∈ Q m × n A\in\mathbb{Q}^{m\times n} A ∈ Q m × n m ≤ n m\le n m ≤ n b ∈ Q m b\in\mathbb{Q}^{m} b ∈ Q m P : = { x ∈ R n ∣ A x = b , x ≥ 0 n } \mathcal{P}:=\{x\in\mathbb{R}^{n}\mid Ax=b,\ x\ge0_n\} P := { x ∈ R n ∣ A x = b , x ≥ 0 n } y ∈ P ∩ Q n y\in\mathcal{P}\cap\mathbb{Q}^{n} y ∈ P ∩ Q n v ( 1 ) , … , v ( k ) v^{(1)},\ldots,v^{(k)} v ( 1 ) , … , v ( k ) P \mathcal{P} P λ 1 , … , λ k ∈ Q > 0 \lambda_1,\ldots,\lambda_k\in\mathbb{Q}_{>0} λ 1 , … , λ k ∈ Q > 0 v ( 1 ) , … , v ( k ) v^{(1)},\ldots,v^{(k)} v ( 1 ) , … , v ( k )
∑ i = 1 k λ i = 1 , y = ∑ i = 1 k λ i v ( i ) . \sum_{i=1}^{k}\lambda_i=1,\qquad y=\sum_{i=1}^{k}\lambda_i v^{(i)}. i = 1 ∑ k λ i = 1 , y = i = 1 ∑ k λ i v ( i ) . in particular k ≤ n − rank ( A ) + 1 k\le n-\operatorname{rank}(A)+1 k ≤ n − rank ( A ) + 1 O ( n ( T c r ( n ) + T i n v ( n ) ) ) O\left(n(\mathcal{T}_{\mathrm{cr}}(n)+\mathcal{T}_{\mathrm{inv}}(n))\right) O ( n ( T cr ( n ) + T inv ( n )) ) poly ( n ) size ( A , b , y ) \operatorname{poly}(n)\operatorname{size}(A,b,y) poly ( n ) size ( A , b , y )
Proof. Set y ( 1 ) : = y y^{(1)}:=y y ( 1 ) := y S i : = supp ( y ( i ) ) S_i:=\operatorname{supp}(y^{(i)}) S i := supp ( y ( i ) ) i = 1 , 2 , … i=1,2,\ldots i = 1 , 2 , … P \mathcal{P} P y ( i ) y^{(i)} y ( i ) v ( i ) v^{(i)} v ( i ) P \mathcal{P} P supp ( v ( i ) ) ⊆ S i \operatorname{supp}(v^{(i)})\subseteq S_i supp ( v ( i ) ) ⊆ S i v ( i ) = y ( i ) v^{(i)}=y^{(i)} v ( i ) = y ( i ) k : = i k:=i k := i v ( i ) v^{(i)} v ( i ) h ( i ) : = y ( i ) − v ( i ) ≠ 0 n h^{(i)}:=y^{(i)}-v^{(i)}\ne0_n h ( i ) := y ( i ) − v ( i ) = 0 n A h ( i ) = 0 m Ah^{(i)}=0_m A h ( i ) = 0 m supp ( h ( i ) ) ⊆ S i \operatorname{supp}(h^{(i)})\subseteq S_i supp ( h ( i ) ) ⊆ S i τ i \tau_i τ i y ( i ) + τ i h ( i ) ∈ P y^{(i)}+\tau_i h^{(i)}\in\mathcal{P} y ( i ) + τ i h ( i ) ∈ P y ( i ) + ( τ i + ϵ ) h ( i ) ∉ P y^{(i)}+(\tau_i+\epsilon)h^{(i)}\notin\mathcal{P} y ( i ) + ( τ i + ϵ ) h ( i ) ∈ / P ϵ > 0 \epsilon>0 ϵ > 0 P \mathcal{P} P τ i \tau_i τ i y ( i + 1 ) : = y ( i ) + τ i h ( i ) y^{(i+1)}:=y^{(i)}+\tau_i h^{(i)} y ( i + 1 ) := y ( i ) + τ i h ( i ) y ( i + 1 ) y^{(i+1)} y ( i + 1 ) y y y y ( i + 1 ) = y + H ( i ) α y^{(i+1)}=y+H^{(i)}\alpha y ( i + 1 ) = y + H ( i ) α H ( i ) = [ h ( 1 ) ⋯ h ( i ) ] H^{(i)}=[h^{(1)}\ \cdots\ h^{(i)}] H ( i ) = [ h ( 1 ) ⋯ h ( i ) ] α \alpha α
[ y + H ( i ) α ] S 1 ∖ S i + 1 = 0 S 1 ∖ S i + 1 . (61) \left[y+H^{(i)}\alpha\right]_{S_1\setminus S_{i+1}}=0_{S_1\setminus S_{i+1}}. \tag*{(61)} [ y + H ( i ) α ] S 1 ∖ S i + 1 = 0 S 1 ∖ S i + 1 . (61) Uniqueness of α \alpha α p t ∈ S t ∖ S t + 1 p_t\in S_t\setminus S_{t+1} p t ∈ S t ∖ S t + 1 h p t ( t ) = − y p t ( t ) / τ t ≠ 0 h^{(t)}_{p_t}=-y^{(t)}_{p_t}/\tau_t\ne0 h p t ( t ) = − y p t ( t ) / τ t = 0 h p t ( u ) = 0 h^{(u)}_{p_t}=0 h p t ( u ) = 0 u > t u>t u > t H S 1 ∖ S i + 1 ( i ) H^{(i)}_{S_1\setminus S_{i+1}} H S 1 ∖ S i + 1 ( i ) size ( H ( i ) ) = poly ( n ) size ( A , b , y ) \operatorname{size}(H^{(i)})=\operatorname{poly}(n)\operatorname{size}(A,b,y) size ( H ( i ) ) = poly ( n ) size ( A , b , y ) size ( α ) = poly ( n ) size ( A , b , y ) \operatorname{size}(\alpha)=\operatorname{poly}(n)\operatorname{size}(A,b,y) size ( α ) = poly ( n ) size ( A , b , y ) size ( y ( i + 1 ) ) = poly ( n ) size ( A , b , y ) \operatorname{size}(y^{(i+1)})=\operatorname{poly}(n)\operatorname{size}(A,b,y) size ( y ( i + 1 ) ) = poly ( n ) size ( A , b , y )
The same indices p t p_t p t v ( 1 ) , … , v ( k ) v^{(1)},\ldots,v^{(k)} v ( 1 ) , … , v ( k ) t < k t<k t < k v p t ( t ) > 0 v^{(t)}_{p_t}>0 v p t ( t ) > 0 v p t ( u ) = 0 v^{(u)}_{p_t}=0 v p t ( u ) = 0 u > t u>t u > t supp ( v ( u ) ) ⊆ S u ⊆ S t + 1 \operatorname{supp}(v^{(u)})\subseteq S_u\subseteq S_{t+1} supp ( v ( u ) ) ⊆ S u ⊆ S t + 1 v ( t ) v^{(t)} v ( t ) v ( t + 1 ) , … , v ( k ) v^{(t+1)},\ldots,v^{(k)} v ( t + 1 ) , … , v ( k ) v ( 1 ) , … , v ( k ) v^{(1)},\ldots,v^{(k)} v ( 1 ) , … , v ( k )
Finally, compute λ \lambda λ
\begin{pmatrix}
v^{(1)} & \cdots& v^{(k)}\\
1 & \cdots& 1
\end{pmatrix}\lambda=
\begin{pmatrix}
y\\
1
\end{pmatrix}. \tag*{(62)}\end{pmatrix}
Existence follows by construction of the v ( i ) v^{(i)} v ( i ) y ( i + 1 ) = y ( i ) + τ i ( y ( i ) − v ( i ) ) y^{(i+1)}=y^{(i)}+\tau_i(y^{(i)}-v^{(i)}) y ( i + 1 ) = y ( i ) + τ i ( y ( i ) − v ( i ) ) τ i > 0 \tau_i>0 τ i > 0 y ( i ) = τ i 1 + τ i v ( i ) + 1 1 + τ i y ( i + 1 ) y^{(i)}=\frac{\tau_i}{1+\tau_i}v^{(i)}+\frac{1}{1+\tau_i}y^{(i+1)} y ( i ) = 1 + τ i τ i v ( i ) + 1 + τ i 1 y ( i + 1 ) i < k i<k i < k y ( k ) = v ( k ) y^{(k)}=v^{(k)} y ( k ) = v ( k ) y = y ( 1 ) y=y^{(1)} y = y ( 1 ) y y y v ( 1 ) , … , v ( k ) v^{(1)},\ldots,v^{(k)} v ( 1 ) , … , v ( k ) λ i > 0 \lambda_i>0 λ i > 0 ∑ i = 1 k λ i = 1 \sum_{i=1}^{k}\lambda_i=1 ∑ i = 1 k λ i = 1 v ( 1 ) , … , v ( k ) v^{(1)},\ldots,v^{(k)} v ( 1 ) , … , v ( k ) (62) having full column rank. As λ \lambda λ y y y v ( 1 ) , … , v ( k ) v^{(1)},\ldots,v^{(k)} v ( 1 ) , … , v ( k ) size ( λ ) = poly ( n ) size ( v ( 1 ) , … , v ( k ) , y ) = poly ( n ) size ( A , b , y ) \operatorname{size}(\lambda)=\operatorname{poly}(n)\operatorname{size}(v^{(1)},\ldots,v^{(k)},y)=\operatorname{poly}(n)\operatorname{size}(A,b,y) size ( λ ) = poly ( n ) size ( v ( 1 ) , … , v ( k ) , y ) = poly ( n ) size ( A , b , y )
Both systems can be solved by inverting a square subsystem that is triangular with nonzero diagonal by the arguments above: for α \alpha α p 1 , … , p i p_1,\ldots,p_i p 1 , … , p i λ \lambda λ p 1 , … , p k − 1 p_1,\ldots,p_{k-1} p 1 , … , p k − 1 n n n T c r ( n ) \mathcal{T}_{\mathrm{cr}}(n) T cr ( n ) O ( n ) O(n) O ( n ) τ i \tau_i τ i T i n v ( n ) \mathcal{T}_{\mathrm{inv}}(n) T inv ( n ) λ \lambda λ O ( n ( T c r ( n ) + T i n v ( n ) ) ) O\left(n(\mathcal{T}_{\mathrm{cr}}(n)+\mathcal{T}_{\mathrm{inv}}(n))\right) O ( n ( T cr ( n ) + T inv ( n )) )
Cones and general polyhedra
The conic case reduces to the bounded one by intersecting with a hyperplane through the given point.
Corollary A.12 (Conic decomposition in standard form). Let A ∈ Q ^ m × n A \in\widehat{\mathbb{Q}}^{m \times n} A ∈ Q m × n m ≤ n m \le n m ≤ n C : = { x ∈ R n ∣ A x = 0 m , x ≥ 0 n } \mathcal{C} := \{x \in\mathbb{R}^{n} \mid Ax = 0_{m},\ x \ge0_{n}\} C := { x ∈ R n ∣ A x = 0 m , x ≥ 0 n } y ∈ C ∩ Q ^ n y \in\mathcal{C} \cap\widehat{\mathbb{Q}}^{n} y ∈ C ∩ Q n y ≠ 0 n y \ne0_{n} y = 0 n r ( 1 ) , … , r ( ℓ ) ∈ Q ^ n r^{(1)},\ldots,r^{(\ell)} \in\widehat{\mathbb{Q}}^{n} r ( 1 ) , … , r ( ℓ ) ∈ Q n C \mathcal{C} C y = ∑ j = 1 ℓ r ( j ) y = \sum_{j=1}^{\ell} r^{(j)} y = ∑ j = 1 ℓ r ( j ) ℓ ≤ n \ell\le n ℓ ≤ n O ( n ( T c r ( n + 1 ) + T i n v ( n + 1 ) ) ) O\left(n\left(\mathcal{T}_{\mathrm{cr}}(n+1)+\mathcal{T}_{\mathrm{inv}}(n+1)\right)\right) O ( n ( T cr ( n + 1 ) + T inv ( n + 1 ) ) ) poly ( n ) size ( A , y ) \operatorname{poly}(n)\operatorname{size}(A,y) poly ( n ) size ( A , y )
Proof. Let σ : = 1 n T y > 0 \sigma:= 1_{n}^{\mathsf{T}}y > 0 σ := 1 n T y > 0
Q : = { x ∈ R n ∣ A x = 0 m , 1 n T x = σ , x ≥ 0 n } ∋ y Q := \{x \in\mathbb{R}^{n} \mid Ax = 0_{m},\ 1_{n}^{\mathsf{T}}x = \sigma,\ x \ge0_{n}\} \ni y Q := { x ∈ R n ∣ A x = 0 m , 1 n T x = σ , x ≥ 0 n } ∋ y is in standard form with m + 1 ≤ n + 1 m+1 \le n+1 m + 1 ≤ n + 1 ( A 1 n T ) \begin{pmatrix} A \\ 1_{n}^{\mathsf{T}} \end{pmatrix} ( A 1 n T ) ( 0 m σ ) \begin{pmatrix} 0_{m} \\ \sigma\end{pmatrix} ( 0 m σ ) O ( size ( A , y ) ) O(\operatorname{size}(A,y)) O ( size ( A , y )) x ≥ 0 n x \ge0_{n} x ≥ 0 n 1 n T x = σ 1_{n}^{\mathsf{T}}x = \sigma 1 n T x = σ 0 ≤ x i ≤ σ 0 \le x_i \le\sigma 0 ≤ x i ≤ σ C \mathcal{C} C 1 n T x = σ 1_{n}^{\mathsf{T}}x = \sigma 1 n T x = σ
Lemma A.11 applied to Q Q Q y y y w ( 1 ) , … , w ( ℓ ) w^{(1)},\ldots,w^{(\ell)} w ( 1 ) , … , w ( ℓ ) Q Q Q λ ∈ Q ^ ≥ 0 ℓ \lambda\in\widehat{\mathbb{Q}}_{\ge0}^{\ell} λ ∈ Q ≥ 0 ℓ ∑ j λ j = 1 \sum_j \lambda_j = 1 ∑ j λ j = 1 y = ∑ j λ j w ( j ) y = \sum_j \lambda_j w^{(j)} y = ∑ j λ j w ( j ) w ( j ) w^{(j)} w ( j ) ∑ j a j w ( j ) = 0 n \sum_j a_jw^{(j)} = 0_n ∑ j a j w ( j ) = 0 n 1 n T 1_n^{\mathsf{T}} 1 n T σ ∑ j a j = 0 \sigma\sum_j a_j = 0 σ ∑ j a j = 0 ∑ j a j = 0 \sum_j a_j = 0 ∑ j a j = 0 a 1 = ⋯ = a ℓ = 0 a_1 = \cdots= a_\ell= 0 a 1 = ⋯ = a ℓ = 0 r ( j ) : = λ j w ( j ) r^{(j)} := \lambda_jw^{(j)} r ( j ) := λ j w ( j )
We are now ready to prove Lemma 5.4 .
Definition A.13 (Minkowski–Weyl decomposition). T M W ( n ) \mathcal{T}_{\mathrm{MW}}(n) T MW ( n ) A ∈ Q ^ m × n A \in\widehat{\mathbb{Q}}^{m \times n} A ∈ Q m × n m ≤ n m \le n m ≤ n b ∈ Q ^ m b \in\widehat{\mathbb{Q}}^{m} b ∈ Q m x ∈ P ∩ Q ^ n x \in\mathcal{P} \cap\widehat{\mathbb{Q}}^{n} x ∈ P ∩ Q n P : = { x ∈ R n ∣ A x = b , x ≥ 0 n } \mathcal{P} := \{x \in\mathbb{R}^{n} \mid Ax = b,\ x \ge0_{n}\} P := { x ∈ R n ∣ A x = b , x ≥ 0 n } x x x Lemma 5.4 .
Theorem A.14. Lemma 5.4 holds with T M W ( n ) = O ( n ( T c r ( n + 1 ) + T i n v ( n + 1 ) ) ) \mathcal{T}_{\mathrm{MW}}(n) = O\left(n\left(\mathcal{T}_{\mathrm{cr}}(n+1)+\mathcal{T}_{\mathrm{inv}}(n+1)\right)\right) T MW ( n ) = O ( n ( T cr ( n + 1 ) + T inv ( n + 1 ) ) ) poly ( n ) size ( A , b ) \operatorname{poly}(n)\operatorname{size}(A,b) poly ( n ) size ( A , b ) poly ( n ) size ( A , b , x ) \operatorname{poly}(n)\operatorname{size}(A,b,x) poly ( n ) size ( A , b , x ) poly ( n ) size ( A , b , x ) \operatorname{poly}(n)\operatorname{size}(A,b,x) poly ( n ) size ( A , b , x )
Proof. Write y : = x y := x y := x Lemma A.11 on P \mathcal{P} P A h ( i ) = 0 m Ah^{(i)} = 0_m A h ( i ) = 0 m y ( i ) + τ h ( i ) y^{(i)}+\tau h^{(i)} y ( i ) + τ h ( i ) P \mathcal{P} P τ ≥ 0 \tau\ge0 τ ≥ 0 h ( i ) ≥ 0 n h^{(i)} \ge0_n h ( i ) ≥ 0 n i i i τ i \tau_i τ i h ( i ) h^{(i)} h ( i ) h ( i ) h^{(i)} h ( i ) Lemma A.11 , with ℓ = 0 \ell= 0 ℓ = 0 k ≤ n + 1 k \le n+1 k ≤ n + 1 k k k h ( k ) ≥ 0 n h^{(k)} \ge0_n h ( k ) ≥ 0 n h ( k ) ≠ 0 n h^{(k)} \ne0_n h ( k ) = 0 n C : = { x ∈ R n ∣ A x = 0 m , x ≥ 0 n } \mathcal{C} := \{x \in\mathbb{R}^{n} \mid Ax = 0_m,\ x \ge0_n\} C := { x ∈ R n ∣ A x = 0 m , x ≥ 0 n } P \mathcal{P} P P \mathcal{P} P Corollary A.12 returns linearly independent extreme rays r ( 1 ) , … , r ( ℓ ) r^{(1)},\ldots,r^{(\ell)} r ( 1 ) , … , r ( ℓ ) P \mathcal{P} P h ( k ) = ∑ j = 1 ℓ r ( j ) h^{(k)} = \sum_{j=1}^{\ell}r^{(j)} h ( k ) = ∑ j = 1 ℓ r ( j )
As in the proof of Lemma A.11 , y y y v ( 1 ) , … , v ( k − 1 ) , y ( k ) v^{(1)},\ldots,v^{(k-1)},y^{(k)} v ( 1 ) , … , v ( k − 1 ) , y ( k ) λ ∈ Q ^ > 0 k \lambda\in\widehat{\mathbb{Q}}_{>0}^{k} λ ∈ Q > 0 k (62) with y ( k ) y^{(k)} y ( k ) v ( k ) v^{(k)} v ( k ) y ( k ) = v ( k ) + ∑ j r ( j ) y^{(k)} = v^{(k)}+\sum_j r^{(j)} y ( k ) = v ( k ) + ∑ j r ( j ) η j : = λ k > 0 \eta_j := \lambda_k > 0 η j := λ k > 0 j ∈ [ ℓ ] j \in[\ell] j ∈ [ ℓ ] ( v ( 1 ) , 1 ) , … , ( v ( k ) , 1 ) , ( r ( 1 ) , 0 ) , … , ( r ( ℓ ) , 0 ) (v^{(1)},1),\ldots,(v^{(k)},1),(r^{(1)},0),\ldots,(r^{(\ell)},0) ( v ( 1 ) , 1 ) , … , ( v ( k ) , 1 ) , ( r ( 1 ) , 0 ) , … , ( r ( ℓ ) , 0 ) R n + 1 \mathbb{R}^{n+1} R n + 1 k + ℓ ≤ n + 1 k+\ell\le n+1 k + ℓ ≤ n + 1 Lemma A.11 and Corollary A.12 .
For the sizes, size ( v ( i ) ) = poly ( n ) size ( A , b ) \operatorname{size}(v^{(i)}) = \operatorname{poly}(n)\operatorname{size}(A,b) size ( v ( i ) ) = poly ( n ) size ( A , b ) Lemma A.10 , and size ( h ( k ) ) = poly ( n ) size ( A , b , y ) \operatorname{size}(h^{(k)}) = \operatorname{poly}(n)\operatorname{size}(A,b,y) size ( h ( k ) ) = poly ( n ) size ( A , b , y ) Lemma A.11 , so every number in the call to Corollary A.12 , and in particular every r ( j ) r^{(j)} r ( j ) poly ( n ) size ( A , h ( k ) ) = poly ( n ) size ( A , b , y ) \operatorname{poly}(n)\operatorname{size}(A,h^{(k)}) = \operatorname{poly}(n)\operatorname{size}(A,b,y) poly ( n ) size ( A , h ( k ) ) = poly ( n ) size ( A , b , y ) n n n O ( n ( T c r ( n ) + T i n v ( n ) ) ) O\left(n(\mathcal{T}_{\mathrm{cr}}(n)+\mathcal{T}_{\mathrm{inv}}(n))\right) O ( n ( T cr ( n ) + T inv ( n )) ) Lemma A.11 . Solving for λ \lambda λ T i n v ( n + 1 ) \mathcal{T}_{\mathrm{inv}}(n+1) T inv ( n + 1 ) O ( n 2 ) O(n^{2}) O ( n 2 ) Corollary A.12 costs O ( n ( T c r ( n + 1 ) + T i n v ( n + 1 ) ) ) O\left(n(\mathcal{T}_{\mathrm{cr}}(n+1)+\mathcal{T}_{\mathrm{inv}}(n+1))\right) O ( n ( T cr ( n + 1 ) + T inv ( n + 1 )) )
Combining the reductions of Section A.1 turns Theorem A.14 into an explicit bound using the naive cubic bound T m u l ( n ) = O ( n 3 ) \mathcal{T}_{\mathrm{mul}}(n) = O(n^{3}) T mul ( n ) = O ( n 3 ) Proposition A.2 .
Corollary A.15 (Explicit running time and space). Assuming T m u l ( n ) = O ( n 3 ) \mathcal{T}_{\mathrm{mul}}(n) = O(n^{3}) T mul ( n ) = O ( n 3 ) Corollaries A.4 and A.5 and lemmas A.6, A.7 and A.10 and Theorem A.14 give
T d e t ( n ) , T a d j ( n ) , T i n v ( n ) = O ( n 4 ) , T b a s i s ( n ) , T c i r c ( n ) = O ( n 5 ) , \mathcal{T}_{\mathrm{det}}(n),\ \mathcal{T}_{\mathrm{adj}}(n),\ \mathcal{T}_{\mathrm{inv}}(n) = O(n^{4}), \qquad\mathcal{T}_{\mathrm{basis}}(n),\ \mathcal{T}_{\mathrm{circ}}(n) = O(n^{5}), T det ( n ) , T adj ( n ) , T inv ( n ) = O ( n 4 ) , T basis ( n ) , T circ ( n ) = O ( n 5 ) , T c r ( n ) = O ( n 6 ) , T M W ( n ) = O ( n 7 ) . (63) \mathcal{T}_{\mathrm{cr}}(n) = O(n^{6}), \qquad\mathcal{T}_{\mathrm{MW}}(n) = O(n^{7}). \tag*{(63)} T cr ( n ) = O ( n 6 ) , T MW ( n ) = O ( n 7 ) . (63) Consequently, for A ∈ Q ^ m × n A \in\widehat{\mathbb{Q}}^{m \times n} A ∈ Q m × n m ≤ n m \le n m ≤ n b ∈ Q ^ m b \in\widehat{\mathbb{Q}}^m b ∈ Q m x ∈ P ∩ Q ^ n x \in\mathcal{P} \cap\widehat{\mathbb{Q}}^n x ∈ P ∩ Q n Lemma 5.4 is computed in O ( n 7 ) O(n^7) O ( n 7 ) poly ( n ) size ( A , b ) \operatorname{poly}(n)\operatorname{size}(A,b) poly ( n ) size ( A , b ) poly ( n ) size ( A , b , x ) \operatorname{poly}(n)\operatorname{size}(A,b,x) poly ( n ) size ( A , b , x ) poly ( n ) size ( A , b , x ) \operatorname{poly}(n)\operatorname{size}(A,b,x) poly ( n ) size ( A , b , x )
Proof. For the running time, substitute T m u l ( n ) = O ( n 3 ) \mathcal{T}_{\mathrm{mul}}(n) = O(n^3) T mul ( n ) = O ( n 3 ) Corollary A.4 gives T d e t ( n ) , T a d j ( n ) = O ( n T m u l ( n ) ) = O ( n 4 ) \mathcal{T}_{\mathrm{det}}(n), \mathcal{T}_{\mathrm{adj}}(n) = O(n\mathcal{T}_{\mathrm{mul}}(n)) = O(n^4) T det ( n ) , T adj ( n ) = O ( n T mul ( n )) = O ( n 4 ) T i n v ( n ) = O ( T a d j ( n ) + T d e t ( n ) ) = O ( n 4 ) \mathcal{T}_{\mathrm{inv}}(n) = O(\mathcal{T}_{\mathrm{adj}}(n) + \mathcal{T}_{\mathrm{det}}(n)) = O(n^4) T inv ( n ) = O ( T adj ( n ) + T det ( n )) = O ( n 4 ) T b a s i s ( n ) = O ( n ( T m u l ( n ) + T d e t ( n ) ) ) = O ( n 5 ) \mathcal{T}_{\mathrm{basis}}(n) = O(n(\mathcal{T}_{\mathrm{mul}}(n) + \mathcal{T}_{\mathrm{det}}(n))) = O(n^5) T basis ( n ) = O ( n ( T mul ( n ) + T det ( n ))) = O ( n 5 ) T c i r c ( n ) = O ( T b a s i s ( n ) + T m u l ( n ) + T i n v ( n ) ) = O ( n 5 ) \mathcal{T}_{\mathrm{circ}}(n) = O(\mathcal{T}_{\mathrm{basis}}(n) + \mathcal{T}_{\mathrm{mul}}(n) + \mathcal{T}_{\mathrm{inv}}(n)) = O(n^5) T circ ( n ) = O ( T basis ( n ) + T mul ( n ) + T inv ( n )) = O ( n 5 ) Lemma A.10 gives T c r ( n ) = O ( n ( T c i r c ( n ) + T i n v ( n ) + T m u l ( n ) ) ) = O ( n 6 ) \mathcal{T}_{\mathrm{cr}}(n) = O(n(\mathcal{T}_{\mathrm{circ}}(n) + \mathcal{T}_{\mathrm{inv}}(n) + \mathcal{T}_{\mathrm{mul}}(n))) = O(n^6) T cr ( n ) = O ( n ( T circ ( n ) + T inv ( n ) + T mul ( n ))) = O ( n 6 ) T M W ( n ) = O ( n ( T c r ( n + 1 ) + T i n v ( n + 1 ) ) ) = O ( n 7 ) \mathcal{T}_{\mathrm{MW}}(n) = O(n(\mathcal{T}_{\mathrm{cr}}(n+1) + \mathcal{T}_{\mathrm{inv}}(n+1))) = O(n^7) T MW ( n ) = O ( n ( T cr ( n + 1 ) + T inv ( n + 1 ))) = O ( n 7 ) (63) .
For the space bound, the algorithm performs O ( n 7 ) O(n^7) O ( n 7 ) poly ( n ) size ( A , b , x ) \operatorname{poly}(n)\operatorname{size}(A,b,x) poly ( n ) size ( A , b , x ) O ( n 7 ) O(n^7) O ( n 7 ) poly ( n ) size ( A , b , x ) \operatorname{poly}(n)\operatorname{size}(A,b,x) poly ( n ) size ( A , b , x )
In particular, Theorem A.14 and corollary A.15 prove Lemma 5.4 . We also collect the bounds stated in Lemma 2.2 .
Proof of Lemma 2.2. Part (i) is Lemma A.6, and part (ii) is Corollaries A.4 and A.5, with the bounds T b a s i s ( n ) = O ( n 5 ) \mathcal{T}_{\mathrm{basis}}(n) = O(n^5) T basis ( n ) = O ( n 5 ) T d e t ( n ) , T a d j ( n ) , T i n v ( n ) = O ( n 4 ) \mathcal{T}_{\mathrm{det}}(n), \mathcal{T}_{\mathrm{adj}}(n), \mathcal{T}_{\mathrm{inv}}(n) = O(n^4) T det ( n ) , T adj ( n ) , T inv ( n ) = O ( n 4 ) (63) . For part (iii), if m ≤ n m \le n m ≤ n Lemma A.8 and (63) give O ( n 5 ) O(n^5) O ( n 5 ) m > n m > n m > n M T M^{\mathsf{T}} M T ( M T ) + = ( M + ) T (M^{\mathsf{T}})^+ = (M^+)^{\mathsf{T}} ( M T ) + = ( M + ) T
