Introduction and Related Work
Many recent studies have shown that mixed-precision hardware can significantly accelerate the solution of linear systems (see [17 ] for a survey). We consider two approaches that have been used to achieve these speedups. The first approach combines a low-precision factorization with iterative refinement to recover a higher accuracy solution of the linear system. For example Haidar et al. [12 ] achieve significant speedups by combining a mixed-precision LU factorization that exploits half-precision NVIDIA Tensor Cores with GMRES-based iterative refinement (GMRES-IR). In addition, GMRES-IR was accelerated using multiple precisions in [4 ] . The strategy in both these works is to use an FP16/TF32/BF16 LU factorization and then use it to precondition the "correction" equation, which is solved with GMRES. Iterative refinement is required to compensate for the fact that FP16 and BF16 both have many fewer mantissa bits than FP64. This implies a larger machine epsilon and hence a less accurate solution. In this work, since we are only
Authors’ Contact Information: Sudhanva Vijaya Kulkarni, Arith Inc, USA, Berkeley, CA, USA, sudhanvakulkarni@berkeley.edu; James W. Demmel, University of California, Berkeley, Berkeley, CA, USA, demmel@berkeley.edu; Laura Grigori, PSI Center for Scientific Computing, Theory and Data and Institute of Mathematics, EPF Lausanne, Switzerland, Lausanne, Switzerland, laura.grigori@epfl.ch. concerned with symmetric positive definite systems, we use the mixed-precision factorization as a preconditioner for a preconditioned conjugate gradient (PCG) rather than GMRES. Mixed-precision PCG with low-precision incomplete Cholesky preconditioners has been analyzed in [5 ] . Low-precision formats like FP16 also provide a much smaller range of representable numbers. Methods to mitigate the effect of the smaller range of available FP16 numbers have been studied in [15 ] . Sahraneshinsamani et al. [32 ] show how pre-pivoting with low-precision GEMMs can eliminate the pivoting bottleneck on GPU-like architectures. Other approaches include using custom data structures such as those described in [3 ] , where a mixed-precision SVD is used to store different blocks of a Block-Low-Rank matrix in different precisions. This has the effect of storing "less significant" (smaller singular value) parts of the blocks in lower precision. Another example is [1 ] , where off-diagonal blocks beyond some user-specified distance away from the diagonal are stored in FP32, and the rest are stored in FP64. A similar idea is applied in [31 ] where the precision to which the off-diagonal blocks are rounded is decided by the magnitude of their Frobenius norm (blocks with a smaller Frobenius norm are stored in lower precision). This idea is further extended in [2 ] while also applying MicroScaling. The Cholesky factorizations described in [1 , 31 ] have the problem of not providing error-bound guarantees, and not accounting for problems such as factorization failure and underflow, which become common in the world of low-precision floats. The concurrent work [6 ] solves this issue by picking the precision of each tile using the ratio between the norm of the tile and the norm of the input matrix. Our work differs from that work in that it also considers emulated GEMMs and can provide either an element-wise or norm-wise error bound rather than only a norm-wise bound. Our algorithm also uses a different precision selection criterion. More recently, FP8/6/4 formats have also become popular due to their utility in machine learning [23 ] . Many FP8 (and shorter) formats are available [28 ] , [25 ] , [35 ] , [11 ] , [20 ] , including the emerging IEEE 3109 standard for low-precision floats [30 ] . When using these low precisions, the Cholesky factorization can break down. We avoid this by using diagonal perturbation, but lose error guaranties in the process.
The second approach relies on emulation, which splits a higher precision GEMM into a sum of many scaled low-precision fixed-point or floating-point GEMMs. Ozaki schemes [26 , 29 ] use lower-precision GEMMs to decompose a floating-point matrix into a sum of matrices whose products can be calculated exactly using low precision and then accumulated using higher precision. This allows FP64 matrix products to be computed using INT8 Tensor Cores, albeit with a weaker roundoff error bound [9 ] . The error bound of a GEMM implementation can receive a grade A, B or C, where the meaning of these grades is displayed in Table 1 . In this paper, we show that using Ozaki (or any "grade B" GEMM as defined in [9 ] ) gives you the same backward error guarantees as the regular floating-point Cholesky [8 ] . Hence, when using emulation, we can simply replace DGEMM or DTRSM with their emulated equivalents, and we do not need an extra refinement phase. Luszczek et al. [21 ] used the Ozaki scheme for the BLAS in an LU decomposition, but did not present an error analysis. Kouya [19 ] also uses Ozaki 2 in an LU decomposition. The Ozaki scheme has also been extended to FP8 [24 ] and FP4 [13 ] tensor cores. NVIDIA recently added a fixed-point INT8 emulation of DGEMM [33 ] to cuBLAS. Other implementations include ozIMMU [27 ] and also the Ozaki 2 implementation [29 ] . We use the cuBLAS implementation for our experiments.
Table 1. GEMM error bound grades for C ^ = fl ( A B ) \hat{C}=\operatorname{fl}(AB) C ^ = fl ( A B ) , where ϵ \epsilon ϵ is machine epsilon and f A f_A f A , f B f_B f B , f C f_C f C are at most low degree polynomial functions of n n n , where for simplicity we assume A A A , B B B and C C C are n × n n\times n n × n matrices.
Contributions
Our contributions may be summarized as follows.
Our main contribution is an adaptive-precision Cholesky factorization that selects the number of Ozaki 1 slices, or the floating-point precision (FP8, FP16, etc.) for GEMM/SYRK and the number of Ozaki 1 slices for TRSM per iteration of Cholesky adaptively to reach a prescribed accuracy.
Table 2. TRSM error bound grades for solving L X = B LX=B L X = B , with computed solution X ^ \hat{X} X ^ , where ϵ \epsilon ϵ is machine epsilon and g A g_A g A , g B g_B g B , g C g_C g C are at most low degree polynomial functions of n n n , where for simplicity we assume L L L , X X X and B B B are n × n n\times n n × n matrices.
In the appendix, we also give an adaptive algorithm for Ozaki 2 that selects the number of moduli for a prescribed accuracy, based on only a first-order error analysis (we keep only the dominant first-order terms of the GEMM error bound).
The adaptive algorithm is enabled by our error analysis: we present an error analysis of Cholesky using emulated BLAS for GEMM/SYRK and TRSM, and an error analysis of Cholesky that uses mixed-precision floating-point Tensor Cores for GEMM/SYRK. We show that an emulated Cholesky obeys the same classical backward error bound as the classical floating-point Cholesky.
We evaluate the emulation algorithm on NVIDIA GH200 GPUs using cuBLAS FP64 fixed-point emulation. We compare 3 strategies, using 1) our adaptive algorithm, 2) static slice counts (7 and 8 slices), and 3) cuBLAS’ automatic dynamic precision scheme [33 ] which decides the number of slices by looking at each GEMM without the context of Cholesky factorization. Our adaptive algorithm is up to 29% faster than the static 7-slice strategy and up to 69% faster than cuBLAS ADP while meeting the same backward error bound. Our success criterion for all three implementations is the double precision backward error bound described in [8 , 14 ] . This means that if Cholesky returns a factor L L L , we check that ∣ a i j − ∑ k l i k l j k ∣ ≤ n ϵ 64 a i i a j j \lvert a_{ij}-\sum_k l_{ik}l_{jk}\rvert\le n\epsilon_{64}\sqrt{a_{ii}a_{jj}} ∣ a ij − ∑ k l ik l j k ∣ ≤ n ϵ 64 a ii a j j , where the subscript i j ij ij denotes the ( i , j ) (i,j) ( i , j ) -th element of the corresponding matrix. We also evaluate the Ozaki 2 variant on synthetic matrices against fixed 12-, 13- and 14-moduli implementations. We test on synthetic matrices and matrices from SuiteSparse [7 , 18 ] .
We use the mixed-precision floating-point version of the algorithm to generate preconditioners (for PCG) with varying tolerances, using a diagonal perturbation to handle factorization failures, and evaluate their effectiveness in reducing the number of CG iterations. If the algorithm is given a higher tolerance, the adaptive Cholesky will use more low-precision GEMMs. On the 13 SuiteSparse matrices we tested, and on synthetic matrices with arithmetic and random spectra, an FP8 preconditioner reduces the number of PCG iterations by up to 12× compared to CG on the power-of-two equilibrated matrix. On synthetic matrices with geometric spectra, the FP8 preconditioner improves the convergence of CG for κ ≤ 10 4 \kappa\le10^4 κ ≤ 1 0 4 . However, the convergence of CG is not improved much on synthetic matrices with κ ≥ 10 8 \kappa\ge10^8 κ ≥ 1 0 8 .
Section 3 proves the lemmas and theorems required for the error analysis of a mixed-precision Cholesky. § 4 carries out the error analysis, and § 5 states and proves the correctness of our adaptive precision algorithm. Our experiments are presented in § 6 .
Background
We first introduce the notation we use throughout the rest of the paper. We use lower case letters for scalars and upper case letters for matrices and submatrices. Additionally, lower case indices are used to denote an element of a matrix and upper case indices are used to denote submatrices. Hence, a i j a_{ij} a ij is the ( i , j ) (i,j) ( i , j ) -th element of the matrix A A A , while A I J A_{IJ} A I J is the ( I , J ) (I,J) ( I , J ) -th submatrix; also, l i k l_{ik} l ik is an entry of L L L and L I K L_{IK} L I K a subblock. Subscripts can be used for both indices (as a i j a_{ij} a ij ) and labels (as ϵ h \epsilon_h ϵ h , ϵ l \epsilon_l ϵ l to denote high and low-precision machine epsilon, respectively). We sometimes refer to the precision using the machine epsilon when it makes sense. Iteration/panel/slice number is denoted with a parenthesized superscript (e.g. ϵ ( J ) \epsilon^{(J)} ϵ ( J ) is the precision chosen at iteration J J J ). We also use MATLAB notation: A [ i 1 : i 2 , j 1 : j 2 ] A[i_1:i_2,j_1:j_2] A [ i 1 : i 2 , j 1 : j 2 ] denotes the submatrix with rows i 1 i_1 i 1 to i 2 i_2 i 2 and columns j 1 j_1 j 1 to j 2 j_2 j 2 . A [ i 1 : i 2 , : ] A[i_1:i_2,:] A [ i 1 : i 2 , : ] refers to rows i 1 i_1 i 1 to i 2 i_2 i 2 and all columns. We use a similar notation for A [ : , j 1 : j 2 ] A[:,j_1:j_2] A [ : , j 1 : j 2 ] . Finally, A [ i , : ] A[i,:] A [ i , : ] is the row i i i and A [ : , j ] A[:,j] A [ : , j ] is the column j j j .
Floating-Point Error Analysis
Floating-point computations are generally not exact. Errors are introduced when a higher precision floating-point number is rounded to a lower precision one. In addition to roundoff errors, there are also underflow and overflow errors. For this paper, we ignore overflow errors and high-precision underflow errors (which could occur during accumulation, but are very small and unlikely). The roundoff model used for high precision binary operations is as follows. For a binary operation ∘ \circ ∘ performed in (high) precision H \mathcal{H} H , we have
fl ( a ∘ b ) = ( a ∘ b ) ( 1 + δ ) (1) \operatorname{fl}(a \circ b) = (a \circ b)(1+\delta) \tag*{(1)} fl ( a ∘ b ) = ( a ∘ b ) ( 1 + δ ) (1)
with ∣ δ ∣ < ϵ h |\delta| < \epsilon_h ∣ δ ∣ < ϵ h , where ϵ h \epsilon_h ϵ h is the machine epsilon. Note that we have ignored the underflow since it is a high-precision operator. If a number a a a is rounded to some lower precision L \mathcal{L} L with gradual underflow, then
fl ( a ) = a ( 1 + δ ) + η (2) \operatorname{fl}(a) = a(1+\delta)+\eta \tag*{(2)} fl ( a ) = a ( 1 + δ ) + η (2)
with ∣ δ ∣ < ϵ l |\delta| < \epsilon_l ∣ δ ∣ < ϵ l , ∣ η ∣ < μ ϵ l |\eta| < \mu\epsilon_l ∣ η ∣ < μ ϵ l , and η ⋅ δ = 0 \eta\cdot\delta= 0 η ⋅ δ = 0 , where μ = 2 e min \mu= 2^{e_{\min}} μ = 2 e m i n and e min e_{\min} e m i n is the smallest exponent in the format [16 ] . We always assume RoundToNearestEven for the rounding mode unless specified otherwise. We also assume gradual underflow for the rest of the document.
Emulation with Ozaki 1
The Ozaki scheme emulates a high-precision matrix product using a sequence of low-precision products [26 , 29 , 37 ] . In this paper, we consider the Ozaki 1 algorithm from [26 ] applied to FP64 matrices with INT8/UINT8 slices. Consider two FP64 matrices A A A , B B B , divided into p ∈ N p \in\mathbb{N} p ∈ N slices, where a slice represents a short set of consecutive mantissa bits in each matrix entry. We can write
A = ∑ q = 1 p A ( q ) + R A ( p ) , B = ∑ q = 1 p B ( q ) + R B ( p ) . (3) A = \sum_{q=1}^{p} A^{(q)} + R_A^{(p)}, \qquad B = \sum_{q=1}^{p} B^{(q)} + R_B^{(p)}. \tag*{(3)} A = q = 1 ∑ p A ( q ) + R A ( p ) , B = q = 1 ∑ p B ( q ) + R B ( p ) . (3)
where each A ( q ) A^{(q)} A ( q ) , B ( q ) B^{(q)} B ( q ) is a scaled low-precision INT8/UINT8 matrix and R A ( p ) = A − ∑ q = 1 p A ( q ) R_A^{(p)} = A - \sum_{q=1}^{p} A^{(q)} R A ( p ) = A − ∑ q = 1 p A ( q ) , R B ( p ) = B − ∑ q = 1 p B ( q ) R_B^{(p)} = B - \sum_{q=1}^{p} B^{(q)} R B ( p ) = B − ∑ q = 1 p B ( q ) are the residuals. The matrix product using p p p slices is approximated by
C ~ ( p ) : = ∑ q = 1 p ∑ r = 1 p − q + 1 A ( q ) B ( r ) (4) \widetilde{C}^{(p)} := \sum_{q=1}^{p} \sum_{r=1}^{p-q+1} A^{(q)}B^{(r)} \tag*{(4)} C ( p ) := q = 1 ∑ p r = 1 ∑ p − q + 1 A ( q ) B ( r ) (4)
which contains 1 2 p ( p + 1 ) \frac{1}{2}p(p+1) 2 1 p ( p + 1 ) low-precision GEMMs. Each A ( q ) B ( r ) A^{(q)}B^{(r)} A ( q ) B ( r ) is computed exactly. We let C ^ ( p ) \widehat{C}^{(p)} C ( p ) denote the final result of accumulating all the products A ( q ) B ( r ) A^{(q)}B^{(r)} A ( q ) B ( r ) using FP64. Then, for the uniform INT8 slice case described in [37 ] , this gives the following error bound
∣ C ^ ( p ) − C ~ ( p ) ∣ ≤ ( 1 2 p ( p + 1 ) − 1 ) ϵ 64 ∑ q = 1 p ∑ r = 1 p − q + 1 ∣ A ( q ) B ( r ) ∣ ≤ ( 1 2 p ( p + 1 ) − 1 ) ϵ 64 ∑ q = 1 p ∑ r = 1 p − q + 1 ∣ A ( q ) ∣ ∣ B ( r ) ∣ ≤ ( 1 2 p ( p + 1 ) − 1 ) ϵ 64 ∣ A ∣ ∣ B ∣ (5) \begin{aligned}
\left|\widehat{C}^{(p)}-\widetilde{C}^{(p)}\right|
\leq\left(\frac{1}{2}p(p+1)-1\right)\epsilon_{64}
\sum_{q=1}^{p}\sum_{r=1}^{p-q+1}\left|A^{(q)}B^{(r)}\right| \\
&\leq\left(\frac{1}{2}p(p+1)-1\right)\epsilon_{64}
\sum_{q=1}^{p}\sum_{r=1}^{p-q+1}\left|A^{(q)}\right|\left|B^{(r)}\right|
\leq\left(\frac{1}{2}p(p+1)-1\right)\epsilon_{64}|A||B| \tag*{(5)}
\end{aligned} C ( p ) − C ( p ) ≤ ( 2 1 p ( p + 1 ) − 1 ) ϵ 64 q = 1 ∑ p r = 1 ∑ p − q + 1 A ( q ) B ( r ) ≤ ( 2 1 p ( p + 1 ) − 1 ) ϵ 64 q = 1 ∑ p r = 1 ∑ p − q + 1 A ( q ) B ( r ) ≤ ( 2 1 p ( p + 1 ) − 1 ) ϵ 64 ∣ A ∣∣ B ∣ (5)
Current implementations of Ozaki 1 FP64 emulation with integers either use INT8 for all slices [37 ] , or use INT8 for the first slice (to capture the sign bit) and UINT8 for the remaining slices [33 ] . The latter approach can capture more mantissa bits while using the same number of slices. An error analysis for the uniform INT8 case was presented in [37 ] . Here, we partially extend that analysis to the cuBLAS implementation described in [33 ] . Let d : N → N d : \mathbb{N} \to\mathbb{N} d : N → N denote a slice width function, where d ( q ) d(q) d ( q ) is the number of mantissa bits represented by the q q q -th slice. Define d ( 0 ) = 0 d(0)=0 d ( 0 ) = 0 . Then, the effective mantissa width of a q q q -slice emulation is
w ( q ) = ∑ r = 0 q d ( r ) . (6) w(q)=\sum_{r=0}^{q}d(r). \tag*{(6)} w ( q ) = r = 0 ∑ q d ( r ) . (6)
Two examples are:
Uniform slices. If every slice is INT8, then d ( q ) = 7 d(q)=7 d ( q ) = 7 for q > 0 q>0 q > 0 , and d ( 0 ) = 0 d(0)=0 d ( 0 ) = 0 . So w ( q ) = 7 q w(q)=7q w ( q ) = 7 q .
Signed leading slice. If the leading slice is stored as signed INT8 and subsequent slices as unsigned UINT8, then d ( 0 ) = w ( 0 ) = 0 d(0)=w(0)=0 d ( 0 ) = w ( 0 ) = 0 , d ( 1 ) = w ( 1 ) = 7 d(1)=w(1)=7 d ( 1 ) = w ( 1 ) = 7 and d ( q ) = 8 d(q)=8 d ( q ) = 8 for q ≥ 2 q\geq2 q ≥ 2 , giving w ( q ) = 7 + 8 ( q − 1 ) = 8 q − 1 w(q)=7+8(q-1)=8q-1 w ( q ) = 7 + 8 ( q − 1 ) = 8 q − 1 .
Error analysis
Consider multiplying A A A , B B B using a p p p -slice Ozaki 1 scheme. Define the vectors g g g and f f f with g i = ufp ( max j ∣ a i j ∣ ) g_i=\operatorname{ufp}(\max_j|a_{ij}|) g i = ufp ( max j ∣ a ij ∣ ) and f j = ufp ( max i ∣ b i j ∣ ) f_j=\operatorname{ufp}(\max_i|b_{ij}|) f j = ufp ( max i ∣ b ij ∣ ) . Here ufp (unit in the first place) is defined as ufp ( y ) = 2 ⌊ log 2 ∣ y ∣ ⌋ \operatorname{ufp}(y)=2^{\lfloor\log_2|y|\rfloor} ufp ( y ) = 2 ⌊ l o g 2 ∣ y ∣ ⌋ for y ≠ 0 y\ne0 y = 0 , and ufp ( 0 ) : = 0 \operatorname{ufp}(0):=0 ufp ( 0 ) := 0 . The error in an Ozaki 1 GEMM consists of 2 parts, the truncation error and the accumulation error. First, we bound the truncation error ∣ A B − C ~ ( p ) ∣ |AB-\widetilde{C}^{(p)}| ∣ A B − C ( p ) ∣ due to splitting the mantissa bits. This requires bounding ∣ A ( q ) ∣ |A^{(q)}| ∣ A ( q ) ∣ , ∣ R A ( q ) ∣ |R_A^{(q)}| ∣ R A ( q ) ∣ , ∣ B ( q ) ∣ |B^{(q)}| ∣ B ( q ) ∣ , ∣ R B ( q ) ∣ |R_B^{(q)}| ∣ R B ( q ) ∣ , for 1 ≤ q ≤ p 1\leq q\leq p 1 ≤ q ≤ p . These can be bounded above using bounds similar to those in [37 ] :
∣ A ( q ) ∣ ≤ 2 − w ( q − 1 ) + 1 g 1 T , ∣ R A ( q ) ∣ ≤ 2 − w ( q ) + 1 g 1 T , |A^{(q)}|\leq2^{-w(q-1)+1}g_1^T,\qquad|R_A^{(q)}|\leq2^{-w(q)+1}g_1^T, ∣ A ( q ) ∣ ≤ 2 − w ( q − 1 ) + 1 g 1 T , ∣ R A ( q ) ∣ ≤ 2 − w ( q ) + 1 g 1 T ,
∣ B ( q ) ∣ ≤ 2 − w ( q − 1 ) + 1 1 f T , ∣ R B ( q ) ∣ ≤ 2 − w ( q ) + 1 1 f T (7) |B^{(q)}|\leq2^{-w(q-1)+1}1f^T,\qquad|R_B^{(q)}|\leq2^{-w(q)+1}1f^T \tag*{(7)} ∣ B ( q ) ∣ ≤ 2 − w ( q − 1 ) + 1 1 f T , ∣ R B ( q ) ∣ ≤ 2 − w ( q ) + 1 1 f T (7)
for every $1\leq q\leq p$. Also, ∣ A ∣ ≤ 2 ⋅ g 1 T |A|\leq2\cdot g_1^T ∣ A ∣ ≤ 2 ⋅ g 1 T , and ∣ B ∣ ≤ 2 ⋅ 1 f T |B|\leq2\cdot1f^T ∣ B ∣ ≤ 2 ⋅ 1 f T , where 1 1 1 is the column vector of all 1s. From (3), we have A B − C ~ ( p ) = ( ∑ q = 1 p A ( q ) + R A ( p ) ) ( ∑ q = 1 p B ( q ) + R B ( p ) ) − ∑ q = 1 p ∑ r = 1 p − q + 1 A ( q ) B ( r ) AB-\widetilde{C}^{(p)}=\left(\sum_{q=1}^{p}A^{(q)}+R_A^{(p)}\right)\left(\sum_{q=1}^{p}B^{(q)}+R_B^{(p)}\right)-\sum_{q=1}^{p}\sum_{r=1}^{p-q+1}A^{(q)}B^{(r)} A B − C ( p ) = ( ∑ q = 1 p A ( q ) + R A ( p ) ) ( ∑ q = 1 p B ( q ) + R B ( p ) ) − ∑ q = 1 p ∑ r = 1 p − q + 1 A ( q ) B ( r ) . To transform the residual into a more convenient form, write ∑ q = 1 p ∑ r = 1 p − q + 1 A ( q ) B ( r ) = ∑ q = 1 p A ( q ) ( ∑ r = 1 p − q + 1 B ( r ) ) \sum_{q=1}^{p}\sum_{r=1}^{p-q+1}A^{(q)}B^{(r)}=\sum_{q=1}^{p}A^{(q)}\left(\sum_{r=1}^{p-q+1}B^{(r)}\right) ∑ q = 1 p ∑ r = 1 p − q + 1 A ( q ) B ( r ) = ∑ q = 1 p A ( q ) ( ∑ r = 1 p − q + 1 B ( r ) ) . Now, since B = ∑ r = 1 p B ( r ) + R B ( p ) B=\sum_{r=1}^{p}B^{(r)}+R_B^{(p)} B = ∑ r = 1 p B ( r ) + R B ( p ) , we can write $\sum_{q=1}^{p}A^{(q)}\left(\sum_{r=1}^{p-q+1}B^{(r)}\right)=\sum_{q=1}^{p}A^{(q)}\left(B-R_B^{(p-q+1)}\right)=\left(\sum_{q=1}^{p}A^{(q)}\right)B-\sum_{q=1}^{p}A^{(q)}R_B^{(p-q+1)}=\left(A-R_A^{(p)}\right)B∑ q = 1 p A ( q ) R B ( p − q + 1 ) \sum_{q=1}^{p} A^{(q)}R_B^{(p-q+1)} ∑ q = 1 p A ( q ) R B ( p − q + 1 ) . Then by the triangle inequality,
∣ A B − C ~ ( p ) ∣ ≤ ∑ q = 1 p ∣ A ( q ) ∣ ∣ R B ( p − q + 1 ) ∣ + ∣ R A ( p ) ∣ ∣ B ∣ ≤ ∑ q = 1 p 2 − w ( q − 1 ) + 1 g 1 T ⋅ 2 − w ( p − q + 1 ) + 1 1 f T + 2 ⋅ 2 − w ( p ) + 1 g 1 T ⋅ 1 f T . \left|AB-\widetilde{C}^{(p)}\right|\leq\sum_{q=1}^{p}\left|A^{(q)}\right|\left|R_B^{(p-q+1)}\right|+\left|R_A^{(p)}\right||B|\leq\sum_{q=1}^{p}2^{-w(q-1)+1}g\mathbf{1}^{T}\cdot2^{-w(p-q+1)+1}\mathbf{1}f^{T}+2\cdot2^{-w(p)+1}g\mathbf{1}^{T}\cdot\mathbf{1}f^{T}. A B − C ( p ) ≤ q = 1 ∑ p A ( q ) R B ( p − q + 1 ) + R A ( p ) ∣ B ∣ ≤ q = 1 ∑ p 2 − w ( q − 1 ) + 1 g 1 T ⋅ 2 − w ( p − q + 1 ) + 1 1 f T + 2 ⋅ 2 − w ( p ) + 1 g 1 T ⋅ 1 f T .
Using 1 T 1 = n \mathbf{1}^{T}\mathbf{1}=n 1 T 1 = n , each term in the first sum equals 4 n 2 − ( w ( q − 1 ) + w ( p − q + 1 ) ) g f T 4n\,2^{-(w(q-1)+w(p-q+1))}gf^{T} 4 n 2 − ( w ( q − 1 ) + w ( p − q + 1 )) g f T . Assuming p > 2 p>2 p > 2 , for q = 1 q=1 q = 1 , w ( 0 ) + w ( p ) = 8 p − 1 w(0)+w(p)=8p-1 w ( 0 ) + w ( p ) = 8 p − 1 , and for q > 1 q>1 q > 1 , w ( q − 1 ) + w ( p − q + 1 ) = 8 ( q − 1 ) − 1 + 8 ( p − q + 1 ) − 1 = 8 p − 2 w(q-1)+w(p-q+1)=8(q-1)-1+8(p-q+1)-1=8p-2 w ( q − 1 ) + w ( p − q + 1 ) = 8 ( q − 1 ) − 1 + 8 ( p − q + 1 ) − 1 = 8 p − 2 , so
∣ A B − C ~ ( p ) ∣ ≤ 4 n p 2 − 8 p + 2 g f T + 2 n 2 − 8 p + 2 g f T ≤ 4 ( p + 1 2 ) n 2 − 8 p + 2 g f T . (8) \left|AB-\widetilde{C}^{(p)}\right|\leq4np\,2^{-8p+2}gf^{T}+2n\,2^{-8p+2}gf^{T}\leq4\left(p+\frac{1}{2}\right)n\,2^{-8p+2}gf^{T}. \tag*{(8)} A B − C ( p ) ≤ 4 n p 2 − 8 p + 2 g f T + 2 n 2 − 8 p + 2 g f T ≤ 4 ( p + 2 1 ) n 2 − 8 p + 2 g f T . (8)
For the accumulation error, we assume that the cuBLAS Ozaki 1 has the same accumulation error as (5) . Adding together truncation and accumulation errors (from (8) and (5) respectively) gives us
∣ A B − C ^ ( p ) ∣ ≤ ∣ A B − C ~ ( p ) ∣ + ∣ C ~ ( p ) − C ^ ( p ) ∣ ≤ 4 ( p + 1 2 ) n 2 − 8 p + 2 g f T + ( 1 2 ( p ( p + 1 ) ) − 1 ) ϵ 64 ∣ A ∣ ∣ B ∣ . (9) \left|AB-\widehat{C}^{(p)}\right|\leq\left|AB-\widetilde{C}^{(p)}\right|+\left|\widetilde{C}^{(p)}-\widehat{C}^{(p)}\right|\leq4\left(p+\frac{1}{2}\right)n2^{-8p+2}gf^{T}+\left(\frac{1}{2}(p(p+1))-1\right)\epsilon_{64}|A||B|. \tag*{(9)} A B − C ( p ) ≤ A B − C ( p ) + C ( p ) − C ( p ) ≤ 4 ( p + 2 1 ) n 2 − 8 p + 2 g f T + ( 2 1 ( p ( p + 1 )) − 1 ) ϵ 64 ∣ A ∣∣ B ∣. (9)
For convenience, we define Θ ( p , n ) = 4 ( p + 1 2 ) n 2 − 8 p + 2 \Theta(p,n)=4\left(p+\frac{1}{2}\right)n2^{-8p+2} Θ ( p , n ) = 4 ( p + 2 1 ) n 2 − 8 p + 2 and Γ ( p ) = 1 2 p ( p + 1 ) − 1 \Gamma(p)=\frac{1}{2}p(p+1)-1 Γ ( p ) = 2 1 p ( p + 1 ) − 1 , and write our error bound as
∣ A B − C ^ ( p ) ∣ ≤ Θ ( p , n ) g f T + Γ ( p ) ϵ 64 ∣ A ∣ ∣ B ∣ . (10) \left|AB-\widehat{C}^{(p)}\right|\leq\Theta(p,n)gf^{T}+\Gamma(p)\epsilon_{64}|A||B|. \tag*{(10)} A B − C ( p ) ≤ Θ ( p , n ) g f T + Γ ( p ) ϵ 64 ∣ A ∣∣ B ∣. (10)
TRSM with Ozaki 1
We consider solving the system L X = B LX=B L X = B where L ∈ R n × n L\in\mathbb{R}^{n\times n} L ∈ R n × n is a lower triangular matrix and B ∈ R n × m B\in\mathbb{R}^{n\times m} B ∈ R n × m is a general rectangular matrix. We want to solve for X X X which has the same dimensions as B B B . There are many possible implementations of TRSM, here we use the textbook blocked implementation of TRSM (shown in Algorithm 1, using block size n b n_b n b ) for simplicity. Off-diagonal updates are computed using emulated GEMM, and the smaller TRSM with the n b × n b n_b\times n_b n b × n b diagonal block L I I L_{II} L I I is done in regular FP64 floating point. We obtain the following theorem.
Theorem 3.1. Let L ∈ R n × n L\in\mathbb{R}^{n\times n} L ∈ R n × n , B ∈ R n × m B\in\mathbb{R}^{n\times m} B ∈ R n × m , and X ∈ R n × m X\in\mathbb{R}^{n\times m} X ∈ R n × m . We solve L X = B LX=B L X = B using a blocked TRSM (with block size n b n_b n b ) where off-diagonal updates are emulated with Ozaki 1 using p p p slices. Then, for block row and column indices I , J I,J I , J , the computed solution X ^ \widehat{X} X satisfies:
∣ L X ^ − B ∣ I J ≤ n b ϵ 64 ∣ L I I ∣ ∣ X ^ I J ∣ + Θ ( p , n ) [ ρ χ T ] I J + ( Γ ( p ) + 1 ) ϵ 64 ( ∑ K = 1 I − 1 ∣ L I K ∣ ∣ X ^ K J ∣ + ∣ B I J ∣ ) (11) \left|L\widehat{X}-B\right|_{IJ}\leq n_b\epsilon_{64}|L_{II}||\widehat{X}_{IJ}|+\Theta(p,n)[\rho\chi^{T}]_{IJ}+(\Gamma(p)+1)\epsilon_{64}\left(\sum_{K=1}^{I-1}|L_{IK}||\widehat{X}_{KJ}|+|B_{IJ}|\right) \tag*{(11)} L X − B I J ≤ n b ϵ 64 ∣ L I I ∣∣ X I J ∣ + Θ ( p , n ) [ ρ χ T ] I J + ( Γ ( p ) + 1 ) ϵ 64 ( K = 1 ∑ I − 1 ∣ L I K ∣∣ X K J ∣ + ∣ B I J ∣ ) (11) where ρ ∈ R n \rho\in\mathbb{R}^{n} ρ ∈ R n is a column vector with ρ i = max j = 1 n ∣ l i j ∣ \rho_i=\max_{j=1}^{n}|l_{ij}| ρ i = max j = 1 n ∣ l ij ∣ , and χ ∈ R m \chi\in\mathbb{R}^{m} χ ∈ R m is a column vector with χ j = max i = 1 n ∣ x ^ i j ∣ \chi_j=\max_{i=1}^{n}|\widehat{x}_{ij}| χ j = max i = 1 n ∣ x ij ∣ , where i , j i,j i , j are row and column indices, and [ ρ χ T ] I J [\rho\chi^{T}]_{IJ} [ ρ χ T ] I J refers to the I , J I,J I , J -th subblock of the n × m n\times m n × m matrix ρ χ T \rho\chi^{T} ρ χ T . Also, Θ ( p , n ) = 4 ( p + 1 2 ) n 2 − 8 p + 2 \Theta(p,n)=4\left(p+\frac{1}{2}\right)n2^{-8p+2} Θ ( p , n ) = 4 ( p + 2 1 ) n 2 − 8 p + 2 , and Γ ( p ) = 1 2 p ( p + 1 ) − 1 \Gamma(p)=\frac{1}{2}p(p+1)-1 Γ ( p ) = 2 1 p ( p + 1 ) − 1 .
Algorithm 1 Blocked Right-Looking TRSM: Solve L X = B LX=B L X = B
Require: Lower-triangular L ∈ R n × n L \in\mathbb{R}^{n \times n} L ∈ R n × n , B ∈ R n × m B \in\mathbb{R}^{n \times m} B ∈ R n × m , block size n b n_b n b
Ensure: X ∈ R n × m X \in\mathbb{R}^{n \times m} X ∈ R n × m such that L X = B LX=B L X = B
Partition L L L and B B B into n / n b × m / n b n/n_b \times m/n_b n / n b × m / n b blocks of size n b × n b n_b \times n_b n b × n b with L I J L_{IJ} L I J , B I J B_{IJ} B I J the ( I , J ) (I,J) ( I , J ) submatrices
for J = 1 , … , m / n b J=1,\ldots,m/n_b J = 1 , … , m / n b do
for I = 1 , … , n / n b I=1,\ldots,n/n_b I = 1 , … , n / n b do
for K = 1 , … , I − 1 K=1,\ldots,I-1 K = 1 , … , I − 1 do
B I J ← B I J − L I K X K J B_{IJ} \leftarrow B_{IJ}-L_{IK}X_{KJ} B I J ← B I J − L I K X K J ▹ GEMM update of trailing panels
end for
X I J ← L I I − 1 B I J X_{IJ} \leftarrow L_{II}^{-1}B_{IJ} X I J ← L I I − 1 B I J ▹ Unblocked TRSM using FP64
end for
end for
Proof. For simplicity, assume n b ∣ n n_b \mid n n b ∣ n and n b ∣ m n_b \mid m n b ∣ m . We need to do a floating-point TRSM on the updated block B ^ I J = B I J − ∑ K = 1 I − 1 L I K X ^ K J + E I J \widehat{B}_{IJ}=B_{IJ}-\sum_{K=1}^{I-1}L_{IK}\widehat{X}_{KJ}+E_{IJ} B I J = B I J − ∑ K = 1 I − 1 L I K X K J + E I J , where E I J E_{IJ} E I J is the error from performing the I − 1 I-1 I − 1 GEMMs with Ozaki 1 using p p p slices. For each L I K , X ^ K J L_{IK},\widehat{X}_{KJ} L I K , X K J , denote their column vectors of row and column ufp maxnorms respectively as g ( I , K ) , f ( J , K ) g^{(I,K)},f^{(J,K)} g ( I , K ) , f ( J , K ) . Note that g ( I , K ) g^{(I,K)} g ( I , K ) and L I K L_{IK} L I K are 0 in all elements for blocks with I < K I<K I < K . Then,
∣ E I J ∣ ≤ ∑ K = 1 I − 1 Θ ( p , n b ) g ( I , K ) f ( J , K ) T + ( Γ ( p ) + I ) ϵ 64 ( ∑ K = 1 I − 1 ∣ L I K ∣ ∣ X ^ K J ∣ + ∣ B I J ∣ ) ≤ ∑ K = 1 n / n b Θ ( p , n b ) g ( I , K ) f ( J , K ) T + ( Γ ( p ) + I ) ϵ 64 ( ∑ K = 1 I − 1 ∣ L I K ∣ ∣ X ^ K J ∣ + ∣ B I J ∣ ) , (12) \begin{aligned}
\lvert E_{IJ}\rvert\le\sum_{K=1}^{I-1}\Theta(p,n_b)g^{(I,K)}f^{(J,K)T}+(\Gamma(p)+I)\epsilon_{64}\left(\sum_{K=1}^{I-1}\lvert L_{IK}\rvert\lvert\widehat{X}_{KJ}\rvert+\lvert B_{IJ}\rvert\right) \\
&\le\sum_{K=1}^{n/n_b}\Theta(p,n_b)g^{(I,K)}f^{(J,K)T}+(\Gamma(p)+I)\epsilon_{64}\left(\sum_{K=1}^{I-1}\lvert L_{IK}\rvert\lvert\widehat{X}_{KJ}\rvert+\lvert B_{IJ}\rvert\right), \tag*{(12)}
\end{aligned} ∣ E I J ∣ ≤ K = 1 ∑ I − 1 Θ ( p , n b ) g ( I , K ) f ( J , K ) T + ( Γ ( p ) + I ) ϵ 64 ( K = 1 ∑ I − 1 ∣ L I K ∣ ∣ X K J ∣ + ∣ B I J ∣ ) ≤ K = 1 ∑ n / n b Θ ( p , n b ) g ( I , K ) f ( J , K ) T + ( Γ ( p ) + I ) ϵ 64 ( K = 1 ∑ I − 1 ∣ L I K ∣ ∣ X K J ∣ + ∣ B I J ∣ ) , (12)
where we increase the upper limit on the first sum from I − 1 I-1 I − 1 to n / n b n/n_b n / n b , which is an upper bound since all extra terms are positive. This is also a more convenient form for the inequality, since it includes the diagonal block. The first term on the right hand side is just the sum of all truncation errors. The second term is a collection of all the accumulation errors. To see why we get an extra Γ ( p ) + I \Gamma(p)+I Γ ( p ) + I factor for the accumulation error, notice that every p p p -slice Ozaki 1 gives us Γ ( p ) + 1 \Gamma(p)+1 Γ ( p ) + 1 FP64 product matrices (which have been multiplied exactly in INT8) to accumulate. After accumulating each of these, we need to sum up the I − 1 I-1 I − 1 L I K X ^ K J L_{IK}\widehat{X}_{KJ} L I K X K J products that we need to subtract from B I J B_{IJ} B I J . Assuming accumulation is done in a nested manner with the Γ ( p ) + 1 \Gamma(p)+1 Γ ( p ) + 1 terms accumulated first, then the I − 1 I-1 I − 1 partial sums accumulated after, we get at most Γ ( p ) + I \Gamma(p)+I Γ ( p ) + I roundoff terms on each element of the result matrix B I J − ∑ K = 1 I − 1 L I K X ^ K J B_{IJ}-\sum_{K=1}^{I-1}L_{IK}\widehat{X}_{KJ} B I J − ∑ K = 1 I − 1 L I K X K J , thus giving us the required factor. Next, each entry of g ( I , K ) , f ( J , K ) g^{(I,K)},f^{(J,K)} g ( I , K ) , f ( J , K ) is bounded above by the corresponding entry of ρ , χ \rho,\chi ρ , χ respectively (since the former is max over submatrices). Thus, we can rewrite (12) as
∣ E I J ∣ ≤ Θ ( p , n ) [ ρ χ T ] I J + ( Γ ( p ) + I ) ϵ 64 ( ∑ K = 1 I − 1 ∣ L I K ∣ ∣ X ^ K J ∣ + ∣ B I J ∣ ) . (13) \lvert E_{IJ}\rvert\le\Theta(p,n)[\rho\chi^{T}]_{IJ}+(\Gamma(p)+I)\epsilon_{64}\left(\sum_{K=1}^{I-1}\lvert L_{IK}\rvert\lvert\widehat{X}_{KJ}\rvert+\lvert B_{IJ}\rvert\right). \tag*{(13)} ∣ E I J ∣ ≤ Θ ( p , n ) [ ρ χ T ] I J + ( Γ ( p ) + I ) ϵ 64 ( K = 1 ∑ I − 1 ∣ L I K ∣ ∣ X K J ∣ + ∣ B I J ∣ ) . (13)
When we perform the n b × n b n_b \times n_b n b × n b TRSM with L I I L_{II} L I I , we know from [14 ] that for every column j j j , X ^ I J \widehat{X}_{IJ} X I J satisfies
( L I I + Δ L I I ( j ) ) X ^ I J [ : , j ] = B ^ I J [ : , j ] ⟹ B ^ I J [ : , j ] − L I I X ^ I J [ : , j ] = Δ L I I ( j ) X ^ I J [ : , j ] , (L_{II}+\Delta L_{II}^{(j)})\widehat{X}_{IJ}[:,j]=\widehat{B}_{IJ}[:,j]\Longrightarrow\widehat{B}_{IJ}[:,j]-L_{II}\widehat{X}_{IJ}[:,j]=\Delta L_{II}^{(j)}\widehat{X}_{IJ}[:,j], ( L I I + Δ L I I ( j ) ) X I J [ : , j ] = B I J [ : , j ] ⟹ B I J [ : , j ] − L I I X I J [ : , j ] = Δ L I I ( j ) X I J [ : , j ] ,
with ∣ Δ L I I ( j ) ∣ ≤ n b ϵ 64 ∣ L I I ∣ \lvert\Delta L_{II}^{(j)}\rvert\le n_b\epsilon_{64}\lvert L_{II}\rvert ∣ Δ L I I ( j ) ∣ ≤ n b ϵ 64 ∣ L I I ∣ for each column 0 ≤ j ≤ n b 0\le j\le n_b 0 ≤ j ≤ n b . So, the inequality per column is ∣ B ^ I J [ : , j ] − L I I X ^ I J [ : , j ] ∣ ≤ n b ϵ 64 ∣ L I I ∣ ∣ X ^ I J [ : , j ] ∣ \lvert\widehat{B}_{IJ}[:,j]-L_{II}\widehat{X}_{IJ}[:,j]\rvert\le n_b\epsilon_{64}\lvert L_{II}\rvert\lvert\widehat{X}_{IJ}[:,j]\rvert ∣ B I J [ : , j ] − L I I X I J [ : , j ]∣ ≤ n b ϵ 64 ∣ L I I ∣ ∣ X I J [ : , j ]∣ . Since this holds for every column, we can say ∣ B ^ I J − L I I X ^ I J ∣ ≤ n b ϵ 64 ∣ L I I ∣ ∣ X ^ I J ∣ \lvert\widehat{B}_{IJ}-L_{II}\widehat{X}_{IJ}\rvert\le n_b\epsilon_{64}\lvert L_{II}\rvert\lvert\widehat{X}_{IJ}\rvert ∣ B I J − L I I X I J ∣ ≤ n b ϵ 64 ∣ L I I ∣ ∣ X I J ∣ . Thus, ∣ B I J \lvert B_{IJ} ∣ B I J ∑ K = 1 I L I K X ^ K J ≤ ∣ ( B I J − ∑ K = 1 I − 1 L I K X ^ K J ) − B ^ I J ∣ + ∣ B ^ I J − L I I X ^ I J ∣ \sum_{K=1}^{I} L_{IK}\widehat{X}_{KJ} \leq|(B_{IJ}-\sum_{K=1}^{I-1}L_{IK}\widehat{X}_{KJ})-\widehat{B}_{IJ}|+|\widehat{B}_{IJ}-L_{II}\widehat{X}_{IJ}| ∑ K = 1 I L I K X K J ≤ ∣ ( B I J − ∑ K = 1 I − 1 L I K X K J ) − B I J ∣ + ∣ B I J − L I I X I J ∣ can be written as
∣ B I J − ∑ K = 1 I L I K X ^ K J ∣ ≤ n b ϵ 64 ∥ L I I ∥ ∥ X ^ I J ∥ + Θ ( p , n ) [ ρ X T ] I J + ( Γ ( p ) + I ) ϵ 64 ( ∑ K = 1 I − 1 ∣ L I K ∣ ∥ X ^ K J ∥ + ∣ B I J ∣ ) . (14) \left|B_{IJ}-\sum_{K=1}^{I}L_{IK}\widehat{X}_{KJ}\right| \leq n_b\epsilon_{64}\lVert L_{II}\rVert\lVert\widehat{X}_{IJ}\rVert+\Theta(p,n)[\rho X^T]_{IJ}+\left(\Gamma(p)+I\right)\epsilon_{64}\left(\sum_{K=1}^{I-1}|L_{IK}|\lVert\widehat{X}_{KJ}\rVert+|B_{IJ}|\right). \tag*{(14)} B I J − K = 1 ∑ I L I K X K J ≤ n b ϵ 64 ∥ L I I ∥ ∥ X I J ∥ + Θ ( p , n ) [ ρ X T ] I J + ( Γ ( p ) + I ) ϵ 64 ( K = 1 ∑ I − 1 ∣ L I K ∣ ∥ X K J ∥ + ∣ B I J ∣ ) . (14)
For simplicity, we relax the bound in (14) as
∣ B I J − ∑ K = 1 I L I K X ^ K J ∣ ≤ Θ ( p , n ) [ ρ X T ] I J + ϵ 64 max ( n b , Γ ( p ) + I ) ( ∑ K = 1 I ∣ L I K ∣ ∥ X ^ K J ∥ + ∣ B I J ∣ ) . (15) \left|B_{IJ}-\sum_{K=1}^{I}L_{IK}\widehat{X}_{KJ}\right| \leq\Theta(p,n)[\rho X^T]_{IJ}+\epsilon_{64}\max\left(n_b,\Gamma(p)+I\right)\left(\sum_{K=1}^{I}|L_{IK}|\lVert\widehat{X}_{KJ}\rVert+|B_{IJ}|\right). \tag*{(15)} B I J − K = 1 ∑ I L I K X K J ≤ Θ ( p , n ) [ ρ X T ] I J + ϵ 64 max ( n b , Γ ( p ) + I ) ( K = 1 ∑ I ∣ L I K ∣ ∥ X K J ∥ + ∣ B I J ∣ ) . (15)
Since I ≤ n n b I\leq\frac{n}{n_b} I ≤ n b n , we obtain
∣ B I J − ∑ K = 1 I L I K X ^ K J ∣ ≤ Θ ( p , n ) [ ρ X T ] I J + ϵ 64 max ( n b , Γ ( p ) + n n b ) ( ∑ K = 1 n / n b ∣ L I K ∣ ∥ X ^ K J ∥ + ∣ B I J ∣ ) ⇒ ∣ B − L X ^ ∣ ≤ Θ ( p , n ) ρ X T + ϵ 64 max ( n b , Γ ( p ) + n n b ) ( ∣ L ∣ ∥ X ^ ∥ + ∣ B ∣ ) . (16) \begin{aligned}
\left|B_{IJ}-\sum_{K=1}^{I}L_{IK}\widehat{X}_{KJ}\right| \leq\Theta(p,n)[\rho X^T]_{IJ}+\epsilon_{64}\max\left(n_b,\Gamma(p)+\frac{n}{n_b}\right)\left(\sum_{K=1}^{n/n_b}|L_{IK}|\lVert\widehat{X}_{KJ}\rVert+|B_{IJ}|\right) \\
&\Rightarrow\quad|B-L\widehat{X}| \leq\Theta(p,n)\rho X^T+\epsilon_{64}\max\left(n_b,\Gamma(p)+\frac{n}{n_b}\right)\left(|L|\lVert\widehat{X}\rVert+|B|\right). \tag*{(16)}
\end{aligned} B I J − K = 1 ∑ I L I K X K J ≤ Θ ( p , n ) [ ρ X T ] I J + ϵ 64 max ( n b , Γ ( p ) + n b n ) K = 1 ∑ n / n b ∣ L I K ∣ ∥ X K J ∥ + ∣ B I J ∣ ⇒ ∣ B − L X ∣ ≤ Θ ( p , n ) ρ X T + ϵ 64 max ( n b , Γ ( p ) + n b n ) ( ∣ L ∣ ∥ X ∥ + ∣ B ∣ ) . (16)
If we would like to remove B B B from the right hand side, define R = B − L X ^ R=B-L\widehat{X} R = B − L X . Then, B = R + L X ^ B=R+L\widehat{X} B = R + L X , and we can rewrite (16) as
∣ R ∣ ≤ Θ ( p , n ) ρ X T + ϵ 64 max ( n b , Γ ( p ) + n n b ) ( 2 ∣ L ∣ ∥ X ^ ∥ + ∣ R ∣ ) ⇒ ∣ R ∣ ≤ 1 1 − ϵ 64 max ( n b , Γ ( p ) + n n b ) ( Θ ( p , n ) ρ X T + ϵ 64 max ( n b , Γ ( p ) + n n b ) ( 2 ∣ L ∣ ∥ X ^ ∥ ) ) . (17) \begin{aligned}
|R|\leq\Theta(p,n)\rho X^T+\epsilon_{64}\max\left(n_b,\Gamma(p)+\frac{n}{n_b}\right)\left(2|L|\lVert\widehat{X}\rVert+|R|\right) \\
&\Rightarrow\quad|R|\leq\frac{1}{1-\epsilon_{64}\max\left(n_b,\Gamma(p)+\frac{n}{n_b}\right)}\left(\Theta(p,n)\rho X^T+\epsilon_{64}\max\left(n_b,\Gamma(p)+\frac{n}{n_b}\right)\left(2|L|\lVert\widehat{X}\rVert\right)\right). \tag*{(17)}
\end{aligned} ∣ R ∣ ≤ Θ ( p , n ) ρ X T + ϵ 64 max ( n b , Γ ( p ) + n b n ) ( 2∣ L ∣ ∥ X ∥ + ∣ R ∣ ) ⇒ ∣ R ∣ ≤ 1 − ϵ 64 max ( n b , Γ ( p ) + n b n ) 1 ( Θ ( p , n ) ρ X T + ϵ 64 max ( n b , Γ ( p ) + n b n ) ( 2∣ L ∣ ∥ X ∥ ) ) . (17)
Corollary 3.1.1. Blocked TRSM with grade “B” GEMM also gets “B”. [9 ]
Proof. We can easily rewrite (17) in terms of row and column norms by noting that ∣ L ∣ ∥ X ^ ∥ ≤ n ρ X T |L|\lVert\widehat{X}\rVert\leq n\rho X^T ∣ L ∣ ∥ X ∥ ≤ n ρ X T . Therefore
∣ R ∣ ≤ 1 1 − ϵ 64 max ( n b , Γ ( p ) + n n b ) ( Θ ( p , n ) ρ X T + ϵ 64 max ( n b , Γ ( p ) + n n b ) ( 2 ∣ L ∣ ∥ X ^ ∥ ) ) ≤ 1 1 − ϵ 64 max ( n b , Γ ( p ) + n n b ) ( Θ ( p , n ) + 2 n ϵ 64 max ( n b , Γ ( p ) + n n b ) ) ( ρ X T ) . \begin{aligned}
|R|\leq\frac{1}{1-\epsilon_{64}\max\left(n_b,\Gamma(p)+\frac{n}{n_b}\right)}\left(\Theta(p,n)\rho X^T+\epsilon_{64}\max\left(n_b,\Gamma(p)+\frac{n}{n_b}\right)\left(2|L|\lVert\widehat{X}\rVert\right)\right) \\
&\leq\frac{1}{1-\epsilon_{64}\max\left(n_b,\Gamma(p)+\frac{n}{n_b}\right)}\left(\Theta(p,n)+2n\epsilon_{64}\max\left(n_b,\Gamma(p)+\frac{n}{n_b}\right)\right)\left(\rho X^T\right).
\end{aligned} ∣ R ∣ ≤ 1 − ϵ 64 max ( n b , Γ ( p ) + n b n ) 1 ( Θ ( p , n ) ρ X T + ϵ 64 max ( n b , Γ ( p ) + n b n ) ( 2∣ L ∣ ∥ X ∥ ) ) ≤ 1 − ϵ 64 max ( n b , Γ ( p ) + n b n ) 1 ( Θ ( p , n ) + 2 n ϵ 64 max ( n b , Γ ( p ) + n b n ) ) ( ρ X T ) .
Although this proof is done only for Ozaki 1, it generalizes to a generic grade B GEMM by changing Θ \Theta Θ and Γ \Gamma Γ appropriately.
MicroScaling
MicroScaling is a strategy used to deal with the limited range of numbers in low-precision floats [28 ] . Given a tensor A A A that is stored in high-precision (say FP32), rather than rounding naively to lower-precision (say FP8), we can divide the tensor into blocks and scale each block before rounding. The scale factors can be stored with their own precision scal , which will be FP32 or E8M0 for our experiments. Throughout the paper, we assume that there is no overflow or underflow for the scale factors. We refer to the post-scaled-and-rounded elements of each block as private elements. For matrices, these blocks can be either a row/column, a sub-row/column, or a submatrix. A simple implementation of MicroScaling on a vector V V V is shown in Algorithm 2. In this algorithm, the ComputeScale function is a helper that computes and returns θ ℓ = absmax ( V ( ℓ ) ) / β \theta_\ell=\operatorname{absmax}(V^{(\ell)})/\beta θ ℓ = absmax ( V ( ℓ ) ) / β rounded to precision scal , where β \beta β can be any number between 1 and the largest finite number in the target format (to avoid overflow), and V ( ℓ ) V^{(\ell)} V ( ℓ ) is an input block. We refer to β \beta β as the "microscaling target" since it tells the algorithm what low-precision value the largest number in the block should map to. When rounding the scale factor, θ ℓ \theta_{\ell} θ ℓ , we use RoundAwayFromZero. This ensures that v i ( ℓ ) / θ ℓ v_i^{(\ell)}/\theta_{\ell} v i ( ℓ ) / θ ℓ does not overflow if β \beta β is within ( 1 + ϵ s c a l ) (1+\epsilon_{scal}) ( 1 + ϵ sc a l ) of the largest finite number of the format. Here ϵ s c a l \epsilon_{scal} ϵ sc a l denotes the machine epsilon of the type the scale factor is stored in. However, if β \beta β is a smaller number, any rounding mode should be fine.
Algorithm 2 MicroScaling (MX) Quantization of a vector V V V
Require: Vector V ∈ R n V \in\mathbb{R}^n V ∈ R n in precision h h h , Block size b m x ≤ n b_{mx} \le n b m x ≤ n , target low-precision l l l , microscaling target β \beta β
Ensure: Quantized vector V ^ \hat{V} V ^ in precision l l l and scale vector θ \theta θ in precision s c a l scal sc a l
Partition V V V into contiguous blocks V ( 1 ) , V ( 2 ) , … , V ( ⌈ n / b m x ⌉ ) V^{(1)}, V^{(2)}, \ldots, V^{(\lceil n/b_{mx}\rceil)} V ( 1 ) , V ( 2 ) , … , V (⌈ n / b m x ⌉) , where V ( ℓ ) V^{(\ell)} V ( ℓ ) contains v i v_i v i for ( ℓ − 1 ) b m x < i ≤ min ( ℓ b m x , n ) (\ell-1)b_{mx}<i\le\min(\ell b_{mx},n) ( ℓ − 1 ) b m x < i ≤ min ( ℓ b m x , n )
for ℓ = 1 , … , ⌈ n / b m x ⌉ \ell=1,\ldots,\lceil n/b_{mx}\rceil ℓ = 1 , … , ⌈ n / b m x ⌉ do
θ ℓ ← ComputeScale ( V ( ℓ ) , β ) \theta_{\ell}\leftarrow\operatorname{ComputeScale}(V^{(\ell)},\beta) θ ℓ ← ComputeScale ( V ( ℓ ) , β )
for each element v i ∈ V ( ℓ ) v_i\in V^{(\ell)} v i ∈ V ( ℓ ) do
u i ← v i / θ ℓ u_i\leftarrow v_i/\theta_{\ell} u i ← v i / θ ℓ \hfill In precision h h h
v ^ i ← RoundToLowPrecision ( u i , l ) \hat{v}_i\leftarrow\operatorname{RoundToLowPrecision}(u_i,l) v ^ i ← RoundToLowPrecision ( u i , l ) \hfill With RoundToNearestEven
end for
end for
return V ^ , θ \hat{V},\theta V ^ , θ
Lemma 3.2. Consider some vector V V V in some higher precision h h h . If we run Microscaling in Algorithm 2 with some block size b m x ≤ len ( V ) b_{mx}\le\operatorname{len}(V) b m x ≤ len ( V ) , and obtain a vector of private elements V ^ \hat{V} V ^ (in precision l l l ), and a vector of scale factors θ \theta θ (in precision s c a l scal sc a l ), then ∀ i ∈ { 1 , … , len ( V ) } \forall i\in\{1,\ldots,\operatorname{len}(V)\} ∀ i ∈ { 1 , … , len ( V )} ,
v ^ i θ ⌈ i / b m x ⌉ = v i ( 1 + δ h ) ( 1 + δ l ) + θ ⌈ i / b m x ⌉ η l , (18) \hat{v}_i\theta_{\lceil i/b_{mx}\rceil}=v_i(1+\delta_h)(1+\delta_l)+\theta_{\lceil i/b_{mx}\rceil}\eta_l, \tag*{(18)} v ^ i θ ⌈ i / b m x ⌉ = v i ( 1 + δ h ) ( 1 + δ l ) + θ ⌈ i / b m x ⌉ η l , (18) where δ l η l = 0 \delta_l\eta_l=0 δ l η l = 0 , and ∣ δ l ∣ < ϵ l |\delta_l|<\epsilon_l ∣ δ l ∣ < ϵ l , ∣ δ h ∣ < ϵ h |\delta_h|<\epsilon_h ∣ δ h ∣ < ϵ h , (and δ h = 0 \delta_h=0 δ h = 0 if the division is exact), ∣ η l ∣ < μ ϵ l |\eta_l|<\mu\epsilon_l ∣ η l ∣ < μ ϵ l , where μ = 2 e min \mu=2^{e_{\min}} μ = 2 e m i n and e min e_{\min} e m i n is the smallest exponent in the lower-precision format (alternatively, μ \mu μ is the smallest normalized number). So, if there is underflow, then ∣ v i ( 1 + δ h ) ∣ ≤ μ θ ⌈ i / b m x ⌉ |v_i(1+\delta_h)|\le\mu\theta_{\lceil i/b_{mx}\rceil} ∣ v i ( 1 + δ h ) ∣ ≤ μ θ ⌈ i / b m x ⌉ and ∣ v ^ i θ ⌈ i / b m x ⌉ − v i ∣ ≤ μ ϵ l θ ⌈ i / b m x ⌉ + ϵ h ∣ v i ∣ |\hat{v}_i\theta_{\lceil i/b_{mx}\rceil}-v_i|\le\mu\epsilon_l\theta_{\lceil i/b_{mx}\rceil}+\epsilon_h|v_i| ∣ v ^ i θ ⌈ i / b m x ⌉ − v i ∣ ≤ μ ϵ l θ ⌈ i / b m x ⌉ + ϵ h ∣ v i ∣ , else ∣ v ^ i θ ⌈ i / b m x ⌉ − v i ∣ ≤ ( ( 1 + ϵ h ) ( 1 + ϵ l ) − 1 ) ∣ v i ∣ ≈ ϵ l ∣ v i ∣ |\hat{v}_i\theta_{\lceil i/b_{mx}\rceil}-v_i|\le((1+\epsilon_h)(1+\epsilon_l)-1)|v_i|\approx\epsilon_l|v_i| ∣ v ^ i θ ⌈ i / b m x ⌉ − v i ∣ ≤ (( 1 + ϵ h ) ( 1 + ϵ l ) − 1 ) ∣ v i ∣ ≈ ϵ l ∣ v i ∣ .
Proof. Following steps 5,6 of Algorithm 2, we can observe that
v ^ i = v i θ ⌈ i / b m x ⌉ ( 1 + δ h ) ( 1 + δ l ) + η l , \hat{v}_i=\frac{v_i}{\theta_{\lceil i/b_{mx}\rceil}}(1+\delta_h)(1+\delta_l)+\eta_l, v ^ i = θ ⌈ i / b m x ⌉ v i ( 1 + δ h ) ( 1 + δ l ) + η l ,
where η l δ l = 0 \eta_l\delta_l=0 η l δ l = 0 . Multiplying by θ ⌈ i / b m x ⌉ \theta_{\lceil i/b_{mx}\rceil} θ ⌈ i / b m x ⌉ on both sides gives us the desired result. □ \square □
There are three important vectors in this context. The first is the input vector V V V , which is stored in some high precision such as FP32. The second is the low-precision V ^ \hat{V} V ^ which, along with the scale factors θ \theta θ , is able to approximate the original V V V . The element v i v_i v i is approximated by v ^ i θ ⌈ i / b m x ⌉ \hat{v}_i\theta_{\lceil i/b_{mx}\rceil} v ^ i θ ⌈ i / b m x ⌉ . We also define a third vector, V ˉ \bar{V} V ˉ with v ˉ i = v ^ i θ ⌈ i / b m x ⌉ \bar{v}_i=\hat{v}_i\theta_{\lceil i/b_{mx}\rceil} v ˉ i = v ^ i θ ⌈ i / b m x ⌉ . Thus, ∣ v ˉ i − v i ∣ |\bar{v}_i-v_i| ∣ v ˉ i − v i ∣ represents the quantization error we get when we microscale V V V . It is important to note that V ˉ \bar{V} V ˉ does not include any rounding error that we get from multiplying the scale factors with the low-precision private elements. We have decided to count this roundoff term in the dot product, since the dequantization of these MX vectors occurs during the dot product. Furthermore, we assume that the scaling of the high-precision elements in step 5 is done via a division, and so we have only ( 1 + δ h ) (1+\delta_h) ( 1 + δ h ) for our roundoff term. If scaling is implemented via a cheaper reciprocal and multiply technique, the roundoff term would be ( 1 + δ h 1 ) ( 1 + δ h 2 ) (1+\delta_{h_1})(1+\delta_{h_2}) ( 1 + δ h 1 ) ( 1 + δ h 2 ) (where ∣ δ h 1 ∣ , ∣ δ h 2 ∣ |\delta_{h_1}|,|\delta_{h_2}| ∣ δ h 1 ∣ , ∣ δ h 2 ∣ are bounded by ϵ h \epsilon_h ϵ h ). Using a power of 2 scale factor would make the scaling exact and so we would not need a δ h \delta_h δ h term at all.
The same lemma can be extended to matrices with any arbitrary MicroScaling layout. For MicroScaling in our experiments, we try 2 kinds of scalings. The first one uses one shared FP32 scale factor for a full row/col of FP8 (E4M3 1 sign bit, 4 exponent bits, 3 mantissa bits) numbers. The second is OCP MX FP8, which is a shared E8M0 scale factor for a contiguous block of 32 private E4M3 elements.
MX GEMM
We consider the problem of multiplying two MicroScaled matrices represented by A ^ \hat{A} A ^ , S A S_A S A and B ^ \hat{B} B ^ , S B S_B S B , where A ^ \hat{A} A ^ has scales S A S_A S A for contiguous sub-rows and B ^ \hat{B} B ^ has scales S B S_B S B for contiguous sub-columns. The algorithm for doing this multiplication is written in Algorithm 3. The matrices S A S_A S A , S B S_B S B are of dimension m × ⌈ k b m x ⌉ m \times\left\lceil\frac{k}{b_{\mathrm{mx}}} \right\rceil m × ⌈ b mx k ⌉ , ⌈ k b m x ⌉ × n \left\lceil\frac{k}{b_{\mathrm{mx}}} \right\rceil\times n ⌈ b mx k ⌉ × n , respectively. Both matrices A ^ , B ^ \hat{A},\hat{B} A ^ , B ^ are assumed to have the same block size b m x b_{\mathrm{mx}} b mx so that each partial dot product of length b m x b_{\mathrm{mx}} b mx uses exactly one scale factor from each S A S_A S A and S B S_B S B . Algorithm 3 describes how we can compute the higher-precision product C C C . There are other possible precision configurations (C C C can be low-precision and microscaled, A A A , B B B can have different block sizes, etc), but we limit our analysis to this case since it is the only relevant case for our mixed-precision Cholesky factorization (described later in Algorithm 5).
Algorithm 3 GEMM of two microscaled matrices A ^ , B ^ \hat{A},\hat{B} A ^ , B ^ with scale factors S A , S B S_A,S_B S A , S B , and microscaling block size b m x b_{\mathrm{mx}} b mx
Require: Matrices A ^ ∈ R m × k \hat{A} \in\mathbb{R}^{m \times k} A ^ ∈ R m × k , B ^ ∈ R k × n \hat{B} \in\mathbb{R}^{k \times n} B ^ ∈ R k × n in precision l l l , Block size b m x ≤ k b_{\mathrm{mx}} \le k b mx ≤ k , scale factors S A ∈ R m × ⌈ k b m x ⌉ S_A \in\mathbb{R}^{m \times\left\lceil\frac{k}{b_{\mathrm{mx}}} \right\rceil} S A ∈ R m × ⌈ b mx k ⌉ , S B ∈ R ⌈ k b m x ⌉ × n S_B \in\mathbb{R}^{\left\lceil\frac{k}{b_{\mathrm{mx}}} \right\rceil\times n} S B ∈ R ⌈ b mx k ⌉ × n , (high) computation precision h h h
Ensure: Output/Accumulation matrix C ∈ R m × n C \in\mathbb{R}^{m \times n} C ∈ R m × n in precision h h h
for i = 1 , … , m i = 1,\ldots,m i = 1 , … , m do
for j = 1 , … , n j = 1,\ldots,n j = 1 , … , n do
c i j ← 0 c_{ij} \leftarrow0 c ij ← 0
for ℓ = 1 , … , ⌈ k / b m x ⌉ \ell= 1,\ldots,\left\lceil k/b_{\mathrm{mx}}\right\rceil ℓ = 1 , … , ⌈ k / b mx ⌉ do
p ℓ ← 0 p_{\ell} \leftarrow0 p ℓ ← 0 Block-local accumulator, precision h h h
for k ′ = ( ℓ − 1 ) b m x + 1 , … , min ( ℓ b m x , k ) k' = (\ell-1)b_{\mathrm{mx}}+1,\ldots,\min(\ell b_{\mathrm{mx}},k) k ′ = ( ℓ − 1 ) b mx + 1 , … , min ( ℓ b mx , k ) do
p ℓ ← p ℓ + a ^ i , k ′ ⋅ b ^ k ′ , j p_{\ell} \leftarrow p_{\ell}+\hat{a}_{i,k'}\cdot\hat{b}_{k',j} p ℓ ← p ℓ + a ^ i , k ′ ⋅ b ^ k ′ , j Inputs in precision l l l , accumulation in precision h h h
end for
c i j ← c i j + ( s A ) i , ℓ ( s B ) ℓ , j p ℓ c_{ij} \leftarrow c_{ij}+(s_A)_{i,\ell}(s_B)_{\ell,j}p_{\ell} c ij ← c ij + ( s A ) i , ℓ ( s B ) ℓ , j p ℓ In precision h h h
end for
end for
end for
return C C C
To write the error bounds in matrix form, we define a broadcast matrix that is an expanded version of the matrix of scale factors. The new larger matrix has the same shape as the low-precision matrix of private elements, thus the scaling can be applied entrywise. Let N = ⌈ k / b m x ⌉ N = \left\lceil k/b_{\mathrm{mx}}\right\rceil N = ⌈ k / b mx ⌉ be the number of blocks along the contraction dimension and let K ℓ = { k ′ : ( ℓ − 1 ) b m x < k ′ ≤ min ( ℓ b m x , k ) } \mathcal{K}_{\ell} = \{k' : (\ell-1)b_{\mathrm{mx}} < k' \le\min(\ell b_{\mathrm{mx}}, k)\} K ℓ = { k ′ : ( ℓ − 1 ) b mx < k ′ ≤ min ( ℓ b mx , k )} be the contraction indices in block ℓ \ell ℓ . Then, for each k ′ ∈ K ℓ k' \in\mathcal{K}_{\ell} k ′ ∈ K ℓ , scale factors S A ∈ R m × N S_A \in\mathbb{R}^{m \times N} S A ∈ R m × N and S B ∈ R N × n S_B \in\mathbb{R}^{N \times n} S B ∈ R N × n , and matrices A ∈ R m × k A \in\mathbb{R}^{m \times k} A ∈ R m × k and B ∈ R k × n B \in\mathbb{R}^{k \times n} B ∈ R k × n , define the broadcasts S ˉ A ∈ R m × k \bar{S}_A \in\mathbb{R}^{m \times k} S ˉ A ∈ R m × k and S ˉ B ∈ R k × n \bar{S}_B \in\mathbb{R}^{k \times n} S ˉ B ∈ R k × n by
( s ˉ A ) i k ′ = ( s A ) i , ⌈ k ′ / b m x ⌉ , ( s ˉ B ) k ′ j = ( s B ) ⌈ k ′ / b m x ⌉ , j . (19) (\bar{s}_A)_{ik'} = (s_A)_{i,\lceil k'/b_{\mathrm{mx}}\rceil}, \qquad(\bar{s}_B)_{k'j} = (s_B)_{\lceil k'/b_{\mathrm{mx}}\rceil,j}. \tag*{(19)} ( s ˉ A ) i k ′ = ( s A ) i , ⌈ k ′ / b mx ⌉ , ( s ˉ B ) k ′ j = ( s B ) ⌈ k ′ / b mx ⌉ , j . (19)
Thus, each replicates its scale factors across the b m x b_{\mathrm{mx}} b mx entries of its block, matching the shapes of A A A and B B B , respectively. The application of scale factors is then just a Hadamard product: S ˉ A ⊙ A \bar{S}_A \odot A S ˉ A ⊙ A scales A A A by rows and B ⊙ S ˉ B B \odot\bar{S}_B B ⊙ S ˉ B scales B B B by columns. Since Hadamard products are commutative, the order of S ˉ A \bar{S}_A S ˉ A and S ˉ B \bar{S}_B S ˉ B has no effect here; we nevertheless place S ˉ A \bar{S}_A S ˉ A on the left and S ˉ B \bar{S}_B S ˉ B on the right for notational consistency with scaling by diagonal matrices. Now, let D A ∈ R m × m D_A \in\mathbb{R}^{m \times m} D A ∈ R m × m and D B ∈ R n × n D_B \in\mathbb{R}^{n \times n} D B ∈ R n × n be the diagonal matrices that hold the largest scale factor of each row of A A A and each column of B B B ,
( d A ) i i = max 1 ≤ ℓ ≤ N ( s A ) i ℓ , ( d B ) j j = max 1 ≤ ℓ ≤ N ( s B ) ℓ j . (20) (d_A)_{ii} = \max_{1 \le\ell\le N}(s_A)_{i\ell}, \qquad(d_B)_{jj} = \max_{1 \le\ell\le N}(s_B)_{\ell j}. \tag*{(20)} ( d A ) ii = 1 ≤ ℓ ≤ N max ( s A ) i ℓ , ( d B ) j j = 1 ≤ ℓ ≤ N max ( s B ) ℓ j . (20)
Since ( s A ) i , ⌈ k ′ / b m x ⌉ ≤ ( d A ) i i (s_A)_{i,\lceil k'/b_{\mathrm{mx}}\rceil} \le(d_A)_{ii} ( s A ) i , ⌈ k ′ / b mx ⌉ ≤ ( d A ) ii and ( s B ) ⌈ k ′ / b m x ⌉ , j ≤ ( d B ) j j (s_B)_{\lceil k'/b_{\mathrm{mx}}\rceil,j} \le(d_B)_{jj} ( s B ) ⌈ k ′ / b mx ⌉ , j ≤ ( d B ) j j for every k ′ k' k ′ , this gives
∣ S ˉ A ⊙ A ∣ ≤ D A ∣ A ∣ , ∣ B ⊙ S ˉ B ∣ ≤ ∣ B ∣ D B , (21) |\bar{S}_A \odot A| \le D_A|A|,\qquad|B \odot\bar{S}_B| \le|B|D_B, \tag*{(21)} ∣ S ˉ A ⊙ A ∣ ≤ D A ∣ A ∣ , ∣ B ⊙ S ˉ B ∣ ≤ ∣ B ∣ D B , (21)
with D B D_B D B on the right since B B B is scaled by columns. We can now state the matrix version of Lemma 3.2.
Lemma 3.3. Let A A A be microscaled along its rows and B B B along its columns, giving A ^ , S A \hat{A},S_A A ^ , S A and B ^ , S B \hat{B},S_B B ^ , S B , and write A ˉ = S ˉ A ⊙ A ^ \bar{A}=\bar{S}_A\odot\hat{A} A ˉ = S ˉ A ⊙ A ^ and B ˉ = B ^ ⊙ S ˉ B \bar{B}=\hat{B}\odot\bar{S}_B B ˉ = B ^ ⊙ S ˉ B for the exactly dequantized A ^ , S A \hat{A},S_A A ^ , S A and B ^ , S B \hat{B},S_B B ^ , S B (these are defined analogously to V ˉ \bar{V} V ˉ , and don’t include the rounding error we get from multiplying the scale factor with the private element). Define A ′ A' A ′ as a i k ′ ′ = a i k ′ a'_{ik'}=a_{ik'} a i k ′ ′ = a i k ′ when a ^ i k ′ \hat{a}_{ik'} a ^ i k ′ doesn’t underflow and 0 0 0 otherwise, and ditto for B ′ B' B ′ . Let M A , M B M_A,M_B M A , M B have complementary sparsity patterns to A ′ , B ′ A',B' A ′ , B ′ with every nonzero entry equal to μ \mu μ (smallest positive normal number in precision l l l ), and set A ′ ′ = ∣ A ′ ∣ + D A M A A''=|A'|+D_AM_A A ′′ = ∣ A ′ ∣ + D A M A , B ′ ′ = ∣ B ′ ∣ + M B D B B''=|B'|+M_BD_B B ′′ = ∣ B ′ ∣ + M B D B . Since all entries are positive, ∣ A ′ ′ ∣ = A ′ ′ |A''|=A'' ∣ A ′′ ∣ = A ′′ and ∣ B ′ ′ ∣ = B ′ ′ |B''|=B'' ∣ B ′′ ∣ = B ′′ . Then, with ϕ = ( 1 + ϵ h ) ( 1 + ϵ l ) − 1 = ϵ l + ϵ h + ϵ h ϵ l \phi=(1+\epsilon_h)(1+\epsilon_l)-1=\epsilon_l+\epsilon_h+\epsilon_h\epsilon_l ϕ = ( 1 + ϵ h ) ( 1 + ϵ l ) − 1 = ϵ l + ϵ h + ϵ h ϵ l ,
∣ A ˉ − A ∣ ≤ ϕ ∣ A ′ ′ ∣ , ∣ B ˉ − B ∣ ≤ ϕ ∣ B ′ ′ ∣ . (22) |\bar{A}-A|\le\phi|A''|,\qquad|\bar{B}-B|\le\phi|B''|. \tag*{(22)} ∣ A ˉ − A ∣ ≤ ϕ ∣ A ′′ ∣ , ∣ B ˉ − B ∣ ≤ ϕ ∣ B ′′ ∣. (22)
Proof. Given row i i i , entry ( i , k ′ ) (i,k') ( i , k ′ ) lies in block ⌈ k ′ / b m x ⌉ \lceil k'/b_{\mathrm{mx}}\rceil ⌈ k ′ / b mx ⌉ and carries the scale factor ( s ˉ A ) i k ′ (\bar{s}_A)_{ik'} ( s ˉ A ) i k ′ , the broadcasted entry from (19), so Lemma 3.2 gives a ˉ i k ′ = a i k ′ ( 1 + δ h ) ( 1 + δ l ) + ( s ˉ A ) i k ′ η l \bar{a}_{ik'}=a_{ik'}(1+\delta_h)(1+\delta_l)+(\bar{s}_A)_{ik'}\eta_l a ˉ i k ′ = a i k ′ ( 1 + δ h ) ( 1 + δ l ) + ( s ˉ A ) i k ′ η l with δ l η l = 0 \delta_l\eta_l=0 δ l η l = 0 . Now, if a ^ i k ′ \hat{a}_{ik'} a ^ i k ′ did not underflow, then η l = 0 \eta_l=0 η l = 0 and ∣ a ˉ i k ′ − a i k ′ ∣ ≤ ( ( 1 + ϵ h ) ( 1 + ϵ l ) − 1 ) ∣ a i k ′ ∣ = ϕ ∣ a i k ′ ∣ |\bar{a}_{ik'}-a_{ik'}|\le((1+\epsilon_h)(1+\epsilon_l)-1)|a_{ik'}|=\phi|a_{ik'}| ∣ a ˉ i k ′ − a i k ′ ∣ ≤ (( 1 + ϵ h ) ( 1 + ϵ l ) − 1 ) ∣ a i k ′ ∣ = ϕ ∣ a i k ′ ∣ . If it did underflow, then δ l = 0 \delta_l=0 δ l = 0 , and ∣ a i k ′ ∣ < ( 1 + ϵ h ) μ ( s ˉ A ) i k ′ |a_{ik'}|<(1+\epsilon_h)\mu(\bar{s}_A)_{ik'} ∣ a i k ′ ∣ < ( 1 + ϵ h ) μ ( s ˉ A ) i k ′ . Using this upper bound on ∣ a i k ′ ∣ |a_{ik'}| ∣ a i k ′ ∣ , we can write a ˉ i k ′ − a i k ′ = δ h a i k ′ + ( s ˉ A ) i k ′ η l ⟹ ∣ a ˉ i k ′ − a i k ′ ∣ ≤ ϵ h ( 1 + ϵ h ) μ ( s ˉ A ) i k ′ + ϵ l μ ( s ˉ A ) i k ′ ≤ ϵ h ϵ l μ ( s ˉ A ) i k ′ + ϵ h μ ( s ˉ A ) i k ′ + ϵ l μ ( s ˉ A ) i k ′ = ϕ μ ( s ˉ A ) i k ′ \bar{a}_{ik'}-a_{ik'}=\delta_ha_{ik'}+(\bar{s}_A)_{ik'}\eta_l\Longrightarrow|\bar{a}_{ik'}-a_{ik'}|\le\epsilon_h(1+\epsilon_h)\mu(\bar{s}_A)_{ik'}+\epsilon_l\mu(\bar{s}_A)_{ik'}\le\epsilon_h\epsilon_l\mu(\bar{s}_A)_{ik'}+\epsilon_h\mu(\bar{s}_A)_{ik'}+\epsilon_l\mu(\bar{s}_A)_{ik'}=\phi\mu(\bar{s}_A)_{ik'} a ˉ i k ′ − a i k ′ = δ h a i k ′ + ( s ˉ A ) i k ′ η l ⟹ ∣ a ˉ i k ′ − a i k ′ ∣ ≤ ϵ h ( 1 + ϵ h ) μ ( s ˉ A ) i k ′ + ϵ l μ ( s ˉ A ) i k ′ ≤ ϵ h ϵ l μ ( s ˉ A ) i k ′ + ϵ h μ ( s ˉ A ) i k ′ + ϵ l μ ( s ˉ A ) i k ′ = ϕ μ ( s ˉ A ) i k ′ , where we used ϵ h 2 ≤ ϵ h ϵ l \epsilon_h^2\le\epsilon_h\epsilon_l ϵ h 2 ≤ ϵ h ϵ l for the last bound. The two cases have disjoint supports, so by (21),
∣ A ˉ − A ∣ ≤ ϕ ∣ A ′ ∣ + ϕ ( S ˉ A ⊙ M A ) ≤ ϕ ( ∣ A ′ ∣ + D A M A ) = ϕ ∣ A ′ ′ ∣ . |\bar{A}-A|\le\phi|A'|+\phi(\bar{S}_A\odot M_A)\le\phi(|A'|+D_AM_A)=\phi|A''|. ∣ A ˉ − A ∣ ≤ ϕ ∣ A ′ ∣ + ϕ ( S ˉ A ⊙ M A ) ≤ ϕ ( ∣ A ′ ∣ + D A M A ) = ϕ ∣ A ′′ ∣.
The argument for B B B is identical, with D B D_B D B multiplying from the right. □
Now, we can state the error bound that we obtain when multiplying two microscaled matrices using Algorithm 3.
Lemma 3.4. With the notation of Lemma 3.3, the output C ^ \hat{C} C ^ of Algorithm 3 satisfies
∣ C ^ − A B ∣ ≤ ϕ ( ∣ A ′ ′ ∣ ∣ B ∣ + ∣ A ∣ ∣ B ′ ′ ∣ + ϕ ∣ A ′ ′ ∣ ∣ B ′ ′ ∣ ) + ( k + 2 ) ϵ h ( ∣ A ∣ + ϕ ∣ A ′ ′ ∣ ) ( ∣ B ∣ + ϕ ∣ B ′ ′ ∣ ) . (23) |\hat{C}-AB|\le\phi\left(|A''||B|+|A||B''|+\phi|A''||B''|\right)+(k+2)\epsilon_h\left(|A|+\phi|A''|\right)\left(|B|+\phi|B''|\right). \tag*{(23)} ∣ C ^ − A B ∣ ≤ ϕ ( ∣ A ′′ ∣∣ B ∣ + ∣ A ∣∣ B ′′ ∣ + ϕ ∣ A ′′ ∣∣ B ′′ ∣ ) + ( k + 2 ) ϵ h ( ∣ A ∣ + ϕ ∣ A ′′ ∣ ) ( ∣ B ∣ + ϕ ∣ B ′′ ∣ ) . (23)
Proof. Algorithm 3 applies scale factors from S A , S B S_A,S_B S A , S B to the length b m x b_{\mathrm{mx}} b mx dot products of blocks of A ^ , B ^ \hat{A},\hat{B} A ^ , B ^ . These dot products have input precision l l l and accumulation and output precision h h h . The exact expression for the ( i , j ) (i,j) ( i , j ) entry of C C C is
c i j = ∑ ℓ = 1 N ( s A ) i ℓ ( s B ) ℓ j ∑ k ′ ∈ K ℓ a ^ i k ′ b ^ k ′ j = ∑ ℓ = 1 N ∑ k ′ ∈ K ℓ ( s A ) i ℓ a ^ i k ′ ( s B ) ℓ j b ^ k ′ j . (24) c_{ij}=\sum_{\ell=1}^{N}(s_A)_{i\ell}(s_B)_{\ell j}\sum_{k'\in K_\ell}\hat{a}_{ik'}\hat{b}_{k'j}=\sum_{\ell=1}^{N}\sum_{k'\in K_\ell}(s_A)_{i\ell}\hat{a}_{ik'}(s_B)_{\ell j}\hat{b}_{k'j}. \tag*{(24)} c ij = ℓ = 1 ∑ N ( s A ) i ℓ ( s B ) ℓ j k ′ ∈ K ℓ ∑ a ^ i k ′ b ^ k ′ j = ℓ = 1 ∑ N k ′ ∈ K ℓ ∑ ( s A ) i ℓ a ^ i k ′ ( s B ) ℓ j b ^ k ′ j . (24)
We can turn the nested sum into just one sum by applying each of the scale factors to their corresponding private elements,
c i j = ∑ ℓ = 1 N ∑ k ′ ∈ K ℓ a ˉ i k ′ b ˉ k ′ j = ∑ k ′ = 1 k a ˉ i k ′ b ˉ k ′ j . c_{ij}=\sum_{\ell=1}^{N}\sum_{k'\in K_\ell}\bar{a}_{ik'}\bar{b}_{k'j}=\sum_{k'=1}^{k}\bar{a}_{ik'}\bar{b}_{k'j}. c ij = ℓ = 1 ∑ N k ′ ∈ K ℓ ∑ a ˉ i k ′ b ˉ k ′ j = k ′ = 1 ∑ k a ˉ i k ′ b ˉ k ′ j .
Counting the rounding error terms carried by the summand a ˉ i k ′ b ˉ k ′ j \bar{a}_{ik'}\bar{b}_{k'j} a ˉ i k ′ b ˉ k ′ j , we get two from applying scale factors, at most b m x − 1 b_{\mathrm{mx}}-1 b mx − 1 from the FMAs forming the block dot product in line 7, and at most N − 1 N-1 N − 1 from accumulating the N N N block results in line 9. Since b m x + N − 1 ≤ k b_{\mathrm{mx}}+N-1\le k b mx + N − 1 ≤ k for every 1 ≤ b m x ≤ k 1\le b_{\mathrm{mx}}\le k 1 ≤ b mx ≤ k , this is at most k + 2 k+2 k + 2 in total, so
c ^ i j = ∑ k ′ = 1 k a ˉ i k ′ b ˉ k ′ j ( 1 + ζ k ′ ) , ∣ ζ k ′ ∣ ≤ ( k + 2 ) ϵ h . (25) \hat{c}_{ij}=\sum_{k'=1}^{k}\bar{a}_{ik'}\bar{b}_{k'j}(1+\zeta_{k'}),\qquad|\zeta_{k'}|\le(k+2)\epsilon_h. \tag*{(25)} c ^ ij = k ′ = 1 ∑ k a ˉ i k ′ b ˉ k ′ j ( 1 + ζ k ′ ) , ∣ ζ k ′ ∣ ≤ ( k + 2 ) ϵ h . (25)
Hence, ∣ c ^ i j − ( A ˉ B ˉ ) i j ∣ ≤ ( k + 2 ) ϵ h ∑ k ′ = 1 k ∣ a ˉ i k ′ ∣ ∣ b ˉ k ′ j ∣ \lvert\hat{c}_{ij}-(\bar{A}\bar{B})_{ij}\rvert\le(k+2)\epsilon_h\sum_{k'=1}^{k}\lvert\bar{a}_{ik'}\rvert\lvert\bar{b}_{k'j}\rvert ∣ c ^ ij − ( A ˉ B ˉ ) ij ∣ ≤ ( k + 2 ) ϵ h ∑ k ′ = 1 k ∣ a ˉ i k ′ ∣ ∣ b ˉ k ′ j ∣ , which in matrix form reads
∣ C ^ − A ˉ B ˉ ∣ ≤ ( k + 2 ) ϵ h ∣ A ˉ ∣ ∣ B ˉ ∣ . (26) \lvert\hat{C}-\bar{A}\bar{B}\rvert\le(k+2)\epsilon_h\lvert\bar{A}\rvert\lvert\bar{B}\rvert. \tag*{(26)} ∣ C ^ − A ˉ B ˉ ∣ ≤ ( k + 2 ) ϵ h ∣ A ˉ ∣ ∣ B ˉ ∣ . (26)
Finally, write E A = A ˉ − A E_A=\bar{A}-A E A = A ˉ − A and E B = B ˉ − B E_B=\bar{B}-B E B = B ˉ − B , so ∣ E A ∣ ≤ ϕ ∣ A ′ ′ ∣ \lvert E_A\rvert\le\phi\lvert A''\rvert ∣ E A ∣ ≤ ϕ ∣ A ′′ ∣ and ∣ E B ∣ ≤ ϕ ∣ B ′ ′ ∣ \lvert E_B\rvert\le\phi\lvert B''\rvert ∣ E B ∣ ≤ ϕ ∣ B ′′ ∣ by (22) , and hence ∣ A ˉ ∣ ≤ ∣ A ∣ + ϕ ∣ A ′ ′ ∣ \lvert\bar{A}\rvert\le\lvert A\rvert+\phi\lvert A''\rvert ∣ A ˉ ∣ ≤ ∣ A ∣ + ϕ ∣ A ′′ ∣ and ∣ B ˉ ∣ ≤ ∣ B ∣ + ϕ ∣ B ′ ′ ∣ \lvert\bar{B}\rvert\le\lvert B\rvert+\phi\lvert B''\rvert ∣ B ˉ ∣ ≤ ∣ B ∣ + ϕ ∣ B ′′ ∣ . Since A ˉ B ˉ − A B = E A B + A E B + E A E B \bar{A}\bar{B}-AB=E_AB+AE_B+E_AE_B A ˉ B ˉ − A B = E A B + A E B + E A E B , we get ∣ A ˉ B ˉ − A B ∣ ≤ ϕ ( ∣ A ′ ′ ∣ ∣ B ∣ + ∣ A ∣ ∣ B ′ ′ ∣ + ϕ ∣ A ′ ′ ∣ ∣ B ′ ′ ∣ ) \lvert\bar{A}\bar{B}-AB\rvert\le\phi\left(\lvert A''\rvert\lvert B\rvert+\lvert A\rvert\lvert B''\rvert+\phi\lvert A''\rvert\lvert B''\rvert\right) ∣ A ˉ B ˉ − A B ∣ ≤ ϕ ( ∣ A ′′ ∣ ∣ B ∣ + ∣ A ∣ ∣ B ′′ ∣ + ϕ ∣ A ′′ ∣ ∣ B ′′ ∣ ) . Adding this to (26) gives (23) . If we have power-of-two scales, the k + 2 k+2 k + 2 factor can also be reduced to k k k since the scaling is exact. □
We also consider an alternative way to multiply microscaled matrices. This alternate method is depicted in Algorithm 4, which is clearly more expensive than Algorithm 3 since it does not use mixed-precision FMAs with low-precision inputs and high-precision accumulation. Instead, everything is done in high precision. Algorithm 4 is useful in cases where low-precision hardware is not available, since it simulates Algorithm 3 while obeying the same error bound as Lemma 3.4.
Figure 4. Simulated GEMM of two microscaled matrices A ^ , B ^ \hat{A},\hat{B} A ^ , B ^ with scale factors S A , S B S_A,S_B S A , S B , and microscaling block size b m x b_{\mathrm{mx}} b mx .
Require: Matrices A ^ ∈ R m × k \hat{A}\in\mathbb{R}^{m\times k} A ^ ∈ R m × k , B ^ ∈ R k × n \hat{B}\in\mathbb{R}^{k\times n} B ^ ∈ R k × n in precision l l l , Block size b m x ≤ k b_{\mathrm{mx}}\le k b mx ≤ k , scale factors S A ∈ R m × ⌈ k / b m x ⌉ S_A\in\mathbb{R}^{m\times\lceil k/b_{\mathrm{mx}}\rceil} S A ∈ R m × ⌈ k / b mx ⌉ , S B ∈ R ⌈ k / b m x ⌉ × n S_B\in\mathbb{R}^{\lceil k/b_{\mathrm{mx}}\rceil\times n} S B ∈ R ⌈ k / b mx ⌉ × n , (high) computation precision h h h
Ensure: Output matrix C ∈ R m × n C\in\mathbb{R}^{m\times n} C ∈ R m × n in precision h h h
1: for i = 1 , … , m i=1,\ldots,m i = 1 , … , m do 2: for k ′ = 1 , … , k k'=1,\ldots,k k ′ = 1 , … , k do 3: a ˉ i , k ′ ← ( S A ) i , ⌈ k ′ / b m x ⌉ ⋅ a ^ i , k ′ \bar{a}_{i,k'}\leftarrow(S_A)_{i,\lceil k'/b_{\mathrm{mx}}\rceil}\cdot\hat{a}_{i,k'} a ˉ i , k ′ ← ( S A ) i , ⌈ k ′ / b mx ⌉ ⋅ a ^ i , k ′ ▷ Dequantize A ^ \hat{A} A ^ ; one rounding per entry, precision h h h 4: end for 5: end for 6: for k ′ = 1 , … , k k'=1,\ldots,k k ′ = 1 , … , k do 7: for j = 1 , … , n j=1,\ldots,n j = 1 , … , n do 8: b ˉ k ′ , j ← ( S B ) ⌈ k ′ / b m x ⌉ , j ⋅ b ^ k ′ , j \bar{b}_{k',j}\leftarrow(S_B)_{\lceil k'/b_{\mathrm{mx}}\rceil,j}\cdot\hat{b}_{k',j} b ˉ k ′ , j ← ( S B ) ⌈ k ′ / b mx ⌉ , j ⋅ b ^ k ′ , j ▷ Dequantize B ^ \hat{B} B ^ ; one rounding per entry, precision h h h 9: end for 10: end for 11: for i = 1 , … , m i=1,\ldots,m i = 1 , … , m do 12: for j = 1 , … , n j=1,\ldots,n j = 1 , … , n do 13: c i j ← 0 c_{ij}\leftarrow0 c ij ← 0 14: for k ′ = 1 , … , k k'=1,\ldots,k k ′ = 1 , … , k do 15: c i j ← c i j + a ˉ i , k ′ ⋅ b ˉ k ′ , j c_{ij}\leftarrow c_{ij}+\bar{a}_{i,k'}\cdot\bar{b}_{k',j} c ij ← c ij + a ˉ i , k ′ ⋅ b ˉ k ′ , j ▷ One length-k k k dot product, entirely in precision h h h 16: end for 17: end for 18: end for 19: return C C C
Lemma 3.5. With the notation of Lemma 3.3, the output C ^ \hat{C} C ^ of Algorithm 4 satisfies (23) as well.
Proof. The two dequantization loops of Algorithm 4 put one rounding error on each entry of A ˉ \bar{A} A ˉ and one on each entry of B ˉ \bar{B} B ˉ . We then do a regular GEMM from lines 11 to 18, which adds k k k roundoff terms [14 ] . This gives k + 2 k+2 k + 2 total roundoff terms, each of which is in high precision and is bounded by ϵ h \epsilon_h ϵ h . Thus, (25) holds, and the rest of the proof is similar to the proof of Lemma 3.4. □
Algorithm 5 Cholesky with statically scheduled emulated GEMM and TRSM
Input: PSD Matrix A A A in precision h h h , block size n b n_b n b , and for 1 ≤ I ≤ n / n b 1 \le I \le n/n_b 1 ≤ I ≤ n / n b :
π g ( I ) \pi_g(I) π g ( I ) : GEMM slice count (emulation) or low-precision format (floating point)
π t ( I ) \pi_t(I) π t ( I ) : TRSM slice count (emulation only)
Output: Lower triangular matrix L ^ \hat{L} L ^ such that A ≈ L ^ L ^ T A \approx\hat{L}\hat{L}^{T} A ≈ L ^ L ^ T
for I = 1 → n / n b I = 1 \to n/n_b I = 1 → n / n b do
L ^ I I ← Potrf ( A I I ) \hat{L}_{II} \leftarrow\operatorname{Potrf}(A_{II}) L ^ I I ← Potrf ( A I I ) ▹ A I I = L ^ I I L ^ I I T \triangleright A_{II} = \hat{L}_{II}\hat{L}_{II}^{T} ▹ A I I = L ^ I I L ^ I I T , in precision h h h
for J = I + 1 → n / n b J = I + 1 \to n/n_b J = I + 1 → n / n b do
L ^ J I ← Trsm ( A J I , L ^ I I ) \hat{L}_{JI} \leftarrow\operatorname{Trsm}(A_{JI},\hat{L}_{II}) L ^ J I ← Trsm ( A J I , L ^ I I ) ▹ L ^ J I L ^ I I T = A J I \triangleright\hat{L}_{JI}\hat{L}_{II}^{T} = A_{JI} ▹ L ^ J I L ^ I I T = A J I ; π t ( I ) \pi_t(I) π t ( I ) slices, or precision h h h
end for
for J = I + 1 → n / n b J = I + 1 \to n/n_b J = I + 1 → n / n b do
for K = I + 1 → J K = I + 1 \to J K = I + 1 → J do
A J K ← A J K − Gemm ( L ^ J I , L ^ K I T ) A_{JK} \leftarrow A_{JK} - \operatorname{Gemm}(\hat{L}_{JI},\hat{L}_{KI}^{T}) A J K ← A J K − Gemm ( L ^ J I , L ^ K I T ) ▹ \triangleright ▹ format or slices π g ( I ) \pi_g(I) π g ( I )
end for
end for
end for
Theorem 3.6. With the notation of Lemma 3.3, Algorithms 3 and 4 satisfy the same bound (23) .
Proof. Direct application of Lemmas 3.4 and 3.5. □
For our experiments, ϵ l \epsilon_l ϵ l will be 0.125, and ϵ h \epsilon_h ϵ h is 2 − 23 2^{-23} 2 − 23 , so ϵ h ≪ ϵ l \epsilon_h \ll\epsilon_l ϵ h ≪ ϵ l , and we can simplify ϕ = ( 1 + ϵ h ) ( 1 + ϵ l ) − 1 ≈ ϵ l \phi= (1 + \epsilon_h)(1 + \epsilon_l) - 1 \approx\epsilon_l ϕ = ( 1 + ϵ h ) ( 1 + ϵ l ) − 1 ≈ ϵ l , and relax (23) to
∣ C ^ − A B ∣ ≲ ϵ l ( ∣ A ′ ′ ∣ ∣ B ∣ + ∣ A ∣ ∣ B ′ ′ ∣ + ϵ l ∣ A ′ ′ ∣ ∣ B ′ ′ ∣ ) + ( k + 2 ) ϵ h ∣ A ∣ ∣ B ∣ . (27) |\hat{C} - AB| \lesssim\epsilon_l (|A^{\prime\prime}||B| + |A||B^{\prime\prime}| + \epsilon_l|A^{\prime\prime}||B^{\prime\prime}|) + (k + 2)\epsilon_h|A||B|. \tag*{(27)} ∣ C ^ − A B ∣ ≲ ϵ l ( ∣ A ′′ ∣∣ B ∣ + ∣ A ∣∣ B ′′ ∣ + ϵ l ∣ A ′′ ∣∣ B ′′ ∣ ) + ( k + 2 ) ϵ h ∣ A ∣∣ B ∣. (27)
Note that if we don’t do any microscaling, then N = 1 N = 1 N = 1 with all scale factors equal to 1, and each D Z D_Z D Z matrix is the identity. If we use global scaling factors θ A \theta_A θ A , θ B \theta_B θ B , then D A = θ A I D_A = \theta_A I D A = θ A I and D B = θ B I D_B = \theta_B I D B = θ B I . Additionally, if either of the matrices is scaled such that we fully avoid underflow when rounding to low precision, the corresponding M Z M_Z M Z will be 0 and ∣ Z ′ ′ ∣ = ∣ Z ′ ∣ = ∣ Z ∣ |Z^{\prime\prime}| = |Z^\prime| = |Z| ∣ Z ′′ ∣ = ∣ Z ′ ∣ = ∣ Z ∣ .
Mixed-Precision Cholesky
To motivate the adaptive algorithm, we first consider the “regular” mixed-precision algorithm described in Algorithm 5. The algorithm can either use an emulated GEMM/SYRK, or a mixed-precision floating-point GEMM/SYRK (where inputs are assumed to be low-precision and accumulation and output are assumed to be high-precision) for the Schur update in line 8. We assume that the algorithm receives a list of precisions/slices to use at each iteration, and the slice count/precision in iteration I I I is given by π g ( I ) \pi_g(I) π g ( I ) for GEMM and π t ( I ) \pi_t(I) π t ( I ) for TRSM. The TRSM is implemented such that the off-diagonal GEMMs in the TRSM are done with emulation (as in Algorithm 1).
Emulation Case
Theorem 4.1. If Algorithm 5 completes successfully with the π g ( I ) \pi_g(I) π g ( I ) slices used for the Ozaki 1 GEMMs and the π t ( I ) \pi_t(I) π t ( I ) slices used for the Ozaki 1 TRSM in iteration I I I , assuming that there is no overflow/underflow, provided n b ϵ 64 < 1 n_b\epsilon_{64} < 1 n b ϵ 64 < 1 and γ e m u ( ψ , Ψ , n , n b ) < 1 \gamma_{\mathrm{emu}}(\psi,\Psi,n,n_b) < 1 γ emu ( ψ , Ψ , n , n b ) < 1 , we obtain A + E = L ^ L ^ T A + E = \hat{L}\hat{L}^{T} A + E = L ^ L ^ T with
∣ e s t ∣ ≤ γ e m u ( ψ , Ψ , n , n b ) 1 − γ e m u ( ψ , Ψ , n , n b ) a s s a t t , (28) |e_{\mathrm{st}}| \le\frac{\gamma_{\mathrm{emu}}(\psi,\Psi,n,n_b)}{1-\gamma_{\mathrm{emu}}(\psi,\Psi,n,n_b)} \sqrt{a_{\mathrm{ss}}a_{\mathrm{tt}}}, \tag*{(28)} ∣ e st ∣ ≤ 1 − γ emu ( ψ , Ψ , n , n b ) γ emu ( ψ , Ψ , n , n b ) a ss a tt , (28) where ψ = min I min ( π g ( I ) , π t ( I ) ) \psi= \min_I \min(\pi_g(I),\pi_t(I)) ψ = min I min ( π g ( I ) , π t ( I )) and Ψ = max I max ( π g ( I ) , π t ( I ) ) \Psi= \max_I \max(\pi_g(I),\pi_t(I)) Ψ = max I max ( π g ( I ) , π t ( I )) for 1 ≤ I ≤ n / n b 1 \le I \le n/n_b 1 ≤ I ≤ n / n b , γ e m u ( ψ , Ψ , n , n b ) = Θ ( ψ , n b ) + 4 M ϵ 64 \gamma_{\mathrm{emu}}(\psi,\Psi,n,n_b) = \Theta(\psi,n_b) + 4M\epsilon_{64} γ emu ( ψ , Ψ , n , n b ) = Θ ( ψ , n b ) + 4 M ϵ 64 with M = M ( Ψ , n b ) = Γ ( Ψ ) + max ( n b , n n b ) 1 − n b ϵ 64 M = M(\Psi,n_b) = \frac{\Gamma(\Psi)+\max(n_b,\frac{n}{n_b})}{1-n_b\epsilon_{64}} M = M ( Ψ , n b ) = 1 − n b ϵ 64 Γ ( Ψ ) + m a x ( n b , n b n ) , Θ ( ψ , n b ) = 4 ( ψ + 1 2 ) n b 2 − 8 ψ + 2 \Theta(\psi,n_b) = 4\left(\psi+\frac{1}{2}\right)n_b2^{-8\psi+2} Θ ( ψ , n b ) = 4 ( ψ + 2 1 ) n b 2 − 8 ψ + 2 , and Γ ( Ψ ) = 1 2 Ψ ( Ψ + 1 ) − 1 \Gamma(\Psi) = \frac{1}{2}\Psi(\Psi+1)-1 Γ ( Ψ ) = 2 1 Ψ ( Ψ + 1 ) − 1 .
Proof. For simplicity, assume n b ∣ n n_b \mid n n b ∣ n . First consider diagonal blocks of A A A . The blocks A I I A_{II} A I I are updated with Ozaki 1 GEMMs/SYRKs, and then we perform an FP64 Cholesky on the updated submatrix A ^ I I u \hat{A}_{II}^{u} A ^ I I u . From [8 ] , we have
∣ A ^ I I u − L ^ I I L ^ I I T ∣ ≤ n b ϵ 64 Ω I I , (29) \left|\hat{A}_{II}^{u}-\hat{L}_{II}\hat{L}_{II}^{T}\right|\leq n_b\epsilon_{64}\Omega_{II}, \tag*{(29)} A ^ I I u − L ^ I I L ^ I I T ≤ n b ϵ 64 Ω I I , (29)
where ω s t = ∣ a ^ s s u a ^ t t u ∣ \omega_{st}=\sqrt{|\hat{a}_{ss}^{u}\hat{a}_{tt}^{u}|} ω s t = ∣ a ^ ss u a ^ tt u ∣ . Next, by triangle inequality,
∣ A I I − ∑ K = 1 I L ^ I K L ^ I K T ∣ = ∣ ( A I I − ∑ K = 1 I − 1 L ^ I K L ^ I K T − A ^ I I u ) + ( A ^ I I u − L ^ I I L ^ I I T ) ∣ ≤ ∣ A I I − ∑ K = 1 I − 1 L ^ I K L ^ I K T − A ^ I I u ∣ + ∣ A ^ I I u − L ^ I I L ^ I I T ∣ . \left|A_{II}-\sum_{K=1}^{I}\hat{L}_{IK}\hat{L}_{IK}^{T}\right|=\left|\left(A_{II}-\sum_{K=1}^{I-1}\hat{L}_{IK}\hat{L}_{IK}^{T}-\hat{A}_{II}^{u}\right)+\left(\hat{A}_{II}^{u}-\hat{L}_{II}\hat{L}_{II}^{T}\right)\right|\leq\left|A_{II}-\sum_{K=1}^{I-1}\hat{L}_{IK}\hat{L}_{IK}^{T}-\hat{A}_{II}^{u}\right|+\left|\hat{A}_{II}^{u}-\hat{L}_{II}\hat{L}_{II}^{T}\right|. A I I − K = 1 ∑ I L ^ I K L ^ I K T = ( A I I − K = 1 ∑ I − 1 L ^ I K L ^ I K T − A ^ I I u ) + ( A ^ I I u − L ^ I I L ^ I I T ) ≤ A I I − K = 1 ∑ I − 1 L ^ I K L ^ I K T − A ^ I I u + A ^ I I u − L ^ I I L ^ I I T .
The first term on the right hand side can be bounded by the Ozaki 1 error bound (10), while accounting for A I I A_{II} A I I in the accumulation roundoff (similar to the proof of Theorem 3.1). Also note that for p ∈ N p\in\mathbb{N} p ∈ N , Θ ( p , n ) \Theta(p,n) Θ ( p , n ) decreases as p p p increases and Γ ( p ) \Gamma(p) Γ ( p ) increases with p p p . Thus Θ ( ψ , n b ) ≥ Θ ( π g ( I ) , n b ) \Theta(\psi,n_b)\geq\Theta(\pi_g(I),n_b) Θ ( ψ , n b ) ≥ Θ ( π g ( I ) , n b ) for every 1 ≤ I ≤ n / n b 1\leq I\leq n/n_b 1 ≤ I ≤ n / n b , and Γ ( Ψ ) ≥ Γ ( π g ( I ) ) \Gamma(\Psi)\geq\Gamma(\pi_g(I)) Γ ( Ψ ) ≥ Γ ( π g ( I )) for 1 ≤ I ≤ n / n b 1\leq I\leq n/n_b 1 ≤ I ≤ n / n b . Ditto for π t \pi_t π t . Using (10) and the same counting argument as in the proof of Theorem 3.1, we can write
∣ A I I − ∑ K = 1 I − 1 L ^ I K L ^ I K T − A ^ I I u ∣ ≤ ∑ K = 1 I − 1 Θ ( π g ( K ) , n b ) ( ρ ( I , K ) ) ( ρ ( I , K ) ) T + ( max K ≤ I Γ ( π g ( K ) ) + I ) ϵ 64 ( ∑ K = 1 I − 1 ∣ L ^ I K ∣ ∣ L ^ I K T ∣ + ∣ A I I ∣ ) ≤ Θ ( ψ , n b ) ∑ K = 1 I − 1 ( ρ ( I , K ) ) ( ρ ( I , K ) ) T + ( Γ ( Ψ ) + I ) ϵ 64 ( ∑ K = 1 I − 1 ∣ L ^ I K ∣ ∣ L ^ I K T ∣ + ∣ A I I ∣ ) . (30) \begin{aligned}
\left|A_{II}-\sum_{K=1}^{I-1}\hat{L}_{IK}\hat{L}_{IK}^{T}-\hat{A}_{II}^{u}\right|&\leq\sum_{K=1}^{I-1}\Theta(\pi_g(K),n_b)(\rho^{(I,K)})(\rho^{(I,K)})^{T}+\left(\max_{K\leq I}\Gamma(\pi_g(K))+I\right)\epsilon_{64}\left(\sum_{K=1}^{I-1}|\hat{L}_{IK}||\hat{L}_{IK}^{T}|+|A_{II}|\right)\\
&\leq\Theta(\psi,n_b)\sum_{K=1}^{I-1}(\rho^{(I,K)})(\rho^{(I,K)})^{T}+\left(\Gamma(\Psi)+I\right)\epsilon_{64}\left(\sum_{K=1}^{I-1}|\hat{L}_{IK}||\hat{L}_{IK}^{T}|+|A_{II}|\right). \tag*{(30)}
\end{aligned} A I I − K = 1 ∑ I − 1 L ^ I K L ^ I K T − A ^ I I u ≤ K = 1 ∑ I − 1 Θ ( π g ( K ) , n b ) ( ρ ( I , K ) ) ( ρ ( I , K ) ) T + ( K ≤ I max Γ ( π g ( K )) + I ) ϵ 64 ( K = 1 ∑ I − 1 ∣ L ^ I K ∣∣ L ^ I K T ∣ + ∣ A I I ∣ ) ≤ Θ ( ψ , n b ) K = 1 ∑ I − 1 ( ρ ( I , K ) ) ( ρ ( I , K ) ) T + ( Γ ( Ψ ) + I ) ϵ 64 ( K = 1 ∑ I − 1 ∣ L ^ I K ∣∣ L ^ I K T ∣ + ∣ A I I ∣ ) . (30)
where ρ ( I , K ) \rho^{(I,K)} ρ ( I , K ) is the length n b n_b n b vector defined as ρ s ( I , K ) = max ( K − 1 ) n b ≤ j ≤ K n b ∣ l ^ s j ∣ \rho_s^{(I,K)}=\max_{(K-1)n_b\leq j\leq Kn_b}|\hat{l}_{sj}| ρ s ( I , K ) = max ( K − 1 ) n b ≤ j ≤ K n b ∣ l ^ s j ∣ for ( I − 1 ) n b ≤ s ≤ I n b (I-1)n_b\leq s\leq In_b ( I − 1 ) n b ≤ s ≤ I n b . Combining (30) and (29) yields
∣ A I I − ∑ K = 1 I L ^ I K L ^ I K T ∣ ≤ Θ ( ψ , n b ) ∑ K = 1 I − 1 ( ρ ( I , K ) ) ( ρ ( I , K ) ) T + ( Γ ( Ψ ) + I ) ϵ 64 ( ∑ K = 1 I − 1 ∣ L ^ I K ∣ ∣ L ^ I K T ∣ + ∣ A I I ∣ ) + n b ϵ 64 Ω I I . (31) \left|A_{II}-\sum_{K=1}^{I}\hat{L}_{IK}\hat{L}_{IK}^{T}\right|\leq\Theta(\psi,n_b)\sum_{K=1}^{I-1}(\rho^{(I,K)})(\rho^{(I,K)})^{T}+\left(\Gamma(\Psi)+I\right)\epsilon_{64}\left(\sum_{K=1}^{I-1}|\hat{L}_{IK}||\hat{L}_{IK}^{T}|+|A_{II}|\right)+n_b\epsilon_{64}\Omega_{II}. \tag*{(31)} A I I − K = 1 ∑ I L ^ I K L ^ I K T ≤ Θ ( ψ , n b ) K = 1 ∑ I − 1 ( ρ ( I , K ) ) ( ρ ( I , K ) ) T + ( Γ ( Ψ ) + I ) ϵ 64 ( K = 1 ∑ I − 1 ∣ L ^ I K ∣∣ L ^ I K T ∣ + ∣ A I I ∣ ) + n b ϵ 64 Ω I I . (31)
Assuming s < t ≤ n s<t\leq n s < t ≤ n , since L L ^ T = A + E L\hat{L}^{T}=A+E L L ^ T = A + E , we can write ∑ k = 1 I l ^ s k l ^ t k = a s t + e s t \sum_{k=1}^{I}\hat{l}_{sk}\hat{l}_{tk}=a_{st}+e_{st} ∑ k = 1 I l ^ s k l ^ t k = a s t + e s t , and ∑ k = 1 I l ^ t k 2 = a t t + e t t \sum_{k=1}^{I}\hat{l}_{tk}^{2}=a_{tt}+e_{tt} ∑ k = 1 I l ^ t k 2 = a tt + e tt . Ditto for s s s . Also, each element, ρ t ( I , K ) \rho_t^{(I,K)} ρ t ( I , K ) of ρ ( I , K ) \rho^{(I,K)} ρ ( I , K ) is bounded by ∑ k = ( K − 1 ) n b + 1 K n b l ^ t k 2 \sqrt{\sum_{k=(K-1)n_b+1}^{Kn_b}\hat{l}_{tk}^{2}} ∑ k = ( K − 1 ) n b + 1 K n b l ^ t k 2 , and so ∑ K = 1 I − 1 ( ρ t ( I , K ) ) 2 ≤ ∑ k = 1 ( I − 1 ) n b l ^ t k 2 \sum_{K=1}^{I-1}(\rho_t^{(I,K)})^2\leq\sum_{k=1}^{(I-1)n_b}\hat{l}_{tk}^{2} ∑ K = 1 I − 1 ( ρ t ( I , K ) ) 2 ≤ ∑ k = 1 ( I − 1 ) n b l ^ t k 2 . Now zoom in to the diagonal elements of the diagonal blocks of A A A in (31) . Looking at the diagonal element a t t a_{tt} a tt , and noting that I ≤ n / n b I\leq n/n_b I ≤ n / n b , we can write
∣ e t t ∣ = ∣ a t t − ∑ k = 1 t l ^ t k 2 ∣ ≤ Θ ( ψ , n b ) ( ∣ a t t ∣ + ∣ e t t ∣ ) + ( Γ ( Ψ ) + n n b ) ϵ 64 ( ∑ k = 1 ( I − 1 ) n b l ^ t k 2 + ∣ a t t ∣ ) + n b ϵ 64 ∣ a ^ t t u ∣ . (32) |e_{tt}|=\left|a_{tt}-\sum_{k=1}^{t}\hat{l}_{tk}^{2}\right|\leq\Theta(\psi,n_b)(|a_{tt}|+|e_{tt}|)+\left(\Gamma(\Psi)+\frac{n}{n_b}\right)\epsilon_{64}\left(\sum_{k=1}^{(I-1)n_b}\hat{l}_{tk}^{2}+|a_{tt}|\right)+n_b\epsilon_{64}|\hat{a}_{tt}^{u}|. \tag*{(32)} ∣ e tt ∣ = a tt − k = 1 ∑ t l ^ t k 2 ≤ Θ ( ψ , n b ) ( ∣ a tt ∣ + ∣ e tt ∣ ) + ( Γ ( Ψ ) + n b n ) ϵ 64 k = 1 ∑ ( I − 1 ) n b l ^ t k 2 + ∣ a tt ∣ + n b ϵ 64 ∣ a ^ tt u ∣. (32)
The entry ( t , t ) (t,t) ( t , t ) of (29) is ∣ a ^ t t u − ∑ k = ( I − 1 ) n b + 1 I n b l ^ t k 2 ∣ ≤ n b ϵ 64 ∣ a ^ t t u ∣ |\hat{a}_{tt}^{u}-\sum_{k=(I-1)n_b+1}^{In_b}\hat{l}_{tk}^{2}|\leq n_b\epsilon_{64}|\hat{a}_{tt}^{u}| ∣ a ^ tt u − ∑ k = ( I − 1 ) n b + 1 I n b l ^ t k 2 ∣ ≤ n b ϵ 64 ∣ a ^ tt u ∣ , so ∣ a ^ t t u ∣ ≤ 1 1 − n b ϵ 64 ∑ k = ( I − 1 ) n b + 1 I n b l ^ t k 2 |\hat{a}_{tt}^{u}|\leq\frac{1}{1-n_b\epsilon_{64}}\sum_{k=(I-1)n_b+1}^{In_b}\hat{l}_{tk}^{2} ∣ a ^ tt u ∣ ≤ 1 − n b ϵ 64 1 ∑ k = ( I − 1 ) n b + 1 I n b l ^ t k 2 . Plugging this into (32),
∣ e t t ∣ ≤ Θ ( ψ , n b ) ( ∣ a t t ∣ + ∣ e t t ∣ ) + ( Γ ( Ψ ) + n n b ) ϵ 64 ( ∑ k = 1 ( I − 1 ) n b l ^ t k 2 ) + ( Γ ( Ψ ) + n n b ) ϵ 64 ∣ a t t ∣ + n b ϵ 64 1 − n b ϵ 64 ( ∑ k = ( I − 1 ) n b + 1 I n b l ^ t k 2 ) |e_{tt}|\leq\Theta(\psi,n_b)(|a_{tt}|+|e_{tt}|)+\left(\Gamma(\Psi)+\frac{n}{n_b}\right)\epsilon_{64}\left(\sum_{k=1}^{(I-1)n_b}\hat{l}_{tk}^{2}\right)+\left(\Gamma(\Psi)+\frac{n}{n_b}\right)\epsilon_{64}|a_{tt}|+\frac{n_b\epsilon_{64}}{1-n_b\epsilon_{64}}\left(\sum_{k=(I-1)n_b+1}^{In_b}\hat{l}_{tk}^{2}\right) ∣ e tt ∣ ≤ Θ ( ψ , n b ) ( ∣ a tt ∣ + ∣ e tt ∣ ) + ( Γ ( Ψ ) + n b n ) ϵ 64 k = 1 ∑ ( I − 1 ) n b l ^ t k 2 + ( Γ ( Ψ ) + n b n ) ϵ 64 ∣ a tt ∣ + 1 − n b ϵ 64 n b ϵ 64 k = ( I − 1 ) n b + 1 ∑ I n b l ^ t k 2
⟹ ∣ e t t ∣ ≤ Θ ( ψ , n b ) ( ∣ a t t ∣ + ∣ e t t ∣ ) + M ϵ 64 ( ∑ k = 1 ( I − 1 ) n b l ^ t k 2 ) + M ϵ 64 ∣ a t t ∣ + M ϵ 64 ( ∑ k = ( I − 1 ) n b + 1 I n b l ^ t k 2 ) \Longrightarrow\ |e_{tt}|\leq\Theta(\psi,n_b)(|a_{tt}|+|e_{tt}|)+M\epsilon_{64}\left(\sum_{k=1}^{(I-1)n_b}\hat{l}_{tk}^{2}\right)+M\epsilon_{64}|a_{tt}|+M\epsilon_{64}\left(\sum_{k=(I-1)n_b+1}^{In_b}\hat{l}_{tk}^{2}\right) ⟹ ∣ e tt ∣ ≤ Θ ( ψ , n b ) ( ∣ a tt ∣ + ∣ e tt ∣ ) + M ϵ 64 k = 1 ∑ ( I − 1 ) n b l ^ t k 2 + M ϵ 64 ∣ a tt ∣ + M ϵ 64 k = ( I − 1 ) n b + 1 ∑ I n b l ^ t k 2
⟹ ∣ e t t ∣ ≤ ( Θ ( ψ , n b ) + M ϵ 64 ) ( ∣ a t t ∣ + ∣ e t t ∣ ) + M ϵ 64 ∣ a t t ∣ . \Longrightarrow\ |e_{tt}|\leq\left(\Theta(\psi,n_b)+M\epsilon_{64}\right)(|a_{tt}|+|e_{tt}|)+M\epsilon_{64}|a_{tt}|. ⟹ ∣ e tt ∣ ≤ ( Θ ( ψ , n b ) + M ϵ 64 ) ( ∣ a tt ∣ + ∣ e tt ∣ ) + M ϵ 64 ∣ a tt ∣.
Thus, for the diagonal elements,
∣ e t t ∣ ≤ Θ ( ψ , n b ) + 2 M ϵ 64 1 − Θ ( ψ , n b ) − M ϵ 64 ∣ a t t ∣ . (33) |e_{tt}|\leq\frac{\Theta(\psi,n_b)+2M\epsilon_{64}}{1-\Theta(\psi,n_b)-M\epsilon_{64}}|a_{tt}|. \tag*{(33)} ∣ e tt ∣ ≤ 1 − Θ ( ψ , n b ) − M ϵ 64 Θ ( ψ , n b ) + 2 M ϵ 64 ∣ a tt ∣. (33)
The same inequality holds for all indices from 1 to n n n . For off-diagonal elements in the diagonal block, (31) becomes
∣ e s t ∣ = ∣ a s t − ∑ k = 1 s l ^ s k l ^ t k ∣ ≤ Θ ( ψ , n b ) ∑ K = 1 I − 1 ρ s ( I , K ) ρ t ( I , K ) + ( Γ ( Ψ ) + n n b ) ϵ 64 ( ∑ k = 1 ( I − 1 ) n b ∣ l ^ s k ∣ ∣ l ^ t k ∣ + ∣ a s t ∣ ) + n b ϵ 64 ω s t (34) \left|e_{st}\right|=\left|a_{st}-\sum_{k=1}^{s}\hat{l}_{sk}\hat{l}_{tk}\right|\leq\Theta(\psi,n_b)\sum_{K=1}^{I-1}\rho_s^{(I,K)}\rho_t^{(I,K)}+\left(\Gamma(\Psi)+\frac{n}{n_b}\right)\epsilon_{64}\left(\sum_{k=1}^{(I-1)n_b}\left|\hat{l}_{sk}\right|\left|\hat{l}_{tk}\right|+\left|a_{st}\right|\right)+n_b\epsilon_{64}\omega_{st} \tag*{(34)} ∣ e s t ∣ = a s t − k = 1 ∑ s l ^ s k l ^ t k ≤ Θ ( ψ , n b ) K = 1 ∑ I − 1 ρ s ( I , K ) ρ t ( I , K ) + ( Γ ( Ψ ) + n b n ) ϵ 64 k = 1 ∑ ( I − 1 ) n b l ^ s k l ^ t k + ∣ a s t ∣ + n b ϵ 64 ω s t (34)
First, using Cauchy-Schwarz, we can bound ∑ K = 1 I − 1 ρ s ( I , K ) ρ t ( I , K ) ≤ ∑ k = 1 t l ^ t k 2 ∑ k = 1 t l ^ s k 2 ≤ ∣ a s s ∣ + ∣ e s s ∣ ∣ a t t ∣ + ∣ e t t ∣ \sum_{K=1}^{I-1}\rho_s^{(I,K)}\rho_t^{(I,K)}\leq\sqrt{\sum_{k=1}^{t}\hat{l}_{tk}^{2}}\sqrt{\sum_{k=1}^{t}\hat{l}_{sk}^{2}}\leq\sqrt{\left|a_{ss}\right|+\left|e_{ss}\right|}\sqrt{\left|a_{tt}\right|+\left|e_{tt}\right|} ∑ K = 1 I − 1 ρ s ( I , K ) ρ t ( I , K ) ≤ ∑ k = 1 t l ^ t k 2 ∑ k = 1 t l ^ s k 2 ≤ ∣ a ss ∣ + ∣ e ss ∣ ∣ a tt ∣ + ∣ e tt ∣ . So,
∣ e s t ∣ ≤ Θ ( ψ , n b ) ( ∣ a t t ∣ + ∣ e t t ∣ ) ( ∣ a s s ∣ + ∣ e s s ∣ ) + ( Γ ( Ψ ) + n n b ) ϵ 64 ( ∑ k = 1 ( I − 1 ) n b ∣ l ^ s k ∣ ∣ l ^ t k ∣ + ∣ a s t ∣ ) + n b ϵ 64 ω s t . (35) \left|e_{st}\right|\leq\Theta(\psi,n_b)\sqrt{\left(\left|a_{tt}\right|+\left|e_{tt}\right|\right)\left(\left|a_{ss}\right|+\left|e_{ss}\right|\right)}+\left(\Gamma(\Psi)+\frac{n}{n_b}\right)\epsilon_{64}\left(\sum_{k=1}^{(I-1)n_b}\left|\hat{l}_{sk}\right|\left|\hat{l}_{tk}\right|+\left|a_{st}\right|\right)+n_b\epsilon_{64}\omega_{st}. \tag*{(35)} ∣ e s t ∣ ≤ Θ ( ψ , n b ) ( ∣ a tt ∣ + ∣ e tt ∣ ) ( ∣ a ss ∣ + ∣ e ss ∣ ) + ( Γ ( Ψ ) + n b n ) ϵ 64 k = 1 ∑ ( I − 1 ) n b l ^ s k l ^ t k + ∣ a s t ∣ + n b ϵ 64 ω s t . (35)
Applying Cauchy-Schwarz to ∑ k = 1 ( I − 1 ) n b ∣ l ^ s k ∣ ∣ l ^ t k ∣ \sum_{k=1}^{(I-1)n_b}\left|\hat{l}_{sk}\right|\left|\hat{l}_{tk}\right| ∑ k = 1 ( I − 1 ) n b l ^ s k l ^ t k and using ∣ a ^ t t ∣ ≤ 1 1 − n b ϵ 64 ∑ k = ( I − 1 ) n b + 1 I n b l ^ t k 2 \left|\hat{a}_{tt}\right|\leq\frac{1}{1-n_b\epsilon_{64}}\sum_{k=(I-1)n_b+1}^{In_b}\hat{l}_{tk}^{2} ∣ a ^ tt ∣ ≤ 1 − n b ϵ 64 1 ∑ k = ( I − 1 ) n b + 1 I n b l ^ t k 2 we can simplify the second and third terms in the right hand side of (35) as follows:
( Γ ( Ψ ) + n n b ) ϵ 64 ( ∑ k = 1 ( I − 1 ) n b ∣ l ^ s k ∣ ∣ l ^ t k ∣ + ∣ a s t ∣ ) + n b ϵ 64 ω s t ≤ M ϵ 64 ∑ k = 1 ( I − 1 ) n b l ^ s k 2 ∑ k = 1 ( I − 1 ) n b l ^ t k 2 + M ϵ 64 ∣ a s t ∣ + M ϵ 64 ∑ k = ( I − 1 ) n b + 1 I n b l ^ s k 2 ∑ k = ( I − 1 ) n b + 1 I n b l ^ t k 2 . \begin{aligned}
\left(\Gamma(\Psi)+\frac{n}{n_b}\right)\epsilon_{64}\left(\sum_{k=1}^{(I-1)n_b}\left|\hat{l}_{sk}\right|\left|\hat{l}_{tk}\right|+\left|a_{st}\right|\right)+n_b\epsilon_{64}\omega_{st}\leq M\epsilon_{64}\sqrt{\sum_{k=1}^{(I-1)n_b}\hat{l}_{sk}^{2}\sum_{k=1}^{(I-1)n_b}\hat{l}_{tk}^{2}}+M\epsilon_{64}\left|a_{st}\right| \\
&+M\epsilon_{64}\sqrt{\sum_{k=(I-1)n_b+1}^{In_b}\hat{l}_{sk}^{2}\sum_{k=(I-1)n_b+1}^{In_b}\hat{l}_{tk}^{2}}.
\end{aligned} ( Γ ( Ψ ) + n b n ) ϵ 64 k = 1 ∑ ( I − 1 ) n b l ^ s k l ^ t k + ∣ a s t ∣ + n b ϵ 64 ω s t ≤ M ϵ 64 k = 1 ∑ ( I − 1 ) n b l ^ s k 2 k = 1 ∑ ( I − 1 ) n b l ^ t k 2 + M ϵ 64 ∣ a s t ∣ + M ϵ 64 k = ( I − 1 ) n b + 1 ∑ I n b l ^ s k 2 k = ( I − 1 ) n b + 1 ∑ I n b l ^ t k 2 .
For non-negative reals x 1 , x 2 , y 1 , y 2 x_1,x_2,y_1,y_2 x 1 , x 2 , y 1 , y 2 , we know that x 1 y 1 + x 2 y 2 ≤ ( x 1 + x 2 ) ( y 1 + y 2 ) \sqrt{x_1y_1}+\sqrt{x_2y_2}\leq\sqrt{(x_1+x_2)(y_1+y_2)} x 1 y 1 + x 2 y 2 ≤ ( x 1 + x 2 ) ( y 1 + y 2 ) . Thus, we can merge l ^ t k \hat{l}_{tk} l ^ t k and l ^ s k \hat{l}_{sk} l ^ s k sums and upper bound the above expression by
M ϵ 64 ∑ k = 1 I n b l ^ s k 2 ∑ k = 1 I n b l ^ t k 2 + M ϵ 64 ∣ a s t ∣ ≤ M ϵ 64 ( ∣ a s s ∣ + ∣ e s s ∣ ) ( ∣ a t t ∣ + ∣ e t t ∣ ) + M ϵ 64 ∣ a s t ∣ . M\epsilon_{64}\sqrt{\sum_{k=1}^{In_b}\hat{l}_{sk}^{2}\sum_{k=1}^{In_b}\hat{l}_{tk}^{2}}+M\epsilon_{64}\left|a_{st}\right|\leq M\epsilon_{64}\sqrt{\left(\left|a_{ss}\right|+\left|e_{ss}\right|\right)\left(\left|a_{tt}\right|+\left|e_{tt}\right|\right)}+M\epsilon_{64}\left|a_{st}\right|. M ϵ 64 k = 1 ∑ I n b l ^ s k 2 k = 1 ∑ I n b l ^ t k 2 + M ϵ 64 ∣ a s t ∣ ≤ M ϵ 64 ( ∣ a ss ∣ + ∣ e ss ∣ ) ( ∣ a tt ∣ + ∣ e tt ∣ ) + M ϵ 64 ∣ a s t ∣ .
Plugging this into (35) , we get
∣ e s t ∣ ≤ ( Θ ( ψ , n b ) + M ϵ 64 ) ( ∣ a s s ∣ + ∣ e s s ∣ ) ( ∣ a t t ∣ + ∣ e t t ∣ ) + M ϵ 64 ∣ a s t ∣ , (36) \left|e_{st}\right|\leq\left(\Theta(\psi,n_b)+M\epsilon_{64}\right)\sqrt{\left(\left|a_{ss}\right|+\left|e_{ss}\right|\right)\left(\left|a_{tt}\right|+\left|e_{tt}\right|\right)}+M\epsilon_{64}\left|a_{st}\right|, \tag*{(36)} ∣ e s t ∣ ≤ ( Θ ( ψ , n b ) + M ϵ 64 ) ( ∣ a ss ∣ + ∣ e ss ∣ ) ( ∣ a tt ∣ + ∣ e tt ∣ ) + M ϵ 64 ∣ a s t ∣ , (36)
Then, to bound ∣ a s t ∣ \left|a_{st}\right| ∣ a s t ∣ , note that ∣ a s t ∣ ≤ ∑ k = 1 s ∣ l ^ t k ∣ ∣ l ^ s k ∣ + ∣ e s t ∣ ≤ ( ∣ a s s ∣ + ∣ e s s ∣ ) ( ∣ a t t ∣ + ∣ e t t ∣ ) + ∣ e s t ∣ \left|a_{st}\right|\leq\sum_{k=1}^{s}\left|\hat{l}_{tk}\right|\left|\hat{l}_{sk}\right|+\left|e_{st}\right|\leq\sqrt{\left(\left|a_{ss}\right|+\left|e_{ss}\right|\right)\left(\left|a_{tt}\right|+\left|e_{tt}\right|\right)}+\left|e_{st}\right| ∣ a s t ∣ ≤ ∑ k = 1 s l ^ t k l ^ s k + ∣ e s t ∣ ≤ ( ∣ a ss ∣ + ∣ e ss ∣ ) ( ∣ a tt ∣ + ∣ e tt ∣ ) + ∣ e s t ∣ . Substituting this into (36) ,
∣ e s t ∣ ≤ ( Θ ( ψ , n b ) + 2 M ϵ 64 ) ( ∣ a s s ∣ + ∣ e s s ∣ ) ( ∣ a t t ∣ + ∣ e t t ∣ ) + M ϵ 64 ∣ e s t ∣ \left|e_{st}\right|\leq\left(\Theta(\psi,n_b)+2M\epsilon_{64}\right)\sqrt{\left(\left|a_{ss}\right|+\left|e_{ss}\right|\right)\left(\left|a_{tt}\right|+\left|e_{tt}\right|\right)}+M\epsilon_{64}\left|e_{st}\right| ∣ e s t ∣ ≤ ( Θ ( ψ , n b ) + 2 M ϵ 64 ) ( ∣ a ss ∣ + ∣ e ss ∣ ) ( ∣ a tt ∣ + ∣ e tt ∣ ) + M ϵ 64 ∣ e s t ∣
⟹ ∣ e s t ∣ ≤ Θ ( ψ , n b ) + 2 M ϵ 64 1 − M ϵ 64 ( ∣ a s s ∣ + ∣ e s s ∣ ) ( ∣ a t t ∣ + ∣ e t t ∣ ) . \Longrightarrow\left|e_{st}\right|\leq\frac{\Theta(\psi,n_b)+2M\epsilon_{64}}{1-M\epsilon_{64}}\sqrt{\left(\left|a_{ss}\right|+\left|e_{ss}\right|\right)\left(\left|a_{tt}\right|+\left|e_{tt}\right|\right)}. ⟹ ∣ e s t ∣ ≤ 1 − M ϵ 64 Θ ( ψ , n b ) + 2 M ϵ 64 ( ∣ a ss ∣ + ∣ e ss ∣ ) ( ∣ a tt ∣ + ∣ e tt ∣ ) .
Substituting e s s , e t t e_{ss},e_{tt} e ss , e tt from (33) then gives
∣ e s t ∣ ≤ Θ ( ψ , n b ) + 2 M ϵ 64 1 − M ϵ 64 ⋅ 1 + M ϵ 64 1 − Θ ( ψ , n b ) − M ϵ 64 ∣ a s s ∣ ∣ a t t ∣ ≤ Θ ( ψ , n b ) + 2 M ϵ 64 1 − M ϵ 64 ⋅ 1 − M ϵ 64 1 − Θ ( ψ , n b ) − 3 M ϵ 64 ∣ a s s ∣ ∣ a t t ∣ = Θ ( ψ , n b ) + 2 M ϵ 64 1 − Θ ( ψ , n b ) − 3 M ϵ 64 ∣ a s s ∣ ∣ a t t ∣ . (37) \begin{aligned}
\left|e_{st}\right|\leq\frac{\Theta(\psi,n_b)+2M\epsilon_{64}}{1-M\epsilon_{64}}\cdot\frac{1+M\epsilon_{64}}{1-\Theta(\psi,n_b)-M\epsilon_{64}}\sqrt{\left|a_{ss}\right|\left|a_{tt}\right|}\leq\frac{\Theta(\psi,n_b)+2M\epsilon_{64}}{1-M\epsilon_{64}}\cdot\frac{1-M\epsilon_{64}}{1-\Theta(\psi,n_b)-3M\epsilon_{64}}\sqrt{\left|a_{ss}\right|\left|a_{tt}\right|} \\
&=\frac{\Theta(\psi,n_b)+2M\epsilon_{64}}{1-\Theta(\psi,n_b)-3M\epsilon_{64}}\sqrt{\left|a_{ss}\right|\left|a_{tt}\right|}. \tag*{(37)}
\end{aligned} ∣ e s t ∣ ≤ 1 − M ϵ 64 Θ ( ψ , n b ) + 2 M ϵ 64 ⋅ 1 − Θ ( ψ , n b ) − M ϵ 64 1 + M ϵ 64 ∣ a ss ∣ ∣ a tt ∣ ≤ 1 − M ϵ 64 Θ ( ψ , n b ) + 2 M ϵ 64 ⋅ 1 − Θ ( ψ , n b ) − 3 M ϵ 64 1 − M ϵ 64 ∣ a ss ∣ ∣ a tt ∣ = 1 − Θ ( ψ , n b ) − 3 M ϵ 64 Θ ( ψ , n b ) + 2 M ϵ 64 ∣ a ss ∣ ∣ a tt ∣ . (37)
To compute the error bound for the off-diagonal blocks, we must use the Ozaki 1 TRSM bound (17) rather than the Cholesky error bound on the diagonal block, and we use (10) for the error bound on GEMM. Let J < I ≤ n / n b J<I\leq n/n_b J < I ≤ n / n b . Bounding ∣ A I J − ∑ K = 1 J L ^ I K L ^ J K T ∣ \left|A_{IJ}-\sum_{K=1}^{J}\hat{L}_{IK}\hat{L}_{JK}^{T}\right| A I J − ∑ K = 1 J L ^ I K L ^ J K T by the Schur-update error ∣ A I J − ∑ K = 1 J − 1 L ^ I K L ^ J K T − A ^ I J u ∣ \left|A_{IJ}-\sum_{K=1}^{J-1}\hat{L}_{IK}\hat{L}_{JK}^{T}-\hat{A}^{u}_{IJ}\right| A I J − ∑ K = 1 J − 1 L ^ I K L ^ J K T − A ^ I J u plus the TRSM error ∣ A ^ I J u − L ^ I J L ^ J J T ∣ \left|\hat{A}^{u}_{IJ}-\hat{L}_{IJ}\hat{L}_{JJ}^{T}\right| A ^ I J u − L ^ I J L ^ J J T , we get
∣ A I J − ∑ K = 1 J L ^ I K L ^ J K T ∣ ≤ ∑ K = 1 J − 1 Θ ( π g ( K ) , n b ) ( ρ ( I , K ) ) ( ρ ( J , K ) ) T + ( max K ≤ J Γ ( π g ( K ) ) + J ) ϵ 64 ( ∑ K = 1 J − 1 ∣ L ^ I K ∣ ∣ L ^ J K T ∣ + ∣ A I J ∣ ) + 1 1 − ( Γ ( π t ( J ) ) + n b ) ϵ 64 ( Θ ( π t ( J ) , n b ) ( ρ ( I , J ) ) ( ρ ( J , J ) ) T + 2 ( Γ ( π t ( J ) ) + n b ) ϵ 64 ∣ L ^ I J ∣ ∣ L ^ J J T ∣ ) (38) \begin{aligned}
\left|A_{IJ}-\sum_{K=1}^{J}\hat{L}_{IK}\hat{L}_{JK}^{T}\right|
&\leq\sum_{K=1}^{J-1}\Theta(\pi_g(K),n_b)(\rho^{(I,K)})(\rho^{(J,K)})^{T}\\
&\quad+\left(\max_{K\leq J}\Gamma(\pi_g(K))+J\right)\epsilon_{64}\left(\sum_{K=1}^{J-1}|\hat{L}_{IK}||\hat{L}_{JK}^{T}|+|A_{IJ}|\right)\\
&\quad+\frac{1}{1-(\Gamma(\pi_t(J))+n_b)\epsilon_{64}}\left(\Theta(\pi_t(J),n_b)(\rho^{(I,J)})(\rho^{(J,J)})^{T}\right.\\
&\qquad\left.+2(\Gamma(\pi_t(J))+n_b)\epsilon_{64}|\hat{L}_{IJ}||\hat{L}_{JJ}^{T}|\right) \tag*{(38)}
\end{aligned} A I J − K = 1 ∑ J L ^ I K L ^ J K T ≤ K = 1 ∑ J − 1 Θ ( π g ( K ) , n b ) ( ρ ( I , K ) ) ( ρ ( J , K ) ) T + ( K ≤ J max Γ ( π g ( K )) + J ) ϵ 64 ( K = 1 ∑ J − 1 ∣ L ^ I K ∣∣ L ^ J K T ∣ + ∣ A I J ∣ ) + 1 − ( Γ ( π t ( J )) + n b ) ϵ 64 1 ( Θ ( π t ( J ) , n b ) ( ρ ( I , J ) ) ( ρ ( J , J ) ) T + 2 ( Γ ( π t ( J )) + n b ) ϵ 64 ∣ L ^ I J ∣∣ L ^ J J T ∣ ) (38)
≤ Θ ( ψ , n b ) ∑ K = 1 J − 1 ( ρ ( I , K ) ) ( ρ ( J , K ) ) T + ( Γ ( Ψ ) + J ) ϵ 64 ( ∑ K = 1 J − 1 ∣ L ^ I K ∣ ∣ L ^ J K T ∣ + ∣ A I J ∣ ) + 1 1 − ( Γ ( Ψ ) + n b ) ϵ 64 ( Θ ( ψ , n b ) ( ρ ( I , J ) ) ( ρ ( J , J ) ) T + 2 ( Γ ( Ψ ) + n b ) ϵ 64 ∣ L ^ I J ∣ ∣ L ^ J J T ∣ ) . \begin{aligned}
&\leq\Theta(\psi,n_b)\sum_{K=1}^{J-1}(\rho^{(I,K)})(\rho^{(J,K)})^{T}+(\Gamma(\Psi)+J)\epsilon_{64}\left(\sum_{K=1}^{J-1}|\hat{L}_{IK}||\hat{L}_{JK}^{T}|+|A_{IJ}|\right)\\
&\quad+\frac{1}{1-(\Gamma(\Psi)+n_b)\epsilon_{64}}\left(\Theta(\psi,n_b)(\rho^{(I,J)})(\rho^{(J,J)})^{T}+2(\Gamma(\Psi)+n_b)\epsilon_{64}|\hat{L}_{IJ}||\hat{L}_{JJ}^{T}|\right).
\end{aligned} ≤ Θ ( ψ , n b ) K = 1 ∑ J − 1 ( ρ ( I , K ) ) ( ρ ( J , K ) ) T + ( Γ ( Ψ ) + J ) ϵ 64 ( K = 1 ∑ J − 1 ∣ L ^ I K ∣∣ L ^ J K T ∣ + ∣ A I J ∣ ) + 1 − ( Γ ( Ψ ) + n b ) ϵ 64 1 ( Θ ( ψ , n b ) ( ρ ( I , J ) ) ( ρ ( J , J ) ) T + 2 ( Γ ( Ψ ) + n b ) ϵ 64 ∣ L ^ I J ∣∣ L ^ J J T ∣ ) .
where the second line is (17) applied to solving L ^ I J L ^ J J T = A ^ I J u \hat{L}_{IJ}\hat{L}_{JJ}^{T}=\hat{A}^{u}_{IJ} L ^ I J L ^ J J T = A ^ I J u with some block size n b ′ ≤ n b n'_b\leq n_b n b ′ ≤ n b . Note that although the theorem is proved for L X = B LX=B L X = B , it also holds for X L T = B XL^{T}=B X L T = B (with appropriate reordering of terms) since all the matrices are n b × n b n_b\times n_b n b × n b in this case. Now, we want to merge the error contributed by TRSM and the error contributed by GEMM/SYRK. Using Θ ( ψ , n b ) ∑ K = 1 J − 1 ( ρ ( I , K ) ) ( ρ ( J , K ) ) T + Θ ( ψ , n b ) 1 − M ϵ 64 ( ρ ( I , J ) ) ( ρ ( J , J ) ) T ≤ Θ ( ψ , n b ) 1 − M ϵ 64 ∑ K = 1 J ( ρ ( I , K ) ) ( ρ ( J , K ) ) T \Theta(\psi,n_b)\sum_{K=1}^{J-1}(\rho^{(I,K)})(\rho^{(J,K)})^{T}+\frac{\Theta(\psi,n_b)}{1-M\epsilon_{64}}(\rho^{(I,J)})(\rho^{(J,J)})^{T}\leq\frac{\Theta(\psi,n_b)}{1-M\epsilon_{64}}\sum_{K=1}^{J}(\rho^{(I,K)})(\rho^{(J,K)})^{T} Θ ( ψ , n b ) ∑ K = 1 J − 1 ( ρ ( I , K ) ) ( ρ ( J , K ) ) T + 1 − M ϵ 64 Θ ( ψ , n b ) ( ρ ( I , J ) ) ( ρ ( J , J ) ) T ≤ 1 − M ϵ 64 Θ ( ψ , n b ) ∑ K = 1 J ( ρ ( I , K ) ) ( ρ ( J , K ) ) T , and noting that Γ ( Ψ ) + J ≤ Γ ( Ψ ) + n n b ≤ M \Gamma(\Psi)+J\leq\Gamma(\Psi)+\frac{n}{n_b}\leq M Γ ( Ψ ) + J ≤ Γ ( Ψ ) + n b n ≤ M , Γ ( Ψ ) + n b ≤ M \Gamma(\Psi)+n_b\leq M Γ ( Ψ ) + n b ≤ M , and M ϵ 64 ≤ 2 M ϵ 64 1 − M ϵ 64 M\epsilon_{64}\leq\frac{2M\epsilon_{64}}{1-M\epsilon_{64}} M ϵ 64 ≤ 1 − M ϵ 64 2 M ϵ 64 , we can rewrite (38) as
∣ A I J − ∑ K = 1 J L ^ I K L ^ J K T ∣ ≤ Θ ( ψ , n b ) 1 − M ϵ 64 ∑ K = 1 J ( ρ ( I , K ) ) ( ρ ( J , K ) ) T + 2 M ϵ 64 1 − M ϵ 64 ∑ K = 1 J ∣ L ^ I K ∣ ∣ L ^ J K T ∣ + M ϵ 64 ∣ A I J ∣ . (39) \left|A_{IJ}-\sum_{K=1}^{J}\hat{L}_{IK}\hat{L}_{JK}^{T}\right|
\leq\frac{\Theta(\psi,n_b)}{1-M\epsilon_{64}}\sum_{K=1}^{J}(\rho^{(I,K)})(\rho^{(J,K)})^{T}
+\frac{2M\epsilon_{64}}{1-M\epsilon_{64}}\sum_{K=1}^{J}|\hat{L}_{IK}||\hat{L}_{JK}^{T}|+M\epsilon_{64}|A_{IJ}|. \tag*{(39)} A I J − K = 1 ∑ J L ^ I K L ^ J K T ≤ 1 − M ϵ 64 Θ ( ψ , n b ) K = 1 ∑ J ( ρ ( I , K ) ) ( ρ ( J , K ) ) T + 1 − M ϵ 64 2 M ϵ 64 K = 1 ∑ J ∣ L ^ I K ∣∣ L ^ J K T ∣ + M ϵ 64 ∣ A I J ∣. (39)
Again, the element ( s , t ) (s,t) ( s , t ) of ∑ K = 1 J ∣ L ^ I K ∣ ∣ L ^ J K T ∣ \sum_{K=1}^{J}|\hat{L}_{IK}||\hat{L}_{JK}^{T}| ∑ K = 1 J ∣ L ^ I K ∣∣ L ^ J K T ∣ , and ∑ K = 1 J ( ρ ( I , K ) ) ( ρ ( J , K ) ) T \sum_{K=1}^{J}(\rho^{(I,K)})(\rho^{(J,K)})^{T} ∑ K = 1 J ( ρ ( I , K ) ) ( ρ ( J , K ) ) T can both be bounded by ( ∣ a s s ∣ + ∣ e s s ∣ ) ( ∣ a t t ∣ + ∣ e t t ∣ ) \sqrt{(|a_{ss}|+|e_{ss}|)(|a_{tt}|+|e_{tt}|)} ( ∣ a ss ∣ + ∣ e ss ∣ ) ( ∣ a tt ∣ + ∣ e tt ∣ ) via Cauchy-Schwarz. Also, as done after (36), we can bound ∣ a s t ∣ ≤ ( ∣ a s s ∣ + ∣ e s s ∣ ) ( ∣ a t t ∣ + ∣ e t t ∣ ) + ∣ e s t ∣ |a_{st}|\leq\sqrt{(|a_{ss}|+|e_{ss}|)(|a_{tt}|+|e_{tt}|)}+|e_{st}| ∣ a s t ∣ ≤ ( ∣ a ss ∣ + ∣ e ss ∣ ) ( ∣ a tt ∣ + ∣ e tt ∣ ) + ∣ e s t ∣ . Hence, the element version of (39) is
∣ e s t ∣ = ∣ a s t − ∑ k = 1 t l ^ s k l ^ t k ∣ ≤ Θ ( ψ , n b ) 1 − M ϵ 64 ∑ K = 1 J ρ s ( I , K ) ρ t ( J , K ) + 2 M ϵ 64 1 − M ϵ 64 ∑ k = 1 t ∣ l ^ s k ∣ ∣ l ^ t k ∣ + M ϵ 64 ∣ a s t ∣ ≤ Θ ( ψ , n b ) + 2 M ϵ 64 1 − M ϵ 64 ( ∣ a s s ∣ + ∣ e s s ∣ ) ( ∣ a t t ∣ + ∣ e t t ∣ ) + M ϵ 64 ( ( ∣ a s s ∣ + ∣ e s s ∣ ) ( ∣ a t t ∣ + ∣ e t t ∣ ) + ∣ e s t ∣ ) . (40) \begin{aligned}
|e_{st}|&=\left|a_{st}-\sum_{k=1}^{t}\hat{l}_{sk}\hat{l}_{tk}\right|\\
&\leq\frac{\Theta(\psi,n_b)}{1-M\epsilon_{64}}\sum_{K=1}^{J}\rho_s^{(I,K)}\rho_t^{(J,K)}
+\frac{2M\epsilon_{64}}{1-M\epsilon_{64}}\sum_{k=1}^{t}|\hat{l}_{sk}||\hat{l}_{tk}|+M\epsilon_{64}|a_{st}|\\
&\leq\frac{\Theta(\psi,n_b)+2M\epsilon_{64}}{1-M\epsilon_{64}}\sqrt{(|a_{ss}|+|e_{ss}|)(|a_{tt}|+|e_{tt}|)}\\
&\quad+M\epsilon_{64}\left(\sqrt{(|a_{ss}|+|e_{ss}|)(|a_{tt}|+|e_{tt}|)}+|e_{st}|\right). \tag*{(40)}
\end{aligned} ∣ e s t ∣ = a s t − k = 1 ∑ t l ^ s k l ^ t k ≤ 1 − M ϵ 64 Θ ( ψ , n b ) K = 1 ∑ J ρ s ( I , K ) ρ t ( J , K ) + 1 − M ϵ 64 2 M ϵ 64 k = 1 ∑ t ∣ l ^ s k ∣∣ l ^ t k ∣ + M ϵ 64 ∣ a s t ∣ ≤ 1 − M ϵ 64 Θ ( ψ , n b ) + 2 M ϵ 64 ( ∣ a ss ∣ + ∣ e ss ∣ ) ( ∣ a tt ∣ + ∣ e tt ∣ ) + M ϵ 64 ( ( ∣ a ss ∣ + ∣ e ss ∣ ) ( ∣ a tt ∣ + ∣ e tt ∣ ) + ∣ e s t ∣ ) . (40)
Using (33) on rows s s s and t t t ,
( 1 − M ϵ 64 ) ∣ e s t ∣ ≤ Θ ( ψ , n b ) + 3 M ϵ 64 − M 2 ϵ 64 2 1 − M ϵ 64 ( ∣ a s s ∣ + ∣ e s s ∣ ) ( ∣ a t t ∣ + ∣ e t t ∣ ) ≤ Θ ( ψ , n b ) + 3 M ϵ 64 − M 2 ϵ 64 2 1 − Θ ( ψ , n b ) − 3 M ϵ 64 ∣ a s s ∣ ∣ a t t ∣ (1-M\epsilon_{64})|e_{st}|\leq\frac{\Theta(\psi,n_b)+3M\epsilon_{64}-M^{2}\epsilon_{64}^{2}}{1-M\epsilon_{64}}\sqrt{(|a_{ss}|+|e_{ss}|)(|a_{tt}|+|e_{tt}|)}
\leq\frac{\Theta(\psi,n_b)+3M\epsilon_{64}-M^{2}\epsilon_{64}^{2}}{1-\Theta(\psi,n_b)-3M\epsilon_{64}}\sqrt{|a_{ss}||a_{tt}|} ( 1 − M ϵ 64 ) ∣ e s t ∣ ≤ 1 − M ϵ 64 Θ ( ψ , n b ) + 3 M ϵ 64 − M 2 ϵ 64 2 ( ∣ a ss ∣ + ∣ e ss ∣ ) ( ∣ a tt ∣ + ∣ e tt ∣ ) ≤ 1 − Θ ( ψ , n b ) − 3 M ϵ 64 Θ ( ψ , n b ) + 3 M ϵ 64 − M 2 ϵ 64 2 ∣ a ss ∣∣ a tt ∣
⟹ ∣ e s t ∣ ≤ Θ ( ψ , n b ) + 3 M ϵ 64 − M 2 ϵ 64 2 1 − M ϵ 64 1 1 − Θ ( ψ , n b ) − 3 M ϵ 64 ∣ a s s ∣ ∣ a t t ∣ ≤ Θ ( ψ , n b ) + 4 M ϵ 64 1 − Θ ( ψ , n b ) − 3 M ϵ 64 ∣ a s s ∣ ∣ a t t ∣ ≤ γ e m u ( ψ , Ψ , n , n b ) 1 − γ e m u ( ψ , Ψ , n , n b ) ∣ a s s ∣ ∣ a t t ∣ . (41) \begin{aligned}
\Longrightarrow|e_{st}|\leq\frac{\Theta(\psi,n_b)+3M\epsilon_{64}-M^{2}\epsilon_{64}^{2}}{1-M\epsilon_{64}}\frac{1}{1-\Theta(\psi,n_b)-3M\epsilon_{64}}\sqrt{|a_{ss}||a_{tt}|} \\
&\leq\frac{\Theta(\psi,n_b)+4M\epsilon_{64}}{1-\Theta(\psi,n_b)-3M\epsilon_{64}}\sqrt{|a_{ss}||a_{tt}|}
\leq\frac{\gamma_{\mathrm{emu}}(\psi,\Psi,n,n_b)}{1-\gamma_{\mathrm{emu}}(\psi,\Psi,n,n_b)}\sqrt{|a_{ss}||a_{tt}|}. \tag*{(41)}
\end{aligned} ⟹ ∣ e s t ∣ ≤ 1 − M ϵ 64 Θ ( ψ , n b ) + 3 M ϵ 64 − M 2 ϵ 64 2 1 − Θ ( ψ , n b ) − 3 M ϵ 64 1 ∣ a ss ∣∣ a tt ∣ ≤ 1 − Θ ( ψ , n b ) − 3 M ϵ 64 Θ ( ψ , n b ) + 4 M ϵ 64 ∣ a ss ∣∣ a tt ∣ ≤ 1 − γ emu ( ψ , Ψ , n , n b ) γ emu ( ψ , Ψ , n , n b ) ∣ a ss ∣∣ a tt ∣ . (41)
where the last step follows from ( Θ ( ψ , n b ) + 4 M ϵ 64 ) ( 1 − M ϵ 64 ) − ( Θ ( ψ , n b ) + 3 M ϵ 64 − M 2 ϵ 64 2 ) = M ϵ 64 ( 1 − Θ ( ψ , n b ) − 3 M ϵ 64 ) ≥ 0 (\Theta(\psi,n_b)+4M\epsilon_{64})(1-M\epsilon_{64})-(\Theta(\psi,n_b)+3M\epsilon_{64}-M^{2}\epsilon_{64}^{2})=M\epsilon_{64}(1-\Theta(\psi,n_b)-3M\epsilon_{64})\geq0 ( Θ ( ψ , n b ) + 4 M ϵ 64 ) ( 1 − M ϵ 64 ) − ( Θ ( ψ , n b ) + 3 M ϵ 64 − M 2 ϵ 64 2 ) = M ϵ 64 ( 1 − Θ ( ψ , n b ) − 3 M ϵ 64 ) ≥ 0 since Θ ( ψ , n b ) + 3 M ϵ 64 ≤ Θ ( ψ , n b ) + 4 M ϵ 64 ≤ 1 \Theta(\psi,n_b)+3M\epsilon_{64}\leq\Theta(\psi,n_b)+4M\epsilon_{64}\leq1 Θ ( ψ , n b ) + 3 M ϵ 64 ≤ Θ ( ψ , n b ) + 4 M ϵ 64 ≤ 1 by assumption.
Since the bound in (41) is looser than the bounds we derived for the elements of the diagonal block in (33) and (37), it applies there too. □
Mixed-precision Floating-point case
Theorem 4.2. Suppose that the factorization in Algorithm 5 uses (potentially diagonally scaled or MX) GEMM with inputs in precision π g ( I ) \pi_g(I) π g ( I ) in iteration I I I , with machine epsilon ϵ π g ( I ) \epsilon_{\pi_g(I)} ϵ π g ( I ) and underflow threshold μ π g ( I ) \mu_{\pi_g(I)} μ π g ( I ) , accumulation in precision h h h , as well as block diagonal factorization and TRSM in precision h h h . Here π g ( I ) \pi_g(I) π g ( I ) is a lower precision, such as FP8 or FP16, and h h h is a higher precision, such as FP64 or FP32. Let ψ \psi ψ be the format that has the largest machine epsilon in all iterations, and let Ψ \Psi Ψ be the format with the largest underflow threshold in all iterations. Assume that the algorithm completes successfully and that there is no overflow in both low and high precision. If rounding blocks of L ^ \hat{L} L ^ (before GEMM) results in underflow on at most ν < n \nu< n ν < n entries of L ^ \hat{L} L ^ per row, and provided n b ϵ h < 1 n_b\epsilon_h < 1 n b ϵ h < 1 and γ f p < 1 \gamma_{\mathrm{fp}} < 1 γ fp < 1 , then A + E = L ^ L ^ T A + E = \hat{L}\hat{L}^{T} A + E = L ^ L ^ T with
∣ e s t ∣ ≤ γ f p 1 − γ f p a s s + ν μ Ψ 2 d s s 2 a t t + ν μ Ψ 2 d t t 2 , (42) \lvert e_{st}\rvert\leq\frac{\gamma_{\mathrm{fp}}}{1-\gamma_{\mathrm{fp}}}\sqrt{a_{ss}+\nu\mu_{\Psi}^{2}d_{ss}^{2}}\sqrt{a_{tt}+\nu\mu_{\Psi}^{2}d_{tt}^{2}}, \tag*{(42)} ∣ e s t ∣ ≤ 1 − γ fp γ fp a ss + ν μ Ψ 2 d ss 2 a tt + ν μ Ψ 2 d tt 2 , (42) where
γ f p = γ f p ( ϵ ψ , n , n b ) = ( 2 ϵ ψ + ϵ ψ 2 ) + 3 C , C = ϵ h max ( n b 1 − n b ϵ h , ( n b + 2 + n n b ) ( 1 + ϵ ψ ) 2 ) , (43) \gamma_{\mathrm{fp}}=\gamma_{\mathrm{fp}}(\epsilon_{\psi},n,n_b)=(2\epsilon_{\psi}+\epsilon_{\psi}^{2})+3C,\qquad C=\epsilon_h\max\left(\frac{n_b}{1-n_b\epsilon_h},\left(n_b+2+\frac{n}{n_b}\right)(1+\epsilon_{\psi})^{2}\right), \tag*{(43)} γ fp = γ fp ( ϵ ψ , n , n b ) = ( 2 ϵ ψ + ϵ ψ 2 ) + 3 C , C = ϵ h max ( 1 − n b ϵ h n b , ( n b + 2 + n b n ) ( 1 + ϵ ψ ) 2 ) , (43) n b n_b n b is the block size used for factorization, ϵ ψ \epsilon_{\psi} ϵ ψ is the machine epsilon of the format ψ \psi ψ , μ Ψ \mu_{\Psi} μ Ψ is the smallest normal number in the low precision format Ψ \Psi Ψ , and d r r d_{rr} d r r is the largest scale factor we encounter in row r r r of L ^ \hat{L} L ^ when scaling L ^ \hat{L} L ^ before rounding to low precision. Concretely, if σ i j \sigma_{ij} σ ij is the scale factor used when scaling l ^ i j \hat{l}_{ij} l ^ ij to l ^ i j / σ i j \hat{l}_{ij}/\sigma_{ij} l ^ ij / σ ij before rounding, followed by GEMM, d r r = 0 d_{rr}=0 d r r = 0 for r ≤ n b r\leq n_b r ≤ n b , and d r r = max k = 1 n b σ r k d_{rr}=\max_{k=1}^{n_b}\sigma_{rk} d r r = max k = 1 n b σ r k for n b < r ≤ n n_b<r\leq n n b < r ≤ n . Additionally, if we set the microscaling target of Algorithm 2 to β \beta β , and store the scale factor with some precision scal, then, provided γ f p ( 1 + λ Ψ ) < 1 \gamma_{\mathrm{fp}}(1+\lambda_{\Psi})<1 γ fp ( 1 + λ Ψ ) < 1 , we can also say
∣ e s t ∣ ≤ γ f p ( 1 + λ Ψ ) 1 − γ f p ( 1 + λ Ψ ) a s s a t t , (44) \lvert e_{st}\rvert\leq\frac{\gamma_{\mathrm{fp}}(1+\lambda_{\Psi})}{1-\gamma_{\mathrm{fp}}(1+\lambda_{\Psi})}\sqrt{a_{ss}a_{tt}}, \tag*{(44)} ∣ e s t ∣ ≤ 1 − γ fp ( 1 + λ Ψ ) γ fp ( 1 + λ Ψ ) a ss a tt , (44) where λ Ψ = ν μ Ψ 2 ( 1 + ϵ s c a l ) 2 β 2 \lambda_{\Psi}=\frac{\nu\mu_{\Psi}^{2}(1+\epsilon_{\mathrm{scal}})^{2}}{\beta^{2}} λ Ψ = β 2 ν μ Ψ 2 ( 1 + ϵ scal ) 2 , and ϵ s c a l \epsilon_{\mathrm{scal}} ϵ scal is the machine epsilon of the precision used to store the scale factor.
Proof. For simplicity, assume n b ∣ n n_b\mid n n b ∣ n . Following the same structure as the proof of Theorem 4.1, the roundoff error on diagonal blocks is the sum of the error from the block diagonal Cholesky n b ϵ h Ω I I n_b\epsilon_h\Omega_{II} n b ϵ h Ω I I and the error from the MX GEMM given by (23) :
∣ A I I − ∑ K = 1 I L ^ I K L ^ I K T ∣ ≤ n b ϵ h Ω I I + ( n b + 2 + n n b ) ϵ h ∣ A I I ∣ + ∑ K = 1 I − 1 [ ϵ π g ( K ) ( ∣ L ^ I K ′ ′ ∣ ∣ L ^ I K T ∣ + ∣ L ^ I K ∣ ∣ L ^ I K ′ ′ T ∣ + ϵ π g ( K ) ∣ L ^ I K ′ ′ ∣ ∣ L ^ I K ′ ′ T ∣ ) |A_{II}-\sum_{K=1}^{I}\hat{L}_{IK}\hat{L}_{IK}^{T}|
\leq n_b\epsilon_h\Omega_{II}+(n_b+2+\frac{n}{n_b})\epsilon_h|A_{II}|+\sum_{K=1}^{I-1}[\epsilon_{\pi_g(K)}(|\hat{L}_{IK}^{\prime\prime}||\hat{L}_{IK}^{T}|+|\hat{L}_{IK}||\hat{L}_{IK}^{\prime\prime T}|+\epsilon_{\pi_g(K)}|\hat{L}_{IK}^{\prime\prime}||\hat{L}_{IK}^{\prime\prime T}|) ∣ A I I − K = 1 ∑ I L ^ I K L ^ I K T ∣ ≤ n b ϵ h Ω I I + ( n b + 2 + n b n ) ϵ h ∣ A I I ∣ + K = 1 ∑ I − 1 [ ϵ π g ( K ) ( ∣ L ^ I K ′′ ∣∣ L ^ I K T ∣ + ∣ L ^ I K ∣∣ L ^ I K ′′ T ∣ + ϵ π g ( K ) ∣ L ^ I K ′′ ∣∣ L ^ I K ′′ T ∣ )
+ ( n b + 2 + n n b ) ϵ h ( ∣ L ^ I K ∣ + ϵ π g ( K ) ∣ L ^ I K ′ ′ ∣ ) ( ∣ L ^ I K ∣ + ϵ π g ( K ) ∣ L ^ I K ′ ′ ∣ ) T ] ≤ n b ϵ h Ω I I + ( n b + 2 + n n b ) ϵ h ∣ A I I ∣ + ∑ K = 1 I − 1 [ ϵ ψ ( ∣ L ^ I K ′ ′ ∣ ∣ L ^ I K T ∣ + ∣ L ^ I K ∣ ∣ L ^ I K ′ ′ T ∣ + ϵ ψ ∣ L ^ I K ′ ′ ∣ ∣ L ^ I K ′ ′ T ∣ ) (45) \begin{aligned}
\qquad+(n_b+2+\frac{n}{n_b})\epsilon_h(|\hat{L}_{IK}|+\epsilon_{\pi_g(K)}|\hat{L}_{IK}^{\prime\prime}|)(|\hat{L}_{IK}|+\epsilon_{\pi_g(K)}|\hat{L}_{IK}^{\prime\prime}|)^{T}] \\
&\leq n_b\epsilon_h\Omega_{II}+(n_b+2+\frac{n}{n_b})\epsilon_h|A_{II}|+\sum_{K=1}^{I-1}[\epsilon_{\psi}(|\hat{L}_{IK}^{\prime\prime}||\hat{L}_{IK}^{T}|+|\hat{L}_{IK}||\hat{L}_{IK}^{\prime\prime T}|+\epsilon_{\psi}|\hat{L}_{IK}^{\prime\prime}||\hat{L}_{IK}^{\prime\prime T}|) \tag*{(45)}
\end{aligned} + ( n b + 2 + n b n ) ϵ h ( ∣ L ^ I K ∣ + ϵ π g ( K ) ∣ L ^ I K ′′ ∣ ) ( ∣ L ^ I K ∣ + ϵ π g ( K ) ∣ L ^ I K ′′ ∣ ) T ] ≤ n b ϵ h Ω I I + ( n b + 2 + n b n ) ϵ h ∣ A I I ∣ + K = 1 ∑ I − 1 [ ϵ ψ ( ∣ L ^ I K ′′ ∣∣ L ^ I K T ∣ + ∣ L ^ I K ∣∣ L ^ I K ′′ T ∣ + ϵ ψ ∣ L ^ I K ′′ ∣∣ L ^ I K ′′ T ∣ ) (45)
+ ( n b + 2 + n n b ) ϵ h ( ∣ L ^ I K ∣ + ϵ ψ ∣ L ^ I K ′ ′ ∣ ) ( ∣ L ^ I K ∣ + ϵ ψ ∣ L ^ I K ′ ′ ∣ ) T ] . \qquad+(n_b+2+\frac{n}{n_b})\epsilon_h(|\hat{L}_{IK}|+\epsilon_{\psi}|\hat{L}_{IK}^{\prime\prime}|)(|\hat{L}_{IK}|+\epsilon_{\psi}|\hat{L}_{IK}^{\prime\prime}|)^{T}]. + ( n b + 2 + n b n ) ϵ h ( ∣ L ^ I K ∣ + ϵ ψ ∣ L ^ I K ′′ ∣ ) ( ∣ L ^ I K ∣ + ϵ ψ ∣ L ^ I K ′′ ∣ ) T ] .
We have applied Theorem 3.6 with A = L ^ I K A=\hat{L}_{IK} A = L ^ I K , B = L ^ I K T B=\hat{L}_{IK}^{T} B = L ^ I K T and ϕ ≈ ϵ π g ( K ) \phi\approx\epsilon_{\pi_g(K)} ϕ ≈ ϵ π g ( K ) as in (27). Also, ∣ L ^ I K ′ ′ ∣ = D I K M I K + ∣ L ^ I K ′ ∣ |\hat{L}_{IK}^{\prime\prime}|=D_{IK}M_{IK}+|\hat{L}_{IK}^{\prime}| ∣ L ^ I K ′′ ∣ = D I K M I K + ∣ L ^ I K ′ ∣ with D I K D_{IK} D I K a diagonal matrix holding the largest scale factor of each row, L ^ I K ′ = L ^ I K \hat{L}_{IK}^{\prime}=\hat{L}_{IK} L ^ I K ′ = L ^ I K in positions where L ^ I K \hat{L}_{IK} L ^ I K doesn’t underflow when rounding to low precision, and zero everywhere else, and M I K M_{IK} M I K a matrix with μ π g ( K ) \mu_{\pi_g(K)} μ π g ( K ) in positions where there is underflow, and zero everywhere else (same as Lemma 3.3). Note that (45) has accumulation error ( n b + 2 + n n b ) ϵ h ( ∣ A I I ∣ + ∑ K = 1 I − 1 ( ∣ L ^ I K ∣ + ϵ ψ ∣ L ^ I K ′ ′ ∣ ) ( ∣ L ^ I K ∣ + ϵ ψ ∣ L ^ I K ′ ′ ∣ ) T ) (n_b+2+\frac{n}{n_b})\epsilon_h(|A_{II}|+\sum_{K=1}^{I-1}(|\hat{L}_{IK}|+\epsilon_{\psi}|\hat{L}_{IK}^{\prime\prime}|)(|\hat{L}_{IK}|+\epsilon_{\psi}|\hat{L}_{IK}^{\prime\prime}|)^{T}) ( n b + 2 + n b n ) ϵ h ( ∣ A I I ∣ + ∑ K = 1 I − 1 ( ∣ L ^ I K ∣ + ϵ ψ ∣ L ^ I K ′′ ∣ ) ( ∣ L ^ I K ∣ + ϵ ψ ∣ L ^ I K ′′ ∣ ) T ) , which is slightly different from the accumulation error in (23). To understand why the accumulation error has the coefficient ( n b + 2 + n n b ) ϵ h (n_b+2+\frac{n}{n_b})\epsilon_h ( n b + 2 + n b n ) ϵ h and the extra ∣ A I I ∣ |A_{II}| ∣ A I I ∣ term, we can count the roundoff terms the same way we did for Lemmas 3.4 and 3.5. Consider an element of L ^ I K \hat{L}_{IK} L ^ I K . When we round L ^ I K \hat{L}_{IK} L ^ I K to low precision and perform MXGEMM L ^ I K L ^ I K T \hat{L}_{IK}\hat{L}_{IK}^{T} L ^ I K L ^ I K T , each element in that product receives at most n b + 2 n_b+2 n b + 2 roundoff terms according to Theorem 3.6. Two terms come from the application of scale factors, one from the multiplication, and the remaining n b − 1 n_b-1 n b − 1 terms from the sum reduction. Then, as in the proof of Theorem 4.1, Algorithm 5 subtracts the I − 1 I-1 I − 1 such products L ^ I K L ^ I K T \hat{L}_{IK}\hat{L}_{IK}^{T} L ^ I K L ^ I K T from A I I A_{II} A I I , which adds at most I − 1 ≤ n n b I-1\leq\frac{n}{n_b} I − 1 ≤ n b n roundoff terms. This gives a total of at most n b + 2 + n n b n_b+2+\frac{n}{n_b} n b + 2 + n b n roundoff terms on all elements of L ^ I K \hat{L}_{IK} L ^ I K and A I I A_{II} A I I . Moving forward, assuming ( s , t ) (s,t) ( s , t ) falls in block ( I , J ) (I,J) ( I , J ) with J ≤ I J\leq I J ≤ I , we will refer to the entry of ϵ ψ ∑ K = 1 J − 1 ( ∣ L ^ I K ′ ′ ∣ ∣ L ^ J K T ∣ + ∣ L ^ I K ∣ ∣ L ^ J K ′ ′ T ∣ + ϵ ψ ∣ L ^ I K ′ ′ ∣ ∣ L ^ J K ′ ′ T ∣ ) \epsilon_{\psi}\sum_{K=1}^{J-1}\left(\lvert\hat{L}_{IK}^{\prime\prime}\rvert\lvert\hat{L}_{JK}^{T}\rvert+\lvert\hat{L}_{IK}\rvert\lvert\hat{L}_{JK}^{\prime\prime T}\rvert+\epsilon_{\psi}\lvert\hat{L}_{IK}^{\prime\prime}\rvert\lvert\hat{L}_{JK}^{\prime\prime T}\rvert\right) ϵ ψ ∑ K = 1 J − 1 ( ∣ L ^ I K ′′ ∣ ∣ L ^ J K T ∣ + ∣ L ^ I K ∣ ∣ L ^ J K ′′ T ∣ + ϵ ψ ∣ L ^ I K ′′ ∣ ∣ L ^ J K ′′ T ∣ ) in position ( s , t ) (s,t) ( s , t ) as ζ s t t r u n c \zeta_{st}^{\mathrm{trunc}} ζ s t trunc , and the corresponding entry of ( n b + 2 + n n b ) ϵ h ∑ K = 1 J − 1 ( ∣ L ^ I K ∣ + ϵ ψ ∣ L ^ I K ′ ′ ∣ ) ( ∣ L ^ J K ∣ + ϵ ψ ∣ L ^ J K ′ ′ ∣ ) T \left(n_b+2+\frac{n}{n_b}\right)\epsilon_h\sum_{K=1}^{J-1}\left(\lvert\hat{L}_{IK}\rvert+\epsilon_{\psi}\lvert\hat{L}_{IK}^{\prime\prime}\rvert\right)\left(\lvert\hat{L}_{JK}\rvert+\epsilon_{\psi}\lvert\hat{L}_{JK}^{\prime\prime}\rvert\right)^T ( n b + 2 + n b n ) ϵ h ∑ K = 1 J − 1 ( ∣ L ^ I K ∣ + ϵ ψ ∣ L ^ I K ′′ ∣ ) ( ∣ L ^ J K ∣ + ϵ ψ ∣ L ^ J K ′′ ∣ ) T as ζ s t L -accum \zeta_{st}^{L\text{-accum}} ζ s t L -accum . Note that we do not include ( n b + 2 + n n b ) ϵ h ∣ A I I ∣ \left(n_b+2+\frac{n}{n_b}\right)\epsilon_h\lvert A_{II}\rvert ( n b + 2 + n b n ) ϵ h ∣ A I I ∣ in ζ s t L -accum \zeta_{st}^{L\text{-accum}} ζ s t L -accum . Now, for the proof of the Cholesky error bound, we will use the same strategy as the proof of Theorem 4.1: we first find a bound on the diagonal elements of the diagonal blocks, then the off-diagonal elements of the diagonal blocks, then all elements in off-diagonal blocks. Defining L ^ I K ′ ′ = 0 \hat{L}_{IK}^{\prime\prime}=0 L ^ I K ′′ = 0 for K ≥ I K\geq I K ≥ I (since we don’t use the corresponding blocks of L L L for the MX GEMMs), we can write L ^ ′ ′ \hat{L}^{\prime\prime} L ^ ′′ for the full-matrix assembly of all L ^ I K ′ ′ \hat{L}_{IK}^{\prime\prime} L ^ I K ′′ , so that l ^ t k \hat{l}_{tk} l ^ t k is defined for every 1 ≤ k , t ≤ n 1\leq k,t\leq n 1 ≤ k , t ≤ n . Also, as in the statement of the Theorem, let d t t d_{tt} d tt be the maximum scale factor in all blocks that contain the row t t t . If row t t t falls into block row I I I , d t t = max K = 1 I − 1 ( D I K ) t − ( I − 1 ) n b , t − ( I − 1 ) n b d_{tt}=\max_{K=1}^{I-1}(D_{IK})_{t-(I-1)n_b,t-(I-1)n_b} d tt = max K = 1 I − 1 ( D I K ) t − ( I − 1 ) n b , t − ( I − 1 ) n b . As in the proof of Theorem 4.1, Ω I I \Omega_{II} Ω I I is the block diagonal Cholesky bound of [8 ] , with entries ω s t = ∣ a ^ s s u ∣ ∣ a ^ t t u ∣ \omega_{st}=\sqrt{\lvert\hat{a}_{ss}^{u}\rvert\lvert\hat{a}_{tt}^{u}\rvert} ω s t = ∣ a ^ ss u ∣ ∣ a ^ tt u ∣ , and A ^ I I u \hat{A}_{II}^{u} A ^ I I u denotes the computed Schur-updated block.
Now zoom in on the diagonal entry ( t , t ) (t,t) ( t , t ) . The diagonal element version of (45) is
∣ e t t ∣ ≤ n b ϵ h ω t t + ( n b + 2 + n n b ) ϵ h ∣ a t t ∣ + ζ t t t r u n c + ζ t t L -accum . (46) \lvert e_{tt}\rvert\leq n_b\epsilon_h\omega_{tt}+\left(n_b+2+\frac{n}{n_b}\right)\epsilon_h\lvert a_{tt}\rvert+\zeta_{tt}^{\mathrm{trunc}}+\zeta_{tt}^{L\text{-accum}}. \tag*{(46)} ∣ e tt ∣ ≤ n b ϵ h ω tt + ( n b + 2 + n b n ) ϵ h ∣ a tt ∣ + ζ tt trunc + ζ tt L -accum . (46)
We already saw in the proof of Theorem 4.1 that
ϵ h n b ω t t = ϵ h n b ∣ a ^ t t u ∣ ≤ ϵ h n b 1 − n b ϵ h ∑ k = ( I − 1 ) n b + 1 I n b l ^ t k 2 . (47) \epsilon_h n_b\omega_{tt}=\epsilon_h n_b\lvert\hat{a}_{tt}^{u}\rvert\leq\frac{\epsilon_h n_b}{1-n_b\epsilon_h}\sum_{k=(I-1)n_b+1}^{In_b}\hat{l}_{tk}^{2}. \tag*{(47)} ϵ h n b ω tt = ϵ h n b ∣ a ^ tt u ∣ ≤ 1 − n b ϵ h ϵ h n b k = ( I − 1 ) n b + 1 ∑ I n b l ^ t k 2 . (47)
We will now focus on ζ t t t r u n c \zeta_{tt}^{\mathrm{trunc}} ζ tt trunc . For ϵ ψ ∑ K = 1 I − 1 ∣ L ^ I K ′ ′ ∣ ∣ L ^ I K T ∣ \epsilon_{\psi}\sum_{K=1}^{I-1}\lvert\hat{L}_{IK}^{\prime\prime}\rvert\lvert\hat{L}_{IK}^{T}\rvert ϵ ψ ∑ K = 1 I − 1 ∣ L ^ I K ′′ ∣ ∣ L ^ I K T ∣ , first define U t ⊆ { 1 , … , ( I − 1 ) n b } U_t\subseteq\{1,\ldots,(I-1)n_b\} U t ⊆ { 1 , … , ( I − 1 ) n b } as the set of k k k for which l ^ t k \hat{l}_{tk} l ^ t k underflows when rounded to low precision. Now, for k ∉ U t k\notin U_t k ∈ / U t , we have ∣ l ^ t k ′ ′ ∣ = ∣ l ^ t k ∣ \lvert\hat{l}_{tk}^{\prime\prime}\rvert=\lvert\hat{l}_{tk}\rvert ∣ l ^ t k ′′ ∣ = ∣ l ^ t k ∣ , and for k ∈ U t k\in U_t k ∈ U t , ∣ l t k ∣ ≤ ( 1 + ϵ h ) d t t μ Ψ \lvert l_{tk}\rvert\leq(1+\epsilon_h)d_{tt}\mu_{\Psi} ∣ l t k ∣ ≤ ( 1 + ϵ h ) d tt μ Ψ and ∣ l ^ t k ∣ ≤ d t t μ Ψ \lvert\hat{l}_{tk}\rvert\leq d_{tt}\mu_{\Psi} ∣ l ^ t k ∣ ≤ d tt μ Ψ . The rest of the l ^ t k ′ ′ \hat{l}_{tk}^{\prime\prime} l ^ t k ′′ with k > ( I − 1 ) n b k>(I-1)n_b k > ( I − 1 ) n b are all 0. Element ( t , t ) (t,t) ( t , t ) of ϵ ψ ∑ K = 1 I − 1 ∣ L ^ I K ′ ′ ∣ ∣ L ^ I K T ∣ \epsilon_{\psi}\sum_{K=1}^{I-1}\lvert\hat{L}_{IK}^{\prime\prime}\rvert\lvert\hat{L}_{IK}^{T}\rvert ϵ ψ ∑ K = 1 I − 1 ∣ L ^ I K ′′ ∣ ∣ L ^ I K T ∣ is ϵ ψ ∑ k = 1 ( I − 1 ) n b ∣ l ^ t k ′ ′ ∣ ∣ l ^ t k ∣ \epsilon_{\psi}\sum_{k=1}^{(I-1)n_b}\lvert\hat{l}_{tk}^{\prime\prime}\rvert\lvert\hat{l}_{tk}\rvert ϵ ψ ∑ k = 1 ( I − 1 ) n b ∣ l ^ t k ′′ ∣ ∣ l ^ t k ∣ . We will try to bound this by applying Cauchy-Schwarz, for which we need a bound on ∑ k = 1 ( I − 1 ) n b ∣ l ^ t k ′ ′ ∣ 2 \sum_{k=1}^{(I-1)n_b}\lvert\hat{l}_{tk}^{\prime\prime}\rvert^2 ∑ k = 1 ( I − 1 ) n b ∣ l ^ t k ′′ ∣ 2
∑ k = 1 ( I − 1 ) n b ∣ l ^ t k ′ ′ ∣ 2 = ∑ k ∈ U t ∣ l ^ t k ′ ′ ∣ 2 + ∑ k ∉ U t ∣ l ^ t k ′ ′ ∣ 2 ≤ ν μ Ψ 2 d t t 2 + ∑ k = 1 ( I − 1 ) n b ∣ l ^ t k ∣ 2 . (48) \sum_{k=1}^{(I-1)n_b}\lvert\hat{l}_{tk}^{\prime\prime}\rvert^2=\sum_{k\in U_t}\lvert\hat{l}_{tk}^{\prime\prime}\rvert^2+\sum_{k\notin U_t}\lvert\hat{l}_{tk}^{\prime\prime}\rvert^2\leq\nu\mu_{\Psi}^{2}d_{tt}^{2}+\sum_{k=1}^{(I-1)n_b}\lvert\hat{l}_{tk}\rvert^2. \tag*{(48)} k = 1 ∑ ( I − 1 ) n b ∣ l ^ t k ′′ ∣ 2 = k ∈ U t ∑ ∣ l ^ t k ′′ ∣ 2 + k ∈ / U t ∑ ∣ l ^ t k ′′ ∣ 2 ≤ ν μ Ψ 2 d tt 2 + k = 1 ∑ ( I − 1 ) n b ∣ l ^ t k ∣ 2 . (48)
Then
ϵ ψ ∑ k = 1 ( I − 1 ) n b ∣ l ^ t k ′ ′ ∣ ∣ l ^ t k ∣ ≤ ϵ ψ ( ∑ k = 1 ( I − 1 ) n b ∣ l ^ t k ′ ′ ∣ 2 ) ( ∑ k = 1 ( I − 1 ) n b l ^ t k 2 ) ≤ ϵ ψ ( ∑ k = 1 ( I − 1 ) n b l ^ t k 2 + ν μ Ψ 2 d t t 2 ) . (49) \epsilon_{\psi}\sum_{k=1}^{(I-1)n_b}\lvert\hat{l}_{tk}^{\prime\prime}\rvert\lvert\hat{l}_{tk}\rvert\leq\epsilon_{\psi}\sqrt{\left(\sum_{k=1}^{(I-1)n_b}\lvert\hat{l}_{tk}^{\prime\prime}\rvert^2\right)\left(\sum_{k=1}^{(I-1)n_b}\hat{l}_{tk}^2\right)}\leq\epsilon_{\psi}\left(\sum_{k=1}^{(I-1)n_b}\hat{l}_{tk}^2+\nu\mu_{\Psi}^{2}d_{tt}^{2}\right). \tag*{(49)} ϵ ψ k = 1 ∑ ( I − 1 ) n b ∣ l ^ t k ′′ ∣ ∣ l ^ t k ∣ ≤ ϵ ψ k = 1 ∑ ( I − 1 ) n b ∣ l ^ t k ′′ ∣ 2 k = 1 ∑ ( I − 1 ) n b l ^ t k 2 ≤ ϵ ψ k = 1 ∑ ( I − 1 ) n b l ^ t k 2 + ν μ Ψ 2 d tt 2 . (49)
The next matrix term in ζ t t t r u n c \zeta_{tt}^{\mathrm{trunc}} ζ tt trunc is ϵ ψ ∑ K = 1 I − 1 ∣ L ^ I K ∣ ∣ L ^ I K ′ ′ T ∣ \epsilon_{\psi}\sum_{K=1}^{I-1}\lvert\hat{L}_{IK}\rvert\lvert\hat{L}_{IK}^{\prime\prime T}\rvert ϵ ψ ∑ K = 1 I − 1 ∣ L ^ I K ∣ ∣ L ^ I K ′′ T ∣ , which is the transpose of ϵ ψ ∑ K = 1 I − 1 ∣ L ^ I K ′ ′ ∣ ∣ L ^ I K T ∣ \epsilon_{\psi}\sum_{K=1}^{I-1}\lvert\hat{L}_{IK}^{\prime\prime}\rvert\lvert\hat{L}_{IK}^{T}\rvert ϵ ψ ∑ K = 1 I − 1 ∣ L ^ I K ′′ ∣ ∣ L ^ I K T ∣ . So, (49) works as an upper bound for the diagonal element of ϵ ψ ∑ K = 1 I − 1 ∣ L ^ I K ∣ ∣ L ^ I K ′ ′ T ∣ \epsilon_{\psi}\sum_{K=1}^{I-1}\lvert\hat{L}_{IK}\rvert\lvert\hat{L}_{IK}^{\prime\prime T}\rvert ϵ ψ ∑ K = 1 I − 1 ∣ L ^ I K ∣ ∣ L ^ I K ′′ T ∣ as well. For ϵ ψ 2 ∑ K = 1 I − 1 ∣ L ^ I K ′ ′ ∣ ∣ L ^ I K ′ ′ T ∣ \epsilon_{\psi}^{2}\sum_{K=1}^{I-1}\lvert\hat{L}_{IK}^{\prime\prime}\rvert\lvert\hat{L}_{IK}^{\prime\prime T}\rvert ϵ ψ 2 ∑ K = 1 I − 1 ∣ L ^ I K ′′ ∣ ∣ L ^ I K ′′ T ∣ , the diagonal term is ϵ ψ 2 ∑ k = 1 ( I − 1 ) n b ∣ l ^ t k ′ ′ ∣ 2 \epsilon_{\psi}^{2}\sum_{k=1}^{(I-1)n_b}\lvert\hat{l}_{tk}^{\prime\prime}\rvert^{2} ϵ ψ 2 ∑ k = 1 ( I − 1 ) n b ∣ l ^ t k ′′ ∣ 2 , which we can get an upper bound on by multiplying (48) by ϵ ψ 2 \epsilon_{\psi}^{2} ϵ ψ 2 . So
ζ t t t r u n c ≤ ( 2 ϵ ψ + ϵ ψ 2 ) ( ∑ k = 1 ( I − 1 ) n b l ^ t k 2 + ν μ Ψ 2 d t t 2 ) . (50) \zeta_{tt}^{\mathrm{trunc}}\leq(2\epsilon_{\psi}+\epsilon_{\psi}^{2})\left(\sum_{k=1}^{(I-1)n_b}\hat{l}_{tk}^{2}+\nu\mu_{\Psi}^{2}d_{tt}^{2}\right). \tag*{(50)} ζ tt trunc ≤ ( 2 ϵ ψ + ϵ ψ 2 ) k = 1 ∑ ( I − 1 ) n b l ^ t k 2 + ν μ Ψ 2 d tt 2 . (50)
To get a bound on ζ t t L a c c u m \zeta^{L_{\mathrm{accum}}}_{tt} ζ tt L accum , observe the accumulation error matrices in (45)
( n b + 2 + n n b ) ϵ h ( ∣ L ^ I K ∣ + ϵ ψ ∣ L ^ I K ′ ′ ∣ ) ( ∣ L ^ I K ∣ + ϵ ψ ∣ L ^ I K ′ ′ ∣ ) T = ( n b + 2 + n n b ) ϵ h ( ∣ L ^ I K ∣ ∣ L ^ I K ∣ T + ϵ ψ ∣ L ^ I K ∣ ∣ L ^ I K ′ ′ ∣ T + ϵ ψ ∣ L ^ I K ′ ′ ∣ ∣ L ^ I K ∣ T + ϵ ψ 2 ∣ L ^ I K ′ ′ ∣ ∣ L ^ I K ′ ′ ∣ T ) . (51) \begin{aligned}
\left(n_b+2+\frac{n}{n_b}\right)\epsilon_h\left(\lvert\widehat{L}_{IK}\rvert+\epsilon_\psi\lvert\widehat{L}^{\prime\prime}_{IK}\rvert\right)\left(\lvert\widehat{L}_{IK}\rvert+\epsilon_\psi\lvert\widehat{L}^{\prime\prime}_{IK}\rvert\right)^T \\
&=\left(n_b+2+\frac{n}{n_b}\right)\epsilon_h\left(\lvert\widehat{L}_{IK}\rvert\lvert\widehat{L}_{IK}\rvert^T+\epsilon_\psi\lvert\widehat{L}_{IK}\rvert\lvert\widehat{L}^{\prime\prime}_{IK}\rvert^T+\epsilon_\psi\lvert\widehat{L}^{\prime\prime}_{IK}\rvert\lvert\widehat{L}_{IK}\rvert^T+\epsilon_\psi^2\lvert\widehat{L}^{\prime\prime}_{IK}\rvert\lvert\widehat{L}^{\prime\prime}_{IK}\rvert^T\right). \tag*{(51)}
\end{aligned} ( n b + 2 + n b n ) ϵ h ( ∣ L I K ∣ + ϵ ψ ∣ L I K ′′ ∣ ) ( ∣ L I K ∣ + ϵ ψ ∣ L I K ′′ ∣ ) T = ( n b + 2 + n b n ) ϵ h ( ∣ L I K ∣ ∣ L I K ∣ T + ϵ ψ ∣ L I K ∣ ∣ L I K ′′ ∣ T + ϵ ψ ∣ L I K ′′ ∣ ∣ L I K ∣ T + ϵ ψ 2 ∣ L I K ′′ ∣ ∣ L I K ′′ ∣ T ) . (51)
On the right hand side of (51) , (49) and (48) already bound the diagonal elements of every term except ∣ L ^ I K ∣ ∣ L ^ I K ∣ T \lvert\widehat{L}_{IK}\rvert\lvert\widehat{L}_{IK}\rvert^T ∣ L I K ∣ ∣ L I K ∣ T , which has diagonal element ∑ k = 1 ( I − 1 ) n b l ^ t k 2 \sum_{k=1}^{(I-1)n_b}\widehat{l}_{tk}^2 ∑ k = 1 ( I − 1 ) n b l t k 2 . So, the diagonal element of the accumulation error matrices can be bounded by
ζ t t L a c c u m ≤ ( n b + 2 + n n b ) ϵ h ( ∑ k = 1 ( I − 1 ) n b l ^ t k 2 + ( 2 ϵ ψ + ϵ ψ 2 ) ( ∑ k = 1 ( I − 1 ) n b l ^ t k 2 + ν μ ψ 2 d t t 2 ) ) ≤ ( n b + 2 + n n b ) ϵ h ( 1 + ϵ ψ ) 2 ( ∑ k = 1 ( I − 1 ) n b l ^ t k 2 + ν μ ψ 2 d t t 2 ) . (52) \begin{aligned}
\zeta^{L_{\mathrm{accum}}}_{tt}\leq\left(n_b+2+\frac{n}{n_b}\right)\epsilon_h\left(\sum_{k=1}^{(I-1)n_b}\widehat{l}_{tk}^2+\left(2\epsilon_\psi+\epsilon_\psi^2\right)\left(\sum_{k=1}^{(I-1)n_b}\widehat{l}_{tk}^2+\nu\mu_\psi^2d_{tt}^2\right)\right) \\
&\leq\left(n_b+2+\frac{n}{n_b}\right)\epsilon_h(1+\epsilon_\psi)^2\left(\sum_{k=1}^{(I-1)n_b}\widehat{l}_{tk}^2+\nu\mu_\psi^2d_{tt}^2\right). \tag*{(52)}
\end{aligned} ζ tt L accum ≤ ( n b + 2 + n b n ) ϵ h k = 1 ∑ ( I − 1 ) n b l t k 2 + ( 2 ϵ ψ + ϵ ψ 2 ) k = 1 ∑ ( I − 1 ) n b l t k 2 + ν μ ψ 2 d tt 2 ≤ ( n b + 2 + n b n ) ϵ h ( 1 + ϵ ψ ) 2 k = 1 ∑ ( I − 1 ) n b l t k 2 + ν μ ψ 2 d tt 2 . (52)
Now, we can rewrite (46) using (52) , (50) and (47) as
∣ e t t ∣ ≤ [ ( 2 ϵ ψ + ϵ ψ 2 ) + ( n b + 2 + n n b ) ϵ h ( 1 + ϵ ψ ) 2 ] ( ∑ k = ( I − 1 ) n b + 1 t l ^ t k 2 + ν μ ψ 2 d t t 2 ) + n b ϵ h 1 − n b ϵ h ∑ k = ( I − 1 ) n b + 1 t l ^ t k 2 + ( n b + 2 + n n b ) ϵ h ∣ a t t ∣ . ⇒ ∣ e t t ∣ ≤ [ ( 2 ϵ ψ + ϵ ψ 2 ) + max { ( n b + 2 + n n b ) ϵ h ( 1 + ϵ ψ ) 2 , n b ϵ h 1 − n b ϵ h } ] ( ∣ a t t ∣ + ∣ e t t ∣ + ν μ ψ 2 d t t 2 ) + ( n b + 2 + n n b ) ϵ h ∣ a t t ∣ ≤ [ ( 2 ϵ ψ + ϵ ψ 2 ) + C ] ( ∣ a t t ∣ + ∣ e t t ∣ + ν μ ψ 2 d t t 2 ) + C ∣ a t t ∣ ≤ γ f p ( ∣ a t t ∣ + ∣ e t t ∣ + ν μ ψ 2 d t t 2 ) , ⇒ ∣ e t t ∣ ≤ γ f p 1 − γ f p ( ∣ a t t ∣ + ν μ ψ 2 d t t 2 ) . (53) \begin{aligned}
\lvert e_{tt}\rvert\leq\left[(2\epsilon_\psi+\epsilon_\psi^2)+\left(n_b+2+\frac{n}{n_b}\right)\epsilon_h(1+\epsilon_\psi)^2\right]\left(\sum_{k=(I-1)n_b+1}^{t}\widehat{l}_{tk}^2+\nu\mu_\psi^2d_{tt}^2\right)+\frac{n_b\epsilon_h}{1-n_b\epsilon_h}\sum_{k=(I-1)n_b+1}^{t}\widehat{l}_{tk}^2+\left(n_b+2+\frac{n}{n_b}\right)\epsilon_h\lvert a_{tt}\rvert. \\
&\Rightarrow\lvert e_{tt}\rvert\leq\left[(2\epsilon_\psi+\epsilon_\psi^2)+\max\left\{\left(n_b+2+\frac{n}{n_b}\right)\epsilon_h(1+\epsilon_\psi)^2,\frac{n_b\epsilon_h}{1-n_b\epsilon_h}\right\}\right]\left(\lvert a_{tt}\rvert+\lvert e_{tt}\rvert+\nu\mu_\psi^2d_{tt}^2\right)+\left(n_b+2+\frac{n}{n_b}\right)\epsilon_h\lvert a_{tt}\rvert\\
&\leq\left[(2\epsilon_\psi+\epsilon_\psi^2)+C\right]\left(\lvert a_{tt}\rvert+\lvert e_{tt}\rvert+\nu\mu_\psi^2d_{tt}^2\right)+C\lvert a_{tt}\rvert\leq\gamma_{\mathrm{fp}}\left(\lvert a_{tt}\rvert+\lvert e_{tt}\rvert+\nu\mu_\psi^2d_{tt}^2\right), \\
&\Rightarrow\lvert e_{tt}\rvert\leq\frac{\gamma_{\mathrm{fp}}}{1-\gamma_{\mathrm{fp}}}\left(\lvert a_{tt}\rvert+\nu\mu_\psi^2d_{tt}^2\right). \tag*{(53)}
\end{aligned} ∣ e tt ∣ ≤ [ ( 2 ϵ ψ + ϵ ψ 2 ) + ( n b + 2 + n b n ) ϵ h ( 1 + ϵ ψ ) 2 ] k = ( I − 1 ) n b + 1 ∑ t l t k 2 + ν μ ψ 2 d tt 2 + 1 − n b ϵ h n b ϵ h k = ( I − 1 ) n b + 1 ∑ t l t k 2 + ( n b + 2 + n b n ) ϵ h ∣ a tt ∣ . ⇒ ∣ e tt ∣ ≤ [ ( 2 ϵ ψ + ϵ ψ 2 ) + max { ( n b + 2 + n b n ) ϵ h ( 1 + ϵ ψ ) 2 , 1 − n b ϵ h n b ϵ h } ] ( ∣ a tt ∣ + ∣ e tt ∣ + ν μ ψ 2 d tt 2 ) + ( n b + 2 + n b n ) ϵ h ∣ a tt ∣ ≤ [ ( 2 ϵ ψ + ϵ ψ 2 ) + C ] ( ∣ a tt ∣ + ∣ e tt ∣ + ν μ ψ 2 d tt 2 ) + C ∣ a tt ∣ ≤ γ fp ( ∣ a tt ∣ + ∣ e tt ∣ + ν μ ψ 2 d tt 2 ) , ⇒ ∣ e tt ∣ ≤ 1 − γ fp γ fp ( ∣ a tt ∣ + ν μ ψ 2 d tt 2 ) . (53)
The same inequalities hold for all the indices from 1 to n n n .
For the off-diagonal elements of the diagonal block, pick some 1 ≤ s < t 1\leq s<t 1 ≤ s < t . The inequality we need to analyze is
∣ e s t ∣ ≤ n b ϵ h ω s t + ( n b + 2 + n n b ) ϵ h ∣ a s t ∣ + ζ s t t r u n c + ζ s t L a c c u m . (54) \lvert e_{st}\rvert\leq n_b\epsilon_h\omega_{st}+\left(n_b+2+\frac{n}{n_b}\right)\epsilon_h\lvert a_{st}\rvert+\zeta^{\mathrm{trunc}}_{st}+\zeta^{L_{\mathrm{accum}}}_{st}. \tag*{(54)} ∣ e s t ∣ ≤ n b ϵ h ω s t + ( n b + 2 + n b n ) ϵ h ∣ a s t ∣ + ζ s t trunc + ζ s t L accum . (54)
Starting with ω s t \omega_{st} ω s t ,
n b ϵ h ω s t = n b ϵ h ∣ a ^ s s ∣ ∣ a ^ t t ∣ ≤ n b ϵ h 1 − n b ϵ h ( ∑ k = ( I − 1 ) n b + 1 s l ^ s k 2 ) ( ∑ k = ( I − 1 ) n b + 1 t l ^ t k 2 ) . (55) n_b\epsilon_h\omega_{st}=n_b\epsilon_h\sqrt{\lvert\widehat{a}_{ss}\rvert\lvert\widehat{a}_{tt}\rvert}\leq\frac{n_b\epsilon_h}{1-n_b\epsilon_h}\sqrt{\left(\sum_{k=(I-1)n_b+1}^{s}\widehat{l}_{sk}^2\right)\left(\sum_{k=(I-1)n_b+1}^{t}\widehat{l}_{tk}^2\right)}. \tag*{(55)} n b ϵ h ω s t = n b ϵ h ∣ a ss ∣ ∣ a tt ∣ ≤ 1 − n b ϵ h n b ϵ h k = ( I − 1 ) n b + 1 ∑ s l s k 2 k = ( I − 1 ) n b + 1 ∑ t l t k 2 . (55)
Next, ∣ a s t ∣ = ∣ ∑ k = 1 s l ^ s k l ^ t k − e s t ∣ ≤ ( ∣ a s s ∣ + ∣ e s s ∣ ) ( ∣ a t t ∣ + ∣ e t t ∣ ) + ∣ e s t ∣ \lvert a_{st}\rvert=\left\lvert\sum_{k=1}^{s}\widehat{l}_{sk}\widehat{l}_{tk}-e_{st}\right\rvert\leq\sqrt{(\lvert a_{ss}\rvert+\lvert e_{ss}\rvert)(\lvert a_{tt}\rvert+\lvert e_{tt}\rvert)}+\lvert e_{st}\rvert ∣ a s t ∣ = ∑ k = 1 s l s k l t k − e s t ≤ (∣ a ss ∣ + ∣ e ss ∣) (∣ a tt ∣ + ∣ e tt ∣) + ∣ e s t ∣ . Then, for ζ s t t r u n c \zeta^{\mathrm{trunc}}_{st} ζ s t trunc , consider the element ( s , t ) (s,t) ( s , t ) of ϵ ψ ( ∑ k = 1 I − 1 ∣ L ^ I K ′ ′ ∣ ∣ L ^ I K T ∣ ) \epsilon_\psi\left(\sum_{k=1}^{I-1}\lvert\widehat{L}^{\prime\prime}_{IK}\rvert\lvert\widehat{L}_{IK}^T\rvert\right) ϵ ψ ( ∑ k = 1 I − 1 ∣ L I K ′′ ∣ ∣ L I K T ∣ ) . This is ϵ ψ ∑ k = 1 ( I − 1 ) n b ∣ l ^ s k ′ ′ ∣ ∣ l ^ t k ∣ \epsilon_\psi\sum_{k=1}^{(I-1)n_b}\lvert\widehat{l}^{\prime\prime}_{sk}\rvert\lvert\widehat{l}_{tk}\rvert ϵ ψ ∑ k = 1 ( I − 1 ) n b ∣ l s k ′′ ∣ ∣ l t k ∣ . This can be bounded above by Cauchy-Schwarz. ∑ k = 1 ( I − 1 ) n b ∣ l ^ s k ′ ′ ∣ 2 ≤ ∑ k = 1 ( I − 1 ) n b l ^ s k 2 + ν μ ψ 2 d s s 2 \sum_{k=1}^{(I-1)n_b}\lvert\widehat{l}^{\prime\prime}_{sk}\rvert^2\leq\sum_{k=1}^{(I-1)n_b}\widehat{l}_{sk}^2+\nu\mu_\psi^2d_{ss}^2 ∑ k = 1 ( I − 1 ) n b ∣ l s k ′′ ∣ 2 ≤ ∑ k = 1 ( I − 1 ) n b l s k 2 + ν μ ψ 2 d ss 2 (follows from (48) ), and ditto for t t t , giving us
ϵ ψ ∑ k = 1 ( I − 1 ) n b ∣ l ^ s k ′ ′ ∣ ∣ l ^ t k ∣ ≤ ϵ ψ ( ∑ k = 1 ( I − 1 ) n b ∣ l ^ s k ′ ′ ∣ 2 ) ( ∑ k = 1 ( I − 1 ) n b l ^ t k 2 ) ≤ ϵ ψ ( ∑ k = 1 ( I − 1 ) n b l ^ s k 2 + ν μ ψ 2 d s s 2 ) ( ∑ k = 1 ( I − 1 ) n b l ^ t k 2 + ν μ ψ 2 d t t 2 ) . (56) \epsilon_\psi\sum_{k=1}^{(I-1)n_b}\lvert\widehat{l}^{\prime\prime}_{sk}\rvert\lvert\widehat{l}_{tk}\rvert\leq\epsilon_\psi\sqrt{\left(\sum_{k=1}^{(I-1)n_b}\lvert\widehat{l}^{\prime\prime}_{sk}\rvert^2\right)\left(\sum_{k=1}^{(I-1)n_b}\widehat{l}_{tk}^2\right)}\leq\epsilon_\psi\sqrt{\left(\sum_{k=1}^{(I-1)n_b}\widehat{l}_{sk}^2+\nu\mu_\psi^2d_{ss}^2\right)\left(\sum_{k=1}^{(I-1)n_b}\widehat{l}_{tk}^2+\nu\mu_\psi^2d_{tt}^2\right)}. \tag*{(56)} ϵ ψ k = 1 ∑ ( I − 1 ) n b ∣ l s k ′′ ∣ ∣ l t k ∣ ≤ ϵ ψ k = 1 ∑ ( I − 1 ) n b ∣ l s k ′′ ∣ 2 k = 1 ∑ ( I − 1 ) n b l t k 2 ≤ ϵ ψ k = 1 ∑ ( I − 1 ) n b l s k 2 + ν μ ψ 2 d ss 2 k = 1 ∑ ( I − 1 ) n b l t k 2 + ν μ ψ 2 d tt 2 . (56)
Clearly, swapping the s s s and t t t doesn’t change (56) , so the same inequality works for the ( s , t ) (s,t) ( s , t ) element of ϵ ψ ( ∑ k = 1 I − 1 ∣ L ^ I K ′ ′ ∣ ∣ L ^ I K T ∣ ) T = ϵ ψ ( ∑ k = 1 I − 1 ∣ L ^ I K ∣ ∣ L ^ I K ′ ′ T ∣ ) \epsilon_\psi\left(\sum_{k=1}^{I-1}\lvert\widehat{L}^{\prime\prime}_{IK}\rvert\lvert\widehat{L}_{IK}^T\rvert\right)^T=\epsilon_\psi\left(\sum_{k=1}^{I-1}\lvert\widehat{L}_{IK}\rvert\lvert\widehat{L}^{\prime\prime T}_{IK}\rvert\right) ϵ ψ ( ∑ k = 1 I − 1 ∣ L I K ′′ ∣ ∣ L I K T ∣ ) T = ϵ ψ ( ∑ k = 1 I − 1 ∣ L I K ∣ ∣ L I K ′′ T ∣ ) . Element ( s , t ) (s,t) ( s , t ) of ϵ ψ 2 ( ∑ k = 1 I − 1 ∣ L ^ I K ′ ′ ∣ ∣ L ^ I K ′ ′ T ∣ ) \epsilon_\psi^2\left(\sum_{k=1}^{I-1}\lvert\widehat{L}^{\prime\prime}_{IK}\rvert\lvert\widehat{L}^{\prime\prime T}_{IK}\rvert\right) ϵ ψ 2 ( ∑ k = 1 I − 1 ∣ L I K ′′ ∣ ∣ L I K ′′ T ∣ ) is ϵ ψ 2 ∑ k = 1 ( I − 1 ) n b ∣ l ^ s k ′ ′ ∣ ∣ l ^ t k ′ ′ ∣ \epsilon_\psi^2\sum_{k=1}^{(I-1)n_b}\lvert\widehat{l}^{\prime\prime}_{sk}\rvert\lvert\widehat{l}^{\prime\prime}_{tk}\rvert ϵ ψ 2 ∑ k = 1 ( I − 1 ) n b ∣ l s k ′′ ∣ ∣ l t k ′′ ∣ , so by the same Cauchy-Schwarz argument, (57) holds for it too after we multiply an extra ϵ ψ \epsilon_\psi ϵ ψ , which means
ζ s t trunc ≤ ( 2 ϵ ψ + ϵ ψ 2 ) ( ∑ k = 1 ( I − 1 ) n b l ^ s k 2 + ν μ 2 d s s 2 ) ( ∑ k = 1 ( I − 1 ) n b l ^ t k 2 + ν μ 2 d t t 2 ) . (57) \zeta^{\operatorname{trunc}}_{st} \le(2\epsilon_\psi+\epsilon_\psi^2)\sqrt{\left(\sum_{k=1}^{(I-1)n_b}\hat{l}_{sk}^{2}+\nu_\mu^2d_{ss}^{2}\right)\left(\sum_{k=1}^{(I-1)n_b}\hat{l}_{tk}^{2}+\nu_\mu^2d_{tt}^{2}\right)}. \tag*{(57)} ζ s t trunc ≤ ( 2 ϵ ψ + ϵ ψ 2 ) k = 1 ∑ ( I − 1 ) n b l ^ s k 2 + ν μ 2 d ss 2 k = 1 ∑ ( I − 1 ) n b l ^ t k 2 + ν μ 2 d tt 2 . (57)
Using (57), we can construct an upper bound on ζ s t L _ a c c u m \zeta^{\mathrm{L\_accum}}_{st} ζ s t L_accum in the same way as we constructed (52),
ζ s t L _ a c c u m ≤ ( n b + 2 + n n b ) ϵ h ( 1 + ϵ ψ ) 2 ( ∑ k = 1 ( I − 1 ) n b l ^ s k 2 + ν μ 2 d s s 2 ) ( ∑ k = 1 ( I − 1 ) n b l ^ t k 2 + ν μ 2 d t t 2 ) . (58) \zeta^{\mathrm{L\_accum}}_{st}\le\left(n_b+2+\frac{n}{n_b}\right)\epsilon_h(1+\epsilon_\psi)^2\sqrt{\left(\sum_{k=1}^{(I-1)n_b}\hat{l}_{sk}^{2}+\nu_\mu^2d_{ss}^{2}\right)\left(\sum_{k=1}^{(I-1)n_b}\hat{l}_{tk}^{2}+\nu_\mu^2d_{tt}^{2}\right)}. \tag*{(58)} ζ s t L_accum ≤ ( n b + 2 + n b n ) ϵ h ( 1 + ϵ ψ ) 2 k = 1 ∑ ( I − 1 ) n b l ^ s k 2 + ν μ 2 d ss 2 k = 1 ∑ ( I − 1 ) n b l ^ t k 2 + ν μ 2 d tt 2 . (58)
Substituting (59), (58), (56) in (55) we get
∣ e s t ∣ ≤ [ ( 2 ϵ ψ + ϵ ψ 2 ) + ( n b + 2 + n n b ) ϵ h ( 1 + ϵ ψ ) 2 ] ( ∑ k = 1 ( I − 1 ) n b l ^ s k 2 + ν μ 2 d s s 2 ) ( ∑ k = 1 ( I − 1 ) n b l ^ t k 2 + ν μ 2 d t t 2 ) + n b ϵ h 1 − n b ϵ h ( ∑ k = ( I − 1 ) n b + 1 s l ^ s k 2 ) ( ∑ k = ( I − 1 ) n b + 1 t l ^ t k 2 ) + ( n b + 2 + n n b ) ϵ h ( ( ∣ a s s ∣ + ∣ e s s ∣ ) ( ∣ a t t ∣ + ∣ e t t ∣ ) + ∣ e s t ∣ ) ≤ [ ( 2 ϵ ψ + ϵ ψ 2 ) + 2 C ] ( ∣ a s s ∣ + ∣ e s s ∣ + ν μ 2 d s s 2 ) ( ∣ a t t ∣ + ∣ e t t ∣ + ν μ 2 d t t 2 ) + C ∣ e s t ∣ . \begin{aligned}
|e_{st}| &\le\left[(2\epsilon_\psi+\epsilon_\psi^2)+\left(n_b+2+\frac{n}{n_b}\right)\epsilon_h(1+\epsilon_\psi)^2\right]\sqrt{\left(\sum_{k=1}^{(I-1)n_b}\hat{l}_{sk}^{2}+\nu_\mu^2d_{ss}^{2}\right)\left(\sum_{k=1}^{(I-1)n_b}\hat{l}_{tk}^{2}+\nu_\mu^2d_{tt}^{2}\right)}\\
&\quad+\frac{n_b\epsilon_h}{1-n_b\epsilon_h}\sqrt{\left(\sum_{k=(I-1)n_b+1}^{s}\hat{l}_{sk}^{2}\right)\left(\sum_{k=(I-1)n_b+1}^{t}\hat{l}_{tk}^{2}\right)}
+\left(n_b+2+\frac{n}{n_b}\right)\epsilon_h\left(\sqrt{(|a_{ss}|+|e_{ss}|)(|a_{tt}|+|e_{tt}|)}+|e_{st}|\right)\\
&\le\left[(2\epsilon_\psi+\epsilon_\psi^2)+2C\right]\sqrt{(|a_{ss}|+|e_{ss}|+\nu_\mu^2d_{ss}^{2})(|a_{tt}|+|e_{tt}|+\nu_\mu^2d_{tt}^{2})}+C|e_{st}|.
\end{aligned} ∣ e s t ∣ ≤ [ ( 2 ϵ ψ + ϵ ψ 2 ) + ( n b + 2 + n b n ) ϵ h ( 1 + ϵ ψ ) 2 ] k = 1 ∑ ( I − 1 ) n b l ^ s k 2 + ν μ 2 d ss 2 k = 1 ∑ ( I − 1 ) n b l ^ t k 2 + ν μ 2 d tt 2 + 1 − n b ϵ h n b ϵ h k = ( I − 1 ) n b + 1 ∑ s l ^ s k 2 k = ( I − 1 ) n b + 1 ∑ t l ^ t k 2 + ( n b + 2 + n b n ) ϵ h ( ( ∣ a ss ∣ + ∣ e ss ∣ ) ( ∣ a tt ∣ + ∣ e tt ∣ ) + ∣ e s t ∣ ) ≤ [ ( 2 ϵ ψ + ϵ ψ 2 ) + 2 C ] ( ∣ a ss ∣ + ∣ e ss ∣ + ν μ 2 d ss 2 ) ( ∣ a tt ∣ + ∣ e tt ∣ + ν μ 2 d tt 2 ) + C ∣ e s t ∣.
⟹ ∣ e s t ∣ ≤ γ f p − C 1 − C ( ∣ a s s ∣ + ∣ e s s ∣ + ν μ 2 d s s 2 ) ( ∣ a t t ∣ + ∣ e t t ∣ + ν μ 2 d t t 2 ) (59) \Longrightarrow|e_{st}|\le\frac{\gamma_{\mathrm{fp}}-C}{1-C}\sqrt{(|a_{ss}|+|e_{ss}|+\nu_\mu^2d_{ss}^{2})(|a_{tt}|+|e_{tt}|+\nu_\mu^2d_{tt}^{2})} \tag*{(59)} ⟹ ∣ e s t ∣ ≤ 1 − C γ fp − C ( ∣ a ss ∣ + ∣ e ss ∣ + ν μ 2 d ss 2 ) ( ∣ a tt ∣ + ∣ e tt ∣ + ν μ 2 d tt 2 ) (59)
⟹ ∣ e s t ∣ ≤ γ f p 1 − γ f p ( ∣ a s s ∣ + ν μ 2 d s s 2 ) ( ∣ a t t ∣ + ν μ 2 d t t 2 ) . (60) \Longrightarrow|e_{st}|\le\frac{\gamma_{\mathrm{fp}}}{1-\gamma_{\mathrm{fp}}}\sqrt{(|a_{ss}|+\nu_\mu^2d_{ss}^{2})(|a_{tt}|+\nu_\mu^2d_{tt}^{2})}. \tag*{(60)} ⟹ ∣ e s t ∣ ≤ 1 − γ fp γ fp ( ∣ a ss ∣ + ν μ 2 d ss 2 ) ( ∣ a tt ∣ + ν μ 2 d tt 2 ) . (60)
where we have substituted (54) for rows s s s and t t t and used the fact that γ f p − C γ f p − γ f p + C > 0 \gamma_{\mathrm{fp}}-C\gamma_{\mathrm{fp}}-\gamma_{\mathrm{fp}}+C>0 γ fp − C γ fp − γ fp + C > 0 since γ f p < 1 \gamma_{\mathrm{fp}}<1 γ fp < 1 .
For off-diagonal block ( I , J ) (I,J) ( I , J ) with J < I ≤ n / n b J<I\le n/n_b J < I ≤ n / n b , instead of a block diagonal Cholesky, we have a TRSM. Unlike the emulation case we proved in Theorem 4.1, we do TRSM entirely in precision h h h . So, the error bound is n b ϵ h ∣ L ^ I J ∣ ∣ L ^ J J T ∣ ≤ n b ϵ h 1 − n b ϵ h ∣ L ^ I J ∣ ∣ L ^ J J T ∣ n_b\epsilon_h|\hat{L}_{IJ}||\hat{L}_{JJ}^{T}|\le\frac{n_b\epsilon_h}{1-n_b\epsilon_h}|\hat{L}_{IJ}||\hat{L}_{JJ}^{T}| n b ϵ h ∣ L ^ I J ∣∣ L ^ J J T ∣ ≤ 1 − n b ϵ h n b ϵ h ∣ L ^ I J ∣∣ L ^ J J T ∣ . The matrix inequality for the error is then
∣ A I J − ∑ K = 1 J L ^ I K L ^ J K T ∣ ≤ n b ϵ h 1 − n b ϵ h ∣ L ^ I J ∣ ∣ L ^ J J T ∣ + ∑ K = 1 J − 1 ϵ π g ( K ) ( ∣ L ^ I K ∣ ∣ L ^ J K T ∣ + ∣ L ^ I K ∣ ∣ L ^ J K T ∣ + ϵ π g ( K ) ∣ L ^ I K ∣ ∣ L ^ J K T ∣ ) + ( n b + 2 + n n b ) ϵ h ( ∑ K = 1 J − 1 ( ∣ L ^ I K ∣ + ϵ π g ( K ) ∣ L ^ I K ∣ ) ( ∣ L ^ J K ∣ + ϵ π g ( K ) ∣ L ^ J K ∣ ) T + ∣ A I J ∣ ) ≤ n b ϵ h 1 − n b ϵ h ∣ L ^ I J ∣ ∣ L ^ J J T ∣ + ∑ K = 1 J − 1 ϵ ψ ( ∣ L ^ I K ∣ ∣ L ^ J K T ∣ + ∣ L ^ I K ∣ ∣ L ^ J K T ∣ + ϵ ψ ∣ L ^ I K ∣ ∣ L ^ J K T ∣ ) + ( n b + 2 + n n b ) ϵ h ( ∑ K = 1 J − 1 ( ∣ L ^ I K ∣ + ϵ ψ ∣ L ^ I K ∣ ) ( ∣ L ^ J K ∣ + ϵ ψ ∣ L ^ J K ∣ ) T + ∣ A I J ∣ ) . (61) \begin{aligned}
\left|A_{IJ}-\sum_{K=1}^{J}\hat{L}_{IK}\hat{L}_{JK}^{T}\right|
&\le\frac{n_b\epsilon_h}{1-n_b\epsilon_h}|\hat{L}_{IJ}||\hat{L}_{JJ}^{T}|+\sum_{K=1}^{J-1}\epsilon_{\pi_g(K)}\left(|\hat{L}_{IK}||\hat{L}_{JK}^{T}|+|\hat{L}_{IK}||\hat{L}_{JK}^{T}|+\epsilon_{\pi_g(K)}|\hat{L}_{IK}||\hat{L}_{JK}^{T}|\right)\\
&\quad+\left(n_b+2+\frac{n}{n_b}\right)\epsilon_h\left(\sum_{K=1}^{J-1}\left(|\hat{L}_{IK}|+\epsilon_{\pi_g(K)}|\hat{L}_{IK}|\right)\left(|\hat{L}_{JK}|+\epsilon_{\pi_g(K)}|\hat{L}_{JK}|\right)^T+|A_{IJ}|\right)\\
&\le\frac{n_b\epsilon_h}{1-n_b\epsilon_h}|\hat{L}_{IJ}||\hat{L}_{JJ}^{T}|+\sum_{K=1}^{J-1}\epsilon_\psi\left(|\hat{L}_{IK}||\hat{L}_{JK}^{T}|+|\hat{L}_{IK}||\hat{L}_{JK}^{T}|+\epsilon_\psi|\hat{L}_{IK}||\hat{L}_{JK}^{T}|\right)\\
&\quad+\left(n_b+2+\frac{n}{n_b}\right)\epsilon_h\left(\sum_{K=1}^{J-1}\left(|\hat{L}_{IK}|+\epsilon_\psi|\hat{L}_{IK}|\right)\left(|\hat{L}_{JK}|+\epsilon_\psi|\hat{L}_{JK}|\right)^T+|A_{IJ}|\right). \tag*{(61)}
\end{aligned} A I J − K = 1 ∑ J L ^ I K L ^ J K T ≤ 1 − n b ϵ h n b ϵ h ∣ L ^ I J ∣∣ L ^ J J T ∣ + K = 1 ∑ J − 1 ϵ π g ( K ) ( ∣ L ^ I K ∣∣ L ^ J K T ∣ + ∣ L ^ I K ∣∣ L ^ J K T ∣ + ϵ π g ( K ) ∣ L ^ I K ∣∣ L ^ J K T ∣ ) + ( n b + 2 + n b n ) ϵ h ( K = 1 ∑ J − 1 ( ∣ L ^ I K ∣ + ϵ π g ( K ) ∣ L ^ I K ∣ ) ( ∣ L ^ J K ∣ + ϵ π g ( K ) ∣ L ^ J K ∣ ) T + ∣ A I J ∣ ) ≤ 1 − n b ϵ h n b ϵ h ∣ L ^ I J ∣∣ L ^ J J T ∣ + K = 1 ∑ J − 1 ϵ ψ ( ∣ L ^ I K ∣∣ L ^ J K T ∣ + ∣ L ^ I K ∣∣ L ^ J K T ∣ + ϵ ψ ∣ L ^ I K ∣∣ L ^ J K T ∣ ) + ( n b + 2 + n b n ) ϵ h ( K = 1 ∑ J − 1 ( ∣ L ^ I K ∣ + ϵ ψ ∣ L ^ I K ∣ ) ( ∣ L ^ J K ∣ + ϵ ψ ∣ L ^ J K ∣ ) T + ∣ A I J ∣ ) . (61)
Using the same ζ s t \zeta_{st} ζ s t notation, we write
∣ e s t ∣ ≤ TRSM error + ( n b + 2 + n n b ) ϵ h ∣ a s t ∣ + ζ s t trunc + ζ s t L _ a c c u m . (62) |e_{st}|\le\text{TRSM error}+\left(n_b+2+\frac{n}{n_b}\right)\epsilon_h|a_{st}|+\zeta^{\operatorname{trunc}}_{st}+\zeta^{\mathrm{L\_accum}}_{st}. \tag*{(62)} ∣ e s t ∣ ≤ TRSM error + ( n b + 2 + n b n ) ϵ h ∣ a s t ∣ + ζ s t trunc + ζ s t L_accum . (62)
As we just saw, the ( s , t ) (s,t) ( s , t ) element of the TRSM error is bounded by n b ϵ h 1 − n b ϵ h ∑ k = ( J − 1 ) n b + 1 t ∣ l ^ s k ∣ ∣ l ^ t k ∣ \frac{n_b\epsilon_h}{1-n_b\epsilon_h}\sum_{k=(J-1)n_b+1}^{t}|\hat{l}_{sk}||\hat{l}_{tk}| 1 − n b ϵ h n b ϵ h ∑ k = ( J − 1 ) n b + 1 t ∣ l ^ s k ∣∣ l ^ t k ∣ , which we can further bound by Cauchy-Schwarz
n b ϵ h 1 − n b ϵ h ∑ k = ( J − 1 ) n b + 1 t ∣ l ^ s k ∣ ∣ l ^ t k ∣ ≤ n b ϵ h 1 − n b ϵ h ∑ k = ( J − 1 ) n b + 1 t l ^ s k 2 ∑ k = ( J − 1 ) n b + 1 t l ^ t k 2 . (63) \frac{n_b\epsilon_h}{1-n_b\epsilon_h}\sum_{k=(J-1)n_b+1}^{t}|\hat{l}_{sk}||\hat{l}_{tk}|
\le\frac{n_b\epsilon_h}{1-n_b\epsilon_h}\sqrt{\sum_{k=(J-1)n_b+1}^{t}\hat{l}_{sk}^{2}\sum_{k=(J-1)n_b+1}^{t}\hat{l}_{tk}^{2}}. \tag*{(63)} 1 − n b ϵ h n b ϵ h k = ( J − 1 ) n b + 1 ∑ t ∣ l ^ s k ∣∣ l ^ t k ∣ ≤ 1 − n b ϵ h n b ϵ h k = ( J − 1 ) n b + 1 ∑ t l ^ s k 2 k = ( J − 1 ) n b + 1 ∑ t l ^ t k 2 . (63)
Next, we need an upper bound on ζ s t t r u n c \zeta^{\mathrm{trunc}}_{st} ζ s t trunc , for which we can slightly modify (58). Looking at (62), we can see that the matrices we include in ζ s t t r u n c \zeta^{\mathrm{trunc}}_{st} ζ s t trunc sum from k = 1 k=1 k = 1 to k = ( J − 1 ) n b < ( I − 1 ) n b k=(J-1)n_b < (I-1)n_b k = ( J − 1 ) n b < ( I − 1 ) n b , rather than k = 1 k=1 k = 1 to k = ( I − 1 ) n b k=(I-1)n_b k = ( I − 1 ) n b (as seen in (45)), so
ζ s t t r u n c ≤ ( 2 ϵ ψ + ϵ ψ 2 ) ( ∑ k = 1 ( J − 1 ) n b l ^ s k 2 + ν μ ψ 2 d s s 2 ) ( ∑ k = 1 ( J − 1 ) n b l ^ t k 2 + ν μ ψ 2 d t t 2 ) ≤ ( 2 ϵ ψ + ϵ ψ 2 ) ( ∣ a s s ∣ + ∣ e s s ∣ + ν μ ψ 2 d s s 2 ) ( ∣ a t t ∣ + ∣ e t t ∣ + ν μ ψ 2 d t t 2 ) . (64) \begin{aligned}
\zeta^{\mathrm{trunc}}_{st} \le(2\epsilon_{\psi}+\epsilon_{\psi}^{2})\sqrt{\left(\sum_{k=1}^{(J-1)n_b}\hat{l}_{sk}^{2}+\nu\mu_{\psi}^{2}d_{ss}^{2}\right)\left(\sum_{k=1}^{(J-1)n_b}\hat{l}_{tk}^{2}+\nu\mu_{\psi}^{2}d_{tt}^{2}\right)} \\
&\le(2\epsilon_{\psi}+\epsilon_{\psi}^{2})\sqrt{\left(|a_{ss}|+|e_{ss}|+\nu\mu_{\psi}^{2}d_{ss}^{2}\right)\left(|a_{tt}|+|e_{tt}|+\nu\mu_{\psi}^{2}d_{tt}^{2}\right)}. \tag*{(64)}
\end{aligned} ζ s t trunc ≤ ( 2 ϵ ψ + ϵ ψ 2 ) k = 1 ∑ ( J − 1 ) n b l ^ s k 2 + ν μ ψ 2 d ss 2 k = 1 ∑ ( J − 1 ) n b l ^ t k 2 + ν μ ψ 2 d tt 2 ≤ ( 2 ϵ ψ + ϵ ψ 2 ) ( ∣ a ss ∣ + ∣ e ss ∣ + ν μ ψ 2 d ss 2 ) ( ∣ a tt ∣ + ∣ e tt ∣ + ν μ ψ 2 d tt 2 ) . (64)
Similarly, for ζ s t L _ a c c u m \zeta^{\mathrm{L\_accum}}_{st} ζ s t L_accum , we can write the bound
ζ s t L _ a c c u m ≤ ( n b + 2 + n n b ) ϵ h ( 1 + ϵ ψ ) 2 ( ∑ k = 1 ( J − 1 ) n b l ^ s k 2 + ν μ ψ 2 d s s 2 ) ( ∑ k = 1 ( J − 1 ) n b l ^ t k 2 + ν μ ψ 2 d t t 2 ) . (65) \zeta^{\mathrm{L\_accum}}_{st}\le\left(n_b+2+\frac{n}{n_b}\right)\epsilon_h(1+\epsilon_{\psi})^{2}\sqrt{\left(\sum_{k=1}^{(J-1)n_b}\hat{l}_{sk}^{2}+\nu\mu_{\psi}^{2}d_{ss}^{2}\right)\left(\sum_{k=1}^{(J-1)n_b}\hat{l}_{tk}^{2}+\nu\mu_{\psi}^{2}d_{tt}^{2}\right)}. \tag*{(65)} ζ s t L_accum ≤ ( n b + 2 + n b n ) ϵ h ( 1 + ϵ ψ ) 2 k = 1 ∑ ( J − 1 ) n b l ^ s k 2 + ν μ ψ 2 d ss 2 k = 1 ∑ ( J − 1 ) n b l ^ t k 2 + ν μ ψ 2 d tt 2 . (65)
Substituting (66), (65), (64) in (63), and using ∣ a s t ∣ = ∣ ∑ k = 1 t l ^ s k l ^ t k − e s t ∣ ≤ ( ∣ a s s ∣ + ∣ e s s ∣ ) ( ∣ a t t ∣ + ∣ e t t ∣ ) + ∣ e s t ∣ |a_{st}|=\left|\sum_{k=1}^{t}\hat{l}_{sk}\hat{l}_{tk}-e_{st}\right|\le\sqrt{(|a_{ss}|+|e_{ss}|)(|a_{tt}|+|e_{tt}|)}+|e_{st}| ∣ a s t ∣ = ∑ k = 1 t l ^ s k l ^ t k − e s t ≤ ( ∣ a ss ∣ + ∣ e ss ∣ ) ( ∣ a tt ∣ + ∣ e tt ∣ ) + ∣ e s t ∣ , we get the same bounds as (60) and (61). Finally, to get (44) we need to bound d r r d_{rr} d r r for 1 ≤ r ≤ n 1\le r\le n 1 ≤ r ≤ n . Since d r r d_{rr} d r r is a scale factor in row r r r ,
d r r ≤ 1 + ϵ s c a l β max k = 1 r ∣ l ^ r k ∣ ≤ 1 + ϵ s c a l β ∣ a r r ∣ + ∣ e r r ∣ . d_{rr}\le\frac{1+\epsilon_{\mathrm{scal}}}{\beta}\max_{k=1}^{r}|\hat{l}_{rk}|\le\frac{1+\epsilon_{\mathrm{scal}}}{\beta}\sqrt{|a_{rr}|+|e_{rr}|}. d r r ≤ β 1 + ϵ scal k = 1 max r ∣ l ^ r k ∣ ≤ β 1 + ϵ scal ∣ a r r ∣ + ∣ e r r ∣ .
Substituting this in eq:54 , we get
( 1 − γ f p ) ∣ e t t ∣ ≤ γ f p ( ∣ a t t ∣ + ν μ ψ 2 ( 1 + ϵ s c a l ) 2 β 2 ( ∣ a t t ∣ + ∣ e t t ∣ ) ) ⇒ ∣ e t t ∣ ≤ γ f p ( 1 + λ Ψ ) 1 − γ f p ( 1 + λ Ψ ) ∣ a t t ∣ . (66) \begin{aligned}
(1-\gamma_{\mathrm{fp}})|e_{tt}|\le\gamma_{\mathrm{fp}}\left(|a_{tt}|+\frac{\nu\mu_{\psi}^{2}(1+\epsilon_{\mathrm{scal}})^{2}}{\beta^{2}}(|a_{tt}|+|e_{tt}|)\right) \\
&\Rightarrow|e_{tt}|\le\frac{\gamma_{\mathrm{fp}}(1+\lambda_{\Psi})}{1-\gamma_{\mathrm{fp}}(1+\lambda_{\Psi})}|a_{tt}|. \tag*{(66)}
\end{aligned} ( 1 − γ fp ) ∣ e tt ∣ ≤ γ fp ( ∣ a tt ∣ + β 2 ν μ ψ 2 ( 1 + ϵ scal ) 2 ( ∣ a tt ∣ + ∣ e tt ∣ ) ) ⇒ ∣ e tt ∣ ≤ 1 − γ fp ( 1 + λ Ψ ) γ fp ( 1 + λ Ψ ) ∣ a tt ∣. (66)
For ∣ e s t ∣ |e_{st}| ∣ e s t ∣ , using the bound on d r r d_{rr} d r r in (59) , we obtain
∣ e s t ∣ ≤ γ f p − C 1 − C ( 1 + λ Ψ ) ( ∣ a s s ∣ + ∣ e s s ∣ ) ( 1 + λ Ψ ) ( ∣ a t t ∣ + ∣ e t t ∣ ) ≤ γ f p ( 1 + λ Ψ ) ( ∣ a s s ∣ + ∣ e s s ∣ ) ( 1 + λ Ψ ) ( ∣ a t t ∣ + ∣ e t t ∣ ) . (67) |e_{st}|\le\frac{\gamma_{\mathrm{fp}}-C}{1-C}\sqrt{(1+\lambda_{\Psi})(|a_{ss}|+|e_{ss}|)}\sqrt{(1+\lambda_{\Psi})(|a_{tt}|+|e_{tt}|)}
\le\gamma_{\mathrm{fp}}\sqrt{(1+\lambda_{\Psi})(|a_{ss}|+|e_{ss}|)}\sqrt{(1+\lambda_{\Psi})(|a_{tt}|+|e_{tt}|)}. \tag*{(67)} ∣ e s t ∣ ≤ 1 − C γ fp − C ( 1 + λ Ψ ) ( ∣ a ss ∣ + ∣ e ss ∣ ) ( 1 + λ Ψ ) ( ∣ a tt ∣ + ∣ e tt ∣ ) ≤ γ fp ( 1 + λ Ψ ) ( ∣ a ss ∣ + ∣ e ss ∣ ) ( 1 + λ Ψ ) ( ∣ a tt ∣ + ∣ e tt ∣ ) . (67)
Substituting (67) above gives us the desired inequality.
Theorem 4.3. Suppose we do a Cholesky with the same assumptions as Theorem 4.2, but we don’t scale (for example, FP16 inputs accumulating into FP32, with FP32 output), and if γ f p < 1 \gamma_{\mathrm{fp}}<1 γ fp < 1 we have A + E = L ^ L ^ T A+E=\hat{L}\hat{L}^{T} A + E = L ^ L ^ T with
∣ e s t ∣ ≤ γ f p 1 − γ f p a s s + ν μ Ψ 2 a t t + ν μ Ψ 2 . (68) |e_{st}|\le\frac{\gamma_{\mathrm{fp}}}{1-\gamma_{\mathrm{fp}}}\sqrt{a_{ss}+\nu\mu_{\Psi}^{2}}\sqrt{a_{tt}+\nu\mu_{\Psi}^{2}}. \tag*{(68)} ∣ e s t ∣ ≤ 1 − γ fp γ fp a ss + ν μ Ψ 2 a tt + ν μ Ψ 2 . (68)
Proof. As discussed in Section 3.4, if we do not microscale then d t t = 1 d_{tt}=1 d tt = 1 . Plugging this into (42) gives us the desired inequality.
Factorization Failures
It is shown in [8 ] and [14 Theorem 10.7] that we can have factorization failures when our machine epsilon gets too large. If Algorithm 5 tries to give us L ^ L ^ T = A + E \hat{L}\hat{L}^{T}=A+E L ^ L ^ T = A + E such that A + E A+E A + E ends up being indefinite, our factorization will fail. The failure/success guarantees from [14 Theorem 10.7] can be adapted to our error bounds in Section 4. Since all of our error bounds take the form γ 1 − γ a i i a j j \frac{\gamma}{1-\gamma}\sqrt{a_{ii}a_{jj}} 1 − γ γ a ii a j j , we can state the following theorem:
Theorem 4.4. Suppose we have a Cholesky factorization algorithm which on a successful run with input A 0 A_0 A 0 , returns L ^ \hat{L} L ^ such that L ^ L ^ T = A 0 + E \hat{L}\hat{L}^{T}=A_0+E L ^ L ^ T = A 0 + E with ∣ e i j ∣ ≤ γ 1 − γ a i i a j j |e_{ij}|\le\frac{\gamma}{1-\gamma}\sqrt{a_{ii}a_{jj}} ∣ e ij ∣ ≤ 1 − γ γ a ii a j j . Here γ \gamma γ can be γ e m u \gamma_{\mathrm{emu}} γ emu , γ f p \gamma_{\mathrm{fp}} γ fp or γ f p ( 1 + λ Ψ ) \gamma_{\mathrm{fp}}(1+\lambda_{\Psi}) γ fp ( 1 + λ Ψ ) from Theorems 4.1, 4.2 or 4.3. Let H = D − 1 A D − 1 H=D^{-1}AD^{-1} H = D − 1 A D − 1 with D = diag i a i i D=\operatorname{diag}_{i}\sqrt{a_{ii}} D = diag i a ii . Then the factorization is guaranteed to succeed if λ min ( H ) > n γ 1 − γ \lambda_{\min}(H)>\frac{n\gamma}{1-\gamma} λ m i n ( H ) > 1 − γ nγ , and the factorization is guaranteed to fail if λ min ( H ) < − n γ 1 − γ \lambda_{\min}(H)<-\frac{n\gamma}{1-\gamma} λ m i n ( H ) < − 1 − γ nγ .
Proof. See [14 Theorem 10.7] . □
We always have λ min ( H ) ≤ 1 \lambda_{\min}(H) \le1 λ m i n ( H ) ≤ 1 since the sum of all eigenvalues of H H H is n n n . Say we do a Cholesky with FP8 (E4M3) MX GEMMs, if we assume no underflow, and ignore higher precision rounding errors, by Theorem 4.2 , we have
γ ≈ 2 ϵ l + ϵ l 2 = 0.265625. \gamma\approx2\epsilon_l + \epsilon_l^2 = 0.265625. γ ≈ 2 ϵ l + ϵ l 2 = 0.265625.
So we cannot guarantee success for n ≥ 3 n \ge3 n ≥ 3 . Similarly, for FP16 inputs and FP32 accumulation, if there is no underflow, γ ≈ 0.00195 \gamma\approx0.00195 γ ≈ 0.00195 , so we cannot guarantee success for n ≥ 511 n \ge511 n ≥ 511 . To deal with this, we propose using a modified Cholesky factorization. An extensive overview and comparison of the various modified Cholesky factorizations is provided in [22 ] . We consider two modified Cholesky methods which perturb the diagonal of A A A during the factorization to ensure the diagonal stays positive. The first is a simple clamping algorithm, which clamps each updated diagonal entry so that it never falls below a user-specified δ c l a m p > 0 \delta^{\mathrm{clamp}} > 0 δ clamp > 0 . The second is the algorithm of Gill, Murray, and Wright [10 ] , which we use without pivoting. We will refer to their algorithm as GMW81. Both algorithms are equivalent to a Cholesky factorization of A + F A+F A + F for some diagonal F ≥ 0 F \ge0 F ≥ 0 , so Theorems 4.2 and 4.3 still apply with a s s a t t \sqrt{a_{ss}a_{tt}} a ss a tt replaced by ( a s s + f ∗ ) ( a t t + f ∗ ) \sqrt{(a_{ss}+f^*)(a_{tt}+f^*)} ( a ss + f ∗ ) ( a tt + f ∗ ) , where f ∗ = max i f i i f^*=\max_i f_{ii} f ∗ = max i f ii (plus the underflow term of Theorem 4.3 if underflow is expected to be significant). It therefore suffices to bound f ∗ f^* f ∗ .
We will use a similar notation to § 4 , for j ≤ i j \le i j ≤ i let a i j u = a i j − ∑ k = 1 j − 1 l i k l j k a^u_{ij}=a_{ij}-\sum_{k=1}^{j-1}l_{ik}l_{jk} a ij u = a ij − ∑ k = 1 j − 1 l ik l j k . Note that a i i u a^u_{ii} a ii u may be negative. Both diagonal perturbation schemes, along with their respective bounds on f ∗ f^* f ∗ are described in the following.
Diagonal Clamping
In this scheme, we set l j j 2 = max ( a j j u , δ c l a m p ) l_{jj}^2=\max(a^u_{jj},\delta^{\mathrm{clamp}}) l j j 2 = max ( a j j u , δ clamp ) , so a j j u + f j j = max ( a j j u , δ c l a m p ) ⇒ f j j = max ( 0 , δ c l a m p − a j j u ) a^u_{jj}+f_{jj}=\max(a^u_{jj},\delta^{\mathrm{clamp}})\Rightarrow f_{jj}=\max(0,\delta^{\mathrm{clamp}}-a^u_{jj}) a j j u + f j j = max ( a j j u , δ clamp ) ⇒ f j j = max ( 0 , δ clamp − a j j u ) . Also, l j j ≥ δ c l a m p l_{jj}\ge\sqrt{\delta^{\mathrm{clamp}}} l j j ≥ δ clamp , and for i > j i>j i > j , ∣ l i j ∣ ≤ 1 δ c l a m p ( ∣ a i j u ∣ + ∑ k = 1 j − 1 ∣ l i k ∣ ∣ l j k ∣ ) |l_{ij}|\le\frac{1}{\sqrt{\delta^{\mathrm{clamp}}}}\left(|a^u_{ij}|+\sum_{k=1}^{j-1}|l_{ik}||l_{jk}|\right) ∣ l ij ∣ ≤ δ clamp 1 ( ∣ a ij u ∣ + ∑ k = 1 j − 1 ∣ l ik ∣∣ l j k ∣ ) . If ∣ l i j ∣ ≤ λ |l_{ij}|\le\lambda ∣ l ij ∣ ≤ λ for all i > j i>j i > j , then
f i i = max ( 0 , δ c l a m p − a i i u + ∑ k = 1 i − 1 l i k 2 ) ⇒ f ∗ ≤ δ c l a m p + max t ∣ a t t ∣ + ( n − 1 ) λ 2 . f_{ii}=\max\left(0,\delta^{\mathrm{clamp}}-a^u_{ii}+\sum_{k=1}^{i-1}l_{ik}^2\right)\Rightarrow f^*\le\delta^{\mathrm{clamp}}+\max_t|a_{tt}|+(n-1)\lambda^2. f ii = max ( 0 , δ clamp − a ii u + k = 1 ∑ i − 1 l ik 2 ) ⇒ f ∗ ≤ δ clamp + t max ∣ a tt ∣ + ( n − 1 ) λ 2 .
We cannot know λ \lambda λ ahead of time; all we can say is that λ \lambda λ is at least O ( 1 / δ c l a m p ) O(1/\sqrt{\delta^{\mathrm{clamp}}}) O ( 1/ δ clamp ) , so a very small δ c l a m p \delta^{\mathrm{clamp}} δ clamp , or a very large δ c l a m p \delta^{\mathrm{clamp}} δ clamp will both lead to large errors. Since the perturbation doesn’t use data from anywhere else in the matrix, this perturbation scheme can be incorporated easily in the blocking used in Algorithm 5, which uses block diagonal Cholesky + TRSM + GEMM/SYRK.
GMW81
The perturbation rule here is l j j 2 = max ( δ G M W , ∣ a j j u ∣ , ( θ j G M W β G M W ) 2 ) l_{jj}^2=\max\left(\delta^{\mathrm{GMW}},|a^u_{jj}|,\left(\frac{\theta_j^{\mathrm{GMW}}}{\beta^{\mathrm{GMW}}}\right)^2\right) l j j 2 = max ( δ GMW , ∣ a j j u ∣ , ( β GMW θ j GMW ) 2 ) , with θ j G M W = max i > j ∣ a i j u ∣ \theta_j^{\mathrm{GMW}}=\max_{i>j}|a^u_{ij}| θ j GMW = max i > j ∣ a ij u ∣ , where β G M W \beta^{\mathrm{GMW}} β GMW and δ G M W \delta^{\mathrm{GMW}} δ GMW are user inputs. Since ∣ l j j ∣ ≥ θ j G M W / β G M W |l_{jj}|\ge\theta_j^{\mathrm{GMW}}/\beta^{\mathrm{GMW}} ∣ l j j ∣ ≥ θ j GMW / β GMW , for every i > j i>j i > j , we have ∣ l i j ∣ = ∣ a i j u / l j j ∣ ≤ β G M W ∣ a i j u ∣ / θ j G M W ≤ β G M W |l_{ij}|=|a^u_{ij}/l_{jj}|\le\beta^{\mathrm{GMW}}|a^u_{ij}|/\theta_j^{\mathrm{GMW}}\le\beta^{\mathrm{GMW}} ∣ l ij ∣ = ∣ a ij u / l j j ∣ ≤ β GMW ∣ a ij u ∣/ θ j GMW ≤ β GMW , so the GMW algorithm gives us an a priori upper bound on the elements of L L L . Now, we want a bound on ∣ a j j u ∣ ⋅ ∣ a i j u ∣ ≤ ∣ a i j ∣ + ∑ k = 1 j − 1 l i k 2 ≤ max t ∣ a t t ∣ + ( j − 1 ) ( β G M W ) 2 |a^u_{jj}|\cdot|a^u_{ij}|\le|a_{ij}|+\sum_{k=1}^{j-1}l_{ik}^2\le\max_t|a_{tt}|+(j-1)(\beta^{\mathrm{GMW}})^2 ∣ a j j u ∣ ⋅ ∣ a ij u ∣ ≤ ∣ a ij ∣ + ∑ k = 1 j − 1 l ik 2 ≤ max t ∣ a tt ∣ + ( j − 1 ) ( β GMW ) 2 . Next, for θ j G M W \theta_j^{\mathrm{GMW}} θ j GMW , we need a bound on ∣ a i j u ∣ |a^u_{ij}| ∣ a ij u ∣ for i > j i>j i > j : ∣ a i j u ∣ ≤ ∣ a i j ∣ + ∑ k = 1 j − 1 ∣ l i k ∣ ∣ l j k ∣ ≤ max s ≠ t ∣ a s t ∣ + ( j − 1 ) ( β G M W ) 2 |a^u_{ij}|\le|a_{ij}|+\sum_{k=1}^{j-1}|l_{ik}||l_{jk}|\le\max_{s\ne t}|a_{st}|+(j-1)(\beta^{\mathrm{GMW}})^2 ∣ a ij u ∣ ≤ ∣ a ij ∣ + ∑ k = 1 j − 1 ∣ l ik ∣∣ l j k ∣ ≤ max s = t ∣ a s t ∣ + ( j − 1 ) ( β GMW ) 2 . Since θ j G M W \theta_j^{\mathrm{GMW}} θ j GMW is the maximum of ∣ a i j u ∣ |a^u_{ij}| ∣ a ij u ∣ over i > j i>j i > j , θ j G M W ≤ max s ≠ t ∣ a s t ∣ + ( j − 1 ) ( β G M W ) 2 \theta_j^{\mathrm{GMW}}\le\max_{s\ne t}|a_{st}|+(j-1)(\beta^{\mathrm{GMW}})^2 θ j GMW ≤ max s = t ∣ a s t ∣ + ( j − 1 ) ( β GMW ) 2 . Finally, since f j j = l j j 2 − a j j u f_{jj}=l_{jj}^2-a^u_{jj} f j j = l j j 2 − a j j u ,
f j j ≤ δ G M W + ( θ j G M W ) 2 ( β G M W ) 2 + 2 ∣ a j j u ∣ ⇒ f ∗ ≤ δ G M W + 2 ( max t ∣ a t t ∣ + ( n − 1 ) ( β G M W ) 2 ) + ( max s ≠ t ∣ a s t ∣ β G M W + ( n − 1 ) β G M W ) 2 . \begin{aligned}
f_{jj}\le\delta^{\mathrm{GMW}}+\frac{(\theta_j^{\mathrm{GMW}})^2}{(\beta^{\mathrm{GMW}})^2}+2|a^u_{jj}| \\
&\Rightarrow f^*\le\delta^{\mathrm{GMW}}+2\left(\max_t|a_{tt}|+(n-1)(\beta^{\mathrm{GMW}})^2\right)+\left(\frac{\max_{s\ne t}|a_{st}|}{\beta^{\mathrm{GMW}}}+(n-1)\beta^{\mathrm{GMW}}\right)^2.
\end{aligned} f j j ≤ δ GMW + ( β GMW ) 2 ( θ j GMW ) 2 + 2∣ a j j u ∣ ⇒ f ∗ ≤ δ GMW + 2 ( t max ∣ a tt ∣ + ( n − 1 ) ( β GMW ) 2 ) + ( β GMW max s = t ∣ a s t ∣ + ( n − 1 ) β GMW ) 2 .
Gill, Murray, and Wright [10 ] prescribe ( β G M W ) 2 = max ( max t ∣ a t t ∣ , max s ≠ t ∣ a s t ∣ n 2 − 1 , ϵ ) (\beta^{\mathrm{GMW}})^2=\max\left(\max_t|a_{tt}|,\frac{\max_{s\ne t}|a_{st}|}{\sqrt{n^2-1}},\epsilon\right) ( β GMW ) 2 = max ( max t ∣ a tt ∣ , n 2 − 1 m a x s = t ∣ a s t ∣ , ϵ ) , where ϵ \epsilon ϵ is the machine precision. It is important to note that the perturbation rule uses all subdiagonal elements in column j j j to compute θ j G M W \theta_j^{\mathrm{GMW}} θ j GMW .
As a consequence of this, we cannot block this algorithm the same way as Algorithm 5. Instead, the TRSM and diagonal block solve need to be merged into one "panel solve" that is applied to a block column of width n b n_b n b .
Adaptive precision Algorithms
In this section, we introduce a framework for adaptive precision Cholesky algorithms which dynamically choose the number of slices (or the precision) of each GEMM or TRSM operation. At each iteration, the algorithms choose the smallest number of slices (or the smallest precision) that ensure that the computed n × n n \times n n × n L ^ \widehat{L} L factor satisfies ∣ a i j − ∑ k = 1 min ( i , j ) l ^ i k l ^ j k ∣ ≤ ϵ ′ a i i a j j |a_{ij} - \sum_{k=1}^{\min(i,j)} \widehat{l}_{ik}\widehat{l}_{jk}| \le\epsilon' \sqrt{a_{ii}a_{jj}} ∣ a ij − ∑ k = 1 m i n ( i , j ) l ik l j k ∣ ≤ ϵ ′ a ii a j j for 1 ≤ i , j ≤ n 1 \le i,j \le n 1 ≤ i , j ≤ n , where A A A is a positive definite n × n n \times n n × n input matrix, and ϵ ′ \epsilon' ϵ ′ is a user-defined threshold. When ϵ ′ \epsilon' ϵ ′ is chosen to be equal to n ϵ 64 n\epsilon_{64} n ϵ 64 , the bound ϵ ′ a i i a j j \epsilon' \sqrt{a_{ii}a_{jj}} ϵ ′ a ii a j j coincides with the classical component-wise error bound of the Cholesky factorization [8 , 14 ] .
Adaptive precision with Ozaki 1
For simplicity, assume n b ∣ n n_b \mid n n b ∣ n , and to be concrete, we focus on the description of the algorithm using emulation based on the Ozaki 1 scheme. The mixed-precision floating-point case, and the Ozaki 2 case are very similar, and we state the algorithm without proof. Algorithm 5 serves as the basis for our adaptive precision algorithm. The key modification is to select the number of slices used in the TRSM and GEMM operations (lines 4 and 8 of Algorithm 5, respectively) at every iteration, instead of using a pre-determined list of slices throughout the factorization. The following theorem defines the conditions under which the number of slices can be changed while ensuring that the output satisfies ∣ a i j − ∑ k = 1 min ( i , j ) l ^ i k l ^ j k ∣ ≤ ϵ ′ a i i a j j |a_{ij} - \sum_{k=1}^{\min(i,j)} \widehat{l}_{ik}\widehat{l}_{jk}| \le\epsilon' \sqrt{a_{ii}a_{jj}} ∣ a ij − ∑ k = 1 m i n ( i , j ) l ik l j k ∣ ≤ ϵ ′ a ii a j j . Given an input matrix A A A , and considering the iteration I I I of Algorithm 5, the input matrix A A A and the calculated factor L ^ \widehat{L} L are partitioned into block matrices of dimension 3 × 3 3 \times3 3 × 3 such that A − L ^ L ^ T A - \widehat{L}\widehat{L}^{T} A − L L T is written as
\begin{pmatrix}
A_{11} & A_{21}^{T} & A_{31}^{T} \\
A_{21} & A_{22} & A_{32}^{T} \\
A_{31} & A_{32} & A_{33}
\end{pmatrix}
-
\begin{pmatrix}
\widehat{L}_{11} & 0 & 0 \\
\widehat{L}_{21} & \widehat{L}_{22} & 0 \\
\widehat{L}_{31} & \widehat{L}_{32} & \widehat{L}_{33}
\end{pmatrix}
\begin{pmatrix}
\widehat{L}_{11} & 0 & 0 \\
\widehat{L}_{21} & \widehat{L}_{22} & 0 \\
\widehat{L}_{31} & \widehat{L}_{32} & \widehat{L}_{33}
\end{pmatrix}^{T}, \tag*{(69)}\end{pmatrix}
where A 11 A_{11} A 11 is ( I − 1 ) n b × ( I − 1 ) n b (I-1)n_b \times(I-1)n_b ( I − 1 ) n b × ( I − 1 ) n b , A 22 A_{22} A 22 is n b × n b n_b \times n_b n b × n b , and A 33 A_{33} A 33 is ( n − I n b ) × ( n − I n b ) (n-In_b) \times(n-In_b) ( n − I n b ) × ( n − I n b ) . The blocks of L ^ \widehat{L} L are defined in a similar way. The only submatrix of L ^ \widehat{L} L whose computation is not complete is L ^ 33 \widehat{L}_{33} L 33 . Once the Schur complement of A 33 A_{33} A 33 is computed as A 33 − L ^ 31 L ^ 31 T − L ^ 32 L ^ 32 T A_{33} - \widehat{L}_{31}\widehat{L}_{31}^{T} - \widehat{L}_{32}\widehat{L}_{32}^{T} A 33 − L 31 L 31 T − L 32 L 32 T , according to Algorithm 5, a mixed-precision Cholesky factorization of that Schur complement is performed. The backward error on A 33 A_{33} A 33 is then ∣ A 33 − L ^ 31 L ^ 31 T − L ^ 32 L ^ 32 T − L ^ 33 L ^ 33 T ∣ |A_{33} - \widehat{L}_{31}\widehat{L}_{31}^{T} - \widehat{L}_{32}\widehat{L}_{32}^{T} - \widehat{L}_{33}\widehat{L}_{33}^{T}| ∣ A 33 − L 31 L 31 T − L 32 L 32 T − L 33 L 33 T ∣ . Denote A 33 ( c u r r ) A_{33}^{(\mathrm{curr})} A 33 ( curr ) as the computed result of A 33 − L ^ 31 L ^ 31 T A_{33} - \widehat{L}_{31}\widehat{L}_{31}^{T} A 33 − L 31 L 31 T and A 33 ( n e x t ) A_{33}^{(\mathrm{next})} A 33 ( next ) as the computed result of A 33 ( c u r r ) − L ^ 32 L ^ 32 T A_{33}^{(\mathrm{curr})} - \widehat{L}_{32}\widehat{L}_{32}^{T} A 33 ( curr ) − L 32 L 32 T . Then, we can write the following telescoping error bound for A 33 A_{33} A 33 ,
∣ A 33 − L ^ 31 L ^ 31 T − L ^ 32 L ^ 32 T − L ^ 33 L ^ 33 T ∣ ≤ ∣ ( A 33 − L ^ 31 L ^ 31 T ) − A 33 ( c u r r ) ∣ + ∣ ( A 33 ( c u r r ) − L ^ 32 L ^ 32 T ) − A 33 ( n e x t ) ∣ + ∣ ( A 33 ( n e x t ) − L ^ 33 L ^ 33 T ) ∣ ≤ E ( 1 ) + E ( 2 ) + E ( 3 ) , (70) \begin{aligned}
|A_{33} - \widehat{L}_{31}\widehat{L}_{31}^{T} - \widehat{L}_{32}\widehat{L}_{32}^{T} - \widehat{L}_{33}\widehat{L}_{33}^{T}|
&\le|(A_{33} - \widehat{L}_{31}\widehat{L}_{31}^{T}) - A_{33}^{(\mathrm{curr})}| \\
&\quad+ |(A_{33}^{(\mathrm{curr})} - \widehat{L}_{32}\widehat{L}_{32}^{T}) - A_{33}^{(\mathrm{next})}| \\
&\quad+ |(A_{33}^{(\mathrm{next})} - \widehat{L}_{33}\widehat{L}_{33}^{T})| \\
&\le E^{(1)} + E^{(2)} + E^{(3)}, \tag*{(70)}
\end{aligned} ∣ A 33 − L 31 L 31 T − L 32 L 32 T − L 33 L 33 T ∣ ≤ ∣ ( A 33 − L 31 L 31 T ) − A 33 ( curr ) ∣ + ∣ ( A 33 ( curr ) − L 32 L 32 T ) − A 33 ( next ) ∣ + ∣ ( A 33 ( next ) − L 33 L 33 T ) ∣ ≤ E ( 1 ) + E ( 2 ) + E ( 3 ) , (70)
where E ( 1 ) E^{(1)} E ( 1 ) , E ( 2 ) E^{(2)} E ( 2 ) , E ( 3 ) E^{(3)} E ( 3 ) are slightly loosened worst case error bounds on ∣ ( A 33 − L ^ 31 L ^ 31 T ) − A 33 ( c u r r ) ∣ |(A_{33} - \widehat{L}_{31}\widehat{L}_{31}^{T}) - A_{33}^{(\mathrm{curr})}| ∣ ( A 33 − L 31 L 31 T ) − A 33 ( curr ) ∣ , ∣ ( A 33 ( c u r r ) − L ^ 32 L ^ 32 T ) − A 33 ( n e x t ) ∣ |(A_{33}^{(\mathrm{curr})} - \widehat{L}_{32}\widehat{L}_{32}^{T}) - A_{33}^{(\mathrm{next})}| ∣ ( A 33 ( curr ) − L 32 L 32 T ) − A 33 ( next ) ∣ and ∣ ( A 33 ( n e x t ) − L ^ 33 L ^ 33 T ) ∣ |(A_{33}^{(\mathrm{next})} - \widehat{L}_{33}\widehat{L}_{33}^{T})| ∣ ( A 33 ( next ) − L 33 L 33 T ) ∣ obtained from (10), and (28), respectively. In the proof of the following Theorem 5.1 we explain why we need to relax the bound and how we achieve this.
Algorithm 6 Blocked Cholesky Factorization with Adaptive Slice Ozaki 1 GEMM
Require: Matrix A ∈ R n × n A \in\mathbb{R}^{n \times n} A ∈ R n × n (SPD) in some high precision h h h , block size n b n_b n b , tolerance ϵ ′ \epsilon' ϵ ′ , max slice count Ψ ′ \Psi' Ψ ′
Ensure: Lower Cholesky factor L L L (in FP64) such that A = L L T A = LL^T A = L L T (in-place)
diag i ← a i i \operatorname{diag}_i \leftarrow a_{ii} diag i ← a ii for i = 1 , … , n i = 1,\ldots,n i = 1 , … , n \hfill ▹ diag ∈ R n \triangleright\operatorname{diag} \in\mathbb{R}^n ▹ diag ∈ R n : original diagonal of A A A in h h h
diag i u ← a i i \operatorname{diag}^{u}_i \leftarrow a_{ii} diag i u ← a ii for i = 1 , … , n i = 1,\ldots,n i = 1 , … , n \hfill ▹ diag u ∈ R n \triangleright\operatorname{diag}^{u} \in\mathbb{R}^n ▹ diag u ∈ R n , in h h h
m i ← 0 m_i \leftarrow0 m i ← 0 for i = 1 , … , n i = 1,\ldots,n i = 1 , … , n \hfill ▹ m ∈ R n \triangleright m \in\mathbb{R}^n ▹ m ∈ R n in h h h
s i ← 0 s_i \leftarrow0 s i ← 0 for i = 1 , … , n i = 1,\ldots,n i = 1 , … , n \hfill ▹ s ∈ R n \triangleright s \in\mathbb{R}^n ▹ s ∈ R n in h h h
g i ← 0 g_i \leftarrow0 g i ← 0 for i = 1 , … , n i = 1,\ldots,n i = 1 , … , n \hfill ▹ g ∈ R n \triangleright g \in\mathbb{R}^n ▹ g ∈ R n in h h h
c ← 0 c \leftarrow0 c ← 0 \hfill ▹ \triangleright ▹ in h h h
ψ ← Ψ ′ \psi\leftarrow\Psi' ψ ← Ψ ′
for I = 1 , … , n / n b I = 1,\ldots,n/n_b I = 1 , … , n / n b do
o ← ( I − 1 ) n b \quad o \leftarrow(I-1)n_b o ← ( I − 1 ) n b
w ← min ( n b , n − o ) \quad w \leftarrow\min(n_b,n-o) w ← min ( n b , n − o )
A I I ← A [ o + 1 : o + w , o + 1 : o + w ] \quad A_{II} \leftarrow A[o+1:o+w,o+1:o+w] A I I ← A [ o + 1 : o + w , o + 1 : o + w ] \hfill ▹ \triangleright ▹ diagonal block
A I I ← Potrf ( A I I ) \quad A_{II} \leftarrow\operatorname{Potrf}(A_{II}) A I I ← Potrf ( A I I ) \hfill ▹ A I I ← L I I L I I T \triangleright A_{II} \leftarrow L_{II}L_{II}^{T} ▹ A I I ← L I I L I I T in h h h
\quad if o + w < n o+w<n o + w < n then
A I + 1 , I ← A [ o + w + 1 : n , o + 1 : o + w ] \qquad A_{I+1,I} \leftarrow A[o+w+1:n,o+1:o+w] A I + 1 , I ← A [ o + w + 1 : n , o + 1 : o + w ] \hfill ▹ \triangleright ▹ off-diagonal panel
A I + 1 , I ← A I + 1 , I ⋅ A I I − T \qquad A_{I+1,I} \leftarrow A_{I+1,I}\cdot A_{II}^{-T} A I + 1 , I ← A I + 1 , I ⋅ A I I − T \hfill ▹ \triangleright ▹ TRSM is emulated using ψ \psi ψ slices
\qquad for i = o + w + 1 , … , n i=o+w+1,\ldots,n i = o + w + 1 , … , n do \hfill ▹ \triangleright ▹ one pass over row i i i of the panel
g i ← max o < k ≤ o + w ∣ l i k ∣ \qquad\qquad g_i \leftarrow\max_{o<k\leq o+w}|l_{ik}| g i ← max o < k ≤ o + w ∣ l ik ∣
s i ← s i + ∑ k = o + 1 o + w l i k 2 \qquad\qquad s_i \leftarrow s_i+\sum_{k=o+1}^{o+w}l_{ik}^{2} s i ← s i + ∑ k = o + 1 o + w l ik 2
diag i u ← diag i u − ∑ k = o + 1 o + w l i k 2 \qquad\qquad\operatorname{diag}^{u}_i \leftarrow\operatorname{diag}^{u}_i-\sum_{k=o+1}^{o+w}l_{ik}^{2} diag i u ← diag i u − ∑ k = o + 1 o + w l ik 2
\qquad end for
\qquad for p 1 = 1 , … , Ψ ′ p_1=1,\ldots,\Psi' p 1 = 1 , … , Ψ ′ do
\qquad\qquad for p 2 = 1 , … , p 1 p_2=1,\ldots,p_1 p 2 = 1 , … , p 1 do
\qquad\qquad\qquad if Θ ( p 2 , n b ) g i 2 + ( max ( c , Γ ( p 2 ) ) + I + 1 ) ϵ 64 ( s i + a i i ) + γ e m u ( p 1 , Ψ ′ , n − I n b , n b ) 1 − γ e m u ( p 1 , Ψ ′ , n − I n b , n b ) a i i ≤ ϵ ′ a i i − m i \Theta(p_2,n_b)g_i^2+(\max(c,\Gamma(p_2))+I+1)\epsilon_{64}(s_i+a_{ii})+\frac{\gamma_{\mathrm{emu}}(p_1,\Psi',n-In_b,n_b)}{1-\gamma_{\mathrm{emu}}(p_1,\Psi',n-In_b,n_b)}a_{ii}\leq\epsilon' a_{ii}-m_i Θ ( p 2 , n b ) g i 2 + ( max ( c , Γ ( p 2 )) + I + 1 ) ϵ 64 ( s i + a ii ) + 1 − γ emu ( p 1 , Ψ ′ , n − I n b , n b ) γ emu ( p 1 , Ψ ′ , n − I n b , n b ) a ii ≤ ϵ ′ a ii − m i for o + w < i ≤ n o+w<i\leq n o + w < i ≤ n and γ e m u ( p 1 , Ψ ′ , n − I n b , n b ) < 1 \gamma_{\mathrm{emu}}(p_1,\Psi',n-In_b,n_b)<1 γ emu ( p 1 , Ψ ′ , n − I n b , n b ) < 1 then
π g ( I ) ← p 2 , ψ ← p 1 \qquad\qquad\qquad\quad\pi_g(I)\leftarrow p_2,\ \psi\leftarrow p_1 π g ( I ) ← p 2 , ψ ← p 1
\qquad\qquad\qquad\quad goto line 29
\qquad\qquad\qquad end if
\qquad\qquad end for
\qquad end for
c ← max ( c , Γ ( π g ( I ) ) ) \qquad\quad c\leftarrow\max(c,\Gamma(\pi_g(I))) c ← max ( c , Γ ( π g ( I )))
\qquad\quad for i = o + w + 1 , … , n i=o+w+1,\ldots,n i = o + w + 1 , … , n do
m i ← m i + Θ ( π g ( I ) , n b ) g i 2 \qquad\qquad m_i\leftarrow m_i+\Theta(\pi_g(I),n_b)g_i^2 m i ← m i + Θ ( π g ( I ) , n b ) g i 2
\qquad\quad end for
A I + 1 , I + 1 ← A [ o + w + 1 : n , o + w + 1 : n ] \qquad\quad A_{I+1,I+1}\leftarrow A[o+w+1:n,o+w+1:n] A I + 1 , I + 1 ← A [ o + w + 1 : n , o + w + 1 : n ] \hfill ▹ \triangleright ▹ trailing submatrix
A I + 1 , I + 1 ← A I + 1 , I + 1 − A I + 1 , I A I + 1 , I T \qquad\quad A_{I+1,I+1}\leftarrow A_{I+1,I+1}-A_{I+1,I}A_{I+1,I}^{T} A I + 1 , I + 1 ← A I + 1 , I + 1 − A I + 1 , I A I + 1 , I T \hfill ▹ \triangleright ▹ GEMM/SYRK is emulated using π g ( I ) \pi_g(I) π g ( I ) slices
\qquad end if
end for
Theorem 5.1. Consider Algorithm 5 that uses a π g ( I ) \pi_g(I) π g ( I ) -slice Ozaki 1 at iteration I I I for the Schur update and a π t ( I ) \pi_t(I) π t ( I ) -slice Ozaki 1 for the TRSM. Then in Equation (70) , E ′ = E ( 1 ) + E ( 2 ) + E ( 3 ) E' = E^{(1)} + E^{(2)} + E^{(3)} E ′ = E ( 1 ) + E ( 2 ) + E ( 3 ) is symmetric positive semidefinite.
Proof. For 1 ≤ K ≤ I 1 \leq K \leq I 1 ≤ K ≤ I , define L ^ 3 ( K ) = L ^ [ I n b + 1 : n , ( K − 1 ) n b + 1 : K n b ] \hat{L}_3^{(K)}=\hat{L}[In_b+1:n,(K-1)n_b+1:Kn_b] L ^ 3 ( K ) = L ^ [ I n b + 1 : n , ( K − 1 ) n b + 1 : K n b ] . Then L ^ 32 = L ^ 3 ( I ) = L ^ [ I n b + 1 : n , ( I − 1 ) n b + 1 : I n b ] \hat{L}_{32}=\hat{L}_3^{(I)}=\hat{L}[In_b+1:n,(I-1)n_b+1:In_b] L ^ 32 = L ^ 3 ( I ) = L ^ [ I n b + 1 : n , ( I − 1 ) n b + 1 : I n b ] and
L ^ 31 L ^ 31 T = ∑ K = 1 I − 1 L ^ 3 ( K ) L ^ 3 ( K ) T \hat{L}_{31}\hat{L}_{31}^{T}=\sum_{K=1}^{I-1}\hat{L}_3^{(K)}\hat{L}_3^{(K)T} L ^ 31 L ^ 31 T = K = 1 ∑ I − 1 L ^ 3 ( K ) L ^ 3 ( K ) T
where the K K K -th GEMM is done with π g ( K ) \pi_g(K) π g ( K ) slices. Our goal now is to apply (10). For this, we define g ( K ) g^{(K)} g ( K ) as the length n − I n b n-I n_b n − I n b vector containing the ufp max norms of the rows of L ^ 3 ( K ) \widehat{L}_3^{(K)} L 3 ( K ) , or equivalently, the ufp max norms of the columns of L ^ 3 ( K ) T \widehat{L}_3^{(K)T} L 3 ( K ) T . Then
E ( 1 ) + E ( 2 ) = ∑ K = 1 I Θ ( π g ( K ) , n b ) g ( K ) g ( K ) T + ( max K ≤ I Γ ( π g ( K ) ) + I + 1 ) ϵ 64 ( ∑ K = 1 I ∣ L ^ 3 ( K ) ∣ ∣ L ^ 3 ( K ) ∣ T + d d T ) (71) E^{(1)}+E^{(2)}=\sum_{K=1}^{I}\Theta(\pi_g(K),n_b)g^{(K)}g^{(K)T}+\left(\max_{K\le I}\Gamma(\pi_g(K))+I+1\right)\epsilon_{64}\left(\sum_{K=1}^{I}\lvert\widehat{L}_3^{(K)}\rvert\lvert\widehat{L}_3^{(K)}\rvert^T+dd^T\right) \tag*{(71)} E ( 1 ) + E ( 2 ) = K = 1 ∑ I Θ ( π g ( K ) , n b ) g ( K ) g ( K ) T + ( K ≤ I max Γ ( π g ( K )) + I + 1 ) ϵ 64 ( K = 1 ∑ I ∣ L 3 ( K ) ∣ ∣ L 3 ( K ) ∣ T + d d T ) (71)
where d d d is a length n − I n b n-I n_b n − I n b vector with d r = a r + I n b , r + I n b d_r=\sqrt{a_{r+I n_b,r+I n_b}} d r = a r + I n b , r + I n b . We use d d T dd^T d d T instead of ∣ A 33 ∣ \lvert A_{33}\rvert ∣ A 33 ∣ since although A 33 A_{33} A 33 is positive semidefinite, ∣ A 33 ∣ \lvert A_{33}\rvert ∣ A 33 ∣ need not be. But, since A 33 A_{33} A 33 is a principal submatrix of A A A , ∣ A 33 ∣ ≤ d d T \lvert A_{33}\rvert\le dd^T ∣ A 33 ∣ ≤ d d T by Cauchy-Schwarz. In equation (72), g ( K ) g ( K ) T g^{(K)}g^{(K)T} g ( K ) g ( K ) T , ∣ L ^ 3 ( K ) ∣ ∣ L ^ 3 ( K ) ∣ T \lvert\widehat{L}_3^{(K)}\rvert\lvert\widehat{L}_3^{(K)}\rvert^T ∣ L 3 ( K ) ∣ ∣ L 3 ( K ) ∣ T , and d d T dd^T d d T , are all positive semi-definite, and since (72) only has positive coefficients, the sum of all matrices is also positive semidefinite. Next, for E ( 3 ) E^{(3)} E ( 3 ) , we apply Theorem 4.1 to A 33 ( n e x t ) A_{33}^{(\mathrm{next})} A 33 ( next ) (under the assumption that Cholesky is successful). Denote its diagonal entries by a r r u a_{rr}^{u} a r r u for 1 ≤ r ≤ n − I n b 1\le r\le n-I n_b 1 ≤ r ≤ n − I n b . Then e i j ( 3 ) = γ e m u ( ψ , Ψ , n − I n b , n b ) 1 − γ e m u ( ψ , Ψ , n − I n b , n b ) a i i u a j j u e_{ij}^{(3)}=\frac{\gamma_{\mathrm{emu}}(\psi,\Psi,n-I n_b,n_b)}{1-\gamma_{\mathrm{emu}}(\psi,\Psi,n-I n_b,n_b)}\sqrt{a_{ii}^{u}a_{jj}^{u}} e ij ( 3 ) = 1 − γ emu ( ψ , Ψ , n − I n b , n b ) γ emu ( ψ , Ψ , n − I n b , n b ) a ii u a j j u , where ψ \psi ψ and Ψ \Psi Ψ are min J min ( π g ( J ) , π t ( J ) ) \min_J\min(\pi_g(J),\pi_t(J)) min J min ( π g ( J ) , π t ( J )) and max J max ( π g ( J ) , π t ( J ) ) \max_J\max(\pi_g(J),\pi_t(J)) max J max ( π g ( J ) , π t ( J )) respectively for I < J ≤ n / n b I<J\le n/n_b I < J ≤ n / n b . This is also positive semidefinite, since it is another outer product of a vector with itself. □
Since E ′ = E ( 1 ) + E ( 2 ) + E ( 3 ) E'=E^{(1)}+E^{(2)}+E^{(3)} E ′ = E ( 1 ) + E ( 2 ) + E ( 3 ) is positive semidefinite, using Cauchy-Schwarz, we obtain e i j ′ ≤ e i i ′ e j j ′ e_{ij}'\le\sqrt{e_{ii}'e_{jj}'} e ij ′ ≤ e ii ′ e j j ′ . Given a tolerance ϵ ′ \epsilon' ϵ ′ , we want our algorithm to ensure that the computed L ^ \widehat{L} L satisfies ∣ a i j − ∑ k = 1 min ( i , j ) l i k l j k ∣ ≤ ϵ ′ a i i a j j \lvert a_{ij}-\sum_{k=1}^{\min(i,j)}l_{ik}l_{jk}\rvert\le\epsilon'\sqrt{a_{ii}a_{jj}} ∣ a ij − ∑ k = 1 m i n ( i , j ) l ik l j k ∣ ≤ ϵ ′ a ii a j j . This requires e i i ′ e j j ′ ≤ ϵ ′ a i i a j j \sqrt{e_{ii}'e_{jj}'}\le\epsilon'\sqrt{a_{ii}a_{jj}} e ii ′ e j j ′ ≤ ϵ ′ a ii a j j at every iteration. Equivalently, it is much cheaper to ensure that e i i ′ ≤ ϵ ′ a i i e_{ii}'\le\epsilon'a_{ii} e ii ′ ≤ ϵ ′ a ii for every 1 ≤ i ≤ n 1\le i\le n 1 ≤ i ≤ n . Using (72) and e i i ( 3 ) = γ e m u ( ψ , Ψ , n − I n b , n b ) 1 − γ e m u ( ψ , Ψ , n − I n b , n b ) a i i u e_{ii}^{(3)}=\frac{\gamma_{\mathrm{emu}}(\psi,\Psi,n-I n_b,n_b)}{1-\gamma_{\mathrm{emu}}(\psi,\Psi,n-I n_b,n_b)}a_{ii}^{u} e ii ( 3 ) = 1 − γ emu ( ψ , Ψ , n − I n b , n b ) γ emu ( ψ , Ψ , n − I n b , n b ) a ii u , we can write
e i i ′ = ∑ K = 1 I − 1 Θ ( π g ( K ) , n b ) ( g i ( K ) ) 2 + Θ ( π g ( I ) , n b ) ( g i ( I ) ) 2 + ( max K ≤ I Γ ( π g ( K ) ) + I + 1 ) ϵ 64 ( ∑ k = 1 I n b l ^ i k 2 + a i i ) + γ e m u ( ψ , Ψ , n − I n b , n b ) 1 − γ e m u ( ψ , Ψ , n − I n b , n b ) a i i u (72) e_{ii}'=\sum_{K=1}^{I-1}\Theta(\pi_g(K),n_b)(g_i^{(K)})^2+\Theta(\pi_g(I),n_b)(g_i^{(I)})^2+\left(\max_{K\le I}\Gamma(\pi_g(K))+I+1\right)\epsilon_{64}\left(\sum_{k=1}^{I n_b}\widehat{l}_{ik}^{\,2}+a_{ii}\right)+\frac{\gamma_{\mathrm{emu}}(\psi,\Psi,n-I n_b,n_b)}{1-\gamma_{\mathrm{emu}}(\psi,\Psi,n-I n_b,n_b)}a_{ii}^{u} \tag*{(72)} e ii ′ = K = 1 ∑ I − 1 Θ ( π g ( K ) , n b ) ( g i ( K ) ) 2 + Θ ( π g ( I ) , n b ) ( g i ( I ) ) 2 + ( K ≤ I max Γ ( π g ( K )) + I + 1 ) ϵ 64 ( k = 1 ∑ I n b l ik 2 + a ii ) + 1 − γ emu ( ψ , Ψ , n − I n b , n b ) γ emu ( ψ , Ψ , n − I n b , n b ) a ii u (72)
Then, bounding e i i ′ e_{ii}' e ii ′ by ϵ ′ a i i \epsilon'a_{ii} ϵ ′ a ii is equivalent to checking the following inequality
Θ ( π g ( I ) , n b ) ( g i ( I ) ) 2 + ( max K ≤ I Γ ( π g ( K ) ) + I + 1 ) ϵ 64 ( ∑ k = 1 I n b l ^ i k 2 + a i i ) + γ e m u ( ψ , Ψ , n − I n b , n b ) 1 − γ e m u ( ψ , Ψ , n − I n b , n b ) a i i u ≤ ϵ ′ a i i − ∑ K = 1 I − 1 Θ ( π g ( K ) , n b ) ( g i ( K ) ) 2 . (73) \begin{aligned}
\Theta(\pi_g(I),n_b)(g_i^{(I)})^2+\left(\max_{K\le I}\Gamma(\pi_g(K))+I+1\right)\epsilon_{64}\left(\sum_{k=1}^{I n_b}\widehat{l}_{ik}^{\,2}+a_{ii}\right)+\frac{\gamma_{\mathrm{emu}}(\psi,\Psi,n-I n_b,n_b)}{1-\gamma_{\mathrm{emu}}(\psi,\Psi,n-I n_b,n_b)}a_{ii}^{u} \\
&\le\epsilon'a_{ii}-\sum_{K=1}^{I-1}\Theta(\pi_g(K),n_b)(g_i^{(K)})^2. \tag*{(73)}
\end{aligned} Θ ( π g ( I ) , n b ) ( g i ( I ) ) 2 + ( K ≤ I max Γ ( π g ( K )) + I + 1 ) ϵ 64 ( k = 1 ∑ I n b l ik 2 + a ii ) + 1 − γ emu ( ψ , Ψ , n − I n b , n b ) γ emu ( ψ , Ψ , n − I n b , n b ) a ii u ≤ ϵ ′ a ii − K = 1 ∑ I − 1 Θ ( π g ( K ) , n b ) ( g i ( K ) ) 2 . (73)
The blocked Cholesky factorization with adaptive precision is presented in Algorithm 6. This algorithm incorporates an efficient adaptive slice selection and is based on the observation that every quantity required to evaluate e i i ′ e_{ii}' e ii ′ in (73) can be computed with O ( n ⋅ n b ) O(n\cdot n_b) O ( n ⋅ n b ) work. First, all elements a i i a_{ii} a ii can be read from A A A and stored in a separate array of length n n n before starting the factorization. Each ( g i ( K ) ) 2 (g_i^{(K)})^2 ( g i ( K ) ) 2 for 1 ≤ K ≤ n / n b 1\le K\le n/n_b 1 ≤ K ≤ n / n b can be evaluated by computing g i ( K ) = ufp ( max k = ( K − 1 ) n b + 1 K n b ∣ l ^ i k ∣ ) g_i^{(K)}=\operatorname{ufp}\left(\max_{k=(K-1)n_b+1}^{K n_b}\lvert\widehat{l}_{ik}\rvert\right) g i ( K ) = ufp ( max k = ( K − 1 ) n b + 1 K n b ∣ l ik ∣ ) before each GEMM. Instead of the ufp, we use the plain max max ( K − 1 ) n b < k ≤ K n b ∣ l ^ i k ∣ \max_{(K-1)n_b<k\le K n_b}\lvert\widehat{l}_{ik}\rvert max ( K − 1 ) n b < k ≤ K n b ∣ l ik ∣ in Algorithm 6 (and in Algorithm 8); since ufp ( x ) ≤ x \operatorname{ufp}(x)\le x ufp ( x ) ≤ x , this only makes the bound larger. An array of length n n n can be used to store and update at each iteration m i = ∑ K = 1 I − 1 Θ ( π g ( K ) , n b ) ( g i ( K ) ) 2 m_i=\sum_{K=1}^{I-1}\Theta(\pi_g(K),n_b)(g_i^{(K)})^2 m i = ∑ K = 1 I − 1 Θ ( π g ( K ) , n b ) ( g i ( K ) ) 2 . We also need to compute ∑ k = ( K − 1 ) n b + 1 K n b l ^ i k 2 \sum_{k=(K-1)n_b+1}^{K n_b}\widehat{l}_{ik}^{\,2} ∑ k = ( K − 1 ) n b + 1 K n b l ik 2 in every iteration, and this computation can be done in the same pass used to find the maximum. The last quantity we need to compute is a i i u a_{ii}^{u} a ii u . This can also be approximated in the same pass, since we need to approximate a i i − ∑ k = 1 I n b l ^ i k 2 a_{ii}-\sum_{k=1}^{I n_b}\widehat{l}_{ik}^{\,2} a ii − ∑ k = 1 I n b l ik 2 . To get an accurate approximation of this expression, we would essentially need an Ozaki style dot product function, but we find in our experiments that using regular double precision for this operation still gives us error guarantees, hence our algorithm assumes that a i i − ∑ k = 1 I n b l ^ i k 2 a_{ii}-\sum_{k=1}^{I n_b}\hat{l}_{ik}^{2} a ii − ∑ k = 1 I n b l ^ ik 2 is computed in double precision. Since there are n / n b n/n_b n / n b iterations, tracking (72) costs O ( n 2 ) O(n^2) O ( n 2 ) work throughout the course of the O ( n 3 ) O(n^3) O ( n 3 ) Cholesky algorithm. So, computing (72) adds minimal overhead to Algorithm 6.
Thus Algorithm 6 uses five additional arrays of length n n n to monitor the roundoff error. Array diag with diag i = a i i \operatorname{diag}_i=a_{ii} diag i = a ii , diag u \operatorname{diag}^{u} diag u , which is updated at every iteration and stores diag i u = a i i u = a i i − ∑ k l i k 2 \operatorname{diag}^{u}_i=a^{u}_{ii}=a_{ii}-\sum_k l_{ik}^{2} diag i u = a ii u = a ii − ∑ k l ik 2 , array g g g which stores the row-wise ufp max norms of the panel with which we are just about to perform SYRK/GEMM, array m m m which stores the error from truncation as m i = ∑ Θ ( π g ( K ) , n b ) ( g i ( K ) ) 2 m_i=\sum\Theta(\pi_g(K),n_b)(g_i^{(K)})^2 m i = ∑ Θ ( π g ( K ) , n b ) ( g i ( K ) ) 2 , and finally s s s which stores the error from accumulation as s i = ( max K ≤ I Γ ( π g ( K ) ) + I + 1 ) ϵ 64 ( ∑ k = 1 I n b l ^ i k 2 + a i i ) s_i=\left(\max_{K\le I}\Gamma(\pi_g(K))+I+1\right)\epsilon_{64}\left(\sum_{k=1}^{I n_b}\hat{l}_{ik}^{2}+a_{ii}\right) s i = ( max K ≤ I Γ ( π g ( K )) + I + 1 ) ϵ 64 ( ∑ k = 1 I n b l ^ ik 2 + a ii ) . Since computing e i i ′ e'_{ii} e ii ′ is cheap, the goal of our adaptive precision algorithm is to solve (73) for π g ( I ) \pi_g(I) π g ( I ) and ψ \psi ψ in every iteration. We do this in lines 21–28 of Algorithm 6. When we want a double precision Cholesky factorization, we set ϵ ′ = n ϵ 64 \epsilon'=n\epsilon_{64} ϵ ′ = n ϵ 64 and set Ψ ′ = 8 \Psi'=8 Ψ ′ = 8 , but Ψ ′ = 7 \Psi'=7 Ψ ′ = 7 should also work. Also note that we default to 8 slices, not FP64, if we can’t find any solutions to (73) in line 23. Different possible solutions exist for π g ( I ) , ψ \pi_g(I),\psi π g ( I ) , ψ . We consider two possibilities, π g ( I ) = ψ \pi_g(I)=\psi π g ( I ) = ψ and π g ( I ) ≤ ψ \pi_g(I)\le\psi π g ( I ) ≤ ψ , Algorithm 6 displays the latter. We enforce π g ( I ) ≤ ψ \pi_g(I)\le\psi π g ( I ) ≤ ψ in the nested loops in lines 21–28 by using the loop index of the ψ \psi ψ (outer) loop as the upper bound of the π g ( I ) \pi_g(I) π g ( I ) (inner) loop. Solutions with π g ( I ) ≥ ψ \pi_g(I)\ge\psi π g ( I ) ≥ ψ are not interesting since SYRK/GEMM is the computationally expensive part of Cholesky, and TRSM is more numerically sensitive than GEMM. So ideally, TRSM should use at least as many slices as GEMM. Experimentally, we found that choosing a tighter Ψ ′ \Psi' Ψ ′ (for example, 7, which also gives us double precision accuracy for sufficiently large n n n ) tends to make Algorithm 6 slower than choosing Ψ ′ = 8 \Psi'=8 Ψ ′ = 8 since there is less flexibility for GEMMs to utilize lower precisions. Additionally, in our experiments, we find that the choice π g ( I ) = ψ \pi_g(I)=\psi π g ( I ) = ψ gives us the same slice choice at every iteration both with and without equilibration by a diagonal matrix of powers of 2, but π g ( I ) ≤ ψ \pi_g(I)\le\psi π g ( I ) ≤ ψ does not have that property (unless all the diagonal values are equal). Finally, although Algorithm 6 provides the bound ϵ ′ a i i a j j \epsilon'\sqrt{a_{ii}a_{jj}} ϵ ′ a ii a j j , we can also change the algorithm to return a factorization that satisfies ϵ ′ max i a i i \epsilon'\max_i a_{ii} ϵ ′ max i a ii or ϵ ′ ∥ A ∥ \epsilon'\lVert A\rVert ϵ ′ ∥ A ∥ by modifying (73) appropriately.
Adaptive precision with mixed-precision floating-point
The blocked floating-point Cholesky factorization with adaptive precision GEMM is presented in Algorithm 7. It relies on an adaptation of (73) to the floating-point case, which requires using Theorem 4.2 in place of Theorem 4.1, and Theorem 3.6 in place of (10) . The floating-point version of (73) then becomes
τ ( ϵ π g ( I ) ) ( ∑ k = ( I − 1 ) n b + 1 I n b l ^ i k 2 + ν π g ( I ) 2 ( d i ( I ) ) 2 ) + ( n b + 2 + n n b ) ϵ h a i i + γ f p ( ϵ ψ , n − I n b , n b ) ( 1 + λ ψ ) 1 − γ f p ( ϵ ψ , n − I n b , n b ) ( 1 + λ ψ ) ϵ ψ a i i u ≤ ϵ ′ a i i − ∑ K = 1 I − 1 τ ( ϵ π g ( K ) ) ( ∑ k = ( K − 1 ) n b + 1 K n b l ^ i k 2 + ν π g ( K ) 2 ( d i ( K ) ) 2 ) , (74) \tau(\epsilon_{\pi_g(I)})\left(\sum_{k=(I-1)n_b+1}^{I n_b}\hat{l}_{ik}^{2}+\nu_{\pi_g(I)}^{2}(d_i^{(I)})^{2}\right)+\left(n_b+2+\frac{n}{n_b}\right)\epsilon_h a_{ii}+\frac{\gamma_{\mathrm{fp}}(\epsilon_{\psi},n-I n_b,n_b)(1+\lambda_{\psi})}{1-\gamma_{\mathrm{fp}}(\epsilon_{\psi},n-I n_b,n_b)(1+\lambda_{\psi})}\epsilon_{\psi}a^{u}_{ii}
\le\epsilon'a_{ii}-\sum_{K=1}^{I-1}\tau(\epsilon_{\pi_g(K)})\left(\sum_{k=(K-1)n_b+1}^{K n_b}\hat{l}_{ik}^{2}+\nu_{\pi_g(K)}^{2}(d_i^{(K)})^{2}\right), \tag*{(74)} τ ( ϵ π g ( I ) ) k = ( I − 1 ) n b + 1 ∑ I n b l ^ ik 2 + ν π g ( I ) 2 ( d i ( I ) ) 2 + ( n b + 2 + n b n ) ϵ h a ii + 1 − γ fp ( ϵ ψ , n − I n b , n b ) ( 1 + λ ψ ) γ fp ( ϵ ψ , n − I n b , n b ) ( 1 + λ ψ ) ϵ ψ a ii u ≤ ϵ ′ a ii − K = 1 ∑ I − 1 τ ( ϵ π g ( K ) ) k = ( K − 1 ) n b + 1 ∑ K n b l ^ ik 2 + ν π g ( K ) 2 ( d i ( K ) ) 2 , (74)
where
τ ( ϵ p ) = { 2 ϵ p + ϵ p 2 + ( n b + 2 + n n b ) ϵ h ( 1 + ϵ p ) 2 ≈ 2 ϵ p + ϵ p 2 , if p ≠ h , ( n b + 2 + n n b ) ϵ h , if p = h . \tau(\epsilon_p)=
\begin{cases}
2\epsilon_p+\epsilon_p^2+\left(n_b+2+\frac{n}{n_b}\right)\epsilon_h(1+\epsilon_p)^2\approx2\epsilon_p+\epsilon_p^2, & \text{if }p\ne h,\\
\left(n_b+2+\frac{n}{n_b}\right)\epsilon_h, & \text{if }p=h.
\end{cases} τ ( ϵ p ) = ⎩ ⎨ ⎧ 2 ϵ p + ϵ p 2 + ( n b + 2 + n b n ) ϵ h ( 1 + ϵ p ) 2 ≈ 2 ϵ p + ϵ p 2 , ( n b + 2 + n b n ) ϵ h , if p = h , if p = h .
since for p = h p=h p = h the inputs of the GEMM are not rounded, and only the accumulation error and the rounding from applying the scale factors remain. Here d i ( K ) d_i^{(K)} d i ( K ) is the scale factor of row i i i in panel K K K . With microscaling, the scale factor for row i i i in panel K K K is at most ( 1 + ϵ s c a l ) max K ∣ l ^ i k ∣ / β (1+\epsilon_{\mathrm{scal}})\max_K|\hat{l}_{ik}|/\beta ( 1 + ϵ scal ) max K ∣ l ^ ik ∣/ β , so ν π g ( K ) 2 ( d i ( K ) ) 2 ≤ λ g ( K ) ∑ k = ( K − 1 ) n b + 1 K n b l ^ i k 2 \nu_{\pi_g(K)}^2(d_i^{(K)})^2\le\lambda_{g(K)}\sum_{k=(K-1)n_b+1}^{K n_b}\hat{l}_{ik}^2 ν π g ( K ) 2 ( d i ( K ) ) 2 ≤ λ g ( K ) ∑ k = ( K − 1 ) n b + 1 K n b l ^ ik 2 with λ p = ν p 2 ( 1 + ϵ s c a l ) 2 β p 2 \lambda_p=\frac{\nu_p^2(1+\epsilon_{\mathrm{scal}})^2}{\beta_p^2} λ p = β p 2 ν p 2 ( 1 + ϵ scal ) 2 , and so τ ( ϵ π g ( K ) ) ( ∑ k = ( K − 1 ) n b + 1 K n b l ^ i k 2 + ν π g ( K ) 2 ( d i ( K ) ) 2 ) \tau(\epsilon_{\pi_g(K)})(\sum_{k=(K-1)n_b+1}^{K n_b}\hat{l}_{ik}^{2}+\nu_{\pi_g(K)}^2(d_i^{(K)})^2) τ ( ϵ π g ( K ) ) ( ∑ k = ( K − 1 ) n b + 1 K n b l ^ ik 2 + ν π g ( K ) 2 ( d i ( K ) ) 2 ) reduces to τ ( ϵ π g ( K ) ) ( 1 + λ π g ( K ) ) ∑ k = ( K − 1 ) n b + 1 K n b l ^ i k 2 \tau(\epsilon_{\pi_g(K)})(1+\lambda_{\pi_g(K)})\sum_{k=(K-1)n_b+1}^{K n_b}\hat{l}_{ik}^2 τ ( ϵ π g ( K ) ) ( 1 + λ π g ( K ) ) ∑ k = ( K − 1 ) n b + 1 K n b l ^ ik 2 . Without scaling, d i ( K ) = 1 d_i^{(K)}=1 d i ( K ) = 1 and it reduces to τ ( ϵ π g ( K ) ) ( ∑ k = ( K − 1 ) n b + 1 K n b l ^ i k 2 + ν π g ( K ) 2 ) \tau(\epsilon_{\pi_g(K)})(\sum_{k=(K-1)n_b+1}^{K n_b}\hat{l}_{ik}^2+\nu_{\pi_g(K)}^2) τ ( ϵ π g ( K ) ) ( ∑ k = ( K − 1 ) n b + 1 K n b l ^ ik 2 + ν π g ( K ) 2 ) . The a i i u a^{u}_{ii} a ii u term is kept in the MX form of (44) since our experiments use MXFP8 (E4M3) as the lowest precision, but if the lowest precision was FP16, we would use Theorem 4.3 instead. Note that in (74) one can also include the diagonal perturbation errors from Section 4.3, by turning a i i a_{ii} a ii into a i i + f ∗ a_{ii}+f^{*} a ii + f ∗ , and by accounting for the perturbation rule in a i i u a_{ii}^{u} a ii u . But there are two problems with this. If we use GMW81, f ∗ f^{*} f ∗ is O ( n 2 ) O(n^{2}) O ( n 2 ) , and assuming β G M W = 1 \beta^{\mathrm{GMW}}=1 β GMW = 1 , and a i i = 1 a_{ii}=1 a ii = 1 for 1 ≤ i ≤ n 1\leq i\leq n 1 ≤ i ≤ n , for n = 40000 n=40000 n = 40000 , f ∗ f^{*} f ∗ is 1.6 × 10 9 1.6\times10^{9} 1.6 × 1 0 9 which is clearly much larger than 1, making the a i i + f ∗ a_{ii}+f^{*} a ii + f ∗ a very impractical bound. If we used a i i + f ∗ a_{ii}+f^{*} a ii + f ∗ the algorithm would essentially default to using low precision for every operation even for modest ϵ ′ \epsilon' ϵ ′ . The other problem, specifically with GMW81, is the fact that a i i u a_{ii}^{u} a ii u cannot be accurately estimated since it depends on θ G M W \theta^{\mathrm{GMW}} θ GMW which depends on the values a j i u a_{ji}^{u} a j i u for j > i j>i j > i which are not known ahead of time. For these reasons, when using diagonal perturbation methods, we treat (74) as a heuristic rather than a strict error guarantee. If we use clamping instead of GMW81, then we don’t have a concrete f ∗ f^{*} f ∗ . For the floating-point case, since TRSM is not emulated, we don’t have the choice of solving for both GEMM precision and TRSM precision. So, rather than having a nested loop like lines 21 to 28 of Algorithm 6, we have only one loop from line 23 to 28 in Algorithm 7.
The inputs of Algorithm 7 are as follows. The matrix A A A is stored in the high precision h h h , which is also the precision used for POTRF, TRSM, and the accumulation in each GEMM. The list P \mathcal{P} P contains the candidate input formats for the GEMMs, ordered from least to most accurate, and each format p ∈ P p\in\mathcal{P} p ∈ P is described by its machine epsilon ϵ p \epsilon_{p} ϵ p and its underflow threshold μ p \mu_{p} μ p . If no format in P \mathcal{P} P satisfies (74) , the GEMM falls back to precision h h h . The underflow count ν \nu ν is an assumed upper bound on the number of entries per row of L ^ \widehat{L} L that underflow when rounded, as in Theorem 4.2, and β \beta β is the microscaling target of Section 3.4. The last input selects whether a diagonal perturbation is used. Diagonal clamping (Section 4.3) only changes the pivots, so it can be done inside POTRF, but GMW81 needs the whole column below each pivot, so POTRF and TRSM are replaced by a single GMW81 factorization of the tall panel. For GMW81, the parameters β p G M W \beta_{p}^{\mathrm{GMW}} β p GMW and δ p G M W \delta_{p}^{\mathrm{GMW}} δ p GMW are given for each format p p p , and at iteration I I I we use those of π g ( I − 1 ) \pi_{g}(I-1) π g ( I − 1 ) , the format of the previous GEMM. This is because the entries of panel I I I were last updated by that GEMM, so their rounding errors are on the scale of ϵ π g ( I − 1 ) \epsilon_{\pi_{g}(I-1)} ϵ π g ( I − 1 ) . The first panel is not updated by any GEMM, so we set π g ( 0 ) = h \pi_{g}(0)=h π g ( 0 ) = h . The values used in our experiments are given in Section 6.
Experiments
In this section we present numerical experiments for the Cholesky decompositions from Algorithms 6 and 7. We consider two types of matrices.
Matrices of the form A = D U Λ U T D A=DU\Lambda U^{T}D A = D U Λ U T D , where U U U is a random orthonormal matrix of the Haar distribution, D D D is a random diagonal matrix with entries 2 e x p 2^{\mathrm{exp}} 2 exp where exp is sampled uniformly from [ − R , R ] [-R,R] [ − R , R ] , and Λ \Lambda Λ is a diagonal matrix with values that are defined by the distributions in Table 3. For Ozaki 1 experiments, R = 511 R=511 R = 511 , for PCG and Ozaki 2, R = 0 R=0 R = 0 .
Matrices from SuiteSparse [7 ] that we consider dense, and thus use our dense Cholesky algorithm to compute their factorization (to test accuracy, not performance).
In the case of the Ozaki 1, double precision is emulated directly using the cuBLAS 13.2 implementation . Our goal is to ensure that ∣ A − L ^ L ^ T ∣ \lvert A-\widehat{L}\widehat{L}^{T}\rvert ∣ A − L L T ∣ satisfies the standard n e 64 a i i a j j ne_{64}\sqrt{a_{ii}a_{jj}} n e 64 a ii a j j bound of [8 ] . In the case of the low-precision floating-point Cholesky decomposition, rather than directly emulating double precision, we sweep many different values of ϵ ′ \epsilon' ϵ ′ and check the effectiveness of the resulting factorizations as preconditioners for a Preconditioned Conjugate Gradient (PCG) solver. The PCG solves A x = b Ax=b A x = b until double precision accuracy is achieved, i.e., until ∥ b − A x ∥ 2 n ∥ A ∥ 2 ∥ x ∥ 2 ≤ ϵ 64 \frac{\lVert b-Ax\rVert_{2}}{\sqrt{n}\lVert A\rVert_{2}\lVert x\rVert_{2}}\leq\epsilon_{64} n ∥ A ∥ 2 ∥ x ∥ 2 ∥ b − A x ∥ 2 ≤ ϵ 64 is achieved. We check this residual at every iteration.
Algorithm 7 Blocked Floating-Point Cholesky Factorization with Adaptive Precision GEMM
Require: Matrix A ∈ R n × n A \in\mathbb{R}^{n \times n} A ∈ R n × n (SPD) in some high precision h h h , block size n b n_b n b , tolerance ϵ ′ \epsilon' ϵ ′ , list of precisions P \mathcal{P} P ordered from least to most accurate, with machine epsilons ϵ p \epsilon_p ϵ p and underflow thresholds μ p \mu_p μ p for p ∈ P p \in\mathcal{P} p ∈ P , (expected) underflow count ν \nu ν , microscaling target β \beta β , perturbation ∈ { none , diagonal clamping , GMW81 } \in\{\text{none},\text{diagonal clamping},\text{GMW81}\} ∈ { none , diagonal clamping , GMW81 } (with β p G M W , δ p G M W \beta_p^{\mathrm{GMW}},\delta_p^{\mathrm{GMW}} β p GMW , δ p GMW for each p ∈ P p \in\mathcal{P} p ∈ P for GMW81)
Ensure: Lower Cholesky factor L L L (in h h h ) such that A = L L T A = LL^{T} A = L L T (in-place), or A + F = L L T A + F = LL^{T} A + F = L L T with diagonal F ≥ 0 F \ge0 F ≥ 0 for diagonal clamping or GMW81
1: diag i ← a i i \operatorname{diag}_i \leftarrow a_{ii} diag i ← a ii for i = 1 , … , n i = 1,\ldots,n i = 1 , … , n ▹ diag ∈ R n \triangleright\operatorname{diag} \in\mathbb{R}^{n} ▹ diag ∈ R n : original diagonal of A A A in h h h 2: diag i u ← a i i \operatorname{diag}^{u}_i \leftarrow a_{ii} diag i u ← a ii for i = 1 , … , n i = 1,\ldots,n i = 1 , … , n ▹ diag u ∈ R n \triangleright\operatorname{diag}^{u} \in\mathbb{R}^{n} ▹ diag u ∈ R n , in h h h 3: s i ← 0 s_i \leftarrow0 s i ← 0 for i = 1 , … , n i = 1,\ldots,n i = 1 , … , n ▹ s ∈ R n \triangleright s \in\mathbb{R}^{n} ▹ s ∈ R n in h h h 4: t i ← 0 t_i \leftarrow0 t i ← 0 for i = 1 , … , n i = 1,\ldots,n i = 1 , … , n ▹ t ∈ R n \triangleright t \in\mathbb{R}^{n} ▹ t ∈ R n in h h h 5: Ψ ← arg max p ∈ P μ p \Psi\leftarrow\arg\max_{p \in\mathcal{P}} \mu_p Ψ ← arg max p ∈ P μ p 6: π g ( 0 ) ← h \pi_g(0) \leftarrow h π g ( 0 ) ← h 7: for I = 1 , … , n / n b I = 1,\ldots,n/n_b I = 1 , … , n / n b do 8: o ← ( I − 1 ) n b o \leftarrow(I-1)n_b o ← ( I − 1 ) n b 9: w ← min ( n b , n − o ) w \leftarrow\min(n_b,n-o) w ← min ( n b , n − o ) 10: A I I ← A [ o + 1 : o + w , o + 1 : o + w ] A_{II} \leftarrow A[o+1:o+w,o+1:o+w] A I I ← A [ o + 1 : o + w , o + 1 : o + w ] ▹ \triangleright ▹ diagonal block 11: A I + 1 , I ← A [ o + w + 1 : n , o + 1 : o + w ] A_{I+1,I} \leftarrow A[o+w+1:n,o+1:o+w] A I + 1 , I ← A [ o + w + 1 : n , o + 1 : o + w ] ▹ \triangleright ▹ off-diagonal panel (empty if o + w = n o+w=n o + w = n ) 12: if perturbation = = = GMW81 then 13: A [ o + 1 : n , o + 1 : o + w ] ← PanelGMW81 ( A [ o + 1 : n , o + 1 : o + w ] , β π g ( I − 1 ) G M W , δ π g ( I − 1 ) G M W ) A[o+1:n,o+1:o+w] \leftarrow\operatorname{PanelGMW81}(A[o+1:n,o+1:o+w],\beta_{\pi_g(I-1)}^{\mathrm{GMW}},\delta_{\pi_g(I-1)}^{\mathrm{GMW}}) A [ o + 1 : n , o + 1 : o + w ] ← PanelGMW81 ( A [ o + 1 : n , o + 1 : o + w ] , β π g ( I − 1 ) GMW , δ π g ( I − 1 ) GMW ) ▹ \triangleright ▹ in h h h 14: else 15: A I I ← Potrf ( A I I ) A_{II} \leftarrow\operatorname{Potrf}(A_{II}) A I I ← Potrf ( A I I ) ▹ \triangleright ▹ in h h h , optionally with diagonal clamping 16: A I + 1 , I ← A I + 1 , I A I I − T A_{I+1,I} \leftarrow A_{I+1,I}A_{II}^{-T} A I + 1 , I ← A I + 1 , I A I I − T ▹ \triangleright ▹ TRSM in precision h h h 17: end if 18: if o + w < n o+w<n o + w < n then 19: for i = o + w + 1 , … , n i=o+w+1,\ldots,n i = o + w + 1 , … , n do ▹ \triangleright ▹ one pass over row i i i of the panel 20: t i ← ∑ k = o + 1 o + w a i k 2 t_i \leftarrow\sum_{k=o+1}^{o+w} a_{ik}^{2} t i ← ∑ k = o + 1 o + w a ik 2 21: diag i u ← diag i u − t i \operatorname{diag}^{u}_i \leftarrow\operatorname{diag}^{u}_i-t_i diag i u ← diag i u − t i 22: end for 23: for p ∈ P p \in\mathcal{P} p ∈ P do 24: if τ ( ϵ p ) ( 1 + λ p ) t i + ( n b + 2 + n n b ) ϵ h a i i + γ f p ( ϵ p , n − I n b , n b ) ( 1 + λ Ψ ) 1 − γ f p ( ϵ p , n − I n b , n b ) ( 1 + λ Ψ ) a i i u ≤ ϵ ′ a i i − s i \tau(\epsilon_p)(1+\lambda_p)t_i+\left(n_b+2+\frac{n}{n_b}\right)\epsilon_h a_{ii}+\frac{\gamma_{\mathrm{fp}}(\epsilon_p,n-In_b,n_b)(1+\lambda_{\Psi})}{1-\gamma_{\mathrm{fp}}(\epsilon_p,n-In_b,n_b)(1+\lambda_{\Psi})}a_{ii}^{u}\le\epsilon'a_{ii}-s_i τ ( ϵ p ) ( 1 + λ p ) t i + ( n b + 2 + n b n ) ϵ h a ii + 1 − γ fp ( ϵ p , n − I n b , n b ) ( 1 + λ Ψ ) γ fp ( ϵ p , n − I n b , n b ) ( 1 + λ Ψ ) a ii u ≤ ϵ ′ a ii − s i for o + w < i ≤ n o+w<i\le n o + w < i ≤ n and γ f p ( ϵ p , n − I n b , n b ) ( 1 + λ Ψ ) < 1 \gamma_{\mathrm{fp}}(\epsilon_p,n-In_b,n_b)(1+\lambda_{\Psi})<1 γ fp ( ϵ p , n − I n b , n b ) ( 1 + λ Ψ ) < 1 then 25: π g ( I ) ← p , ψ ← p \pi_g(I)\leftarrow p,\psi\leftarrow p π g ( I ) ← p , ψ ← p 26: goto line 29 27: end if 28: end for 29: for i = o + w + 1 , … , n i=o+w+1,\ldots,n i = o + w + 1 , … , n do 30: s i ← s i + τ ( ϵ π g ( I ) ) ( 1 + λ π g ( I ) ) t i s_i\leftarrow s_i+\tau(\epsilon_{\pi_g(I)})(1+\lambda_{\pi_g(I)})t_i s i ← s i + τ ( ϵ π g ( I ) ) ( 1 + λ π g ( I ) ) t i 31: end for 32: A I + 1 , I + 1 ← A [ o + w + 1 : n , o + w + 1 : n ] A_{I+1,I+1}\leftarrow A[o+w+1:n,o+w+1:n] A I + 1 , I + 1 ← A [ o + w + 1 : n , o + w + 1 : n ] ▹ \triangleright ▹ trailing submatrix 33: microscale A I + 1 , I A_{I+1,I} A I + 1 , I and round it to format π g ( I ) \pi_g(I) π g ( I ) 34: A I + 1 , I + 1 ← A I + 1 , I + 1 − A I + 1 , I A I + 1 , I T A_{I+1,I+1}\leftarrow A_{I+1,I+1}-A_{I+1,I}A_{I+1,I}^{T} A I + 1 , I + 1 ← A I + 1 , I + 1 − A I + 1 , I A I + 1 , I T ▹ \triangleright ▹ MX GEMM/SYRK, accumulating in h h h 35: end if 36: end for
Ozaki 1 Emulation on Synthetic Matrices and SuiteSparse
We first discuss the performance of the blocked Cholesky factorization with an adaptive number of slices based on the Ozaki 1 scheme, described in Algorithm 6. We use the cuBLAS Ozaki 1 implementation and our experiments are run on a GH200 with the CUDA toolkit version 13.2. CUDA 13.2 provides emulated DSYRK only on Blackwell and not on Hopper. However, emulated DGEMM is available on Hopper. Thus, in our experiments, the Cholesky factorization is implemented using emulated DGEMM instead of DSYRK. Because our DGEMM-based implementation does not exploit symmetry, it performs twice as many floating-point operations as a DSYRK-based implementation. Although this roughly doubles the wall-clock times, the relative speedups should remain comparable if DGEMM is replaced by DSYRK. We compare four emulation variants: fixed 7 slice Ozaki 1, fixed 8 slice Ozaki 1, cuBLAS Automatic Dynamic Precision (Ozaki ADP) and our adaptive factorization from Algorithm 6 (labeled Custom Adaptive in Figure 1 , Figure 2 , Figure 3 , 4).
Figure 1. Performance in GFLOPs = ( n 3 / 3 ) (n^3/3) ( n 3 /3 ) /factorization_time, of the Ozaki 1 emulation schemes on the synthetic distributions.
Figure 2. Slice (mantissa bit) profile per panel for Ozaki 1 on the synthetic distributions.
Figure 3. Backward error, max i , j ∣ a i j − ∑ k l i k l j k ∣ / a i i a j j \max_{i,j} \lvert a_{ij} - \sum_k l_{ik}l_{jk}\rvert/\sqrt{a_{ii}a_{jj}} max i , j ∣ a ij − ∑ k l ik l j k ∣ / a ii a j j , of the Ozaki 1 emulation schemes on the synthetic distributions.
As all the emulation variants are slower than the traditional floating-point DGEMM available on Hopper, we restrict our performance comparison to emulation based factorizations. CUBLAS_EMULATION_STRATEGY is set to eager so that DGEMMs and DTRSMs always use the Ozaki 1 scheme. For the synthetic matrices in § 6.1 , Algorithm 6 uses a block size of 2048. Figure 1 displays the performance of all four algorithms for the synthetic matrices in our test set as their dimension n n n increases. We use n = 2048 ∗ i n = 2048 * i n = 2048 ∗ i where i ∈ [ 1 , 2 , 3 , 4 , 6 , 8 , 12 , 16 , 20 ] i \in[1, 2, 3, 4, 6, 8, 12, 16, 20] i ∈ [ 1 , 2 , 3 , 4 , 6 , 8 , 12 , 16 , 20 ] . The condition number is 10 14 10^{14} 1 0 14 for the first three distributions of Table 3 , and 10 12 10^{12} 1 0 12 for the half-half distribution (since 10 14 10^{14} 1 0 14 failed in this case even for native FP64). Figure 2 presents the number of slices (or bits) selected for each GEMM and TRSM by our adaptive algorithm and the number of slices (bits) selected by cuBLAS ADP for each GEMM. We report both the number of slices and the number of bits, since the cuBLAS API tracks the number of bits required for Ozaki 1. We refer to the sequence of slice counts used for GEMM and TRSM at successive iterations as the bit profile of the matrix. Figure 2 displays the bit profile of matrices with n = 40960 n = 40960 n = 40960 and the largest condition number for which the Cholesky factorization did not fail in our experiments. The number on top of each graph represents the speedup of our adaptive algorithm over cuBLAS ADP. As can be seen from these bit profiles, ADP chooses 8 – 10 slices for all GEMMs in the factorization, while the adaptive algorithm chooses fewer slices when the matrix has more rapidly decaying eigenvalues, as in the case of geometric and half-half eigenvalue distributions. Even for the more "uniform" cases of arithmetic and random distributions, the adaptive algorithm chooses a much smaller slice count for GEMMs, and uses mostly 7 slices for TRSMs, and thus speedups are obtained for these cases as well. Figure 3 displays the backward error obtained by the different Ozaki 1 emulation schemes. Our algorithm leads to a backward error smaller than the limit n ϵ 64 n\epsilon_{64} n ϵ 64 . The loss in accuracy relative to native FP64 is higher for the arithmetic and random distributions compared to the geometric and half-half distributions.
Table 3. Eigenvalue distributions used for our synthetic matrices of dimension n × n n \times n n × n and condition number κ \kappa κ .
Figure 4 displays the results obtained for matrices from the SuiteSparse collection. For the SuiteSparse experiments, we use Algorithm 6 with a block size of 512. Ozaki Fixed-7 is the fastest, Ozaki Fixed-8 is the slowest, and for matrices with n > 10 4 n > 10^4 n > 1 0 4 , Ozaki Fixed-7 is 1.00 to 1.02 times faster than Custom Adaptive, and 1.12 to 1.15 times faster than Ozaki Fixed-8. Additionally, all 3 variants produce nearly identical backward errors, in the range from 1.4 × 10 − 4 1.4 \times10^{-4} 1.4 × 1 0 − 4 to 1.4 × 10 − 2 1.4 \times10^{-2} 1.4 × 1 0 − 2 times the target upper bound on n ϵ 64 n\epsilon_{64} n ϵ 64 . We do not include a slice count for cuBLAS ADP since it returned very high slice counts for the SuiteSparse matrices, which caused the algorithm to default to regular FP64. This is likely because ADP is run for each GEMM and returns a larger number of slices when the dynamic range of each matrix (max i j ∣ a i j ∣ / min i j : a i j ≠ 0 ∣ a i j ∣ \max_{ij} \lvert a_{ij}\rvert/\min_{ij:a_{ij}\ne0}\lvert a_{ij}\rvert max ij ∣ a ij ∣ / min ij : a ij = 0 ∣ a ij ∣ ) is large. For most of these matrices, the number of GEMM slices used in our adaptive algorithm remains close to 7. The number of TRSM slices stays at 8 for some of the SuiteSparse examples (bcsstk14, crystm01, bcsstk16 and s3rmt3m3), but falls to 7 for all other SuiteSparse matrices that we tested. In Figure 5 we display the average number of slices used for GEMMs and TRSMs for the entire factorization. Note that the average is unweighted. So, the average slice count for the GEMMs is computed as 1 ⌈ n / n b ⌉ ∑ K = 1 ⌈ n / n b ⌉ π g ( K ) \frac{1}{\lceil n/n_b\rceil}\sum_{K=1}^{\lceil n/n_b\rceil}\pi_g(K) ⌈ n / n b ⌉ 1 ∑ K = 1 ⌈ n / n b ⌉ π g ( K ) . Thus, the performance of Algorithm 6 on SuiteSparse matrices corresponds to that of an algorithm that uses a fixed number of 7 slices.
Figure 5. average number of slices for GEMMs and TRSMs on SuiteSparse matrices
Algorithm 7 with PCG on Synthetic Matrices and SuiteSparse
In the case of the low-precision floating-point Cholesky decomposition in Algorithm 7, we use the produced factorization as a preconditioner for the preconditioned Conjugate Gradient (PCG) to solve a linear system A x = b Ax=b A x = b . For the SuiteSparse matrices, before factorization, the matrix A A A is equilibrated using a diagonal matrix D D D whose diagonal elements d i i d_{ii} d ii are chosen to be the power of two closest to a i i \sqrt{a_{ii}} a ii . The equilibrated matrix H = D − 1 A D − 1 H=D^{-1}AD^{-1} H = D − 1 A D − 1 is then factored using Algorithm 7 to obtain L ^ \hat{L} L ^ . We do not equilibrate the synthetic matrices. For the synthetic matrices, CG on equilibrated matrices does not yield much benefit over CG on the original matrix. We consider microscaling with two different block sizes and scaling formats for the case where FP8 (E4M3) is used in Algorithm 7. The first strategy, described in Section 3.4, is OCP MXFP8-style scaling with a block size of 32 along each row and E8M0 scale factors. Since cuBLAS does not support this on Hopper, we use the LoFloat simulator [34 ] , along with Algorithm 4 to simulate the MX GEMM. The second strategy uses a single FP32 scale factor for each row of the panel of L L L (this scaled GEMM is available in cuBLASLt). For both scaling methods, β \beta β is chosen as 448, the largest finite number in E4M3. The amount of underflow per row at n = 16384 n=16384 n = 16384 for both strategies was smaller than 10 for all synthetic test matrices, and we did not see any difference in the quality of the preconditioner. All our experiments implement Algorithm 7 + GMW81 with a block size of 256. We let the PCG run for either 30000 iterations or until we reach
∥ b − A x ∥ 2 n ∥ A ∥ 2 ∥ x ∥ 2 ≤ ϵ 64 \frac{\lVert b-Ax\rVert_2}{\sqrt{n}\lVert A\rVert_2\lVert x\rVert_2} \le\epsilon_{64} n ∥ A ∥ 2 ∥ x ∥ 2 ∥ b − A x ∥ 2 ≤ ϵ 64
We set FP32 to be the highest available precision and perform panel factorization in FP32. Finding the best ϵ ′ \epsilon' ϵ ′ is a tuning problem that is beyond the scope of this paper. The iteration count and the distribution of FLOPs across Cholesky and PCG for synthetic matrices is shown in Figure 6 . We measure the total number of FLOPs in each precision plus the number of PCG iterations needed for convergence. If there is no point in a bar, that means that the algorithm failed to converge within 30000 iterations. We see that an FP8 preconditioner leads to quick convergence for the arithmetic cases, and it can somewhat speed up convergence for the
Figure 6. PCG iterations and FLOP share per precision for the Geometric (top) and Arithmetic (bottom) distributions. FLOP share is computed as (#FLOPs in precision X)/(total number of FLOPs)
Figure 4 (PDF p. 32). Performance in GFLOPs = ( n 3 / 3 ) (n^3/3) ( n 3 /3 ) /factorization_time of fixed-7, fixed-8 and Adaptive on selected SuiteSparse matrices
geometric case if the condition number is small enough. However, for large condition numbers, FP8 fails to produce a good preconditioner when singular values have the geometric distribution. We don’t present the random case since it behaves the same way as the arithmetic case, and we don’t present the half-half case since it is very easy for CG to solve without any preconditioning. These results suggest that a preconditioner produced using FP8 GEMMs may be useful for some matrices as long as the FP8 GEMMs are fast enough to make up for the cost of increased number of PCG iterations performed in double precision.
For SuiteSparse, we test on 13 matrices with n < 16384 n < 16384 n < 16384 using different ϵ ′ \epsilon^\prime ϵ ′ . The list of matrices is bcsstk14, bcsstk15, crystm01, bcsstk16, s3rmt3m3, bcsstk38, msc10848, bcsstk17, t2dah_e, bcsstk18, crystm02, Pres_Poisson, Dubcova1. Similarly to the Ozaki 1 case, the SuiteSparse matrices do not show any interesting distribution of precisions, and the factorizations choose mainly one precision for the whole factorization. For SuiteSparse, ϵ ′ \epsilon^\prime ϵ ′ falls into three possible ranges: ϵ ′ ≤ 10 − 3 \epsilon^\prime\le10^{-3} ϵ ′ ≤ 1 0 − 3 where all GEMMs are computed in single precision, 10 − 3 < ϵ ′ < 0.125 10^{-3} < \epsilon^\prime< 0.125 1 0 − 3 < ϵ ′ < 0.125 where all GEMMs are in FP16, and ϵ ′ ≥ 0.125 \epsilon^\prime\ge0.125 ϵ ′ ≥ 0.125 where all GEMMs are in FP8. So, rather than presenting the number of iterations for each ϵ ′ \epsilon^\prime ϵ ′ , we present the number of iterations for each precision. We find that the combination of MXFP8 and GMW81 is able to form an effective preconditioner for many matrices. Figure 7 displays the number of PCG iterations required for each precision to reach double precision on all the SuiteSparse matrices mentioned above. As seen in Figure 7 , using low-precision GEMMs along with diagonal perturbation can form an effective preconditioner for Conjugate Gradient on many SuiteSparse matrices.
Figure 7. Number of PCG iterations for SuiteSparse matrices.
In our experiments, Algorithm 7 uses the GMW81 path described in § 4.3 since it outperformed diagonal clamping. We use the "optimal" β G M W \beta^{\mathrm{GMW}} β GMW suggested in [10 ] . GMW81 outperformed the diagonal For synthetic matrices, we set δ F P 8 G M W \delta_{\mathrm{FP8}}^{\mathrm{GMW}} δ FP8 GMW and δ F P 16 G M W \delta_{\mathrm{FP16}}^{\mathrm{GMW}} δ FP16 GMW to the unit roundoff of each format (2 − 4 2^{-4} 2 − 4 and 2 − 11 2^{-11} 2 − 11 ). Doing this for FP32 led to some failures for the ill-conditioned geometric synthetic matrices. However, increasing δ F P 32 G M W \delta_{\mathrm{FP32}}^{\mathrm{GMW}} δ FP32 GMW to 2 − 18 2^{-18} 2 − 18 seems to fix this. We use the same δ F P 32 G M W \delta_{\mathrm{FP32}}^{\mathrm{GMW}} δ FP32 GMW for SuiteSparse. To choose δ F P 8 G M W \delta_{\mathrm{FP8}}^{\mathrm{GMW}} δ FP8 GMW for SuiteSparse, we sweep [ 1.0 , 0.5 , 0.3 , 0.1 ] [1.0, 0.5, 0.3, 0.1] [ 1.0 , 0.5 , 0.3 , 0.1 ] and find that δ F P 8 G M W = 0.5 \delta_{\mathrm{FP8}}^{\mathrm{GMW}} = 0.5 δ FP8 GMW = 0.5 works best. For δ F P 16 G M W \delta_{\mathrm{FP16}}^{\mathrm{GMW}} δ FP16 GMW , we sweep [ 10 − 3 , 3 × 10 − 3 , 10 − 2 , 3 × 10 − 2 ] [10^{-3}, 3 \times10^{-3}, 10^{-2}, 3 \times10^{-2}] [ 1 0 − 3 , 3 × 1 0 − 3 , 1 0 − 2 , 3 × 1 0 − 2 ] and find that 3 × 10 − 3 3 \times10^{-3} 3 × 1 0 − 3 works best across our test set. Note that the best value of δ G M W \delta^{\mathrm{GMW}} δ GMW can depend on the matrix. For example, δ F P 8 G M W = 0.3 \delta_{\mathrm{FP8}}^{\mathrm{GMW}} = 0.3 δ FP8 GMW = 0.3 gives a better preconditioner than δ F P 8 G M W = 0.5 \delta_{\mathrm{FP8}}^{\mathrm{GMW}} = 0.5 δ FP8 GMW = 0.5 on many of the tested SuiteSparse matrices, but on bcsstk38 it is worse than CG on the equilibrated A.
Conclusions and Future Work
We have presented an analysis of floating-point mixed-precision Cholesky and a Cholesky performed via Ozaki 1 fixed-point emulation. We have also presented an algorithm to dynamically reduce precision/slices throughout the Cholesky factorization. For emulation, we show up to 29% speedup over using 7 fixed slices throughout the factorization, and up to 69% speedup over cuBLAS ADP. We also use the adaptive Cholesky, with a diagonal perturbation to handle factorization failures, to construct preconditioners for PCG: on the 13 SuiteSparse matrices we tested, an FP8 preconditioner reduces the number of PCG iterations by up to 12 × 12 \times 12 × over CG on our test matrices. Future work includes constructing stress tests to see if the error bounds in § 4 are attainable. Tighter bounds could let us use low precision more often, thus giving us more speedup. Developing a high performance implementation and adapting our algorithm to the left-looking case is also of interest. Applying our algorithms to CholeskyQR, or developing similar algorithms for LU also seem like promising directions for future work.
AI usage disclosure
Claude Opus 5 and Claude Opus 5.5 were used to improve the writing in some places, and get rid of notational collisions throughout the paper. The LLMs were also able to find an error in the proof of Theorem 4.2 which allowed the authors to remove an extra ( 1 + ϵ h ) (1+\epsilon_h) ( 1 + ϵ h ) factor from the bound. They were also used to unify the proofs of Theorem 4.2 and Theorem 4.3 given an initial proof sketch of how to unify the old versions of 4.2 and 4.3 (which treated both MX and no MX cases separately). LLMs were also used to set up and run the experiments on the Alps cluster. ChatGPT 6 Astra-Pro and Claude Opus 5.5 were also used to review the paper. All code and proofs were checked by the authors.
Acknowledgments
The experiments performed in this work on Alps were supported through a grant from the Swiss National Supercomputing Centre (CSCS) under project RandESC.
Appendix
Adaptive precision with Ozaki 2
In the appendix, we derive an adaptive algorithm for Ozaki 2 using the same methods we used in § 5 . Results from our experiments are presented in Section 9.2. The resulting algorithm is Algorithm 8. For the GEMM error bound, we use only the dominant O ( t ) O(t) O ( t ) terms from Equation (16) in [36 ] . The paper uses, t = 1 2 5 ( ∏ ℓ = 1 N p ℓ − 1 ) t = \frac{1}{\sqrt{2^{5}(\prod_{\ell=1}^{N}p_{\ell}-1)}} t = 2 5 ( ∏ ℓ = 1 N p ℓ − 1 ) 1 , where p ℓ p_{\ell} p ℓ is the ℓ \ell ℓ -th largest coprime in the list of coprime numbers from Equation (10) of [36 ] , and N N N being the number of coprimes used by the GEMM. We are interested in controlling how many co-prime numbers we need at any given iteration of Algorithm 5. So, rather than just t t t , we will use the notation t ( N ) = 1 2 5 ( ∏ ℓ = 1 N p ℓ − 1 ) t(N) = \frac{1}{\sqrt{2^{5}(\prod_{\ell=1}^{N}p_{\ell}-1)}} t ( N ) = 2 5 ( ∏ ℓ = 1 N p ℓ − 1 ) 1 . Now, to understand the error analysis of a Cholesky with Ozaki 2, it is sufficient to see what the error on the diagonal, ∣ e i i ∣ = ∣ a i i − ∑ k = 1 i − 1 l ^ i k 2 ∣ |e_{ii}| = |a_{ii} - \sum_{k=1}^{i-1}\hat{l}_{ik}^{2}| ∣ e ii ∣ = ∣ a ii − ∑ k = 1 i − 1 l ^ ik 2 ∣ looks like. The rest of the proof will follow the exact same steps as the proof of Theorem 4.1 . The dominant term of the bound from [36 ] , Eq. (16), is t ( N ) ( ∣ A ∣ 1 ( 2 β T + 2 α ′ ν T ∣ B ∣ ) t(N)(|A| \mathbf{1} (2\beta^{T}+2\alpha^{\prime}\nu^{T}|B|) t ( N ) ( ∣ A ∣ 1 ( 2 β T + 2 α ′ ν T ∣ B ∣ ) , with ν = ( 1 , … , 1 ) T \nu= (1,\ldots,1)^{T} ν = ( 1 , … , 1 ) T . Where, all terms are defined as in [36 ] . Since we are concerned with GEMMs of the form L ^ I K L ^ I K T \hat{L}_{IK}\hat{L}_{IK}^{T} L ^ I K L ^ I K T , A = B T = L ^ I K A = B^{T} = \hat{L}_{IK} A = B T = L ^ I K , α = β \alpha= \beta α = β . For A ∈ R m × k A \in\mathbb{R}^{m \times k} A ∈ R m × k and A T = B A^{T} = B A T = B
α i ′ = α i + 1 2 q i , α i = ⌈ log 2 max 1 ≤ h ≤ k ∣ a i h ∣ ⌉ , q i = log 2 max 1 ≤ j ≤ n c ˉ i j , C ~ = ∣ diag ( 2 5 − α ) ∣ A ∣ ∣ ∣ A T ∣ diag ( 2 5 − α ) ∣ ∣ , \begin{aligned}
\alpha_i^{\prime} &= \alpha_i + \frac{1}{2}q_i, \qquad\alpha_i = \left\lceil\log_2 \max_{1 \le h \le k}|a_{ih}| \right\rceil, \qquad q_i = \log_2 \max_{1 \le j \le n}\bar{c}_{ij},\\
\widetilde{C} &= \left|\operatorname{diag}(2^{5-\alpha})|A|\right| \left|A^{T}\left|\operatorname{diag}(2^{5-\alpha})\right|\right|,
\end{aligned} α i ′ C = α i + 2 1 q i , α i = ⌈ log 2 1 ≤ h ≤ k max ∣ a ih ∣ ⌉ , q i = log 2 1 ≤ j ≤ n max c ˉ ij , = diag ( 2 5 − α ) ∣ A ∣ A T diag ( 2 5 − α ) ,
where q q q is the vector e ′ e' e ′ of [36 ] . So, we can write the inequality on the block diagonal as
∣ A I I − ∑ K = 1 I L ^ I K L ^ I K T ∣ ≤ ∑ K = 1 I − 1 t ( π g ( K ) ) ( ∣ L ^ I K ∣ v ( 2 α ′ ( K ) ) T + 2 α ′ ( K ) v T ∣ L ^ I K T ∣ ) + n b ϵ h Ω I I , (75) \left|A_{II}-\sum_{K=1}^{I}\hat{L}_{IK}\hat{L}_{IK}^{T}\right|\leq\sum_{K=1}^{I-1}t(\pi_g(K))\left(\left|\hat{L}_{IK}\right|v\left(2^{\alpha'(K)}\right)^{T}+2^{\alpha'(K)}v^{T}\left|\hat{L}_{IK}^{T}\right|\right)+n_b\epsilon_h\Omega_{II}, \tag*{(75)} A I I − K = 1 ∑ I L ^ I K L ^ I K T ≤ K = 1 ∑ I − 1 t ( π g ( K )) ( L ^ I K v ( 2 α ′ ( K ) ) T + 2 α ′ ( K ) v T L ^ I K T ) + n b ϵ h Ω I I , (75)
where α ′ ( K ) = α ( K ) + 1 2 q ( K ) \alpha'(K)=\alpha(K)+\frac{1}{2}q(K) α ′ ( K ) = α ( K ) + 2 1 q ( K ) is the vector α ′ \alpha' α ′ of [36 ] for the GEMM with panel K K K , and q ( K ) q(K) q ( K ) denotes the vector e ′ ( = f ′ ) e'(=f') e ′ ( = f ′ ) of [36 ] . For row i i i of block I I I , the i i i -th entry of both ∣ L ^ I K ∣ v \left|\hat{L}_{IK}\right|v L ^ I K v and v T ∣ L ^ I K T ∣ v^{T}\left|\hat{L}_{IK}^{T}\right| v T L ^ I K T is r i ( K ) = ∑ k = ( K − 1 ) n b + 1 K n b ∣ l ^ i k ∣ r_i^{(K)}=\sum_{k=(K-1)n_b+1}^{Kn_b}|\hat{l}_{ik}| r i ( K ) = ∑ k = ( K − 1 ) n b + 1 K n b ∣ l ^ ik ∣ , so the ( i , i ) (i,i) ( i , i ) entry of each of the two terms in (76) is t ( π g ( K ) ) 2 α i ′ ( K ) r i ( K ) t(\pi_g(K))2^{\alpha'_i(K)}r_i^{(K)} t ( π g ( K )) 2 α i ′ ( K ) r i ( K ) . On the diagonal element, we then get
∣ e i i ∣ ≤ ∑ K = 1 I − 1 2 t ( π g ( K ) ) 2 α i ′ ( K ) r i ( K ) + n b ϵ h a i i u = ∑ K = 1 I − 1 2 t ( π g ( K ) ) 2 α i ( K ) 2 q i ( K ) / 2 r i ( K ) + n b ϵ h a i i u . (76) |e_{ii}|\leq\sum_{K=1}^{I-1}2t(\pi_g(K))2^{\alpha'_i(K)}r_i^{(K)}+n_b\epsilon_h a_{ii}^{u}=\sum_{K=1}^{I-1}2t(\pi_g(K))2^{\alpha_i^{(K)}}2^{q_i^{(K)}/2}r_i^{(K)}+n_b\epsilon_h a_{ii}^{u}. \tag*{(76)} ∣ e ii ∣ ≤ K = 1 ∑ I − 1 2 t ( π g ( K )) 2 α i ′ ( K ) r i ( K ) + n b ϵ h a ii u = K = 1 ∑ I − 1 2 t ( π g ( K )) 2 α i ( K ) 2 q i ( K ) /2 r i ( K ) + n b ϵ h a ii u . (76)
Getting an upper bound on 2 α i ( K ) 2^{\alpha_i^{(K)}} 2 α i ( K ) is straightforward, since α i ( K ) = ⌈ log 2 max ( K − 1 ) n b < k ≤ K n b ∣ l ^ i k ∣ ⌉ ⟹ 2 α i ( K ) ≤ g i ( K ) \alpha_i^{(K)}=\lceil\log_2\max_{(K-1)n_b<k\leq Kn_b}|\hat{l}_{ik}|\rceil\Longrightarrow2^{\alpha_i^{(K)}}\leq g_i^{(K)} α i ( K ) = ⌈ log 2 max ( K − 1 ) n b < k ≤ K n b ∣ l ^ ik ∣ ⌉ ⟹ 2 α i ( K ) ≤ g i ( K ) , where g i ( K ) = max ( K − 1 ) n b < k ≤ K n b ∣ l ^ i k ∣ g_i^{(K)}=\max_{(K-1)n_b<k\leq Kn_b}|\hat{l}_{ik}| g i ( K ) = max ( K − 1 ) n b < k ≤ K n b ∣ l ^ ik ∣ . Now, q ( K ) q^{(K)} q ( K ) is computed by first scaling ∣ L ^ I K ∣ |\hat{L}_{IK}| ∣ L ^ I K ∣ by the diagonal matrix diag ( 2 5 − α ( K ) ) \operatorname{diag}(2^{5-\alpha^{(K)}}) diag ( 2 5 − α ( K ) ) , and computing C ˉ = ⌈ diag ( 2 5 − α ( K ) ) ∣ L ^ I K ∣ ⌉ ⌈ diag ( 2 5 − α ( K ) ) ∣ L ^ I K ∣ ⌉ T \bar{C}=\lceil\operatorname{diag}(2^{5-\alpha^{(K)}})|\hat{L}_{IK}|\rceil\lceil\operatorname{diag}(2^{5-\alpha^{(K)}})|\hat{L}_{IK}|\rceil^{T} C ˉ = ⌈ diag ( 2 5 − α ( K ) ) ∣ L ^ I K ∣ ⌉ ⌈ diag ( 2 5 − α ( K ) ) ∣ L ^ I K ∣ ⌉ T . We then compute q i ( K ) = log 2 max h c ˉ i h q_i^{(K)}=\log_2\max_h\bar{c}_{ih} q i ( K ) = log 2 max h c ˉ ih . Since 2 5 − α i ( K ) ∣ l ^ i k ∣ < 2 6 2^{5-\alpha_i^{(K)}}|\hat{l}_{ik}|<2^6 2 5 − α i ( K ) ∣ l ^ ik ∣ < 2 6 , every entry of ⌈ diag ( 2 5 − α ( K ) ) ∣ L ^ I K ∣ ⌉ \lceil\operatorname{diag}(2^{5-\alpha^{(K)}})|\hat{L}_{IK}|\rceil ⌈ diag ( 2 5 − α ( K ) ) ∣ L ^ I K ∣ ⌉ lies in [ 0 , 2 6 ] [0,2^6] [ 0 , 2 6 ] , so c ˉ h h ≤ 2 12 n b \bar{c}_{hh}\leq2^{12}n_b c ˉ hh ≤ 2 12 n b for every h h h . Since C ˉ \bar{C} C ˉ is positive semidefinite, c ˉ i h ≤ c ˉ i i c ˉ h h ≤ 2 12 n b \bar{c}_{ih}\leq\sqrt{\bar{c}_{ii}\bar{c}_{hh}}\leq2^{12}n_b c ˉ ih ≤ c ˉ ii c ˉ hh ≤ 2 12 n b . Then, we get
2 q i ( K ) / 2 ≤ 2 6 n b ⟹ 2 α i ′ ( K ) ≤ 2 6 n b g i ( K ) . (77) 2^{q_i^{(K)}/2}\leq2^6\sqrt{n_b}\Longrightarrow2^{\alpha'_i(K)}\leq2^6\sqrt{n_b}g_i^{(K)}. \tag*{(77)} 2 q i ( K ) /2 ≤ 2 6 n b ⟹ 2 α i ′ ( K ) ≤ 2 6 n b g i ( K ) . (77)
Finally, since every ∣ l ^ i k ∣ ≤ g i ( K ) |\hat{l}_{ik}|\leq g_i^{(K)} ∣ l ^ ik ∣ ≤ g i ( K ) in panel K K K , r i ( K ) ≤ n b g i ( K ) r_i^{(K)}\leq n_b g_i^{(K)} r i ( K ) ≤ n b g i ( K ) . Substituting this and (78) into each term of (77) gives
2 t ( π g ( K ) ) 2 α i ′ ( K ) r i ( K ) ≤ 2 t ( π g ( K ) ) ⋅ 2 6 n b g i ( K ) ⋅ n b g i ( K ) = 2 7 n b n b t ( π g ( K ) ) ( g i ( K ) ) 2 . (78) 2t(\pi_g(K))2^{\alpha'_i(K)}r_i^{(K)}\leq2t(\pi_g(K))\cdot2^6\sqrt{n_b}g_i^{(K)}\cdot n_b g_i^{(K)}=2^7n_b\sqrt{n_b}t(\pi_g(K))(g_i^{(K)})^2. \tag*{(78)} 2 t ( π g ( K )) 2 α i ′ ( K ) r i ( K ) ≤ 2 t ( π g ( K )) ⋅ 2 6 n b g i ( K ) ⋅ n b g i ( K ) = 2 7 n b n b t ( π g ( K )) ( g i ( K ) ) 2 . (78)
The rest follows from the proof of Theorem 4.1 , using ( g i ( K ) ) 2 ≤ ∑ k = ( K − 1 ) n b + 1 K n b l ^ i k 2 (g_i^{(K)})^2\leq\sum_{k=(K-1)n_b+1}^{Kn_b}\hat{l}_{ik}^2 ( g i ( K ) ) 2 ≤ ∑ k = ( K − 1 ) n b + 1 K n b l ^ ik 2 , with γ o z 2 ( ψ , Ψ , n , n b ) = 2 7 n b n b t ( ψ ) \gamma_{\mathrm{oz2}}(\psi,\Psi,n,n_b)=2^7n_b\sqrt{n_b}\,t(\psi) γ oz2 ( ψ , Ψ , n , n b ) = 2 7 n b n b t ( ψ ) in place of γ e m u \gamma_{\mathrm{emu}} γ emu .
Ozaki 2 Emulation on synthetic matrices
Ozaki 2 is also a grade B GEMM so we expect it to have similar properties to Ozaki 1 in terms of accuracy, and thus slice selection. We use the implementation from [29 ] for our experiments. We compare our adaptive algorithm (Algorithm 8 ) with fixed 12, 13 and 14 moduli implementations. Similar to Ozaki 1, Figure 8 shows that the moduli profile is decreasing for geometric and pretty much constant for arithmetic, with GEMMs being less precise than TRSM. We also stay within the n ϵ 64 n\epsilon_{64} n ϵ 64 bound despite only using the O ( t ) O(t) O ( t ) terms to write Algorithm 8 .
Figure 8. Moduli profile per panel for adaptive Ozaki 2 on the Geometric and Arithmetic distributions.
Figure 9. Performance of the Ozaki 2 emulation schemes on the synthetic distributions.
Figure 10. Backward error of the Ozaki 2 emulation schemes on the synthetic distributions.
Algorithm 8 Blocked Cholesky Factorization with Adaptive Moduli Ozaki 2 GEMM
Require: Matrix A ∈ R n × n A \in\mathbb{R}^{n \times n} A ∈ R n × n (SPD) in some high precision h h h , block size n b n_b n b , tolerance ϵ ′ \epsilon' ϵ ′ , max moduli count Ψ ′ \Psi' Ψ ′
Ensure: Lower Cholesky factor L L L (in FP64) such that A = L L T A = LL^{T} A = L L T (in-place)
diag i ← a i i \operatorname{diag}_{i} \gets a_{ii} diag i ← a ii for i = 1 , … , n i = 1,\ldots,n i = 1 , … , n ▹ diag ∈ R n \triangleright\operatorname{diag} \in\mathbb{R}^{n} ▹ diag ∈ R n : original diagonal of A A A in h h h
diag i u ← a i i \operatorname{diag}^{u}_{i} \gets a_{ii} diag i u ← a ii for i = 1 , … , n i = 1,\ldots,n i = 1 , … , n ▹ diag u ∈ R n \triangleright\operatorname{diag}^{u} \in\mathbb{R}^{n} ▹ diag u ∈ R n in h h h
m i ← 0 m_i \gets0 m i ← 0 for i = 1 , … , n i = 1,\ldots,n i = 1 , … , n ▹ m ∈ R n \triangleright m \in\mathbb{R}^{n} ▹ m ∈ R n in h h h
g i ← 0 g_i \gets0 g i ← 0 for i = 1 , … , n i = 1,\ldots,n i = 1 , … , n ▹ g ∈ R n \triangleright g \in\mathbb{R}^{n} ▹ g ∈ R n in h h h
ψ ← Ψ ′ \psi\gets\Psi' ψ ← Ψ ′
for I = 1 , … , n / n b I = 1,\ldots,n/n_b I = 1 , … , n / n b do
o ← ( I − 1 ) n b o \gets(I-1)n_b o ← ( I − 1 ) n b
w ← min ( n b , n − o ) w \gets\min(n_b,n-o) w ← min ( n b , n − o )
A I , I ← A [ o + 1 : o + w , o + 1 : o + w ] A_{I,I} \gets A[o+1:o+w,o+1:o+w] A I , I ← A [ o + 1 : o + w , o + 1 : o + w ] ▹ \triangleright ▹ diagonal block
A I , I ← Potrf ( A I , I ) A_{I,I} \gets\operatorname{Potrf}(A_{I,I}) A I , I ← Potrf ( A I , I ) ▹ A I , I ← L I , I L I , I T \triangleright A_{I,I} \gets L_{I,I}L_{I,I}^{T} ▹ A I , I ← L I , I L I , I T in h h h
if o + w < n o+w<n o + w < n then
A I + 1 , I ← A [ o + w + 1 : n , o + 1 : o + w ] A_{I+1,I} \gets A[o+w+1:n,o+1:o+w] A I + 1 , I ← A [ o + w + 1 : n , o + 1 : o + w ] ▹ \triangleright ▹ off-diagonal panel
A I + 1 , I ← A I + 1 , I ⋅ A I , I − T A_{I+1,I} \gets A_{I+1,I}\cdot A_{I,I}^{-T} A I + 1 , I ← A I + 1 , I ⋅ A I , I − T ▹ \triangleright ▹ TRSM is emulated using ψ \psi ψ moduli
for i = o + w + 1 , … , n i=o+w+1,\ldots,n i = o + w + 1 , … , n do ▹ \triangleright ▹ one pass over row i i i of the panel
g i ← max o < k ≤ o + w ∣ l i k ∣ g_i \gets\max_{o<k\leq o+w}|l_{ik}| g i ← max o < k ≤ o + w ∣ l ik ∣
diag i u ← diag i u − ∑ k = o + 1 o + w l i k 2 \operatorname{diag}^{u}_{i} \gets\operatorname{diag}^{u}_{i}-\sum_{k=o+1}^{o+w}l_{ik}^{2} diag i u ← diag i u − ∑ k = o + 1 o + w l ik 2
end for
for p 1 = 1 , … , Ψ ′ p_1=1,\ldots,\Psi' p 1 = 1 , … , Ψ ′ do
for p 2 = 1 , … , p 1 p_2=1,\ldots,p_1 p 2 = 1 , … , p 1 do
if 2 n b n b t ( p 2 ) g i 2 + γ o z 2 ( p 1 , Ψ ′ , n − I n b , n b ) 1 − γ o z 2 ( p 1 , Ψ ′ , n − I n b , n b ) a i i u ≤ ϵ ′ a i i u − m i 2^{n_b}\sqrt{n_b}\,t(p_2)g_i^2+\frac{\gamma_{\mathrm{oz2}}(p_1,\Psi',n-In_b,n_b)}{1-\gamma_{\mathrm{oz2}}(p_1,\Psi',n-In_b,n_b)}a_{ii}^{u}\leq\epsilon'a_{ii}^{u}-m_i 2 n b n b t ( p 2 ) g i 2 + 1 − γ oz2 ( p 1 , Ψ ′ , n − I n b , n b ) γ oz2 ( p 1 , Ψ ′ , n − I n b , n b ) a ii u ≤ ϵ ′ a ii u − m i for o + w < i ≤ n o+w<i\leq n o + w < i ≤ n then
π g ( I ) ← p 2 , ψ ← p 1 \pi_g(I)\gets p_2,\ \psi\gets p_1 π g ( I ) ← p 2 , ψ ← p 1
goto line 26
end if
end for
end for
for i = o + w + 1 , … , n i=o+w+1,\ldots,n i = o + w + 1 , … , n do
m i ← m i + 2 n b n b t ( π g ( I ) ) g i 2 m_i\gets m_i+2^{n_b}\sqrt{n_b}\,t(\pi_g(I))g_i^2 m i ← m i + 2 n b n b t ( π g ( I )) g i 2
end for
A I + 1 , I + 1 ← A [ o + w + 1 : n , o + w + 1 : n ] A_{I+1,I+1}\gets A[o+w+1:n,o+w+1:n] A I + 1 , I + 1 ← A [ o + w + 1 : n , o + w + 1 : n ] ▹ \triangleright ▹ trailing submatrix
A I + 1 , I + 1 ← A I + 1 , I + 1 − A I + 1 , I A I + 1 , I T A_{I+1,I+1}\gets A_{I+1,I+1}-A_{I+1,I}A_{I+1,I}^{T} A I + 1 , I + 1 ← A I + 1 , I + 1 − A I + 1 , I A I + 1 , I T ▹ \triangleright ▹ GEMM/SYRK is emulated using π g ( I ) \pi_g(I) π g ( I ) moduli
end if
end for
References