betaFeedback

WorksMathematics

Adaptive Precision Cholesky with Error Guarantees

Abstract

Modern accelerators increasingly support low-precision formats such as 8-bit floats and integers to speed up machine learning. We show that a Cholesky factorization in which both GEMM and TRSM are implemented using the fixed-point Ozaki 1 scheme satisfies the same backward error bounds as the standard floating-point Cholesky factorization, and that the same holds for a Cholesky factorization computed using a low-precision MicroScaled GEMM. We then use our error analyses to construct an adaptive precision Cholesky factorization that exploits low-precision formats while respecting a prescribed element-wise error bound, as long as the factorization succeeds. We test it in two settings. First, we use it to choose the number of Ozaki 1 slices at each iteration, with SYRK/GEMM and TRSM emulated in INT8 fixed-point arithmetic to attain double-precision accuracy. Second, we use it to build a low-precision Cholesky preconditioner for the conjugate gradient method (PCG), in which SYRK/GEMM take low-precision inputs and accumulate in high precision, with the input format selected adaptively for each panel; if the factorization breaks down, diagonal perturbation still yields a valid preconditioner, at the cost of the error bound guarantees. On an NVIDIA GH200, the adaptive Ozaki 1-based factorization is up to 29% faster than a right-looking Cholesky using a fixed slice count of 7, and up to 69% faster than one using cuBLAS' Automatic Dynamic Precision (ADP), while maintaining the same error guarantees. For PCG, FP8 factorizations with diagonal perturbation form effective preconditioners on the 13 tested SuiteSparse matrices, reducing the number of PCG iterations by up to 12×12\times over CG on the same matrix diagonally scaled by powers of two. For synthetic matrices with arithmetic spectra, and geometric spectra with κ≤104κ\le 10^4, FP8 preconditioners also reduce the number of CG iterations.

Introduction and Related Work

Many recent studies have shown that mixed-precision hardware can significantly accelerate the solution of linear systems (see [17] for a survey). We consider two approaches that have been used to achieve these speedups. The first approach combines a low-precision factorization with iterative refinement to recover a higher accuracy solution of the linear system. For example Haidar et al. [12] achieve significant speedups by combining a mixed-precision LU factorization that exploits half-precision NVIDIA Tensor Cores with GMRES-based iterative refinement (GMRES-IR). In addition, GMRES-IR was accelerated using multiple precisions in [4]. The strategy in both these works is to use an FP16/TF32/BF16 LU factorization and then use it to precondition the "correction" equation, which is solved with GMRES. Iterative refinement is required to compensate for the fact that FP16 and BF16 both have many fewer mantissa bits than FP64. This implies a larger machine epsilon and hence a less accurate solution. In this work, since we are only

Authors’ Contact Information: Sudhanva Vijaya Kulkarni, Arith Inc, USA, Berkeley, CA, USA, [email protected]; James W. Demmel, University of California, Berkeley, Berkeley, CA, USA, [email protected]; Laura Grigori, PSI Center for Scientific Computing, Theory and Data and Institute of Mathematics, EPF Lausanne, Switzerland, Lausanne, Switzerland, [email protected]. concerned with symmetric positive definite systems, we use the mixed-precision factorization as a preconditioner for a preconditioned conjugate gradient (PCG) rather than GMRES. Mixed-precision PCG with low-precision incomplete Cholesky preconditioners has been analyzed in [5]. Low-precision formats like FP16 also provide a much smaller range of representable numbers. Methods to mitigate the effect of the smaller range of available FP16 numbers have been studied in [15]. Sahraneshinsamani et al. [32] show how pre-pivoting with low-precision GEMMs can eliminate the pivoting bottleneck on GPU-like architectures. Other approaches include using custom data structures such as those described in [3], where a mixed-precision SVD is used to store different blocks of a Block-Low-Rank matrix in different precisions. This has the effect of storing "less significant" (smaller singular value) parts of the blocks in lower precision. Another example is [1], where off-diagonal blocks beyond some user-specified distance away from the diagonal are stored in FP32, and the rest are stored in FP64. A similar idea is applied in [31] where the precision to which the off-diagonal blocks are rounded is decided by the magnitude of their Frobenius norm (blocks with a smaller Frobenius norm are stored in lower precision). This idea is further extended in [2] while also applying MicroScaling. The Cholesky factorizations described in [1, 31] have the problem of not providing error-bound guarantees, and not accounting for problems such as factorization failure and underflow, which become common in the world of low-precision floats. The concurrent work [6] solves this issue by picking the precision of each tile using the ratio between the norm of the tile and the norm of the input matrix. Our work differs from that work in that it also considers emulated GEMMs and can provide either an element-wise or norm-wise error bound rather than only a norm-wise bound. Our algorithm also uses a different precision selection criterion. More recently, FP8/6/4 formats have also become popular due to their utility in machine learning [23]. Many FP8 (and shorter) formats are available [28], [25], [35], [11], [20], including the emerging IEEE 3109 standard for low-precision floats [30]. When using these low precisions, the Cholesky factorization can break down. We avoid this by using diagonal perturbation, but lose error guaranties in the process.

The second approach relies on emulation, which splits a higher precision GEMM into a sum of many scaled low-precision fixed-point or floating-point GEMMs. Ozaki schemes [26, 29] use lower-precision GEMMs to decompose a floating-point matrix into a sum of matrices whose products can be calculated exactly using low precision and then accumulated using higher precision. This allows FP64 matrix products to be computed using INT8 Tensor Cores, albeit with a weaker roundoff error bound [9]. The error bound of a GEMM implementation can receive a grade A, B or C, where the meaning of these grades is displayed in Table 1. In this paper, we show that using Ozaki (or any "grade B" GEMM as defined in [9]) gives you the same backward error guarantees as the regular floating-point Cholesky [8]. Hence, when using emulation, we can simply replace DGEMM or DTRSM with their emulated equivalents, and we do not need an extra refinement phase. Luszczek et al. [21] used the Ozaki scheme for the BLAS in an LU decomposition, but did not present an error analysis. Kouya [19] also uses Ozaki 2 in an LU decomposition. The Ozaki scheme has also been extended to FP8 [24] and FP4 [13] tensor cores. NVIDIA recently added a fixed-point INT8 emulation of DGEMM [33] to cuBLAS. Other implementations include ozIMMU [27] and also the Ozaki 2 implementation [29]. We use the cuBLAS implementation for our experiments.

GradeError Bound
A∣C^−C∣≤fA(n) ϵ∣A∣∣B∣\lvert\hat{C}-C\rvert\le f_A(n)\,\epsilon\lvert A\rvert\lvert B\rvert
B∣C^−C∣ij≤fB(n) ϵ∥A(i,:)∥∥B(:,j)∥\lvert\hat{C}-C\rvert_{ij} \le f_B(n)\,\epsilon\lVert A(i,:)\rVert\lVert B(:,j)\rVert
C∥C^−C∥≤fC(n) ϵ∥A∥∥B∥\lVert\hat{C}-C\rVert\le f_C(n)\,\epsilon\lVert A\rVert\lVert B\rVert

Table 1. GEMM error bound grades for C^=fl⁡(AB)\hat{C}=\operatorname{fl}(AB), where ϵ\epsilon is machine epsilon and fAf_A, fBf_B, fCf_C are at most low degree polynomial functions of nn, where for simplicity we assume AA, BB and CC are n×nn\times n matrices.

Contributions

Our contributions may be summarized as follows.

  • Our main contribution is an adaptive-precision Cholesky factorization that selects the number of Ozaki 1 slices, or the floating-point precision (FP8, FP16, etc.) for GEMM/SYRK and the number of Ozaki 1 slices for TRSM per iteration of Cholesky adaptively to reach a prescribed accuracy.

GradeError Bound
A∣LX^−B∣≤gA(n) ϵ∣L∣∣X^∣\lvert L\hat{X}-B\rvert\le g_A(n)\,\epsilon\lvert L\rvert\lvert\hat{X}\rvert
B∣LX^−B∣ij≤gB(n) ϵ∥L(i,:)∥∥X^(:,j)∥\lvert L\hat{X}-B\rvert_{ij} \le g_B(n)\,\epsilon\lVert L(i,:)\rVert\lVert\hat{X}(:,j)\rVert
C∥LX^−B∥≤gC(n) ϵ∥L∥∥X^∥\lVert L\hat{X}-B\rVert\le g_C(n)\,\epsilon\lVert L\rVert\lVert\hat{X}\rVert

Table 2. TRSM error bound grades for solving LX=BLX=B, with computed solution X^\hat{X}, where ϵ\epsilon is machine epsilon and gAg_A, gBg_B, gCg_C are at most low degree polynomial functions of nn, where for simplicity we assume LL, XX and BB are n×nn\times n matrices.

In the appendix, we also give an adaptive algorithm for Ozaki 2 that selects the number of moduli for a prescribed accuracy, based on only a first-order error analysis (we keep only the dominant first-order terms of the GEMM error bound).

  • The adaptive algorithm is enabled by our error analysis: we present an error analysis of Cholesky using emulated BLAS for GEMM/SYRK and TRSM, and an error analysis of Cholesky that uses mixed-precision floating-point Tensor Cores for GEMM/SYRK. We show that an emulated Cholesky obeys the same classical backward error bound as the classical floating-point Cholesky.

  • We evaluate the emulation algorithm on NVIDIA GH200 GPUs using cuBLAS FP64 fixed-point emulation. We compare 3 strategies, using 1) our adaptive algorithm, 2) static slice counts (7 and 8 slices), and 3) cuBLAS’ automatic dynamic precision scheme [33] which decides the number of slices by looking at each GEMM without the context of Cholesky factorization. Our adaptive algorithm is up to 29% faster than the static 7-slice strategy and up to 69% faster than cuBLAS ADP while meeting the same backward error bound. Our success criterion for all three implementations is the double precision backward error bound described in [8, 14]. This means that if Cholesky returns a factor LL, we check that ∣aij−∑klikljk∣≤nϵ64aiiajj\lvert a_{ij}-\sum_k l_{ik}l_{jk}\rvert\le n\epsilon_{64}\sqrt{a_{ii}a_{jj}}, where the subscript ijij denotes the (i,j)(i,j)-th element of the corresponding matrix. We also evaluate the Ozaki 2 variant on synthetic matrices against fixed 12-, 13- and 14-moduli implementations. We test on synthetic matrices and matrices from SuiteSparse [7, 18].

  • We use the mixed-precision floating-point version of the algorithm to generate preconditioners (for PCG) with varying tolerances, using a diagonal perturbation to handle factorization failures, and evaluate their effectiveness in reducing the number of CG iterations. If the algorithm is given a higher tolerance, the adaptive Cholesky will use more low-precision GEMMs. On the 13 SuiteSparse matrices we tested, and on synthetic matrices with arithmetic and random spectra, an FP8 preconditioner reduces the number of PCG iterations by up to 12× compared to CG on the power-of-two equilibrated matrix. On synthetic matrices with geometric spectra, the FP8 preconditioner improves the convergence of CG for κ≤104\kappa\le10^4. However, the convergence of CG is not improved much on synthetic matrices with κ≥108\kappa\ge10^8.

Section 3 proves the lemmas and theorems required for the error analysis of a mixed-precision Cholesky. § 4 carries out the error analysis, and § 5 states and proves the correctness of our adaptive precision algorithm. Our experiments are presented in § 6.

Background

We first introduce the notation we use throughout the rest of the paper. We use lower case letters for scalars and upper case letters for matrices and submatrices. Additionally, lower case indices are used to denote an element of a matrix and upper case indices are used to denote submatrices. Hence, aija_{ij} is the (i,j)(i,j)-th element of the matrix AA, while AIJA_{IJ} is the (I,J)(I,J)-th submatrix; also, likl_{ik} is an entry of LL and LIKL_{IK} a subblock. Subscripts can be used for both indices (as aija_{ij}) and labels (as ϵh\epsilon_h, ϵl\epsilon_l to denote high and low-precision machine epsilon, respectively). We sometimes refer to the precision using the machine epsilon when it makes sense. Iteration/panel/slice number is denoted with a parenthesized superscript (e.g. ϵ(J)\epsilon^{(J)} is the precision chosen at iteration JJ). We also use MATLAB notation: A[i1:i2,j1:j2]A[i_1:i_2,j_1:j_2] denotes the submatrix with rows i1i_1 to i2i_2 and columns j1j_1 to j2j_2. A[i1:i2,:]A[i_1:i_2,:] refers to rows i1i_1 to i2i_2 and all columns. We use a similar notation for A[:,j1:j2]A[:,j_1:j_2]. Finally, A[i,:]A[i,:] is the row ii and A[:,j]A[:,j] is the column jj.

Floating-Point Error Analysis

Floating-point computations are generally not exact. Errors are introduced when a higher precision floating-point number is rounded to a lower precision one. In addition to roundoff errors, there are also underflow and overflow errors. For this paper, we ignore overflow errors and high-precision underflow errors (which could occur during accumulation, but are very small and unlikely). The roundoff model used for high precision binary operations is as follows. For a binary operation ∘\circ performed in (high) precision H\mathcal{H}, we have

fl⁡(a∘b)=(a∘b)(1+δ)(1)\operatorname{fl}(a \circ b) = (a \circ b)(1+\delta) \tag*{(1)}

with ∣δ∣<ϵh|\delta| < \epsilon_h, where ϵh\epsilon_h is the machine epsilon. Note that we have ignored the underflow since it is a high-precision operator. If a number aa is rounded to some lower precision L\mathcal{L} with gradual underflow, then

fl⁡(a)=a(1+δ)+η(2)\operatorname{fl}(a) = a(1+\delta)+\eta \tag*{(2)}

with ∣δ∣<ϵl|\delta| < \epsilon_l, ∣η∣<μϵl|\eta| < \mu\epsilon_l, and η⋅δ=0\eta\cdot\delta= 0, where μ=2emin⁡\mu= 2^{e_{\min}} and emin⁡e_{\min} is the smallest exponent in the format [16]. We always assume RoundToNearestEven for the rounding mode unless specified otherwise. We also assume gradual underflow for the rest of the document.

Emulation with Ozaki 1

The Ozaki scheme emulates a high-precision matrix product using a sequence of low-precision products [26, 29, 37]. In this paper, we consider the Ozaki 1 algorithm from [26] applied to FP64 matrices with INT8/UINT8 slices. Consider two FP64 matrices AA, BB, divided into p∈Np \in\mathbb{N} slices, where a slice represents a short set of consecutive mantissa bits in each matrix entry. We can write

A=∑q=1pA(q)+RA(p),B=∑q=1pB(q)+RB(p).(3)A = \sum_{q=1}^{p} A^{(q)} + R_A^{(p)}, \qquad B = \sum_{q=1}^{p} B^{(q)} + R_B^{(p)}. \tag*{(3)}

where each A(q)A^{(q)}, B(q)B^{(q)} is a scaled low-precision INT8/UINT8 matrix and RA(p)=A−∑q=1pA(q)R_A^{(p)} = A - \sum_{q=1}^{p} A^{(q)}, RB(p)=B−∑q=1pB(q)R_B^{(p)} = B - \sum_{q=1}^{p} B^{(q)} are the residuals. The matrix product using pp slices is approximated by

C~(p):=∑q=1p∑r=1p−q+1A(q)B(r)(4)\widetilde{C}^{(p)} := \sum_{q=1}^{p} \sum_{r=1}^{p-q+1} A^{(q)}B^{(r)} \tag*{(4)}

which contains 12p(p+1)\frac{1}{2}p(p+1) low-precision GEMMs. Each A(q)B(r)A^{(q)}B^{(r)} is computed exactly. We let C^(p)\widehat{C}^{(p)} denote the final result of accumulating all the products A(q)B(r)A^{(q)}B^{(r)} using FP64. Then, for the uniform INT8 slice case described in [37], this gives the following error bound

∣C^(p)−C~(p)∣≤(12p(p+1)−1)ϵ64∑q=1p∑r=1p−q+1∣A(q)B(r)∣≤(12p(p+1)−1)ϵ64∑q=1p∑r=1p−q+1∣A(q)∣∣B(r)∣≤(12p(p+1)−1)ϵ64∣A∣∣B∣(5)\begin{aligned} \left|\widehat{C}^{(p)}-\widetilde{C}^{(p)}\right| \leq\left(\frac{1}{2}p(p+1)-1\right)\epsilon_{64} \sum_{q=1}^{p}\sum_{r=1}^{p-q+1}\left|A^{(q)}B^{(r)}\right| \\ &\leq\left(\frac{1}{2}p(p+1)-1\right)\epsilon_{64} \sum_{q=1}^{p}\sum_{r=1}^{p-q+1}\left|A^{(q)}\right|\left|B^{(r)}\right| \leq\left(\frac{1}{2}p(p+1)-1\right)\epsilon_{64}|A||B| \tag*{(5)} \end{aligned}

References

  1. [1]Sameh Abdullah, Hatem Ltaief, Ying Sun, Marc G. Genton, and David E. Keyes. 2019. Geostatistical Modeling and Prediction Using Mixed Precision Tiled Cholesky Factorization. In 2019 IEEE 26th International Conference on High Performance Computing, Data, and Analytics (HiPC), 152–162. doi:10.1109/HiPC.2019.00028.
  2. [2]Abdourauf Abdurakhmanov, Theo Mary, Hatem Ltaief, David E Keyes, and Suhaib A. Fahmy. 2026. Error-Controlled Mixed-Precision Cholesky Factorization with MX Microscaling Formats. ACM Transactions on Mathematical Software. https://hal.science/hal-05641535.
  3. [3]Patrick Amestoy, Olivier Boiteau, Alfredo Buttari, Matthieu Gresset, Fabienne Jézéquel, and et al. 2022. Mixed Precision Low Rank Approximations and their Application to Block Low Rank LU Factorization. IMA Journal of Numerical Analysis. hal-03251738v3. doi:10.1093/imanum/drac037.DOI
  4. [4]Patrick Amestoy, Alfredo Buttari, Nicholas J. Higham, Jean-Yves L’Excellent, Theo Mary, and Bastien Vieublé. 2024. Five-precision gmres-based iterative refinement. SIAM Journal on Matrix Analysis and Applications, 45, 1, 529–552. eprint: https://doi.org/10.1137/23M1549079.
  5. [5]Thomas Bake, Erin Carson, and Yuxin Ma. 2025. Forward and backward error bounds for a mixed precision preconditioned conjugate gradient algorithm. (2025). https://arxiv.org/abs/2510.11379 arXiv: 2510.11379 [math.NA].
  6. [6]Qinglei Cao, Theo Mary, Rabab Alomairy, Hatem Ltaief, and David E. Keyes. 2026. Right Precision, Right Place: Runtime Adaptations for Mixed-Precision Cholesky Factorization. HAL preprint hal-05793453. (2026). https://hal.science/hal-05793453v1.
  7. [7]Timothy A. Davis and Yifan Hu. 2011. The University of Florida Sparse Matrix Collection. ACM Trans. Math. Softw., 38, 1, (Dec. 2011). doi:10.1145/2049662.2049663.DOI
  8. [8]J. W. Demmel. 1989. On Floating-Point Errors in Cholesky. Tech. rep. CS-89-87. LAPACK Working Note 14. University of Tennessee at Knoxville, (Oct. 1989).
  9. [9]James Demmel, Greg Henry, Igor Kozachenko, Julien Langou, Xiaoye Sherry Li, Jason Riedy, and Jackson Vanover. 2026. How to grade the accuracy of the BLAS. (2026). https://arxiv.org/abs/2609.12307 arXiv: 2609.12307 [math.NA].
  10. [10]Philip E. Gill, Walter Murray, and Margaret H. Wright. 1981. Practical Optimization. Academic Press, London.
  11. [11]Gustafson and Yenamoto. 2017. Beating floating point at its own game: posit arithmetic. Supercomput. Front. Innov.: Int. J., 4, 2, (June 2017), 71–86. doi:10.14529/jsfi170206.DOI
  12. [12]Azzam Haidar, Panruo Wu, Stanimire Tomov, and Jack Dongarra. 2017. Investigating half precision arithmetic to accelerate dense linear system solvers. In Proceedings of the 8th Workshop on Latest Advances in Scalable Algorithms for Large-Scale Systems (ScaLA ’17). Association for Computing Machinery, Denver, Colorado. ISBN: 9781450351256. https://doi.org/10.1145/3148226.3148327.
  13. [13]Shun-ichiro Hayashi, Daichi Mukunoki, Tetsuya Hoshino, and Takahiro Katagiri. 2026. DGEMM with Ozaki Scheme I/II on FP4 Tensor Cores: A Base-13 E2M1 Limb Representation. (2026). https://arxiv.org/abs/2608.06812 arXiv: 2608.06812 [cs.DC].
  14. [14]Nicholas J. Higham. 2002. Accuracy and Stability of Numerical Algorithms. (Second ed.). Society for Industrial and Applied Mathematics. eprint: https://epubs.siam.org/doi/pdf/10.1137/1.9780898718027.DOI
  15. [15]Nicholas J. Higham, Srikara Pranesh, and Mawussi Zounon. 2019. Squeezing a matrix into half precision, with an application to solving linear systems. SIAM Journal on Scientific Computing, 41, 4, A2536–A2551. eprint: https://doi.org/10.1137/18M1229511.
  16. [16]IEEE. 2019. IEEE Standard for Floating-Point Arithmetic. IEEE Std 754-2019 (Revision of IEEE Std 754-2008). Standard. Institute of Electrical and Electronics Engineers. New York, NY, USA, (July 2019). ISBN: 978-1-5044-5924-2. doi:10.1109/IEEESTD.2019.8766229.DOI
  17. [17]Aditya Kashi, Hao Lu, Wesley Brewer, David Rogers, Michael Matheson, Mallikarjun Shankar, and Feiyi Wang. 2024. Mixed-precision numerics in scientific applications: survey and perspectives. (2024). https://arxiv.org/abs/2412.19322 arXiv: 2412.19322 [cs.CE].
  18. [18]Scott P. Kolodziej, Mohsen Aznaveh, Matthew Bullock, Jarrett David, Timothy A. Davis, Matthew Henderson, Yifan Hu, and Read Sandstrom. 2019. The suitesparse matrix collection website interface. Journal of Open Source Software, 4, 35, 1244. doi:10.21105/joss.01244.DOI
  19. [19]Tomonori Kouya. 2026. Accelerating Multiple-Precision LU Decomposition with Ozaki Scheme II. (2026). https://arxiv.org/abs/2609.29554 arXiv: 2609.29554 [math.NA].
  20. [20]Yuanyong Luo et al. 2024. Ascend hifloat8 format for deep learning. (2024). https://arxiv.org/abs/2409.16626 arXiv: 2409.16626 [cs.LG].
  21. [21]Piotr Luszczek et al. 2025. Performance and numerical aspects of decompositional factorization with fp64 floating-point emulation in int8. In 2025 IEEE High Performance Extreme Computing Conference (HPEC), 1–7. doi:10.1109/HPEC67600.2025.11196192.DOI
  22. [22]P. McSweeney. 2020. Modified Cholesky decomposition and applications. https://mcsweeney90.github.io/files/modified-cholesky-decomposition-and-applications.pdf.
  23. [23]Paulius Micekievicius et al. 2022. FP8 Formats for Deep Learning. (2022). https://arxiv.org/abs/2209.05433.
  24. [24]Daichi Mukunoki. 2025. DGEMM without FP64 Arithmetic – Using FP64 Emulation and FP8 Tensor Cores with Ozaki Scheme. (2025). https://arxiv.org/abs/2508.00441 arXiv: 2508.00441 [cs.PF].
  25. [25]B. Noune, P. Jones, D. Justus, D. Masters, and C. Luschi. 2022. 8-bit numerical formats for deep neural networks. Tech. rep. arXiv cs.LG. https://arxiv.org/abs/2206.02915.
  26. [26]H. Ootomo, K. Ozaki, and R. Yokota. 2024. DGEMM on Integer Matrix Multiplication Unit. Tech. rep. 2306.11975v4. arXiv.arxiv.org/abs/2306.11975
  27. [27]Hiroyuki Ootomo, Katsuhisa Ozaki, and Rio Yokota. 2024. Dgemm on integer matrix multiplication unit. (2024). https://arxiv.org/abs/2306.11975 arXiv: 2306.11975 [cs.DC].
  28. [28]Open Compute Project. 2023. OCP Microscaling Formats (MX) Specification Version 1.0. Accessed: 2024-12-02. Open Compute Project Foundation, (Sept. 2023). https://www.opencompute.org/documents/ocp-microscaling-formats-mx-v1-0-spec-final-pdf.
  29. [29]Katsuhisa Ozaki, Yuki Uchino, and Toshiyuki Imamura. 2025. Ozaki Scheme II: A GEMM-oriented emulation of floating-point matrix multiplication using an integer modular technique. (2025). https://arxiv.org/abs/2504.08009 arXiv: 2504.08009 [cs.MS].
  30. [30]P3109. 2024. Public repository. GitHub repository, main branch. Accessed: 2024-12-02. (2024). https://github.com/P3109/Public/tree/main.
  31. [31]Jie Ren, Hatem Ltaief, Sameh Abdulah, and David E. Keyes. [n.d.] Accelerating Mixed-Precision Out-of-Core Cholesky Factorization with Static Task Scheduling. Published on October 13, 2024. (). https://arxiv.org/pdf/2410.09819.
  32. [32]Nima Sahraneshin samani, Sandra Catalán, and José R. Herrero. 2024. Mixed-precision pre-pivoting strategy for the LU factorization. The Journal of Supercomputing, 81, 1, (Oct 2024), 87. https://doi.org/10.1007/s11227-024-06523-w.
  33. [33]Angelika Schwarz et al. 2026. Guaranteed DGEMM Accuracy While Using Reduced Precision Tensor Cores Through Extensions of the Ozaki Scheme. In Proceedings of the Supercomputing Asia and International Conference on High Performance Computing in Asia Pacific Region (SCA/HPCAsia 2026). ACM, (Jan. 2026), 91–101. doi:10.1145/3773656.3773670.DOI
  34. [34]Laura Grigori Sudhanva Kulkarni James Demmel and David Chen. 2025. LoFloat: a C++ library for low-precision floating-point experimentation. https://github.com/SudhanvaKulkarni123/LoFloat. GitHub repository. Accessed: 2026-10-13. (2025).
  35. [35]Tesla, Inc. 2023. Tesla Dojo Technology: A guide to Tesla’s configurable floating point formats and arithmetic. Tech. rep. Tesla, Inc. https://web.archive.org/web/20230503235751/https://tesla-cdn.thron.com/static/XMMU3S_tesla-dojo-technology_1WDVZN.pdf.
  36. [36]Yuki Uchino, Katsuhisa Ozaki, and Toshiyuki Imamura. 2026. Error Analysis of Matrix Multiplication Emulation Using Ozaki-II Scheme. (2026). https://arxiv.org/abs/2602.02549 arXiv: 2602.02549 [math.NA].
  37. [37]Yuki Uchino, Katsuhisa Ozaki, and Toshiyuki Imamura. 2025. Performance enhancement of the Ozaki Scheme on integer matrix multiplication unit. The International Journal of High Performance Computing Applications, 39, 3, (Jan. 2025), 462–476. doi:10.1177/10943420241313064.DOI

Paper details

vlnc:W_4qf8xu9f
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Mathematics

Claude Opus 5 (bare), Claude Opus 5.5, ChatGPT 6 Astra-Pro

View statement
Looking for related papers…
Contents