WorksMathematics
Low-rank Low Precision Training of Gaussian Processes
Abstract
Training Gaussian processes (GPs) at large scale is computationally challenging because evaluating the log-marginal likelihood and its gradients involves large, dense kernel matrices. We develop a scalable low-rank GP training approach that combines AnchorNet landmark selection with a fully independent training conditional (FITC) correction. The resulting objective and its analytic gradients can be evaluated using low-rank factors, diagonal quantities, and small dense matrices, without forming the full kernel matrix or its derivatives. The complete training procedure is implemented in single precision and evaluated on nine large-scale regression datasets. Across these problems, the method reaches the reference predictive accuracy while substantially reducing training time, although the required approximation rank varies across datasets. We also study the numerical behavior of the full-kernel predictive solve and show that single precision can become limiting for ill-conditioned problems. Finally, we investigate further reducing selected matrix operations to fp16 or TF32. These lower-precision variants provide only modest additional speedups and can become unreliable when the reduced system is poorly conditioned.
Introduction
Surrogate models can serve as accurate and computationally efficient replacements for expensive models or can represent an underlying unknown function from observed data [13]. Gaussian processes (GPs) [27] are widely used for this purpose in Bayesian optimization and machine learning because they provide both predictions at unseen inputs and a principled quantification of predictive uncertainty.
The performance of a GP depends strongly on the choice of covariance or kernel function and its associated hyperparameters [20]. A central task in GP training is therefore to estimate these hyperparameters from the available training data [19]. Both hyperparameter training and posterior prediction require matrix computations involving a large, dense covariance matrix. A direct implementation based on matrix factorization has cubic complexity in the number of training points, which quickly becomes prohibitive for large datasets. Consequently, considerable effort has been devoted to structured and low-rank approximations that reduce these computational costs [27, 1, 29, 8, 18, 25].
Further computational gains are possible by exploiting reduced-precision arithmetic on modern hardware, where single- and half-precision operations can provide substantially higher throughput and lower memory traffic than double precision. This has motivated increasing interest in low-precision algorithms for large-scale GP computations [15]. At the same time, reduced precision introduces additional numerical challenges because GP kernel matrices can become severely ill-conditioned during hyperparameter optimization.
The contributions of this paper are two-fold. First, we develop a scalable low-rank GP training approach based on landmarks selected by the AnchorNet algorithm [4] together with a fully independent training conditional (FITC) correction. We derive the corresponding objective and analytic gradients so that both the objective and its derivatives can be evaluated using low-rank factors and small dense matrices, without forming the full kernel matrix or its derivatives. Second, we study the numerical behavior of this training procedure in reduced precision on modern GPU architectures. We demonstrate that the complete low-rank training procedure can be carried out effectively in single precision and investigate the extent to which selected matrix operations can be further reduced to half precision or TF32. Our experiments also identify the conditioning regimes in which these lower-precision computations become unreliable.
The remainder of the paper is organized as follows. In § 2, we review the GP formulation and the kernel models used in this work. In § 3, we derive the low-rank FITC objective and its gradients. Section 4 describes the AnchorNet landmark-selection procedure, and § 5 summarizes the resulting training algorithm. § 6 presents the numerical experiments, including the effects of the approximation rank and the numerical behavior of the predictive solve. Finally, § 7 investigates the use and limitations of precision below fp32.
Gaussian Processes
A Gaussian process (GP) is a collection of random variables such that every finite subset has a joint Gaussian distribution [27]. It is completely characterized by its mean function
and covariance, or kernel, function
We write
and assume a zero mean, , throughout this work.
Let contain the training inputs as its rows, and define the corresponding latent function values by
We observe
where and the observation noise is independent of the latent process. For test inputs , define
where each kernel matrix is formed by evaluating at the corresponding pairs of inputs. Let collect the latent function values at the test inputs. For fixed hyperparameters, the joint distribution of the training observations and test function values is
Conditioning on the training observations gives
The posterior mean predicts the latent function values, while quantifies their uncertainty conditional on the chosen hyperparameters. For noisy test observations with independent Gaussian noise of variance , the predictive covariance is . Figure 1 illustrates the prior and posterior of a one-dimensional GP.

Figure 1. A one-dimensional GP before and after conditioning on training data.
In the following sections, we also use the shorthand , , and .
Kernel model. The kernel determines the covariance structure of the GP. We begin with the squared-exponential kernel
where is the length scale and is the signal scale, so that is the signal variance. This kernel is symmetric and positive definite.
Define the squared Euclidean distance matrix by
The corresponding kernel matrix is
where the exponential is applied entrywise.
In our experiments, we use the squared-exponential kernel with automatic relevance determination (ARD), which assigns a separate positive length scale to each input coordinate:
The hyperparameter vector is
and contains parameters. The noise scale enters the observation covariance . A large relative to the variation in the th input coordinate makes the kernel less sensitive to changes along that coordinate.
For each coordinate , define by
so that
The ARD kernel matrix can then be written as
with the exponential again applied entrywise. The isotropic kernel is recovered by setting for all . The distance matrices provide convenient notation for the kernel and its derivatives; the low-rank implementation described below forms only the required blocks.
Large-scale computation. The main computational difficulty in GP regression is that is generally dense. For , the observation covariance is symmetric positive definite in exact arithmetic. A direct Cholesky factorization requires operations and storage, which becomes prohibitive for large datasets [27]. This has motivated approaches based on inducing points, structured kernel approximations, and matrix-free iterative methods.
Inducing-point methods use a smaller collection of inputs to construct a low-rank representation of the covariance structure; see, e.g., [18]. Examples include the subset of regressors (SOR) [21] and the fully independent training conditional (FITC) approximation [23]. FITC augments the low-rank training covariance with a diagonal correction that restores the marginal variances at the training inputs. Structured kernel interpolation, developed by Wilson and Nickisch [28], provides another way to approximate kernel matrices efficiently. Such approximations can be combined with Krylov methods for linear solves and with stochastic trace estimation for log-determinant computations [9]. These techniques are used in scalable GP software such as GPyTorch [11].
Efficient trace estimation is also important in these computations [22, 24, 16, 7, 17, 26, 30]. Matrix-free approaches evaluate the action of without explicitly storing the dense matrix [25, 14]. For posterior-mean computations, the resulting linear systems are closely related to those arising in kernel ridge regression and can benefit from preconditioning, including low-rank preconditioners; see, e.g., [6, 3, 10, 31].
Our method uses AnchorNet to select landmarks for a low-rank kernel approximation and combines this approximation with the FITC diagonal correction. The resulting training objective and its gradients are evaluated using low-rank factors, diagonal quantities, and small dense matrices, as developed in § 3.
Training Gaussian Processes
Exact GP training estimates the hyperparameters by minimizing the negative log-marginal likelihood
For large datasets, the main computational bottlenecks are the linear solve with required by the quadratic term and the evaluation of . A direct Cholesky factorization requires operations and storage [27, 2], while alternative large-scale approaches typically rely on iterative solves and trace-estimation techniques [11, 8].
To reduce these costs, we construct a low-rank approximation using landmarks selected by AnchorNet [4]. The corresponding kernel blocks satisfy
The construction of the landmarks and kernel blocks is described in Section 4. The landmark locations are distinct and remain fixed throughout hyperparameter optimization.
In the single-precision implementation, the landmark kernel matrix is stabilized before factorization:
where is selected as described in § 5. For notational simplicity, in the remainder of this section, and denote the stabilized matrix and its inverse, respectively. With this convention, the low-rank approximation is
All derivatives below are taken with the landmark set and the accepted value of held fixed. Changes in caused by the stabilization procedure are not differentiated.
The FITC-corrected low-rank model
The low-rank approximation can underestimate the marginal variances at the training inputs. For nonnegative stabilization, it satisfies
in exact arithmetic, and hence
When , the standard Nyström approximation gives equality at the landmark locations. Positive stabilization generally removes this interpolation property and increases the diagonal deficit.
We use the fully independent training conditional (FITC) correction [23] to restore the marginal variances. Let be the diagonal matrix whose entries are
The approximate covariance of the noisy training observations is
Thus, approximates the latent covariance , while approximates . Since and ,
so both and are positive definite in exact arithmetic.
The approximate negative log-marginal likelihood used for training is
The uncorrected low-rank model is recovered by setting .
Efficient evaluation of the objective
The inverse and log-determinant in (5) can be evaluated using low-rank factors, diagonal operations, and factorizations. Define
Applying the Sherman–Morrison–Woodbury formula gives
Similarly, the matrix determinant lemma gives
Although is generally nonsymmetric, it is similar to a symmetric positive definite matrix. Let
Then
Since is positive definite, in exact arithmetic. The similarity transformation also gives
The inverse action in (7) can equivalently be written as
Thus, all dense factorizations required for evaluating the objective involve only matrices. The formulas can be evaluated using diagonal scaling and triangular solves, without explicitly forming the inverse matrices.
Gradient evaluation
To optimize the hyperparameters, we differentiate (5). For any hyperparameter , the standard identities for the derivative of a log-determinant and a matrix inverse give
Using
we separate the gradient into low-rank and diagonal contributions:
where
The diagonal of can be obtained without forming the full inverse. Using and (9),
The second diagonal is evaluated from blocks of the low-rank factors, without forming an matrix. It remains to evaluate the derivatives of and the associated reduced quantities.
Length-scale derivatives. For an ARD length scale , all three factors in
depend on . The matrix product rule [12] therefore gives
Using the ARD kernel (1) and the componentwise squared-distance matrices defined in § 2, the factor derivatives are
Here denotes the Hadamard product and nyst denotes the landmark indices identified by AnchorNet. With fixed, the stabilization term is independent of . Moreover, the diagonal of is zero, so its Hadamard product with the stabilization term vanishes. Thus, (14) remains valid under our convention that denotes the stabilized matrix.
The derivative in (3.12) admits the low-rank factorization
The factors have dimensions , , .
This block representation can be evaluated termwise without explicitly assembling the three factors. In particular,
so the quadratic-form contribution requires no derivative matrix.
The low-rank trace contribution can also be reduced to an computation. Since the derivative of is accounted for separately in (3.3), we differentiate with held fixed:
Differentiating the determinant identity with fixed gives
The diagonal contribution follows from , which is independent of the length scales:
Combining these expressions yields
Signal-scale derivative. The signal variance scales every kernel block. Because the stabilization has the form with fixed , the stabilized landmark matrix also scales proportionally to . Consequently,
The product rule then gives
As in the length-scale calculation, the low-rank contribution holds fixed:
Since
the diagonal derivative is
The resulting signal-scale gradient is
Noise derivative. The low-rank approximation and the diagonal correction are independent of the noise scale , while
Thus, only the diagonal contribution in (4) remains:
All quantities in (19), (4), and (3.23) can be evaluated using low-rank factors, diagonal quantities, and reduced matrices. Neither the full kernel matrix nor any derivative matrix needs to be formed. The componentwise squared-distance matrices are likewise not stored explicitly; the required Hadamard products are generated blockwise together with the corresponding kernel blocks.
Positive parameterization
To enforce positivity without constrained optimization, we introduce unconstrained parameters
and parameterize
and
Since
the derivatives with respect to the unconstrained optimization variables follow from the chain rule. Writing for the component corresponding to , we obtain
This formula also applies to the noise parameter because its additional floor is constant.
AnchorNet Low-Rank Approximation
We use AnchorNet [4] to select landmark points for the low-rank kernel approximation. Given a set of landmarks , the standard Nyström approximation is
where , , and . Here is the number of distinct landmarks.
Let
denote the training locations stored in the rows of . Let be a positive integer specifying the target net size. The construction has two stages: it first generates anchor points adapted to the data geometry and then maps them to training points. All nearest-point ties are resolved by a fixed rule.
Anchor construction.
Compute the smallest axis-aligned bounding box containing and generate an initial low-discrepancy set
Possible constructions include Halton points and the adaptive tensor grids described in [4].
Assign each training point to one nearest point in with respect to the norm. Specifically, choose
according to the fixed tie-breaking rule and define
Discard empty groups and relabel the remaining groups as .
For each nonempty group , compute its smallest axis-aligned bounding box . Let denote its -dimensional Lebesgue measure. For the volume-based allocation, assume
and set
When , generate a low-discrepancy set with points. If , set .
Form the anchor net
The ceiling allocation gives
because . Thus, the number of distinct anchors need not equal the target size .
From anchors to landmarks. For each anchor , select a nearest training point,
using the fixed tie-breaking rule. The landmark set is
with repeated selections removed. Consequently,
and .
The volume-based allocation is designed to distribute anchors across the data geometry. Mapping these anchors to the data produces landmarks intended to improve the approximation quality and numerical behavior of the Nyström approximation [4]. The landmark-selection stage costs and depends only on the input locations, requiring neither kernel matrix entries nor kernel matrix–vector products [4]. The selected landmarks define the low-rank approximation described in § 3, with stabilization implemented as described in § 5.
Algorithm
5. Algorithm. Algorithm 5.1 summarizes the training and prediction procedure. The landmarks are selected once and remain fixed during optimization, with denoting the resulting approximation rank. Initialization and optimizer settings are given in Appendix § 9.
Training is performed in fp32. The stabilization routine factors
starting with and increasing it by a factor of ten after an unsuccessful attempt, up to seven attempts. Failure is reported if no acceptable factor is obtained; this did not occur in the reported experiments. The gradient formulas in § 3 use and its inverse, with the accepted held fixed. Final factor reconstruction repeats the same stabilization procedure. The operator clips the Euclidean norm of the unconstrained gradient to 10.
The factor computations are blocked by default. We select the unblocked implementation only when an initial timing test shows that it is at least 20% faster and sufficient memory is available. Kernel products during prediction can also be evaluated in blocks.
The low-rank predictive mean uses only reduced solves and kernel products with the landmarks. For the full-kernel mean, datasets with fewer than 60,000 training Algorithm 5.1 FITC Gaussian process training with AnchorNet landmarks
Input: Training data ; test inputs ; target net size ; iteration budget ; initial learning rate ; low-rank predictive-mean flag
Output: Hyperparameters and predictive mean
;
initialize the Adam optimizer state
for do
set using (A.2)
form , , and
compute as in Section 3 and factor
evaluate and using the stabilized factors, with fixed
compute using (26)
if then
recompute at the final
factor and solve
solve
evaluate
else
solve at the final
evaluate
points use a dense factorization; larger datasets use preconditioned CG with KeOps matrix–vector products. The default fp32 preconditioner uses the first landmarks. Additional fp64 solves establish the full-kernel reference predictions on 3DRoad, Buzz, and HouseElectric, where fp32 does not attain sufficient accuracy; see § 6 and appendix § 10.
Numerical experiments
We evaluate the low-rank model described in Algorithm 5.1, which we refer to as ANGP (AnchorNet Gaussian process),1 on the nine UCI regression datasets used by Maddox et al. [15]. The datasets are obtained through the public bayesian_benchmarks interface and range from to training points, with input dimension ranging from 3 to 90, as summarized in Table 1. We use the standard preprocessing provided by bayesian_benchmarks; in particular, the input features and targets are standardized.
| dataset | RMSE ANGP | RMSE Maddox | time [s] ANGP | time [s] Maddox | speed-up | ||
| PoleTele | (13.5K, 26) | 100 | 0.1075 ± 0.0085 | 0.117 ± 0.004 | 2.2 | 58 | 26.6× |
| Elevators | (14.9K, 18) | 400 | 0.3607 ± 0.0037 | 0.364 ± 0.004 | 3.4 | 58 | 17.4× |
| Bike | (15.6K, 17) | 100 | 0.0707 ± 0.0071 | 0.074 ± 0.003 | 2.4 | 62 | 26.4× |
| Kin40K | (36K, 8) | 1600 | 0.0981 ± 0.0018 | 0.099 ± 0.001 | 17.4 | 66 | 3.8× |
| Protein | (41.2K, 9) | 1600 | 0.5406 ± 0.0068 | 0.550 ± 0.006 | 20.5 | 71 | 3.5× |
| 3DRoad | (391.4K, 3) | 3000 | 0.1915 ± 0.0040 | 0.194 ± 0.010 | 373 | 1,260 | 3.4× |
| Song | (463.8K, 90) | 3000 | 0.7605 ± 0.0031 | 0.761 ± 0.004 | 1,404 | 24,357 | 17.4× |
| Buzz | (524.9K, 77) | 50 | 0.2672 ± 0.0056 | 0.300 ± 0.010 | 22.7 | 25,436 | 1,120.3× |
| HouseElectric | (1844.4K, 11) | 800 | 0.0518 ± 0.0003 | 0.052 ± 0.002 | 431 | 42,751 | 99.1× |
Table 1. Fit time comparison with Maddox [15 Table 1].
Throughout the experiments, we use the same kernel, hyperparameter initialization, optimizer, and optimization budget for all datasets. The only model parameter varied across datasets is the number of AnchorNet landmarks, which determines the approximation rank . We use the squared-exponential kernel with automatic relevance determination (ARD) in (1), as in [15], resulting in hyperparameters.
The hyperparameters are initialized using the data-driven procedure described in Appendix § 9: the length scales are initialized using the median pairwise distance heuristic, the signal scale using the standard deviation of the target, and the noise scale using the mode of its prior. We optimize the FITC negative log-marginal likelihood using Adam for a fixed budget of 100 iterations with a cosine-decayed learning rate. We use neither a validation split nor early stopping and report the final iterate for every run. With the initialization and learning-rate schedule used here, we did not observe divergence, collapse of a length scale, or activation of the noise floor in the reported experiments.
Each evaluation of the FITC objective and its gradient has cost in our implementation. The unblocked implementation stores intermediate quantities, whereas the blocked implementation processes the training data in row blocks of size and requires working memory. At the beginning of each fit, we time one objective-and-gradient evaluation using both implementations. The blocked implementation is used by default; the unblocked variant is selected only when it is at least 20% faster and fits in available memory. Additional implementation details are given in Appendix § 9.
For prediction, datasets with fewer than 60,000 training points use a dense full-kernel factorization. For the larger datasets, the full-kernel predictive mean is computed using preconditioned conjugate gradients (CG), with matrix–vector products evaluated through KeOps [5]. We also evaluate the predictive mean obtained directly from the fitted low-rank model, which avoids the full-kernel linear solve.
Each configuration is evaluated on five data splits, and we report the mean and standard deviation. For each dataset, we test several landmark counts and report every tested configuration in Table 2. When we subsequently identify the cheapest rank that reaches a reference RMSE, this should be interpreted as a retrospective cost–accuracy comparison rather than as a rank-selection procedure used during training.
| PoleTele (0.117) | Elevators (0.364) | Bike (0.074) | Kin40K (0.099) | Protein (0.550) | 3DRoad* (0.194) | Song (0.761) | Buzz* (0.300) | HouseElectric* (0.052) | |
| 50 | 0.1251 | 0.3806 | 0.0924 | 0.2663 | 0.7535 | 0.6319 | 0.7740 | 0.2672 | 0.0551 |
| 100 | 0.1075 | 0.3771 | 0.0707 | 0.2135 | 0.7086 | 0.5389 | 0.7710 | 0.2630 | 0.0549 |
| 200 | 0.1015 | 0.3667 | 0.0497 | 0.1764 | 0.6765 | 0.4944 | 0.7685 | 0.2655 | 0.0537 |
| 250 | 0.1000 | 0.3649 | 0.0480 | 0.1743 | 0.6539 | 0.4828 | 0.7677 | 0.2642 | 0.0534 |
| 400 | 0.1011 | 0.3607 | 0.0406 | 0.1371 | 0.6151 | 0.3991 | 0.7661 | 0.2648 | 0.0526 |
| 500 | 0.1016 | 0.3592 | 0.0389 | 0.1306 | 0.5952 | 0.3805 | 0.7652 | 0.2638 | 0.0522 |
| 800 | 0.1011 | 0.3565 | 0.0349 | 0.1106 | 0.5649 | 0.3015 | 0.7640 | 0.2622 | 0.0518 |
| 1000 | 0.1034 | 0.3555 | 0.0334 | 0.1051 | 0.5524 | 0.2871 | 0.7634 | 0.2628 | 0.0516 |
| 1600 | 0.1043 | 0.3545 | 0.0303 | 0.0981 | 0.5406 | 0.2502 | 0.7620 | 0.0506 | |
| 2000 | 0.1055 | 0.3542 | 0.0931 | 0.5343 | 0.2316 | 0.7613 | 0.0490 | ||
| 3000 | 0.1076 | 0.3539 | 0.0835 | 0.5282 | 0.1915 | 0.7605 | 0.0467 | ||
| 4000 | 0.0759 | 0.5247 | 0.1753 | 0.7598 | |||||
| 6000 | 0.0683 | 0.5216 | 0.1511 | ||||||
| 8000 | 0.0649 | 0.5207 | |||||||
| speed-up | 26.6× | 17.4× | 26.4× | 3.8× | 3.5× | 3.4× | 17.4× | 1120.3× | 99.1× |
Table 2. Test RMSE from the posterior mean solve against rank. The last row reports the speedup for hyperparameter fitting at the smallest rank that achieves at least the same accuracy as Maddox.
All ANGP timings are measured on an NVIDIA A40 GPU with 48 GB of memory. We compare against the V100 (32 GB) timings reported by Maddox et al. [15], denoted by Maddox in the tables. Following their experimental setup, the reported fitting times include hyperparameter optimization but exclude the predictive solve.
The two sets of timings are obtained on different hardware. The A40 has roughly a 2.4× advantage in nominal single-precision throughput over the V100, so part of the observed speedup is due to hardware rather than the algorithm alone. We therefore report the raw timings and interpret a speedup divided by 2.4 as an approximate hardware-normalized comparison. Under this normalization, ANGP remains faster than the reference on all datasets, although the advantage on 3DRoad becomes small.
The remainder of this section examines three aspects of the method: the cost–accuracy tradeoff as the approximation rank varies, the difference between the low-rank and full-kernel predictive means, and the numerical behavior of the full-kernel predictive solve in single precision.
Comparison with the reference
Table 1 reports, for each dataset, the cheapest tested rank whose mean test RMSE reaches the reference value reported in [15]. As noted above, this selection is performed retrospectively from the tested ranks and is intended to summarize the cost required to attain the reference accuracy.
Whenever the fp32 full-kernel solve converges, we compute the reported RMSE using that predictive mean. For 3DRoad, Buzz, and HouseElectric, the fp32 CG solve does not attain sufficient accuracy because of finite-precision effects. For these datasets, we instead report the RMSE obtained from the corresponding fp64 full-kernel solve; these entries are marked by * in Table 1.
The Protein RMSE reported in [15 Table 1] is 0.055. This value is inconsistent with both the half-precision result (0.635) in the same table and the values 0.481–0.509 reported in the appendix of that paper. We therefore interpret 0.055 as a misplaced decimal point and use 0.55 as the reference value throughout.
Table 2 shows that ANGP reaches the reference RMSE on all nine datasets for at least one of the tested ranks. Increasing generally improves the quality of the low-rank model but also increases the fitting cost, which grows quadratically with in the dominant low-rank operations. The rank required to reach the reference accuracy varies substantially across datasets.
For example, 3DRoad requires to reach the reference accuracy. At , ANGP is already slower than Maddox even before accounting for the hardware difference; see Table 4. Across the datasets, the raw fitting speedups range from 3.4× on 3DRoad to 1120× on Buzz. On 3DRoad, the observed speedup is only somewhat larger than the approximately 2.4× hardware advantage of the A40, leaving about 1.4× after this approximate hardware normalization. One contributing factor is that 3DRoad requires a relatively large approximation rank while having only input dimensions, making the kernel matrix–vector products used by the reference method comparatively inexpensive.
| eff. | fit [s] | ref | low-rank mean RMSE | low-rank mean time [s] | float32 CG RMSE | float32 CG time [s] | float64 CG RMSE | float64 CG time [s] | |
| 50 | 49 | 4.10 | 307.3 | 0.6931 0.0051 | 0.01 | 0.6319 0.0211 | 33 | 0.6319 0.0211 | 47.6 |
| 100 | 100 | 6.26 | 201.1 | 0.6320 0.0069 | 0.01 | 0.5389 0.0110 | 33 | 0.5389 0.0110 | 94.1 |
| 200 | 198 | 10.98 | 114.7 | 0.5771 0.0030 | 0.03 | 0.4944 0.0115 | 33 | 0.4944 0.0115 | 110 |
| 250 | 249 | 13.32 | 94.6 | 0.5641 0.0018 | 0.03 | 0.4828 0.0112 | 33 | 0.4828 0.0113 | 128 |
| 400 | 397 | 22.54 | 55.9 | 0.5232 0.0062 | 0.06 | 0.3992 0.0194 | 33 | 0.3991 0.0195 | 487 |
| 500 | 496 | 27.45 | 45.9 | 0.5026 0.0033 | 0.07 | 0.3805 0.0102 | 33 | 0.3805 0.0101 | 553 |
| 800 | 795 | 50.96 | 24.7 | 0.4479 0.0073 | 0.15 | 0.3015 0.0112 | 33 | 0.3015 0.0113 | 1,420 |
| 1000 | 989 | 67.23 | 18.7 | 0.4241 0.0051 | 0.19 | 0.2876 0.0064 | 33 | 0.2871 0.0062 | 1,588 |
| 1600 | 1573 | 135.55 | 9.3 | 0.3811 0.0056 | 0.39 | 0.2557 0.0077 | 33 | 0.2502 0.0065 | 2,091 |
| 2000 | 1959 | 192.69 | 6.5 | 0.3627 0.0072 | 0.51 | 0.2371 0.0050 | 33 | 0.2316 0.0066 | 2,469 |
| 3000 | 2921 | 372.89 | 3.4 | 0.3302 0.0038 | 0.90 | 0.2041 0.0052 | 33 | 0.1915 0.0040 | 2,950 |
| 4000 | 3869 | 743.44 | 1.7 | 0.3062 0.0030 | 1.42 | 0.1953 0.0069 | 33 | 0.1753 0.0019 | 3,221 |
| 6000 | 5732 | 1,419.64 | 0.9 | 0.2716 0.0040 | 2.91 | 0.1822 0.0057 | 33 | 0.1511 0.0020 | 3,556 |
Table 4. Table 4: Test RMSE and prediction time of the three predictive means on 3DRoad. The reference RMSE is 0.194.
The effect of rank
Table 2 reports the test RMSE for every rank tested. Entries that match or improve upon the reference value in [15 Table 1] are shown in bold. The rank required to reach the reference varies by roughly two orders of magnitude and does not exhibit a simple dependence on either the number of training points or the input dimension .
As in the previous subsection, the full-kernel predictive mean is computed in fp64 for 3DRoad, Buzz, and HouseElectric when the fp32 CG solve does not attain sufficient accuracy. On Buzz, increasing the model rank does not lead to a corresponding improvement in the full-kernel predictive RMSE, and some of the best results occur at relatively small ranks.
The reference RMSE is reached at rank 100 for PoleTele and Bike, at rank 1600 for Kin40K, and at rank 3000 for both Song and 3DRoad. The variation cannot be explained by or alone. For example, Song, with , requires rank 3000, whereas HouseElectric, with , reaches the reference at rank 800. Similarly, 3DRoad has only input dimensions but requires a larger rank than PoleTele, which has . The required rank is instead strongly influenced by how rapidly the fitted kernel spectrum decays, together with the geometry of the particular dataset; these properties are not known before fitting.
| PoleTele (0.117) | Elevators (0.364) | Bike (0.074) | Kin40K (0.099) | Protein (0.550) | 3DRoad (0.194) | Song (0.761) | Buzz (0.300) | HouseElectric (0.052) | |
| 50 | 0.3595 | 0.3983 | 0.1812 | 0.6752 | 0.7912 | 0.6931 | 0.8333 | 0.4272 | 0.1012 |
| 100 | 0.3162 | 0.3903 | 0.1566 | 0.5743 | 0.7623 | 0.6320 | 0.8227 | 0.3796 | 0.0708 |
| 200 | 0.2744 | 0.3772 | 0.1114 | 0.4130 | 0.7325 | 0.5771 | 0.8145 | 0.3701 | 0.0596 |
| 250 | 0.2632 | 0.3742 | 0.1107 | 0.3757 | 0.7230 | 0.5641 | 0.8111 | 0.3619 | 0.0586 |
| 400 | 0.2438 | 0.3668 | 0.0932 | 0.3222 | 0.7004 | 0.5232 | 0.8069 | 0.3547 | 0.0566 |
| 500 | 0.2308 | 0.3646 | 0.0864 | 0.2965 | 0.6825 | 0.5026 | 0.8040 | 0.3442 | 0.0560 |
| 800 | 0.2010 | 0.3606 | 0.0646 | 0.2512 | 0.6565 | 0.4479 | 0.7988 | 0.3323 | 0.0542 |
| 1000 | 0.1882 | 0.3585 | 0.0687 | 0.2250 | 0.6437 | 0.4241 | 0.7966 | 0.3292 | 0.0536 |
| 1600 | 0.1698 | 0.3572 | 0.0517 | 0.1805 | 0.6180 | 0.3811 | 0.7922 | 0.0534 | |
| 2000 | 0.1549 | 0.3561 | 0.1613 | 0.6094 | 0.3627 | 0.7904 | 0.0526 | ||
| 3000 | 0.1416 | 0.3549 | 0.1344 | 0.5892 | 0.3302 | 0.7862 | 0.0522 | ||
| 4000 | 0.1198 | 0.1198 | 0.5805 | 0.3062 | 0.7838 | ||||
| 6000 | 0.1013 | 0.1013 | 0.5692 | 0.2716 | |||||
| 8000 | 0.0915 | 0.5598 |
Table 3. Test RMSE from the low-rank predictive mean for the fits of Table 2.
The rank also exposes a clear accuracy–cost tradeoff. On Bike, for example, the RMSE decreases from 0.0707 at rank 100 to 0.0303 at rank 1600, while the fitting time increases from 2.4 s to 10.2 s. Rank 100 is sufficient to match the reference, but a larger rank can substantially improve predictive accuracy at additional cost. A similar effect occurs on Kin40K, where the RMSE decreases to 0.0649 at rank 8000, compared with the reference value 0.099, although rank 1600 is already sufficient to reach the reference. Thus, the cheapest rank that matches the reference should not be interpreted as an accuracy-optimal rank.
Table 3 reports the same trained models but uses the predictive mean of the low-rank FITC model instead of solving the full-kernel system. This predictor requires no iterative full-kernel solve, can be evaluated in closed form using the low-rank factors, and therefore cannot suffer from iterative convergence failure. Across all reported configurations, its RMSE has a median relative increase of approximately 27% compared with the full-kernel predictive mean. It reaches the reference accuracy on Elevators, Bike, and Kin40K. The gap generally decreases as increases, but at markedly different rates across datasets. On HouseElectric, for example, the low-rank mean is within approximately 4% of the full-kernel result at , whereas on Buzz it remains at least 24% worse over the tested ranks.
| eff. | fit [s] | ref | low-rank mean RMSE | low-rank mean time [s] | float32 CG RMSE | float32 CG time [s] | float64 CG RMSE | float64 CG time [s] | |
| 50 | 46 | 22.71 | 1120.3 | 0.4272 0.0236 | 0.01 | 0.2783 0.0182 | 1,343 | 0.2672 0.0056 | 17,142 |
| 100 | 83 | 39.68 | 641.0 | 0.3796 0.0168 | 0.02 | 0.2709 0.0158 | 1,341 | 0.2630 0.0029 | 19,022 |
| 200 | 151 | 74.94 | 339.4 | 0.3701 0.0139 | 0.04 | 0.3021 0.0833 | 1,345 | 0.2655 0.0045 | 20,687 |
| 250 | 175 | 93.16 | 273.0 | 0.3619 0.0100 | 0.05 | 0.2723 0.0057 | 1,337 | 0.2642 0.0047 | 20,396 |
| 400 | 244 | 149.77 | 169.8 | 0.3547 0.0160 | 0.08 | 0.3047 0.0265 | 1,344 | 0.2648 0.0039 | 20,945 |
| 500 | 281 | 186.65 | 136.3 | 0.3442 0.0071 | 0.10 | 0.3132 0.0282 | 1,341 | 0.2638 0.0040 | 20,949 |
| 800 | 393 | 308.35 | 82.5 | 0.3323 0.0080 | 0.18 | 0.3563 0.0582 | 1,344 | 0.2622 0.0032 | 22,623 |
| 1000 | 461 | 389.44 | 65.3 | 0.3292 0.0033 | 0.25 | 0.3414 0.0271 | 1,336 | 0.2628 0.0030 | 22,060 |
Table 5. Table 5: Test RMSE and prediction time of the three predictive means on Buzz. The reference RMSE is 0.300.
Tables 4–6 compare the low-rank predictive mean with the fp32 and fp64 full-kernel predictive solves on the three largest datasets for which these comparisons are most informative. Results are averaged over five splits. The “effective rank” (eff.) denotes the numerical rank of ; when it is substantially smaller than the nominal landmark count, additional landmarks contribute little independent numerical information. The CG columns report predictive-solve times, whereas the column ref reports the fitting-time speedup relative to Maddox. Entries that reach the Maddox reference RMSE are shown in bold.
The low-rank predictive mean requires at most 4.1 s in these experiments, compared with 33 s to 25 min for the fp32 full-kernel CG solve and up to 24 h for the fp64 solve. Consequently, for the low-rank predictor, fitting dominates prediction time, whereas the full-kernel solve can itself become a dominant component of the total cost.
HouseElectric illustrates the advantage of avoiding the full-kernel solve. The fp32
CG solve does not attain the required accuracy at any tested rank, while the low-rank mean reaches an RMSE of 0.0536 at , within the standard deviation of the reference (0.052 ± 0.002). Its prediction requires 0.9 s following a 544 s fit, and the fitting stage is 78.5× faster than the reported Maddox reference. The complete low-rank pipeline in this case remains in single precision. On 3DRoad and Buzz, by contrast, the low-rank predictive mean is inexpensive but does not reach the accuracy of the full-kernel predictor, so the latter remains necessary when the highest predictive accuracy is required.
The predictive-mean solve in single precision
For datasets with , we compute the full-kernel predictive mean using a dense factorization. For the four larger datasets, we instead use preconditioned CG in fp32, with kernel matrix–vector products evaluated using KeOps [5]. The preconditioner is constructed from the low-rank AnchorNet approximation using at most 200 landmarks.
The behavior of the fp32 solve varies substantially across datasets. It converges on Song, stagnates at a relative residual of approximately on 3DRoad, varies strongly across splits on Buzz, and does not attain the required accuracy on House-Electric; see Table 4–Table 6.
| eff. | fit [s] | ref | low-rank mean RMSE | low-rank mean time [s] | float32 CG RMSE | float32 CG time [s] | float64 CG RMSE | float64 CG time [s] | |
| 50 | 50 | 24.17 | 1768.7× | 0.03 | 1,522 | 12,353 | |||
| 100 | 90 | 44.45 | 961.8× | 0.06 | 834 | 11,040 | |||
| 200 | 157 | 86.92 | 491.8× | 0.14 | 223 | 13,651 | |||
| 250 | 181 | 108.54 | 393.9× | 0.17 | 336 | 13,651 | |||
| 400 | 252 | 191.95 | 222.7× | 0.33 | 272 | 17,570 | |||
| 500 | 286 | 238.73 | 179.1× | 0.40 | 289 | 19,528 | |||
| 800 | 393 | 431.20 | 99.1× | 0.74 | 305 | 24,752 | |||
| 1000 | 449 | 544.37 | 78.5× | 0.91 | 272 | 26,128 | |||
| 1600 | 778 | 1,022.81 | 41.8× | 1.73 | 535 | 42,254 | |||
| 2000 | 1011 | 1,368.83 | 31.2× | 2.26 | 1,812 | 56,746 | |||
| 3000 | 1723 | 2,459.89 | 17.4× | 4.13 | 1,811 | 86,133 |
Table 6. Test RMSE and prediction time of the three predictive means on House-Electric. The reference RMSE is 0.052.
This behavior is closely related to conditioning. Since is positive semidefinite,
When is small, reducing can therefore increase the condition number approximately in proportion to . In finite precision, the attainable accuracy of an iterative solve generally deteriorates as the product of the unit roundoff and the condition number grows. Thus, the small fitted noise levels are an important contributor to the observed loss of accuracy in fp32. For example, the fitted values are approximately on Buzz and on HouseElectric.
The preconditioner can introduce an additional finite-precision difficulty. Let
and let denote the low-rank factor constructed from the first landmarks. We compute a thin QR factorization
The preconditioner is then applied in the form
When a substantial component of lies in the span of , the two vectors and can be close, so their subtraction is susceptible to cancellation in single precision. The resulting rounding error is subsequently multiplied by . This amplification factor is approximately 16 on Buzz and 400 on HouseElectric. The observation is consistent with Figure 3, where increasing the preconditioner rank on HouseElectric can worsen the true residual.

Figure 3. True residual of CG for different preconditioner ranks for HouseElectric in fp32, at model rank 500 (left) and 3000 (right).
Repeating the full-kernel solve in fp64 removes this accuracy barrier in the cases considered here. However, the A40 has much lower fp64 than fp32 throughput, so these reference solves can require many hours. We therefore use fp64 only when needed to establish the predictive accuracy attainable by the full-kernel model. A more detailed analysis of these solves is provided in Appendix § 10.
Mixed-precision experiments
The training algorithm described above is implemented primarily in fp32. Profiling shows that approximately two-thirds of the fitting time is spent in dense matrix multiplications (GEMMs) and triangular solves (TRSMs), making these operations natural candidates for lower-precision arithmetic. We therefore investigate whether selected operations can be performed with fp16 inputs while retaining fp32 accumulation. Our experiments show that the attainable speedup is modest and, more importantly, that the resulting computation is not reliable for all datasets.
In the implementation considered here, the triangular solves associated with the Cholesky factors remain in fp32. Consequently, only four large GEMMs in each objective-and-gradient evaluation are candidates for reduced precision. One of these forms the matrix
which is similar to and is the matrix factored by Cholesky. In exact arithmetic,
The remaining three GEMMs appear only in the gradient and therefore do not directly affect either the objective value or the Cholesky factorization.
We consider two mixed-precision variants, both using fp16 inputs with fp32 accumulation. The first performs the GEMM forming and one gradient GEMM in reduced precision; we refer to this as the two-GEMM variant. The second performs all four large GEMMs in reduced precision. For the four-GEMM variant, the GEMM inputs are scaled by powers of two to avoid overflow; without this scaling, one gradient block exceeds the dynamic range of fp16 near the fitted solution and produces NaN length-scale gradients. The two-GEMM variant is useful for isolating the stability of the GEMM that forms , whereas the four-GEMM variant represents the largest precision reduction considered without changing the triangular solves.
Table 7 compares single fits on 3DLoad with using fp32 and the two mixed-precision variants. All runs use split 73, the ARD kernel, and 100 Adam iterations with the same cosine learning-rate schedule. The two-GEMM variant produces nearly the same fitted hyperparameters and test RMSE as the fp32 run: 0.3261 compared with 0.3258, while reducing the fit time from 465 s to 449 s, a speedup of only 1.04×. Using all four GEMMs in reduced precision lowers the fit time to 424 s (1.10× speedup), with a test RMSE of 0.3277.
| fp32 | 2 GEMMs in fp16 | 4 GEMMs in fp16 | |
| Fit time (s) | 465 | 449 | 424 |
| Final train NLL | 0.2365 | 0.2373 | 0.2406 |
| Median length scale | 0.03906 | 0.03906 | 0.03911 |
| 0.5930 | 0.5907 | 0.5899 | |
| 0.2308 | 0.2308 | 0.2303 | |
| Effective rank | 2921 | 2919 | 2931 |
| Test RMSE | 0.3258 | 0.3261 | 0.3277 |
| Reduced-precision M used | – | 74 of 100 | 79 of 100 |
Table 7. Single fits on 3DRoad with using fp32 and two mixed-precision variants.
Because these results are based on a single fit for each precision configuration, the small differences in objective value and RMSE should not be interpreted as systematic accuracy degradation. At the fp32 fitted hyperparameters, however, we can compare the gradients directly. The relative gradient error is 1.65% for the two-GEMM variant and 1.35% for the four-GEMM variant. Thus, for this particular problem, reducing the precision of these GEMMs introduces only a modest perturbation to an individual gradient evaluation but provides correspondingly modest performance gains.
The limited speedup is consistent with the operation mix of the algorithm. The triangular solves, kernel evaluations, and several other operations remain in fp32. For the two-GEMM variant, the speedup of a single objective-and-gradient evaluation grows with rank, from no measurable improvement at to approximately 1.13× at on 3DRoad; see Table 8. At , using all four GEMMs in reduced precision gives an evaluation speedup of approximately 1.15×. At the level of a complete fit, however, the observed speedup remains close to 1.1×.
| Dataset | Factorization | Rel. gradient error | Time fp32 / mixed (s) | ||
| 3DRoad | 250 | succeeds | 0.029 | 0.16 / 0.17 | |
| 3DRoad | 1000 | succeeds | 0.034 | 0.95 / 0.91 | |
| 3DRoad | 3000 | succeeds | 0.016 | 4.58 / 4.20 | |
| 3DRoad | 6000 | succeeds | 0.012 | 13.6 / 12.0 | |
| Buzz | 50 to 1000 | to | fails (7/8) | 4.2 | – |
| HouseElectric | 50 to 3000 | to | fails (9/11) | 3.0 and 29.6 | – |
Table 8. Single objective-and-gradient evaluations with the two-GEMM mixed-precision variant. The condition number is evaluated from the fp32 baseline matrix.
Stability of the reduced-precision factorization
The main numerical difficulty is the GEMM used to form M. To distinguish exact-arithmetic properties from finite-precision effects, let
denote the exact matrix, let denote the matrix obtained using the fp32 baseline computation, and let denote the corresponding matrix when the selected GEMM is evaluated with fp16 inputs and fp32 accumulation.
Although in exact arithmetic, rounding in the matrix products can perturb the computed matrices. We write
As a rough error model, the norm of can grow proportionally to the fp16 unit roundoff and the scale of the matrix product. By Weyl’s inequality, positive definiteness of the computed matrix is guaranteed whenever
This suggests the heuristic condition
although the actual behavior also depends on the structure and scaling of the matrix multiplication.
Table 8 reports one mixed-precision objective-and-gradient evaluation per rank at hyperparameters obtained from the fp32 fit on split 73. On 3DRoad, the reduced-precision factorization succeeds at every tested rank between 50 and 6000, even though at . Thus, the heuristic condition is conservative for this problem.
The behavior is substantially worse on Buzz and HouseElectric. Their estimated condition numbers lie between approximately and . The reduced-precision factorization fails for 7 of the 8 tested ranks on Buzz and for 9 of the 11 tested ranks on HouseElectric. At the few ranks where the factorization succeeds, the relative gradient errors are already large. We therefore do not attempt complete reduced-precision fits on these two datasets.
These experiments also show that conditioning at the final fitted hyperparameters does not characterize the entire optimization trajectory. Table 9 follows the 3DRoad two-GEMM fit through selected Adam iterations. The exact-arithmetic matrix remains positive definite by construction, but the computed matrices can lose this property because of rounding.
| Iteration | Factorization | |||||
| 0 | 1.00 | 1.414 | 1.000 | 0.997 | succeeds | |
| 7 | 2.07 | 0.611 | 1.000 | 0.975 | succeeds | |
| 8 to 10 | 2.25 to 2.61 | 0.55 to 0.57 | to | 0.999 | Inf | overflow |
| 16 | 3.64 | 0.634 | 0.936 | 0.222 | succeeds | |
| 18 to 40 | 4.07 to 1.44 | 0.60 to 0.30 | to | 0.87 to 0.99 | to | fails |
| 44 | 1.01 | 0.291 | 0.997 | 0.535 | succeeds | |
| Final | 0.59 | 0.231 | 1.00 | – | succeeds |
Table 9. Computed spectra along the 3DRoad mixed-precision fit. denotes the fp32 baseline matrix and the matrix obtained from the reduced-precision GEMM.
Between iterations 18 and 40, the signal-to-noise ratio ranges from approximately 25 to 120, compared with 6.6 near the final iterate. Over this part of the trajectory, the largest eigenvalue of the fp32 baseline matrix grows to between and , while the reduced-precision matrix becomes indefinite, with its smallest eigenvalue between and . At iterations 8–10, the reduced-precision GEMM instead overflows.
Because the reduced-precision matrix still admits a Cholesky factorization at iteration 7, despite a large spectral range, the condition number is best interpreted as an indicator of numerical risk rather than a deterministic breakdown criterion.
To allow the mixed-precision fit to complete, we recompute the GEMM forming in fp32 whenever the Cholesky factorization of fails. Without this fallback, the 3DRoad fit terminates at iteration 8. With the fallback, the reduced-precision matrix is accepted in 74 of the 100 Adam iterations for the two-GEMM variant and in 79 of 100 iterations for the four-GEMM variant. The power-of-two scaling used in the latter also prevents the observed overflow. The fallback, however, detects only factorization failure; it does not guarantee that a successful reduced-precision factorization produces sufficiently accurate gradients. Instead of recomputing in fp32, the failed factorization can be repaired by raising the eigenvalues of below 1 to 1, which uses . On 3DRoad, this keeps all four GEMMs in fp16 in every iteration and gives a test RMSE of 0.3259, compared with 0.3258 in fp32. Such a modified factorization is therefore a possible fix when the loss of positive definiteness is the primary failure, but not on Buzz and HouseElectric, where the relative gradient error remains between 5 and 650 after the repair.
We also tested TF32, which retains a larger dynamic range but has reduced mantissa precision, and observed behavior qualitatively similar to fp16. This indicates that overflow is not the only limitation: even when scaling prevents overflow, insufficient precision can still perturb M enough to compromise its factorization or substantially degrade the gradient.
Buzz and HouseElectric are also the datasets for which is already close to numerically rank deficient in fp32, with observed numerical ranks of 443 out of 1000 and 1724 out of 3000, respectively. This is consistent with their greater sensitivity to further precision reduction.
Overall, these experiments show that selected GEMMs in the AnchorNet–FITC training procedure can be executed below fp32 for moderately conditioned problems, but the resulting performance gain is small and the approach is not robust for ill-conditioned cases. In particular, a Cholesky-based training method requires the computed reduced matrix to remain positive definite, which cannot in general be guaranteed after aggressive precision reduction. This observation is consistent with the use of iterative methods rather than Cholesky factorization for half-precision GP training in [15].
Conclusion
We developed a scalable Gaussian process training approach that combines AnchorNet landmark selection with an FITC correction and evaluates the resulting objective and analytic gradients using low-rank factors and small dense matrices. Across nine regression datasets, using the same kernel, initialization, optimizer, and optimization budget while varying the approximation rank, ANGP reaches the reference predictive accuracy on all nine datasets. The corresponding fitting speedups range from to , although these timings are obtained on an A40 GPU with roughly a single-precision throughput advantage over the V100 used for the reference results.
Our experiments also highlight an important distinction between training and prediction in reduced precision. The low-rank training procedure can be carried out effectively in single precision, whereas the full-kernel predictive solve becomes numerically challenging on 3DRoad, Buzz, and HouseElectric. On these datasets, the fp32 iterative solve does not attain sufficient accuracy and an fp64 solve is required to establish the full-kernel reference prediction. The low-rank predictive mean avoids this solve entirely and requires only a few seconds. It reaches the reference accuracy on Elevators, Bike, and Kin40K, and on HouseElectric it reaches the reference within its reported standard deviation while keeping the complete training-and-prediction pipeline in single precision.
We further investigated whether selected dense matrix operations in the training procedure can be reduced below fp32. Using fp16 or TF32 for selected GEMMs provides only modest additional speedups and can become unreliable when the reduced system is ill-conditioned. In particular, rounding can destroy the positive definiteness required by the Cholesky factorization even though the corresponding matrix is positive definite in exact arithmetic. These results suggest that fp32 provides a useful balance between performance and numerical robustness for the proposed training procedure, while more aggressive precision reduction requires additional stabilization.
The low-rank predictive mean is therefore an attractive first choice when prediction cost is important. When its accuracy is insufficient, the full-kernel predictive mean can be computed as a second stage, using higher precision when required by the conditioning of the problem. This suggests a practical adaptive strategy in which the low-rank prediction is computed first and its accuracy is assessed on validation data before deciding whether the more expensive full-kernel solve is necessary. Developing a reliable criterion for this decision, together with automatic selection of the approximation rank, is an interesting direction for future work.
Acknowledgments. The authors used Claude to assist with coding, including parameter initialization, parameter sweeps, and plotting and table formatting, and ChatGPT to assist with manuscript editing and revision. All AI-assisted code and manuscript content were checked and verified by the authors. The authors assume responsibility for all content.
Experimental setup
Initialization. All datasets use the same initialization. The length scales start at the median pairwise distance of a random subsample of 2,000 training inputs,
with the same value for all length scales. The signal scale starts at , since the targets are standardized, and the noise at the mode of the prior on , i.e. . A large initial noise keeps well conditioned at the start; a small one multiplies the rounding errors by and was the main cause of failed fits in single precision. Each split is a single run from this starting point, without restarts.
Optimization. We run Adam for iterations with gradients clipped to norm 10 and the learning rate
and .
The decay allows one learning rate for all datasets: 3DRoad has to move 4.1 in from its starting point, PoleTele only 0.12. There is no validation split and no early stopping; we report the final iterate. In none of the fits reported here did a length scale collapse or the noise reach its floor.
Gradient path. The blocked and unblocked gradient implementations agree to in test RMSE. The unblocked path holds several arrays in memory, while the blocked path processes the data in blocks of 20,000 rows. We therefore use the blocked path by default and switch to the unblocked one only if a single timed evaluation shows it to be at least 20% faster (§ 6). This margin does not guarantee that the unblocked path fits in memory; in the runs reported here it did.
The predictive-mean solve
The exact predictive mean requires the solution of
at the full training size. We use preconditioned CG with KeOps matrix–vector products, at most 200 iterations, and an AnchorNet preconditioner built from the first landmarks of the model. Figure 2 shows a fp32 solve on Buzz at rank 250 over 500 iterations. Up to about iteration 250, the CG recurrence residual, the preconditioned residual norm, and the true residual agree. After that, the first two keep decreasing to and , while the true residual stays at . A stopping criterion based on either cheap quantity would therefore report convergence that has not happened. The preconditioned norm is particularly misleading, since removes the residual component in the span of the landmarks, which matters most for prediction. The test RMSE is flat from iteration 200 onward (0.2659 at iteration 200, 0.2675 at 500), so more iterations do not help: the remaining error is a floor set by single precision.

Figure 2. Residuals (top) and test RMSE (bottom) of fp32 CG with a rank-200 AnchorNet preconditioner for Buzz with rank 250.
On HouseElectric, the fp32 solve does not converge for any preconditioner rank in below the model rank, at and (Figure 3). The preconditioner here is built from the landmarks of the model, so its rank is at most the model rank. The true residual worsens as the preconditioner rank grows, and no preconditioner is the best setting. This is consistent with the factor in , which amplifies its own rounding error.
In fp64, with a rank-1000 preconditioner, the solve converges on both datasets (Figure 4). On HouseElectric, the true residual reaches at rank 500. These solves provide the reference values in Table 4–Table 6. On Buzz, the fp32 solve at rank 250 gives 0.2723, only 2.4% above the fp64 value of 0.2659, while the low-rank mean gives 0.3619. There, the loss from single precision is much smaller than the loss from the low-rank predictive mean. On HouseElectric, the opposite holds: fp32 fails, and the low-rank mean is within 0.002 to 0.006 of the fp64 result from rank 200 upward.

Figure 4. True residual (top) and test RMSE (bottom) of the fp64 solves on HouseElectric (left) and Buzz (right). Dotted lines show the low-rank predictive mean at the same rank, in the same color.
References
- [1]N. Alder and R. Herbrich, Energy-efficient Gaussian processes using low-precision arithmetic, in Forty-first International Conference on Machine Learning, 2024.
- [2]C. M. Bishop and N. M. Nasrabadi, Pattern recognition and machine learning, vol. 4, Springer, 2006.
- [3]D. Cai, E. Chow, and Y. Xi, Posterior covariance structures in gaussian processes, SIAM Journal on Matrix Analysis and Applications, 46 (2025), pp. 1640–1673, https://doi.org/10.1137/24M1684918.
- [4]D. Cai, J. Nagy, and Y. Xi, Fast deterministic approximation of symmetric indefinite kernel matrices with high dimensional datasets, SIAM J. Matrix Anal. Appl., 43 (2022), pp. 1003–1028, https://doi.org/10.1137/21M142627.
- [5]B. Charlier, J. Feydy, J. A. Glaunès, F.-D. Collin, and G. Durif, Kernel operations on the GPU, with Autodiff, without memory overflows, J. Mach. Learn. Res., 22 (2021), pp. 1–6.
- [6]Y. Chen, E. N. Epperly, J. A. Tropp, and R. J. Webber, Randomly pivoted Cholesky: Practical approximation of a kernel matrix with few entry evaluations, Comm. Pure Appl. Math., 78 (2025), pp. 995–1041.arxiv.org/abs/2207.06503
- [7]A. Cortinovis and D. Kressner, On randomized trace estimates for indefinite matrices with an application to determinants, Found. Comput. Math., 22 (2022), pp. 875–903, https://doi.org/10.1007/s10208-021-09525-9.
- [8]K. Dong, D. Eriksson, H. Nickisch, D. Bindel, and A. G. Wilson, Scalable log determinants for Gaussian process kernel learning, Adv. Neural Inf. Process. Syst., 30 (2017).arxiv.org/abs/1711.03481
- [9]K. Dong, D. Eriksson, H. Nickisch, D. Bindel, and A. G. Wilson, Scalable log determinants for Gaussian process kernel learning, in Proceedings of the 31st International Conference on Neural Information Processing Systems, 2017, p. 6330–6340, https://dl.acm.org/doi/pdf/10.5555/3295222.3295380.arxiv.org/abs/1711.03481
- [10]E. N. Epperly, J. A. Tropp, and R. J. Webber, Embrace rejection: Kernel matrix approximation by accelerated randomly pivoted Cholesky, SIAM J. Matrix Anal. Appl., 46 (2025), pp. 2527–2557.arxiv.org/abs/2410.03969
- [11]J. Gardner, G. Pleiss, K. Q. Weinberger, D. Bindel, and A. G. Wilson, GPyTorch: Blackbox matrix-matrix Gaussian process inference with GPU acceleration, in Proceedings of the 32nd International Conference on Neural Information Processing Systems, 2018, pp. 7587–7597, https://dl.acm.org/doi/pdf/10.5555/3327757.3327857.
- [12]M. B. Giles, Collected matrix derivative results for forward and reverse mode algorithmic differentiation, in Advances in Automatic Differentiation, Springer, 2008, pp. 35–44.DOI
- [13]R. B. Gramacy, Surrogates: Gaussian process modeling, design, and optimization for the applied sciences, Chapman and Hall/CRC, 2020.DOI
- [14]H. Huang, T. Xu, Y. Xi, and E. Chow, Hogp: A high-performance python package for gaussian process, Journal of Open Source Software, 11 (2026), p. 8621, https://doi.org/10.21105/joss.08621, https://joss.theoj.org/papers/10.21105/joss.08621.
- [15]W. J. Maddox, A. Pattonski, and A. G. Wilson, Low-precision arithmetic for fast Gaussian processes, in Uncertainty in Artificial Intelligence, PMLR, 2022, pp. 1306–1316.arxiv.org/abs/2207.06856
- [16]R. A. Meyer, C. Musco, C. Musco, and D. P. Woodruff, Hutch++: Optimal stochastic trace estimation, in Symposium on Simplicity in Algorithms, 2021, pp. 142–155, https://doi.org/10.1137/1.9781611976496.16.
- [17]D. Persson, A. Cortinovis, and D. Kressner, Improved variants of the Hutch++ algorithm for trace estimation, SIAM J. Matrix Anal. Appl., 43 (2022), pp. 1162–1185, https://doi.org/10.1137/21M1447623.
- [18]J. Quinonero-Candela and C. E. Rasmussen, A unifying view of sparse approximate Gaussian process regression, J. Mach. Learn. Res., 6 (2005), pp. 1939–1959.
- [19]C. Rasmussen and C. Williams, Gaussian Processes for Machine Learning, Adaptive Computation and Machine Learning, The MIT Press, 2006, https://doi.org/10.7551/mitpress/3206.001.0001.
- [20]B. Schölkopf and A. J. Smola, Learning with kernels: support vector machines, regularization, optimization, and beyond, The MIT Press, 2001.
- [21]B. W. Silverman, Some aspects of the spline smoothing approach to non-parametric regression curve fitting, J. R. Stat. Soc. Ser. B Methodol., 47 (1985), pp. 1–21, https://doi.org/10.1111/j.2517-6161.1985.tb01327.x.
- [22]M. Skorski, A modern analysis of Hutchinson’s trace estimator, arXiv:2012.12895, (2020), https://doi.org/10.48550/arXiv.2012.12895.
- [23]E. Snelson and Z. Ghahramani, Sparse Gaussian processes using pseudo-inputs, in Proceedings of the 18th International Conference on Neural Information Processing Systems, 2005, p. 1257–1264.
- [24]S. Ubaru, J. Chen, and Y. Saad, Fast estimation of tr(f(A)) via stochastic Lanczos quadrature, SIAM J. Matrix Anal. Appl., 38 (2017), pp. 1075–1099, https://doi.org/10.1137/16M1104974.
- [25]T. Wagner, F. Nestler, T. Xu, Y. Xi, and M. Stoll, Preconditioned additive Gaussian processes with Fourier acceleration, Found. Data Sci., 12 (2027), pp. 131–157, https://doi.org/10.3934/fods.2026013.
- [26]J. Wenger, G. Pleiss, P. Hennig, J. Cunningham, and J. Gardner, Preconditioning for scalable Gaussian process hyperparameter optimization, in International Conference on Machine Learning, PMLR, 2022, pp. 23751–23780.arxiv.org/abs/2107.00243
- [27]C. K. Williams and C. E. Rasmussen, Gaussian processes for machine learning, vol. 2, The MIT Press, 2006.DOI
- [28]A. Wilson and H. Nickisch, Kernel interpolation for scalable structured Gaussian processes (KISS-GP), in Proceedings of the 32nd International Conference on International Conference on Machine Learning - Volume 37, 2015, pp. 1775–1784, https://dl.acm.org/doi/10.5555/3045118.3045307.arxiv.org/abs/1503.01057
- [29]K. Wu, J. Wenger, H. T. Jones, G. Pleiss, and J. Gardner, Large-scale Gaussian processes via alternating projection, in International Conference on Artificial Intelligence and Statistics, PMLR, 2024, pp. 2620–2628.arxiv.org/abs/2310.17137
- [30]T. Xu, D. Cai, H. Huang, E. Chow, and Y. Xi, Preconditioned truncated single-sample estimators for scalable stochastic optimization, 2026, https://arxiv.org/abs/2510.24587.
- [31]S. Zhao, T. Xu, H. Huang, E. Chow, and Y. Xi, An adaptive factorized Nyström preconditioner for regularized kernel matrices, SIAM Journal on Scientific Computing, 46 (2024), pp. A2351–A2376, https://doi.org/10.1137/23M1565139.