betaFeedback

WorksComputer Science

Variance Restoration over Covariance Rank: Memory-Budgeted Analytic Continual Learning

Abstract

Analytic continual learning replaces iterative gradient updates with closed-form ridge regression over frozen pre-trained representations, summarizing all past sessions in two running sufficient statistics: the feature autocorrelation matrix ΣtΣ_t and the class cross-correlation matrix ΓtΓ_t. Lifting backbone features into a wide random code of dimension M≫DM \gg D substantially improves linear separability across tasks, but forces ΣtΣ_t to grow quadratically as O(M2)\mathcal{O}(M^2). Under a fixed persistent memory envelope \Bcal, a learner faces a fundamental structural trade-off: store exact pairwise co-activations inside a narrower feature code, or expand to a wide code while compressing cross-coordinate interactions. We observe that the coordinate self-variances v_t = \diag(Σ_t) and class statistics ΓtΓ_t grow only linearly with MM, whereas off-diagonal interactions dominate memory. We introduce \method (Coordinate-Restored Analytic Continual Learning), which retains (vt,Γt)(v_t, Γ_t) exactly, maintains off-diagonal co-activations through a streaming rank-qq factor FtF_t, restores exact diagonal variances via \Sighat_t = F_t^\top F_t + \Diag(v_t - \diag(F_t^\top F_t)), and solves the full M×CM\times C normal equations via the Woodbury identity. Across five class-incremental benchmarks on frozen ViT-B/16, \method matches dense full-covariance readouts within 0.090.09 points while cutting persistent state by 90.7%90.7\% (35.935.9 vs.\ 385.3385.3 MiB). Under equal memory envelopes of 1616–8080 MiB, prioritizing expansion width MM over covariance rank qq outperforms exact narrow readouts in 1717 of 1818 configurations by up to +2.19+2.19 points.

Introduction

Class-incremental learning (CIL) requires a recognition system to assimilate new categories sequentially without reprocessing historical data or degrading previously acquired decision boundaries [1, 2]. Whereas rehearsal, weight-regularization, and parameter-isolation paradigms treat catastrophic forgetting as an optimization artifact of stochastic gradient descent, analytic continual learning reformulates incremental adaptation as exact recursive least squares over a frozen backbone [3, 4, 5, 6]. Because the cumulative ridge solution depends on past sessions solely through two additive sufficient statistics—the feature autocorrelation Σt\Sigma_t and the feature–label cross-correlation Γt\Gamma_t—analytic learners achieve order-invariant updates without storing raw exemplars.

To unlock strong linear separability from frozen features fθ(x)∈RDf_\theta(x) \in\mathbb{R}^D, recent representation-matching pipelines project embeddings into a high-dimensional random space of width M≫DM \gg D via dense nonlinear projections [7, 8, 9] or biologically inspired sparse TopK expansions [10, 11]. However, widening the readout code shifts the bottleneck from compute to persistent memory: at M=10,000M=10{,}000, storing a dense FP32 autocorrelation matrix Σt∈RM×M\Sigma_t \in\mathbb{R}^{M \times M} requires 381.5 MiB, whereas the class cross-correlation Γt∈RM×C\Gamma_t \in\mathbb{R}^{M \times C} for C=100C=100 classes requires only 3.8 MiB. When multiple incremental heads coexist on an edge accelerator or share a single frozen vision foundation model, this O(M2)O(M^2) covariance footprint dictates the maximum affordable width.

Constrained by a fixed memory envelope B\mathcal{B}, an analytic learner must choose between two competing designs that prior work has never disentangled: exact covariance at narrow width (MnM_n chosen so Mn2+MnC≃BM_n^2+M_nC \simeq\mathcal{B}) versus compressed covariance at wide width (M≫MnM \gg M_n paired with a rank-qq approximation of Σt\Sigma_t). Existing sketching methods such as Frequent Directions (FD) [12], Robust FD (RFD) [13, 14], and LoRanPAC [15] study spectral compression at a fixed dimension, while expansion methods vary MM without bounding memory [7, 10]. Because evaluations across papers simultaneously alter backbones, regularization rules, and memory budgets, it remains unclear whether a fixed byte budget should buy more coordinates or higher covariance fidelity.

Figure 1 reveals the structural asymmetry that governs this choice. In the cumulative normal equations, both the class cross-correlation Γt\Gamma_t (MCMC scalars) and the coordinate self-variances vt=diag⁡(Σt)v_t = \operatorname{diag}(\Sigma_t) (MM scalars) grow linearly with width; only the off-diagonal co-activations scale quadratically. Yet when Σt\Sigma_t is compressed into a streaming rank-qq factor FtTFtF_t^{\mathsf{T}}F_t, repeated spectral truncation erodes 20%–33% of the median coordinate self-variance across a ten-session stream (Figure 3a), leaving off-subspace feature directions with zero data curvature.

Figure 1. The width–interaction allocation trade-off in analytic continual learning. (a) Dense autocorrelation storage (Σt\Sigma_t) scales quadratically as O(M2)O(M^2), whereas CORE-ACL (q=840q=840) scales linearly with width MM and reduces persistent memory by 90.7% at M=104M=10^4. (b) Under fixed memory envelopes (B∈{16,36}\mathcal{B} \in\{16,36\} MiB) with GCV regularization, reallocating bytes from covariance rank qq to expansion width MM consistently improves stage-average accuracy Aˉ\bar{\mathcal{A}} over exact narrow ridge, except when rank falls below a dataset-specific floor (q=318q=318 on CIFAR-100). (c) Stage-average accuracy saturates rapidly as interaction rank qq grows, matching the 385 MiB full-covariance ceiling (dotted lines) already at q=840q=840 (35.9 MiB).

Figure 3. Mechanism diagnostics. (a) Median fraction of coordinate self-variance ∥Ft,:,i∥22/vi\lVert F_{t,:,i}\rVert_2^2/v_i captured by the rank-qq factor across sessions; CORE-ACL restores the diagonal to 1.0 exactly. (b) Session-1 CUB-200 accuracy across forced penalties γ\gamma: small γ\gamma triggers Γ⊥\Gamma_\perp blow-up in unreserved LR and uncalibrated CORE-ACL, whereas CORE-ACL⋆^\star selects γ=107\gamma=10^7 (star). (c) Cumulative accuracy trajectory AtA_t across 10 CIFAR-100 sessions.

Built on this insight, we propose CORE-ACL (Coordinate-Restored Analytic Continual Learning). CORE-ACL maintains (vt,Γt)(v_t,\Gamma_t) exactly in O(M+MC)O(M+MC) space, summarizes off-diagonal interactions via a two-stage rank-qq factor Ft∈Rq×MF_t \in\mathbb{R}^{q \times M}, and restores the exact diagonal variance through Σ^t=FtTFt+Diag⁡(vt−diag⁡(FtTFt))\widehat{\Sigma}_t = F_t^{\mathsf{T}}F_t + \operatorname{Diag}(v_t - \operatorname{diag}(F_t^{\mathsf{T}}F_t)) [16]. Applying the Woodbury identity yields the exact full-width M×CM \times C ridge classifier in O(Mq2+q3+MqC)O(Mq^2 + q^3 + MqC) time without materializing any M×MM \times M matrix. Furthermore, we uncover a critical pathology in full-width sketched readouts: when the ridge penalty γt\gamma_t is selected via hold-out validation on an untruncated subset (nfit≤qn_{\mathrm{fit}} \le q) and refitted on the full session (nrefit>qn_{\mathrm{refit}} > q), off-subspace label projections Γ⊥\Gamma_\perp are amplified by [17]. Our rank-calibrated hold-out rule (CORE-ACL*) matches the truncation ratio during validation, eliminating this collapse.

Our main contributions are summarized as follows:

  • We formulate the memory-budgeted width–interaction problem for analytic continual learning and isolate when expansion width MM outweighs covariance rank qq under matched storage, backbones, and class orders.

  • We propose CORE-ACL, a variance-restored streaming ridge solver that preserves exact per-coordinate variances and class cross-correlations while compressing only off-diagonal interactions into a rank-qq factor.

  • We establish Loewner positivity, Frobenius diagonal optimality, and a nullspace amplification theorem that explains hold-out refit collapse and motivates our rank-calibrated selection rule (CORE-ACL*).

  • Across CIFAR-100, CUB-200, VTAB-50, ImageNet-R, and ImageNet-A, CORE-ACL matches dense full-covariance readouts within 0.09 points using 90.7% less persistent state, while outperforming state-matched narrow exact ridge across diverse backbones and stream horizons.

Related Work

Pre-trained continual learning and parameter isolation. With strong pre-trained Vision Transformers (ViTs) [18, 19], exemplar-free CIL has shifted toward lightweight adaptation modules [1, 2]. Prompt-tuning approaches such as L2P [20], DualPrompt [21], and CODA-Prompt [22] steer frozen attention layers via task-queried prompt pools. Subspace and adapter methods—including EASE [23], InfLoRA [24], MOS [25], MoA [26], and rank-wise expert routing [27, 28]—allocate task-specific low-rank subspaces to limit gradient interference. Recent metric and transport frameworks further align frozen visual branches across sessions [29, 30, 31]. Unlike these methods, which modify backbone features via iterative gradient descent, CORE-ACL operates in the purely analytic regime with a fixed feature map and optimizes how the classifier’s sufficient statistics are budgeted.

Analytic continual learners and random feature lifting. Starting from ACIL [3], recursive least-squares updates have enabled exemplar-free continual learning with exact equivalence to joint ridge regression. Subsequent works introduced dual-stream compensation (DS-AL) [4], generalized objectives (GACL) [5], forward-only projections (F-OAL) [32], and representation enhancement (REAL) [6], with recent extensions to keyword spotting, autonomous driving, continual graphs, and long-tailed spectral rectification [33, 34, 35, 36, 37]. In parallel, RanPAC [7] and RanDumb [8] showed that lifting frozen features into M≫DM \gg D random coordinates before analytic fitting rivals complex prompt-tuning pipelines. Fly-CL [10, 38] replaced dense projections with sparse TopK Kenyon-cell expansions, while multi-view boosting (MVP) [9], guided projections (SCL-MGSM) [11], analytic contrastive projection (AnaCP) [39], class-level statistical estimators [40], and temporal-spectral reconstruction [41] further enriched lifted representations. However, these methods either retain a dense M×MM \times M autocorrelation matrix or compress it at a single fixed width without evaluating equal-memory narrow exact alternatives.

Streaming matrix sketches and low-rank-plus-diagonal models. Deterministic matrix sketching via Frequent Directions (FD) [12] and Robust FD (RFD) [13] approximates streaming Gram matrices by shrinking singular values, and streaming ridge sketches bound regression suboptimality [14]. LoRanPAC [15] applies continual truncated SVD to lifted features and restricts the classifier to the retained subspace. Diagonal-corrected low-rank covariances have classic roots in sparse Gaussian processes [16]. We unify these perspectives by treating spectral truncation, shrinkage, and exact variance restoration as competing allocations under a strict byte budget B\mathcal{B}.

Preliminaries and Problem Formulation

Analytic Readouts on Lifted Representations

We study exemplar-free class-incremental learning across TT sequential sessions {D1,…,DT}\{D_1,\ldots,D_T\} with disjoint label sets. Session tt arrives with ntn_t samples (Xt,Yt)(X_t,Y_t), where Yt∈{0,1}nt×CY_t \in\{0,1\}^{n_t \times C} denotes one-hot targets over the total class vocabulary CC. A frozen backbone fθ:X→RDf_\theta:\mathcal{X}\to\mathbb{R}^{D} extracts embeddings that are lifted into an MM-dimensional sparse code via a fixed random projection P∈RM×DP \in\mathbb{R}^{M \times D} with ss non-zero Gaussian entries per row followed by TopKk_k activation [10]:

ϕM(x)=TopK⁡k(Pfθ(x))∈RM.(1)\phi_M(x)=\operatorname{TopK}_k(Pf_\theta(x))\in\mathbb{R}^{M}. \tag*{(1)}

Stacking the lifted session features into Φt∈Rnt×M\Phi_t \in\mathbb{R}^{n_t \times M}, the cumulative ridge classifier at session tt with regularization penalty γt>0\gamma_t>0 is given in closed form by

Wt∗=(Σt+γtIM)−1Γt,Σt=∑j=1tΦjTΦj,Γt=∑j=1tΦjTYj.(2)W_t^*=(\Sigma_t+\gamma_t I_M)^{-1}\Gamma_t,\qquad\Sigma_t=\sum_{j=1}^{t}\Phi_j^\mathsf{T}\Phi_j,\qquad\Gamma_t=\sum_{j=1}^{t}\Phi_j^\mathsf{T}Y_j. \tag*{(2)}

At test time, an input xx is assigned to arg max⁡c∈{1,…,C}ϕM(x)TWt,:,c\operatorname*{arg\,max}_{c\in\{1,\ldots,C\}}\phi_M(x)^\mathsf{T}W_{t,:,c}. Because raw samples (Φj,Yj)(\Phi_j,Y_j) are discarded immediately after session jj, the learner’s persistent continuation state consists solely of the sufficient statistics required to compute Wt+1∗W_{t+1}^* when session t+1t+1 arrives.

The Memory-Budgeted Allocation Problem

The persistent state (Σt,Γt)(\Sigma_t,\Gamma_t) decomposes into two structurally distinct components according to their scaling in the expansion width MM:

  • Linear per-coordinate statistics O(M+MC)\mathcal{O}(M+MC): The class cross-correlation Γt∈RM×C\Gamma_t\in\mathbb{R}^{M\times C} (MCMC scalars) and the coordinate self-variance vector vt=diag⁡(Σt)∈RMv_t=\operatorname{diag}(\Sigma_t)\in\mathbb{R}^{M} (MM scalars). At M=10,000M=10{,}000 and C=100C=100, (vt,Γt)(v_t,\Gamma_t) occupies only 3.85 MiB in FP32.

  • Quadratic cross-coordinate interactions O(M2)\mathcal{O}(M^2): The off-diagonal co-activation entries of Σt\Sigma_t, which require M2M^2 scalars (381.47 MiB in dense FP32 at M=10,000M=10{,}000).

Given a fixed persistent memory budget B\mathcal{B} (in bytes or FP32 scalars), an exact narrow learner selects the largest width MnM_n satisfying Mn2+MnC≤BM_n^2+M_nC\leq\mathcal{B} and solves Eq. (2) without approximation. Conversely, a wide factored learner fixes a wide dimension M>MnM>M_n and approximates Σt\Sigma_t with a compressed representation Σ^t\widehat{\Sigma}_t whose total persistent state fits within B\mathcal{B}. Our central question is how B\mathcal{B} should be partitioned between coordinate width MM and interaction rank qq, and what structural invariants Σ^t\widehat{\Sigma}_t must preserve so that wide compression reliably beats narrow exactness.

CORE-ACL: Coordinate-Restored Analytic Learning

CORE-ACL exploits the linear scaling of (vt,Γt)(v_t,\Gamma_t) by storing both without loss while compressing only the quadratic off-diagonal interactions into a rank-qq factor Ft∈Rq×MF_t\in\mathbb{R}^{q\times M} (q≪Mq\ll M), as depicted in Figure 2.

Figure 2. Overview of CORE-ACL at session tt. I. Frozen backbone features are lifted into a wide sparse random code Φt∈Rnt×M\Phi_t \in\mathbb{R}^{n_t \times M}. II. Under persistent memory budget B\mathcal{B}, CORE-ACL splits state maintenance into an exact linear pathway for coordinate self-variances vtv_t and class cross-correlations Γt\Gamma_t, and a rank-qq spectral merge pathway for off-diagonal interactions FtF_t. III. Exact diagonal variance restoration repairs the truncated covariance into Σ^t\widehat{\Sigma}_t, and a Woodbury normal-equation solve yields the full-width classifier Wt∈RM×CW_t \in\mathbb{R}^{M \times C}.

Two-Stage Streaming Update

Upon receiving session Φt∈Rnt×M\Phi_t \in\mathbb{R}^{n_t \times M} and labels Yt∈{0,1}nt×CY_t \in\{0,1\}^{n_t \times C}, CORE-ACL updates the exact linear statistics:

vt=vt−1+∑i=1ntϕt,i⊙ϕt,i,Γt=Γt−1+ΦtTYt.(3)v_t = v_{t-1} + \sum_{i=1}^{n_t} \phi_{t,i} \odot\phi_{t,i}, \qquad\Gamma_t = \Gamma_{t-1} + \Phi_t^{\mathsf{T}}Y_t. \tag*{(3)}

For any matrix X∈Rℓ×MX \in\mathbb{R}^{\ell\times M} with economy singular value decomposition X=USVTX = USV^{\mathsf{T}}, define the rank-qq spectral truncation operator Tq(X)=S1:b,1:bV:,1:bT∈Rb×M\mathcal{T}_q(X) = S_{1:b,1:b}V_{:,1:b}^{\mathsf{T}} \in\mathbb{R}^{b \times M} with b=min⁡(q,ℓ,M)b = \min(q,\ell,M), so that Tq(X)TTq(X)\mathcal{T}_q(X)^{\mathsf{T}}\mathcal{T}_q(X) is the best rank-bb Frobenius approximation to XTXX^{\mathsf{T}}X. CORE-ACL compresses the current session and merges it with the historical factor Ft−1F_{t-1}:

Ut=Tq(Φt),Ft=Tq ⁣([Ft−1Ut]),F0=00×M.(4)U_t = \mathcal{T}_q(\Phi_t), \qquad F_t = \mathcal{T}_q\!\left(\begin{bmatrix}F_{t-1}\\U_t\end{bmatrix}\right), \qquad F_0 = 0_{0 \times M}. \tag*{(4)}

Each session discards a positive semidefinite (PSD) session tail Ωtsess=ΦtTΦt−UtTUt⪰0\Omega_t^{\mathrm{sess}} = \Phi_t^{\mathsf{T}}\Phi_t - U_t^{\mathsf{T}}U_t \succeq0 and a PSD merge tail Ωtmerge=[Ft−1;Ut]T[Ft−1;Ut]−FtTFt⪰0\Omega_t^{\mathrm{merge}} = [F_{t-1};U_t]^{\mathsf{T}}[F_{t-1};U_t] - F_t^{\mathsf{T}}F_t \succeq0. Consequently, the persistent state (Ft,vt,Γt)(F_t,v_t,\Gamma_t) requires exactly (qt+1+C)M(q_t + 1 + C)M scalars—scaling linearly in MM and remaining strictly independent of the number of sessions tt.

Exact Variance Restoration and Woodbury Readout

Let Ωt=Σt−FtTFt\Omega_t = \Sigma_t - F_t^{\mathsf{T}}F_t denote the cumulative discarded covariance and ρt=diag⁡(Ωt)\rho_t = \operatorname{diag}(\Omega_t) its diagonal. Because vt=diag⁡(Σt)v_t = \operatorname{diag}(\Sigma_t) is tracked exactly, CORE-ACL computes the exact per-coordinate variance deficit directly from its state and forms the variance-restored covariance operator:

Σ^t=FtTFt+Diag⁡(ρt),ρt=[vt−diag⁡(FtTFt)]+.(5)\widehat{\Sigma}_t = F_t^{\mathsf{T}}F_t + \operatorname{Diag}(\rho_t), \qquad\rho_t = \left[v_t-\operatorname{diag}(F_t^{\mathsf{T}}F_t)\right]_+. \tag*{(5)}

Algorithm 1: CORE-ACL (and CORE-ACL⋆^\star) session update

Input: Frozen fθf_\theta, sparse lift PP, state (Ft−1,vt−1,Γt−1)(F_{t-1},v_{t-1},\Gamma_{t-1}), session data (Xt,Yt)(X_t,Y_t), rank qq, penalty grid G\mathcal{G}

Output: Updated state (Ft,vt,Γt)(F_t,v_t,\Gamma_t) and readout WtW_t

Φt←TopK⁡κ(Pfθ(Xt));\Phi_t \leftarrow\operatorname{TopK}_\kappa(Pf_\theta(X_t));

vt←vt−1+∑i=1ntϕt,i⊙ϕt,i;Γt←Γt−1+ΦtTYt;v_t \leftarrow v_{t-1}+\sum_{i=1}^{n_t}\phi_{t,i}\odot\phi_{t,i};\quad\Gamma_t\leftarrow\Gamma_{t-1}+\Phi_t^{\mathsf{T}}Y_t;

Ut←Tq(Φt);γt←SELECTPENALTY⁡(G);U_t\leftarrow\mathcal{T}_q(\Phi_t);\quad\gamma_t\leftarrow\operatorname{SELECTPENALTY}(\mathcal{G});

Ft←Tq([Ft−1;Ut]);ρt←[vt−diag⁡(FtTFt)]+;F_t\leftarrow\mathcal{T}_q([F_{t-1};U_t]);\quad\rho_t\leftarrow[v_t-\operatorname{diag}(F_t^{\mathsf{T}}F_t)]_+;

Dγt←Diag⁡(ρt+γt1M);Ht←Dγt−1FtT;D_{\gamma_t}\leftarrow\operatorname{Diag}(\rho_t+\gamma_t\mathbf{1}_M);\quad H_t\leftarrow D_{\gamma_t}^{-1}F_t^{\mathsf{T}};

Nt←Dγt−1Γt;N_t\leftarrow D_{\gamma_t}^{-1}\Gamma_t;

Solve (Iqt+FtDγt−1FtT)Jt=FtNt(I_{q_t}+F_tD_{\gamma_t}^{-1}F_t^{\mathsf{T}})J_t=F_tN_t by Cholesky;

Wt←Nt−HtJt;W_t\leftarrow N_t-H_tJ_t;

return (Ft,vt,Γt,Wt)(F_t,v_t,\Gamma_t,W_t);

In exact arithmetic, diag⁡(Σ^t)=vt\operatorname{diag}(\widehat{\Sigma}_t)=v_t, meaning 100% of every coordinate’s self-energy is preserved and the approximation error Δt=Σt−Σ^t=Ωt−Diag⁡(ρt)\Delta_t=\Sigma_t-\widehat{\Sigma}_t=\Omega_t-\operatorname{Diag}(\rho_t) is purely off-diagonal. Defining the diagonal preconditioner Dγt=Diag⁡(ρt+γt1M)D_{\gamma_t}=\operatorname{Diag}(\rho_t+\gamma_t\mathbf{1}_M), the Woodbury identity solves the M×MM\times M normal equations (Σ^t+γtIM)Wt=Γt(\widehat{\Sigma}_t+\gamma_t I_M)W_t=\Gamma_t via a compact qt×qtq_t\times q_t Cholesky factorization:

Wt=Dγt−1Γt−Dγt−1FtT(Iqt+FtDγt−1FtT)−1FtDγt−1Γt.(6)W_t = D_{\gamma_t}^{-1}\Gamma_t-D_{\gamma_t}^{-1}F_t^{\mathsf{T}}\left(I_{q_t}+F_tD_{\gamma_t}^{-1}F_t^{\mathsf{T}}\right)^{-1}F_tD_{\gamma_t}^{-1}\Gamma_t. \tag*{(6)}

Crucially, because Dγt−1ΓtD_{\gamma_t}^{-1}\Gamma_t feeds the entire M×CM\times C cross-correlation matrix into WtW_t, the classifier is not restricted to row⁡(Ft)\operatorname{row}(F_t), in contrast to subspace-projected PCR or LoRAN-PAC [15].

Regularization Selection and Rank Calibration

We evaluate two standard penalty selection paradigms for choosing γt∈G\gamma_t \in\mathcal{G}. Under task-local Generalized Cross-Validation (GCV) [17], γt\gamma_t minimizes the leave-one-out proxy on the current session’s singular spectrum without splitting data. Under hold-out validation and refit, 90% of session tt (nfitn_{\mathrm{fit}} samples) updates a temporary state to score γ∈G\gamma\in\mathcal{G} on the remaining 10%, after which the learner refits on all nrefit=ntn_{\mathrm{refit}} = n_t samples. As we prove in Theorem 1, if nfit≤q<nrefitn_{\mathrm{fit}} \le q < n_{\mathrm{refit}} on session 1, the validation state is exact (Γ⊥val=0\Gamma_{\perp}^{\mathrm{val}} = 0) whereas the refit state is truncated (Γ⊥refit≠0\Gamma_{\perp}^{\mathrm{refit}} \ne0), causing small γt\gamma_t choices to blow up. Our rank-calibrated rule, CORE-ACL⋆, truncates the validation factor at proportional rank qval=⌊q⋅nfit/nrefit⌋<nfitq_{\mathrm{val}} = \lfloor q \cdot n_{\mathrm{fit}}/n_{\mathrm{refit}} \rfloor< n_{\mathrm{fit}} during SELECTPENALTY, exposing off-subspace sensitivity before γt\gamma_t is locked in.

Theoretical Analysis

We now formalize the algebraic invariants of variance restoration, its regularization equivalence, and the mechanism of off-subspace amplification. Complete proofs and extensions are provided in Appendix § 8.

Proposition 1 (Loewner Positivity and Optimal Diagonal Restoration). In exact arithmetic under Eqs. (3)–(5), for every session t≥1t \ge1: (i) Ωt=∑j=1t(Ωjsess+Ωjmerge)⪰0\Omega_t = \sum_{j=1}^{t}(\Omega_j^{\mathrm{sess}} + \Omega_j^{\mathrm{merge}}) \succeq0, ρt≥0\rho_t \ge0, Σ^t⪰0\widehat{\Sigma}_t \succeq0, and diag⁡(Σ^t)=vt\operatorname{diag}(\widehat{\Sigma}_t) = v_t; (ii) the residual error Δt=Σt−Σ^t\Delta_t = \Sigma_t - \widehat{\Sigma}_t satisfies −Diag⁡(ρt)⪯Δt⪯Ωt-\operatorname{Diag}(\rho_t) \preceq\Delta_t \preceq\Omega_t and ∥Δt∥2≤∥Ωt∥2≤tr⁡(Ωt)\lVert\Delta_t\rVert_2 \le\lVert\Omega_t\rVert_2 \le\operatorname{tr}(\Omega_t); and (iii) for any fixed factor FtF_t, u⋆=ρtu^\star= \rho_t is the unique minimizer of ∥Σt−FtTFt−Diag⁡(u)∥F2=∥Δt∥F2+∥ρt−u∥22\lVert\Sigma_t - F_t^{\mathsf{T}}F_t - \operatorname{Diag}(u)\rVert_F^2 = \lVert\Delta_t\rVert_F^2 + \lVert\rho_t-u\rVert_2^2.*

Proposition 1 guarantees that Σ^t+γtIM\widehat{\Sigma}_t + \gamma_t I_M is strictly positive definite for all γt>0\gamma_t > 0 and that ρt\rho_t achieves the largest possible Frobenius reduction over unrestored low-rank ridge (u=0u=0, reduction ∥ρt∥22\lVert\rho_t\rVert_2^2) and over isotropic shift (u=ρˉt1u=\bar{\rho}_t\mathbf{1}, reduction ∥ρt−ρˉt1∥22\lVert\rho_t-\bar{\rho}_t\mathbf{1}\rVert_2^2).

Proposition 2 (Coordinate-Adaptive Regularization View). For fixed (Ft,Γt,ρt)(F_t,\Gamma_t,\rho_t) and γ>0\gamma> 0, the readout Wt=(Σ^t+γIM)−1ΓtW_t = (\widehat{\Sigma}_t+\gamma I_M)^{-1}\Gamma_t uniquely minimizes the coordinate-weighted objective

L(W)=∥FtW∥F2+∑i=1M(γ+ρt,i)∥Wi,:∥22−2⟨Γt,W⟩.(7)\mathcal{L}(W)=\lVert F_tW\rVert_F^2+\sum_{i=1}^{M}(\gamma+\rho_{t,i})\lVert W_{i,:}\rVert_2^2-2\langle\Gamma_t,W\rangle. \tag*{(7)}

Consequently, isotropic restoration FtTFt+ρˉtIMF_t^{\mathsf{T}}F_t+\bar{\rho}_t I_M simply reparameterizes unrestored low-rank ridge WLR(γ)=(FtTFt+γIM)−1ΓtW_{\mathrm{LR}}(\gamma)=(F_t^{\mathsf{T}}F_t+\gamma I_M)^{-1}\Gamma_t via Wiso(γ)=WLR(γ+ρˉt)W_{\mathrm{iso}}(\gamma)=W_{\mathrm{LR}}(\gamma+\bar{\rho}_t), whereas CORE-ACL imposes a heterogeneous coordinate penalty γ+ρt,i\gamma+\rho_{t,i} tailored to each coordinate’s discarded variance.

Next, let Πt=VtVtT\Pi_t=V_tV_t^{\mathsf{T}} denote the orthogonal projector onto row⁡(Ft)\operatorname{row}(F_t), and decompose the class cross-correlation into in-subspace and nullspace components: Γt=ΠtΓt+(IM−Πt)Γt≜Γ∥+Γ⊥\Gamma_t=\Pi_t\Gamma_t+(I_M-\Pi_t)\Gamma_t\triangleq\Gamma_{\parallel}+\Gamma_{\perp}.

Theorem 1 (Nullspace Amplification under Truncating Refit). For unrestored low-rank ridge at penalty γ>0\gamma>0, the classifier decomposes orthogonally as

WLR(γ)=(FtTFt+γIM)−1Γ∥+γ−1Γ⊥,Γ∥TΓ⊥=0.(8)W_{\mathrm{LR}}(\gamma)=(F_t^{\mathsf{T}}F_t+\gamma I_M)^{-1}\Gamma_{\parallel}+\gamma^{-1}\Gamma_{\perp},\qquad\Gamma_{\parallel}^{\mathsf{T}}\Gamma_{\perp}=0. \tag*{(8)}

A label component aligned with a retained singular vector of singular value σi>0\sigma_i>0 is scaled by (σi2+γ)−1(\sigma_i^2+\gamma)^{-1}, whereas any nullspace component in Γ⊥\Gamma_{\perp} is scaled by γ−1\gamma^{-1}—a relative amplification of (σi2+γ)/γ(\sigma_i^2+\gamma)/\gamma. Whenever the factor is exact (FtTFt=Σ^tF_t^{\mathsf{T}}F_t=\widehat{\Sigma}_t), Γ⊥=0\Gamma_{\perp}=0. Under CORE-ACL, any unit vector u∈row⁡(Ft)⊥u\in\operatorname{row}(F_t)^\perp receives Rayleigh curvature

uT(Σ^t+γIM)u=γ+∑i=1Mρt,iui2∈[γ,γ+max⁡iρt,i].(9)u^{\mathsf{T}}(\widehat{\Sigma}_t+\gamma I_M)u=\gamma+\sum_{i=1}^{M}\rho_{t,i}u_i^2\in\left[\gamma,\gamma+\max_i\rho_{t,i}\right]. \tag*{(9)}

Theorem 1 pinpoints why hold-out validation can catastrophically fail on session 1: when nfit≤q<nrefitn_{\mathrm{fit}}\le q<n_{\mathrm{refit}}, the temporary validation factor is exact (Γ⊥val=0\Gamma_{\perp}^{\mathrm{val}}=0), so tiny penalties (γ=102\gamma=10^2) incur zero nullspace error. Upon refitting on all nrefitn_{\mathrm{refit}} samples, truncation creates a non-zero Γ⊥refit\Gamma_{\perp}^{\mathrm{refit}} that is multiplied by γ−1\gamma^{-1}. Early in the stream, FtF_t captures >98%>98\% of vtv_t, so ρt,i\rho_{t,i} is small and Eq. (9) stays near γ\gamma. By contrast, methods with an explicit spectral shrinkage floor α>0\alpha>0 (such as RFD) cap relative amplification at (σi2+γ+α)/(γ+α)(\sigma_i^2+\gamma+\alpha)/(\gamma+\alpha), and CORE-ACL⋆ prevents the tiny γ\gamma from being chosen by enforcing qval<nfitq_{\mathrm{val}}<n_{\mathrm{fit}} during validation.

Experiments

Benchmarks and Evaluation Protocol. We evaluate on five standard exemplar-free CIL benchmarks: 10-session splits of CIFAR-100 [42], CUB-200-2011 [43], ImageNet-R [44], and ImageNet-A [45], plus a 5-session split of VTAB-50 [46] (partition details in Appendix § 12). Following the pre-trained representation protocol of RanPAC and Fly-CL [7, 10], we use a frozen ViT-B/16 backbone (augreg2-in21k_ft_in1k, D=768D=768) [18] and lift features to M=10,000M=10{,}000 coordinates with s=300s=300 non-zeros per row and TopK sparsity k/M=0.3k/M=0.3. We report stage-average accuracy Aˉ=1T∑t=1TAt\bar{A}=\frac{1}{T}\sum_{t=1}^{T}A_t and final accuracy ATA_T, together with persistent state size in MiB (1 MiB=2201\ \mathrm{MiB}=2^{20} bytes, FP32 storage, FP64 normal-equation solve).

Comparison with State-of-the-Art Methods

Table 1 compares CORE-ACL (q=840q=840, GCV penalty selection on {106,…,109}\{10^6,\ldots,10^9\}) against prompt-tuning, adapter, and analytic continual learners across all five benchmarks. Three findings stand out:

  1. Near-lossless compression of the full lifted covariance: Compared with Fly-CL [10], which stores the exact 10,000×10,00010{,}000\times10{,}000 Gram matrix of the identical sparse expansion (385.3 MiB), CORE-ACL (35.9 MiB)

Table 1. Benchmark comparison on frozen ViT-B/16 across five class-incremental datasets (mean over 5 seeds). We report stage-average accuracy Aˉ\bar{A} (%), final accuracy ATA_T (%), and persistent continuation state (MiB, on CIFAR-100). Best overall accuracy is bold; best among compact-state analytic methods (< 60 MiB) is _underlined_.

MethodCIFAR-100 (T=10T=10) Aˉ↑\bar{A} \uparrowCIFAR-100 (T=10T=10) AT↑A_T \uparrowCUB-200 (T=10T=10) Aˉ↑\bar{A} \uparrowCUB-200 (T=10T=10) AT↑A_T \uparrowVTAB-50 (T=5T=5) Aˉ↑\bar{A} \uparrowVTAB-50 (T=5T=5) AT↑A_T \uparrowImageNet-R (T=10T=10) Aˉ↑\bar{A} \uparrowImageNet-R (T=10T=10) AT↑A_T \uparrowImageNet-A (T=10T=10) Aˉ↑\bar{A} \uparrowImageNet-A (T=10T=10) AT↑A_T \uparrowState MiB↓\downarrow
Prompt-tuning, subspace, and adapter continual learners
L2P [20]87.9681.8277.7266.8883.8273.4172.3465.1849.1541.08327.8
DualPrompt [21]88.6283.1979.4869.6584.1175.7071.6264.7553.4245.20328.6
CODA-Prompt [22]90.4185.3482.6474.1287.9080.4575.1868.9256.8448.76331.2
EASE [23]91.7486.9186.9281.6290.6884.9476.0870.1463.8555.9248.3
InfLoRA [24]90.8485.0479.3867.4186.5177.7274.6568.1059.1250.84102.1
Analytic continual learners and lifted covariance matchers
ACIL [3]92.1489.0889.0283.1294.9392.3873.8267.4562.1854.36518.2
DS-AL [4]92.3189.1889.6484.0595.0892.6474.5668.3263.4555.80524.5
GACL [5]92.4689.2590.1284.6895.1692.7875.1269.0464.2856.72518.2
REAL [6]92.5889.3490.8585.7495.2892.9675.6469.7865.4057.95518.2
RanPAC [7]92.8889.0092.5488.9695.1993.0276.5270.8970.1862.68385.3
LoRanPAC [15]93.2989.7992.7589.4295.3693.3377.6673.0670.8264.014218.3
Fly-CL (dense Σt\Sigma_t) [10]92.8388.8291.4887.1295.0692.5176.7471.1469.8562.08385.3
Compact-state analytic learners (< 60 MiB persistent memory)
F-OAL [32]92.0287.3590.3485.1295.9294.0674.9068.8564.0856.428.4
CORE-ACL (q=840q=840)_92.81__88.78__91.52__87.26_95.0792.52_76.65__71.02__69.78__61.96_35.9

Table 1.

stays within 0.09 points in Aˉ\bar{A} (−0.02, +0.04, +0.01, −0.09, −0.07 across the five benchmarks) and within 0.14 points in ATA_T, cutting persistent state by 90.7%.

  1. Superior accuracy over classical analytic and prompt baselines: Under a single-backbone feature stream, CORE-ACL surpasses classical recursive ridge learners (ACIL, DS-AL, GACL, REAL) by +0.67 to + 2.50 points on CUB-200 and by +4.38 to + 7.60 points on ImageNet-A, while outperforming EASE and CODA-Prompt across all five datasets.

  1. Honest positioning against unbudgeted and multi-block upper bounds: LoRanPAC [15] achieves the highest overall accuracy on four datasets, but its materialized implementation consumes 879–4,218 MiB (24–117× larger than CORE-ACL). Conversely, F-OAL [32] uses only 8.4 MiB and leads on VTAB-50 by fusing features from all 12 ViT blocks, but trails CORE-ACL by 0.79, 1.18, 1.75, and 5.70 points on CIFAR-100, CUB-200, ImageNet-R, and ImageNet-A.

Iso-Memory Width vs. Covariance Rank Allocation

To isolate the allocation question directly, we fix the persistent memory budget at B∈{16,36,80}\mathcal{B} \in\{16, 36, 80\} MiB and compare an Exact Narrow readout (largest rank MnM_n fitting in B\mathcal{B}) against wide factored readouts (M=10,000M=10{,}000, rank qq filling B\mathcal{B}). Under GCV selection (Figure 1b and Appendix Table 6), trading covariance rank for expansion width improves Aˉ\bar{A} in 17 of 18 width–rank cells across CIFAR-100, CUB-200, and VTAB-50:

  • Fine-grained and multi-domain streams favor maximal width: On CUB-200 and VTAB-50 at the tightest 16 MiB budget, pushing width to M=10,000M=10{,}000 (leaving ranks q=218q=218 and q=368q=368) beats Exact Narrow (Mn≈2,000M_n \approx2{,}000) by +2.19 [+1.68, +2.70] and +1.72 [+1.33, +2.11] points.

  • Rank starvation creates an interior optimum on dense spectra: On CIFAR-100 at 16 MiB, full width (M=10,000M=10{,}000, q=318q=318) suffers from rank starvation and drops −0.62 [−0.85, −0.39] points below Exact Narrow (90.52% vs. 91.14%), whereas an intermediate allocation (M=5,000M=5{,}000, q=738q=738) gains +0.78 [+0.48, +1.08] points (91.92%). Even against a stronger Packed Exact Narrow baseline that stores only the upper triangle 12Mn(Mn+1)\frac{1}{2}M_n(M_n+1) (widening MnM_n by 2≈1.41×\sqrt{2} \approx1.41\times), CORE-ACL maintains a clear lead of +1.06 to +1.65 points on CUB-200 and VTAB-50 (Appendix § 13.§ 5).

Table 2 evaluates the same B=36\mathcal{B}=36 MiB envelope when γt∈{102,…,109}\gamma_t \in\{10^2,\ldots,10^9\} is chosen by hold-out validation and refit. On CIFAR-100, variance-corrected rules (RFD, FD+var, CORE-ACL, CORE-ACL*) all achieve significant gains of +0.57 to + 0.60 points over Exact Narrow, recovering > 80% of the unbudgeted wide ceiling (92.35%), whereas unrestored LR (+0.07) does not. On CUB-200, however, standard hold-out validation triggers the Theorem 1 pathology: because Session-1 fit splits (nfit≈850n_{\mathrm{fit}} \approx850) barely fit inside qq while refits (nrefit≈945n_{\mathrm{refit}} \approx945) exceed qq, standard validation picks γ1=102\gamma_1=10^2 on two seeds and collapses Unrestored LR (86.12%), FD (86.34%), and uncalibrated CORE-ACL (88.68%). RFD avoids collapse via its isotropic shrinkage floor α\alpha, and CORE-ACL* eliminates the collapse at the source by calibrating the validation truncation rank (qvalq_{\mathrm{val}}), restoring CUB-200 accuracy to 91.89% (+0.45 over Exact Narrow).

Readout RuleCIFAR-100 Aˉ\bar{A}CIFAR-100 Δnar\Delta_{\mathrm{nar}}CUB-200 Aˉ\bar{A}CUB-200 Δnar\Delta_{\mathrm{nar}}VTAB-50 Aˉ\bar{A}VTAB-50 Δnar\Delta_{\mathrm{nar}}
Exact Narrow (MnM_n)91.620.0091.440.0093.810.00
Unrestored LR91.69↑0.07\uparrow0.0786.12↓5.32\downarrow5.3294.21↑0.40\uparrow0.40
Frequent Directions91.81↑0.19\uparrow0.1986.34↓5.10\downarrow5.1094.02↑0.21\uparrow0.21
Robust FD (RFD)92.22↑0.60\uparrow0.6091.88↑0.44\uparrow0.4494.56↑0.75\uparrow0.75
FD + Exact Var.92.19↑0.57\uparrow0.5791.58↑0.14\uparrow0.1494.35↑0.54\uparrow0.54
CORE-ACL (std. val.)92.20↑0.58\uparrow0.5888.68↓2.76\downarrow2.7694.40↑0.59\uparrow0.59
CORE-ACL⋆^\star (ours)92.21↑0.59\uparrow0.5991.89↑0.45\uparrow0.4594.41↑0.60\uparrow0.60
Unbudgeted Wide92.35+0.7392.02+0.5894.88+1.07

Table 2. Iso-memory comparison at B=36B=36 MiB under hold-out validation and refit (3 seeds, Aˉ\bar{A} in %). Unrestored LR, FD, and RFD receive q+1q+1 rows to match the MM variance scalars of CORE-ACL. CORE-ACL⋆^\star applies rank-calibrated hold-out selection.

Mechanism and Component Ablations

What must the state retain? Table 3 ablates the components of (Ft,vt,Γt)(F_t,v_t,\Gamma_t) under a shared GCV grid. Discarding off-diagonal interactions entirely (Diagonal only, Σ^t=Diag⁡(vt)\widehat{\Sigma}_t=\operatorname{Diag}(v_t)) degrades Aˉ\bar{A} by 9.17, 27.90, and 58.25 points across the three benchmarks, confirming that cross-coordinate co-activations are indispensable under standard regularization scales. Restricting the readout to the retained subspace (Subspace PCR, which projects Γt←ΠtΓt\Gamma_t\leftarrow\Pi_t\Gamma_t) drops VTAB-50 stage-average accuracy by 5.09 points (89.98% vs. 95.07%) while slightly raising final accuracy ATA_T (92.82% vs. 92.52%), demonstrating that off-subspace cross-correlations Γ⊥\Gamma_\perp are vital during early sessions before the rank-qq factor spans the full class geometry. Restoring exact coordinate variances in CORE-ACL improves CIFAR-100 by +0.63 points in Aˉ\bar{A} and +1.69 points in ATA_T over Unrestored LR on the exact same factor FtF_t.

State ConfigurationCIFAR-100 Aˉ\bar{A}CIFAR-100 ATA_TCUB-200 Aˉ\bar{A}CUB-200 ATA_TVTAB-50 Aˉ\bar{A}VTAB-50 ATA_T
Diagonal (vtv_t, Γt\Gamma_t) only83.6477.8963.6252.9436.8231.78
Subspace PCR (ΠtWt\Pi_t W_t)92.1987.7691.4286.4889.9892.82
Unrestored LR (FtF_t, Γt\Gamma_t)92.1887.0991.5087.3495.0492.48
Frequent Directions (FD)91.8085.8291.6487.4495.0892.61
Robust FD (RFD)92.6588.4991.3186.7295.0692.55
CORE-ACL (FtF_t, vtv_t, Γt\Gamma_t)92.8188.7891.5287.2695.0792.52

Table 3. Ablation of retained sufficient-statistic components (M=10,000M=10{,}000, q=840q=840, shared GCV grid, 5 seeds). Reporting both Aˉ\bar{A} and ATA_T (%) exposes the intermediate-vs.-final trade-off of span restriction.

Variance erosion and nullspace amplification. Figure 3a tracks the median variance capture ratio ∥Ft,:,i∥22/vt,i\lVert F_{t,:,i}\rVert_2^2/v_{t,i} across sessions t=1,…,10t=1,\ldots,10. As new classes rotate the feature distribution, a fixed rank qq retains a steadily shrinking share of coordinate variance, falling to 0.671 on CIFAR-100 at q=256q=256 (0.795 at q=840q=840). Without variance restoration, this lost energy directly weakens diagonal conditioning and causes the 8.18-point late-session accuracy deficit seen in Figure 4a. Meanwhile, Figure 3b visualizes Theorem 1 on Session 1 of CUB-200: although off-subspace label energy accounts for <0.035%<0.035\% of ∥Γ1∥F\lVert\Gamma_1\rVert_F, forcing γ=102\gamma=10^2 amplifies Γ⊥\Gamma_\perp until ∥(Σ^1+γI)−1Γ⊥∥F/∥W1∥F>0.998\lVert(\widehat{\Sigma}_1+\gamma I)^{-1}\Gamma_\perp\rVert_F/\lVert W_1\rVert_F>0.998, collapsing accuracy to 21.4% (LR) and 49.8% (CORE-ACL). CORE-ACL⋆^\star detects this amplification on the pre-truncated validation split and selects γ=107\gamma=10^7 (92.4%).

Figure 4. Stream fidelity, solver speedup, and cross-backbone transfer. (a) Session-wise gap At−AtfullA_t-A_t^{\mathrm{full}} relative to dense full covariance. (b) Normal-equation solve latency and temporary buffer on RTX 3090 (15.0× speedup and memory reduction). (c) Paired ΔAˉ\Delta\bar{A} gains of CORE-ACL over state-matched Exact Narrow across ResNet-50, DINOv2-g/14, and 20-session horizons.

Generalization and Computational Efficiency

Backbone and horizon generality. Figure 4c and Appendix Table 7 verify that the width-over-rank allocation holds across diverse visual backbones and stream lengths at matched 36 MiB state. On ResNet-50 [47], where raw features are less linearly separable, CORE-ACL beats state-matched Exact Narrow by +0.66, +3.38, and +0.43 points on CIFAR-100, CUB-200, and VTAB-50. On DINOv2 ViT-g/14 (D=1536D=1536) [19], gains remain consistently positive (+0.39, +0.94, +0.64), and on longer 20- and 50-session streams CORE-ACL matches dense Fly-CL within 0.06 points (92.76% vs. 92.80% on 50-session CIFAR-100).

Compute vs. memory trade-off. As shown in Figure 4b, the Woodbury normal-equation solve in Eq. (6) takes 4.9 ms and 102 MiB of temporary buffer versus 73.4 ms and 1,532 MiB for dense M×MM \times M Cholesky factorization—a 15.0× reduction in both latency and solver workspace. End-to-end session fitting on CIFAR-100 takes 11.4 s per session for CORE-ACL (dominated by the session SVD Tq(Φt)T_q(\Phi_t)) versus 24.6 s for RFD and 0.8 s for Exact Narrow (Appendix § 13.§ 6). Thus, CORE-ACL specifically optimizes persistent intersession memory and normal-equation solve cost rather than feature-extraction or SVD wall-clock time.

Conclusion

We investigated how a fixed persistent memory budget in analytic continual learning should be allocated between feature expansion width and covariance interaction fidelity. Because coordinate self-variances and class cross-correlations scale linearly with width whereas off-diagonal co-activations scale quadratically, CORE-ACL preserves the linear statistics exactly, compresses interactions into a streaming rank-qq factor, and restores exact diagonal variances via a fast Woodbury readout. Across five CIL benchmarks, CORE-ACL matches dense 385 MiB covariance readouts within 0.09 points in just 35.9 MiB, and demonstrates that spending bytes on width consistently beats narrow exactness once rank starvation and nullspace refit amplification (CORE-ACL⋆^\star) are controlled.

Mathematical Proofs and Extended Theory

Throughout this appendix, we assume exact arithmetic, a fixed feature lifting map ϕM:X→RM\phi_M : \mathcal{X} \to\mathbb{R}^{M}, a class vocabulary of size C≥2C \ge2, and ridge regularization parameter γ>0\gamma> 0. Rank-deficient session matrices and empty initial factors are permitted. For any matrix X∈Rℓ×MX \in\mathbb{R}^{\ell\times M}, let b(X)=min⁡(q,rank⁡(X),M)b(X) = \min(q,\operatorname{rank}(X), M) denote the number of singular rows retained by Tq(X)T_q(X), and write qt=b(Mt)q_t = b(M_t) where Mt=[Ft−1;Ut]M_t = [F_{t-1}; U_t] is the stacked factor at session tt. The session truncation residual and the merge truncation residual are defined as

Ωtsess=ΦtTΦt−UtTUt,Ωtmerge=MtTMt−FtTFt,Mt=[Ft−1Ut].(10)\Omega_t^{\mathrm{sess}} = \Phi_t^{\mathsf{T}}\Phi_t - U_t^{\mathsf{T}}U_t,\qquad\Omega_t^{\mathrm{merge}} = M_t^{\mathsf{T}}M_t - F_t^{\mathsf{T}}F_t,\qquad M_t = \begin{bmatrix} F_{t-1} \\ U_t \end{bmatrix}. \tag*{(10)}

Both Ωtsess\Omega_t^{\mathrm{sess}} and Ωtmerge\Omega_t^{\mathrm{merge}} are positive semidefinite (PSD) by the Eckart–Young–Mirsky theorem, since each equals a sum of outer products of discarded right singular vectors weighted by squared singular values.

Proof of Proposition 1 (Loewner Positivity, Operator Sandwich, and Optimal Diagonal Restoration)

Proof. We establish FtTFt⪯ΣtF_t^{\mathsf{T}}F_t \preceq\Sigma_t by induction over sessions t≥0t \ge0. At t=0t = 0, F0=0F_0 = 0 and Σ0=0\Sigma_0 = 0. Suppose Ft−1TFt−1⪯Σt−1F_{t-1}^{\mathsf{T}}F_{t-1} \preceq\Sigma_{t-1}. Because Ut=Tq(Φt)U_t = T_q(\Phi_t) discards only non-negative singular modes, UtTUt⪯ΦtTΦtU_t^{\mathsf{T}}U_t \preceq\Phi_t^{\mathsf{T}}\Phi_t. By block-matrix multiplication,

MtTMt=Ft−1TFt−1+UtTUt⪯Σt−1+ΦtTΦt=Σt.(11)M_t^{\mathsf{T}}M_t = F_{t-1}^{\mathsf{T}}F_{t-1} + U_t^{\mathsf{T}}U_t \preceq\Sigma_{t-1} + \Phi_t^{\mathsf{T}}\Phi_t = \Sigma_t. \tag*{(11)}

Applying TqT_q to MtM_t again drops a PSD tail Ωtmerge⪰0\Omega_t^{\mathrm{merge}} \succeq0, yielding FtTFt=MtTMt−Ωtmerge⪯MtTMt⪯ΣtF_t^{\mathsf{T}}F_t = M_t^{\mathsf{T}}M_t - \Omega_t^{\mathrm{merge}} \preceq M_t^{\mathsf{T}}M_t \preceq\Sigma_t. Consequently, the cumulative discarded covariance satisfies

Ωt=Σt−FtTFt=Ωt−1+Ωtsess+Ωtmerge=∑j=1t(Ωjsess+Ωjmerge)⪰0.(12)\Omega_t = \Sigma_t - F_t^{\mathsf{T}}F_t = \Omega_{t-1} + \Omega_t^{\mathrm{sess}} + \Omega_t^{\mathrm{merge}} = \sum_{j=1}^{t}\left(\Omega_j^{\mathrm{sess}} + \Omega_j^{\mathrm{merge}}\right) \succeq0. \tag*{(12)}

Because every diagonal entry of a PSD matrix is non-negative, ρt,i=(Ωt)i,i≥0\rho_{t,i} = (\Omega_t)_{i,i} \ge0 for all i∈{1,…,M}i \in\{1,\ldots,M\}. Thus Σ^t=FtTFt+Diag⁡(ρt)\widehat{\Sigma}_t = F_t^{\mathsf{T}}F_t + \operatorname{Diag}(\rho_t) is a sum of two PSD matrices and satisfies Σ^t⪰0\widehat{\Sigma}_t \succeq0. Since vt=diag⁡(Σt)v_t = \operatorname{diag}(\Sigma_t) is accumulated without approximation, diag⁡(Σ^t)=diag⁡(FtTFt)+vt−diag⁡(FtTFt)=vt\operatorname{diag}(\widehat{\Sigma}_t) = \operatorname{diag}(F_t^{\mathsf{T}}F_t) + v_t - \operatorname{diag}(F_t^{\mathsf{T}}F_t) = v_t.

For part (ii), write the approximation residual as Δt=Σt−Σ^t=Ωt−Diag⁡(ρt)\Delta_t = \Sigma_t - \widehat{\Sigma}_t = \Omega_t - \operatorname{Diag}(\rho_t). Since Diag⁡(ρt)⪰0\operatorname{Diag}(\rho_t) \succeq0, we immediately have Δt⪯Ωt\Delta_t \preceq\Omega_t. Conversely, since Ωt=Δt+Diag⁡(ρt)⪰0\Omega_t = \Delta_t + \operatorname{Diag}(\rho_t) \succeq0, we have −Diag⁡(ρt)⪯Δt-\operatorname{Diag}(\rho_t) \preceq\Delta_t. Because 0≤ρt,i=eiTΩtei≤∥Ωt∥20 \le\rho_{t,i} = e_i^{\mathsf{T}}\Omega_t e_i \le\|\Omega_t\|_2, the extreme eigenvalues of Δt\Delta_t lie in [−∥Ωt∥2,∥Ωt∥2][-\|\Omega_t\|_2,\|\Omega_t\|_2], which proves ∥Δt∥2≤∥Ωt∥2≤tr⁡(Ωt)=∑i=1Mρt,i\|\Delta_t\|_2 \le\|\Omega_t\|_2 \le\operatorname{tr}(\Omega_t) = \sum_{i=1}^{M}\rho_{t,i}.

For part (iii), fix FtF_t and consider any candidate diagonal vector u∈RMu \in\mathbb{R}^{M}. Decomposing Σt−FtTFt−Diag⁡(u)=Δt+Diag⁡(ρt−u)\Sigma_t - F_t^{\mathsf{T}}F_t - \operatorname{Diag}(u) = \Delta_t + \operatorname{Diag}(\rho_t-u), note that Δt\Delta_t has a strictly zero diagonal while Diag⁡(ρt−u)\operatorname{Diag}(\rho_t-u) is purely diagonal. Hence ⟨Δt,Diag⁡(ρt−u)⟩F=0\langle\Delta_t,\operatorname{Diag}(\rho_t-u)\rangle_F = 0, and the Pythagorean identity in Frobenius norm gives

∥Σt−FtTFt−Diag⁡(u)∥F2=∥Δt∥F2+∥ρt−u∥22,(13)\left\|\Sigma_t - F_t^{\mathsf{T}}F_t - \operatorname{Diag}(u)\right\|_F^2 = \|\Delta_t\|_F^2 + \|\rho_t-u\|_2^2, \tag*{(13)}

which is uniquely minimized at u⋆=ρtu^\star= \rho_t. Setting u=0u = 0 yields an excess Frobenius error of ∥ρt∥22\|\rho_t\|_2^2, and restricting u=c1Mu = c\mathbf{1}_M to an isotropic shift yields a minimum at c=ρˉt=1M∑i=1Mρt,ic = \bar{\rho}_t = \frac{1}{M}\sum_{i=1}^{M}\rho_{t,i} with excess Frobenius error ∥ρt−ρˉt1M∥22\|\rho_t-\bar{\rho}_t\mathbf{1}_M\|_2^2.

Proof of Proposition 2 (Coordinate-Adaptive Regularization Equivalence)

Proof. Expanding the Frobenius norm in Eq. (7) trace-wise gives

∥FtW∥F2+∑i=1M(γ+ρt,i)∥Wi,:∥22−2⟨Γt,W⟩=tr⁡(WT(FtTFt+Diag⁡(ρt)+γIM)W)−2tr⁡(WTΓt).(14)\|F_tW\|_F^2 + \sum_{i=1}^{M}(\gamma+\rho_{t,i})\|W_{i,:}\|_2^2 - 2\langle\Gamma_t,W\rangle= \operatorname{tr}\left(W^{\mathsf{T}}\left(F_t^{\mathsf{T}}F_t+\operatorname{Diag}(\rho_t)+\gamma I_M\right)W\right) - 2\operatorname{tr}(W^{\mathsf{T}}\Gamma_t). \tag*{(14)}

Since Σ^t+γIM=FtTFt+Diag⁡(ρt)+γIM≻0\widehat{\Sigma}_t+\gamma I_M = F_t^{\mathsf{T}}F_t+\operatorname{Diag}(\rho_t)+\gamma I_M \succ0 by Proposition 1, the objective is strictly convex in WW. Setting its matrix derivative 2(Σ^t+γIM)W−2Γt=02(\widehat{\Sigma}_t+\gamma I_M)W-2\Gamma_t=0 yields the unique global minimizer Wt=(Σ^t+γIM)−1ΓtW_t=(\widehat{\Sigma}_t+\gamma I_M)^{-1}\Gamma_t. When ρt\rho_t is replaced by an isotropic scalar ρˉt1M\bar{\rho}_t\mathbf{1}_M, the system matrix becomes FtTFt+(γ+ρˉt)IMF_t^{\mathsf{T}}F_t+(\gamma+\bar{\rho}_t)I_M, which coincides with the un-restored low-rank ridge operator WLR(γ+ρˉt)W_{\mathrm{LR}}(\gamma+\bar{\rho}_t).

Proof of Theorem 1 and Nullspace Stabilization Guarantees

Proof. Let Ft∈Rqt×MF_t \in\mathbb{R}^{q_t \times M} have economy SVD Ft=PFSFVF⊤F_t = P_F S_F V_F^\top with positive singular values σ1≥⋯≥σr>0\sigma_1 \ge\cdots\ge\sigma_r > 0 (r=rank⁡(Ft)≤qtr = \operatorname{rank}(F_t) \le q_t). The orthogonal projector onto row⁡(Ft)=col⁡(VF)\operatorname{row}(F_t) = \operatorname{col}(V_F) is Πt=VFVF⊤\Pi_t = V_F V_F^\top. Because Ft⊤Ft+γIM=VF(SF2+γIr)VF⊤+γ(IM−Πt)F_t^\top F_t + \gamma I_M = V_F(S_F^2 + \gamma I_r)V_F^\top+ \gamma(I_M - \Pi_t), its inverse acts as VF(SF2+γIr)−1VF⊤V_F(S_F^2 + \gamma I_r)^{-1}V_F^\top on col⁡(VF)\operatorname{col}(V_F) and as γ−1I\gamma^{-1}I on col⁡(VF)⊥\operatorname{col}(V_F)^\perp. Applying this inverse to Γt=Γ∥+Γ⊥\Gamma_t = \Gamma_\parallel+ \Gamma_\perp gives Eq. (8), where the two terms lie in orthogonal subspaces column by column. A unit component along vF,iv_{F,i} is scaled by (σi2+γ)−1(\sigma_i^2 + \gamma)^{-1} whereas a unit component in col⁡(VF)⊥\operatorname{col}(V_F)^\perp is scaled by γ−1\gamma^{-1}, giving the relative ratio (σi2+γ)/γ(\sigma_i^2 + \gamma)/\gamma. If the factor is exact (Ft⊤Ft=ΣtF_t^\top F_t = \Sigma_t), then row⁡(Ft)=row⁡([Φ1;…;Φt])\operatorname{row}(F_t) = \operatorname{row}([\Phi_1; \ldots; \Phi_t]), which contains the column space of Γt=∑j≤tΦj⊤Yj\Gamma_t = \sum_{j \le t} \Phi_j^\top Y_j, so Γ⊥=0\Gamma_\perp= 0. Finally, for any unit vector u∈row⁡(Ft)⊥u \in\operatorname{row}(F_t)^\perp, Ftu=0F_tu = 0, so u⊤(Σ^t+γIM)u=u⊤Diag⁡(ρt)u+γ=γ+∑i=1Mρt,iui2u^\top(\widehat{\Sigma}_t + \gamma I_M)u = u^\top\operatorname{Diag}(\rho_t)u + \gamma= \gamma+ \sum_{i=1}^M \rho_{t,i}u_i^2. □\square

Proposition 3 (Why Spectral Shrinkage Floors Prevent Refit Blow-Up). *Consider a spectral sketch with an isotropic floor α>0\alpha> 0, Σ^t(α)=Ft⊤Ft+αIM\widehat{\Sigma}_t^{(\alpha)} = F_t^\top F_t + \alpha I_M (as in RFD [13]). For any penalty γ>0\gamma> 0, off-subspace components in Γ⊥\Gamma_\perp are scaled by (γ+α)−1(\gamma+ \alpha)^{-1}, capping the relative amplification against mode σi\sigma_i at

Ramp(γ;α)=σi2+γ+αγ+α≤1+σi2α.(15)\mathcal{R}_{\mathrm{amp}}(\gamma;\alpha) = \frac{\sigma_i^2 + \gamma+ \alpha}{\gamma+ \alpha} \le1 + \frac{\sigma_i^2}{\alpha}. \tag*{(15)}

In particular, even as γ→0+\gamma\to0^+, the amplification remains bounded by 1+σi2/α<∞1 + \sigma_i^2/\alpha< \infty, whereas for unrestored low-rank ridge (α=0\alpha= 0) it diverges as O(γ−1)O(\gamma^{-1}).

Proposition 4 (Rank-Calibrated Validation Exposes Nullspace Sensitivity). *Suppose session 1 has nfit≤q<nrefitn_{\mathrm{fit}} \le q < n_{\mathrm{refit}} and rank⁡(Φfit)=nfit\operatorname{rank}(\Phi_{\mathrm{fit}}) = n_{\mathrm{fit}}. Standard validation at capacity qq retains all nfitn_{\mathrm{fit}} singular modes (Γ⊥val=0\Gamma_\perp^{\mathrm{val}} = 0). Under CORE-ACL⋆, the validation truncation rank qval=⌊q⋅nfit/nrefit⌋q_{\mathrm{val}} = \lfloor q \cdot n_{\mathrm{fit}}/n_{\mathrm{refit}} \rfloor satisfies qval≤nfit−1q_{\mathrm{val}} \le n_{\mathrm{fit}} - 1, forcing at least nfit−qval≥1n_{\mathrm{fit}} - q_{\mathrm{val}} \ge1 non-zero singular directions into the validation nullspace so that Γ⊥val≠0\Gamma_\perp^{\mathrm{val}} \ne0 whenever the held-out tail carries label correlation.

Proposition 5 (Retrospective Margin Transfer Bound). *Fit CORE-ACL (WtW_t) and unbudgeted wide ridge (Wt⋆W_t^\star) on the same lifted features at the same session tt and penalty γ>0\gamma> 0. Then

Wt−Wt⋆=(Σ^t+γIM)−1ΔtWt⋆,∥Wt−Wt⋆∥F≤∥ΔtWt⋆∥Fγ+min⁡iρt,i≤tr⁡(Ωt)∥Γt∥Fγ(γ+min⁡iρt,i).(16)W_t - W_t^\star= (\widehat{\Sigma}_t + \gamma I_M)^{-1}\Delta_t W_t^\star,\qquad\lVert W_t - W_t^\star\rVert_F \le\frac{\lVert\Delta_t W_t^\star\rVert_F}{\gamma+ \min_i \rho_{t,i}} \le\frac{\operatorname{tr}(\Omega_t)\lVert\Gamma_t\rVert_F}{\gamma(\gamma+ \min_i \rho_{t,i})}. \tag*{(16)}

Let μwide(z)=z⊤Wt,:,y⋆−max⁡c≠yz⊤Wt,:,c⋆\mu_{\mathrm{wide}}(z) = z^\top W^\star_{t,:,y} - \max_{c \ne y} z^\top W^\star_{t,:,c} denote the signed margin of z=ϕM(x)z = \phi_M(x) under Wt⋆W_t^\star, and define the margin-violation event Ξ={∣μwide(z)∣≤2∥z∥2∥Wt−Wt⋆∥F}\Xi= \{|\mu_{\mathrm{wide}}(z)| \le2\lVert z\rVert_2\lVert W_t - W_t^\star\rVert_F\}. For any narrow exact readout WnarrowW_{\mathrm{narrow}} evaluated on the same test distribution,

At(CORE−ACL)−At(narrow)≥At(wide)−At(narrow)−Pr⁡(Ξ).(17)A_t(\mathrm{CORE}-\mathrm{ACL}) - A_t(\mathrm{narrow}) \ge A_t(\mathrm{wide}) - A_t(\mathrm{narrow}) - \Pr(\Xi). \tag*{(17)}

Proof. Using the resolvent identity A−1−B−1=A−1(B−A)B−1A^{-1} - B^{-1} = A^{-1}(B-A)B^{-1} with A=Σ^t+γIMA = \widehat{\Sigma}_t + \gamma I_M and B=Σt+γIMB = \Sigma_t + \gamma I_M, we obtain Wt−Wt⋆=(Σ^t+γIM)−1(Σt−Σ^t)Wt⋆=(Σ^t+γIM)−1ΔtWt⋆W_t - W_t^\star= (\widehat{\Sigma}_t + \gamma I_M)^{-1}(\Sigma_t - \widehat{\Sigma}_t)W_t^\star= (\widehat{\Sigma}_t + \gamma I_M)^{-1}\Delta_t W_t^\star. Since Σ^t+γIM⪰(γ+min⁡iρt,i)IM\widehat{\Sigma}_t + \gamma I_M \succeq(\gamma+ \min_i \rho_{t,i})I_M, submultiplicativity and Proposition 1(ii) yield the norm bounds. For any test vector zz, Cauchy–Schwarz gives max⁡c∣z⊤(Wt−Wt⋆):,c∣≤∥z∥2∥Wt−Wt⋆∥F\max_c |z^\top(W_t - W_t^\star)_{:,c}| \le\lVert z\rVert_2\lVert W_t - W_t^\star\rVert_F. Thus, outside Ξ\Xi, the top-scoring class of WtW_t coincides with that of Wt⋆W_t^\star, establishing the accuracy transfer inequality. □\square

Worked Two-Coordinate Analytical Example

To see clearly why Theorem 1 distinguishes early-session refits from late-session updates, consider a 2-dimensional lifted space (M=2M=2) with rank-1 factor F=(1,0)F = (1, 0), ridge penalty γ=10−2\gamma= 10^{-2}, and class cross-correlation Γ=(0,2×10−2)⊤\Gamma= (0, 2 \times10^{-2})^\top, so that all label correlation lies orthogonal to row⁡(F)\operatorname{row}(F).

  • Unrestored Low-Rank Ridge (WLRW_{\mathrm{LR}}): The inverted matrix is diag⁡(1.01,0.01)−1\operatorname{diag}(1.01, 0.01)^{-1}, yielding readout weights WLR=(0,2.0)⊤W_{\mathrm{LR}} = (0, 2.0)^\top. A tiny off-subspace cross-correlation of 0.020.02 is amplified 100×100\times into a dominant weight of 2.02.0.

  • CORE-ACL in Late Stream (ρ2=0.50\rho_2 = 0.50, substantial discarded variance): The variance-restored matrix is diag⁡(1.01,0.51)\operatorname{diag}(1.01, 0.51), yielding WCORE=(0,0.02/0.51)⊤≈(0,0.039)⊤W_{\mathrm{CORE}} = (0, 0.02/0.51)^\top\approx(0, 0.039)^\top. The restored self-variance acts as a natural coordinate stabilizer.

  • CORE-ACL on Session-1 Refit (ρ2=0.01\rho_2 = 0.01, minimal discarded variance): Because the factor F1F_1 just crossed the rank cap qq, 99%99\% of coordinate variance is still inside F1F_1 (ρ2=0.01\rho_2 = 0.01). The system matrix is diag⁡(1.01,0.02)\operatorname{diag}(1.01, 0.02), returning WCORE=(0,1.0)⊤W_{\mathrm{CORE}} = (0, 1.0)^\top—still a 50×50\times blow-up. This explains why diagonal variance restoration alone cannot prevent Session-1 hold-out collapse at γ=102\gamma= 10^2, and why CORE-ACL⋆’s rank-calibrated validation split is essential.

Algorithmic and Numerical Implementation Details

Algorithm 2: Task-Local GCV Penalty Selection

Input: Session features Φt∈Rnt×M\Phi_t \in\mathbb{R}^{n_t \times M}, targets Yt∈{0,1}nt×CY_t \in\{0,1\}^{n_t \times C}, economy SVD Φt=LtStVt⊤\Phi_t = L_t S_t V_t^\top, grid G\mathcal{G}

Ψt←Lt⊤Yt\Psi_t \leftarrow L_t^\top Y_t;

for γ∈G\gamma\in\mathcal{G} do

νi(γ)←σi2/(σi2+γ)\nu_i(\gamma) \leftarrow\sigma_i^2/(\sigma_i^2+\gamma); df⁡(γ)←∑iνi(γ)\operatorname{df}(\gamma) \leftarrow\sum_i \nu_i(\gamma);

Y^t(γ)←LtDiag⁡(ν(γ))Ψt\widehat{Y}_t(\gamma) \leftarrow L_t\operatorname{Diag}(\nu(\gamma))\Psi_t;

GCV⁡(γ)←∥Yt−Y^t(γ)∥F2/nt(1−df⁡(γ)/nt)2\operatorname{GCV}(\gamma) \leftarrow\frac{\lVert Y_t-\widehat{Y}_t(\gamma)\rVert_F^2/n_t}{(1-\operatorname{df}(\gamma)/n_t)^2};

return arg⁡min⁡γ∈GGCV⁡(γ)\arg\min_{\gamma\in\mathcal{G}}\operatorname{GCV}(\gamma) (breaking ties toward smaller γ\gamma);

Algorithm 3: Hold-Out Validation with Rank Calibration (CORE-ACL*)

Input: Pre-session state (Ft−1,vt−1,Γt−1)(F_{t-1},v_{t-1},\Gamma_{t-1}), session data (Φt,Yt)(\Phi_t,Y_t), grid G\mathcal{G}, target capacity qq

Partition (Φt,Yt)(\Phi_t,Y_t) into stratified 90% fit split (Φfit,Yfit)(\Phi_{\mathrm{fit}},Y_{\mathrm{fit}}) (nfitn_{\mathrm{fit}} rows) and 10% val split (Φval,Yval)(\Phi_{\mathrm{val}},Y_{\mathrm{val}});

if using CORE-ACL* then

qval←max⁡(1,⌊q⋅nfit/nt⌋)q_{\mathrm{val}} \leftarrow\max(1,\lfloor q\cdot n_{\mathrm{fit}}/n_t\rfloor); // proportional truncation rank matching refit ratio

else

qval←qq_{\mathrm{val}} \leftarrow q; // standard uncalibrated validation

Form temporary state (F~t,v~t,Γ~t)(\widetilde{F}_t,\widetilde{v}_t,\widetilde{\Gamma}_t) from (Ft−1,vt−1,Γt−1)(F_{t-1},v_{t-1},\Gamma_{t-1}) and (Φfit,Yfit)(\Phi_{\mathrm{fit}},Y_{\mathrm{fit}}) using truncation rank qvalq_{\mathrm{val}};

Evaluate validation accuracy on (Φval,Yval)(\Phi_{\mathrm{val}},Y_{\mathrm{val}}) for every γ∈G\gamma\in\mathcal{G} via Eq. (6); select best γt\gamma_t;

Discard temporary state; update (Ft,vt,Γt)(F_t,v_t,\Gamma_t) on full (Φt,Yt)(\Phi_t,Y_t) at rank qq and solve for WtW_t at γt\gamma_t;

Numerical precision and clipping safeguards. All persistent states (Ft,vt,Γt)(F_t,v_t,\Gamma_t) are stored in IEEE-754 FP32. Before computing ρt=[vt−diag⁡(Ft⊤Ft)]+\rho_t=[v_t-\operatorname{diag}(F_t^\top F_t)]_+, we verify that min⁡i(vt,i−∥Ft,:,i∥22)≥−256εfp32max⁡(1,∥vt∥∞)\min_i(v_{t,i}-\lVert F_{t,:,i}\rVert_2^2)\ge-256\varepsilon_{\mathrm{fp32}}\max(1,\lVert v_t\rVert_\infty) to ensure that any negative entry is purely floating-point rounding rather than a violation of Loewner positivity. During the Woodbury normal-equation solve (Eq. (6)), FtF_t, ρt\rho_t, Γt\Gamma_t are cast to FP64 so that the qt×qtq_t \times q_t inner system Iqt+FtDγt−1Ft⊤I_{q_t}+F_tD_{\gamma_t}^{-1}F_t^\top is factored at double precision before casting WtW_t back to FP32. Across all stored systems, the relative normal-equation residual ∥(Σ^t+γtIM)Wt−Γt∥F/(∥Σ^t+γtIM∥2∥Wt∥F+∥Γt∥F)\lVert(\widehat{\Sigma}_t+\gamma_t I_M)W_t-\Gamma_t\rVert_F/(\lVert\widehat{\Sigma}_t+\gamma_t I_M\rVert_2\lVert W_t\rVert_F+\lVert\Gamma_t\rVert_F) remains below 10−1110^{-11}.

Extended Discussion and Positioning

Why Width Outweighs Covariance Rank Under a Fixed Byte Envelope

In an MM-dimensional random feature space, adding a new coordinate costs Mold+1+CM_{\mathrm{old}}+1+C scalars for a dense covariance learner, but only q+1+Cq+1+C scalars for CORE-ACL. At M=10,000M=10{,}000 and q=840q=840, each coordinate is >10×>10\times cheaper under CORE-ACL. Geometrically, random sparse lifting with TopK activation increases the minimum principal angle between class prototypes [7, 10], turning nonlinearly entangled backbone embeddings into linearly separable directions. Once classes are separated along distinct sparse coordinates, the normal equations primarily require accurate per-coordinate scaling (vtv_t) and leading shared principal components (FtF_t) to suppress background interference. Only when qq falls below the effective spectral dimension of the dataset (e.g., q=318q=318 on CIFAR-100 at 16 MiB, Figure 6a) does unmodeled off-diagonal correlation outweigh the separability gain of higher width.

Figure 6. Spectral decay and session-by-task accuracy heatmaps. (a) Normalized singular spectrum of the retained factor FTFTTF_TF_T^{\mathsf{T}}: CIFAR-100 exhibits a flatter tail than CUB-200 and VTAB-50, explaining why q=256q = 256 underfits CIFAR-100. (b, c) CIFAR-100 session-by-task accuracy matrices at 36 MiB for Exact Narrow vs. CORE-ACL.

Relationship to Class-Statistics and Subspace-Partitioned Analytic Learners

Recent ICML and NeurIPS studies have explored complementary routes to compact continual learning. Cao et al. [40] and Momeni et al. (AnaCP) [39] maintain class-conditional means or shrunk class covariances in the unexpanded DD-dimensional backbone space. While effective on CLIP-scale encoders, operating in RD\mathbb{R}^D forfeits the +1.5+1.5 to +3.5+3.5 point separability gain provided by M=10,000M=10{,}000 lifted coordinates on standard ViT-B/16 and ResNet-50 backbones. Conversely, LoRanPAC [15] lifts features to MM dimensions and truncates the SVD, but restricts the readout weights to row(Ft)(ΠtWt)(F_t)(\Pi_tW_t) and stores unmerged per-class/per-session factors in its released implementation (879–4,218 MiB). CORE-ACL bridges both worlds: it retains the full MM-dimensional cross-correlation Γt\Gamma_t without subspace projection while compressing the shared covariance into a single fixed-budget tuple (Ft,vt,Γt)(F_t, v_t, \Gamma_t).

Design Alternatives We Evaluated and Rejected

  • Per-class factored covariances: Storing a separate rank-qcq_c factor per class scales memory as O(CqcM)\mathcal{O}(Cq_cM) and prevents pooling statistical strength across classes to estimate shared background directions.

  • Hard nullspace projection of incoming tasks: Projecting Φt\Phi_t into row⁡(Ft−1)⊥\operatorname{row}(F_{t-1})^\perp eliminates cross-task covariance drift but destroys shared semantic features needed by fine-grained classes (such as bird species in CUB-200).

  • Trainable diagonal regularizers: Learning ρt\rho_t by gradient descent on each session breaks the closed-form recursive nature of analytic CIL and risks overfitting to the current session’s classes since old exemplars are absent. Proposition 1(iii) shows that ρt=vt−diag⁡(Ft⊤Ft)\rho_t = v_t - \operatorname{diag}(F_t^\top F_t) is already Frobenius-optimal in closed form.

Detailed Experimental Settings

Datasets and Incremental Splits.

  • CIFAR-100 [42]: 50,000 train and 10,000 test images (100 classes), split uniformly into T=10T=10 sessions (10 classes/session), plus T=20T=20 (5 classes/session) and T=50T=50 (2 classes/session) stress tests.

  • CUB-200-2011 [43]: Fine-grained recognition of 200 bird species (9,430 train / 2,358 test images in our local disjoint partition), evaluated over T=10T=10 (20 classes/session), T=20T=20, and T=50T=50 sessions.

  • VTAB-50 [46]: 50-class multi-domain benchmark (1,796 train / 8,619 test images across five visual domains), evaluated over T=5T=5 (10 classes/session) and T=10T=10 sessions.

  • ImageNet-R [44] & ImageNet-A [45]: 200-class domain-shifted and natural-adversarial benchmarks (T=10T=10, 20 classes/session), stressing covariance stability under severe distribution shift.

Exact Width and Rank Configurations under Memory Envelopes.

For a persistent memory budget B\mathcal{B} (in bytes, 4 bytes per FP32 scalar), Exact Narrow chooses the largest integer MnM_n such that Mn2+MnC≤B/4M_n^2 + M_nC \le\mathcal{B}/4, while Packed Exact Narrow chooses the largest MpM_p such that 12Mp(Mp+1)+MpC≤B/4\frac{1}{2}M_p(M_p + 1) + M_pC \le\mathcal{B}/4. At full width M=10,000M = 10{,}000, CORE-ACL chooses q=⌊(B/4−M−MC)/M⌋q = \lfloor(\mathcal{B}/4 - M - MC)/M\rfloor, while LR, FD, and RFD use q+1q + 1 rows:

Dataset (CC)Envelope B\mathcal{B}Dense Exact MnM_nPacked Exact MpM_pCORE-ACL Rank qq (M=104M=10^4)
CIFAR-100 (C=100C=100)16 / 36 / 80 MiB1,998 / 3,024 / 4,5322,802 / 4,250 / 6,382318 / 840 / 1,996
CUB-200 (C=200C=200)16 / 36 / 80 MiB1,950 / 2,975 / 4,4822,708 / 4,154 / 6,284218 / 740 / 1,896
VTAB-50 (C=50C=50)16 / 36 / 80 MiB2,022 / 3,048 / 4,5562,850 / 4,299 / 6,431368 / 890 / 1,796{sup}†`

Table 1 (PDF p. 14).

(†On VTAB-50 at 80 MiB, qq is capped by the 1,796 total training examples, leaving the state at 70.5 MiB.)

Comprehensive Supplementary Experiments

Full Benchmark Metrics Including Backward Transfer (BWT)

Sensitivity to Retained Subspace Rank qq

Complete Iso-Memory Width–Rank Grid (16 / 36 / 80 MiB)

Backbone Architectures and Long-Horizon Streams

Hold-Out Validation Across All Three Envelopes and Packed Baselines

Spectral Decay, Hyperparameter Sweeps, and Solver Profiling

DatasetMethodAˉ\bar{A} (%) ↑\uparrowATA_T (%) ↑\uparrowBWT (%) ↑\uparrowPersist. MiB ↓\downarrowTotal MiB ↓\downarrow
CIFAR-100L2P [20]87.96±1.9887.96 \pm1.9881.82±1.5181.82 \pm1.51−10.24±1.08-10.24 \pm1.08327.8327.8
CIFAR-100CODA-Prompt [22]90.41±1.2490.41 \pm1.2485.34±0.6885.34 \pm0.68−6.12±0.45-6.12 \pm0.45331.2331.2
CIFAR-100EASE [23]91.74±0.8891.74 \pm0.8886.91±0.2186.91 \pm0.21−6.42±0.58-6.42 \pm0.5848.348.3
CIFAR-100ACIL [3]92.14±1.1292.14 \pm1.1289.08±0.0689.08 \pm0.06−3.39±0.46-3.39 \pm0.46518.2566.2
CIFAR-100REAL [6]92.58±0.9492.58 \pm0.9489.34±0.0989.34 \pm0.09−3.58±0.34-3.58 \pm0.34518.2566.2
CIFAR-100F-OAL [32]92.02±0.9192.02 \pm0.9187.35±0.1487.35 \pm0.14−5.28±0.30-5.28 \pm0.308.411.7
CIFAR-100RanPAC [7]92.88±0.8992.88 \pm0.8989.00±0.1089.00 \pm0.10−4.34±0.26-4.34 \pm0.26385.3418.4
CIFAR-100LoRanPAC [15]93.29±0.8193.29 \pm0.8189.79±0.1189.79 \pm0.11−3.78±0.19-3.78 \pm0.194218.34549.5
CIFAR-100Fly-CL [10]92.83±0.8792.83 \pm0.8788.82±0.1288.82 \pm0.12−4.36±0.28-4.36 \pm0.28385.3423.4
CIFAR-100CORE-ACL (q=840q=840)92.81±0.8692.81 \pm0.8688.78±0.1188.78 \pm0.11−4.41±0.27-4.41 \pm0.2735.974.1
CUB-200L2P [20]77.72±1.8677.72 \pm1.8666.88±1.8266.88 \pm1.82−10.81±2.68-10.81 \pm2.68328.1328.1
CUB-200CODA-Prompt [22]82.64±1.4582.64 \pm1.4574.12±1.1274.12 \pm1.12−8.14±1.52-8.14 \pm1.52331.5331.5
CUB-200EASE [23]86.92±1.2986.92 \pm1.2981.62±0.1981.62 \pm0.19−5.89±0.47-5.89 \pm0.4751.851.8
CUB-200ACIL [3]89.02±0.4489.02 \pm0.4483.12±0.2783.12 \pm0.27−6.05±0.35-6.05 \pm0.35524.5572.5
CUB-200REAL [6]90.85±0.5190.85 \pm0.5185.74±0.2485.74 \pm0.24−4.92±0.38-4.92 \pm0.38524.5572.5
CUB-200F-OAL [32]90.34±0.6090.34 \pm0.6085.12±0.2885.12 \pm0.28−5.29±0.52-5.29 \pm0.529.212.9
CUB-200RanPAC [7]92.54±0.4592.54 \pm0.4588.96±0.6588.96 \pm0.65−3.61±0.48-3.61 \pm0.48389.1426.0
CUB-200LoRanPAC [15]92.75±0.2892.75 \pm0.2889.42±0.1489.42 \pm0.14−3.62±0.27-3.62 \pm0.273177.73546.9
CUB-200Fly-CL [10]91.48±0.5891.48 \pm0.5887.12±0.2687.12 \pm0.26−4.48±0.33-4.48 \pm0.33389.1431.1
CUB-200CORE-ACL (q=840q=840)91.52±0.5591.52 \pm0.5587.26±0.2987.26 \pm0.29−4.47±0.36-4.47 \pm0.3639.781.7
VTAB-50L2P [20]83.82±3.3183.82 \pm3.3173.41±4.2573.41 \pm4.25−11.88±4.42-11.88 \pm4.42327.7327.7
VTAB-50CODA-Prompt [22]87.90±1.9487.90 \pm1.9480.45±2.3880.45 \pm2.38−8.42±2.15-8.42 \pm2.15331.0331.0
VTAB-50EASE [23]90.68±2.1690.68 \pm2.1684.94±2.3284.94 \pm2.32−8.04±2.68-8.04 \pm2.686.56.5
VTAB-50ACIL [3]94.93±0.5394.93 \pm0.5392.38±0.4192.38 \pm0.41−3.48±0.58-3.48 \pm0.58515.1563.1
VTAB-50REAL [6]95.28±0.4895.28 \pm0.4892.96±0.3592.96 \pm0.35−3.12±0.49-3.12 \pm0.49515.1563.1
VTAB-50F-OAL [32]95.92±0.3495.92 \pm0.3494.06±0.2894.06 \pm0.28−2.82±0.55-2.82 \pm0.558.011.1
VTAB-50RanPAC [7]95.19±0.4595.19 \pm0.4593.02±0.3193.02 \pm0.31−3.05±0.45-3.05 \pm0.45383.4414.6
VTAB-50LoRanPAC [15]95.36±0.4095.36 \pm0.4093.33±0.2593.33 \pm0.25−2.79±0.35-2.79 \pm0.35879.31191.3
VTAB-50Fly-CL [10]95.06±0.4595.06 \pm0.4592.51±0.3192.51 \pm0.31−3.49±0.58-3.49 \pm0.58383.4419.6
VTAB-50CORE-ACL (q=840q=840)95.07±0.4695.07 \pm0.4692.52±0.3292.52 \pm0.32−3.48±0.62-3.48 \pm0.6234.070.2

Table 4. Complete 5-seed benchmark results (mean ±\pm SD) on CIFAR-100, CUB-200, and VTAB-50, complementing Table 1 with Backward Transfer (BWT, %) and total non-backbone deployment footprint (Total MiB, including projection PP and readout WTW_T).

Rank qqCIFAR-100 Aˉ\bar{A} (%)CIFAR-100 ATA_T (%)CUB-200 Aˉ\bar{A} (%)CUB-200 ATA_T (%)VTAB-50 Aˉ\bar{A} (%)VTAB-50 ATA_T (%)CIFAR State (MiB)
12881.24±1.1281.24 \pm1.1277.92±0.3877.92 \pm0.3890.76±0.5990.76 \pm0.5985.51±0.1685.51 \pm0.1694.74±0.5094.74 \pm0.5091.76±0.3991.76 \pm0.398.7
25687.52±0.9987.52 \pm0.9980.58±0.1580.58 \pm0.1591.42±0.5791.42 \pm0.5786.82±0.2686.82 \pm0.2695.12±0.4395.12 \pm0.4392.72±0.2492.72 \pm0.2413.6
51292.04±0.9192.04 \pm0.9187.08±0.2387.08 \pm0.2391.64±0.4791.64 \pm0.4787.48±0.1887.48 \pm0.1895.09±0.4495.09 \pm0.4492.59±0.2992.59 \pm0.2923.4
84092.81±0.8692.81 \pm0.8688.78±0.1188.78 \pm0.1191.52±0.5591.52 \pm0.5587.26±0.2987.26 \pm0.2995.07±0.4695.07 \pm0.4692.52±0.3292.52 \pm0.3235.9
153692.84±0.8692.84 \pm0.8688.81±0.0888.81 \pm0.0891.45±0.5891.45 \pm0.5887.10±0.2787.10 \pm0.2795.06±0.4595.06 \pm0.4592.51±0.3192.51 \pm0.3162.5
307292.83±0.8592.83 \pm0.8588.82±0.0788.82 \pm0.0791.44±0.5991.44 \pm0.5987.09±0.2387.09 \pm0.2395.06±0.4595.06 \pm0.4592.51±0.3192.51 \pm0.31121.0

Table 5. Sensitivity of CORE-ACL to retained interaction rank q∈{128,256,512,840,1536,3072}q \in\{128, 256, 512, 840, 1536, 3072\} at M=10,000M = 10{,}000 (5 seeds, GCV).

Table 6. Iso-memory width–rank allocations under GCV (3 seeds). In 17 of 18 wide allocations, CORE-ACL improves over Exact Narrow; the sole negative cell is CIFAR-100 at 16 MiB full width (q=318q = 318), where intermediate width (M=5,000M = 5{,}000, q=738q = 738) succeeds.

EnvelopeAllocation (MM / qq)CIFAR-100 A‾\overline{A} (%)CIFAR-100 ATA_T (%)CUB-200 A‾\overline{A} (%)CUB-200 ATA_T (%)VTAB-50 A‾\overline{A} (%)VTAB-50 ATA_T (%)
16 MiBExact Narrow (MnM_n / MnM_n)91.1486.9689.3884.4592.9889.82
16 MiBIntermediate (M=5kM=5\mathrm{k} / qmidq_{\mathrm{mid}})91.9288.1290.7686.0494.1291.45
16 MiBFull Width (M=10kM=10\mathrm{k} / qfullq_{\mathrm{full}})90.5285.3491.5787.1894.7092.36
36 MiBExact Narrow (MnM_n / MnM_n)91.4887.5289.9485.0893.6690.84
36 MiBIntermediate (M=5kM=5\mathrm{k} / qmidq_{\mathrm{mid}})91.9488.1590.6885.8294.1691.50
36 MiBFull Width (M=10kM=10\mathrm{k} / qfullq_{\mathrm{full}})92.3788.7591.5287.3894.7192.29
80 MiBExact Narrow (MnM_n / MnM_n)91.8588.0690.6685.8694.1591.48
80 MiBIntermediate (M=6.4kM=6.4\mathrm{k} / qmidq_{\mathrm{mid}})92.0888.4591.1486.5594.5291.94
80 MiBFull Width (M=10kM=10\mathrm{k} / qfullq_{\mathrm{full}})92.3688.7391.4887.1694.7392.31

Table 6.

Table 7. Generalization across ResNet-50, DINOv2 ViT-g/14, and 20-/50-session horizons at matched 36 MiB state (5 seeds; 3 for T=50T=50).

Backbone / HorizonMethodCIFAR-100 A‾\overline{A} (%)CIFAR-100 ATA_T (%)CUB-200 A‾\overline{A} (%)CUB-200 ATA_T (%)VTAB-50 A‾\overline{A} (%)VTAB-50 ATA_T (%)
ResNet-50 (T=10T=10)Fly-CL (dense Σt\Sigma_t)84.1576.7478.3169.5893.3189.90
ResNet-50 (T=10T=10)Exact Narrow (36 MiB)82.6073.9875.1865.9692.8589.15
ResNet-50 (T=10T=10)CORE-ACL (36 MiB)83.2675.8278.5669.6693.2889.80
DINOv2 ViT-g/14 (T=10T=10)Fly-CL (dense Σt\Sigma_t)96.2994.1993.6891.2496.7294.72
DINOv2 ViT-g/14 (T=10T=10)Exact Narrow (36 MiB)95.8693.5092.7589.8096.0793.76
DINOv2 ViT-g/14 (T=10T=10)CORE-ACL (36 MiB)96.2594.1093.6991.3296.7194.70
ViT-B/16 (T=20/20/10T=20/20/10)Fly-CL (dense Σt\Sigma_t)93.0488.8191.6687.0695.6192.80
ViT-B/16 (T=20/20/10T=20/20/10)Exact Narrow (36 MiB)92.2787.5690.2885.2494.7491.45
ViT-B/16 (T=20/20/10T=20/20/10)CORE-ACL (36 MiB)93.0388.7591.6987.1595.6092.78
ViT-B/16 (T=50T=50)Fly-CL (dense Σt\Sigma_t)92.8088.7492.0287.1495.7192.50
ViT-B/16 (T=50T=50)Exact Narrow (36 MiB)92.0487.4890.6285.3894.8691.52
ViT-B/16 (T=50T=50)CORE-ACL (36 MiB)92.7688.7092.1087.3595.7092.49

Table 7.

Figure 5. Supplementary controls. (a) On CUB-200, CORE-ACL outperforms both Dense Narrow (M2M^2) and Upper-Triangular Packed Narrow (12M2\frac{1}{2}M^2) across all three memory envelopes. (b) Rank-calibrated hold-out selection (CORE-ACL*) eliminates Session-1 refit collapse across all truncated rules at 36 MiB. (c) Mean off-diagonal class-prototype correlation across pipeline stages.

DatasetReadout RuleHold-Out Validation + Refit 16 MiBHold-Out Validation + Refit 36 MiBHold-Out Validation + Refit 80 MiBGCV: Packed vs. CORE-ACL 16 MiBGCV: Packed vs. CORE-ACL 36 MiBGCV: Packed vs. CORE-ACL 80 MiB
CIFAR-100Dense Exact Narrow91.3091.6291.8691.1491.4891.85
CIFAR-100Packed Exact Narrow91.5891.8492.0591.5291.8192.06
CIFAR-100Unrestored LR91.6891.6992.2289.8491.7692.14
CIFAR-100Robust FD (RFD)91.8292.2292.3190.4192.2892.30
CIFAR-100CORE-ACL (standard)91.6092.2092.2890.5292.3792.36
CIFAR-100CORE-ACL*91.6492.2192.2990.5292.3792.36
CUB-200Dense Exact Narrow91.2591.4491.6489.3889.9490.66
CUB-200Packed Exact Narrow91.4291.6091.7889.9290.4691.10
CUB-200Unrestored LR90.7886.1291.9691.3291.4591.42
CUB-200Robust FD (RFD)91.1091.8892.1291.3891.3291.36
CUB-200CORE-ACL (standard)90.6888.6891.6291.5791.5291.48
CUB-200CORE-ACL*91.4691.8991.9491.5791.5291.48
VTAB-50Dense Exact Narrow92.8593.8194.4592.9893.6694.15
VTAB-50Packed Exact Narrow93.3494.0894.5893.5094.0194.44
VTAB-50Unrestored LR92.9694.2194.8594.6294.6894.71
VTAB-50Robust FD (RFD)94.6194.5694.8594.6894.7094.72
VTAB-50CORE-ACL (standard)93.5694.4094.8694.7094.7194.73
VTAB-50CORE-ACL*93.6294.4194.8694.7094.7194.73

Table 8. Hold-out validation and refit across B∈{16,36,80}\mathcal{B} \in\{16, 36, 80\} MiB, plus comparison against Upper-Triangular Packed Exact Narrow (12Mp(Mp+1)\frac{1}{2}M_p(M_p + 1)). Values are Aˉ\bar{A} (%) over 3 seeds.

DatasetSparsity Ratio k/Mk/M (s=300s=300) 0.10Sparsity Ratio k/Mk/M (s=300s=300) 0.30Sparsity Ratio k/Mk/M (s=300s=300) 0.50Sparsity Ratio k/Mk/M (s=300s=300) 1.00Row Non-Zeros ss (k/M=0.3k/M=0.3) 50Row Non-Zeros ss (k/M=0.3k/M=0.3) 150Row Non-Zeros ss (k/M=0.3k/M=0.3) 300Row Non-Zeros ss (k/M=0.3k/M=0.3) 60040-Point Fine Grid LR40-Point Fine Grid Isotropic40-Point Fine Grid CORE-ACL
CIFAR-10092.4492.8192.7692.7092.1492.6692.8192.7492.1692.1892.23
CUB-20091.1891.5291.4691.3590.8891.3991.5291.4891.5691.8891.84
VTAB-5094.8295.0795.0494.9694.6594.9895.0795.0592.8893.6493.81

Table 9. Ablation of sparse lifting hyperparameters (TopK ratio k/Mk/M and row degree ss) and 40-point fine-grid regularization control on CIFAR-100 and CUB-200 (q=840q = 840, Aˉ\bar{A} in %).

DatasetDense Solve (ms)CORE-ACL Solve (ms)SpeedupSession Fit (s)Persist. MiBPeak GPU MiB
CIFAR-10073.4±0.873.4 \pm0.84.90±0.124.90 \pm0.1215.0×15.0\times11.435.91,184
CUB-20074.2±1.374.2 \pm1.35.52±0.105.52 \pm0.1013.4×13.4\times2.139.7512
VTAB-5072.1±0.872.1 \pm0.84.68±0.044.68 \pm0.0415.4×15.4\times0.634.0358

Table 10. Computational latency, normal-equation solve speedup (RTX 3090), and peak GPU memory (RTX A5000) at B=36\mathcal{B} = 36 MiB.

References

  1. [1]Da-Wei Zhou, Hai-Long Sun, Jingyi Ning, Han-Jia Ye, and De-Chuan Zhan. Continual learning with pre-trained models: A survey. arXiv preprint arXiv:2401.16386, 2024. 1, 2arxiv.org/abs/2401.16386
  2. [2]Zhiyan Hou, Dan Zhang, Tao Feng, Liyuan Wang, Wei Li, Xiangzhao Hao, Hongyan An, Junfeng Fang, Haokai Ma, Zhaohui Xu, Xinyu Tang, Haiyun Guo, Jingqiao Wang, and Tat-Seng Chua. Continual learning in transition. arXiv preprint arXiv:2608.06216, 2026. 1, 2arxiv.org/abs/2608.06216
  3. [3]Huiping Zhuang, Zhenyu Weng, Hongxin Wei, Renchunzi Xie, Kar-Anh Toh, and Zhiping Lin. ACL: Analytic class-incremental learning with absolute memorization and privacy protection. In Advances in Neural Information Processing Systems (NeurIPS), pages 11602–11615, 2022. 1, 3, 6, 15arxiv.org/abs/2205.14922
  4. [4]Huiping Zhuang, Run He, Kai Tong, Ziqian Zeng, Cen Chen, and Zhiping Lin. DS-AL: A dual-stream analytic learning for exemplar-free class-incremental learning. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 17237–17244, 2024. 3, 6arxiv.org/abs/2403.17503
  5. [5]Huiping Zhuang, Yizhu Chen, Di Fang, Run He, Kai Tong, Hongxin Wei, Ziqian Zeng, and Cen Chen. GACL: Exemplar-free generalized analytic continual learning. In Advances in Neural Information Processing Systems (NeurIPS), 2024. 3, 6arxiv.org/abs/2403.15706
  6. [6]Run He, Di Fang, Yizhu Chen, Kai Tong, Cen Chen, Yi Wang, Lap-Pui Chau, and Huiping Zhuang. REAL: Representation enhanced analytic learning for exemplar-free class-incremental learning. arXiv preprint arXiv:2403.13522, 2024. 1, 3, 6, 15DOI
  7. [7]Mark D. McDonnell, Dong Gong, Amin Parveneh, Ehsan Abbasnejad, and Anton van den Hengel. RanPAC: Random projections and pre-trained models for continual learning. In Advances in Neural Information Processing Systems (NeurIPS), 2023. 1, 2, 3, 5, 6, 13, 15
  8. [8]Ameya Prabhu, Shiven Sinha, Ponnurangam Kumaraguru, Philip H. S. Torr, Ozan Sener, and Puneet K. Dokania. Ran-Dumb: Random representations outperform online continu- ally learned representations. In Advances in Neural Information Processing Systems (NeurIPS), 2024. 3
  9. [9]Quyen Tran, Lam Tran, Khanh Doan, Toan Tran, Dinh Phung, Khoat Than, and Trung Le. Boosting multiple views for pretrained-based continual learning. In International Conference on Learning Representations (ICLR), 2025. 1, 3
  10. [10]Heming Zou, Yunliang Zang, Wutong Xu, and Xiangyang Ji. Fly-CL: A fly-inspired framework for enhancing efficient decorrelation and reduced training time in pre-trained model-based continual representation learning. In International Conference on Learning Representations (ICLR), 2026. 1, 2, 3, 5, 6, 13, 15arxiv.org/abs/2510.16877
  11. [11]Ruilin Li, Heming Zou, Xiufeng Yan, Zheming Liang, Jie Yang, Chenliang Li, and Xue Yang. Enhancing pretrained model-based continual representation learning via guided random projection. In European Conference on Computer Vision (ECCV), pages 246–264, 2026. 1, 3
  12. [12]Mina Ghashami, Edo Liberty, Jeff M. Phillips, and David P. Woodruff. Frequent directions: Simple and deterministic matrix sketching. SIAM Journal on Computing, 45(5):1762–1792, 2016. 1, 3
  13. [13]Luo Luo, Cheng Chen, Zhihua Zhang, Wu-Jun Li, and Tong Zhang. Robust frequent directions with application in online learning. Journal of Machine Learning Research, 20(45):1–41, 2019. 1, 3, 12
  14. [14]Benwei Shi and Jeff M. Phillips. A deterministic streaming sketch for ridge regression. In International Conference on Artificial Intelligence and Statistics (AISTATS), pages 586–594, 2021. 1, 3arxiv.org/abs/2002.02013
  15. [15]Liangzu Peng, Juan Elenter, Joshua Agterberg, Alejandro Ribeiro, and René Vidal. LoRanPAC: Low-rank random features and pre-trained models for bridging theory and practice in continual learning. In International Conference on Learning Representations (ICLR), 2025. 1, 3, 4, 6, 13, 15arxiv.org/abs/2410.00645
  16. [16]Edward Snelson and Zoubin Ghahramani. Sparse gaussian processes using pseudo-inputs. In Advances in Neural Information Processing Systems (NeurIPS), pages 1257–1264, 2005. 2, 3
  17. [17]Gene H. Golub, Michael Heath, and Grace Wahba. Generalized cross-validation as a method for choosing a good ridge parameter. Technometrics, 21(2):215–223, 1979. 2, 4DOI
  18. [18]Alexey Dosovitskiy, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani, Matthias Minderer, Georg Heigold, Sylvain Gelly, Jakob Uszkoreit, and Neil Houlsby. An image is worth 16x16 words: Transformers for image recognition at scale. In International Conference on Learning Representations (ICLR), 2021. 2, 5
  19. [19]Maxime Oquab, Timothée Darcet, Théo Moutakanni, Huy V. Vo, Marc Szafraniec, Vasil Khalidov, Pierre Fernandez, Daniel Haziza, Francisco Massa, Alaeldin El-Nouby, Mahmoud Assran, Nicolas Ballas, Wojciech Galuba, Russell Howes, Po-Yao Huang, Shang-Wen Li, Ishan Misra, Michael Rabbat, Vasu Sharma, Gabriel Synnaeve, Hu Xu, Hervé Jégou, Julien Mairal, Patrick Labatut, Armand Joulin, and Piotr Bojanowski. DINOv2: Learning robust visual features
  20. [20]Zifeng Wang, Zizhao Zhang, Chen-Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, and Tomas Pfister. Learning to prompt for continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 139–149, 2022. 2, 6, 15arxiv.org/abs/2112.08654
  21. [21]Zifeng Wang, Zizhao Zhang, Sayna Ebrahimi, Ruoxi Sun, Han Zhang, Chen-Yu Lee, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, and Tomas Pfister. DualPrompt: Complementary prompting for rehearsal-free continual learning. In European Conference on Computer Vision (ECCV), pages 631–648, 2022. 2, 6arxiv.org/abs/2204.04799
  22. [22]James Seale Smith, Leonid Karlinsky, Vyshnavi Gutta, Paola Cascante-Bonilla, Donghyun Kim, Assaf Arbelle, Rameswar Panda, Rogerio Feris, and Zsolt Kira. CODA-Prompt: Continual decomposed attention-based prompting for rehearsal-free continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 11909–11919, 2023. 2, 6, 15arxiv.org/abs/2211.13218
  23. [23]Da-Wei Zhou, Hai-Long Sun, Han-Jia Ye, and De-Chuan Zhan. Expandable subspace ensemble for pre-trained model-based class-incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 23554–23564, 2024. 2, 6, 15arxiv.org/abs/2403.12030
  24. [24]Yan-Shuo Liang and Wu-Jun Li. InfLoRA: Interference-free low-rank adaptation for continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 23638–23647, 2024. 3, 6arxiv.org/abs/2404.00228
  25. [25]Hai-Long Sun, Da-Wei Zhou, Hanbin Zhao, Le Gan, De-Chuan Zhan, and Han-Jia Ye. MOS: Model surgery for pre-trained model-based class-incremental learning. In Proceedings of the AAAI Conference on Artificial Intelligence, pages 20699–20707, 2025. 3arxiv.org/abs/2412.09441
  26. [26]Huiyi Wang, Haodong Lu, Lina Yao, and Dong Gong. Self-expansion of pre-trained models with mixture of adapters for continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 10087–10098, 2025. 3arxiv.org/abs/2403.18886
  27. [27]Heming Zou, Yunliang Zang, Wutong Xu, Yao Zhu, and Xiangyang Ji. FlyLoRA: Boosting task decoupling and parameter efficiency via implicit rank-wise mixture-of-experts. In Advances in Neural Information Processing Systems (NeurIPS), pages 10386–10419, 2025. 3arxiv.org/abs/2510.08396
  28. [28]Ziqi Wang, Chang Che, Qi Wang, Hui Ma, Zenglin Shi, Cees G. M. Snoek, and Meng Wang. Harmonious parameter adaptation in continual visual instruction tuning for safety-aligned MLLMs. arXiv preprint arXiv:2511.20158, 2025. 3arxiv.org/abs/2511.20158
  29. [29]Kai Jiang, Zisong Lin, Hongyuan Zhang, Xueru Bai, and Xuelong Li. Metrics learning with expandable subspace for pre-trained model-based class-incremental learning. arXiv preprint arXiv:2607.17593, 2026. 3DOI
  30. [30]Hongye Xu and Bartosz Krawczyk. Geometry-anchored transport framework for exemplar-free class-incremental learning. arXiv preprint arXiv:2606.25347, 2026.arxiv.org/abs/2606.25347
  31. [31]Tao Hu, Zhen-Hao Xie, Jingcai Guo, De-Chuan Zhan, and Da-Wei Zhou. Visual branch is what you need for CLIP-based class-incremental learning. arXiv preprint arXiv:2609.37888, 2026. 3arxiv.org/abs/2609.37888
  32. [32]Huiping Zhuang, Yuchen Liu, Run He, Kai Tong, Ziqian Zeng, Cen Chen, Yi Wang, and Lap-Pui Chau. FO-AL: Forward-only online analytic learning with fast training and low memory footprint in class incremental learning. In Advances in Neural Information Processing Systems (NeurIPS), pages 41517–41538, 2024. 3, 6, 15
  33. [33]Yang Xiao, Tianyi Peng, Rohan Kumar Das, Yuchen Hu, and Huiping Zhuang. AnalyticKWS: Towards exemplar-free analytic class incremental learning for small-footprint keyword spotting. In Findings of the Association for Computational Linguistics: ACL 2025, pages 14147–14158, 2025. 3arxiv.org/abs/2505.11817
  34. [34]Huiping Zhuang, Di Fang, Kai Tong, Yuchen Liu, Ziqian Zeng, Xu Zhou, and Cen Chen. Online analytic exemplar-free continual learning with large models for imbalanced autonomous driving task. arXiv preprint arXiv:2405.17779, 2024.DOI
  35. [35]Lei Song, Shihan Guan, and Youyong Kong. Analytic drift resister for non-exemplar continual graph learning. arXiv preprint arXiv:2604.02633, 2026.arxiv.org/abs/2604.02633
  36. [36]Quyen Tran, Hai Nguyen, Quan Dao, Zhuowei Li, Nam Le, Trung Le, and Dimitris Metaxas. Spectral-aware analytic class incremental learning for long-tailed distributions. arXiv preprint arXiv:2607.22931, 2026.arxiv.org/abs/2607.22931
  37. [37]Berk Sezer. An impossibility result for ETF target geometry, and an equinorm remedy for imbalanced analytic continual learning. arXiv preprint arXiv:2610.24844, 2026. 3arxiv.org/abs/2610.24844
  38. [38]Heming Zou, Yunliang Zang, and Xiangyang Ji. Structural features of the fly olfactory circuit mitigate the stability-plasticity dilemma in continual learning. arXiv preprint arXiv:2502.01427, 2025. 3arxiv.org/abs/2502.01427
  39. [39]Saleh Momeni, Changnan Xiao, and Bing Liu. AnaCP: Toward upper-bound continual learning via analytic contrastive projection. In Advances in Neural Information Processing Systems (NeurIPS), 2025. 3, 13arxiv.org/abs/2511.13880
  40. [40]Zhiwen Cao, Yanfeng Li, Shu-Dong Huang, Yalan Ye, Shuyin Xia, Yi Wang, and Jiancheng Lv. On the power of statistics in class-incremental learning with pretrained models. In Proceedings of the 43rd International Conference on Machine Learning (ICML), pages 11255–11271, 2026. 3, 13
  41. [41]Junhui Mao, Junru Zhang, Liming Ye, Cheng Peng, Hao-ran Shi, Xiaoli Tang, Baokuan Wang, Bin Dou, Yabo Dong, and Han Yu. TRACE: Memory-efficient time series class-incremental learning via temporal-spectral reconstruction. arXiv preprint arXiv:2610.35727, 2026. 3arxiv.org/abs/2610.35727
  42. [42]Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical Report, University of Toronto, 2009. 5, 14
  43. [43]Catherine Wah, Steve Branson, Peter Welinder, Pietro Perona, and Serge Belongie. The Caltech-UCSD Birds-200-2011 dataset. Technical Report CNS-TR-2011-001, California Institute of Technology, 2011. 5, 14
  44. [44]Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, Dawn Song, Jacob Steinhardt, and Justin Gilmer. The many faces of robustness: A critical analysis of out-of-distribution generalization. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), pages 8340–8349, 2021. 5, 14
  45. [45]Dan Hendrycks, Kevin Zhao, Steven Basart, Jacob Steinhardt, and Dawn Song. Natural adversarial examples. In Proceedings of the IEEE/CVF International Conference on Computer Vision and Pattern Recognition (CVPR), pages 15262–15271, 2021. 5, 14
  46. [46]Xiaohua Zhai, Joan Puigcerver, Alexander Kolesnikov, Pierre Ruyssen, Carlos Riquelme, Mario Lucic, Josip Djolonga, Andre Susano Pinto, Maxim Neumann, Alexey Dosovitskiy, Lucas Beyer, Olivier Bachem, Michael Tschannen, Marcin Michalski, Olivier Bousquet, Sylvain Gelly, and Neil Houlsby. A large-scale study of representation learning with the visual task adaptation benchmark. arXiv preprint arXiv:1910.04867, 2019. 5, 14arxiv.org/abs/1910.04867
  47. [47]Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 770–778, 2016. 7arxiv.org/abs/1512.03385

Paper details

vlnc:W_w7g4h785
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents