betaFeedback

WorksComputer Science

DuoCL: Dual-Path Online Continual Learning

Abstract

Online continual learning (OCL) requires learning from a data stream whose arrival rate is independent of model training speed. When training cannot keep pace, computationally expensive gradient-based methods update less frequently, limiting adaptation. Analytic methods reduce this overhead, but their frozen backbones cannot adapt representations to changing distributions. We propose DuoCL, a dual-path OCL framework whose fast analytic path accumulates feature statistics from every incoming minibatch while its slow path periodically adapts the backbone. The slow path adjusts its sample count to the available compute budget without interrupting fast-path learning. Feature alignment compensates for drift in the accumulated classifier statistics. Across four datasets and three computational budgets, DuoCL improves online accuracy by an average of 9.67 percentage points over the strongest gradient-based baseline in each setting. DuoCL consistently outperforms analytic learning under distribution shifts, with gains of up to 4.00 percentage points on CIFAR-C, and achieves higher accuracy than prior methods across disjoint class-incremental, domain-incremental, and Si-Blurry settings.

Introduction

Continual learning (CL) [14, 16, 47] aims to enable machine learning models to learn incrementally from non-stationary data streams while mitigating catastrophic forgetting [16, 42, 57] of previously acquired knowledge. In conventional offline CL, models are trained on a sequence of tasks, typically allowing multiple training epochs over each task’s data before proceeding to the next task [31, 55]. In contrast, online continual learning (OCL) processes a continuous stream of data, with model updates performed under limited computational and memory budgets [10].

Existing OCL methods primarily focus on mitigating catastrophic forgetting through experience replay [13, 28], regularization [1, 8, 9], knowledge distillation [34], and sampling strategies [3, 4, 68]. However, these methods have different computational requirements, which can limit how frequently models update when data arrives independently of model training time. A recent benchmark [18] demonstrates that computationally expensive methods may miss learning opportunities when model training cannot keep up with the data arrival rate, with a simple baseline outperforming more sophisticated OCL methods.

Recent work [75] has explored approaches to support rapid adaptation under computational constraints. System-level approaches employ fine-grained pipeline parallelism to process high-frequency data streams, using gradient compensation to mitigate stale gradients. Other methods reduce training computation through selective [59] and adaptive layer freezing [58], restricting gradient-based updates to selected backbone layers. Analytic CL methods [6, 17, 43, 48, 61] [20] offer a different approach by freezing the backbone and updating the classifier in closed form using accumulated feature statistics. However, restricting backbone updates can limit representation plasticity, particularly when pretrained features cannot adequately capture the incoming data distribution. This motivates a central question: Can a model keep pace with the data arrival rate while periodically adapting its backbone representations under limited computational and memory budgets?

We propose DuoCL, a dual-path OCL framework that combines continuous updates to analytic statistics with periodic gradient-based backbone adaptation (Fig. 1). The fast path accumulates feature statistics for every incoming minibatch, while the slow path adapts the backbone using recent samples. Coordinating these paths under limited computational and memory budgets requires determining when slow-path training occurs and how backbone updates are incorporated without interrupting fast-path learning. DuoCL addresses this through a bounded buffer of recent samples and a periodic slow-path update schedule. However, backbone adaptation changes the feature representations underlying previously accumulated classifier statistics, introducing representation drift. To mitigate this drift, DuoCL employs feature alignment to maintain consistency between the updated backbone and the analytic classifier. Our main contributions are as follows:

Figure 1. Comparison of OCL approaches under a computational budget. (a) Frozen analytic methods update a closed-form classifier from accumulated feature statistics GG and CC for every incoming minibatch at low cost, but the fixed backbone limits adaptation. (b) Gradient-based OCL methods adapt the backbone, but must skip minibatches for training when their computational demand exceeds the budget (ρ>1\rho> 1). (c) DuoCL accumulates feature statistics for every incoming minibatch through the fast path, while the slow path periodically adapts the backbone using samples retained from the previous window and aligns GG and CC with the updated features.

  • We propose DuoCL, a dual-path OCL framework that combines continuous analytic statistics updates with periodic gradient-based backbone adaptation.

  • We develop a budget-aware slow path that supports periodic backbone adaptation under computational and memory constraints using bounded buffers, with feature alignment to mitigate representation drift between the updated backbone and analytic classifier statistics.

  • Across four datasets and three computational budgets, DuoCL improves online accuracy by an average of 9.67 percentage points over the strongest gradient-based baseline in each setting.

  • DuoCL consistently outperforms analytic learning under distribution shifts, with gains of up to 4.00 percentage points on CIFAR-C, and achieves higher accuracy than prior methods across disjoint class-incremental, domain-incremental, and Si-Blurry settings.

Compute-constrained online continual learning. OCL methods mitigate forgetting through replay, regularization, distillation, and sample selection [2, 4, 9, 13, 34], or adapt prompts and low-rank adapters of pretrained models online [46, 66] (Fig. 1b). These methods typically assume that training on each minibatch completes before the next minibatch arrives. [18] evaluates OCL methods without this assumption and shows that a method kk times more expensive than experience replay can train on only one in every kk minibatches. Under such computational constraints, simple replay often outperforms more sophisticated methods [15, 18, 51, 71]. Budget-aware methods reduce the cost of each gradient update through layer freezing [58, 59] or throughput-oriented replay [64], while system-level approaches schedule retraining [7, 29, 30] or distribute backpropagation across additional pipeline workers [75]. These approaches still learn from incoming minibatches through backpropagation, so minibatches are either skipped or processed with additional computational resources. In contrast, DuoCL learns from every incoming minibatch by accumulating feature statistics using features already computed during inference, incurring little additional computation, and restricts backpropagation to a slow path whose budget is determined by the data arrival rate and device throughput (§ 4.1).

Analytic continual learning. Analytic CL methods replace iterative classifier training with closed-form ridge-regression or discriminant solutions computed from accumulated feature statistics [22, 43, 45, 48, 76]. These methods are exemplar-free and, for a fixed feature extractor, equivalent to joint training on all observed data [76]. Recursive formulations update the classifier online for every minibatch [53, 77], and the same statistics have been used in federated CL [6, 17, 61] [20]. Such methods perform strongly under limited online budgets [52, 53], but they rely on a frozen pretrained backbone, since updating it would invalidate the accumulated statistics. They therefore cannot adapt their representations when the incoming data deviate from the pretraining distribution. DuoCL addresses the limitation by adapting the backbone throughout the stream while aligning the accumulated statistics with backbone updates.

Combining analytic and gradient-based learning. Fast and slow learners motivated by complementary learning systems theory [41] typically train both components with backpropagation, differing in learning rate or objective [5, 49, 70]. Other methods combine a training-free or analytic classifier with gradient-based tuning [72, 74], while drift compensation methods correct stored class statistics after backbone updates [19, 39, 56, 69], including through @page 2/12 starts_mid=0 ends_mid=1

Figure 1 (PDF p. 2). Comparison of OCL approaches under a computational budget. (a) Frozen analytic methods update a closed-form classifier from accumulated feature statistics GG and CC for every incoming minibatch at low cost, but the fixed backbone limits adaptation. (b) Gradient-based OCL methods adapt the backbone, but must skip minibatches for training when their computational demand exceeds the budget (ρ>1\rho> 1). (c) DuoCL accumulates feature statistics for every incoming minibatch through the fast path, while the slow path periodically adapts the backbone using samples retained from the previous window and aligns GG and CC with the updated features.

  • We propose DuoCL, a dual-path OCL framework that combines continuous analytic statistics updates with periodic gradient-based backbone adaptation.

  • We develop a budget-aware slow path that supports periodic backbone adaptation under computational and memory constraints using bounded buffers, with feature alignment to mitigate representation drift between the updated backbone and analytic classifier statistics.

  • Across four datasets and three computational budgets, DuoCL improves online accuracy by an average of 9.67 percentage points over the strongest gradient-based baseline in each setting.

  • DuoCL consistently outperforms analytic learning under distribution shifts, with gains of up to 4.00 percentage points on CIFAR-C, and achieves higher accuracy than prior methods across disjoint class-incremental, domain-incremental, and Si-Blurry settings.

Compute-constrained online continual learning. OCL methods mitigate forgetting through replay, regularization, distillation, and sample selection [2, 4, 9, 13, 34], or adapt prompts and low-rank adapters of pretrained models online [46, 66] (Fig. 1b). These methods typically assume that training on each minibatch completes before the next minibatch arrives. [18] evaluates OCL methods without this assumption and shows that a method kk times more expensive than experience replay can train on only one in every kk minibatches. Under such computational constraints, simple replay often outperforms more sophisticated methods [15, 18, 51, 71]. Budget-aware methods reduce the cost of each gradient update through layer freezing [58, 59] or throughput-oriented replay [64], while system-level approaches schedule retraining [7, 29, 30] or distribute backpropagation across additional pipeline workers [75]. These approaches still learn from incoming minibatches through backpropagation, so minibatches are either skipped or processed with additional computational resources. In contrast, DuoCL learns from every incoming minibatch by accumulating feature statistics using features already computed during inference, incurring little additional computation, and restricts backpropagation to a slow path whose budget is determined by the data arrival rate and device throughput (§ 4.1).

Analytic continual learning. Analytic CL methods replace iterative classifier training with closed-form ridge-regression or discriminant solutions computed from accumulated feature statistics [22, 43, 45, 48, 76]. These methods are exemplar-free and, for a fixed feature extractor, equivalent to joint training on all observed data [76]. Recursive formulations update the classifier online for every minibatch [53, 77], and the same statistics have been used in federated CL [6, 17, 61] [20]. Such methods perform strongly under limited online budgets [52, 53], but they rely on a frozen pretrained backbone, since updating it would invalidate the accumulated statistics. They therefore cannot adapt their representations when the incoming data deviate from the pretraining distribution. DuoCL addresses the limitation by adapting the backbone throughout the stream while aligning the accumulated statistics with backbone updates.

Combining analytic and gradient-based learning. Fast and slow learners motivated by complementary learning systems theory [41] typically train both components with backpropagation, differing in learning rate or objective [5, 49, 70]. Other methods combine a training-free or analytic classifier with gradient-based tuning [72, 74], while drift compensation methods correct stored class statistics after backbone updates [19, 39, 56, 69], including through a closed-form least-squares transformation of second-order statistics [24, 54]. These methods are designed for task-based CL, where the backbone is trained for multiple epochs on each task and the statistics are corrected once per task using the full task data. SIESTA [21] updates its output layer online and consolidates the full network in periodic offline sleep phases, which assume scheduled downtime with no incoming data. To our knowledge, DuoCL is the first to combine analytic and gradient-based learning concurrently within a single-pass data stream. The fast path accumulates feature statistics for every incoming minibatch, the slow path adapts the backbone within the computational budget in every window, and a closed-form transformation estimated from a small set of retained samples aligns the accumulated classifier statistics with the backbone after each update, without requiring task boundaries, multiple training epochs, or interruptions to the stream.

Method

Problem Definition

Online continual learning. Following [10, 18], we consider a data stream arriving at rr samples per second and grouped into minibatches of bb samples. Each minibatch arrival defines a time step t∈{1,…,T}t \in\{1, \ldots, T\}, with consecutive arrivals separated by b/rb/r seconds. At step tt, the model predicts Y^t=f(Xt;θt)\hat{Y}_t = f(X_t; \theta_t) for the incoming minibatch XtX_t, drawn from a distribution πt\pi_t that may change over time. The ground-truth labels YtY_t are then revealed to evaluate the predictions, after which the model may learn from (Xt,Yt)(X_t, Y_t) subject to a fixed computational budget and a memory budget of at most MM stored samples. Online performance is evaluated on every arrival, including minibatches skipped for training. Figure 2 illustrates the stream and an example training schedule.

Figure 2. Problem Definition: Minibatches of bb samples arrive every b/rb/r seconds. With ρ=3\rho= 3, training on XtX_t spans three arrival intervals, so Xt+1X_{t+1} and Xt+2X_{t+2} are skipped for training. Prediction continues on every arrival.

Computational budget. Minibatches arrive independently of training progress, so the computation that training can use between consecutive arrivals depends on both the stream rate and the device throughput. For a device providing PP FLOP/s for training, this computational budget per time step is

Favail=Pbr.(1)F_{\mathrm{avail}} = \frac{Pb}{r}. \tag*{(1)}

We specify FavailF_{\mathrm{avail}} directly to express different stream rates and hardware capabilities through a common computational budget. Let FreqF_{\mathrm{req}} denote the training FLOPs a method requires per time step to learn from every incoming minibatch. We characterize its demand relative to this budget by

ρ=FreqFavail.(2)\rho= \frac{F_{\mathrm{req}}}{F_{\mathrm{avail}}}. \tag*{(2)}

Update frequency. When ρ≤1\rho\le1, a method’s training procedure can keep pace with the stream. When ρ>1\rho> 1, queueing every incoming minibatch for training would create a growing backlog that eventually exceeds the fixed memory budget. Maintaining this training procedure therefore requires skipping some minibatches. The average number of minibatches skipped per update is

S=max⁡(0,ρ−1).(3)S = \max(0, \rho- 1). \tag*{(3)}

Predictions continue on every arrival using the latest available model, with parameter updates taking effect only after their computation completes.

DuoCL Overview

As illustrated in Figure 3, DuoCL consists of two coordinated learning paths: a fast path for analytic classifier learning and a slow path for periodic backbone adaptation. For each incoming minibatch, the backbone ϕθ\phi_\theta extracts features, which the fast path incorporates into accumulated classifier statistics. The analytic classifier WW is periodically reconstructed from these statistics. The slow path is triggered every KK incoming minibatches, with each set of KK minibatches forming a window. During each window, the slow path adapts the backbone using samples retained from the previous window, while a new subset of samples is collected for the next update. Since backbone weight updates change the feature representations, DuoCL aligns the accumulated classifier statistics and updates the classifier.

Figure 3. Overview of DuoCL over two consecutive windows of KK minibatches. The fast path predicts every minibatch and updates GG and CC after label revelation. Reservoir sampling collects up to mm current-window samples in Qcurr\mathcal{Q}_{\mathrm{curr}}, while the slow path adapts the backbone using previous-window samples in Qprev\mathcal{Q}_{\mathrm{prev}}. At the next window boundary, feature alignment updates the accumulated statistics and the classifier WW is rebuilt.

Fast Path: Online Analytic Learning

In OCL, predictions must be made for every incoming minibatch, even when the computational and memory budgets prevent training on all incoming samples. Since online prediction already requires extracting backbone features, DuoCL’s fast path reuses these features and the subsequently revealed labels to accumulate feature statistics from every minibatch.

To achieve this, we adopt the ridge-regression formulation used in analytic continual learning [43, 45, 48]. We maintain two accumulated statistics: a Gram matrix GG that captures second-order feature information, and a matrix CC that accumulates class-wise feature information. For each incoming minibatch XtX_t containing bb samples, let Ft=ϕθ(Xt)∈Rb×dF_t = \phi_\theta(X_t) \in\mathbb{R}^{b \times d} denote the extracted features, where dd is the feature dimension. Predictions are made using the latest available classifier WW: Y^t=arg max⁡j(FtW):,j\hat{Y}_t = \operatorname{arg\,max}_j (F_tW)_{:,j}. After observing the ground-truth labels, let Yt∈Rb×cY_t \in\mathbb{R}^{b \times c} denote the one-hot encoding of the labels, where cc is the number of classes. We update the accumulated statistics as G←G+FtTFtG \leftarrow G + F_t^{\mathsf{T}}F_t and C←C+FtTYtC \leftarrow C + F_t^{\mathsf{T}}Y_t, where G∈Rd×dG \in\mathbb{R}^{d \times d} and C∈Rd×cC \in\mathbb{R}^{d \times c} are initialized to zero. Each column of CC accumulates features belonging to a particular class, while GG captures second-order feature statistics used to construct the classifier. The resulting ridge-regression classifier W∈Rd×cW \in\mathbb{R}^{d \times c} has the closed-form solution

W=(G+λI)−1C,(4)W = (G + \lambda I)^{-1}C, \tag*{(4)}

where λ>0\lambda> 0 is the regularization coefficient. While GG and CC can be updated incrementally using the extracted features, reconstructing WW requires solving a d×dd \times d linear system, with O(d3)O(d^3) computational complexity for a conventional dense solve. Performing this operation after every arrival can be expensive under high data arrival rates. To reduce this cost, DuoCL updates GG and CC for every incoming minibatch but reconstructs WW once every KK minibatches. This amortizes the classifier reconstruction cost while allowing every incoming minibatch to contribute to the accumulated statistics. Unlike conventional frozen-backbone analytic learning, DuoCL also periodically adapts the backbone, requiring the accumulated statistics to be aligned with the updated representations, as in § 4.5.

Slow Path: Periodic Backbone Adaptation

Although the fast path incorporates every incoming minibatch into the analytic classifier statistics, it does not adapt the backbone. Performing backpropagation on every incoming minibatch can exceed the available computational budget. To address this, DuoCL periodically adapts the backbone using a bounded subset of recent samples.

Budget-Aware Sample Selection and Scheduling. The slow path is triggered every KK incoming minibatches, with each group of KK minibatches forming a window. During each window, Qcurr\mathcal{Q}_{\mathrm{curr}} collects incoming samples, while the slow path adapts the backbone using samples retained in Qprev\mathcal{Q}_{\mathrm{prev}} from the previous window. We first determine the number of samples mm retained in each buffer based on the available computational and memory budgets. With bb samples per minibatch, each window contains KbKb samples and provides a total computational budget of KFavailKF_{\mathrm{avail}}. The slow path performs ss gradient steps using mm retained samples. Let FslowF_{\mathrm{slow}} denote the profiled FLOPs required for one minibatch-equivalent of slow-path training. The retained sample count is constrained by

smbFslow≤KFavail,(5)\frac{sm}{b}F_{\mathrm{slow}} \le KF_{\mathrm{avail}}, \tag*{(5)}

where KFavailKF_{\mathrm{avail}} denotes the available computation budget over a window. Since both buffers can coexist, their combined storage must also satisfy 2m≤M2m \le M. Accordingly, we determine the maximum retained sample count as

m=min⁡(Kb,⌊KbFavailsFslow⌋,⌊M2⌋).(6)m = \min\left(Kb,\left\lfloor\frac{KbF_{\mathrm{avail}}}{sF_{\mathrm{slow}}}\right\rfloor,\left\lfloor\frac{M}{2}\right\rfloor\right). \tag*{(6)}

Since each window contains KbKb samples, which may exceed the buffer size mm, we employ uniform reservoir sampling to select a subset for backbone adaptation. The first mm samples are stored in QcurrQ_{\mathrm{curr}}. For each subsequent sample n>mn > m, we retain it with probability m/nm/n, replacing a uniformly selected sample in the buffer. This ensures that every sample in the window has an equal probability of being retained while the buffer size remains bounded by mm. Meanwhile, the slow path performs ss gradient steps using samples from QprevQ_{\mathrm{prev}}. At the end of the window, the backbone update is applied, the previous samples are discarded, and the buffers are updated as Qprev←QcurrQ_{\mathrm{prev}} \leftarrow Q_{\mathrm{curr}} and Qcurr←∅Q_{\mathrm{curr}} \leftarrow\varnothing. Thus, DuoCL performs backbone adaptation using recently retained samples without maintaining a persistent historical replay buffer.

Slow-Path Training Objective. Let (XQ,YQ)(X_Q, Y_Q) denote the retained samples and their corresponding labels in QprevQ_{\mathrm{prev}}. For backbone adaptation, we employ three complementary loss terms:

Lslow=Lmain+αLaux+βLdrift,(7)\mathcal{L}_{\mathrm{slow}} = \mathcal{L}_{\mathrm{main}} + \alpha\mathcal{L}_{\mathrm{aux}} + \beta\mathcal{L}_{\mathrm{drift}}, \tag*{(7)}

where α\alpha and β\beta control the contributions of the auxiliary and drift losses, respectively. The main cross-entropy loss adapts the backbone features using the existing analytic classifier:

Lmain=CE⁡(FW^,YQ),(8)\mathcal{L}_{\mathrm{main}} = \operatorname{CE}(F\widehat{W},Y_Q), \tag*{(8)}

where F=ϕθ(XQ)F = \phi_\theta(X_Q) denotes the features of the retained samples, YQY_Q denotes their labels, and W^\widehat{W} is a fixed copy of the analytic classifier. During slow-path training, gradients propagate through the classifier to update the backbone, while W^\widehat{W} remains unchanged.

To provide an additional supervised signal for backbone adaptation, we employ a training-only auxiliary classifier gψg_\psi, with loss

Laux=CE⁡(gψ(F),YQ).(9)\mathcal{L}_{\mathrm{aux}} = \operatorname{CE}(g_\psi(F),Y_Q). \tag*{(9)}

However, backbone updates can also change predictions for previously observed classes. To limit these changes, we employ drift regularization that penalizes deviations from the logits computed before backbone adaptation:

Ldrift=∥(FW^):,O−(FpreW^):,O∥F2,(10)\mathcal{L}_{\mathrm{drift}} = \left\|(F\widehat{W})_{:,O} - (F_{\mathrm{pre}}\widehat{W})_{:,O}\right\|_F^2, \tag*{(10)}

where FpreF_{\mathrm{pre}} denotes the features before adaptation and OO denotes previously observed classes absent from the retained samples. To maintain a consistent loss scale across updates, we normalize the drift penalty by the number of retained samples and previously observed classes when applying its weight β\beta. The drift loss is zero when OO is empty. For CNN-based backbones, the slow path directly updates the backbone parameters. For Transformer-based backbones, we employ low-rank adaptation (LoRA) [27], keeping the pre-trained backbone weights frozen while updating the LoRA adapters in the query and value projections. Both configurations use the same slow-path training objective.

Drift-Aware Feature Alignment

Slow-path backbone adaptation changes the feature representations used to accumulate the classifier statistics GG and CC. Consequently, directly reusing these statistics with the updated backbone can introduce inconsistencies between the classifier and the new feature space. Recomputing the statistics from all previously observed samples is impractical under the computational and memory constraints of OCL. To address this, inspired by prior work on feature space drift compensation [24, 67], DuoCL estimates a feature transformation using the retained samples and aligns the accumulated statistics with the updated backbone.

For the retained samples XQX_Q, we extract features before and after backbone adaptation: Fpre=ϕθold(XQ)F_{\mathrm{pre}} = \phi_{\theta_{\mathrm{old}}}(X_Q) and Fpost=ϕθnew(XQ)F_{\mathrm{post}} = \phi_{\theta_{\mathrm{new}}}(X_Q), where θold\theta_{\mathrm{old}} and θnew\theta_{\mathrm{new}} denote the backbone parameters before and after adaptation, respectively. We estimate an old-to-new feature transformation TT using regularized least squares:

T∗=arg⁡min⁡T ∥FpreT−Fpost∥F2+γ∥T−I∥F2,(11)T^* = \underset{T}{\arg\min}\ \left\|F_{\mathrm{pre}}T - F_{\mathrm{post}}\right\|_F^2 + \gamma\left\|T-I\right\|_F^2, \tag*{(11)}

where γ>0\gamma> 0 is the regularization coefficient that encourages the transformation to remain close to the identity matrix. This prevents excessive changes to the feature space when estimating the transformation from a limited number of retained samples. The optimization admits the closed-form solution

T∗=(FpreTFpre+γI)−1(FpreTFpost+γI).(12)T^* = \left(F_{\mathrm{pre}}^{\mathsf{T}}F_{\mathrm{pre}}+\gamma I\right)^{-1}\left(F_{\mathrm{pre}}^{\mathsf{T}}F_{\mathrm{post}}+\gamma I\right). \tag*{(12)}

Rather than recomputing the classifier statistics from previously observed samples, we transform the accumulated statistics into the updated feature space: G←T∗TGT∗G \leftarrow T^{* \mathsf{T}}GT^* and C←T∗TCC \leftarrow T^{* \mathsf{T}}C. The transformation is applied to the accumulated statistics, including minibatches processed while the slow-path update was pending. Finally, the analytic classifier WW is reconstructed from the aligned statistics using Eq. (4). This allows DuoCL to retain information accumulated from the incoming stream while periodically adapting its backbone representations.

Experiment

Experimental Setup

Dataset and Baseline

Our evaluation covers ten widely used image classification benchmarks. CIFAR-10 and CIFAR-100 [33] are standard image classification benchmarks. CLEAR-10 and CLEAR-100 [37] capture the natural temporal evolution of visual concepts, while CLOC [10] provides a large-scale stream with naturally occurring distribution shifts. CIFAR-C [25] evaluates robustness under common image corruptions. ImageNet-R [26] and ImageNet-S [63] evaluate visual distribution shifts through artistic renditions and sketches, respectively. CUB-200 [62] provides a fine-grained classification benchmark covering 200 bird species. CORe50 [38] evaluates continual object recognition across sessions with varying viewpoints and environments.

We compare DuoCL against representative baselines: ER [13], ACE [9], LwF [34], RWalk [11], PoLRS [10], MIR [2], GSS [4], and aL-SAR [58]. For pretrained ViT-based comparisons, we compare against AGEM [12], ER [13], MIR [2], GDumb [50], DER++ [8], PCR [36], LODE (with DER++ [8]) [35], EMA (with DER++ [8] and with RAR [73]) [60], EWC++ [11], L2P [65], MVP [46], Online-LoRA [66], and FGH [44]. Additional implementation details, including dataset splits and hyperparameter configurations, are provided in the supplementary material.

Metrics

We evaluate DuoCL using three metrics: (i) average online accuracy AonlineA_{\mathrm{online}} [18], the percentage of incoming samples correctly classified before learning their labels, including samples skipped for training; (ii) final accuracy AFinalA_{\mathrm{Final}} [40, 66], computed after training on the final task; and (iii) area under the curve of accuracy AAUCA_{\mathrm{AUC}} [32], which measures model performance throughout training. We report AonlineA_{\mathrm{online}} for the online setting, AFinalA_{\mathrm{Final}} for disjoint class-incremental and domain-incremental settings, and both AFinalA_{\mathrm{Final}} and AAUCA_{\mathrm{AUC}} for Si-Blurry. See Appendix for detailed definitions.

Results

Impact of Data Arrival Rate

To evaluate performance under varying data arrival rates, we vary the computational budget available for training per incoming minibatch, FavailF_{\mathrm{avail}}, as defined in § 4.1. At fixed training throughput PP and minibatch size bb, a lower FavailF_{\mathrm{avail}} corresponds to a higher data arrival rate rr. We consider three computational budgets: High, Medium, and Low, corresponding to 3×3\times, 2×2\times, and 1×1\times the FLOPs required for one backbone forward pass, respectively, as shown in Table 2. We profile the training FLOPs of each baseline and report the resulting compute ratios ρ\rho and skipped mini-

BudgetCIFAR-10/100CLEAR-10/100
High (3× FP)2.223,138.92
Medium (2× FP)1.482,092.61
Low (1× FP)0.741,046.31

Table 2. Available computational budget FavailF_{\mathrm{avail}} (GFLOPs per arrival). FP denotes one backbone forward pass on an incoming minibatch.

batches SS in Table 3. aL-SAR [58] explicitly accounts for computational cost through adaptive layer freezing. The memory budgets used in these experiments are provided in the appendix. For all methods in Table 1, we use an ImageNet-1K-pretrained ResNet-18 [23] with a batch size of 10 for CIFAR-10 and CIFAR-100, and an ImageNet-1K-pretrained ResNet-50 [23] with a batch size of 128 for CLEAR-10 and CLEAR-100. Here, UB represents an unconstrained variant of DuoCL that retains all samples KbKb in each window and performs slow-path adaptation without computational restrictions. Table 1 shows the average online accuracy of all methods under three computational budgets. Under the Low budget, existing OCL methods skip multiple minibatches during training, limiting their learning opportunities. In contrast, DuoCL continuously updates its accumulated feature statistics GG and CC, while periodically adapting the backbone using buffered samples. As a result, DuoCL outperforms the strongest existing OCL baseline by 8.53, 17.04, 12.06, and 21.26 percentage points on CIFAR-10, CIFAR-100, CLEAR-10, and CLEAR-100, respectively, under the Low budget, with an average improvement of 9.67 percentage points across all datasets and budgets. Furthermore, DuoCL achieves accuracy within 0.79 percentage points of UB across all settings, suggesting that continuous fast-path learning combined with reservoir-based slow-path adaptation can approach unconstrained performance under limited computational and memory budgets.

MethodFavailF_{\mathrm{avail}}CIFAR-10CIFAR-100CLEAR-10CLEAR-100
ER [13]High64.3528.5685.2573.06
Medium61.1825.0480.6666.89
Low54.0419.4369.0952.19
ACE [9]High60.9924.0387.0476.63
Medium57.7021.8583.1872.36
Low51.9817.5474.9962.66
LwF [34]High63.179.9082.4869.58
Medium58.258.1578.3962.24
Low51.316.4365.7144.71
RWalk [11]High57.639.1678.0160.56
Medium54.118.1969.0551.90
Low46.835.8751.6730.55
PoLRS [10]High45.4717.4862.7248.68
Medium44.5714.4659.6045.71
Low41.8111.1118.4721.23
MIR [2]High49.258.1872.2950.53
Medium44.907.2261.0437.84
Low39.024.8539.0517.36
GSS [4]High40.269.6751.4330.60
Medium34.317.9331.6618.82
Low28.186.0916.074.76
aL-SAR [58]High61.6223.9684.6965.58
Medium60.1722.4883.7363.33
Low55.7318.9280.5758.07
DuoCL (Ours)High64.8036.6492.7083.96
Medium64.6836.6092.6783.94
Low64.2636.4792.6383.92
UB65.0536.8692.7084.00
ImprovementHigh+0.45+8.08+5.66+7.34
Medium+3.51+11.56+8.94+11.59
Low+8.53+17.04+12.06+21.26

Table 1. Average online accuracy (%, ↑) under three computational budgets, averaged over three seeds. Improvement denotes the absolute percentage-point gain of DuoCL over the strongest existing OCL baseline for each dataset and budget. Best and second-best results, excluding the unconstrained UB, are shown in bold and <u>underlined</u>, respectively.

MethodCompute Ratio (ρ\rho) HighCompute Ratio (ρ\rho) MediumCompute Ratio (ρ\rho) LowSkipped Minibatches (SS) HighSkipped Minibatches (SS) MediumSkipped Minibatches (SS) Low
ER [13]236125
ACE [9]236125
LwF [34]2.67481.6737
RWalk [11]46123511
PoLRS [10]69185817
MIR [2]57.501546.5014
GSS [4]121836111735
DuoCL (Fast Path)<0.01< 0.01<0.01< 0.010.01000

Table 3. Compute ratio (ρ\rho) and skipped minibatches per update (SS) under three computational budgets.

Impact of Distribution Shift

To demonstrate the importance of dual-path learning under distribution shifts commonly encountered in OCL, we compare DuoCL with Analytic, a variant that uses only the fast path without backbone adaptation. We first evaluate both methods using ResNet-18 on a CIFAR-C stream consisting

of clean images followed by Gaussian noise and contrast corruptions at severity level 4. On CIFAR-C, DuoCL improves test accuracy over Analytic by 3.61 and 4.00 percentage points under Gaussian noise and contrast corruption, respectively (Figure 4). On CLOC, using ResNet-50, DuoCL consistently achieves higher monthly online accuracy (Figure 5), supporting the benefit of backbone adaptation under naturally evolving distributions.

Figure 4. Accuracy under distribution shifts on CIFAR-C.

Figure 5. Comparison of online accuracy between DuoCL and Analytic on the CLOC dataset under naturally occurring temporal distribution shifts.

Performance under Different OCL Settings

We evaluate DuoCL under disjoint class-incremental learning (CIL), where new classes are introduced sequentially across tasks and do not reappear in subsequent tasks. We consider CIFAR-100, ImageNet-R, ImageNet-S, and CUB-200 with the CIL setup using ViT-B/16 and ViT-S/16 backbones. We set the memory budget to M=500M = 500 samples and the available computational budget FavailF_{\mathrm{avail}} to the FLOPs corresponding to one training pass per incoming minibatch, consistent with prior work [66]. The first three datasets are divided into 10 tasks, while CUB-200 uses 5 tasks. UB represents the offline upper bound obtained through supervised fine-tuning on the complete dataset with i.i.d. samples. DuoCL achieves the highest final accuracy among OCL methods in seven of eight comparisons, improving upon Analytic by up to 3.08 percentage points (Table 5). Under domain-incremental learning (DIL), the input distribution changes while the class set remains fixed. On CORe50, DuoCL exceeds Online-LoRA [66] by 1.81 and 1.58 percentage points with ViT-B/16 and ViT-S/16, respectively (Table 6).

BackboneMethodCIFAR-100ImageNet-RImageNet-SCUB-200
ViT-B/16AGEM [12]12.67 ± 1.875.60 ± 2.740.16 ± 0.0410.84 ± 1.57
ER [13]44.85 ± 1.8340.99 ± 3.9630.21 ± 0.7031.66 ± 0.83
EWC++ [11]10.61 ± 0.743.86 ± 2.020.32 ± 0.2826.14 ± 3.46
MIR [2]48.36 ± 3.1141.51 ± 2.9930.33 ± 3.8131.64 ± 2.97
GDumb [50]41.00 ± 19.978.87 ± 1.361.65 ± 0.229.09 ± 1.03
PCR [36]48.48 ± 0.1546.11 ± 3.0338.75 ± 0.2241.11 ± 1.43
DER++ [8]36.64 ± 6.1130.90 ± 8.046.47 ± 0.0626.61 ± 1.27
LODE (DER++) [35]44.29 ± 1.4842.20 ± 6.469.97 ± 2.2939.20 ± 4.25
EMA (DER++) [60]42.28 ± 4.3641.75 ± 1.9816.88 ± 2.2335.26 ± 3.31
EMA (RAR) [60]47.10 ± 0.8230.04 ± 0.3314.06 ± 0.3733.34 ± 1.11
Online-LoRA [66]49.40 ± 1.3648.18 ± 0.6347.06 ± 0.2441.46 ± 0.31
FGH [44]76.94 ± 0.8964.85 ± 0.7155.71 ± 2.9675.22 ± 1.55
Analytic85.55 ± 0.1868.24 ± 0.1259.12 ± 0.3082.86 ± 0.56
DuoCL (Ours)87.78 ± 0.1671.31 ± 0.1457.98 ± 0.4982.91 ± 0.54
UB89.50 ± 0.0476.78 ± 0.4463.82 ± 0.0282.81 ± 1.07
ViT-S/16AGEM [12]7.43 ± 2.152.35 ± 0.872.75 ± 2.861.40 ± 0.17
ER [13]31.91 ± 2.0632.73 ± 0.2019.53 ± 1.4421.81 ± 3.02
EWC++ [11]6.80 ± 2.131.32 ± 0.834.08 ± 3.242.07 ± 0.54
MIR [2]29.08 ± 1.1434.73 ± 0.2913.96 ± 1.9522.95 ± 1.12
GDumb [50]10.87 ± 4.945.33 ± 1.092.90 ± 0.323.28 ± 0.99
PCR [36]32.89 ± 1.4721.96 ± 0.2714.37 ± 0.9522.28 ± 2.73
DER++ [8]17.67 ± 4.0422.17 ± 4.2718.15 ± 0.6629.53 ± 2.37
LODE (DER++) [35]28.65 ± 3.0631.65 ± 0.7217.59 ± 0.8426.81 ± 0.89
EMA (DER++) [60]12.76 ± 0.6520.89 ± 0.3512.93 ± 0.1335.79 ± 5.27
EMA (RAR) [60]19.21 ± 2.1616.11 ± 0.3514.50 ± 2.7134.53 ± 1.04
Online-LoRA [66]32.16 ± 0.2433.21 ± 0.5022.45 ± 0.4337.41 ± 1.16
FGH [44]74.01 ± 1.2252.39 ± 4.3748.09 ± 2.2876.46 ± 0.65
Analytic80.08 ± 0.0654.70 ± 0.0746.89 ± 0.2183.22 ± 0.67
DuoCL (Ours)82.62 ± 0.1157.78 ± 0.0149.93 ± 0.1883.82 ± 0.56
UB86.55 ± 0.0169.94 ± 0.3459.28 ± 0.1173.91 ± 1.16

Table 5. Final accuracy (AFinalA_{\mathrm{Final}}, %, ↑\uparrow) under disjoint CIL. Best and second-best non-UB results are shown in bold and _underlined_, respectively.

MethodViT-B/16ViT-S/16
AGEM [12]80.15 ± 2.9778.22 ± 3.51
ER [13]85.85 ± 1.3578.99 ± 3.85
EWC++ [11]78.65 ± 6.5179.03 ± 4.54
MIR [2]74.35 ± 4.0786.49 ± 0.81
GDumb [50]77.20 ± 3.4975.64 ± 2.92
PCR [36]87.16 ± 0.7375.20 ± 1.48
DER++ [8]81.88 ± 7.0689.33 ± 0.62
LODE (DER++) [35]77.02 ± 2.2283.48 ± 5.84
L2P [65]87.97 ± 0.3786.47 ± 0.23
MVP [46]84.82 ± 0.5479.85 ± 0.33
Online-LoRA [66]93.71 ± 0.0190.96 ± 0.02
Analytic93.57 ± 0.0089.19 ± 0.00
DuoCL (Ours)95.52 ± 0.0892.54 ± 0.14
UB95.60 ± 0.0193.56 ± 0.01

Table 6. Final accuracy (AFinalA_{\mathrm{Final}}, %, ↑\uparrow) under domain-incremental learning on CORE50 [38]. Best and second-best non-UB results are shown in bold and _underlined_, respectively.

We further evaluate DuoCL in the stochastic incremental-blurry (Si-Blurry) setting [46], where class distributions change stochastically, and classes overlap across tasks. We consider CIFAR-100, ImageNet-R, and ImageNet-S, each divided into five tasks, using the same computational and memory budgets. DuoCL outperforms

Analytic across all 12 AAUCA_{\mathrm{AUC}} and AFinalA_{\mathrm{Final}} comparisons (Table 4). On CIFAR-100, final accuracy improves by 4.93 and 5.69 percentage points with ViT-B/16 and ViT-S/16, respectively.

BackboneMethodCIFAR-100 AAUCA_{\mathrm{AUC}}CIFAR-100 AFinalA_{\mathrm{Final}}ImageNet-R AAUCA_{\mathrm{AUC}}ImageNet-R AFinalA_{\mathrm{Final}}ImageNet-S AAUCA_{\mathrm{AUC}}ImageNet-S AFinalA_{\mathrm{Final}}
ViT-B/16L2P [65]43.01 ± 9.3739.86 ± 2.2822.71 ± 1.8627.08 ± 2.4910.02 ± 0.4213.58 ± 4.04
MVP [46]47.52 ± 9.7444.49 ± 0.9327.79 ± 0.6231.64 ± 1.7710.68 ± 0.4513.99 ± 1.73
Online-LoRA [66]60.12 ± 5.7961.70 ± 6.2945.05 ± 1.5948.00 ± 6.0130.81 ± 2.0930.22 ± 4.36
Analytic69.86 ± 6.0381.70 ± 0.0059.28 ± 0.3158.82 ± 0.0747.20 ± 2.5044.85 ± 0.24
DuoCL (Ours)74.74 ± 5.2986.63 ± 0.1660.95 ± 0.2961.65 ± 0.3948.53 ± 2.7046.15 ± 0.71
Offline UB89.50 ± 0.0476.78 ± 0.4463.82 ± 0.02
ViT-S/16L2P [65]37.82 ± 12.1930.88 ± 1.3924.31 ± 1.8321.83 ± 2.132.00 ± 0.123.61 ± 1.08
MVP [46]40.31 ± 9.5235.55 ± 2.1127.04 ± 1.0926.67 ± 3.702.27 ± 0.143.72 ± 0.77
Online-LoRA [66]52.84 ± 7.9758.72 ± 1.4439.47 ± 1.9336.61 ± 4.6315.35 ± 0.9220.18 ± 1.84
Analytic64.70 ± 7.5177.93 ± 0.0044.85 ± 0.6843.56 ± 0.2433.35 ± 2.1732.06 ± 0.31
DuoCL (Ours)68.75 ± 7.2483.62 ± 0.3547.69 ± 0.6248.90 ± 0.7936.49 ± 2.4035.88 ± 0.78
UB86.55 ± 0.0169.94 ± 0.3459.28 ± 0.11

Table 4. Performance comparison under Si-Blurry CIL (%, ↑\uparrow). Each dataset reports area under the accuracy curve (AAUCA_{\mathrm{AUC}}) and final accuracy (AFinalA_{\mathrm{Final}}). Best and second-best non-UB results are shown in bold and _underlined_, respectively.

Ablation on Individual Components

Figure 6 presents the component ablation on CIFAR-100 under Si-Blurry using ViT-B/16. The fast-path-only baseline achieves 81.70% final accuracy, increasing to 85.29% with slow-path adaptation using the main cross-entropy loss, 85.95% with drift regularization, and 86.63% with the auxiliary loss, demonstrating the contribution of each component.

Component ablation bar chart on Si-Blurry CIFAR-100

Figure 6. Component ablation on Si-Blurry CIFAR-100.

Conclusion

We presented DuoCL, a dual-path OCL framework that combines continuous analytic learning with periodic backbone adaptation under computational and memory constraints. DuoCL improves average online accuracy by 9.67 percentage points over existing baselines and demonstrates the benefits of backbone adaptation under distribution shifts and diverse OCL settings.

LLM Usage Statement

OpenAI’s ChatGPT and Anthropic’s Claude were used to assist with editing throughout the manuscript, including grammar and clarity, as well as figure refinement, code implementation, and managing experiments. The authors reviewed and verified the final content and take full responsibility for the submitted work.

References

  1. [1]Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuytelaars. Memory aware synapses: Learning what (not) to forget. In Computer Vision – ECCV 2018, pages 144–161, Cham, 2018. Springer International Publishing. 1arxiv.org/abs/1711.09601
  2. [2]Rahaf Aljundi, Eugene Belilovsky, Tinne Tuytelaars, Laurent Charlin, Massimo Caccia, Min Lin, and Lucas Page-Caccia. Online continual learning with maximal interfered retrieval. In Advances in Neural Information Processing Systems. Curran Associates, Inc., 2019. 2, 6, 7, 8
  3. [3]Rahaf Aljundi, Eugene Belilovsky, Tinne Tuytelaars, Laurent Charlin, Massimo Caccia, Min Lin, and Lucas Page-Caccia. Online continual learning with minimal interfered retrieval. In Advances in Neural Information Processing Systems. Curran Associates, Inc., 2019. 1
  4. [4]Rahaf Aljundi, Min Lin, Baptiste Goujaud, and Yoshua Bengio. Gradient based sample selection for online continual learning. In Advances in Neural Information Processing Systems. Curran Associates, Inc., 2019. 1, 2, 6, 7
  5. [5]Elnaz Arani, Fahad Sarfraz, and Bahram Zonooz. Learning fast, learning slow: A general continual learning method based on complementary learning system. In International Conference on Learning Representations, 2022. 2arxiv.org/abs/2201.12604
  6. [6]Jebacyril Ariockiaraj, Dhruv Parikh, Jayashree Adivarahan, Rajgopal Kannan, and Viktor Prasanna. Accurate and resource-efficient federated continual learning, 2026. 1, 2
  7. [7]Romil Bhardwaj, Zhengxu Xia, Ganesh Ananthanarayanan, Junchen Jiang, Yuanchao Shu, Nikolaos Kariakaris, Kevin Hsieh, Paramvir Bahl, and Ion Stoica. Ekya: Continuous learning of video analytics models on edge compute servers. In 19th USENIX Symposium on Networked Systems Design and Implementation (NSDI 22), pages 119–135, Renton, WA, 2022. USENIX Association. 2
  8. [8]Pietro Buzze­ga, Matteo Boschini, Angelo Porrello, Davide Abati, and SIMONE CALDERARA. Dark experience for general continual learning: a strong, simple baseline. In
  9. [9]Lucas Caccia, Rahaf Aljundi, Nader Asadi, Tinne Tuytelaars, Joelle Pineau, and Eugene Belilovsky. New insights on reducing abrupt representation change in online continual learning. In International Conference on Learning Representations, 2022. 1, 2, 6, 7
  10. [10]Zhipeng Cai, Ozan Sener, and Vladlen Koltun. Online Continual Learning with Natural Distribution Shifts: An Empirical Study with Visual Data . In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 8261–8270, Los Alamitos, CA, USA, 2021. IEEE Computer Society. 1, 3, 6, 7
  11. [11]Arslan Chaudhry, Puneet K. Dokania, Thalaiyasingam Ajanthan, and Philip H. S. Torr. Riemannian walk for incremental learning: Understanding forgetting and intransigence. In Computer Vision – ECCV 2018, pages 556–572, Cham, 2018. Springer International Publishing. 6, 7, 8DOI
  12. [12]Arslan Chaudhry, Marc’Aurelio Ranzato, Marcus Rohrbach, and Mohamed Elhoseiny. Efficient lifelong learning with a-GEM. In International Conference on Learning Representations, 2019. 6, 8arxiv.org/abs/1812.00420
  13. [13]Arslan Chaudhry, Marcus Rohrbach, Mohamed Elhoseiny, Thalaiyasingam Ajanthan, P. Dokania, P. Torr, and M. Ranzato. Continual learning with tiny episodic memories. In Workshop on multi-task and lifelong reinforcement learning, 2019. 1, 2, 6, 7, 8
  14. [14]Zhiyuan Chen and Bing Liu. Lifelong machine learning, second edition. Synthesis Lectures on Artificial Intelligence and Machine Learning, 12:1–207, 2018. 1DOI
  15. [15]Botos Csaba, Wenxuan Zhang, Matthias Müller, Ser-Nam Lim, Mohamed Elhoseiny, Philip Torr, and Adel Bibi. Label delay in online continual learning. In Advances in Neural Information Processing Systems, pages 119976–120012. Curran Associates, Inc., 2024. 2arxiv.org/abs/2312.00923
  16. [16]Matthias De Lange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Aleš Leonardis, Gregory Slabaugh, and Tinne Tuytelaars. A continual learning survey: Defying forgetting in classification tasks. IEEE transactions on pattern analysis and machine intelligence, 44(7):3366–3385, 2021. 1DOI
  17. [17]Eros Fanì, Raffaello Camoriano, Barbara Caputo, and Marco Cicone. Accelerating heterogeneous federated learning with closed-form classifiers. In Proceedings of the 41st International Conference on Machine Learning, pages 13029–13048. PMLR, 2024. 1, 2
  18. [18]Yasir Ghunaim, Adel Bibi, Kumail Alhamoud, Motasem Alfarra, Hasan Abed Al Kader Hammoud, Ameya Prabhu, Philip H.S. Torr, and Bernard Ghanem. Real-time evaluation in online continual learning: A new hope. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 11888–11897, 2023. 1, 2, 3, 6
  19. [19]Dipam Goswami, Albin Soutif-Cormerais, Yuyang Liu, Sandesh Kamath, Bartlomiej Twardowski, and Joost van de Weijer. Resurrecting old classes with new data for exemplar-free continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 28525–28534, 2024. 2 [20] Zenghao Guan, Guojun Zhu, Yucan Zhou, Wu Liu, Weiping Wang, Jiebo Luo, and Xiaoyan Gu. Enhancing federated class-incremental learning via spatial-temporal statistics aggregation. In Proceedings of the ACM Web Conference 2026, pages 5356–5367, 2026. 1, 2
  20. [21]Md Yousuf Harun, Jhair Gallardo, Tyler L. Hayes, Ronald Kemker, and Christopher Kanan. SIESTA: Efficient online continual learning with sleep. Transactions on Machine Learning Research, 2023. 3arxiv.org/abs/2303.10725
  21. [22]Tyler L. Hayes and Christopher Kanan. Lifelong machine learning with deep streaming linear discriminant analysis. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 887–896, 2020. 2arxiv.org/abs/1909.01520
  22. [23]Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition, pages 770–778, 2016. 6arxiv.org/abs/1512.03385
  23. [24]Run He, Di Fang, Yicheng Xu, Yawen Cui, Ming Li, Cen Chen, Ziqian Zeng, and Huiping Zhuang. Semantic shift estimation via dual-projection and classifier reconstruction for exemplar-free class-incremental learning. In Forty-second International Conference on Machine Learning, 2025. 3, 5arxiv.org/abs/2503.05423
  24. [25]Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. arXiv preprint arXiv:1903.12261, 2019. 6
  25. [26]Dan Hendrycks, Steven Basart, Norman Mu, Saurav Kadavath, Frank Wang, Evan Dorundo, Rahul Desai, Tyler Zhu, Samyak Parajuli, Mike Guo, Dawn Song, Jacob Steinhardt, and Justin Gilmer. The many faces of robustness: A critical analysis of out-of-distribution generalization. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 8320–8329, 2021. 6
  26. [27]Edward J Hu, yelong shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022. 5arxiv.org/abs/2106.09685
  27. [28]David Isele and Akansel Cosgun. Selective experience replay for lifelong learning. Proceedings of the AAAI Conference on Artificial Intelligence, 32(1), 2018. 1arxiv.org/abs/1802.10269
  28. [29]Mehrdad Khani, Pouya Hamadanian, Arash Nasr-Esfahany, and Mohammad Alizadeh. Real-time video inference on edge devices via adaptive model streaming. In 2021 IEEE/CVF International Conference on Computer Vision (ICCV), pages 4552–4562, 2021. 2arxiv.org/abs/2006.06628
  29. [30]Mehrdad Khani, Ganesh Ananthanarayanan, Kevin Hsieh, Junchen Jiang, Ravi Netravali, Yuanchao Shu, Mohammad Alizadeh, and Victor Bahl. RECL: Responsive Resource-Efficient continuous learning for video analytics. In 20th USENIX Symposium on Networked Systems Design and Implementation (NSDI 23), pages 917–932, Boston, MA, 2023. USENIX Association. 2
  30. [31]James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell. Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences, 114(13):3521–3526, 2017. 1DOI
  31. [32]Hyunseo Koh, Dahyun Kim, Jung-Woo Ha, and Jonghyun Choi. Online continual learning on class incremental blurry task configuration with anytime inference. In ICLR, 2022. 6arxiv.org/abs/2110.10031
  32. [33]Alex Krizhevsky. Learning multiple layers of features from tiny images. 2009. 6
  33. [34]Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE transactions on pattern analysis and machine intelligence, 40(12):2935–2947, 2017. 1, 2, 6, 7
  34. [35]Yan-Shuo Liang and Wu-Jun Li. Loss decoupling for task-agnostic continual learning. In Advances in Neural Information Processing Systems, pages 11151–11167. Curran Associates, Inc., 2023. 6, 8
  35. [36]Huiwei Lin, Baoquan Zhang, Shanshan Feng, Xutao Li, and Yunming Ye. Pcr: Proxy-based contrastive replay for online class-incremental continual learning. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 24246–24255, 2023. 6, 8arxiv.org/abs/2304.04408
  36. [37]Zhiqiu Lin, Jia Shi, Deepak Pathak, and Deva Ramanan. The clear benchmark: Continual learning on real-world imagery. In Thirty-fifth Conference on Neural Information Processing Systems Datasets and Benchmarks Track, 2021. 6arxiv.org/abs/2201.06289
  37. [38]Vincenzo Lomonaco and Davide Maltoni. Core50: a new dataset and benchmark for continuous object recognition. In Proceedings of the 1st Annual Conference on Robot Learning, pages 17–26. PMLR, 2017. 6, 8arxiv.org/abs/1705.03550
  38. [39]Simone Magistri, Tomaso Trinci, Albin Soutif-Cormerais, Joost van de Weijer, and Andrew Bagdanov. Elastic feature consolidation for cold start exemplar-free incremental learning. In International Conference on Learning Representations, pages 31909–31931, 2024.
  39. [40]Zheda Mai, Ruiwen Li, Hyunwoo Kim, and Scott Sanner. Supervised Contrastive Replay: Revisiting the Nearest Class Mean Classifier in Online Class-Incremental Continual Learning . In 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pages 3584–3594, Los Alamitos, CA, USA, 2021. IEEE Computer Society. 6arxiv.org/abs/2103.13885
  40. [41]James L McClelland, Bruce L McNaughton, and Randall C O’Reilly. Why there are complementary learning systems in the hippocampus and neocortex: insights from the successes and failures of connectionist models of learning and memory. Psychological review, 102(3):419, 1995. 2
  41. [42]Michael McCloskey and Neal J. Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. pages 109–165. Academic Press, 1989. 1DOI
  42. [43]Mark D McDonnell, Dong Gong, Amin Parveneh, Ehsan Abbasnejad, and Anton Van den Hengel. Ranpac: Random projections and pre-trained models for continual learning. Advances in Neural Information Processing Systems, 36:12022–12053, 2023. 1, 2, 3
  43. [44]Nicolas Michel, Maorong Wang, Jiangpeng He, and Toshihiko Yamasaki. From offline to online memory-free and task-free continual learning via fine-grained hypergradients. Transactions on Machine Learning Research, 2026. 6, 8arxiv.org/abs/2502.18762
  44. [45]Saleh Momeni, Changnan Xiao, and Bing Liu. Anacp: Toward upper-bound continual learning via analytic contrastive projection. In Advances in Neural Information Processing Systems, pages 72923–72944. Curran Associates, Inc., 2025. 2, 3arxiv.org/abs/2511.13880
  45. [46]Jun-Yeong Moon, Keon-Hee Park, Jung Uk Kim, and Gyeong-Moon Park. Online class incremental learning on stochastic blurry task boundary via mask and visual prompt tuning. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pages 11697–11707, 2023. 2, 6, 7, 8arxiv.org/abs/2308.09303
  46. [47]German I. Parisi, Ronald Kemker, Jose L. Part, Christopher Kanan, and Stefan Wermter. Continual lifelong learning with neural networks: A review. Neural Networks, 113:54–71, 2019. 1DOI
  47. [48]Liangzu Peng, Juan Elenter, Joshua Agterberg, Alejandro Ribeiro, and Rene Vidal. Lornapc: Low-rank random features and pre-trained models for bridging theory and practice in continual learning. In The Thirteenth International Conference on Learning Representations, 2025. 1, 2, 3arxiv.org/abs/2410.00645
  48. [49]Quang Pham, Chenghao Liu, and Steven Hoi. Dualnet: Continual learning, fast and slow. In Advances in Neural Information Processing Systems, pages 16131–16144. Curran Associates, Inc., 2021. 2arxiv.org/abs/2110.00175
  49. [50]Ameya Prabhu, Philip H. S. Torr, and Puneet K. Dokania. Gdumb: A simple approach that questions our progress in continual learning. In Computer Vision – ECCV 2020, pages 524–540, Cham, 2020. Springer International Publishing. 6, 8DOI
  50. [51]Ameya Prabhu, Hasan Abed Al Kader Hammoud, Puneet K. Dokania, Philip H.S. Torr, Ser-Nam Lim, Bernard Ghanem, and Adel Bibi. Computationally budgeted continual learning: What does matter? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3698–3707, 2023. 2arxiv.org/abs/2303.11165
  51. [52]Ameya Prabhu, Zhipeng Cai, Puneet Dokania, Philip Torr, Vladlen Koltun, and Ozan Sener. Online continual learning without the storage constraint, 2023. 2arxiv.org/abs/2305.09253
  52. [53]Ameya Prabhu, Shiven Sinha, Ponnurangam Kumaraguru, Philip Torr, Ozan Sener, and Puneet K. Dokania. Random representations outperform online continually learned representations. In Advances in Neural Information Processing Systems, pages 37988–38006. Curran Associates, Inc., 2024. 2arxiv.org/abs/2402.08823
  53. [54]Xuan Rao, Simian Xu, Zheng Li, Bo Zhao, Derong Liu, Mingming Ha, and Cesare Alippi. Compensating distribution drifts in continual learning with pre-trained vision transformers. Proceedings of the AAAI Conference on Artificial Intelligence, 40(30):25090–25098, 2026. 3DOI
  54. [55]Sylvestre-Alvise Rebuffi, Alexander Kolesnikov, Georg Sperl, and Christoph H. Lampert. icarl: Incremental classifier and representation learning. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pages 5533–5542, 2017. 1arxiv.org/abs/1611.07725
  55. [56]Grzegorz Rypeś, Sebastian Cygert, Tomasz Trzcinski, and Bartł omiej Twardowski. Task-recency bias strikes back: Adapting covariances in exemplar-free class incremental learning. In Advances in Neural Information Processing Systems, pages 63628–63289. Curran Associates, Inc., 2024. 2
  56. [57]Jonathan Schwarz, Wojciech Czarnecki, Elena Luketina, Agnieszka Grabska-Barwinska, Yee Whye Teh, Razvan Pascanu, and Raia Hadsell. Progress & compress: A scalable framework for continual learning. In International conference on machine learning, pages 4528–4537. PMLR, 2018. 1arxiv.org/abs/1805.06370
  57. [58]Minhyuk Seo, Hyunseo Koh, and Jonghyun Choi. Budgeted online continual learning by adaptive layer freezing and frequency-based sampling. In International Conference on Learning Representations, pages 47499–47533, 2025. 1, 2, 6arxiv.org/abs/2410.15143
  58. [59]Amelia Sorrenti, Giovanni Bellitto, Federica Proietto Salan-itri, Matteo Pennisi, Concetto Spampinato, and Simone Palazzo. Selective freezing for efficient continual learning. In 2023 IEEE/CVF International Conference on Computer Vision Workshops (ICCVW), pages 3542–3551, 2023. 1, 2DOI
  59. [60]Albin Soutif-Cormerais, Antonio Carta, and Joost van de Weijer. Improving online continual learning performance and stability with temporal ensembles. In Proceedings of The 2nd Conference on Lifelong Learning Agents, pages 828–845. PMLR, 2023. 6, 8
  60. [61]Jianheng Tang, Jingyu He, Kejia Fan, Run He, Jingchao Wang, Anfeng Liu, Houbing Herbert Song, Leye Wang, Zhanxing Zhu, Huiping Zhuang, and Yunhuai Liu. Acfl: Achieving spatio-temporal invariance to data heterogeneity in federated continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Findings, pages 7768–7778, 2026. 1, 2DOI
  61. [62]C. Wah, S. Branson, P. Welinder, P. Perona, and S. Belongie. Caltech-ucsd birds-200-2011. Technical Report CNS-TR-2011-001, California Institute of Technology, 2011. 6
  62. [63]Haohan Wang, Songwei Ge, Zachary Lipton, and Eric P. Xing. Learning robust global representations by penalizing local predictive power. In Advances in Neural Information Processing Systems, pages 10506–10518, 2019. 6
  63. [64]Xinrui Wang, Chuanxing Geng, Wenhai Wan, Shao-yuan Li, and Songcan Chen. Forgetting, ignorance or myopia: Revisiting key challenges in online continual learning. In Advances in Neural Information Processing Systems, pages 58341–58375. Curran Associates, Inc., 2024. 2arxiv.org/abs/2409.19245
  64. [65]Zifeng Wang, Zizhao Zhang, Chen-Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, and Thomas Pfister. Learning to prompt for continual learning. In 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 139–149, 2022. 6, 8arxiv.org/abs/2112.08654
  65. [66]Xiwen Wei, Guihong Li, and Radu Marculescu. Online-lora: Task-free online continual learning via low rank adaptation. In 2025 IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 6634–6645, 2025. 2, 6, 7, 8arxiv.org/abs/2411.05663
  66. [67]Hongye Xu and Bartosz Krawczyk. Two-way is better than one: Bidirectional alignment with cycle consistency for exemplar-free class-incremental learning. In The Fourteenth International Conference on Learning Representations, 2026. 5arxiv.org/abs/2606.05675
  67. [68]Jaehong Yoon, Divyam Madaan, Eunho Yang, and Sung Ju Hwang. Online coreset selection for rehearsal-based continual learning. In International Conference on Learning Representations, 2022. 1arxiv.org/abs/2106.01085
  68. [69]Lu Yu, Bartlomiej Twardowski, Xiailei Liu, Luis Herranz, Kai Wang, Yongmei Cheng, Shangling Jui, and Joost van de Weijer. Semantic drift compensation for class-incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 2020. 2arxiv.org/abs/2004.00440
  69. [70]Gengwei Zhang, Liyuan Wang, Guoliang Kang, Ling Chen, and Yunchao Wei. Slca: Slow learner with classifier alignment for continual learning on a pre-trained model. In 2023 IEEE/CVF International Conference on Computer Vision (ICCV), pages 19091–19101, 2023. 2arxiv.org/abs/2303.05118
  70. [71]Wenxuan Zhang, Youssef Mohamed, Bernard Ghanem, Philip Torr, Adel Bibi, and Mohamed Elhoseiny. Continual learning on a diet: Learning from sparsely labeled streams under constrained computation. In International Conference on Learning Representations, pages 31516–31532, 2024. 2arxiv.org/abs/2404.12766
  71. [72]Xueyi Zhang, Chengwei Zhang, Zheng Li, Xiyu Wang, Siqi Cai, Mingrui Lao, Yanming Guo, and Huiping Zhuang. Rep deep & machine learning: Exemplar-free continual video action recognition via slow-fast collaborative learning. Proceedings of the AAAI Conference on Artificial Intelligence, 40(42):36075–36083, 2026. 2DOI
  72. [73]Yaqian Zhang, Bernhard Pfahringer, Eibe Frank, Albert Bifet, Nick Jin Sean Lim, and Yunzhe Jia. A simple but strong baseline for online continual learning: Repeated augmented rehearsal. In Advances in Neural Information Processing Systems, pages 14771–14783. Curran Associates, Inc., 2022. 6
  73. [74]Linglan Zhao, Xuerui Zhang, Ke Yan, Shouhong Ding, and Weiran Huang. Safe: Slow and fast parameter-efficient tuning for continual learning with pre-trained models. In Advances in Neural Information Processing Systems, pages 113772–113796. Curran Associates, Inc., 2024. 2arxiv.org/abs/2411.02175
  74. [75]Yuhao Zhou, Yuxin Tian, Jindi Lv, Mingjia Shi, Yuanxi Li, Qing Ye, Shuhao Zhang, and Jiancheng Lv. Ferret: An efficient online continual learning framework under varying memory constraints. In 2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 4850–4861, 2025. 1, 2arxiv.org/abs/2503.12053
  75. [76]HUIPING ZHUANG, Zhenyu Weng, Hongxin Wei, RENCHUNZI XIE, Kar-Ann Toh, and Zhiping Lin. Acil: Analytic class-incremental learning with absolute memorization and privacy protection. In Advances in Neural Information Processing Systems, pages 11602–11614. Curran Associates, Inc., 2022. 2arxiv.org/abs/2205.14922
  76. [77]Huiping Zhuang, Yuchen Liu, Run He, Kai Tong, Ziqian Zeng, Cen Chen, Yi Wang, and Lap-Pui Chau. Foal: Forward-only online analytic learning with fast training and low memory footprint in class incremental learning. In Advances in Neural Information Processing Systems, pages 41517–41538. Curran Associates, Inc., 2024. 2arxiv.org/abs/2403.15751

Paper details

vlnc:W_cvz52439
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 10

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

OpenAI’s ChatGPT, Anthropic’s Claude

View statement
Looking for related papers…
Contents