betaFeedback

WorksComputer Science

From Joint Risk to Decision Invariance: Decision-Preserving Regularization for Exemplar-Free Class-Incremental Learning

Abstract

Exemplar-free continual learning seeks to minimize joint risk over all seen tasks, yet historical data are unavailable for direct optimization. We observe that, for classification, preserving the model's decisions on previously learned classes helps retain old knowledge. This motivates Decision-Preserving Regularization (DPR). Penalizing changes in all pairwise old-class logit margins reduces algebraically to a centered relative-logit penalty. We further apply spectral whitening to remove the classifier's spectral scale bias. Because the global classifier supplies the decision constraints for the next stage, it must remain calibrated across all seen classes despite the drift of stored old-class statistics during representation updates. We therefore fit a closed-form affine map from paired current-task features to transport old-class Gaussian means and covariances into the updated feature space, then calibrate the global classifier using these distributions and current-task features. Across CIFAR-100, Tiny-ImageNet, ImageNet-Subset, and ImageNet-1K under cold-start protocols, comparisons with state-of-the-art methods show that DPR achieves higher accuracy in most settings while remaining highly efficient. Code is available at https://github.com/bnufw/dpr.

Introduction

Exemplar-free class-incremental learning (EFCIL) trains a single model on a sequence of disjoint class sets without retaining historical images, so that inference distinguishes all seen classes without a task identifier [2, 16]. The ideal goal of continual learning is to minimize the joint empirical risk over all observed tasks. Because historical data are absent, this joint objective cannot be optimized directly. Under the cold-start protocol, a small initial task provides an incomplete representation and the feature extractor must adapt substantially in subsequent stages to separate new classes [14]. Updating the backbone on current data alone shifts old-class

features across established decision boundaries and causes catastrophic forgetting [5, 17].

Existing EFCIL methods approximate old-task retention through surrogate feature regularizers and post-hoc drift compensators, yet both designs remain indirect proxies for the old model’s classification decisions. Isotropic feature distillation penalizes all feature displacements equally and restricts new-class plasticity. Elastic Feature Consolidation (EFC) [14] estimates an Empirical Feature Matrix over training samples to constrain sensitive directions, while prior methods partition feature or gradient updates [9, 20, 25]. However, data-dependent sensitivity matrices require sample-level curvature estimation and dense matrix storage without directly enforcing decision invariance. Meanwhile, as the backbone evolves on new classes, stored old-class means and covariances drift out of alignment with the updated feature space. Prior methods compensate for this shift by training auxiliary mapping networks across dozens of epochs [6, 22] or optimizing multi-step adversarial perturbations through the backbone [8]. Can we instead reformulate the intractable joint-learning objective directly into a tractable classification target? For classification, retaining old knowledge does not require reproducing historical risk or freezing the feature vector numerically. It requires only that the model’s old-class classification decisions remain invariant under the frozen old classifier head. Translating this principle into a continuous objective requires preserving the relative ordering of old-class logits, which we formulate by keeping all pairwise logit margins between old classes invariant. By algebraic identity, penalizing all pairwise margin drifts over KK old classes collapses into a single quadratic norm on the row-centered old classifier head WcW_c, allowing arbitrary shared logit offsets while preserving both arg max predictions and conditional softmax probabilities. However, directly penalizing centered logit drift weights each discriminative direction by the squared spectral scale of WcW_c. We introduce Decision-Preserving Regularization (DPR), which applies spectral whitening to the centered classifier head to decouple spectral scale bias and extract an orthonormal discriminative basis V∈Rd×rV \in\mathbb{R}^{d \times r}, constraining all rr active decision directions on an equal footing while applying a mild smoothness penalty to the complementary space for new-class learning.

Because DPR extracts its discriminative constraints directly from the old classifier head, sustaining decision preservation across stages relies on a well-calibrated global classifier over all seen classes; yet representation updates on new classes train only a local task head and cause stored old-class distributions to drift. Rather than training iterative drift networks as in LDC [6] and BiCyc [22] or backpropagating adversarial perturbations as in ADC [8], we extract paired features from the frozen old and updated backbones in a single forward pass over current data and solve ridge regression with an identity prior in closed form for a shared affine map. This affine transformation transports both stored old-class means and covariances into the updated representation space in milliseconds. We then sample synthetic old-class features from the transported Gaussian distributions alongside real current-task features to calibrate a single bias-free linear classifier head directly in feature space. Unlike Mahalanobis classifiers such as AdaGauss [19] and BiCyc [22], our calibrated linear head requires no extra projection layers and allows all stored class statistics to be discarded after incremental training.

Because the derived objective reduces to a single low-rank matrix product per sample during training and a closed-form linear solve afterward, DPR is simple, efficient, and effective (Figure 1). Across CIFAR-100 [12], Tiny-ImageNet [21], ImageNet-Subset [3, 4], and large-scale ImageNet-1K [3] under ten- and twenty-task cold-start protocols, DPR outperforms prior EFCIL baselines such as BiCyc and DPCR. On ImageNet-Subset, DPR improves final accuracy over BiCyc by 6.24 and 6.60 pp on the two task splits, and it scales to ImageNet-1K with gains of up to 3.83 pp over BiCyc. Component-wise profiling on Tiny-ImageNet confirms that spectral whitening, decision-preserving regularization, closed-form affine transport, and classifier calibration together add only 1.39% to per-task training time, completing ten-task ImageNet-Subset training in 5.04 hours (1.87× faster than BiCyc). Formulating continual classification directly from decision invariance thus yields a simple and practical alternative to empirical sensitivity matrices and auxiliary drift networks.

Figure 1. Final accuracy on ImageNet-Subset. Bubble area indicates full-run wall-clock time for 10 tasks; smaller is faster. (§ 4.6).

In summary, our contributions are threefold:

  • We reformulate the joint-learning objective as new-class learning with old-class decision preservation and derive a tractable relative-logit regularizer.

  • We propose DPR, combining spectral whitening for decision preservation with closed-form affine compensation for global classifier calibration.

  • Experiments on four cold-start benchmarks demonstrate improved accuracy and a better balance between old-class retention and new-class learning, with low computational overhead.

Related Work

Continual learning and directional constraints. Continual learning methods preserve previously acquired knowledge through parameter regularization, knowledge distillation, replay, or architectural expansion [2, 16]. Parameter-level regularizers such as EWC [11], SI [24], and MAS [1] estimate weight importance and penalize deviations from old parameters, while distillation methods constrain output logits or intermediate activations [4, 10, 13]. To balance stability and plasticity more selectively, directional and gradient-projection methods partition the representation or gradient updates [20, 25]. Rather than estimating empirical covariance or gradient directions from data, our approach reformulates old-task retention as classification-decision invariance and derives the active discriminative directions algebraically from pairwise logit-margin preservation under the frozen linear classifier head.

Exemplar-free representation regularization. When historical exemplars cannot be stored, representation learning must rely only on current-task data and compact model states. PASS [26] and SSRE [27] combine prototype augmentation with self-supervised or structural constraints, often starting from a large initial task. Under the more demanding cold-start setting, EFC [14] and EFC++ [15] compute an Empirical Feature Matrix over training samples to regularize sensitive feature directions, and DPCR [9] couples dual-projection constraints with classifier reconstruction. Whereas EFC requires O(Bd2)\mathcal{O}(Bd^2) per-batch operations and stores a dense d×dd \times d sensitivity matrix, DPR applies spectral whitening to the row-centered old classifier head to constrain the rr active decision directions in O(Bdr)\mathcal{O}(Bdr) time.

Prototype drift compensation. Summarizing old classes via feature statistics enables decision boundary maintenance without raw images. FeTrIL [18] translates pseudo-features under a frozen backbone, and FeCAM [7] exploits per-class covariance matrices for Mahalanobis classification. When the backbone adapts across tasks, stored prototypes and covariances suffer from representation drift. SDC [23] estimates semantic drift via kernel-weighted displacements, ADC [8] synthesizes adversarial samples through the backbone to estimate drift, LDC [6] trains learnable projection networks across epochs, AdaGauss [19] adapts low-dimensional covariances to mitigate task-recency bias, and BiCyc [22] optimizes bidirectional alignment networks with cycle consistency. In contrast to iterative drift networks and Mahalanobis classifiers that retain auxiliary projection layers at inference, we solve a single ridge-regularized affine system in closed form to transport both old-class means and covariances.

Method

We revisit the continual learning objective for classification, using the idealized joint-learning objective to motivate decision-preserving feature learning (§ 3.1). We then derive a tractable centered relative-logit penalty from pairwise decision margins (§ 3.2) and apply spectral whitening to remove spectral scale bias from the feature training loss (§ 3.3). Finally, because DPR relies on a well-calibrated global classifier at each stage to anchor subsequent decision constraints, we compensate for the drift of stored old-class statistics via a closed-form affine transformation and calibrate the global classifier (§ 3.4). Figure 2 provides an overview of DPR.

Figure 2. Overview of DPR. (a) Decision-preserving regularization preserves pairwise old-class margins while allowing a shared logit shift. Spectral whitening of the centered old classifier yields a basis VV for uniformly penalizing feature drift along old-class discriminative directions during new-class learning. (b) A closed-form affine map fitted to paired current-task features compensates stored old-class means and covariances for representation drift. Gaussian old-class features sampled from the compensated statistics are combined with real current-task features to calibrate the global head WTW_T for unified inference and next-stage decision regularization.

From Joint-Learning Upper Bound to Decision Preservation

Consider a sequence of NN incremental tasks {Dk}k=1N\{\mathcal{D}_k\}_{k=1}^{N} with mutually disjoint class sets Ck\mathcal{C}_k. At incremental stage tt, the learner accesses only the current dataset Dt\mathcal{D}_t without retaining any historical exemplars. Let Cold=⋃k=1t−1Ck\mathcal{C}_{\mathrm{old}}=\bigcup_{k=1}^{t-1}\mathcal{C}_k denote the set of K=∣Cold∣K=|\mathcal{C}_{\mathrm{old}}| previously seen classes, and C1:t=Cold∪Ct\mathcal{C}_{1:t}=\mathcal{C}_{\mathrm{old}}\cup\mathcal{C}_t denote all classes observed up to stage tt. We decouple the network into a feature extractor fθt:X→Rdf_{\theta_t}:\mathcal{X}\to\mathbb{R}^{d} and bias-free linear classification heads, where Wt∈R∣Ct∣×dW_t\in\mathbb{R}^{|\mathcal{C}_t|\times d} denotes the task-specific classifier head for task tt, and WT∈R∣C1:t∣×dW_T\in\mathbb{R}^{|\mathcal{C}_{1:t}|\times d} denotes the global classifier head covering tasks 1 through tt.

The general goal of continual learning is to approximate the performance upper bound of joint training over all observed tasks [2, 13, 16]:

(θt∗,WT∗)=arg⁡min⁡θt,WT∑k=1tE(x,y)∼Dk[Lce(WTfθt(x),y)](1)(\theta_t^*, W_T^*) = \underset{\theta_t,W_T}{\arg\min}\sum_{k=1}^{t}\mathbb{E}_{(x,y)\sim\mathcal{D}_k}\left[\mathcal{L}_{\mathrm{ce}}\left(W_Tf_{\theta_t}(x),y\right)\right] \tag*{(1)}

However, because historical data {Dk}k=1t−1\{\mathcal{D}_k\}_{k=1}^{t-1} are unavailable, directly optimizing Eq. (1) is intractable.

We notice that for classification problems, a sufficient condition of retaining old knowledge is to preserve the model’s classification decisions on old classes. Let fθt−1f_{\theta_{t-1}} be the frozen feature extractor from stage t−1t-1 and WT−1=[w1,…,wK]⊤W_{T-1}=[\mathbf{w}_1,\ldots,\mathbf{w}_K]^\top be the corresponding frozen global classifier head over Cold\mathcal{C}_{\mathrm{old}}. To approach the joint-learning goal in Eq. (1), we formulate an idealized feature-learning objective at stage tt: training the current-task head WtW_t on Dt\mathcal{D}_t while preserving decisions among old classes under WT−1W_{T-1}:

min⁡θt,WtE(x,y)∼Dt[Lce(Wtfθt(x),y)]s.t.arg⁡max⁡c∈Cold wc⊤fθt(x)=arg⁡max⁡c∈Cold wc⊤fθt−1(x).(2)\begin{aligned} \min_{\theta_t,W_t}\quad&\mathbb{E}_{(x,y)\sim\mathcal{D}_t}\left[\mathcal{L}_{\mathrm{ce}}(W_t f_{\theta_t}(x),y)\right]\\ \mathrm{s.t.}\quad&\underset{c\in\mathcal{C}_{\mathrm{old}}}{\arg\max}\ w_c^{\top}f_{\theta_t}(x)=\underset{c\in\mathcal{C}_{\mathrm{old}}}{\arg\max}\ w_c^{\top}f_{\theta_{t-1}}(x). \tag*{(2)} \end{aligned}

Equation (2) translates intractable historical risk minimization into a classification-decision invariance constraint anchored by the old global head WT−1W_{T-1}.

Derivation of Relative-Logit Regularization

For the discontinuous arg max constraint in Eq. (2), we construct a continuous penalty based on a sufficient condition for decision preservation.

For any input xx, the model’s relative preference between two old classes i,j∈Coldi,j\in\mathcal{C}_{\mathrm{old}} under WT−1W_{T-1} is governed by their pairwise logit margin:

mij(s)(x)=(wi−wj)⊤fθs(x),s∈{t−1,t}.(3)m_{ij}^{(s)}(x)=(w_i-w_j)^{\top}f_{\theta_s}(x),\qquad s\in\{t-1,t\}. \tag*{(3)}

Preserving the arg max prediction requires that the margin between the winning class and every other old class does not flip sign. To retain the full discriminative geometry among old classes and obtain a smooth differentiable objective, we tighten sign preservation into a sufficient condition: keeping all pairwise logit differences across old classes invariant. Denoting the feature drift by δf(x)=fθt(x)−fθt−1(x)\delta f(x)=f_{\theta_t}(x)-f_{\theta_{t-1}}(x), this condition takes the form

mij(t)(x)=mij(t−1)(x)⟺(wi−wj)⊤δf(x)=0,∀i,j∈Cold.(4)\begin{aligned} m_{ij}^{(t)}(x)&=m_{ij}^{(t-1)}(x)\\ &\Longleftrightarrow(w_i-w_j)^{\top}\delta f(x)=0,\quad\forall i,j\in\mathcal{C}_{\mathrm{old}}. \tag*{(4)} \end{aligned}

Penalizing the sum of squared margin drifts over all K(K−1)/2K(K-1)/2 old-class pairs yields a quadratic soft constraint. Let δℓi(x)=wi⊤δf(x)\delta\ell_i(x)=w_i^{\top}\delta f(x) denote the ii-th old-logit drift and δℓ‾(x)=1K∑i=1Kδℓi(x)\overline{\delta\ell}(x)=\frac{1}{K}\sum_{i=1}^{K}\delta\ell_i(x) denote the mean old-logit drift. By algebraic identity, the pairwise margin penalty reduces to a centered relative-logit norm:

Llogit(x)=1K∑1≤i<j≤K((wi−wj)⊤δf(x))2=∑i=1K(δℓi(x)−δℓ‾(x))2=∥Wcδf(x)∥22,(5)\begin{aligned} \mathcal{L}_{\mathrm{logit}}(x)&=\frac{1}{K}\sum_{1\leq i<j\leq K}\left((w_i-w_j)^{\top}\delta f(x)\right)^2\\ &=\sum_{i=1}^{K}\left(\delta\ell_i(x)-\overline{\delta\ell}(x)\right)^2=\left\|W_c\delta f(x)\right\|_2^2, \tag*{(5)} \end{aligned}

where Wc=(IK−1K11⊤)WT−1∈RK×dW_c=\left(I_K-\frac{1}{K}\mathbf{1}\mathbf{1}^{\top}\right)W_{T-1}\in\mathbb{R}^{K\times d} is the row-centered old classifier head.

This reduction shows that preserving all O(K2)O(K^2) pairwise margins simplifies to an ℓ2\ell_2 penalty on centered relative logits. Because Eq. (5) permits all old-class logits to share an arbitrary scalar translation a(x)a(x), namely WT−1fθt(x)=WT−1fθt−1(x)+a(x)1W_{T-1}f_{\theta_t}(x)=W_{T-1}f_{\theta_{t-1}}(x)+a(x)\mathbf{1}, achieving Llogit(x)=0\mathcal{L}_{\mathrm{logit}}(x)=0 strictly preserves not only the arg max classification decision and relative class rankings, but also the exact conditional probability distribution softmax⁡(WT−1fθt(x))\operatorname{softmax}(W_{T-1}f_{\theta_t}(x)) among old classes.

Spectral Whitening and Feature Training

To examine how Llogit(x)\mathcal{L}_{\mathrm{logit}}(x) constrains feature drift in the representation space, we perform the compact singular value decomposition (SVD) on the centered old classifier head WcW_c:

Wc=USV⊤,S=diag⁡(σ1,…,σr),(6)W_c=USV^{\top},\qquad S=\operatorname{diag}(\sigma_1,\ldots,\sigma_r), \tag*{(6)}

where r=rank⁡(Wc)≤min⁡(d,K−1)r=\operatorname{rank}(W_c)\leq\min(d,K-1). Substituting Eq. (6) into Eq. (5) yields

Llogit(x)=∥USV⊤δf(x)∥22=∥SV⊤δf(x)∥22=∑k=1rσk2(vk⊤δf(x))2.(7)\begin{aligned} \mathcal{L}_{\mathrm{logit}}(x)&=\left\|USV^{\top}\delta f(x)\right\|_2^2=\left\|SV^{\top}\delta f(x)\right\|_2^2\\ &=\sum_{k=1}^{r}\sigma_k^2\left(v_k^{\top}\delta f(x)\right)^2. \tag*{(7)} \end{aligned}

Equation (7) reveals mathematically that although Llogit(x)\mathcal{L}_{\mathrm{logit}}(x) eliminates the redundant shared logit offset, it couples the regularization weight along each discriminative direction vkv_k to the squared singular value σk2\sigma_k^2 of WcW_c. As our empirical analysis in § 4.5 verifies, protecting the discriminative basis VV identified by WcW_c is far more effective than protecting a random orthonormal basis of equal rank rr, and applying spectral whitening across VV outperforms weighting directions by σk2\sigma_k^2 when controlling for average regularization scale. Motivated by these observations, we apply the spectral whitening transform S−1U⊤S^{-1}U^{\top} to the relative-logit drift to strip away σk2\sigma_k^2 and constrain all active discriminative directions uniformly:

Ldis(x)=∥S−1U⊤Wcδf(x)∥22=∥V⊤δf(x)∥22.(8)\mathcal{L}_{\mathrm{dis}}(x)=\left\|S^{-1}U^{\top}W_c\delta f(x)\right\|_2^2=\left\|V^{\top}\delta f(x)\right\|_2^2. \tag*{(8)}

Because WT−1W_{T-1} is frozen during stage tt, the whitened discriminative basis VV is extracted once before task training, and evaluating Eq. (8) requires only O(dr)O(dr) operations per sample.

We add a mild global drift penalty to promote overall feature-space smoothness. Highlighting the contribution of Ldis\mathcal{L}_{\mathrm{dis}}, we write the stage-tt feature training objective as (with δf(x)=fθt(x)−fθt−1(x)\delta f(x)=f_{\theta_t}(x)-f_{\theta_{t-1}}(x)):

Ltrain(θt,Wt)=E(x,y)∼Dt[Lce(Wtfθt(x),y)+λ1∥V⊤δf(x)∥22+λ2∥(Id−VV⊤)δf(x)∥22],(9)\begin{aligned} \mathcal{L}_{\mathrm{train}}(\theta_t,W_t)&=\mathbb{E}_{(x,y)\sim\mathcal{D}_t}\left[\mathcal{L}_{\mathrm{ce}}(W_tf_{\theta_t}(x),y)\right.\\ &\left.\qquad+\lambda_1\left\|V^{\top}\delta f(x)\right\|_2^2+\lambda_2\left\|(I_d-VV^{\top})\delta f(x)\right\|_2^2\right], \tag*{(9)} \end{aligned}

where λ1,λ2>0\lambda_1,\lambda_2>0 are balancing hyperparameters. The regularization equals λ2∥δf(x)∥22+(λ1−λ2)∥V⊤δf(x)∥22\lambda_2\|\delta f(x)\|_2^2+(\lambda_1-\lambda_2)\|V^{\top}\delta f(x)\|_2^2, combining global drift control with direction-specific weighting and requiring only one low-rank projection.

Affine Statistics Compensation and Global Classifier Calibration

Because DPR derives the discriminative basis VV directly from the frozen old classifier head, sustaining its effectiveness across tasks relies on a well-calibrated global classifier WTW_T over all seen classes C1:t\mathcal{C}_{1:t} at the end of each stage. However, Eq. (9) trains only the local head WtW_t, and the update from fθt−1f_{\theta_{t-1}} to fθtf_{\theta_t} causes stored old-class distributions to undergo location and covariance drift, preventing direct alignment with new-class features. We therefore introduce a closed-form affine transformation to compensate for old-class statistics drift and calibrate WTW_T for both unified inference and next-stage decision regularization.

Affine drift estimation and statistics compensation. Once feature training converges, we freeze both fθt−1f_{\theta_{t-1}} and fθtf_{\theta_t}, and extract paired features Z=[fθt−1(x1),…,fθt−1(xn)]∈Rd×nZ = [f_{\theta_{t-1}}(x_1), \ldots, f_{\theta_{t-1}}(x_n)] \in\mathbb{R}^{d \times n} and Z′=[fθt(x1),…,fθt(xn)]∈Rd×nZ' = [f_{\theta_t}(x_1), \ldots, f_{\theta_t}(x_n)] \in\mathbb{R}^{d \times n} over the nn samples of DtD_t. We fit a shared affine transformation z′≈Az+bz' \approx Az + b to the cross-stage feature drift via ridge regression with an identity prior:

min⁡A,b∥Z′−(AZ+b1n⊤)∥F2+ρ∥A−Id∥F2(10)\min_{A,b} \left\| Z' - (AZ + b\mathbf{1}_n^\top) \right\|_F^2 + \rho\left\| A-I_d \right\|_F^2 \tag*{(10)}

where ρ>0\rho> 0 shrinks weakly supported directions toward the identity map. Let zˉ,zˉ′∈Rd\bar{z}, \bar{z}' \in\mathbb{R}^d denote the sample means and Z~,Z~′\widetilde{Z}, \widetilde{Z}' denote the centered feature matrices; Eq. (10) admits the closed-form solution

A=(Z~′Z~⊤+ρId)(Z~Z~⊤+ρId)−1,b=zˉ′−Azˉ.(11)A = \left(\widetilde{Z}'\widetilde{Z}^{\top}+\rho I_d\right)\left(\widetilde{Z}\widetilde{Z}^{\top}+\rho I_d\right)^{-1},\quad b=\bar{z}'-A\bar{z}. \tag*{(11)}

For each old class c∈Coldc \in\mathcal{C}_{\mathrm{old}}, we maintain only its first- and second-order statistics (μc,Σc)(\mu_c,\Sigma_c) in memory and transport them to the updated feature space:

μ^c=Aμc+b,Σ^c=AΣcA⊤,∀c∈Cold.(12)\widehat{\mu}_c=A\mu_c+b,\qquad\widehat{\Sigma}_c=A\Sigma_c A^{\top},\qquad\forall c\in\mathcal{C}_{\mathrm{old}}. \tag*{(12)}

The means and covariances of current-task classes c∈Ctc \in\mathcal{C}_t are computed directly from the cached features Z′Z'.

Global classifier calibration. With fθtf_{\theta_t} frozen, we sample synthetic old-class features from Gaussian distributions N(μ^c,Σ^c)\mathcal{N}(\widehat{\mu}_c,\widehat{\Sigma}_c) in the feature space of fθtf_{\theta_t} and jointly train the global classifier head WT∈R∣C1:t∣×dW_T \in\mathbb{R}^{|\mathcal{C}_{1:t}|\times d} alongside real current-task features:

Lcal(WT)=E(x,y)∼Dt[Lce(WTfθt(x),y)]+Ec∈ColdEz^∼N(μ^c,Σ^c)[Lce(WTz^,c)].(13)\mathcal{L}_{\mathrm{cal}}(W_T)=\mathbb{E}_{(x,y)\sim D_t}\left[\mathcal{L}_{\mathrm{ce}}\left(W_Tf_{\theta_t}(x),y\right)\right]+\mathbb{E}_{c\in\mathcal{C}_{\mathrm{old}}}\mathbb{E}_{\widehat{z}\sim\mathcal{N}(\widehat{\mu}_c,\widehat{\Sigma}_c)}\left[\mathcal{L}_{\mathrm{ce}}\left(W_T\widehat{z},c\right)\right]. \tag*{(13)}

Algorithm 1 summarizes each incremental task.

Algorithm 1 DPR at task t>1t>1

Require: DtD_t, fθt−1f_{\theta_{t-1}}, WT−1W_{T-1}, old-class moments

  1. Extract VV from centered WT−1W_{T-1} via spectral whitening (Eqs. (6), (8)).

  2. Optimize (θt,Wt)(\theta_t,W_t) using Ltrain\mathcal{L}_{\mathrm{train}} (Eq. (9)).

  3. Solve (A,b)(A,b) (Eq. (11)); transport old-class moments (Eq. (12)) and compute new-class moments.

  4. Freeze fθtf_{\theta_t}; calibrate WTW_T using Lcal\mathcal{L}_{\mathrm{cal}} (Eq. (13)).

Ensure: fθtf_{\theta_t}, WTW_T, and updated moments for task t+1t+1

Inference: y^=arg⁡max⁡c∈C1:t(WTfθt(x))c\hat{y}=\arg\max_{c\in\mathcal{C}_{1:t}}(W_Tf_{\theta_t}(x))_c.

Experiments

In this section, we first present the experimental setup (§ 4.1), compare DPR with state-of-the-art EFCIL methods on standard and large-scale benchmarks (Sec. 4.2), and then analyze component contributions, decision preservation and plasticity, the discriminative basis and spectral whitening, and runtime (§ 4.3 to § 4.6).

Experimental Settings

Datasets and metrics. Our main evaluation uses CIFAR-100 [12], Tiny-ImageNet [21], and ImageNet-Subset [3], and we further evaluate scalability on full ImageNet-1K [3]. Following the cold-start EFCIL protocol [14], classes are split evenly into T=10T=10 or T=20T=20 sequential tasks without a pre-trained or large initial task. CIFAR-100 and ImageNet-Subset contain 100 classes (10 or 5 classes per task), Tiny-ImageNet contains 200 classes (20 or 10 classes per task), and ImageNet-1K contains 1,000 classes (100 or 50 classes per task). CIFAR-100 and Tiny-ImageNet provide 500 training images per class and 100 or 50 test images per class, respectively. ImageNet-Subset uses the standard 100-class split [4] with approximately 1,300 training and 50 validation images per class. Let ata_t denote classification accuracy over all classes seen through task tt (C1:t\mathcal{C}_{1:t}), evaluated after global classifier calibration without task identifiers. We report final accuracy Acc=aT\mathrm{Acc}=a_T (↑\uparrow) and average anytime accuracy AAA=1T∑t=1Tat\mathrm{AAA}=\frac{1}{T}\sum_{t=1}^{T}a_t (↑\uparrow) in percent as mean ±\pm standard deviation over five runs.

Main Results

Competing methods and implementation details. We compare DPR against eleven representative baselines: EWC [11], LwF [13], PASS [26], FeTrIL [18], SSRE [27], EFC [14], ADC [8], LDC [6], AdaGauss [19], DPCR [9], and BiCyc [22]. Following standard cold-start EF CIL settings [14, 15], all methods use a ResNet-18 backbone (d=512d = 512) trained from scratch. We optimize incremental representation learning (Eq. (9)) using Adam with weight decay 2×10−42 \times10^{-4}, a fixed learning rate, 100 epochs per task, and batch size 64. The learning rate is 10−410^{-4} for both the backbone and local head on CIFAR-100 and Tiny-ImageNet, and 10−510^{-5} for the backbone and 10−410^{-4} for the head on ImageNet-Subset. Across all datasets, we fix the regularization coefficients to λ1=0.5\lambda_1 = 0.5 and λ2=0.01\lambda_2 = 0.01. Closed-form affine compensation (Eq. (11)) uses ridge parameter ρ=10−3\rho= 10^{-3}, and global classifier calibration (Eq. (13)) trains the bias-free linear head WTW_T for 50 epochs (batch size 256) using SGD with fixed learning rate 10−310^{-3} on sampled Gaussian features and cached current-task features. All experiments are conducted on a single NVIDIA GeForce RTX 3090 Ti GPU.

Main Results

DPR achieves the highest final accuracy and AAA on all three datasets with 10 tasks, and on Tiny-ImageNet and ImageNet-Subset with 20 tasks (Table 1 and Table 2). The final-

MethodVenueCIFAR-100 (T=10T = 10) Acc ↑CIFAR-100 (T=10T = 10) AAA ↑Tiny-ImageNet (T=10T = 10) Acc ↑Tiny-ImageNet (T=10T = 10) AAA ↑ImageNet-Subset (T=10T = 10) Acc ↑ImageNet-Subset (T=10T = 10) AAA ↑
EWC [11]PNAS’1730.90 ± 1.950.40 ± 1.718.50 ± 1.834.30 ± 2.325.10 ± 2.840.60 ± 3.3
LwF [13]ECCV’1631.90 ± 1.151.80 ± 1.527.10 ± 1.539.60 ± 2.033.40 ± 2.251.50 ± 1.6
PASS [26]CVPR’2130.80 ± 1.248.30 ± 1.124.50 ± 0.639.50 ± 1.026.40 ± 1.345.70 ± 0.2
FeTrIL [18]WACV’2334.90 ± 0.551.20 ± 1.131.00 ± 0.945.30 ± 1.836.20 ± 1.252.60 ± 0.6
SSRE [27]CVPR’2231.65 ± 1.147.26 ± 1.024.15 ± 0.738.82 ± 0.926.65 ± 1.243.76 ± 0.3
EFC [14]ICLR’2443.50 ± 0.858.10 ± 1.234.50 ± 1.147.90 ± 1.550.90 ± 1.161.30 ± 1.2
ADC [8]CVPR’2446.50 ± 1.261.40 ± 1.632.30 ± 1.543.00 ± 1.938.30 ± 1.255.50 ± 1.5
LDC [6]ECCV’2445.40 ± 1.659.50 ± 1.934.20 ± 1.146.80 ± 1.651.40 ± 1.269.40 ± 0.6
AdaGauss [19]NeurIPS’2446.80 ± 1.260.90 ± 1.032.90 ± 0.945.80 ± 1.351.10 ± 1.265.00 ± 1.4
DPCR [9]ICML’2550.20 ± 0.762.80 ± 1.133.90 ± 1.846.90 ± 0.949.90 ± 0.864.80 ± 1.1
BiCyc [22]ICLR’2650.60 ± 0.963.20 ± 1.335.40 ± 0.849.10 ± 1.452.70 ± 0.966.80 ± 1.4
DPR (ours)—51.31 ± 0.864.04 ± 1.240.36 ± 0.951.57 ± 1.458.94 ± 1.069.43 ± 1.3

Table 1. Final accuracy (Acc) and average anytime accuracy (AAA), in percent. Higher is better. Bold: best; _underline_: second best. Values are mean ± standard deviation.

MethodVenueCIFAR-100 (T=20T = 20) Acc ↑CIFAR-100 (T=20T = 20) AAA ↑Tiny-ImageNet (T=20T = 20) Acc ↑Tiny-ImageNet (T=20T = 20) AAA ↑ImageNet-Subset (T=20T = 20) Acc ↑ImageNet-Subset (T=20T = 20) AAA ↑
EWC [11]PNAS’1717.00 ± 1.634.20 ± 2.111.30 ± 1.926.80 ± 2.513.70 ± 2.129.20 ± 2.5
LwF [13]ECCV’1617.60 ± 1.239.20 ± 1.715.20 ± 1.631.50 ± 2.118.60 ± 1.641.30 ± 1.9
PASS [26]CVPR’2117.60 ± 0.831.10 ± 1.318.50 ± 1.430.40 ± 1.914.40 ± 1.231.70 ± 0.4
FeTrIL [18]WACV’2323.30 ± 1.437.90 ± 1.225.90 ± 1.239.90 ± 1.226.60 ± 1.542.40 ± 2.1
SSRE [27]CVPR’2218.75 ± 0.932.45 ± 1.218.55 ± 1.330.62 ± 1.817.45 ± 1.131.15 ± 0.5
EFC [14]ICLR’2432.40 ± 0.947.00 ± 1.328.40 ± 1.242.10 ± 1.638.60 ± 1.250.50 ± 1.5
ADC [8]CVPR’2435.10 ± 1.451.70 ± 1.818.10 ± 1.636.00 ± 2.125.10 ± 1.343.40 ± 1.7
LDC [6]ECCV’2435.50 ± 1.951.90 ± 2.324.90 ± 2.238.20 ± 2.728.50 ± 1.746.50 ± 2.7
AdaGauss [19]NeurIPS’2437.90 ± 1.054.40 ± 0.827.50 ± 1.239.50 ± 1.142.60 ± 1.657.40 ± 1.9
DPCR [9]ICML’2539.80 ± 1.254.80 ± 0.925.60 ± 0.739.30 ± 0.637.30 ± 1.654.70 ± 0.7
BiCyc [22]ICLR’2641.50 ± 1.156.50 ± 1.3_30.20_ ± 1.1_44.20_ ± 1.3_43.80_ ± 1.4_58.20_ ± 1.8
DPR (ours)—41.22 ± 1.154.82 ± 1.535.25 ± 1.147.37 ± 1.450.40 ± 1.363.39 ± 1.6

Table 2. Final accuracy (Acc) and average anytime accuracy (AAA), in percent. Higher is better. Bold: best; _underline_: second best. Values are mean ± standard deviation.

accuracy gains over the strongest baseline are largest on Tiny-ImageNet and ImageNet-Subset, ranging from 4.96 to 6.60 pp. With 20 tasks, AAA also improves by 3.17 pp on Tiny-ImageNet and 5.19 pp on ImageNet-Subset, showing that the benefit extends to average performance over the learning sequence. The task-wise curves in Figure 3 compare DPR with EFC, DPCR, AdaGauss, and BiCyc, showing its advantage through the later stages of learning.

Figure 3. Accuracy over all seen classes after each task on (a) Tiny-ImageNet, 10 tasks; (b) ImageNet-Subset, 10 tasks; and (c) ImageNet-Subset, 20 tasks. Final gains are in pp relative to the strongest displayed baseline.

Large-scale scenarios. DPR also outperforms BiCyc on full ImageNet-1K under both task splits (Table 3), using the same incremental-stage configuration as ImageNet-Subset. Final accuracy improves by 3.83 and 3.43 pp for 10 and 20 tasks, respectively, while AAA improves by 3.93 and 1.04 pp. These gains show that DPR remains effective as the number of classes grows to 1,000.

MethodT=10T = 10 (100 cls/task) Acc ↑T=10T = 10 (100 cls/task) AAA ↑T=20T = 20 (50 cls/task) Acc ↑T=20T = 20 (50 cls/task) AAA ↑
BiCyc [22]38.4852.3732.7449.12
DPR (ours)42.3156.3036.1750.16

Table 3. Large-scale evaluation on ImageNet-1K. Final accuracy (Acc, %) and average anytime accuracy (AAA, %) under 10- and 20-task cold-start splits.

λ1\lambda_1λ2\lambda_20.010.020.03
0.451.1550.4450.33
0.551.3151.0450.45
0.651.1351.0050.36

Table 4b. (b) Sensitivity on T=10T = 10 (Acc, %).

λ1\lambda_1λ2\lambda_20.010.020.03
0.441.5341.1140.36
0.541.2241.0440.82
0.641.4040.9640.74

Table 4c. (c) Sensitivity on T=20T = 20 (Acc, %).

Component Ablations

Regularization terms. On CIFAR-100 with 10 tasks, we remove each component in turn (Table 4a). Removing the whitened decision-preserving penalty Ldis\mathcal{L}_{\mathrm{dis}} reduces final accuracy by 9.91 pp and AAA by 4.50 pp, showing its central role in preserving performance. Adding the global feature-drift penalty δf\delta f to Ldis\mathcal{L}_{\mathrm{dis}} improves final accuracy by 2.04 pp and also improves AAA, providing an additional benefit by constraining feature drift. We further analyze sensitivity to λ1∈{0.4,0.5,0.6}\lambda_1 \in\{0.4, 0.5, 0.6\} and λ2∈{0.01,0.02,0.03}\lambda_2 \in\{0.01, 0.02, 0.03\} (Table 4b, c). Final accuracy varies by 0.98 and 1.17 pp across the grid for 10 and 20 tasks, respectively, indicating limited sensitivity within this neighborhood.

Enabled componentsAccuracy (%)
Ldis\mathcal{L}_{\mathrm{dis}}δf\delta fAffineAcc ↑\uparrowAAA ↑\uparrow
✓✗✓49.2763.57
✗✓✓41.4059.54
✓✓✗46.9561.93
✓✓✓51.3164.04

Table 4. Component ablations and coefficient sensitivity on CIFAR-100. (a) Checkmarks indicate enabled components; Acc and AAA are percentages. (b, c) Final accuracy (%) across coefficient pairs. Boxes mark (λ1,λ2)=(0.5,0.01)(\lambda_1, \lambda_2) = (0.5, 0.01); darker cells indicate higher accuracy within each panel. (a) Component ablations (T=10T = 10).

Affine compensation. Using untransported old-class means and covariances during global classifier calibration lowers final accuracy by 4.36 pp and AAA by 2.12 pp (Table 4a). Affine transport therefore keeps the stored distributions aligned with the evolving representation.

Decision Preservation and New-Class Plasticity

We compare DPR with isotropic feature distillation (FD) on 10-task Tiny-ImageNet to examine how directional protection affects old decisions and new-class learning. At each stage, both updates start from the same preceding DPR model, head, and stored statistics, and apply the same affine compensation and classifier calibration. Each FD branch runs one incremental stage; subsequent stages start from the DPR trajectory. FD penalizes μt(s)∥δf∥22\mu_t(s)\lVert\delta f\rVert_2^2, where μt(s)=s[λ1r+λ2(d−r)]/d\mu_t(s) = s[\lambda_1r + \lambda_2(d-r)]/d and s=1s = 1 matches the trace of DPR’s regularization matrix.

Preserving correct old-class decisions. DPR reduces decision damage at every incremental stage (Figure 4a). With the old head fixed and prediction restricted to old classes, we measure the fraction of previously correct old-class examples that become incorrect after the backbone update. Averaged equally over stages 2 to 10, this rate falls from 6.02% for trace-matched FD to 3.97% for DPR, a reduction of 2.05 pp. Thus, concentrating protection on discriminative directions better preserves old-class decisions during new-class training.

Figure 4. Protecting discriminative directions preserves decisions while allowing new-class learning. (a) On 10-task Tiny-ImageNet, DPR (blue) reduces fixed-old-head correct-to-wrong rates at every stage relative to trace-matched feature distillation (FD, gray). (b) After calibration, old/new class accuracies are averaged equally over stages 2, 5, and 10; DPR achieves the highest old/new mean across all FD strengths. (c) On 10-task CIFAR-100, replacing VV with a random basis QQ loses 4.62 pp; normalized spectral weighting stays within 0.74 pp of whitening.

Old/new-class tradeoff. DPR achieves the highest mean of old- and new-class accuracy across the FD coefficient sweep (Figure 4b). We evaluate s∈{0.1,0.3,1,3,10}s \in\{0.1, 0.3, 1, 3, 10\} at stages 2, 5, and 10. After calibration, predictions cover all seen classes; old- and new-class macro accuracies are averaged equally over these three stages. DPR attains 49.56% old-class and 52.57% new-class accuracy, exceeding both metrics of the two weakest FD configurations. Relative to trace-matched FD, DPR improves new-class accuracy by 2.57 pp with a 1.09 pp decrease in old-class accuracy. Its old/new mean reaches 51.06%, compared with 50.87% for the best FD setting. Stronger isotropic penalties further reduce new-class accuracy, supporting selective protection of the old classifier’s discriminative directions.

Discriminative Basis and Spectral Whitening

Discriminative basis. To test whether the row-centered old classifier head WcW_c identifies the directions critical for retention, we replace VV on 10-task CIFAR-100 with a random orthonormal basis Q∈Rd×rQ \in\mathbb{R}^{d \times r} of identical rank rr under the same whitened form, updating the complementary projector to Id−QQTI_d - QQ^\mathsf{T}. Replacing VV with QQ lowers accuracy at every incremental stage (Figure 4c), reducing final accuracy from 51.04% to 46.42% (−4.62-4.62 pp) and AAA from 63.81% to 61.08% (−2.73-2.73 pp). This supports protecting the discriminative directions identified by the row space of WcW_c.

Spectral whitening. Next, we retain the classifier-derived basis VV and compare spectral whitening against normalized spectral weights αk=σk2/(r−1∑j=1rσj2)\alpha_k = \sigma_k^2/(r^{-1}\sum_{j=1}^{r}\sigma_j^2), whose mean is normalized to one so that the overall regularization scale matches λ1\lambda_1. Spectral whitening achieves 51.04% final accuracy and 63.81% AAA, compared with 51.00% and 63.72% for normalized spectral weighting. Stage-wise differences remain below 0.74 pp across all ten stages (Figure 4c). Whitening therefore removes dependence on the classifier’s singular-value scales while retaining comparable accuracy and a small final advantage, supporting Eq. (8).

Efficiency

As shown in Figure 1, DPR achieves 58.94% and 50.40% final accuracy on ImageNet-Subset with 10 and 20 tasks, respectively. Its 10-task training takes 5.04 hours, compared with 9.40 hours for BiCyc, and is 1.05×, 2.12×, and 2.41× faster than EFC, AdaGauss, and DPCR, respectively.

Component-wise runtime efficiency. Our three method-specific components account for only 1.39% of per-task runtime on 10-task Tiny-ImageNet (Figure 5). Averaged over Stages 2 to 10, an incremental task takes 717.62 s: whitening and drift regularization cost 3.75 s, closed-form affine compensation 3.78 s, and global classifier calibration 2.48 s, totaling 10.00 s. Basis extraction runs once per task, affine compensation uses a closed-form solve, and calibration operates directly on features. See the supplementary material for per-task runtimes.

Figure 5. Method-specific operations account for only 1.39% of runtime. Runtime shares computed from mean component times over Stages 2 to 10 on Tiny-ImageNet (T=10T = 10). Whitening and drift regularization (teal), affine compensation (gold), and classifier calibration (rose) total 10.00 s of the 717.62 s average task; grey denotes all other operations.

Storage Analysis. Like prior methods, DPR stores per-class means and symmetric covariances (μc\mu_c, Σc\Sigma_c) during training (Table 5). Extra parameters are counted beyond the backbone and classifier, and “Discardable” indicates whether stored

MethodPrototypesCovariancesExtra Params.Discardable
EFC [14]✓✓×✓
AdaGauss [19]✓✓✓×
DPCR [9]✓✓×✓
BiCyc [22]✓✓✓×
DPR (ours)✓✓×✓

Table 5. Storage analysis of compared methods.

statistics can be removed at inference. Although AdaGauss and BiCyc store lower-dimensional statistics, their projection layers introduce extra parameters, and these statistics must be retained for Mahalanobis inference.

Conclusion

We presented Decision-Preserving Regularization (DPR) for cold-start exemplar-free class-incremental learning. Reformulating the intractable joint-learning objective as classification-decision invariance under the frozen old classifier reduces pairwise margin preservation to a relative-logit penalty on the row-centered classifier head. Applying spectral whitening decouples the classifier’s spectral scaling and extracts an orthonormal discriminative basis directly from the classifier weights, constraining active decision directions in O(dr)\mathcal{O}(d r) time per sample while leaving complementary directions lightly regularized for new-class adaptation. Because anchoring decision invariance across sequential stages relies on a well-calibrated global classifier over all seen classes, we pair whitened regularization with closed-form ridge-regression affine transport of old-class Gaussian moments and feature-space classifier calibration. Together, these two components make DPR simple, efficient, and effective across CIFAR-100, Tiny-ImageNet, ImageNet-Subset, and ImageNet-1K.

Limitations. Fixed (λ1,λ2)=(0.5,0.01)(\lambda_1,\lambda_2) = (0.5, 0.01) performs robustly across benchmarks, although longer task sequences mildly favor λ1=0.4\lambda_1 = 0.4 (Table 4b, Table 4c). The rank r≤min⁡(d,K−1)r \le\min(d, K - 1) can reach dd when K>dK > d and WcW_c has full column rank; adaptive spectral truncation is a promising direction for thousand-class learning.

LLM Usage Statement

We used OpenAI’s ChatGPT and Codex to assist with language polishing and internal manuscript review. The authors are responsible for the final content.

Per-Task Runtime Analysis

Figure A1 expands the mean runtime breakdown in § 4.6 of the main paper into individual incremental tasks on Tiny-ImageNet (T=10T = 10, ResNet-18, d=512d = 512). Across Stages 2 to 10, the three method-specific components take 7.34 to 12.13 s per task, or 1.03% to 1.68% of total runtime, as the number of old classes grows from 20 to 180.

Figure A1. Method-specific runtime remains below 2% at every incremental stage. Per-task wall-clock time on 10-task Tiny-ImageNet. Stacked components use the same colors as Figure 5 in the main paper; grey denotes all other operations. The broken axis separates component costs from total task time. Labels show total seconds, component totals, and their shares.

The basis extraction takes 2.82 ms per task on average. Directional regularization costs 0.24 ms per batch, compared with 32.22 ms for classification forward/backward; using ∥(Id−VV⊤)δf∥22=∥δf∥22−∥V⊤δf∥22\lVert(I_d - V V^\top)\delta f \rVert_2^2 = \lVert\delta f \rVert_2^2 - \lVert V^\top\delta f \rVert_2^2 reduces this to 0.15 ms. Affine compensation takes 3.52 s for paired feature extraction, 5.05 ms for the ridge solve, and 0.25 s for old-class moment transport. Global classifier calibration takes 0.68 to 4.11 s as the number of seen classes increases from 40 to 200. Runtime shares use the full incremental-task time, including backbone training and the frozen teacher forward pass; the regularization cost is estimated from batch microbenchmarks.

References

  1. [1]Rahaf Aljundi, Francesca Babiloni, Mohamed Elhoseiny, Marcus Rohrbach, and Tinne Tuytelaars. Memory aware synapses: Learning what (not) to forget. In ECCV, pages 139–154, 2018.arxiv.org/abs/1711.09601
  2. [2]Matthias De Lange, Rahaf Aljundi, Marc Masana, Sarah Parisot, Xu Jia, Aleš Leonardis, Gregory Slabaugh, and Tinne Tuytelaars. A continual learning survey: Defying forgetting in classification tasks. IEEE TPAMI, 44(7):3366–3385, 2021.DOI
  3. [3]Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. ImageNet: A large-scale hierarchical image database. In CVPR, pages 248–255, 2009.
  4. [4]Arthur Douillard, Matthieu Cord, Charles Ollion, Thomas Robert, and Eduardo Valle. PODNet: Pooled outputs distillation for small-tasks incremental learning. In ECCV, pages 86–102, 2020.arxiv.org/abs/2004.13513
  5. [5]Robert M. French. Catastrophic forgetting in connectionist networks. Trends in Cognitive Sciences, 3(4):128–135, 1999.
  6. [6]Alex Gomez-Villa, Dipam Goswami, Kai Wang, Andrew D. Bagdanov, Bartłomiej Twardowski, and Joost van de Weijer. Exemplar-free continual representation learning via learnable drift compensation. In ECCV, pages 473–490, 2024.
  7. [7]Dipam Goswami, Yuyang Liu, Bartłomiej Twardowski, and Joost van de Weijer. FeCAM: Exploiting the heterogeneity of class distributions in exemplar-free continual learning. In NeurIPS, pages 6582–6595, 2023.arxiv.org/abs/2309.14062
  8. [8]Dipam Goswami, Albin Soutif-Cormerais, Yuyang Liu, Sandesh Kamath, Bartłomiej Twardowski, and Joost van de Weijer. Restructuring old classes with new data for exemplar-free continual learning. In CVPR, pages 28525–28534, 2024.arxiv.org/abs/2405.19074
  9. [9]Run He, Di Fang, Yicheng Xu, et al. Semantic shift estimation via dual-projection and classifier reconstruction for exemplar-free class-incremental learning. In ICML, 2025.arxiv.org/abs/2503.05423
  10. [10]Saihui Hou, Xinyu Pan, Chen Change Loy, Zilei Wang, and Dahua Lin. Learning a unified classifier incrementally via rebalancing. In CVPR, pages 831–839, 2019.DOI
  11. [11]James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, et al. Overcoming catastrophic forgetting in neural networks. Proceedings of the National Academy of Sciences, 114(13):3521–3526, 2017.DOI
  12. [12]Alex Krizhevsky. Learning multiple layers of features from tiny images. Technical report, University of Toronto, 2009.
  13. [13]Zhizhong Li and Derek Hoiem. Learning without forgetting. IEEE TPAMI, 40(12):2935–2947, 2018.
  14. [14]Simone Magistri, Tomaso Trinci, Albin Soutif-Cormerais, Joost van de Weijer, and Andrew D. Bagdanov. Elastic feature consolidation for cold start exemplar-free incremental learning. In ICLR, 2024.
  15. [15]Simone Magistri, Tomaso Trinci, Albin Soutif-Cormerais, Joost van de Weijer, and Andrew D. Bagdanov. EFC++: Elastic feature consolidation with prototype re-balancing for cold start exemplar-free incremental learning. arXiv preprint arXiv:2503.10439, 2025.DOI
  16. [16]Marc Masana, Xialei Liu, Bartłomiej Twardowski, Mikel Menta, Andrew D. Bagdanov, and Joost van de Weijer. Class-incremental learning: Survey and performance evaluation on image classification. IEEE TPAMI, 45(5):5513–5533, 2023.DOI
  17. [17]Michael McCloskey and Neal J. Cohen. Catastrophic interference in connectionist networks: The sequential learning problem. In Psychology of Learning and Motivation, pages 109–165. Elsevier, 1989.DOI
  18. [18]Grégoire Petit, Adrian Popescu, Hugo Schindler, David Picard, and Bertrand Delozeide. FeTrIL: Feature translation for exemplar-free class-incremental learning. In WACV, pages 3911–3920, 2023.
  19. [19]Grzegorz Rypeść, Sebastian Cygert, Tomasz Trzciński, and Bartłomiej Twardowski. Task-recency bias strikes back: Adapting covariances in exemplar-free class incremental learning. In NeurIPS, pages 63268–63289, 2024.
  20. [20]Xiao Wang, Tianze Chen, Qiming Ge, et al. Orthogonal subspace learning for language model continual learning. In Findings of EMNLP, pages 10658–10671, 2023.arxiv.org/abs/2310.14152
  21. [21]Jiayu Wu, Qixiang Zhang, and Guoxu Xu. Tiny ImageNet challenge. Technical report, Stanford University, 2017.
  22. [22]Hongye Xu and Bartosz Krawczyk. Two-way is better than one: Bidirectional alignment with cycle consistency for exemplar-free class-incremental learning. In ICLR, 2026.arxiv.org/abs/2606.05675
  23. [23]Lu Yu, Bartłomiej Twardowski, Xialei Liu, Luis Herranz, Kai Wang, Yongmei Cheng, Shangling Jui, and Joost van de Weijer. Semantic drift compensation for class-incremental learning. In CVPR, pages 6982–6991, 2020.arxiv.org/abs/2004.00440
  24. [24]Friedemann Zenke, Ben Poole, and Surya Ganguli. Continual learning through synaptic intelligence. In ICML, pages 3987–3995, 2017.arxiv.org/abs/1703.04200
  25. [25]Zhen Zhao, Zhizhong Zhang, Xin Tan, et al. Rethinking gradient projection continual learning: Stability/plasticity feature space decoupling. In CVPR, pages 3718–3727, 2023.DOI
  26. [26]Fei Zhu, Xu-Yao Zhang, Chuang Wang, Fei Yin, and Cheng-Lin Liu. Prototype augmentation and self-supervision for incremental learning. In CVPR, pages 5871–5880, 2021.DOI
  27. [27]Kai Zhu, Wei Zhai, Yang Cao, Jiebo Luo, and Zheng-Jun Zha. Self-sustaining representation expansion for non-exemplar class-incremental learning. In CVPR, pages 9296–9305, 2022.arxiv.org/abs/2203.06359

Paper details

vlnc:W_3b8fkjzh
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

OpenAI’s ChatGPT, Codex

View statement
Looking for related papers…
Contents