betaFeedback

WorksComputer Science

Hidden Trajectories Don't Lie: Correcting Corruption Drift for Test-Time Adaptation

Abstract

Test-time adaptation (TTA) adapts pretrained models to distribution shifts encountered during deployment. However, most existing methods rely on the weight representations of the final layer or signals derived from the model's output probability distribution. We study hidden trajectories: the ordered sequence of intermediate representations produced across a deep neural network. We observe that clean and corrupted trajectories become more separable as corruption severity increases. We introduce Drift-TTA, which adds zero-initialized low-rank adapters to a frozen backbone and trains them to align corrupted hidden trajectories with those from cleaner inputs. The approach supports two variants: a source-dependent version trained with clean targets, and a source-free setting that learns from corrupted images at different severities. We develop an online TTA approach that freezes the backbone and adapters while updating a scalar gate to control adaptation strength. We further cast trajectory alignment as MAP inference under a Gaussian observation model and derive a KL-based transfer bound characterizing test-time trajectory misalignment in terms of offline residual error, residual dimensionality, and distribution shift between offline and test corruptions. We evaluate Drift-TTA on CIFAR-10-C, CIFAR-100-C, and ImageNet-C using supervised ViTs and CLIP models. On CIFAR-100-C at severity 5, our source-free online approach improves accuracy over no TTA by 13.3 percentage points with ViT-B/16 and 11.6 percentage points with CLIP ViT-B/16, reaching 74.9% and 47.4%, respectively. Under the same setting, prior TTA baselines yield ECEs of 18.00% for ViT and 14.70% for CLIP, while our approach reduces them to 9.53% and 5.77%, respectively. These results demonstrate that hidden trajectory alignment provides an effective signal for both offline correction and lightweight online test-time adaptation.

Introduction

Test time adaptation (TTA) adapts a pretrained model to unlabeled test data during deployment, when corruptions such as sensor noise, weather, blur, and compression artifacts shift inputs away from the clean source distribution (Hendrycks and Dietterich, 2019). Many recent methods guide adaptation using prediction-derived objectives, learned update mechanisms, or auxiliary self-supervised signals ([13]; Xiao and Snoek, 2024). Some minimize entropy or use related confidence and energy objectives to update a small set of model parameters, including the affine parameters of batch normalization layers (Wang et al., 2021; [6]), while others adapt representations or features directly ([17]; [5]). These studies demonstrate the potential of TTA to recover much of the accuracy lost on standard corruption benchmarks, often with modest computation and without source labels during test time. Entropy minimization is the primary approach in TTA, with several methods extending it through related confidence- and energy-based objectives [7, 12]. However, these objectives encourage the model to produce more confident predictions, but do not directly optimize whether those predictions are correct. As a result, an incorrect prediction can become more confident without necessarily being corrected.

Furthermore, intermediate features remain comparatively underexplored in TTA, both in how they change as corruption severity increases and in whether they can provide a complementary signal for adaptation. To examine this, we look beyond the final output or a single representation and follow how features evolve through the network. Chain-of-Embedding (Wang et al., 2025a) provides a framework for this by representing a forward pass as a trajectory of hidden states whose geometry can differ across outcomes. We extend this view to vision under corruption and examine whether clean and corrupted inputs trace distinguishable trajectories through a ViT or CLIP image tower. Across all models we test, features derived from the hidden trajectory separate clean and corrupted inputs more clearly as corruption severity increases. This observation motivates Drift-TTA, which adds zero-initialized low-rank adapters [9] to a frozen backbone and trains them so that corrupted inputs produce hidden trajectories closer to those of their clean counterparts. Figure 1 summarizes this intuition. Corruption progressively displaces the hidden trajectory as severity increases, while Drift-TTA learns a correction in the opposite direction and shifts the resulting trajectory features back toward the clean distribution.

Figure 1. Corruption progressively shifts hidden trajectories, while Drift-TTA moves them back toward clean. (a) Schematic of severity-dependent trajectory drift and the correction learned by Drift-TTA. (b) Distributions of the trajectory magnitude and angle features for clean and corrupted inputs at severities 1–5, before and after applying Drift-TTA. Each column compares clean inputs with one corruption severity. Without adaptation, the corrupted distributions move farther from clean as severity increases. With Drift-TTA, they shift back toward the clean distribution.

Our contributions are summarized as follows:

  • We introduce hidden trajectories as a vision-domain extension of the chain-of-embeddings framework previously studied in large language models: A hidden trajectory captures the ordered sequence of mean-pooled representations across backbone layers, and we show that trajectory-derived features increasingly separate clean from corrupted inputs as corruption severity grows across all tested vision models.

  • We introduce hidden-trajectory alignment as a deployment-efficient mechanism for corruption correction: Drift-TTA instantiates this idea by attaching zero-initialized low-rank adapters to a frozen ViT or CLIP image encoder and training them offline to align corrupted hidden trajectories with those of cleaner inputs. The formulation supports both source-dependent adaptation using clean targets and source-free adaptation using corrupted/further-corrupted image pairs, with the latter requiring no clean source data. Once trained, the adapters remain fixed at inference, eliminating test-time gradients and per-sample optimization. The same formulation extends directly to CLIP zero-shot classification.

  • We introduce an online variant of Drift-TTA that adapts the strength of this learned trajectory correction at test time: Drift-TTA-online keeps the backbone and trained adapters frozen and updates only one scalar per adapter branch to control its correction strength. Each scalar scales the correction vector produced by its adapter and is optimized using prediction entropy on each unlabeled target batch. The same procedure applies to both source-dependent and source-free adapters.

  • We provide a probabilistic interpretation and theoretical bounds for hidden-trajectory alignment: We interpret the trajectory-alignment objective as regularized MAP estimation under a Gaussian observation model. We then derive a KL-based transfer bound and an explicit Gaussian form relating test-time residual error to offline residual error, residual dimensionality, and offline-to-test distribution shift.

  • We establish that hidden-trajectory alignment consistently improves corruption robustness and calibration across models: On CIFAR-100-C at severity 5, source-dependent Drift-TTA-online improves mean accuracy by 16.6% and 13.0% over the unadapted ViT-B/16 and CLIP ViT-B/16, respectively, while exceeding the strongest compared TTA baselines by 2.5% and 6.2%. It also improves calibration, reducing mean ECE relative to these baselines by 9.94% for ViT and 9.69% for CLIP.

Background and Related Work

This section introduces the test time adaptation setting, reviews existing methods, and summarizes Chain-of-Embedding Wang et al. (2025a), which motivates this work.

Test Time Adaptation

Test time adaptation modifies a pretrained model using unlabeled test samples during test time.

The labels of the test samples are never available for adaptation, and many methods also assume that the data used to train the pretrained model are unavailable at test time (Wang et al., 2021).

Let fωf_{\omega} denote a model with pretrained parameters ω\omega, trained on source inputs xsx_s and labels ysy_s. At test time, the model receives inputs xtx_t without their labels yty_t. Let α\alpha denote the variables adapted at test time, and let LtL_t be an unlabeled test time objective.

Definition 2.1 (Test time adaptation). Optimization-based test time adaptation estimates

α⋆=arg⁡min⁡α Lt(fω,α(xt)),y^t=fω,α⋆(xt),(1)\alpha^{\star} = \underset{\alpha}{\arg\min}\ L_t\left(f_{\omega,\alpha}(x_t)\right), \qquad\hat{y}_t = f_{\omega,\alpha^{\star}}(x_t), \tag*{(1)}

without using yty_t. Adaptation is online when each test batch xtx_t is used as it arrives.

Existing methods differ mainly in the objective LtL_t that guides adaptation, the variables α\alpha they adapt, and how they use the test inputs xtx_t. A common approach is to use the model predictions themselves to guide adaptation. Tent Wang et al. (2021) minimizes prediction entropy while updating the affine parameters of normalization layers. ETA Niu et al. [18] builds on entropy minimization by selecting reliable test samples and regularizing important model parameters using Fisher information, while SAR Niu et al. [19] combines entropy minimization with sample filtering and sharpness-aware optimization to improve stability. For continual test streams, CMF Lee and Chang [11] further filters the information used during adaptation to reduce forgetting. Other methods use related signals from the model output, learn how the update itself should be performed, or use auxiliary tasks during adaptation [6, 7, 1]. Despite these differences, the model predictions remain the main signal used for adaptation.

Other methods use information from the model representations themselves. FOA Niu et al. [20] combines prediction entropy with feature statistics and adapts prompts and activations using derivative-free optimization. PEA Ma et al. [14] instead uses feature statistics across intermediate layers to progressively align hidden representations without backpropagation. Related work also operates on latent features or their geometry [17, 2, 12, 5, 4] (Wang et al. 2023). These methods show that intermediate representations can also provide useful signals for adaptation.

Drift-TTA: Low-Rank Alignment of Hidden Trajectories

For vision-language models such as CLIP [21], TTA has also been extended to prompts, tokens, memories, and class representations [3] (Zhang et al. 2025; Wang et al. 2025b). TPT [22] adapts the text prompt for each test image using prediction entropy. TDA [10] and DMN-ZS (Zhang et al., 2024b) instead accumulate information from test samples using feature caches or dynamic memories. BCA (Zhou et al., 2025) adapts class embeddings together with their priors, while DPE (Zhang et al., 2024a) evolves visual and textual class prototypes using entropy and cross-modal alignment. More recently, BAT [16] adapts the LayerNorm parameters of both the vision and text encoders using entropy and alignment, while BITTA [23] additionally uses high-entropy samples for unlearning. Other work also considers how adaptation is applied across layers or maintained over longer test streams [15] (Vray et al. 2025).

These methods are commonly evaluated on the -C benchmarks (Hendrycks and Dietterich, 2019), including CIFAR-10-C, CIFAR-100-C, and ImageNet-C, which apply fifteen corruption families at five severity levels. Table 1 compares representative methods by their adaptation signal, updated model components, and test-time setting.

MethodAdaptation signalModules updatedAccessGranularityPersistenceMulti-layer
TENT (Wang et al., 2021)EntropyNorm. affineOnlineBatchContinual
SAR [19]Entropy + sharpnessNorm. affineOnlineBatchContinual
EATA [18]Entropy + FisherNorm. affineOnlineBatchContinual
FOA [20]Entropy + feature statisticsPrompt + activationsOnlineBatchContinual✓
CMF [11]Entropy + filteringModel parametersOnlineBatchContinual
PEA [14]Feature statisticsHidden featuresOnlineBatchContinual✓
TPT [22]EntropyText promptOnlineInstanceEpisodic
TDA [10]Entropy + pseudo-labelsFeature cacheOnlineInstanceContinual
BCA (Zhou et al., 2025)Likelihood + priorClass embeddings + priorsOnlineInstanceContinual
DMN-ZS (Zhang et al., 2024b)Entropy + feature memoryDynamic memoryOnlineInstanceContinual
DPE (Zhang et al., 2024a)Entropy + alignmentClass prototypesOnlineInstanceContinual
BAT [16]Entropy + alignmentVision/text LayerNormOnlineBatchContinual
BITTA [23]Entropy + unlearningVision/text LayerNormOnlineBatchContinual
Drift-TTA (ours)Hidden trajectory alignmentLoRA adaptersOfflineInstance–✓
Drift-TTA-online (ours)Hidden trajectory alignmentScalar gatesOnlineInstance / BatchEpisodic✓

Table 1. Comparison of representative TTA methods. “Adaptation signal” denotes the information used to guide adaptation, while “Modules updated” denotes what is changed during adaptation. We describe the test-time setting along three axes: “Access” indicates whether adaptation uses test samples as they arrive (online) or after they are available together (offline). “Granularity” indicates whether the method can adapt using a single test sample (instance), a group of test samples (batch), or both. “Persistence” indicates whether test-time changes are reset after adaptation (episodic) or kept for later test samples (continual). “Multi-layer” indicates whether representations from multiple network layers are used during adaptation.

Table 1 also shows how methods differ in their use of test samples. Online, instance-level, episodic adaptation processes each sample as it arrives and resets before the next, making it more constrained than offline, batch-level, or continual settings. Drift-TTA-online supports this setting and can also operate on batches when multiple samples are available.

Overall, existing TTA methods use a range of adaptation signals, including model outputs, feature statistics, memories, pseudo-labels, cross-modal alignment, and intermediate representations.

Chain-of-Embedding. Chain-of-Embedding (CoE) (Wang et al., 2025a) studies how hidden representations evolve across layers in large language models. It shows that geometric summaries of this sequence can separate correct from incorrect generations. We transfer this view from language model outputs to vision models under corrupted inputs. Specifically, we compare clean and corrupted versions of the same input across network depth to study how corruption changes the learned representations.

ParadigmClean imagesClean labelsUnlabeled corrupted imagesBefore testingAt test time
Fine tuning✓✓✓
Unsupervised DA✓✓✓✓
Test time training✓✓✓
Online TTA (e.g. TENT)✓✓
Drift-TTA, source dependent✓✓✓
Drift-TTA, source free✓✓
Drift-TTA-online, source dependent✓✓✓✓
Drift-TTA-online, source free✓✓✓

Table 2. Comparison of adaptation paradigms. The table compares the data required beyond the pretrained model and when adaptation is performed. For Drift-TTA, source dependent and source free denote two training settings: source dependent uses clean images as trajectory targets for corrupted inputs, while source free uses corrupted and further corrupted images without clean images or labels. After training, either setting can be used directly with the learned adapters fixed or with Drift-TTA-online, which additionally updates only scalar gates at test time.

Drift-TTA: Low-Rank Alignment of Hidden Trajectories

Drift-TTA operates on the sequence of hidden representations produced across network depth. For an input xx, let zlz_l denote the spatial representation after block ll of a backbone with LL blocks. We define the hidden trajectory as

H(x)=(h1,…,hL),hl=MeanPool⁡(zl)∈Rd,(2)\mathcal{H}(x) = (h_1,\ldots,h_L), \qquad h_l = \operatorname{MeanPool}(z_l) \in\mathbb{R}^{d}, \tag*{(2)}

where mean pooling excludes the class or prefix token for transformer backbones.

For analysis, we include the input embedding h0h_0 and define

Mag⁡(H)=1L∑l=0L−1∥hl+1−hl∥2∥hL−h0∥2,Ang⁡(H)=1L∑l=0L−1∠(hl+1,hl)∠(hL,h0),(3)\operatorname{Mag}(\mathcal{H}) = \frac{\frac{1}{L}\sum_{l=0}^{L-1}\lVert h_{l+1}-h_l\rVert_2}{\lVert h_L-h_0\rVert_2}, \qquad\operatorname{Ang}(\mathcal{H}) = \frac{\frac{1}{L}\sum_{l=0}^{L-1}\angle(h_{l+1},h_l)}{\angle(h_L,h_0)}, \tag*{(3)}

where ∠(a,b)=arccos⁡(aTb∥a∥2∥b∥2)\angle(\mathbf{a},\mathbf{b})=\arccos\left(\frac{\mathbf{a}^{\mathsf{T}}\mathbf{b}}{\lVert\mathbf{a}\rVert_2\lVert\mathbf{b}\rVert_2}\right). These features are used only to analyze and visualize how corruption changes intermediate representations across network depth; training directly aligns the hidden states hlh_l.

Drift-TTA adds zero-initialized low-rank adapters within the frozen backbone and trains them to move corrupted hidden trajectories toward those produced by cleaner inputs. The same trajectory-alignment objective is used for source-dependent and source-free training. Table 2 summarizes these settings, and Figure 2 illustrates the method.

Low-Rank Trajectory Adapters

Each adapter is a trainable residual branch on a linear map of the frozen backbone. Let S⊆{1,…,L}\mathcal{S}\subseteq\{1,\ldots,L\} denote the adapted blocks and Ll\mathcal{L}_l the selected linear maps in block ll, so that (S,{Ll}l∈S)(\mathcal{S},\{\mathcal{L}_l\}_{l\in\mathcal{S}}) defines the adapter placement. For transformer backbones, Ll\mathcal{L}_l contains token-wise linear maps within the attention and MLP sublayers. For CLIP, the same procedure is applied only to the image encoder, while the text encoder and class prototypes remain frozen. The qq-th map in block ll computes yl,q=Wl,qxl,q+bl,qy_{l,q}=\mathbf{W}_{l,q}x_{l,q}+b_{l,q}, with weights Wl,q∈Rdout×din\mathbf{W}_{l,q}\in\mathbb{R}^{d_{\mathrm{out}}\times d_{\mathrm{in}}}, input xl,q∈Rdinx_{l,q}\in\mathbb{R}^{d_{\mathrm{in}}}, and bias bl,q∈Rdoutb_{l,q}\in\mathbb{R}^{d_{\mathrm{out}}}. Following LoRA [9], a rank-rr adapter adds the correction

Δl,q(xl,q)=αl,qAl,qBl,qxl,q,Bl,q∈Rr×din,Al,q∈Rdout×r,(4)\Delta_{l,q}(x_{l,q})=\alpha_{l,q}\mathbf{A}_{l,q}\mathbf{B}_{l,q}x_{l,q}, \qquad\mathbf{B}_{l,q}\in\mathbb{R}^{r\times d_{\mathrm{in}}}, \qquad\mathbf{A}_{l,q}\in\mathbb{R}^{d_{\mathrm{out}}\times r}, \tag*{(4)}

where Al,q\mathbf{A}_{l,q} and Bl,q\mathbf{B}_{l,q} are trainable low-rank weights and the scalar gate αl,q\alpha_{l,q} sets the strength of the correction. The adapted output is y~l,q=yl,q+Δl,q(xl,q)\widetilde{y}_{l,q}=y_{l,q}+\Delta_{l,q}(x_{l,q}), where the effective correction Al,qBl,q\mathbf{A}_{l,q}\mathbf{B}_{l,q} has rank at most rr and Wl,q\mathbf{W}_{l,q} and bl,qb_{l,q} remain fixed. We initialize Al,q=0\mathbf{A}_{l,q}=0 and Bl,q\mathbf{B}_{l,q} with small random values, so the adapted model initially produces the same computation as the frozen backbone. Only the adapter weights receive gradients during offline training, with αl,q=1\alpha_{l,q}=1. Once trained, these weights remain fixed and are used either directly with unit strength or with gates adapted online (Section 3.3).

Figure 2. Architecture of Drift-TTA. A frozen pretrained backbone is augmented with zero-initialized low-rank adapters attached to selected linear maps. The mean-pooled representations h1,…,hL\mathbf{h}_1,\ldots,\mathbf{h}_L produced across the backbone form the hidden trajectory. The learned adapters move corrupted trajectories toward their clean counterparts while keeping the pretrained backbone frozen. After training, the adapters can be used directly with fixed weights. For online adaptation, Drift-TTA-online keeps the backbone and adapter weights frozen and updates only the strength of each learned correction.

Trajectory Alignment Objective

Each training example is a triplet of a student input x\mathbf{x}, which may be augmented or corrupted, a cleaner teacher view xtgt\mathbf{x}^{\mathrm{tgt}}, and an anchor view xanc\mathbf{x}^{\mathrm{anc}}. The adapted backbone processes x\mathbf{x}, while the frozen backbone processes xtgt\mathbf{x}^{\mathrm{tgt}} to provide the reference trajectory and decision. Let hlθ(x)\mathbf{h}_l^\theta(\mathbf{x}) and hl0(xtgt)\mathbf{h}_l^0(\mathbf{x}^{\mathrm{tgt}}) denote the mean-pooled hidden states of the adapted and frozen models after block ll, where θ\theta contains all adapter parameters. Their decisions oθ(x)\mathbf{o}^\theta(\mathbf{x}) and o0(xtgt)\mathbf{o}^0(\mathbf{x}^{\mathrm{tgt}}) are classifier logits for supervised ViTs. Since CLIP has no task-specific classification head, its decision is instead the vector of zero-shot image–text similarity scores. We omit the learned temperature from these scores so that their scale does not dominate the trajectory terms, which leaves the predicted class unchanged. With the aligned blocks T\mathcal{T} running from the first adapted block to block LL, the alignment loss matches both the trajectory and the decision:

Lalign(θ)=∑l∈Twl∥hlθ(x)−sg⁡[hl0(xtgt)]∥22+λdec∥oθ(x)−sg⁡[o0(xtgt)]∥22.(5)\mathcal{L}_{\mathrm{align}}(\theta)=\sum_{l\in\mathcal{T}}w_l\left\lVert\mathbf{h}_l^\theta(\mathbf{x})-\operatorname{sg}\left[\mathbf{h}_l^0(\mathbf{x}^{\mathrm{tgt}})\right]\right\rVert_2^2+\lambda_{\mathrm{dec}}\left\lVert\mathbf{o}^\theta(\mathbf{x})-\operatorname{sg}\left[\mathbf{o}^0(\mathbf{x}^{\mathrm{tgt}})\right]\right\rVert_2^2. \tag*{(5)}

where sg⁡[⋅]\operatorname{sg}[\cdot] stops gradients through the frozen teacher. The effective low-rank corrections are regularized by

Lctrl(θ)=∑l∈S∑q∈Ll∥Al,qBl,q∥F2.(6)\mathcal{L}_{\mathrm{ctrl}}(\theta)=\sum_{l\in\mathcal{S}}\sum_{q\in\mathcal{L}_l}\left\lVert A_{l,q}B_{l,q}\right\rVert_F^2. \tag*{(6)}

Because the adapters remain active for every input, the quietness loss uses the anchor view to limit unnecessary changes to inputs that do not require further correction:

Lquiet(θ)=∑l∈Twl∥hlθ(xanc)−sg⁡[hl0(xanc)]∥22+λdec∥oθ(xanc)−sg⁡[o0(xanc)]∥22.(7)\mathcal{L}_{\mathrm{quiet}}(\theta)=\sum_{l\in\mathcal{T}}w_l\left\lVert\mathbf{h}_l^\theta(\mathbf{x}^{\mathrm{anc}})-\operatorname{sg}\left[\mathbf{h}_l^0(\mathbf{x}^{\mathrm{anc}})\right]\right\rVert_2^2+\lambda_{\mathrm{dec}}\left\lVert\mathbf{o}^\theta(\mathbf{x}^{\mathrm{anc}})-\operatorname{sg}\left[\mathbf{o}^0(\mathbf{x}^{\mathrm{anc}})\right]\right\rVert_2^2. \tag*{(7)}

The complete training objective is

L(θ)=Lalign(θ)+λctrlLctrl(θ)+λquietLquiet(θ).(8)\mathcal{L}(\theta)=\mathcal{L}_{\mathrm{align}}(\theta)+\lambda_{\mathrm{ctrl}}\mathcal{L}_{\mathrm{ctrl}}(\theta)+\lambda_{\mathrm{quiet}}\mathcal{L}_{\mathrm{quiet}}(\theta). \tag*{(8)}

This objective is shared by two training settings, which differ only in how the student, teacher, and anchor views are constructed.

Source-dependent training. Source-dependent training starts from a clean source image xclean\mathbf{x}^{\mathrm{clean}} and corrupts it with a corruption operator C\mathcal{C}, giving the student input x=C(xclean)\mathbf{x}=\mathcal{C}(\mathbf{x}^{\mathrm{clean}}). The clean image serves as both the teacher view and the anchor, xtgt=xanc=xclean\mathbf{x}^{\mathrm{tgt}}=\mathbf{x}^{\mathrm{anc}}=\mathbf{x}^{\mathrm{clean}}. The adapters therefore learn to move the corrupted trajectory back toward the hidden trajectory and decision of the clean forward pass.

Source-free training. When clean images are unavailable, source-free training starts from a corrupted image xcurx^{\mathrm{cur}} and corrupts it further, giving the student input x=x+=Worsen⁡(xcur)x=x^{+}=\operatorname{Worsen}(x^{\mathrm{cur}}) with the same corruption family at a higher effective severity. The current image then serves as both the cleaner teacher view and the anchor, xtgt=xanc=xcurx^{\mathrm{tgt}}=x^{\mathrm{anc}}=x^{\mathrm{cur}}, so no clean images or class labels are required. The adapters thus learn to move the trajectory of x+x^{+} back toward that of xcurx^{\mathrm{cur}}, although this target trajectory remains corrupted. At deployment, the learned correction is applied to the current corrupted input and partially reverses the same direction of displacement. For stochastic corruptions, the worsening operation preserves the underlying corruption realization whenever possible, since an independent draw can change the displacement direction and weaken the teacher correspondence.

Online Test-Time Adaptation

Adapters trained in either setting apply each learned correction with fixed strength, αl,q=1\alpha_{l,q}=1. Drift-TTA-online instead adapts the gates αl,q\alpha_{l,q} in equation (4) at test time, while the backbone and adapter weights remain frozen. Collecting these gates as α={αl,q}\boldsymbol{\alpha}=\{\alpha_{l,q}\} gives the test-time variables in Definition 2.1. Setting αl,q=0\alpha_{l,q}=0 disables a correction. This adds one online parameter per adapter branch: 36 gates for the supervised ViTs and 24 for CLIP in our 12-block models. For each unlabeled test batch Bt\mathcal{B}_t, the gates are initialized to one and updated by minimizing the prediction entropy

Lonline(α;Bt)=−1∣Bt∣∑x∈Bt∑cpα,c(x)log⁡pα,c(x),(9)\mathcal{L}_{\mathrm{online}}(\boldsymbol{\alpha};\mathcal{B}_t)=-\frac{1}{|\mathcal{B}_t|}\sum_{x\in\mathcal{B}_t}\sum_c p_{\boldsymbol{\alpha},c}(x)\log p_{\boldsymbol{\alpha},c}(x), \tag*{(9)}

where pα(x)=softmax⁡(sα(x))\mathbf{p}_{\boldsymbol{\alpha}}(x)=\operatorname{softmax}(\mathbf{s}_{\boldsymbol{\alpha}}(x)), cc indexes the output classes, and the prediction scores sα\mathbf{s}_{\boldsymbol{\alpha}} are the classifier logits for supervised ViTs and the temperature-scaled image–text similarity scores for CLIP. After KK gradient updates, we predict the same batch with the calibrated gates and reset them before processing the next batch, which also supports per-image adaptation when ∣Bt∣=1|\mathcal{B}_t|=1.

Probabilistic Interpretation and Theoretical Bounds

Alignment as regularized Gaussian MAP. For a student–teacher–anchor triplet z=(x,xtgt,xanc)z=(x,x^{\mathrm{tgt}},x^{\mathrm{anc}}), we can decompose equation (8) into the following:

ℓθ(z)=Lalign(θ;z)+λquietLquiet(θ;z)+λctrlLctrl(θ).(10)\ell_{\theta}(z)=\mathcal{L}_{\mathrm{align}}(\theta;z)+\lambda_{\mathrm{quiet}}\mathcal{L}_{\mathrm{quiet}}(\theta;z)+\lambda_{\mathrm{ctrl}}\mathcal{L}_{\mathrm{ctrl}}(\theta). \tag*{(10)}

If we stack the frozen teacher and anchor readings into tt, and the corresponding adapted outputs into fθ(v)f_{\theta}(v), where v=(x,xanc)v=(x,x^{\mathrm{anc}}), then the data-fit term can be written as ℓdata,θ(z)=∥t−fθ(v)∥22\ell_{\mathrm{data},\theta}(z)=\lVert t-f_{\theta}(v)\rVert_{2}^{2}. Let p(t∣v,θ)=N(t;fθ(v),12I)p(t\mid v,\theta)=\mathcal{N}\left(t;f_{\theta}(v),\frac{1}{2}I\right), and place the regularizing prior π(θ)∝exp⁡[−λctrlLctrl(θ)]\pi(\theta)\propto\exp[-\lambda_{\mathrm{ctrl}}\mathcal{L}_{\mathrm{ctrl}}(\theta)], assuming the prior is proper on the parameter domain. Then, by Bayes’ rule:

arg⁡min⁡θ ℓθ(z)=arg⁡max⁡θ p(θ∣t,v).(11)\underset{\theta}{\arg\min}\ \ell_{\theta}(z)=\underset{\theta}{\arg\max}\ p(\theta\mid t,v). \tag*{(11)}

KL variational bound for the test-time residual. For a learned parameter vector θ\theta, let rθ(z)=t−fθ(v)∈Rdtr_{\theta}(z)=t-f_{\theta}(v)\in\mathbb{R}^{d_t} be its weighted residual, and let PθP_{\theta} and QθQ_{\theta} be the residual distributions induced by offline and test examples, respectively. We define:

Roff(θ)=EPθ∥r∥2,Rte(θ)=EQθ∥r∥2,κθ=KL⁡(Qθ∥Pθ).(12)R_{\mathrm{off}}(\theta)=\mathbb{E}_{P_{\theta}}\lVert r\rVert^{2},\qquad R_{\mathrm{te}}(\theta)=\mathbb{E}_{Q_{\theta}}\lVert r\rVert^{2},\qquad\kappa_{\theta}=\operatorname{KL}(Q_{\theta}\Vert P_{\theta}). \tag*{(12)}

We further set η>0\eta>0 with finite Zη=EPθeη∥r∥2Z_{\eta}=\mathbb{E}_{P_{\theta}}e^{\eta\lVert r\rVert^{2}}, let the tilted distribution dPθ,η=eη∥r∥2 dPθ/ZηdP_{\theta,\eta}=e^{\eta\lVert r\rVert^{2}}\,dP_{\theta}/Z_{\eta}. Using κθ−ηRte(θ)+log⁡Zη=KL⁡(Qθ∥Pθ,η)≥0\kappa_{\theta}-\eta R_{\mathrm{te}}(\theta)+\log Z_{\eta}=\operatorname{KL}(Q_{\theta}\Vert P_{\theta,\eta})\geq0, we obtain the following KL change-of-measure bound:

Rte(θ)≤Roff(θ)+inf⁡η>0κθ+log⁡EPθeη(∥r∥2−Roff(θ))η.(13)R_{\mathrm{te}}(\theta)\leq R_{\mathrm{off}}(\theta)+\inf_{\eta>0}\frac{\kappa_{\theta}+\log\mathbb{E}_{P_{\theta}}e^{\eta(\lVert r\rVert^{2}-R_{\mathrm{off}}(\theta))}}{\eta}. \tag*{(13)}

Convert into an explicit Gaussian bound. If we further assume the actual offline residual law is Gaussian Pθ=N(0,σ2I)P_{\theta}=\mathcal{N}(0,\sigma^{2}I), with σ2>0\sigma^{2}>0, substituting this into equation (13) and choosing η=κθ/[σ2(dt+2κθ)]\eta=\sqrt{\kappa_{\theta}}/[\sigma^{2}(\sqrt{d_t}+2\sqrt{\kappa_{\theta}})] gives

Rte(θ)≤Roff(θ)+2σ2(dtκθ+κθ).(14)R_{\mathrm{te}}(\theta)\leq R_{\mathrm{off}}(\theta)+2\sigma^{2}\left(\sqrt{d_t\kappa_{\theta}}+\kappa_{\theta}\right). \tag*{(14)}

An intuitive explanation for equation (14) is that the test-time residual is bounded by the offline residual, the residual dimensionality, and the KL divergence between the offline and test residual distributions. Note that these bounds apply to fixed adapters under additional residual-distribution assumptions; they do not guarantee clean-trajectory recovery, online adaptation gains, or improved accuracy. Detailed proof is given in Appendix § 11.

Experiments

Drift-TTA is applied to both unimodal classifiers and CLIP-based classification models. In this section, we report comprehensive validation for our approach. The experimental setting is described in § 5.1. Main results are presented in § 5.2.

Experimental Setup

Datasets and models. We evaluate Drift-TTA on CIFAR-10-C, CIFAR-100-C, and ImageNet-C ((Hendrycks and Dietterich, 2019)), each containing 15 corruption types at five severity levels. For unimodal classification, ViT-Tiny is used on CIFAR-10-C, while ViT-B/16 is used on CIFAR-100-C and ImageNet-C. Each model is pretrained on the corresponding clean dataset. For CLIP-based classification, we use a pretrained CLIP ViT-B/16 model. Both source-dependent and source-free variants of Drift-TTA are evaluated.

Training Protocol. Drift-TTA adds a lightweight LoRA adapter (rank 4, on all transformer blocks) to a frozen backbone. For each corruption family we train one adapter on the other 14 families (all five severities) and evaluate on the held-out family; Table 3 reports this leave-one-family-out accuracy. The adapter weights are trained offline and remain frozen at test time. Drift-TTA-online updates only their scalar gates using prediction entropy on the current unlabeled target batch. Source-dependent is trained for 4000 steps, while source-free adapters are trained for 2400 steps. A held-out validation split is used only to monitor convergence. For supervised ViT runs, the final checkpoint is reported. For CLIP, we select among epochs 1,3,5,7,9 using unsupervised validation loss; no target labels are used for model selection..

MethodGaussianShotImpulseDefocusGlassMotionZoomSnowFrostFogBrightnessContrastElasticPixelateJPEGMean
CIFAR-100-CViT-B/1642.442.454.173.147.572.875.080.579.757.186.044.064.547.757.561.6
TENT42.643.454.873.447.273.075.280.379.554.386.039.864.446.857.261.2
SAR42.944.554.073.347.673.074.979.579.153.685.446.363.946.955.461.3
EATA42.543.854.573.247.973.075.080.179.755.685.742.065.148.957.261.6
FOA43.549.055.374.355.375.878.581.081.976.087.276.270.155.660.768.0
CMF56.565.366.077.061.176.479.581.081.777.886.182.969.668.765.773.0
PEA62.164.871.581.265.980.082.682.783.682.187.486.470.770.864.475.7
Drift-TTA-online (SF)73.672.245.383.470.281.684.982.984.077.987.181.471.963.763.874.9
Drift-TTA-online (SD)77.378.575.084.571.981.685.183.784.875.285.675.175.671.966.978.2
CLIPViT-B/1619.621.425.342.520.143.248.048.449.741.757.034.429.223.932.535.8
TPT17.219.025.442.720.042.748.349.250.242.557.838.030.225.232.536.1
TDA23.424.332.642.320.242.046.948.548.341.457.032.033.033.433.437.2
BCA12.513.525.939.919.240.545.347.946.840.755.827.131.631.631.334.0
DMN-ZS19.822.029.842.518.343.448.349.151.143.956.735.831.826.531.336.7
DPE22.724.230.045.522.244.850.750.050.843.659.636.631.527.934.238.3
BAT21.425.229.950.425.748.254.952.150.748.263.543.334.831.837.241.2
BITTA26.728.735.250.726.149.055.252.651.448.763.746.635.330.837.842.6
Drift-TTA-online (SF)38.738.336.953.539.353.259.155.558.250.262.252.341.633.738.547.4
Drift-TTA-online (SD)45.347.842.954.339.153.258.057.359.249.061.748.243.730.441.648.8

Table 3. Test-time adaptation accuracy (%) on CIFAR-100-C at severity 5. Results are reported across all fifteen corruption types for supervised ViT-B/16 and CLIP ViT-B/16. Drift-TTA-online combines a frozen trajectory-repair adapter with online entropy-based gate calibration under source-dependent (SD) and source-free (SF) training. The best result in each column is shown in green and the second best in blue.

Evaluation. Each adapted model is compared with the corresponding model without adaptation on the same corrupted images. Source-dependent experiments use 2,000 CIFAR and 5,000 ImageNet images, while source-free experiments use 10,000 CIFAR and 50,000 ImageNet images. CLIP experiments use 10,000 CIFAR and 5,000 ImageNet images. We primarily report the change in accuracy relative to no adaptation, denoted by Δ\Deltapp. Recovery and changes in clean accuracy are also reported when applicable. All experiments use a single seed, with an expected run-to-run variation of approximately 0.3%.

Main Results

Calibration. Accuracy alone does not show whether a model’s confidence remains calibrated after adaptation. Table 4 reports mean expected calibration error (ECE) for the unadapted models, PEA and BITTA, and the fixed and online variants of Drift-TTA. For both source dependent and source free training, the better Drift-TTA variant has lower mean ECE than the corresponding prior TTA baseline across all model–dataset settings. Appendix § 14 provides the metric definition, evaluation protocol, and complete per-corruption results.

Method / deploymentViTCLIP
CIFAR-10-CCIFAR-100-CImageNet-CCIFAR-10-CCIFAR-100-CImageNet-C
No-TTA13.5110.1011.544.876.304.12
PEA (ViT) / BITTA (CLIP)11.4418.0033.4413.5814.7018.04
Adapter only (SD)11.728.4717.823.3310.334.51
+ Online gate (SD)8.218.0613.033.535.0112.89
ΔSD\Delta_{\mathrm{SD}}↓3.23\downarrow3.23↓9.94\downarrow9.94↓20.41\downarrow20.41↓10.25\downarrow10.25↓9.69\downarrow9.69↓13.53\downarrow13.53
Adapter only (SF)14.509.9617.433.869.964.51
+ Online gate (SF)10.719.5314.722.285.778.01
ΔSF\Delta_{\mathrm{SF}}↓0.73\downarrow0.73↓8.47\downarrow8.47↓18.72\downarrow18.72↓11.30\downarrow11.30↓8.93\downarrow8.93↓14.90\downarrow14.90

Table 4. Mean expected calibration error (ECE, %, ↓\downarrow) on the corruption benchmarks at severity 5. PEA and BITTA are the ViT and CLIP baselines, respectively. “Adapter only” uses the frozen Drift-TTA adapter; “+ Online” adds entropy-based gate calibration. For each setting, ΔSD\Delta_{\mathrm{SD}} and ΔSF\Delta_{\mathrm{SF}} select the lower ECE of these two variants and report its reduction from the corresponding baseline in percentage. Green ↓\downarrow values indicate lower ECE. Teal, orange, purple, and gray denote no adaptation, source dependent (SD), source free (SF), and the prior TTA baselines.

Accuracy. Table 3 compares Drift-TTA with representative TTA baselines on CIFAR-100-C at severity 5. For ViT-B/16, source-dependent Drift-TTA-online reaches 78.2% mean accuracy, compared with 61.6% without adaptation and 75.7% for the strongest compared baseline, PEA. For CLIP ViT-B/16, it reaches 48.8%, compared with 35.8% without adaptation and 42.6% for BITTA. Source-free Drift-TTA-online reaches 74.9% and 47.4%, respectively. Full CIFAR-10-C and ImageNet-C results are provided in Appendix § 12.

Representation structure. Figure 3 provides a qualitative view of how Drift-TTA changes the CLIP image representation under Frost corruption. Without adaptation, samples from several classes overlap in the projected space. The fixed source dependent and source free adapters produce more distinct class groups, and online gate calibration further improves this separation. The clearest structure is obtained with the source dependent adapter and online TTA.

Figure 3. UMAP of CLIP ViT-B/16 image embeddings on Frost-corrupted CIFAR-10-C at severity 5. Columns show no adaptation, source-free Drift-TTA without and with online TTA, and source-dependent Drift-TTA without and with online TTA. Points are colored by their ground-truth class. Each Drift-TTA adapter is trained without Frost under the leave-one-family-out protocol, and each panel is projected independently using UMAP with Euclidean distance.

Conclusion

Conclusion

In this work, we studied hidden trajectories of neural networks for test-time adaptation under distribution shift. We observed that clean and corrupted trajectories become more separable as corruption severity increases, motivating Drift-TTA. Our approach uses low-rank adapters to correct trajectory drift while keeping the pretrained backbone frozen. Drift-TTA supports both source-dependent and source-free training, and can be used either as a fixed correction or with lightweight online adaptation using unlabeled test data. We also provided a probabilistic interpretation of trajectory alignment and theoretical bounds for offline generalization and source-free correction. Experiments on CIFAR-10-C, CIFAR-100-C, and ImageNet-C with supervised ViTs and CLIP demonstrate improved robustness to corruption, while reducing expected calibration error (ECE) relative to prior TTA baselines. Overall, these results highlight hidden trajectory alignment as an effective signal for test-time adaptation.

Acknowledgments

We sincerely thank our colleagues Hesam Salehipour and Mehran Ebrahimi for their thoughtful feedback, valuable discussions, and insightful suggestions, which helped improve this work.

AI Use Statement

We used generative AI tools to improve the clarity and readability of the manuscript, assist in writing mathematical proofs, and summarize existing literature. All AI-assisted content was reviewed and verified by the authors. The authors take full responsibility for the final content, including the mathematical arguments, claims, and references.

Method Details

Adapter Implementation

Each adapter is realized as a forward hook on a target linear layer, so no model code is modified. The hook computes the low-rank correction AlBlxA_lB_lx and adds it to the layer’s output, which already contains Wlx+blW_lx + b_l. A single flag enables or disables all hooks at once, which makes the paired “adapters-on versus adapters-off” evaluation trivial. The up-projection AlA_l is zero-initialized and the down-projection BlB_l is drawn from N(0,10−3)\mathcal{N}(0, 10^{-3}), so the branch is an exact no-op at initialization; only AlA_l, BlB_l receive gradients. Each trained set of adapter factors is stored as a single checkpoint per variant. We reiterate that AlBlA_lB_l is a learned side branch and not a factorization of WlW_l: no singular-value decomposition or low-rank approximation of the pretrained weights is ever computed. In the supervised ViT the adapted layers are the two MLP projections and the attention output projection of each block (mlp.fc1, mlp.fc2, attn.proj), excluding the fused qkv projection; in the CLIP image tower they are the two MLP projections of each residual attention block (mlp.c_fc, mlp.c_proj).

Training Details

Adapters are trained with Adam at learning rate 10−310^{-3} and batch size 64 (32 on ImageNet). The source budget is 2,000 images for CIFAR and 3,000 for ImageNet (about three per class). The objective uses uniform trajectory weights wl=1w_l=1 over the aligned blocks, a decision weight λdec=1.0\lambda_{\mathrm{dec}}=1.0, a control weight λctrl=10−3\lambda_{\mathrm{ctrl}}=10^{-3}, and a quietness weight λquiet=0.05\lambda_{\mathrm{quiet}}=0.05. Supervised runs use a fixed number of epochs (five for CIFAR, two for ImageNet); the CLIP runs train to Emax⁡=9E_{\max}=9 and select the checkpoint on the grid {1,3,5,7,9}\{1,3,5,7,9\} by label-free validation loss.

Probabilistic View and Generalization Account

Weighted Gaussian Likelihood and MAP

Let T\mathcal{T} be the aligned blocks. Write hl0h_l^0 and o0o^0 for the frozen hidden states and decisions, and hlθh_l^\theta and oθo^\theta for their adapted counterparts. The nonnegative weights wlw_l, λdec\lambda_{\mathrm{dec}}, λquiet\lambda_{\mathrm{quiet}}, and λctrl\lambda_{\mathrm{ctrl}} are fixed. Define col to concatenate vector blocks and set

t=col⁡({wlhl0(xtgt)}l∈T,λdeco0(xtgt),{λquietwlhl0(xanc)}l∈T,λquietλdeco0(xanc)),fθ(v)=col⁡({wlhlθ(x)}l∈T,λdecoθ(x),{λquietwlhlθ(xanc)}l∈T,λquietλdecoθ(xanc)).(15)\begin{aligned} t={}&\operatorname{col}\left(\left\{\sqrt{w_l}h_l^0(x^{\mathrm{tgt}})\right\}_{l\in\mathcal{T}},\sqrt{\lambda_{\mathrm{dec}}}o^0(x^{\mathrm{tgt}}),\right.\\ &\left.\left\{\sqrt{\lambda_{\mathrm{quiet}}w_l}h_l^0(x^{\mathrm{anc}})\right\}_{l\in\mathcal{T}},\sqrt{\lambda_{\mathrm{quiet}}\lambda_{\mathrm{dec}}}o^0(x^{\mathrm{anc}})\right),\\ f_\theta(v)={}&\operatorname{col}\left(\left\{\sqrt{w_l}h_l^\theta(x)\right\}_{l\in\mathcal{T}},\sqrt{\lambda_{\mathrm{dec}}}o^\theta(x),\right.\\ &\left.\left\{\sqrt{\lambda_{\mathrm{quiet}}w_l}h_l^\theta(x^{\mathrm{anc}})\right\}_{l\in\mathcal{T}},\sqrt{\lambda_{\mathrm{quiet}}\lambda_{\mathrm{dec}}}o^\theta(x^{\mathrm{anc}})\right). \tag*{(15)} \end{aligned}

Omit zero-weight blocks and let dt≥1d_t\geq1 be the resulting dimension. Teacher and anchor readings are fixed observations during optimization. Let Ω(θ)=∑l∈S∑q∈Ll∥Al,qBl,q∥F2\Omega(\theta)=\sum_{l\in\mathcal{S}}\sum_{q\in\mathcal{L}_l}\lVert A_{l,q}B_{l,q}\rVert_F^2 be the actual control penalty in equation (6).

Lemma B.1 (Gaussian likelihood and regularized MAP). The construction equation 15 satisfies ∥t−fθ(v)∥22=ℓdata,θ(z)\lVert t-f_\theta(v)\rVert_2^2=\ell_{\mathrm{data},\theta}(z). Suppose Θ⊂Rdθ\Theta\subset\mathbb{R}^{d_\theta} is compact with positive Lebesgue volume and all outputs and Ω\Omega are continuous on Θ\Theta, where dθd_\theta is the number of parameter coordinates. For fixed sample size N≥1N\geq1, the prior density

πN(θ)=1Θ(θ)e−NλctrlΩ(θ)AN,AN=∫Θe−NλctrlΩ(u) du(16)\pi_N(\theta)=\frac{\mathbf{1}_{\Theta}(\theta)e^{-N\lambda_{\mathrm{ctrl}}\Omega(\theta)}}{A_N},\qquad A_N=\int_{\Theta}e^{-N\lambda_{\mathrm{ctrl}}\Omega(u)}\,du \tag*{(16)}

is proper. Under the product observation likelihood for DN=(z1,…,zN)D_N=(z_1,\ldots,z_N), its MAP solutions minimize J^N(θ)=N−1∑iℓdata,θ(zi)+λctrlΩ(θ)\widehat{J}_N(\theta)=N^{-1}\sum_i\ell_{\mathrm{data},\theta}(z_i)+\lambda_{\mathrm{ctrl}}\Omega(\theta).

Proof. Squaring the concatenated residual and summing the block norms gives exactly the terms of Lalign+λquietLquiet\mathcal{L}_{\mathrm{align}}+\lambda_{\mathrm{quiet}}\mathcal{L}_{\mathrm{quiet}}. The chosen Gaussian density is

p(t∣v,θ)=π−dt/2exp⁡[−∥t−fθ(v)∥2].p(t\mid v,\theta)=\pi^{-d_t/2}\exp[-\lVert t-f_\theta(v)\rVert^2].

Continuity and compactness imply 0≤Ω(θ)≤MΩ<∞0\leq\Omega(\theta)\leq M_\Omega<\infty. Consequently 0<vol⁡(Θ)e−NλctrlMΩ≤AN≤vol⁡(Θ)<∞0<\operatorname{vol}(\Theta)e^{-N\lambda_{\mathrm{ctrl}}M_\Omega}\leq A_N\leq\operatorname{vol}(\Theta)<\infty. Let TN=(t1,…,tN)T_N=(t_1,\ldots,t_N) and VN=(v1,…,vN)V_N=(v_1,\ldots,v_N). Bayes’ rule gives, on Θ\Theta,

−log⁡p(θ∣TN,VN)=∑i=1N[−log⁡p(ti∣vi,θ)]−log⁡πN(θ)+log⁡p(TN∣VN)=NJ^N(θ)+CN,(17)\begin{aligned} -\log p(\theta\mid T_N,V_N) &=\sum_{i=1}^{N}\left[-\log p(t_i\mid v_i,\theta)\right]-\log\pi_N(\theta)+\log p(T_N\mid V_N)\\ &=N\widehat{J}_N(\theta)+C_N, \tag*{(17)} \end{aligned}

where CN=Ndtlog⁡π/2+log⁡AN+log⁡p(TN∣VN)C_N=Nd_t\log\pi/2+\log A_N+\log p(T_N\mid V_N) is independent of θ\theta. The evidence is finite and positive because the likelihood is bounded and strictly positive and the prior is proper. Minimizing this expression proves the assertion; continuity on Θ\Theta ensures a minimizer exists. Taking N=1N=1 proves equation (11). □

The product likelihood is an analytical observation model; this algebra does not establish that the actual readings were independently sampled from its Gaussians. With a uniform prior, MAP minimizes data fit alone. With equation (16), the prior depends on θ\theta and cannot be dropped from optimization. Its exponent uses the actual product penalty ∥AB∥F2\lVert AB\rVert_F^2, not ∥θ∥2\lVert\theta\rVert^2. A Gaussian likelihood in tt also does not imply a Gaussian posterior in the nonlinear adapter parameters.

Variational Change of Distribution

For probability measures Q≪PQ \ll P, define KL⁡(Q∥P)=∫log⁡(dQ/dP) dQ\operatorname{KL}(Q\Vert P)=\int\log(\mathrm{d}Q/\mathrm{d}P)\,\mathrm{d}Q. The following lemma is stated for a general loss so that it applies both to alignment and to entropy.

Lemma B.2 (KL variational transfer bound). Let L≥0L \ge0 be measurable, EPL\mathbb{E}_{P}L and EQL\mathbb{E}_{Q}L finite, and κ=KL⁡(Q∥P)<∞\kappa=\operatorname{KL}(Q\Vert P)<\infty. Suppose Zη=EPeηL<∞Z_{\eta}=\mathbb{E}_{P}e^{\eta L}<\infty for at least one η>0\eta>0. For each such η\eta, define dPη=eηLdP/Zη\mathrm{d}P_{\eta}=e^{\eta L}\mathrm{d}P/Z_{\eta}. Then

KL⁡(Q∥Pη)=κ−ηEQL+log⁡Zη,\operatorname{KL}(Q\Vert P_{\eta})=\kappa-\eta\mathbb{E}_{Q}L+\log Z_{\eta},
EQL≤EPL+κ+log⁡EPeη(L−EPL)η.(18)\mathbb{E}_{Q}L\le\mathbb{E}_{P}L+\frac{\kappa+\log\mathbb{E}_{P}e^{\eta(L-\mathbb{E}_{P}L)}}{\eta}. \tag*{(18)}

The inequality becomes an equality precisely when Q=PηQ=P_{\eta} for this η\eta.

Proof. Since 1≤Zη<∞1\le Z_{\eta}<\infty, PηP_{\eta} is a probability measure equivalent to PP. The Radon–Nikodym chain rule gives, QQ-almost surely,

log⁡dQdPη=log⁡dQdP−ηL+log⁡Zη.\log\frac{\mathrm{d}Q}{\mathrm{d}P_{\eta}}=\log\frac{\mathrm{d}Q}{\mathrm{d}P}-\eta L+\log Z_{\eta}.

All terms are integrable under the assumptions, so integration proves the identity. Nonnegativity of KL implies ηEQL≤κ+log⁡Zη\eta\mathbb{E}_{Q}L\le\kappa+\log Z_{\eta}. Subtract ηEPL\eta\mathbb{E}_{P}L from the right-hand exponential-moment expression and divide by η>0\eta>0 to obtain the bound. KL is zero exactly when the two measures agree, giving the equality condition.

Apply the lemma with L(r)=∥r∥2L(r)=\lVert r\rVert^{2}, P=PθP=P_{\theta}, and Q=QθQ=Q_{\theta} will derive the KL divergence formula κθ−ηRte(θ)+log⁡Zη=KL⁡(Qθ∥Pθ,η)≥0\kappa_{\theta}-\eta R_{\mathrm{te}}(\theta)+\log Z_{\eta}=\operatorname{KL}(Q_{\theta}\Vert P_{\theta,\eta})\ge0. Taking the infimum over admissible η\eta proves equation (13). These are distributions of the same residual function under two triplet laws, so both sides use the same teacher/anchor construction and fixed θ\theta. A test triplet distribution must specify its reference views even when those references are unavailable to the deployed algorithm.

The expression ηEQL−KL⁡(Q∥P)≤log⁡Zη\eta\mathbb{E}_{Q}L-\operatorname{KL}(Q\Vert P)\le\log Z_{\eta} has the same variational structure as an ELBO. Here ZηZ_{\eta} normalizes an exponential loss moment, rather than a predictive likelihood. The transfer result follows from this identity, independently of the MAP argument. It requires distributional information through κ\kappa; fitting the offline objective alone does not determine that quantity.

Gaussian Residual Bound

The actual offline residual law is an additional modeling assumption. The covariance σ2I\sigma^{2}I below describes this law and need not equal the observation covariance I/2I/2 used in Lemma B.1.

Lemma B.3 (Gaussian quadratic exponential moment). Let r∼N(μ,Σ)r\sim\mathcal{N}(\mu,\Sigma) in Rd\mathbb{R}^{d}, where Σ\Sigma is symmetric positive semidefinite. Define

m=∥μ∥2+tr⁡Σ,a=tr⁡(Σ2)+2μTΣμ,b=∥Σ∥op.m=\lVert\mu\rVert^{2}+\operatorname{tr}\Sigma,\qquad a=\operatorname{tr}(\Sigma^{2})+2\mu^{\mathsf T}\Sigma\mu,\qquad b=\lVert\Sigma\rVert_{\mathrm{op}}.

Then m=E∥r∥2m=\mathbb{E}\lVert r\rVert^{2}. If b>0b>0 and 0<η<(2b)−10<\eta<(2b)^{-1},

log⁡Eeη(∥r∥2−m)≤aη21−2bη.(19)\log\mathbb{E}e^{\eta(\lVert r\rVert^{2}-m)} \le\frac{a\eta^{2}}{1-2b\eta}. \tag*{(19)}

In particular, if μ=0\mu=0 and Σ=σ2I\Sigma=\sigma^{2}I,

log⁡Eeη(∥r∥2−dtσ2)=−dt2log⁡(1−2σ2η)−dtσ2η≤dtσ4η21−2σ2η.(20)\log\mathbb{E}e^{\eta(\lVert r\rVert^{2}-d_{t}\sigma^{2})} =-\frac{d_{t}}{2}\log(1-2\sigma^{2}\eta)-d_{t}\sigma^{2}\eta \le\frac{d_{t}\sigma^{4}\eta^{2}}{1-2\sigma^{2}\eta}. \tag*{(20)}

Proof. Diagonalize Σ=Udiag⁡(λ1,…,λdt)U⊤\Sigma= U \operatorname{diag}(\lambda_1,\ldots,\lambda_{d_t})U^{\top}, where UU is orthogonal and λj≥0\lambda_j \ge0. Writing ν=U⊤μ\nu= U^{\top}\mu, the squared norm has the distribution ∑j(νj+λjgj)2\sum_j(\nu_j+\sqrt{\lambda_j}g_j)^2 for independent standard normal variables gjg_j. Its expectation is mm. Completing the square in the one-dimensional Gaussian integral gives

Eeη(νj+λjgj)2=(1−2ηλj)−1/2exp⁡(ηνj21−2ηλj).\mathbb{E}e^{\eta(\nu_j+\sqrt{\lambda_j}g_j)^2}=(1-2\eta\lambda_j)^{-1/2}\exp\left(\frac{\eta\nu_j^2}{1-2\eta\lambda_j}\right).

Indeed, combining the exponent with −gj2/2-g_j^2/2 gives a quadratic with coefficient $-(1-2\eta\lambda_j)/2; integrating its completed square gives the prefactor and displayed exponent. The formula also holds when λj=0\lambda_j=0. Independence and centering therefore give

log⁡Eeη(∥r∥2−m)=∑j[−12log⁡(1−2ηλj)−ηλj+2η2νj2λj1−2ηλj].(21)\log\mathbb{E}e^{\eta(\lVert r\rVert^2-m)}=\sum_j\left[-\frac{1}{2}\log(1-2\eta\lambda_j)-\eta\lambda_j+\frac{2\eta^2\nu_j^2\lambda_j}{1-2\eta\lambda_j}\right]. \tag*{(21)}

For 0≤u<10\le u<1, the convergent power series satisfies −log⁡(1−u)−u=∑j=2∞uj/j≤u2/[2(1−u)]-\log(1-u)-u=\sum_{j=2}^{\infty}u^j/j\le u^2/[2(1-u)]. Applying this with u=2ηλju=2\eta\lambda_j bounds the first two terms by η2λj2/(1−2ηλj)\eta^2\lambda_j^2/(1-2\eta\lambda_j). Since λj≤b\lambda_j\le b, summation bounds the whole expression by η2∑j(λj2+2νj2λj)/(1−2bη)=aη2/(1−2bη)\eta^2\sum_j(\lambda_j^2+2\nu_j^2\lambda_j)/(1-2b\eta)=a\eta^2/(1-2b\eta). Substitution of λj=σ2\lambda_j=\sigma^2 and νj=0\nu_j=0 proves equation 20.

Corollary B.4 (Gaussian transfer). Suppose Pθ=N(μ,Σ)P_\theta=\mathcal{N}(\mu,\Sigma) and QθQ_\theta has finite second moment and finite κθ=KL⁡(Qθ∥Pθ)\kappa_\theta=\operatorname{KL}(Q_\theta\Vert P_\theta). With a,ba,b from Lemma B.3,

Rte(θ)≤Roff(θ)+2aκθ+2bκθ.(22)R_{\mathrm{te}}(\theta)\le R_{\mathrm{off}}(\theta)+2\sqrt{a\kappa_\theta}+2b\kappa_\theta. \tag*{(22)}

The isotropic centered case is exactly equation 14.

Proof. For b>0b>0, one has a>0a>0. Lemmas B.2 and B.3 imply

Rte−Roff≤κθη+aη1−2bη.R_{\mathrm{te}}-R_{\mathrm{off}}\le\frac{\kappa_\theta}{\eta}+\frac{a\eta}{1-2b\eta}.

If κθ>0\kappa_\theta>0, choose η=κθ/(a+2bκθ)\eta=\sqrt{\kappa_\theta}/(\sqrt{a}+2b\sqrt{\kappa_\theta}), which lies in (0,(2b)−1)(0,(2b)^{-1}). Then κθ/η=aκθ+2bκθ\kappa_\theta/\eta=\sqrt{a\kappa_\theta}+2b\kappa_\theta and aη/(1−2bη)=aκθa\eta/(1-2b\eta)=\sqrt{a\kappa_\theta}, proving the bound. If κθ=0\kappa_\theta=0, then Qθ=PθQ_\theta=P_\theta and the risks agree; equivalently take η↓0\eta\downarrow0. If b=0b=0, PθP_\theta is a point mass, and finite KL again forces Qθ=PθQ_\theta=P_\theta. Finally, for μ=0\mu=0, Σ=σ2I\Sigma=\sigma^2I, substitute a=dtσ4a=d_t\sigma^4, b=σ2b=\sigma^2, and Roff=dtσ2R_{\mathrm{off}}=d_t\sigma^2.

Writing Js(θ)=Rs(θ)+λctrlΩ(θ)J_s(\theta)=R_s(\theta)+\lambda_{\mathrm{ctrl}}\Omega(\theta) for s∈{off,te}s\in\{\mathrm{off},\mathrm{te}\} gives Jte−Joff=Rte−RoffJ_{\mathrm{te}}-J_{\mathrm{off}}=R_{\mathrm{te}}-R_{\mathrm{off}}. Thus the same transfer bound holds for the regularized objective. No estimate of the control penalty in terms of adapter rank is needed.

Additional Results

The main text presents the primary CIFAR-100-C comparison in Table 3. Here, Tables 5 and 6 provide the full per-corruption results on CIFAR-10-C and ImageNet-C, respectively, for supervised ViT and CLIP-based classification. These tables compare the source-free and source-dependent variants of Drift-TTA with the unadapted models and prior TTA baselines.

Single-Model Generalization to Unseen Corruption Families

The per-corruption tables above follow the leave-one-family-out (LOFO) protocol, in which a separate Drift-TTA adapter is trained for each held-out corruption; a full fifteen-family table therefore aggregates fifteen independently trained adapters. Prior test-time adaptation methods instead deploy a single model that must cope with any corruption, so this asymmetry could be raised against a direct comparison. To remove it, we train one Drift-TTA adapter on the fifteen standard corruption families and evaluate it, with no further training, on the four validation corruptions distributed with the -C benchmarks that are disjoint from the training families: *speckle noise, Gaussian blur, spatter, and saturate. This is a strict, single-model, apples-to-apples setting: exactly as for the baselines, one model adapts to all four unseen corruptions, in both its offline form (the frozen adapter) and its online form (the adapter with entropy-based gate calibration), under source-dependent (SD) and source-free (SF) training. Table 7–Table 9 report accuracy at severity 5 on ImageNet-C, CIFAR-10-C, and CIFAR-100-C, against the strongest single-model baseline for each backbone (PEA for ViT, BITTA for CLIP). Across all three benchmarks and both backbones, Drift-TTA improves over the unadapted model and over the baseline, showing that the trajectory-repair prior transfers to corruption families never seen during training.

MethodSpeckleGaussian BlurSpatterSaturateMean
ImageNet-C
ViT
ViT-B/1664.344.969.173.462.9
PEA68.450.473.977.967.6
Drift-TTA (SF)69.453.069.774.366.6
Drift-TTA (SD)71.154.969.973.867.4
Drift-TTA-online (SF)69.854.569.974.567.2
Drift-TTA-online (SD)71.956.970.574.268.4
CLIP
ViT-B/1625.821.338.349.533.7
BITTA36.425.838.651.138.0
Drift-TTA (SF)37.525.441.251.238.8
Drift-TTA (SD)38.827.445.749.740.4
Drift-TTA-online (SF)39.127.941.550.739.8
Drift-TTA-online (SD)38.127.045.050.140.1

Table 7. Test-time adaptation accuracy (%) on ImageNet-C for the four held-out (unseen) corruption families at severity 5. A single Drift-TTA adapter is trained on the fifteen standard corruption families and evaluated, without further training, on four disjoint validation corruptions. “Drift-TTA” is the frozen adapter (offline); “Drift-TTA-online” adds online entropy-based gate calibration; SD and SF denote source-dependent and source-free training. The best result in each column (within each backbone) is shown in green and the second best in blue.

MethodSpeckleGaussian BlurSpatterSaturateMean
CIFAR-100-C
ViT
ViT-B/1638.870.685.380.768.9
PEA64.380.987.385.579.5
Drift-TTA (SF)71.183.987.184.281.6
Drift-TTA (SD)80.784.486.481.883.4
Drift-TTA-online (SF)74.684.187.184.582.6
Drift-TTA-online (SD)80.384.686.981.983.4
CLIP
ViT-B/1621.941.258.254.443.9
BITTA30.250.960.159.350.0
Drift-TTA (SF)35.652.162.659.452.4
Drift-TTA (SD)50.455.363.259.957.2
Drift-TTA-online (SF)39.954.862.960.454.5
Drift-TTA-online (SD)50.057.064.160.157.8

Table 9. Test-time adaptation accuracy (%) on CIFAR-100-C for the four held-out (unseen) corruption families at severity 5. A single Drift-TTA adapter is trained on the fifteen standard corruption families and evaluated, without further training, on four disjoint validation corruptions. “Drift-TTA” is the frozen adapter (offline); “Drift-TTA-online” adds online entropy-based gate calibration; SD and SF denote source-dependent and source-free training. The best result in each column (within each backbone) is shown in green and the second best in blue.

MethodGaussianShotImpulseDefocusGlassMotionZoomSnowFrostFogBrightnessContrastElasticPixelateJPEGMean
CIFAR-10-C ViT
ViT-Tiny43.849.869.382.171.082.384.691.889.587.595.583.483.251.881.476.5
TENT43.850.169.681.671.282.684.691.789.788.095.280.583.549.081.676.2
SAR45.454.770.081.969.881.783.989.087.583.693.383.281.751.579.575.8
EATA43.849.869.382.171.082.384.691.889.587.595.583.483.251.781.476.5
FOA62.771.677.588.276.487.791.191.690.989.195.285.385.276.581.383.3
CMF66.271.671.187.875.687.590.589.690.588.194.886.383.383.383.283.3
PEA61.966.077.888.577.488.390.791.991.589.995.586.985.681.782.083.7
Drift-TTA (SF)82.080.873.792.885.990.393.992.792.988.895.490.188.063.983.686.3
Drift-TTA (SD)83.885.276.293.482.790.094.392.793.089.195.387.688.672.284.187.2
CLIP
ViT-B/1638.041.754.471.740.967.973.673.977.470.384.562.353.847.659.461.2
TPT39.944.958.872.343.570.074.875.578.572.785.170.857.252.361.363.8
TDA42.946.063.372.346.969.874.777.579.471.685.962.360.563.762.965.3
BCA31.534.554.266.535.664.970.673.976.368.584.055.451.555.856.058.6
DMN-ZS45.049.559.873.648.870.974.877.177.570.186.267.662.855.362.165.4
DPE34.838.757.873.144.368.675.974.378.971.286.964.457.552.458.562.5
BAT60.565.563.480.152.380.482.083.183.681.389.682.467.560.568.173.3
BITTA62.667.465.080.656.380.782.383.584.381.989.983.369.165.569.374.8
Drift-TTA (SF)69.269.367.881.971.880.984.183.786.482.490.084.474.265.170.277.4
Drift-TTA (SD)73.878.375.481.774.681.184.784.487.082.390.584.475.468.671.179.5

Table 5. Per-corruption test-time adaptation accuracy (%) on CIFAR-10-C for unimodal ViT and CLIP-based classification across the fifteen corruption types. The best result in each column is shown in green and the second best in blue.

MethodGaussianShotImpulseDefocusGlassMotionZoomSnowFrostFogBrightnessContrastElasticPixelateJPEGMean
ImageNet-C ViT
ViT-B/1656.756.857.546.935.653.144.862.262.565.777.632.646.066.967.655.5
TENT57.158.159.244.743.256.650.662.860.565.278.059.749.868.268.658.8
SAR59.962.262.854.154.159.154.563.565.665.378.264.458.369.269.762.7
EATA57.359.159.953.649.458.251.863.062.965.777.862.055.765.868.560.7
FOA61.563.564.356.955.161.060.968.470.973.680.966.061.873.573.666.1
CMF60.061.260.956.656.862.460.869.367.972.778.865.269.473.872.065.9
PEA57.058.458.953.250.159.860.469.168.772.980.263.569.972.072.064.5
Drift-TTA (SF)62.061.461.956.540.558.750.064.463.264.577.155.749.468.768.060.2
Drift-TTA (SD)61.962.862.958.342.664.750.764.361.963.677.453.047.970.868.160.7
CLIP
ViT-B/1613.014.413.724.115.825.026.632.630.737.055.017.613.633.033.525.7
TPT8.89.49.624.516.225.124.033.832.337.855.619.114.335.934.825.4
TDA9.611.511.224.814.623.925.035.132.838.756.516.314.238.833.825.8
BCA6.77.77.226.015.024.925.933.932.937.956.117.314.339.635.825.4
DMN-ZS11.812.212.223.215.024.023.131.529.736.553.116.912.131.633.124.4
DPE10.313.517.326.418.027.524.935.233.239.656.520.216.036.835.427.4
BAT20.223.519.927.221.031.228.736.331.541.056.426.223.939.939.231.1
BITTA22.524.122.428.023.931.929.336.932.241.955.926.422.740.439.731.9
Drift-TTA (SF)27.228.426.726.623.232.929.036.432.539.754.425.822.538.535.231.9
Drift-TTA (SD)26.127.226.426.420.233.128.037.632.941.153.825.820.539.034.331.5

Table 6. Per-corruption test-time adaptation accuracy (%) on ImageNet-C for unimodal ViT and CLIP-based classification across the fifteen corruption types. The best result in each column is shown in green and the second best in blue.

Calibration (Expected Calibration Error)

Beyond accuracy, we report expected calibration error (ECE, %, ↓\downarrow) at severity 5 for all three benchmarks and both backbones. For each setting we compare the unadapted model (No-TTA) and the strongest prior test-time-adaptation baseline (PEA for the unimodal ViT, BITTA for CLIP) against Drift-TTA (the frozen repair adapter) and Drift-TTA-online (the same adapter with its per-branch gates tuned online by entropy minimization), under both source-dependent (SD) and source-free (SF) training. ECE uses 15 equal-width bins over the top-label confidence, on the same corrupted evaluation sets and configurations used for the accuracy tables. The best SD and SF variants reduce mean ECE relative to PEA and BITTA across all model–dataset settings. Table 4 summarizes these results in the main text. Table 10–Table 12 give the full per-corruption breakdown.

Method / deploymentGaussianShotImpulseDefocusGlassMotionZoomSnowFrostFogBrightnessContrastElasticPixelateJPEGMean
No-TTA11.338.243.4512.729.1712.8414.4615.9617.1122.0714.3315.9217.8413.3313.8613.51
PEA3.593.755.5414.327.7314.0215.8213.4214.8115.8913.7114.3613.6910.9410.0811.44
Drift-TTA: adapter only (SD)10.119.974.9213.859.5512.5014.6412.6814.7116.8212.3513.6113.696.1710.3011.72
Drift-TTA: adapter + online gate (SD)6.266.606.168.746.007.929.668.7210.6811.348.788.898.738.136.518.21
Δ\Delta best SD vs. PEA↑2.67↑2.85↓0.62↓5.58↓1.73↓6.10↓6.16↓4.70↓4.13↓4.55↓4.93↓5.47↓4.96↓4.77↓3.57↓3.23
Drift-TTA: adapter only (SF)12.029.888.8816.7316.3114.9416.4914.8518.0619.9813.5317.4916.737.7913.7414.50
Drift-TTA: adapter + online gate (SF)9.137.704.2812.5311.3710.5312.3910.4513.3414.9010.1113.0112.4110.008.5410.71
Δ\Delta best SF vs. PEA↑5.54↑3.95↓1.26↓1.79↑3.64↓3.49↓3.43↓2.97↓1.47↓0.99↓3.60↓1.35↓1.28↓3.15↓1.54↓0.73
No-TTA12.589.724.373.193.751.932.395.016.004.435.524.592.074.742.744.87
BITTA23.0018.3521.6110.3225.528.999.417.707.278.814.336.9116.6217.4717.4213.58
Drift-TTA: adapter only (SD)2.352.503.122.801.692.093.284.165.654.895.295.242.651.692.603.33
Drift-TTA: adapter + online gate (SD)10.125.338.253.083.612.780.901.021.310.741.881.582.346.393.613.53
Δ\Delta best SD vs. BITTA↓20.65↓15.85↓18.49↓7.52↓23.83↓6.90↓8.51↓6.68↓5.96↓8.07↓2.45↓5.33↓14.28↓15.78↓14.82↓10.25
Drift-TTA: adapter only (SF)3.083.251.873.194.192.234.305.285.575.835.135.351.903.812.853.86
Drift-TTA: adapter + online gate (SF)4.221.735.280.841.570.981.172.292.001.142.021.582.253.152.532.28
Δ\Delta best SF vs. BITTA↓19.92↓16.62↓19.74↓9.48↓23.95↓8.01↓8.24↓5.41↓5.13↓6.41↓2.02↓4.79↓14.86↓14.04↓15.52↓11.30

Table 10. Expected calibration error (ECE, %, ↓) on CIFAR-10-C at severity 5. All 15 corruption families are reported for both backbones. PEA is the ViT prior-TTA baseline and BITTA is the CLIP prior-TTA baseline. Drift-TTA is shown with its adapter alone and with additional online gate tuning, under source-dependent (SD) and source-free (SF) training. In each Δ\Delta row, the lower ECE of the two variants is selected independently per column and subtracted from PEA/BITTA. Green ↓ values denote how many percentage points the selected ECE is below the baseline; red ↑ values denote how many points it is above. Method-cell tints follow the visualization palette: teal for no adaptation, orange for SD, purple for SF, and gray for PEA/BITTA.

Table 12: Expected calibration error (ECE, %, ↓) on ImageNet-C at severity 5. All 15 corruption families are reported for both backbones. PEA is the ViT prior-TTA baseline and BITTA is the CLIP prior-TTA baseline. Drift-TTA is shown with its adapter alone and with additional online gate tuning, under source-dependent (SD) and source-free (SF) training. In each ∆ row, the lower ECE of the two variants is selected independently per column and subtracted from PEA/BITTA. Green ↓ values denote how many percentage points the selected ECE is below the baseline; red ↑ values denote how many points it is above. Method-cell tints follow the visualization palette: teal for no adaptation, orange for SD, purple for SF, and gray for PEA/BITTA.

Method / deploymentGaussianShotImpulseDefocusGlassMotionZoomSnowFrostFogBrightnessContrastElasticPixelateJPEGMean
ViT No-TTA12.857.4611.689.994.992.684.684.4020.9848.118.9820.515.974.735.1511.54
PEA36.0734.2534.5931.6736.0233.9139.1833.5938.8740.0118.9337.4437.9323.6125.4833.44
Drift-TTA: adapter only (SD)21.2820.2221.2818.719.4019.3811.1715.5118.8033.8014.8225.637.6516.9212.7517.82
Drift-TTA: adapter + online gate (SD)16.2214.8516.3113.414.1614.255.6010.9914.0629.3311.8619.082.5712.989.7413.03
∆ best SD vs. PEA ↓↓19.85↓19.40↓18.28↓18.26↓31.86↓19.66↓33.58↓22.60↓24.81↓10.68↓7.07↓18.36↓35.36↓10.63↓15.74↓20.41
Drift-TTA: adapter only (SF)19.3115.0216.2815.236.4113.728.2815.6024.6343.9815.2934.326.4113.5313.4017.43
Drift-TTA: adapter + online gate (SF)16.1712.2013.2812.843.6611.315.5012.9021.6840.4012.9732.203.5411.1510.9914.72
∆ best SF vs. PEA ↓↓19.90↓22.05↓21.31↓18.83↓32.36↓22.60↓33.68↓20.69↓17.19↑0.39↓5.96↓5.24↓34.39↓12.46↓14.49↓18.72
CLIP No-TTA7.777.307.830.964.711.322.211.764.681.743.152.778.812.564.224.12
BITTA18.7820.1920.4719.6221.0816.6119.1216.0720.1315.5910.2516.1323.3915.5617.5618.04
Drift-TTA: adapter only (SD)5.594.625.032.909.432.046.231.895.872.082.321.7610.952.154.874.51
Drift-TTA: adapter + online gate (SD)15.6414.2814.0812.4418.7811.5114.689.5814.388.896.449.7419.8611.4411.5612.89
∆ best SD vs. BITTA ↓↓13.19↓15.57↓15.44↓16.72↓11.65↓14.57↓12.89↓14.18↓14.26↓13.51↓7.93↓14.37↓12.44↓13.41↓12.69↓13.53
Drift-TTA: adapter only (SF)2.051.903.431.314.533.131.911.825.271.642.373.268.523.472.553.14
Drift-TTA: adapter + online gate (SF)6.937.498.656.6912.005.729.646.6212.797.105.324.0114.444.688.138.01
∆ best SF vs. BITTA ↓↓16.73↓18.29↓17.04↓18.31↓16.55↓13.48↓17.21↓14.25↓14.86↓13.95↓7.88↓12.87↓14.87↓12.09↓15.01↓14.90

Table 12.

ECE computation. We use the standard binned estimator of Guo et al. [8]. For each test image the top-label confidence is p^=max⁡ksoftmax⁡(z)k\hat{p} = \max_k \operatorname{softmax}(z)_k and the prediction is correct when y^=\hat{y} = y.y. Confidences are partitioned into M=15M=15 equal-width bins {Bm}m=1M\{B_m\}_{m=1}^{M} over [0,1][0,1], and ECE =∑m=1M∣Bm∣N∣acc⁡(Bm)−conf⁡(Bm)∣=\sum_{m=1}^{M}\frac{|B_m|}{N}\left|\operatorname{acc}(B_m)-\operatorname{conf}(B_m)\right|, reported as a percentage. For the unimodal ViT rows the logits zz are the classifier outputs; for the CLIP rows zz are the temperature-scaled image–text cosine scores (logit_scale ⋅⟨fI,fT⟩\cdot\langle f_I,f_T\rangle), i.e. the model’s own predictive distribution. Confidences are collected from

MethodSpeckleGaussian BlurSpatterSaturateMean
CIFAR-10-C
ViT
ViT-Tiny54.877.594.794.080.3
PEA70.387.095.194.786.8
Drift-TTA (SF)76.692.895.094.689.8
Drift-TTA (SD)88.494.195.394.193.0
Drift-TTA-online (SF)81.693.594.694.291.0
Drift-TTA-online (SD)88.393.895.194.192.8
CLIP
ViT-B/1645.370.482.883.470.5
BITTA68.181.787.789.281.7
Drift-TTA (SF)66.280.686.287.880.2
Drift-TTA (SD)79.781.587.187.784.0
Drift-TTA-online (SF)74.483.588.289.683.9
Drift-TTA-online (SD)80.484.189.490.286.0

Table 8. Test-time adaptation accuracy (%) on CIFAR-10-C for the four held-out (unseen) corruption families at severity 5. A single Drift-TTA adapter is trained on the fifteen standard corruption families and evaluated, without further training, on four disjoint validation corruptions. “Drift-TTA” is the frozen adapter (offline); “Drift-TTA-online” adds online entropy-based gate calibration; SD and SF denote source-dependent and source-free training. The best result in each column (within each backbone) is shown in green and the second best in blue.

Method / deploymentGaussianShotImpulseDefocusGlassMotionZoomSnowFrostFogBrightnessContrastElasticPixelateJPEGMean
No-TTA16.8418.229.467.6219.096.037.315.537.235.376.3010.744.4016.2411.0110.10
PEA17.6917.4922.7214.1018.1316.9113.9316.5116.7423.0613.9820.1422.5619.4316.6418.00
Drift-TTA: adapter only (SD)6.547.496.618.245.237.758.698.9610.3212.947.9916.797.915.934.758.47
Drift-TTA: adapter + online gate (SD)9.088.418.107.849.387.267.817.197.906.377.095.287.5010.4111.258.06
Δ\Delta best SD vs. PEA↓11.15↓10.00↓16.11↓6.26↓12.90↓9.65↓6.12↓9.32↓8.84↓16.69↓6.89↓14.86↓15.06↓13.50↓11.89↓9.94
Drift-TTA: adapter only (SF)6.914.9812.488.405.928.479.409.5513.7018.489.5622.207.576.705.089.96
Drift-TTA: adapter + online gate (SF)7.226.469.247.487.286.487.157.228.5612.237.0814.906.999.999.049.53
Δ\Delta best SF vs. PEA↓10.78↓12.51↓10.24↓6.62↓12.21↓10.43↓6.78↓9.29↓8.18↓10.83↓6.90↓5.24↓15.57↓12.73↓11.56↓8.47
No-TTA10.513.534.247.491.009.048.5011.979.835.6811.346.746.264.795.966.30
BITTA21.4521.6415.6711.7719.9411.9510.7510.8112.0012.916.7911.5517.9219.5215.8814.70
Drift-TTA: adapter only (SD)10.5213.077.8912.069.4612.4112.5613.2012.559.8711.809.3610.432.557.2710.33
Drift-TTA: adapter + online gate (SD)3.934.922.795.514.543.925.916.036.443.735.544.484.119.803.485.01
Δ\Delta best SD vs. BITTA↓17.52↓16.72↓12.88↓6.26↓15.40↓8.03↓4.84↓4.78↓5.56↓9.18↓1.25↓7.07↓13.81↓9.72↓12.40↓9.69
Drift-TTA: adapter only (SF)8.549.056.7111.419.6012.3312.9613.6713.449.1012.938.5310.394.036.719.96
Drift-TTA: adapter + online gate (SF)4.545.603.656.367.917.116.938.037.465.177.404.956.062.502.895.77
Δ\Delta best SF vs. BITTA↓16.91↓16.04↓12.02↓5.41↓12.03↓4.84↓3.82↓2.78↓4.54↓7.74↓0.61↓6.60↓11.86↓17.02↓12.99↓8.93

Table 11. Expected calibration error (ECE, %, ↓) on CIFAR-100-C at severity 5. All 15 corruption families are reported for both backbones. PEA is the ViT prior-TTA baseline and BITTA is the CLIP prior-TTA baseline. Drift-TTA is shown with its adapter alone and with additional online gate tuning, under source-dependent (SD) and source-free (SF) training. In each Δ\Delta row, the lower ECE of the two variants is selected independently per column and subtracted from PEA/BITTA. Green ↓ values denote how many percentage points the selected ECE is below the baseline; red ↑ values denote how many points it is above. Method-cell tints follow the visualization palette: teal for no adaptation, orange for SD, purple for SF, and gray for PEA/BITTA.

each method’s final prediction (post-adaptation for the adapted rows) on exactly the corrupted test sets used for the accuracy tables, all at severity 5, and binned with a single shared implementation so every row uses an identical metric.

Baselines. The PEA (ViT) and BITTA (CLIP) rows are obtained by running the authors’ official implementations under their reported settings and computing ECE with the same estimator; no calibration-specific tuning is applied. PEA uses source-statistic alignment with a continual stream at batch size 64; BITTA uses its bilateral information-aware objective with one gradient step per batch at batch sizes 200/200/64 (CIFAR-10-C/CIFAR-100-C/ImageNet-C). As a correctness check, the recomputed top-1 accuracy of every baseline reproduces the corresponding entry in the accuracy tables to the first decimal, confirming that the calibration numbers are measured on the same models and predictions.

Hidden-Trajectory Drift under Corruption

This section explains how each panel of Figure 4 is constructed and how it should be interpreted. We first define the trajectory quantities shown in the figure, then examine whether the association between trajectory drift and prediction error remains after controlling for corruption severity. Figure 5 provides the corresponding analysis on ImageNet-C.

Figure 4. Hidden-trajectory drift on CIFAR-100-C. A frozen ViT-B/16 is evaluated on CIFAR-100-C without adaptation. (a) Mean hidden trajectories for clean inputs and Gaussian noise at severities 1–5, projected onto the first two principal components. (b) Relative clean-to-corrupted drift across network depth, averaged over all 15 corruption families. (c) Relative trajectory drift versus top-1 error across the 75 corruption-family and severity conditions (Pearson r=0.73r=0.73; Spearman ρs=0.78\rho_s=0.78). (d) Top-1 accuracy across severity for four corruption families. (e) Magnitude and angle feature distributions for clean inputs and Gaussian noise at severities 1–5.

Hidden-trajectory drift on ImageNet-C across corruption severities

Figure 5. Hidden-trajectory drift on ImageNet-C. A frozen ViT-B/16 is evaluated on the full ImageNet-C validation set without adaptation. Panels (a) and (e) use fog as a representative corruption family, while panels (b)–(d) summarize all fifteen corruption families following Figure 4. Across the 75 corruption-family and severity conditions, relative trajectory drift is associated with top-1 error (Pearson r=0.78r=0.78; Spearman ρs=0.87\rho_s=0.87). The clean point in (c) is shown as a reference and is excluded from both correlations.

Experimental setup. We evaluate a frozen ViT-B/16 without test-time adaptation. CIFAR-100-C contains N=10,000N=10{,}000 images per corruption condition, while ImageNet-C contains N=50,000N=50{,}000. Each corrupted image is paired with its clean counterpart, and both are processed by the same frozen model. Consequently, trajectory drift and prediction error are measured on the same images. Panels (a) and (e) use one representative corruption family—Gaussian noise for CIFAR-100-C and fog for ImageNet-C—while panels (b)–(d) summarize all fifteen corruption families.

Panel (a): Mean hidden trajectories. For corruption family cc and severity ss, let xic,sx_i^{c,s} denote the corrupted version of clean image xix_i. At trajectory position ℓ∈{0,…,L}\ell\in\{0,\ldots,L\}, we average the hidden representation over all images:

hˉℓc,s=1N∑i=1Nhℓ(xic,s).(23)\bar{\mathbf{h}}_{\ell}^{c,s}=\frac{1}{N}\sum_{i=1}^{N}\mathbf{h}_{\ell}\left(x_i^{c,s}\right). \tag*{(23)}

This gives one mean trajectory for clean inputs and one for each corruption severity. The six trajectories are projected jointly onto the first two principal components. Each line therefore shows how the mean representation changes from the input embedding h0\mathbf{h}_0 to the final block representation hL\mathbf{h}_L. The projection is used only for visualization; all drift measurements are computed in the original representation space.

Panel (b): Relative drift across network depth. We first compute the mean norm of the clean representation at each trajectory position:

νl=1N∑i=1N∥hl(xi)∥2.(24)\nu_l = \frac{1}{N}\sum_{i=1}^{N} \lVert\mathbf{h}_l(x_i) \rVert_2 . \tag*{(24)}

The relative drift for corruption family cc and severity ss is then

δlc,s=1N∑i=1N∥hl(xic,s)−hl(xi)∥2νl.(25)\delta_l^{c,s} = \frac{\frac{1}{N}\sum_{i=1}^{N} \lVert\mathbf{h}_l(x_i^{c,s})-\mathbf{h}_l(x_i) \rVert_2}{\nu_l}. \tag*{(25)}

The numerator measures how far the corrupted representation moves from its clean counterpart. Dividing by νl\nu_l accounts for differences in representation scale across network depth. Panel (b) averages this quantity over all fifteen corruption families and plots one curve per severity. The clean curve is zero because every clean representation is compared with itself.

Panel (c): Trajectory drift and prediction error. For each corruption-family and severity condition, we average the relative drift across the complete trajectory:

Dc,s=1L+1∑l=0Lδlc,s.(26)D^{c,s} = \frac{1}{L+1}\sum_{l=0}^{L}\delta_l^{c,s}. \tag*{(26)}

Panel (c) contains one point for each of the 15×5=7515 \times5 = 75 conditions. Its horizontal coordinate is the trajectory drift Dc,sD^{c,s}, divided by the largest value among the 75 conditions so that the displayed values lie between zero and one:

D~c,s=Dc,smax⁡c′,s′Dc′,s′.(27)\widetilde{D}^{c,s} = \frac{D^{c,s}}{\max_{c',s'} D^{c',s'}}. \tag*{(27)}

This final scaling changes only the displayed axis and does not affect either correlation. The vertical coordinate is the top-1 error over the same images. Pearson rr measures the linear association between drift and error, while Spearman ρs\rho_s measures whether conditions with larger drift generally have larger error, independent of a linear relationship. The clean point is shown at zero drift as a reference and is excluded from both calculations.

Panel (d): Accuracy across severity. Panel (d) provides a direct view of the corresponding accuracy loss. We first retain corruption families whose accuracy decreases monotonically from clean through severity 5. Among these families, the panel shows the four with the largest mean accuracy loss across severities 1–5. Each group contains the clean accuracy followed by the accuracy at the five corruption severities.

Panel (e): Magnitude and angle distributions. Panel (e) provides an image-level view of trajectory geometry. Each image is represented by the magnitude and angle features defined in § 4. The magnitude feature measures the average distance traveled between adjacent hidden states, normalized by the distance between h0h_0 and hLh_L. The angle feature similarly measures the average change in orientation between adjacent states, normalized by the angle between the trajectory endpoints. Each column compares the clean distribution with one corruption severity. Increasing separation between the contours indicates that individual corrupted images follow different hidden trajectories from their clean counterparts.

Controlling for corruption severity. Across the 75 CIFAR-100-C conditions, relative trajectory drift and top-1 error have Pearson correlation r=0.73r=0.73 and Spearman correlation ρs=0.78\rho_s=0.78. Both quantities generally increase with corruption severity, so we additionally test whether the association remains when severity is held fixed.

Within each severity level, Pearson correlation remains positive and ranges from 0.59 to 0.71 across the fifteen corruption families. After subtracting the corresponding severity-level mean from both drift and error, their residual Pearson correlation is r=0.63r=0.63. The corresponding partial Spearman correlation is ρs=0.67\rho_s=0.67.

The five severity levels of a corruption family are related and should not be treated as independent observations. We therefore resample complete corruption-family profiles, retaining all five severities whenever a family is selected. This family-cluster bootstrap gives a 95% confidence interval of [0.39, 0.83] for the severity-controlled Pearson correlation. A complementary permutation test that reassigns complete five-severity drift profiles across corruption families gives p<0.01p<0.01. Thus, the observed association is not explained only by the shared increase of drift and error with corruption severity.

ImageNet-C. Figure 5 repeats the analysis on ImageNet-C, using fog as the representative corruption family in panels (a) and (e). Across the 75 conditions, relative trajectory drift has Pearson correlation r=0.78r=0.78 and Spearman correlation ρs=0.87\rho_s=0.87 with top-1 error. Within-severity Pearson correlations range from 0.52 to 0.69. After removing severity-level means, the residual Pearson correlation is r=0.53r=0.53, while the partial Spearman correlation is ρs=0.69\rho_s=0.69. The family-cluster confidence interval for the severity-controlled Pearson correlation is [0.19, 0.77], and the corresponding family-profile permutation test gives p<0.01p<0.01.

Class-Structure Visualizations under Adaptation

To visualize how adaptation changes the class structure of the representations used for classification, we project these features to two dimensions using UMAP and t-SNE and color each point by its true class. For each backbone, we show five model states: No Adaptation, source-free Drift-TTA without and with online gate adaptation, and source-dependent Drift-TTA without and with online gate adaptation. All panels use the complete CIFAR-10-C test set of 10,000 images at severity 5 under the leave-one-family-out protocol. We show six corruption families spanning noise, blur, weather, and digital corruptions.

Each panel is projected independently, so absolute positions and axis directions should not be compared across panels. The intended comparison is the compactness and separation of the class distributions within each panel. UMAP and t-SNE provide complementary views and show the same qualitative progression: relative to No Adaptation, the fixed Drift-TTA adapters generally produce more compact and better-separated class distributions, while online gate adaptation further improves this structure. Figure 6 and Figure 7 show the CLIP image tower, while Figure 8 and Figure 9 show the ViT.

Figure 6. UMAP of CLIP ViT-B/16 image embeddings on CIFAR-10-C (severity 5), colored by class. Rows are six corruption families; columns are, left to right, No Adaptation, source-free Drift-TTA without and with the online gate, and source-dependent Drift-TTA without and with the online gate. Column groups are color-coded (teal: no adaptation; purple: source-free; orange: source-dependent). Points are the L2L_2-normalized image embedding used for zero-shot classification, projected with UMAP (Euclidean metric); each panel is embedded independently. Leave-one-family-out: each column uses only the adapter trained without the family shown in that row.

Figure 7. t-SNE of CLIP ViT-B/16 image embeddings on CIFAR-10-C (severity 5), colored by class. Rows are six corruption families; columns are, from left to right, No Adaptation, source-free Drift-TTA without and with online gate adaptation, and source-dependent Drift-TTA without and with online gate adaptation. Points are the L2L_{2}-normalized image embeddings used for zero-shot classification, projected independently with t-SNE using Euclidean distance and perplexity 30. Layout, color coding, and the leave-one-family-out protocol match Figure 6.

Figure 8. UMAP of ViT image features on CIFAR-10-C (severity 5), colored by class. Layout, color coding, and protocol match Figure 6; here the points are the pre-classifier pooled embedding of the ViT. As with CLIP, both Drift-TTA priors tighten the class structure and the online gate sharpens it further.

Figure 9. t-SNE of ViT image features on CIFAR-10-C (severity 5), colored by class. Layout, color coding, projection settings, and protocol match Figure 7; here, the points are the L2L_{2}-normalized pre-classifier pooled embeddings of the ViT. The class structure follows the same qualitative progression as the corresponding UMAP visualization in Figure 8.

Per-Family Trajectory-Feature Distributions

This section extends the trajectory-feature view of Figure 1(b) from a single case to individual corruption families. Each image reduces every input to two scalar summaries of its hidden trajectory, the magnitude feature Mag⁡(H)\operatorname{Mag}(H) and the angle feature Ang⁡(H)\operatorname{Ang}(H), and draws kernel-density contours of the clean reference (green) against corruption severities 1–5 (columns). The three rows are, top to bottom, the unadapted model (No Adaptation), the frozen offline Drift-TTA adapter, and the same adapter with the online entropy gate enabled. All densities use CIFAR-10-C, CIFAR-100-C, or ImageNet-C at the stated severities under the leave-one-family-out protocol, so each panel reports its family using only the adapter that never saw that family.

The intended reading is spatial. In the examples shown, the corrupted density moves away from the clean contours as severity increases, while the offline adapter shifts it back toward clean and the online gate further strengthens this correction. We show one illustrative corruption family for each of the twelve backbone×\timesprior×\timesdataset settings, covering supervised ViT and CLIP, source-free and source-dependent training, and all three datasets.

Figure 10. Trajectory-feature distributions, supervised ViT (source-free). Each panel plots Mag⁡(H)\operatorname{Mag}(H) (horizontal) against Ang⁡(H)\operatorname{Ang}(H) (vertical); green is the clean reference and colored contours are severities 1–5 (columns), while the three rows are No Adaptation, offline Drift-TTA, and online Drift-TTA. Top: glass blur on CIFAR-10-C; bottom: shot noise on CIFAR-100-C. Leave-one-family-out.

Figure 11. Trajectory-feature distributions, supervised ViT. Green is the clean reference and colored contours are severities 1–5 (columns); the three rows are No Adaptation, offline Drift-TTA, and online Drift-TTA. Top: snow on ImageNet-C (source-free); bottom: pixelate on CIFAR-10-C (source-dependent). Leave-one-family-out.

Figure 12. Trajectory-feature distributions, supervised ViT (source-dependent). Green is the clean reference and colored contours are severities 1–5 (columns); the three rows are No Adaptation, offline Drift-TTA, and online Drift-TTA. Top: impulse noise on CIFAR-100-C; bottom: fog on ImageNet-C. Leave-one-family-out.

Figure 13. Trajectory-feature distributions, CLIP (source-free). Green is the clean reference and colored contours are severities 1–5 (columns); the three rows are No Adaptation, offline Drift-TTA, and online Drift-TTA. Top: motion blur on CIFAR-10-C; bottom: elastic transform on CIFAR-100-C. Leave-one-family-out.

Figure 14. Trajectory-feature distributions, CLIP. Green is the clean reference and colored contours are severities 1–5 (columns); the three rows are No Adaptation, offline Drift-TTA, and online Drift-TTA. Top: zoom blur on ImageNet-C (source-free); bottom: defocus blur on CIFAR-10-C (source-dependent). Leave-one-family-out.

Figure 15. Trajectory-feature distributions, CLIP (source-dependent). Green is the clean reference and colored contours are severities 1–5 (columns); the three rows are No Adaptation, offline Drift-TTA, and online Drift-TTA. Top: Gaussian noise on CIFAR-100-C; bottom: contrast on ImageNet-C. Leave-one-family-out.

References

  1. [1]Motasem Alfarra, Alvaro Correia, Bernard Ghanem, and Christos Louizos. Test-time adaptation with source based auxiliary tasks. Transactions on Machine Learning Research, 2026.
  2. [2]Wenxuan Bao, Ruxi Deng, and Jingrui He. Mint: A simple test-time adaptation of vision-language models against common corruptions. In Advances in Neural Information Processing Systems (NeurIPS), volume 38, 2025a.
  3. [3]Wenxuan Bao, Ruxi Deng, Ruizhong Qiu, Tianxin Wei, Hanghang Tong, and Jingrui He. Latte: Collaborative test-time adaptation of vision-language models in federated learning. In IEEE/CVF International Conference on Computer Vision (ICCV), pages 133–143, 2025b.
  4. [4]Dian Chen, Dequan Wang, Trevor Darrell, and Sayna Ebrahimi. Contrastive test-time adaptation. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 295–305, 2022.arxiv.org/abs/2204.10377
  5. [5]Younggeol Cho, Youngrae Kim, Junho Yoon, Seunghoon Hong, and Dongman Lee. Feature augmentation based test-time adaptation. In IEEE/CVF Winter Conference on Applications of Computer Vision (WACV), pages 6838–6847, 2025.arxiv.org/abs/2410.14178
  6. [6]Wonjeong Choi, Do-Yeon Kim, Jungwuk Park, Jungmoon Lee, Younghyun Park, Dong-Jun Han, and Jaekyun Moon. Adaptive energy alignment for accelerating test-time adaptation. In International Conference on Learning Representations (ICLR), 2025.
  7. [7]Qi Deng, Shuaicheng Niu, Ronghao Zhang, Yaofo Chen, Runhao Zeng, Jian Chen, and Xiping Hu. Learning to generate gradients for test-time adaptation via test-time training layers. In AAAI Conference on Artificial Intelligence (AAAI), volume 39, pages 16235–16243, 2025.arxiv.org/abs/2412.16901
  8. [8]Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. On calibration of modern neural networks. In International Conference on Machine Learning (ICML), volume 70, pages 1321–1330, 2017. Dan Hendrycks and Thomas Dietterich. Benchmarking neural network robustness to common corruptions and perturbations. In International Conference on Learning Representations (ICLR), 2019.
  9. [9]Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations (ICLR), 2022.arxiv.org/abs/2106.09685
  10. [10]Adilbek Karmanov, Dayan Guan, Shijian Lu, Abdulmotaleb El Saddik, and Eric Xing. Efficient test-time adaptation of vision-language models. In IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 14162–14171, 2024.arxiv.org/abs/2403.18293
  11. [11]Jae-Hong Lee and Joon-Hyuk Chang. Continual momentum filtering on parameter space for online test-time adaptation. In International Conference on Learning Representations (ICLR), 2024.
  12. [12]Mingxi Lei, Chunwei Ma, Meng Ding, Yufan Zhou, Ziyun Huang, and Jinhui Xu. TTVD: Towards a geometric framework for test-time adaptation based on voronoi diagram. In International Conference on Learning Representations (ICLR), 2025.
  13. [13]Jian Liang, Ran He, and Tieniu Tan. A comprehensive survey on test-time adaptation under distribution shifts. International Journal of Computer Vision, pages 31–64, 2025.DOI
  14. [14]Xiao Ma, Young D. Kwon, Pan Zhou, and Dong Ma. Architecture-agnostic test-time adaptation via backprop-free embedding alignment. In International Conference on Learning Representations (ICLR), 2026.
  15. [15]Sarthak Kumar Maharana, Baoming Zhang, and Yunhui Guo. PALM: Pushing adaptive learning rate mechanisms for continual test-time adaptation. In AAAI Conference on Artificial Intelligence (AAAI), volume 39, pages 19378–19386, 2025a.
  16. [16]Sarthak Kumar Maharana, Baoming Zhang, Leonid Karlinsky, Rogerio Feris, and Yunhui Guo. BATCLIP: Bimodal online test-time adaptation for CLIP. In IEEE/CVF International Conference on Computer Vision (ICCV), pages 1569–1579, 2025b.DOI
  17. [17]Alexander Murphy, Michal Danilowski, Soumyajit Chatterjee, and Abhirun Ghosh. NEO — no-optimization test-time adaptation through latent re-centering. In International Conference on Learning Representations (ICLR), 2026.arxiv.org/abs/2510.05635
  18. [18]Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang, Yaofo Chen, Shijian Zheng, Peilin Zhao, and Mingkui Tan. Efficient test-time model adaptation without forgetting. In International Conference on Machine Learning (ICML), pages 16888–16905. PMLR, 2022.arxiv.org/abs/2204.02610
  19. [19]Shuaicheng Niu, Jiaxiang Wu, Yifan Zhang, Zhiquan Wen, Yaofo Chen, Peilin Zhao, and Mingkui Tan. Towards stable test-time adaptation in dynamic wild world. In International Conference on Learning Representations (ICLR), 2023.arxiv.org/abs/2302.12400
  20. [20]Shuaicheng Niu, Chunyan Miao, Guohao Chen, Pengcheng Wu, and Peilin Zhao. Test-time model adaptation with only forward passes. In International Conference on Machine Learning (ICML), volume 235. PMLR, 2024.arxiv.org/abs/2404.01650
  21. [21]Alec Radford, Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, and Ilya Sutskever. Learning transferable visual models from natural language supervision. In International Conference on Machine Learning (ICML), 2021.arxiv.org/abs/2103.00020
  22. [22]Manli Shu, Weili Nie, De-An Huang, Zhiding Yu, Tom Goldstein, Anima Anandkumar, and Chaowei Xiao. Test-time prompt tuning for zero-shot generalization in vision-language models. Advances in Neural Information Processing Systems, 35:14274–14289, 2022.
  23. [23]Jingwei Sun, Jianing Zhu, Jiangchao Yao, Gang Niu, Masashi Sugiyama, and Bo Han. Bilateral information-aware test-time adaptation for vision-language models. In International Conference on Learning Representations (ICLR), 2026.

Paper details

vlnc:W_xd7qv7m7
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 10

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

generative AI tools

View statement
Looking for related papers…
Contents