betaFeedback

WorksComputer Science

Inherited Adapter Experts for Replay-Free Domain-Incremental Learning

Abstract

Replay-free domain-incremental learning trains a model on the same classes in successive domains without keeping earlier images. Per-domain learners freeze a pre-trained ViT and train an additional expert for each new domain: the small set of parameters and the classification head learned on that domain. Test images carry no domain label, so a router selects one expert for each image. Recent work trains this router. However, a training-free Mahalanobis router costs the analyzed learners under 2.7 points of final accuracy relative to always selecting each image's own-domain expert. The experts, not this router, limit accuracy. Our learner, Lamarck, therefore uses this router and strengthens the experts. Its adapters, small trainable layers that correct every image token after each deep ViT block, add 4.4 points of final accuracy on the deepfake benchmark CDDB-Hard. In addition, each expert is initialized from its predecessor at no extra cost: the domains share their classes, so an image misrouted to a later expert loses less. Against twelve frozen-backbone baselines under one protocol, Lamarck is first on CDDB-Hard by 5.2 points, second on DomainNet and OfficeHome, and highest in mean accuracy.

Introduction

Domain-incremental learning (DIL) trains one model on a fixed set of classes in successive stages. Each stage brings a new kind of image, called a domain [29, 32]. For example, a bicycle is learned from clipart, then doodles, then photographs, and a deepfake detector learns each new source of fake images [15]. We study the replay-free setting, in which no image of an earlier domain is kept; only model parameters and feature statistics remain. Training on a new domain then lowers accuracy on the earlier ones (forgetting), and a test image does not say which domain it comes from.

Prompt-based continual learning (PCL) freezes a pre-trained ViT and trains a classification head and prompts. Prompts are a few learnable vectors that the frozen blocks attend to as extra tokens [36, 37]. Many prompt learners retrieve prompts for each image from a prompt pool by matching the image’s feature to learned keys [13, 25, 37]. Prompt learners differ in what they share across domains. Shared-pool learners train one pool and one head on every domain in turn, so forgetting can remain: on DomainNet, CODA-Prompt [25] averages 71.3% right after learning a domain but keeps 50.6% after the last (Sec. S7).

To avoid this forgetting, per-domain learners train an additional expert [1, 3] for each new domain: the small set of parameters and the head learned on that domain. Each expert is frozen and kept, so later training cannot lower its accuracy. An S-iPrompt [35] expert has one prompt and one head, and a SOYO [34] expert adds adapters, small trainable layers after all twelve blocks. Since the domain of a test image is unknown, a router (a domain identifier or selector in prior work) selects one expert for each image, and only that expert classifies it. Such a learner can thus make two kinds of error: the router selects another expert, or the image’s own-domain expert misclassifies it. Subsequent per-domain learners target the first kind of error by training the router [20, 31, 34]. Meanwhile, the nine prompt learners we compare place prompts at the input or in the early blocks (1–5) and train nothing in the deep blocks (6–12).

We run every expert of S-iPrompt and SOYO on every test image (§ 5). A training-free router built from feature statistics costs them under 2.7 points of final accuracy (ACC: after the last domain, averaged over domains) relative to always selecting each image’s own-domain expert. It also gives both learners higher ACC than their own routers on all three benchmarks (Table 1 and Table 2). This router picks the domain whose mean of frozen-ViT features is nearest in Mahalanobis distance [14]. It misroutes 3.6 to 16.2% of the test images, but the expert it selects predicts the same classes and is often still correct. The cost of routing is thus roughly the misrouting rate times the accuracy an image loses at the wrong expert. The experts, not this router, limit accuracy, even for our learner, Lamarck (Figure 1, left). Under this router on DomainNet, SOYO’s experts score 3.4 ACC points higher than S-iPrompt’s, more than a perfect router would add to either learner. The gap is uneven: in own-domain accuracy, the experts are nearly equal on real photographs but far apart on QuickDraw doodles (Table S2).

Figure 1. Most of the final error is expert error (left); inheritance lowers the accuracy lost at later experts (right). Left: Lamarck’s final error (100−final accuracy) on DomainNet (DN), OfficeHome (OH) and CDDB-Hard (CDDB), split into the error of each image’s own-domain expert (Expert) and the error added by the Mahalanobis router (Routing). Dashed (Prompts only): Lamarck without adapters. Right: later-expert cost (Eq. (5)), the accuracy a domain’s images lose at an expert trained after their own domain. Shown for Lamarck, Lamarck with only the adapters re-initialized for every domain (Reset adapters), Lamarck trained as independent experts (Independent) and SOYO [34]. Five-seed test means.

DNOHCDDB
Expert bankDRA↑Mah.↑Proto.↑SE↓DRA↑Mah.↑Proto.↑SE↓DRA↑Mah.↑Proto.↑SE↓
S-iPrompt69.26±0.0866.79±0.0864.95±0.1423.43±0.1088.81±0.2387.89±0.1986.64±0.386.28±0.2286.99±2.6587.12±1.9784.54±1.151.27±1.09
SOYO72.81±0.0770.19±0.0868.31±0.1719.96±0.0687.51±0.5086.36±0.4684.61±0.626.50±0.3488.15±1.2386.46±1.3584.17±1.293.64±1.83
Lamarck, prompts only69.72±0.1667.32±0.1665.62±0.3222.90±0.1188.91±0.3388.14±0.2287.21±0.146.91±0.1189.35±1.1888.57±0.9287.10±1.323.27±0.51
Lamarck (Ours)71.93±0.1269.46±0.1467.75±0.1320.59±0.1389.29±0.2488.47±0.2587.43±0.256.22±0.3094.78±0.5392.94±0.6291.61±0.731.80±0.22

Table 1. Under the Mahalanobis router, the routing loss is smaller than the ACC spread across expert banks. ACC of each expert bank under two routers on the same frozen features: Mahalanobis (Mah., Eq. (8)) and prototype (Proto.; cosine similarity to eight Gaussian-mixture means per domain). DRA (Eq. (4)): domain labels select own-domain experts. Routing loss = DRA−Mah. SE: share of images that every stored expert misclassifies. TEST, five-seed mean±s.d. Table 2 reports S-iPrompt and SOYO with their own routers.

MethodVenueDN AIA↑DN ACC↑OH AIA↑OH ACC↑CDDB AIA↑CDDB ACC↑Mean AIA↑Mean ACC↑
Joint linear probe––64.08±0.0464.08 \pm0.04–86.95±0.3786.95 \pm0.37–80.63±0.3480.63 \pm0.34–77.22
Shared-pool learners: one prompt pool and classifier shared by all domains
CODA-Prompt [25]CVPR 202354.55±0.1054.55 \pm0.1050.62±0.1450.62 \pm0.1484.46±0.2184.46 \pm0.2186.62±0.2386.62 \pm0.2387.04±0.5887.04 \pm0.5872.92±2.3372.92 \pm2.3375.3570.05
OS-Prompt [13]ECCV 202452.60±0.1252.60 \pm0.1249.74±0.1949.74 \pm0.1984.22±0.2784.22 \pm0.2786.31±0.5186.31 \pm0.5186.49±0.4386.49 \pm0.4372.84±2.1472.84 \pm2.1474.4469.63
OS-Prompt++ [13]ECCV 202452.55±0.0752.55 \pm0.0749.74±0.2349.74 \pm0.2384.26±0.3184.26 \pm0.3186.33±0.5186.33 \pm0.5186.53±0.3986.53 \pm0.3972.69±2.2172.69 \pm2.2174.4569.59
KA-Prompt [39]ICML 202549.84±0.0349.84 \pm0.0350.60±0.0450.60 \pm0.0483.73±0.4783.73 \pm0.4785.90±0.2285.90 \pm0.2281.84±0.3381.84 \pm0.3374.96±0.3974.96 \pm0.3971.8070.49
CPS-Prompt [17]CVPR 202653.64±0.1053.64 \pm0.1049.97±0.1449.97 \pm0.1483.96±0.3583.96 \pm0.3585.84±0.4285.84 \pm0.4285.75±0.7185.75 \pm0.7171.45±1.4771.45 \pm1.4774.4569.09
UniPrompt-CL [19]WACV 202751.77±0.2051.77 \pm0.2047.55±0.2947.55 \pm0.2985.61±0.2785.61 \pm0.2787.03±0.3487.03 \pm0.3489.42±0.5189.42 \pm0.5176.14±0.8976.14 \pm0.8975.6070.24
Per-domain learners: parameters trained and kept for each domain
S-iPrompt [35]NeurIPS 202265.51±0.1065.51 \pm0.1065.81±0.0365.81 \pm0.0384.71±0.3784.71 \pm0.3786.99±0.4086.99 \pm0.4090.23±0.8190.23 \pm0.8185.60±1.9285.60 \pm1.9280.1579.47
HiDe-Prompt [31]NeurIPS 202364.54±0.0864.54 \pm0.0864.51±0.0864.51 \pm0.0885.92±0.3685.92 \pm0.3688.12±0.2188.12 \pm0.2191.14±0.3491.14 \pm0.3486.39±0.6186.39 \pm0.6180.5379.67
EASE [41]CVPR 202462.09±0.0362.09 \pm0.0361.36±0.0261.36 \pm0.0285.00±0.1085.00 \pm0.1086.66±0.0886.66 \pm0.0879.79±0.7779.79 \pm0.7767.14±0.7967.14 \pm0.7975.6371.72
SOYO [34]CVPR 202569.33±0.0869.33 \pm0.0869.91±0.0869.91 \pm0.0882.91±0.8582.91 \pm0.8585.05±0.6085.05 \pm0.6089.96±0.4989.96 \pm0.4979.24±0.9579.24 \pm0.9580.7378.07
Hash [16]CVPR 202662.42±0.6162.42 \pm0.6158.89±0.7058.89 \pm0.7086.31±0.2086.31 \pm0.2088.64±0.1288.64 \pm0.1292.37±0.3692.37 \pm0.3687.75±0.8887.75 \pm0.8880.3778.43
CONEC-LoRA [20]arXiv 202667.97±0.0567.97 \pm0.0568.15±0.1368.15 \pm0.1386.28±0.2386.28 \pm0.2387.80±0.1687.80 \pm0.1690.31±1.4590.31 \pm1.4579.96±2.3479.96 \pm2.3481.5278.64
Lamarck (Ours)–68.89±0.2068.89 \pm0.2069.46±0.1469.46 \pm0.1485.85±0.6385.85 \pm0.6388.47±0.2588.47 \pm0.2597.16±0.1797.16 \pm0.1792.94±0.6292.94 \pm0.6283.9783.62

Table 2. Lamarck has the best ACC among frozen-backbone learners on CDDB and averaged over the three benchmarks, and the second best on DN and OH. Five-seed TEST accuracy (%, mean±s.d.). For a fair comparison, all methods are trained under one protocol (§ 6.1), each with its own inference rule. Bold, underlined: best and second best. The joint linear probe, trained on all domains at once, is a reference outside this comparison. SLCA [40] and DUCT [43], which fine-tune the backbone, are in Table S5.

DomainS-iPromptSOYOLamarck, prompts onlyLamarck (Ours)
DomainNet
Clipart77.86±0.1377.86 \pm0.1380.85±0.2580.85 \pm0.2577.89±0.1577.89 \pm0.1580.34±0.3180.34 \pm0.31
Infograph43.54±0.1943.54 \pm0.1946.76±0.1546.76 \pm0.1543.54±0.1943.54 \pm0.1946.11±0.4846.11 \pm0.48
Painting74.20±0.1074.20 \pm0.1076.08±0.1876.08 \pm0.1874.54±0.2374.54 \pm0.2375.55±0.1275.55 \pm0.12
QuickDraw63.86±0.2263.86 \pm0.2272.29±0.1872.29 \pm0.1864.12±0.1164.12 \pm0.1169.79±0.0969.79 \pm0.09
Real86.88±0.0486.88 \pm0.0487.41±0.0487.41 \pm0.0487.21±0.1987.21 \pm0.1987.17±0.0987.17 \pm0.09
Sketch69.22±0.2069.22 \pm0.2073.47±0.1473.47 \pm0.1471.00±0.1671.00 \pm0.1672.64±0.1772.64 \pm0.17
OfficeHome
Art83.88±0.8983.88 \pm0.8981.55±1.3481.55 \pm1.3483.25±1.2583.25 \pm1.2583.79±1.4883.79 \pm1.48
Clipart82.38±0.6782.38 \pm0.6781.73±1.2481.73 \pm1.2483.12±0.4983.12 \pm0.4984.05±0.4784.05 \pm0.47
Product96.20±0.3096.20 \pm0.3095.37±0.3295.37 \pm0.3295.98±0.2095.98 \pm0.2096.12±0.4596.12 \pm0.45
Real World92.77±0.4992.77 \pm0.4991.40±0.4691.40 \pm0.4693.31±0.4693.31 \pm0.4693.19±0.4293.19 \pm0.42
CDDB-Hard
GauGAN97.85±0.3897.85 \pm0.3899.78±0.1099.78 \pm0.1098.08±0.4898.08 \pm0.4899.58±0.1199.58 \pm0.11
BigGAN91.75±0.7291.75 \pm0.7299.31±0.4199.31 \pm0.4194.62±0.6094.62 \pm0.6099.31±0.3499.31 \pm0.34
Wild76.95±1.2876.95 \pm1.2887.84±0.6487.84 \pm0.6487.16±1.0587.16 \pm1.0592.29±0.3492.29 \pm0.34
WFIR89.62±0.9589.62 \pm0.9572.00±5.8572.00 \pm5.8591.12±1.2091.12 \pm1.2097.88±0.7197.88 \pm0.71
SAN78.79±11.9378.79 \pm11.9381.82±0.0081.82 \pm0.0075.76±4.7975.76 \pm4.7984.85±2.1484.85 \pm2.14

Table S2. Own-domain accuracy (%) ↑. As Table 3, with S-iPrompt; TEST, mean±s.d. over five seeds. WFIR: WhichFaceIsReal.

Lamarck, a per-domain learner, therefore uses this router and strengthens the experts. Each expert retrieves prompts from its own prompt pool in the early blocks, as prompt learners do. After each deep block, an adapter [4, 8] adds a learned correction to every image token. Against the same experts without adapters (Lamarck’s prompt-only experts), the adapters leave own-domain accuracy on DomainNet’s real photographs unchanged but raise it by 5.7 points on QuickDraw doodles. They raise ACC on all three benchmarks, from 0.3 points on OfficeHome to 4.4 on the deepfake benchmark CDDB-Hard (Table 1 and Table 4). On CDDB-Hard, deep-block prompts with as many parameters as the adapters recover most of this gain, and the adapter form adds the rest (§ 6.3).

Expert bankACC↑ OHACC↑ CDDBLater-expert cost↓ OHLater-expert cost↓ CDDB
Lamarck (Ours)88.47±0.2592.94±0.625.04±0.4815.75±0.84
w/o adapters−0.33±0.19−4.36±1.27−1.08±0.45−2.18±1.76
w/o prompts+0.16±0.28−0.23±0.70+0.20±0.48−0.48±2.43
w/o adapter inheritance (reset)−0.15±0.15−0.21±0.39+1.51±0.63+12.06±1.62
w/o inheritance (independent)−0.54±0.28−0.58±0.75+8.37±0.78+22.78±1.43
w/o adapters and inheritance−0.95±0.19−3.51±0.66+7.55±0.54+19.74±0.94

Table 4. Removing the adapters lowers ACC most on CDDB, and removing inheritance raises the later-expert cost. Top row: Lamarck. Other rows: per-seed change after removing components. ACC: Mahalanobis router. Later-expert cost: Eq. (5). TEST, five-seed mean±s.d.

No earlier image is kept, but the previous expert is. Because all domains share the classes, this expert is already a classifier for the next domain. Lamarck therefore initializes each expert, adapters included, from its predecessor, as HiDe-Prompt [31] does for its prompts. This inheritance (hence the name) adds no parameters or computation. Experts trained later are then more accurate on earlier domains than independently trained ones (82.1% against 59.4% on CDDB-Hard; chance: 50%), so an image misrouted to a later expert loses less (Figure 1, right). On CDDB-Hard, the inherited adapters account for about half of this advantage (§ 6.4). Inheritance matters more as more images reach later experts. Under the Mahalanobis router, which misroutes few such images, inherited experts lead independent ones by at most 0.6 ACC points. With a weaker router fitted on two images per domain, they lead by 1.7 to 4.4.

Lamarck thus combines a training-free router, adapter experts and inheritance (Figure 2). Like Lamarck, SOYO keeps adapter experts, but it trains its router and independent experts. We compare Lamarck with twelve replay-free learners trained under one protocol on the same frozen ViT, over five seeds on DomainNet (DN), OfficeHome (OH) and CDDB-Hard (CDDB). Lamarck leads on CDDB by 5.2 ACC points, is second on DN and OH, and has the highest mean ACC, 83.6 against 79.7 for HiDe-Prompt (Table 2). Inference runs the ViT twice per image, once to route and once to classify, as in other two-pass learners (Figure 3).

Figure 2. Lamarck. (a) One expert on the frozen backbone. Prompts retrieved from the expert’s prompt pool enter the early blocks, and after each deep block a residual adapter corrects every image token. An MLP readout on the final CLS feature computes the logits. (b) Expert tt is initialized from expert t−1t-1, trained on domain tt and then frozen. (c) A training-free router selects the domain whose mean feature is nearest to the frozen-ViT feature z(x)z(x) in shrunk Mahalanobis distance, and only that domain’s expert classifies xx.

Figure 3. Lamarck has the highest mean ACC at the inference cost of a two-pass learner. Mean ACC over the three benchmarks (Table 2) against GFLOPs per image on DN (log scale). Throughput in Table S9 falls as GFLOPs rise (rank correlation −0.94). SOYO costs most: its released inference runs the ViT once per domain.

Our contributions are:

  • We show on three benchmarks that a training-free Mahalanobis router costs the per-domain learners S-iPrompt, SOYO and Lamarck under 2.7 ACC points. On each benchmark, this cost is less than the largest ACC difference between their experts under that router.

  • We propose Lamarck, whose experts have prompts in the early blocks and adapters on every image token after each deep block, and test their form, tokens and placement.

  • We show that inheritance, adapters included, adds no parameters or computation. It makes later experts more accurate on earlier domains, so its gain grows as a router misroutes more images to them.

Related Work

Prompt-based continual learning. L2P [37] queries a shared pool of input prompts with the image’s feature from a separate, unprompted pass of the frozen ViT. Dual-Prompt [36] places prompts as attention prefixes in blocks chosen by search: a shared prompt in blocks 1–2 and a per-task prompt in blocks 3–5. Subsequent learners prompt the same blocks, mostly with prefixes [13, 17, 19, 25, 39]. CODA-Prompt, KA-Prompt and CPS-Prompt weight or select prompts with the unprompted-pass query, whereas OS-Prompt, OS-Prompt++ and UniPrompt-CL use the class token of the prompted pass. None trains parameters in the deep blocks. Lamarck’s experts use OS-Prompt’s query, but in its last prompt-only expert, retrieval is nearly input-independent (§ 6.3), in line with single-prompt results in class-incremental [10] and online task-free [22] settings.

Expert retention and routing. S-iPrompt [35] routes its input-prompt experts by the nearest k-means centre of frozen features. SOYO [34] pairs an input prompt with adapters after all twelve blocks and trains its router on features resampled from a Gaussian mixture. G2D [3] trains its router on generated data. Others propose different experts or routers [28, 33, 38]. In Hash [16], the “router” gates a shared prompt pool per task. Like many routers [21, 28, 35], Lamarck’s router uses statistics of frozen features in a shrunk Mahalanobis score [6, 14]. Prior work separates within-task and task-identity prediction [12, 31], reports domain-identification accuracy and accuracy with domain labels [20, 34, 35], or evaluates every prompt with every head [2]. Because all DIL experts predict the same classes, a misrouted image can still be classified correctly. We therefore measure and lower the accuracy lost per misroute.

Adapters and feature adaptation. Prompts [11] add tokens to a frozen network’s input, whereas adapters and low-rank updates change the transformation it applies: residual bottleneck adapters [8], their parallel form for ViTs in AdaptFormer [4], and LoRA [9]. Prefix tuning can be written as an adapter on the attention output [7], but cannot change the relative attention among image tokens in its layer [24]. Adapter-based continual learners combine or select per-task adapters [26, 41] or adapt a single model [18, 27, 42], mostly for a growing label space. For DIL, CONEC-LoRA [20] keeps per-domain low-rank updates in blocks 7–12 and trains its router. DUCT [43] instead consolidates a fine-tuned backbone and its head across domains. Among our baselines, SOYO, EASE [41] and CONEC-LoRA train per-domain parameters in the deep blocks, like Lamarck. Adapters in all blocks (1–12) change Lamarck’s ACC by at most 0.3 (Table 5 and Table S10).

VariantACC↑ OHACC↑ CDDBLater-expert cost↓ OHLater-expert cost↓ CDDB
Lamarck (Ours), r=32r=3288.47±0.2592.94±0.625.04±0.4815.75±0.84
CLS token only (r=64r=64)88.15±0.6392.30±0.724.60±0.8923.53±1.52
Blocks 1–12 (r=16r=16)88.54±0.4093.20±0.975.41±0.6613.17±0.78
Deep prompts, no adapters88.22±0.2691.74±0.194.96±0.2816.59±2.63

Table 5. Adapters on every image token give higher ACC than CLS-only adapters or prompts in the deep blocks. Each variant changes only Lamarck’s adapters (rr: rank) and still uses inheritance. The last row replaces the adapters with 64 prompt tokens in each deep block, matching their parameter count. Columns as in Table 4. TEST, five-seed mean±s.d.

BenchmarkAdaptersVAL ACC↑TEST ACC↑Later-expert cost↓
DomainNetr=8r=869.11 ± 0.1869.18 ± 0.0431.43
r=32r=3269.21 ± 0.1169.46 ± 0.1431.47
r=64r=6469.37 ± 0.0869.56 ± 0.0531.34
Blocks 1–12–69.49 ± 0.0531.46
OfficeHomer=8r=888.95 ± 0.1888.50 ± 0.494.79
r=32r=3289.06 ± 0.5888.47 ± 0.255.04
r=64r=6489.11 ± 0.4188.43 ± 0.205.18
Blocks 1–12–88.54 ± 0.405.41
CDDB-Hardr=8r=884.33 ± 0.6691.93 ± 1.3512.42
r=32r=3284.70 ± 0.6092.94 ± 0.6215.75
r=64r=6485.27 ± 0.9892.63 ± 0.2417.61
Blocks 1–12–93.20 ± 0.9713.17

Table S10. Accuracy depends little on the rank and placement of the adapters. Rows r=8r=8, 32, 64 (post-hoc sweep): Lamarck’s adapters in blocks 6–12 at rank rr (Lamarck uses r=32r=32); Blocks 1–12: adapters of rank 16 in all twelve blocks, with Lamarck’s prompts and inheritance kept. ACC under the Mahalanobis router on VAL and TEST (%, mean±s.d. over five seeds) and TEST later-expert cost (Eq. (5); five-seed mean); –: not computed.

Function-preserving initialization. Expert Gate [1] initializes a new expert from the most related previous expert, and HiDe-Prompt [31] initializes each task’s prompt from the previous one. Net2Net [5] initializes a larger model with exactly the function of a smaller one. Likewise, LoRA zero-initializes one of its two factors [9], so training starts from the pre-trained function. Lamarck applies both ideas to whole experts, and § 6.4 measures the effect.

Preliminaries

Replay-free domain-incremental learning. A learner receives the training sets D1,…,DTD_1, \ldots, D_T of TT domains in sequence. Each DtD_t is drawn from Pt(x,y)P_t(x, y) over images xx and labels y∈Yy \in\mathcal{Y}. All domains share the label set Y\mathcal{Y} and differ in their input distributions Pt(x)P_t(x). At stage tt only DtD_t is available: from earlier stages, the learner may keep model parameters and feature statistics but no image. Stage boundaries are known during training, but test images carry no domain label. After every stage, the learner must classify images of all seen domains.

Experts and routers. An expert ψe\psi_e consists of the parameters and classification head trained on domain ee and then frozen. With the frozen ViT backbone Θ\Theta (M=12M=12 blocks), ψe\psi_e is a complete classifier over Y\mathcal{Y}. A per-domain learner keeps its experts in an expert bank and, after stage tt, predicts

y^(x)=hρt(x)(x),ρt(x)∈{1,…,t},(1)\hat{y}(x) = h_{\rho_t(x)}(x), \qquad\rho_t(x) \in\{1,\ldots,t\}, \tag*{(1)}

where the router ρt\rho_t selects one expert per image and he(x)h_e(x) is the class predicted by expert ee.

Prompts and prompt-only experts. A prompt is a few learnable vectors that a frozen block attends to as extra tokens, in the token sequence [11] or as a prefix in its attention [36]. A prompt pool holds key-prompt pairs as rows. Prompt retrieval weights the rows by matching a feature of the image, the query, to the keys. A prompt-only expert has only prompts and a head. In Lamarck’s prompt-only expert, the first NN blocks receive prompts, retrieved with the query of OS-Prompt [13]. At block i≤Ni \le N, the CLS token QiQ_i of the incoming sequence fif_i queries a pool of ZZ rows of keys and one-token prompts, Ki,Pi∈RZ×768K_i, P_i \in\mathbb{R}^{Z \times768}. Each stage makes ⌊Z/T⌋\lfloor Z/T \rfloor new rows visible, plus the remainder at the last stage. Retrieval at stage tt uses all ZtZ_t visible rows, including the frozen rows of earlier stages. With cosine similarity γ\gamma, the retrieved prompt

ϕi=∑k=1Ztexp⁡γ(Qi,Ki,k)∑k′exp⁡γ(Qi,Ki,k′)Pi,k(2)\phi_i = \sum_{k=1}^{Z_t} \frac{\exp\gamma(Q_i,K_{i,k})}{\sum_{k'} \exp\gamma(Q_i,K_{i,k'})} P_{i,k} \tag*{(2)}

is prepended to the sequence, fi+1=Θi([ϕi,fi])f_{i+1} = \Theta_i([\phi_i, f_i]), and stays there, as in UniPrompt-CL [19]. Through attention, the prompt tokens change which features the frozen blocks combine, not the transformation each block applies.

Metrics. Let At,jA_{t,j} be the accuracy on the test split (TEST) of domain jj after stage tt. We report ACC and the average incremental accuracy (AIA),

ACC=1T∑j=1TAT,j,AIA=1T∑t=1T1t∑j=1tAt,j,(3)\mathrm{ACC} = \frac{1}{T}\sum_{j=1}^{T} A_{T,j}, \qquad\mathrm{AIA} = \frac{1}{T}\sum_{t=1}^{T}\frac{1}{t}\sum_{j=1}^{t} A_{t,j}, \tag*{(3)}

which weight domains equally, as in KA-Prompt [39]. To separate the experts’ errors from the router’s, let ae,ja_{e,j} be the accuracy of expert ee on domain jj after the last stage, and let C(x)C(x) be the set of experts that classify image xx correctly:

DRA=1T∑jaj,j,routing loss=DRA−ACC,\mathrm{DRA} = \frac{1}{T}\sum_j a_{j,j}, \qquad\text{routing loss} = \mathrm{DRA} - \mathrm{ACC},
SE=1T∑jPr⁡x∈TESTj[C(x)=∅].(4)\mathrm{SE} = \frac{1}{T}\sum_j \Pr_{x\in\mathrm{TEST}_j}[C(x)=\varnothing]. \tag*{(4)}

The domain-routed accuracy (DRA) is the accuracy when the domain label selects each image’s own-domain expert. The shared error (SE) is the share of images that no router can recover. The quantities below evaluate images of domain jj at a later expert e>je>j or an earlier expert e<je<j. Each cost is the accuracy lost relative to the own-domain expert:

later-expert accuracy=2T(T−1)∑e>jae,j,later-expert cost=2T(T−1)∑e>j(aj,j−ae,j),earlier-expert cost=2T(T−1)∑e<j(aj,j−ae,j).(5)\begin{aligned} \text{later-expert accuracy} &= \frac{2}{T(T-1)}\sum_{e>j} a_{e,j},\\ \text{later-expert cost} &= \frac{2}{T(T-1)}\sum_{e>j}(a_{j,j}-a_{e,j}),\\ \text{earlier-expert cost} &= \frac{2}{T(T-1)}\sum_{e<j}(a_{j,j}-a_{e,j}). \tag*{(5)} \end{aligned}

Lamarck

Prompt base. Each Lamarck expert ψe\psi_e (Figure 2a) adds adapters to the prompt-only expert of § 3, whose head is an MLP readout. As in most compared prompt learners, prompts enter the N=5N=5 early blocks [13, 19], each with a pool of Z=100Z=100 rows [13, 25].

Deep adapters. After each deep block b∈B={N+1,…,M}b \in\mathcal{B}=\{N+1,\ldots,M\}, a residual bottleneck adapter [8] corrects every image token uu (the CLS and 196 patch tokens, not prompt tokens):

u←u+UbGELU⁡(DbLN⁡(u)),(6)u \leftarrow u + U_b \operatorname{GELU}(D_b\operatorname{LN}(u)), \tag*{(6)}

with down- and up-projections Db∈Rr×dD_b \in\mathbb{R}^{r \times d}, Ub∈Rd×rU_b \in\mathbb{R}^{d \times r} (d=768d=768, rank r=32r=32), no biases and parameter-free layer normalization. Correcting only the CLS token is less accurate (Table 5). Unlike prompts, the adapters change the features themselves. They add 2rd∣B∣≈0.34M2rd|\mathcal{B}| \approx0.34\mathrm{M} parameters per expert, against 86M in the frozen backbone.

Readout and training. The readout is a two-layer MLP ffcf_{\mathrm{fc}} with hidden width 384 and ReLU. It maps f^\hat{f}, the CLS feature after the final layer normalization, to logits scaled by the sigmoid σ\sigma of a learned scalar ww:

logits⁡(x)=σ(w)ffc(f^),L=CE⁡(logits⁡(x),y).(7)\operatorname{logits}(x)=\sigma(w)f_{\mathrm{fc}}(\hat{f}),\qquad\mathcal{L}=\operatorname{CE}(\operatorname{logits}(x),y). \tag*{(7)}

At stage tt, Lamarck trains the new prompt rows, all adapters and the readout on DtD_t, then freezes and stores expert ψt\psi_t.

Inheritance. Lamarck initializes expert tt with the prompt rows, adapters and readout of its predecessor and adds only new prompt rows (Figure 2b). The first expert’s up-projections UbU_b start at zero, so it begins as a prompt-only expert. Inheritance needs only the stored predecessor, not earlier images, and adds no parameters or computation. Because each expert starts from its predecessor, an image misrouted to a later expert loses less accuracy than with independently trained experts (Figure 4; § 6.4).

Final-stage accuracy matrices for three expert banks

Figure 4. Inherited adapters make later experts more accurate on earlier domains. Final-stage accuracy (colour, %) of every expert (columns) on every CDDB domain (rows) in three expert banks. Above the diagonal: later experts on earlier domains. Outlined: own-domain experts. Dot area: share of a domain’s images for which the Mahalanobis router selects each expert (same in all panels). Chance: 50%; TEST, five-seed means.

Mahalanobis router. An unprompted pass gives the ℓ2\ell_2-normalized CLS feature z(x)z(x). For every seen domain ee, the router keeps the mean μe\mu_e and the covariance of zz over the training split (TRAIN), and the number of images. It pools the covariances into the within-domain covariance SS and shrinks SS toward a scaled identity, Σ=(1−λ)S+λtr⁡SdI\Sigma=(1-\lambda)S+\lambda\frac{\operatorname{tr}S}{d}I, with λ=0.01\lambda=0.01 chosen on the validation split (VAL). The router selects the domain whose mean is nearest to z(x)z(x) in shrunk Mahalanobis distance [6, 14]:

ρt(x)=arg min⁡e≤t (z(x)−μe)TΣ−1(z(x)−μe).(8)\rho_t(x)=\underset{e\leq t}{\operatorname{arg\,min}}\ \left(z(x)-\mu_e\right)^{\mathsf{T}}\Sigma^{-1}\left(z(x)-\mu_e\right). \tag*{(8)}

It keeps only these feature statistics and needs no training. Inference costs one unprompted and one prompted pass (Alg. S1).

The Experts Limit Accuracy

To separate routing loss from expert error, we analyze four expert banks (§ 6.1): S-iPrompt’s prompt-only experts, SOYO’s adapter experts, and Lamarck’s experts with and without adapters.

Evaluation without retraining. We evaluate every stored expert on the test images of every seen domain. With each learner’s own router, this evaluation reproduces its routed predictions exactly (Sec. S2). This extends SOYO’s evaluation of own-domain experts [34] to every expert–domain pair. It also extends the evaluation of every prompt with every head in [2] to whole experts, including adapter experts. We route all banks with two training-free routers on frozen features: the Mahalanobis router (Eq. (8)) and a prototype router. The prototype router fits eight Gaussian-mixture means per domain on TRAIN and selects the domain of the mean most cosine-similar to the image’s frozen feature. All banks thus receive the same routes and differ only in their experts. A misrouted image can be classified correctly where its own-domain expert errs, so ACC can exceed DRA (Eq. (4)), as for S-iPrompt on CDDB under the Mahalanobis router.

The routing loss is small. Under the Mahalanobis router, the routing loss is at most 2.62 points on DN, 1.15 on OH and 1.84 on CDDB (Table 1). The expert error (100−DRA) is larger: 11.2 to 30.7 for S-iPrompt and SOYO and 5.2 to 28.1 for full Lamarck (Table 1). The router misroutes 11.9%, 16.2% and 3.6% of TEST images (Table S11), but the expert it selects is often still correct. Replacing the experts under this router can change ACC by more than the routing loss: even the three banks without Lamarck’s adapters span 3.40 points on DN (SOYO over S-iPrompt), 1.78 on OH and 2.11 on CDDB (Table 1). The prototype router ranks the banks identically.

RoutingDomainNet Mis.↓−reset−indep.OfficeHome Mis.↓−reset−indep.CDDB-Hard Mis.↓−reset−indep.
Domain-label routing0.0−0.41 ± 0.12−0.07 ± 0.040.0−0.01 ± 0.19+0.23 ± 0.200.0+0.31 ± 0.99+0.55 ± 0.86
Mahalanobis router11.9−0.41 ± 0.16+0.08 ± 0.0716.2+0.15 ± 0.15+0.54 ± 0.283.6+0.21 ± 0.39+0.58 ± 0.75
Prototype router19.5−0.32 ± 0.16+0.33 ± 0.0636.4+0.12 ± 0.14+1.10 ± 0.3410.5+0.77 ± 0.47+1.47 ± 0.81
Last expert only88.2+1.32 ± 0.61+6.17 ± 0.5872.1+0.73 ± 0.42+4.80 ± 0.6698.5+7.94 ± 3.38+17.75 ± 2.38

Table S11. Inheritance changes ACC little under accurate routing and helps most without a router. TEST ACC of inherited experts minus that of experts with reset adapters (−reset) or of independent experts (−indep.); points, mean±s.d. of the per-seed differences over five seeds. Mis.: share (%) of all TEST images that are not sent to their own domain’s expert.

Experiments

Setup

Benchmarks. As in prior work, we use DN [23] (six domains from clipart to sketch, 345 classes), OH [30] (four domains, 65 classes) and CDDB [15] (five domains from GauGAN to SAN, real vs. fake). DUCT [43] and CONEC-LoRA [20] evaluate on all three, and S-Prompts [35] and SOYO [34] evaluate on DN and CDDB. Each benchmark’s domains are learned in a fixed order. DN keeps its official test set as TEST, and we split OH and CDDB into TRAIN, VAL and TEST (Sec. S1). CDDB’s domains are unbalanced: its two smallest, WhichFaceIsReal (WFIR) and SAN, contain 160 and 33 of its 2,140 TEST images.

Baselines. The twelve baselines (Table 2) are six shared-pool learners and six per-domain learners. We exclude G2D [3] because it trains its router with a second pre-trained model, a diffusion model fine-tuned on each domain. Two controls of Lamarck isolate inheritance: the reset adapters control re-initializes only the adapters for every domain, and the independent experts control starts every expert from the first expert’s initialization. All Lamarck variants and controls use the readout of Eq. (7).

Protocol. For a fair comparison, all methods in Table 2 are trained under one protocol: a frozen ViT-B/16 (ImageNet-21k→1k), identical splits and domain orders, FP32, and five training seeds as in OS-Prompt [13] and CODA-Prompt [25]. Every method learns each domain for 15 epochs at batch size 128, the budget DUCT [43] releases for these benchmarks. Lamarck and ten of the baselines use Adam with initial learning rate 10−310^{-3} for every trained parameter group, as released with CODA-Prompt, OS-Prompt, OS-Prompt++ and CPS-Prompt. EASE and CONEC-LoRA instead use their released SGD settings. Each method has its own schedule, weight decay and auxiliary phases (Table S1). On VAL, we select which tokens the adapters correct (Sec. S9), and we tune no learning rate on our splits for any method. Lamarck uses the simpler of two readouts that differ by at most 0.2 points on VAL and TEST (Sec. S1). We report TEST accuracy, with differences paired by seed and computed before rounding.

MethodOptimizerLRScheduleWDPrompt length / pool / moduleAuxiliary phases (epochs)Other settings
Joint linear probeAdam10−310^{-3}cosine0––linear head on the pooled TRAIN domains
SLCASGD0.01multi-step5×10−45 \times10^{-4}fine-tuned backboneclassifier alignment (5)LR step at epoch 12; backbone LR ×0.01\times0.01; alignment on 256 sampled features per class with logit normalization 0.1; prediction modulo ∣Y∣|Y| over the per-domain heads
DUCTSGD0.1multi-step‡^{\ddagger}5×10−45 \times10^{-4}fine-tuned backboneclassifier retraining (5; LR 10−310^{-3})backbone LR ×0.01\times0.01; merge ratio 0.25 (OfficeHome, DomainNet) / 0.5 (CDDB-Hard) and optimal-transport regularization 0.1 (OfficeHome, CDDB-Hard) / 0.01 (DomainNet), as in the released configurations; prediction modulo ∣Y∣|Y| over the per-domain heads; classes without training images left out of the transport; log-domain Sinkhorn solver
CODA-PromptAdam10−310^{-3}cosine0pool 100, length 8, blocks 1–5–orthogonality weight 0 (released script)
OS-PromptAdam10−310^{-3}cosine0pool 100, length 8, blocks 1–5–query regularizer 0
OS-Prompt++Adam10−310^{-3}cosine0pool 100, length 8, blocks 1–5–query regularizer 10−410^{-4}
KA-PromptAdam10−310^{-3}constant0.01pool 100 split evenly over domains, length 8, top-kk 3; general prompt in blocks 1–2, per-domain prompts in blocks 3–5exponential-moving-average warm-up (1)τ\tau 0.01; auxiliary weight 0.1
CPS-PromptAdam10−310^{-3}cosine0pool 100, length 8, blocks 1–5–reduction 0.5; critical-score sampling; phase ratio 0.8; temperature 1.0; gradient clipping 1.0
UniPrompt-CLAdam10−310^{-3}cosine0pool 500, 100 rows added per domain (earlier rows frozen), 5 prompted blocks–selection-loss weight 10−410^{-4} (released training code)
S-iPromptAdam10−310^{-3}cosine2×10−42 \times10^{-4}length 8, one prompt set per domain–5 kk-means keys per domain; image-only
HiDe-PromptAdam10−310^{-3}cosine0one prompt per domain, length 8 (4 key + 4 value), blocks 1–5task-identity head (20), classifier alignment (30)contrastive weight 0.5; prompt momentum 0.01; gradient clipping 1.0; prediction modulo ∣Y∣|Y| over the (domain, class) outputs
EASESGD0.05cosine5×10−35 \times10^{-3}one adapter of width 64 per domain (released configuration)–prototype reweighting, α\alpha 0.1; each domain is a task of ∣Y∣|Y| classes, prediction modulo ∣Y∣|Y|
SOYOAdam10−310^{-3}cosine2×10−42 \times10^{-4}length 8; adapter width 16router (30)2 Gaussian-mixture components
HashAdam10−310^{-3}cosine0pool 100, length 8 (4 key + 4 value), general prompt 5 key + 5 value, top-kk 2 with softmax weights, blocks 1–5task-identity head (50), classifier alignment (30)gate activation sigmoid, general prompt and prompted blocks as in the released code; prompt momentum 0.01; gradient clipping 1.0; prediction modulo ∣Y∣|Y| over the (domain, class) outputs
CONEC-LoRASGD0.02cosine2×10−42 \times10^{-4}LoRA rank 8 on query and value, shared in blocks 1–6 and per domain in blocks 7–12router (15; LR 2×10−32 \times10^{-3})router on all 12 blocks; 2 Gaussian-mixture components
Lamarck (Ours)Adam10−310^{-3}cosine0pool 100, length 1, blocks 1–5; adapters of rank r=32r=32 after blocks 6–12–MLP readout; training-free Mahalanobis router

Table S1. Training settings in our runs. All methods learn each domain for 15 epochs at batch size 128. The optimizer and initial learning rate (LR) apply to every trained parameter group and phase unless a cell states otherwise; WD: weight decay; ∣Y∣|Y|: number of classes.

Main results

Lamarck is first on CDDB and in mean ACC. Lamarck has the best ACC and AIA on CDDB, 5.18 ACC points above Hash (Table 2). Its mean ACC is also the highest: 83.62 against 79.67 for HiDe-Prompt, the best frozen-ViT baseline. In ACC it is second on DN (69.46 against 69.91 for SOYO) and on OH (0.17 points below Hash). The CDDB lead also holds with every TEST image weighted equally and with SAN removed (Table S6). With the CDDB order reversed. Lamarck reaches 93.15 ACC and leads Hash and HiDe-Prompt in every seed, on average by 4.70 and 6.54 points (Table S7). SLCA and DUCT, which fine-tune the ViT, lead on OH (90.14 and 88.87 vs. 88.47 ACC) but trail Lamarck on CDDB and DN (Table S5). On DN, forgetting leaves the six shared-pool learners at least 8 ACC points below every per-domain learner: right after learning a domain, CODA-Prompt averages 71.3%, as high as SOYO’s 71.0%, but ends at 50.6% (Sec. S7).

MethodDomainNet AIA↑DomainNet ACC↑OfficeHome AIA↑OfficeHome ACC↑CDDB-Hard AIA↑CDDB-Hard ACC↑
SLCA [40]68.71 ± 0.0567.97 ± 0.0788.16 ± 0.4190.14 ± 0.4695.52 ± 0.2289.35 ± 1.37
DUCT [43]63.58 ± 0.0659.96 ± 0.1286.71 ± 0.1188.87 ± 0.3381.65 ± 2.5567.58 ± 3.86
Lamarck (Ours)68.89 ± 0.2069.46 ± 0.1485.85 ± 0.6388.47 ± 0.2597.16 ± 0.1792.94 ± 0.62

Table S5.

MethodEqualImagesEqual w/o SANImages w/o SANGauGAN (800)BigGAN (320)Wild (827)WhichFaceIsReal (160)SAN (33)
Joint linear probe80.63 ± 0.3482.46 ± 0.1882.45 ± 0.2582.60 ± 0.1994.90 ± 0.2981.12 ± 0.9071.29 ± 0.5282.50 ± 1.1773.33 ± 1.36
SLCA [40] (fine-tuned backbone)89.35 ± 1.3788.45 ± 1.4990.17 ± 1.9788.49 ± 1.5488.88 ± 3.2291.06 ± 3.4385.88 ± 1.2194.88 ± 2.2786.06 ± 3.46
DUCT [43] (fine-tuned backbone)67.58 ± 3.8674.55 ± 4.8773.27 ± 4.4175.02 ± 4.9389.95 ± 5.4787.00 ± 5.8459.49 ± 6.5856.62 ± 2.8844.85 ± 4.49
CODA-Prompt [25]72.92 ± 2.3369.88 ± 1.8969.48 ± 2.3969.62 ± 1.9273.30 ± 2.7175.94 ± 1.5364.69 ± 1.5664.00 ± 5.6986.67 ± 7.30
OS-Prompt [13]72.84 ± 2.1469.46 ± 2.7268.93 ± 2.3869.16 ± 2.7474.83 ± 2.9579.06 ± 1.3461.57 ± 4.3260.25 ± 2.7188.48 ± 2.54
OS-Prompt++ [13]72.69 ± 2.2169.57 ± 2.9969.05 ± 2.7869.29 ± 3.0474.60 ± 3.4779.62 ± 1.4461.96 ± 4.4060.00 ± 3.2287.27 ± 1.36
KA-Prompt [39]74.96 ± 0.3974.06 ± 0.5674.91 ± 0.4374.04 ± 0.5778.20 ± 1.1069.50 ± 1.3070.30 ± 1.1081.62 ± 1.3075.15 ± 1.36
CPS-Prompt [17]71.45 ± 1.4768.01 ± 1.9067.65 ± 1.5167.72 ± 1.9176.92 ± 1.5974.12 ± 2.6157.41 ± 3.2862.12 ± 1.5786.67 ± 1.66
UniPrompt-CL [19]76.14 ± 0.8976.60 ± 0.6576.09 ± 0.4776.60 ± 0.6378.33 ± 1.5381.50 ± 1.8974.27 ± 2.2470.25 ± 3.4476.36 ± 4.49
S-iPrompt [35]85.60 ± 1.9286.39 ± 0.6387.15 ± 0.3486.50 ± 0.5595.80 ± 0.6186.31 ± 0.2677.00 ± 1.2889.50 ± 0.8179.39 ± 8.41
HiDe-Prompt [31]86.39 ± 0.6187.90 ± 0.4388.60 ± 0.4888.06 ± 0.4396.53 ± 0.3789.38 ± 0.9179.11 ± 0.7389.38 ± 1.3377.58 ± 1.66
EASE [41]67.14 ± 0.7975.38 ± 0.5172.56 ± 0.9975.85 ± 0.5297.60 ± 0.4672.12 ± 1.8959.03 ± 0.3561.50 ± 2.0145.45 ± 0.00
SOYO [34]79.24 ± 0.9591.21 ± 0.5087.53 ± 1.1091.92 ± 0.5099.05 ± 0.2198.00 ± 0.3687.84 ± 0.6465.25 ± 3.9946.06 ± 1.36
Hash [16]87.75 ± 0.8888.80 ± 0.5589.99 ± 0.6788.96 ± 0.5496.22 ± 0.6891.81 ± 1.0780.31 ± 0.6591.62 ± 1.8078.79 ± 2.14
CONEC-LoRA [20]79.96 ± 2.3486.36 ± 4.7388.28 ± 3.5186.99 ± 4.8498.67 ± 0.5795.62 ± 1.2372.45 ± 11.3686.38 ± 3.8446.67 ± 3.46
Lamarck (Ours)92.94 ± 0.6296.01 ± 0.1396.93 ± 0.2596.31 ± 0.1599.42 ± 0.1198.12 ± 0.4992.29 ± 0.3497.88 ± 0.7176.97 ± 2.71

Table S6.

MethodOriginal order AIA↑Original order ACC↑Reversed order AIA*↑Reversed order ACC↑
HiDe-Prompt [31]91.14 ± 0.3486.39 ± 0.6183.71 ± 1.0886.62 ± 0.56
Hash [16]92.37 ± 0.3687.75 ± 0.8886.30 ± 0.7788.46 ± 1.07
Lamarck, reset adapters97.13 ± 0.1692.73 ± 0.6390.18 ± 1.8293.12 ± 0.89
Lamarck (Ours)97.16 ± 0.1792.94 ± 0.6290.12 ± 1.4493.15 ± 0.96

Table S7.

Lamarck costs as much as other two-pass learners. At test time, Lamarck needs 71.2 GFLOPs per image against 70.3–88.1 (Figure 3, Table S9). Single-pass learners, all shared-pool, need 35–36 GFLOPs but trail Lamarck by over 13 points in mean ACC. Lamarck stores 9.84M parameters (11.5% of the backbone): each expert has its own prompt pool, adapters and readout. EASE and CONEC-LoRA store more (Table S9).

MethodPasses↓GFLOPs↓Param. (M)↓img/s↑Mem. (GB)↓GFLOPs↓img/s↑Mem. (GB)↓
CODA-Prompt [25]270.34.11159.10.90105.9100.96.15
OS-Prompt [13]135.13.72314.80.8670.8146.36.09
OS-Prompt++ [13]135.13.72314.61.18105.9100.56.45
KA-Prompt [39]288.16.76130.01.45283.038.215.68
CPS-Prompt [17]270.34.11159.70.9769.7151.23.09
UniPrompt-CL [19]135.91.96326.60.8273.8144.35.38
S-iPrompt [35]271.71.65154.50.8774.5148.36.26
HiDe-Prompt [31]270.35.73158.61.20{sup}§`106.1102.213.87
EASE [41]{sup}†`6212.416.6851.51.0569.8151.25.72
SOYO [34]7255.53.4643.50.8674.9143.56.73
Hash [16]271.09.12154.71.21{sup}§`73.1142.614.30
CONEC-LoRA [20]{sup}†`270.423.16146.21.23123.784.76.24
Lamarck (Ours)271.29.84147.61.7374.3141.66.25

Table S9. Cost of the timed learners on DomainNet after the last domain. One V100, FP32, batch size 64; protocol in Sec. S8. Passes: encoder forward passes per test image; Param.: parameters added to the backbone and used at test time. {sup}§Without the class statistics (9.1 GB) that the released code loads with the checkpoint but never reads at test time; training memory includes them. {sup}†Timed on the final-stage architecture with initial weights.

The adapters strengthen the experts

Lamarck’s lead comes from its experts. With all four banks routed identically, Lamarck’s experts are the most accurate on OH and CDDB, under both routers and in DRA (Table 1). Under the Mahalanobis router, their lead over SOYO’s experts reaches 6.5 points (92.94 vs. 86.46 on CDDB). Despite their adapters, SOYO’s independently trained experts are less accurate than Lamarck’s promptonly experts on OH and CDDB. On DN they are the most accurate (70.19 vs. 69.46 for Lamarck under the Mahalanobis router).

The adapters raise accuracy over Lamarck’s prompt-only experts. Under the Mahalanobis router, the adapters add 4.36 ACC points on CDDB, 2.14 on DN and 0.33 on OH (Table 1 and Table 4). Of this gain, images routed to their own-domain expert contribute +4.75 points on CDDB, +2.00 on DN and +0.35 on OH, whereas misrouted images contribute between −0.39 and +0.14. The adapters also raise accuracy on all three benchmarks under the prototype router and in DRA (Table 1). They also lower SE on all three benchmarks, from 3.27% to 1.80% on CDDB (Table 1). On DINOv2 ViT-B/14, they raise ACC in every seed on OH and CDDB (from 91.16 to 94.01 on CDDB; Tab. S16).

The adapters help where prompt-only experts fall short. In own-domain accuracy on DN, Lamarck’s prompt-only experts are within 0.2 points of SOYO’s adapter experts on real photographs but trail on the other five domains, from 1.5 points on painting to 8.2 on QuickDraw (Table 3). S-iPrompt’s independently trained prompt-only experts share this shortfall: over the DN domains, they average 0.45 points below Lamarck’s prompt-only experts (Table S2). On DN, the adapters’ gain follows exactly the reverse order of this shortfall, from none on real photographs to 5.67 points on QuickDraw, where the gap to SOYO’s experts narrows to 2.5 points. The gain is 1.50 to 9.09 points across the CDDB domains but at most 0.93 on any OH domain.

DomainSOYOPrompts onlyLamarck (Ours)Δ
Own-domain accuracy (%) ↑
DNClipart80.85±0.2580.85 \pm0.2577.89±0.1577.89 \pm0.1580.34±0.3180.34 \pm0.31+2.46±0.29+2.46 \pm0.29
DNInfograph46.76±0.1546.76 \pm0.1543.54±0.1943.54 \pm0.1946.11±0.4846.11 \pm0.48+2.58±0.55+2.58 \pm0.55
DNPainting76.08±0.1876.08 \pm0.1874.54±0.2374.54 \pm0.2375.55±0.1275.55 \pm0.12+1.02±0.24+1.02 \pm0.24
DNQuickDraw72.29±0.1872.29 \pm0.1864.12±0.1164.12 \pm0.1169.79±0.0969.79 \pm0.09+5.67±0.16+5.67 \pm0.16
DNReal87.41±0.0487.41 \pm0.0487.21±0.1987.21 \pm0.1987.17±0.0987.17 \pm0.09−0.04±0.23-0.04 \pm0.23
DNSketch73.47±0.1473.47 \pm0.1471.00±0.1671.00 \pm0.1672.64±0.1772.64 \pm0.17+1.64±0.27+1.64 \pm0.27
OHArt81.55±1.3481.55 \pm1.3483.25±1.2583.25 \pm1.2583.79±1.4883.79 \pm1.48+0.53±0.50+0.53 \pm0.50
OHClipart81.73±1.2481.73 \pm1.2483.12±0.4983.12 \pm0.4984.05±0.4784.05 \pm0.47+0.93±0.53+0.93 \pm0.53
OHProduct95.37±0.3295.37 \pm0.3295.98±0.2095.98 \pm0.2096.12±0.4596.12 \pm0.45+0.14±0.40+0.14 \pm0.40
OHReal World91.40±0.4691.40 \pm0.4693.31±0.4693.31 \pm0.4693.19±0.4293.19 \pm0.42−0.11±0.42-0.11 \pm0.42
CDDBGauGAN99.78±0.1099.78 \pm0.1098.08±0.4898.08 \pm0.4899.58±0.1199.58 \pm0.11+1.50±0.54+1.50 \pm0.54
CDDBBigGAN99.31±0.4199.31 \pm0.4194.62±0.6094.62 \pm0.6099.31±0.3499.31 \pm0.34+4.69±0.58+4.69 \pm0.58
CDDBWild87.84±0.6487.84 \pm0.6487.16±1.0587.16 \pm1.0592.29±0.3492.29 \pm0.34+5.13±1.19+5.13 \pm1.19
CDDBWFIR72.00±5.8572.00 \pm5.8591.12±1.2091.12 \pm1.2087.88±0.7187.88 \pm0.71+6.75±1.35+6.75 \pm1.35
CDDBSAN81.82±0.0081.82 \pm0.0075.76±4.7975.76 \pm4.7984.85±2.1484.85 \pm2.14+9.09±6.78+9.09 \pm6.78

Table 3. The adapters raise own-domain accuracy most on QuickDraw and on CDDB. Own-domain accuracy: ad,ja_{d,j} in Eq. (4). Columns: SOYO’s adapter experts, Lamarck’s prompt-only experts and Lamarck. Δ\Delta: gain from the adapters. TEST, five-seed mean±s.d. SAN has 33 TEST images.

Adapting every image token raises ACC over CLS-only adapters and deep-block prompts. Adapters that correct only the CLS token in the deep blocks give no ACC gain over Lamarck’s prompt-only experts on OH and reach 92.30 on CDDB, against 92.94 for Lamarck (Table 5). On CDDB, adapting every image token also lowers the later-expert cost (15.75 vs. 23.53 for CLS-only adapters). With as many parameters as the adapters, deep-block prompts recover 3.17 of the adapters’ 4.36 points on CDDB. The adapter form adds the other 1.19 points. On OH it adds 0.25 (Table 5).

Adapters in the deep blocks match the ACC of adapters in all twelve blocks; rank matters little. Adapters of rank 16 in all twelve blocks match Lamarck’s ACC on DN, OH and CDDB (within 0.3 points; Table 5 and Table S10). Across ranks 8, 32 and 64, VAL and TEST ACC vary by at most 0.4 points on DN, 0.2 on OH and 1.1 on CDDB. Selecting the VAL-best rank, r=64r=64, would change TEST ACC by at most 0.4 points.

Prompt retrieval is nearly input-independent. In Lamarck’s last prompt-only expert, the first prompted block’s query (Eq. (2)) is the learned class token plus a position embedding, identical for every image, so all images receive the same prompt. In blocks 2–5, retrieval weights barely depend on the image. On average, an image’s weights differ from the dataset-average weights by less than 0.01 in total variation (at most 0.02 for any image), and their entropy is at least 98.4% of the maximum on every benchmark (Table S3). This expert therefore uses essentially one fixed prompt per block. Querying every pool with the router’s frozen feature makes retrieval input-dependent but changes OH ACC under the prototype router by at most 0.15 points (Table S4). Moreover, when the deep blocks carry adapters, removing the prompts changes ACC by less than 0.25 points on OH and CDDB (Table 4).

BenchmarkBlockEntropy/log ZZLargest weightTV meanTV max
DomainNet199.14%0.01690.00000.0000
299.01%0.01770.00080.0027
398.47%0.01790.00390.0090
499.56%0.01560.00810.0162
598.96%0.02110.00750.0200
OfficeHome199.94%0.01170.00000.0000
299.89%0.01240.00050.0016
399.95%0.01140.00130.0034
499.95%0.01180.00160.0041
599.95%0.01220.00220.0051
CDDB-Hard199.95%0.01210.00000.0000
299.92%0.01290.00040.0013
399.95%0.01170.00120.0025
499.92%0.01400.00150.0033
599.90%0.01350.00220.0045

Table S3. Prompt retrieval is nearly uniform and nearly input-independent. Retrieval weights of Lamarck’s last prompt-only expert (one seed per benchmark) over up to 256 VAL images per domain and 100 visible pool rows (uniform weight 0.01), per prompted block: entropy relative to its maximum, mean largest weight, and the total-variation (TV) distance of an image’s weights from the average weights (mean and maximum over images).

τ\tauVAL Δ\DeltaAIA↑VAL Δ\DeltaACC↑TEST Δ\DeltaAIA↑TEST Δ\DeltaACC↑
0.03−0.27±0.35-0.27 \pm0.35 (1/5)−0.15±0.17-0.15 \pm0.17 (1/5)−0.14±0.37-0.14 \pm0.37 (2/5)0.12±0.330.12 \pm0.33 (4/5)
0.10.01±0.280.01 \pm0.28 (2/5)0.10±0.280.10 \pm0.28 (3/5)−0.02±0.41-0.02 \pm0.41 (4/5)0.08±0.200.08 \pm0.20 (3/5)
0.30.12±0.250.12 \pm0.25 (4/5)0.11±0.230.11 \pm0.23 (2/5)−0.09±0.44-0.09 \pm0.44 (2/5)0.07±0.140.07 \pm0.14 (4/5)

Table S4. Querying every prompt pool with the router’s frozen feature. Change in OfficeHome accuracy against the prompt-only experts (same seeds) under the prototype router, for softmax temperatures τ\tau of the retrieval; points, mean±s.d. over five seeds; in parentheses, seeds with a positive change.

Inheritance lowers the later-expert cost

Inheritance raises later-expert accuracy. On every benchmark, Lamarck has a lower later-expert cost than reset adapters and every bank without inheritance (Table 4 and Table 6). Without adapters, inheritance also lowers this cost, on CDDB from 35.49 to 13.58. Adding reset adapters doubles it to 27.81, but inherited adapters raise it only to 15.75. Lamarck’s later-expert cost exceeds that of its prompt-only experts by 1.1–2.2 points on each benchmark, because adapters raise own-domain accuracy more than later-expert accuracy. Over reset adapters, inheriting the adapters raises later-expert accuracy by 2.29, 1.47 and 12.06 points on DN,

Expert bankExpertsLater-expert accuracy↑ DNLater-expert accuracy↑ OHLater-expert accuracy↑ CDDBLater-expert cost↓ DNLater-expert cost↓ OHLater-expert cost↓ CDDBEarlier-expert cost↓ DNEarlier-expert cost↓ OHEarlier-expert cost↓ CDDB
S-iPromptindependent31.52 ± 0.1173.22 ± 0.2157.64 ± 1.0535.19 ± 0.1912.22 ± 0.4533.38 ± 1.4238.77 ± 0.1212.31 ± 0.5130.11 ± 5.15
SOYOindependent33.07 ± 0.1366.78 ± 0.5854.47 ± 0.5537.04 ± 0.1517.13 ± 1.2540.00 ± 0.9740.02 ± 0.1917.27 ± 0.5830.24 ± 1.95
Prompts only, independentindependent30.28 ± 0.2272.48 ± 0.4058.61 ± 0.8736.48 ± 0.1812.59 ± 0.3035.49 ± 0.7040.53 ± 0.2613.40 ± 0.9134.89 ± 1.88
Lamarck, independentindependent30.85 ± 0.1672.31 ± 0.5959.45 ± 1.0038.52 ± 0.1913.42 ± 0.8438.53 ± 1.1041.20 ± 0.1513.71 ± 0.8937.81 ± 1.48
Lamarck, reset adaptersprompts inherited35.54 ± 0.3379.42 ± 0.4870.06 ± 1.4134.02 ± 0.306.55 ± 0.8227.81 ± 1.4239.34 ± 0.1412.13 ± 1.0837.84 ± 1.49
Lamarck, prompts onlyinherited37.28 ± 0.1381.37 ± 0.2580.59 ± 1.1929.56 ± 0.083.96 ± 0.7313.58 ± 1.2437.70 ± 0.1011.85 ± 0.8432.33 ± 2.30
Lamarck (Ours)inherited37.83 ± 0.1380.89 ± 0.1982.12 ± 0.8731.47 ± 0.185.04 ± 0.4815.75 ± 0.8437.65 ± 0.1611.93 ± 0.7438.93 ± 1.46

Table 6. The two fully inherited banks have the highest later-expert accuracy and the lowest later-expert cost. Metrics as defined in Eq. (5). Experts: what each expert inherits from its predecessor. TEST, five-seed mean±s.d.

OH and CDDB (Table 6). In Figure 4, every cell above the diagonal gains over reset adapters, whereas the other cells’ mean changes by under half a point. Inheritance still lowers the later-expert cost on reversed CDDB, from 34.25 to 27.85 (Tab. S14), and on DINOv2 for OH and CDDB (Table S17).

On DINOv2 ViT-B/14, inheritance changes routed ACC by at most 0.2 points, raises ACC with the last expert only and lowers the later-expert cost on both datasets. TEST ACC (%) of Lamarck (inherited adapters) and of Lamarck with reset adapters, and Lamarck −- reset (points, mean±\pms.d. of the per-seed differences); last row: later-expert cost (Eq. (5)). Five seeds. Mis.: share (%) of all TEST images that are not sent to their own domain’s expert, equal in both arms because the routers read the unprompted feature.

OfficeHomeCDDB-Hard
RoutingMis.↓\downarrowLamarckReset−-resetMis.↓\downarrowLamarckReset−-reset
Domain-label routing0.090.18±0.2690.18 \pm0.2690.27±0.3490.27 \pm0.34−0.08±0.23-0.08 \pm0.230.094.97±0.2994.97 \pm0.2995.08±0.3695.08 \pm0.36−0.11±0.41-0.11 \pm0.41
Mahalanobis router20.389.34±0.0989.34 \pm0.0989.15±0.4089.15 \pm0.40+0.20±0.39+0.20 \pm0.396.194.01±0.9994.01 \pm0.9994.17±0.9894.17 \pm0.98−0.16±0.42-0.16 \pm0.42
Prototype router35.288.61±0.3088.61 \pm0.3088.56±0.3588.56 \pm0.35+0.06±0.27+0.06 \pm0.2711.392.64±0.6592.64 \pm0.6592.47±0.6292.47 \pm0.62+0.17±0.47+0.17 \pm0.47
Last expert only72.186.04±0.5386.04 \pm0.5384.53±0.3584.53 \pm0.35+1.50±0.58+1.50 \pm0.5898.584.60±1.3884.60 \pm1.3877.24±1.7477.24 \pm1.74+7.36±1.88+7.36 \pm1.88
Later-expert cost↓\downarrow−-5.93±0.325.93 \pm0.329.01±1.159.01 \pm1.15−3.08±0.98-3.08 \pm0.98−-10.02±0.8610.02 \pm0.8618.31±1.5518.31 \pm1.55−8.29±1.66-8.29 \pm1.66

Table S17. Table S17. On DINOv2 ViT-B/14, inheritance changes routed ACC by at most 0.2 points, raises ACC with the last expert only and lowers the later-expert cost on both datasets. TEST ACC (%) of Lamarck (inherited adapters) and of Lamarck with reset adapters, and Lamarck −- reset (points, mean±\pms.d. of the per-seed differences); last row: later-expert cost (Eq. (5)). Five seeds. Mis.: share (%) of all TEST images that are not sent to their own domain’s expert, equal in both arms because the routers read the unprompted feature.

Inheritance adds more ACC as more images reach later experts. Lamarck’s ACC difference over reset adapters grows with FF, the share of TEST images routed to a later expert, averaged over domains, for router and injected errors alike (Figure 5). Under the Mahalanobis router, the difference is +0.15 on OH, +0.21 on CDDB and −0.41 on DN, where reset adapters have higher own-domain accuracy (Table S11). On CDDB, fitting the router on kk TRAIN images per domain raises FF from 1.28% to 18.37% at k=2k=2 and the difference to +2.32 points; it is positive in every seed for every kk (Tab. S13). Its slope across kk, 0.129±0.050 points per point of FF, is close to the 0.121 (12.06/100) implied by the 12.06-point gap in later-expert cost (Table 4), and the OH and DN slopes are likewise close to their implied values (Table 6 and S13). When the last expert classifies every image, the difference is +1.32, +0.73 and +7.94 on DN, OH and CDDB (Table S11), whereas errors injected toward earlier experts do not raise it (Table S12). Over independent experts, Lamarck’s ACC lead is 0.08 to 0.58 points under the Mahalanobis router and 1.67 to 4.42 under the k=2k=2 router (Table S11 and S13).

Figure 5. On OH and CDDB, inherited adapters add more ACC as more images reach later experts through router or injected errors. TEST ACC: Lamarck minus reset adapters. Lines: up to 30% of images moved from their Mahalanobis routes to a random later or any wrong expert. kk-shot: the Mahalanobis router fitted on k=2k=2 to 256 TRAIN images per domain. Euclidean, cosine: nearest TRAIN mean. Bands, bars: ±1 s.d. over five seeds.

DifferenceExtra errors toOfficeHome p=0p=0OfficeHome p=10%p=10\%OfficeHome p=30%p=30\%CDDB-Hard p=0p=0CDDB-Hard p=10%p=10\%CDDB-Hard p=30%p=30\%
Lamarck − reset adapterslater experts0.15 ± 0.15 (4/5)0.25 ± 0.15 (5/5)0.43 ± 0.18 (5/5)0.21 ± 0.39 (3/5)1.02 ± 0.31 (5/5)2.69 ± 0.34 (5/5)
any wrong expert0.15 ± 0.15 (4/5)0.21 ± 0.15 (5/5)0.35 ± 0.15 (5/5)0.21 ± 0.39 (3/5)0.74 ± 0.35 (5/5)1.85 ± 0.42 (5/5)
earlier experts0.15 ± 0.15 (4/5)0.15 ± 0.14 (4/5)0.15 ± 0.12 (4/5)0.21 ± 0.39 (3/5)0.18 ± 0.34 (4/5)0.06 ± 0.28 (3/5)
Lamarck − independent expertslater experts0.54 ± 0.28 (5/5)1.13 ± 0.24 (5/5)2.32 ± 0.19 (5/5)0.58 ± 0.75 (4/5)2.22 ± 0.73 (5/5)5.58 ± 0.69 (5/5)
any wrong expert0.54 ± 0.28 (5/5)1.02 ± 0.23 (5/5)1.97 ± 0.19 (5/5)0.58 ± 0.75 (4/5)1.64 ± 0.70 (5/5)3.80 ± 0.59 (5/5)
earlier experts0.54 ± 0.28 (5/5)0.64 ± 0.25 (5/5)0.79 ± 0.22 (5/5)0.58 ± 0.75 (4/5)0.57 ± 0.72 (4/5)0.49 ± 0.63 (4/5)
Prompts only: inherited − independentlater experts0.62 ± 0.17 (5/5)1.23 ± 0.17 (5/5)2.39 ± 0.20 (5/5)−0.85 ± 0.96 (1/5)0.80 ± 0.90 (4/5)4.09 ± 0.80 (5/5)
any wrong expert0.62 ± 0.17 (5/5)1.13 ± 0.19 (5/5)2.08 ± 0.22 (5/5)−0.85 ± 0.96 (1/5)0.36 ± 0.81 (3/5)2.76 ± 0. …
earlier experts0.62 ± 0.17 (5/5)0.71 ± 0.19 (5/5)0.84 ± 0.25 (5/5)−0.85 ± 0.96 (1/5)−0.68 ± 0.87 (1/5)−0.46 ± 0.55 (1/5)
Expert bankMahalanobis routerLast expert onlyMahalanobis routerLast expert only
Lamarck, independent87.93 ± 0.2680.79 ± 0.6792.35 ± 0.6159.31 ± 1.28
Lamarck, reset adapters88.32 ± 0.1684.85 ± 0.4992.73 ± 0.6369.12 ± 2.22
Lamarck (Ours)88.47 ± 0.2585.59 ± 0.1492.94 ± 0.6277.06 ± 2.15

Table S12. Routing-error injection. TEST ACC difference for three pairs of expert banks when an extra share pp of the TEST images goes from the Mahalanobis routes to a random later, any wrong or earlier expert (50 paired draws per seed); the first two rows give the lines of Figure 5; points, mean±s.d. over five seeds; in parentheses, seeds with a positive difference. Bottom: ACC with the Mahalanobis router and with the last expert only.

Conclusion

Per-domain learners select one expert for each test image with a router, and recent work trains this router. For the learners we analyze, a training-free Mahalanobis router costs under 2.7 ACC points; on each benchmark, this is less than the largest ACC difference between their expert banks, so the experts limit accuracy. Lamarck uses this router and strengthens the experts. Its adapters correct every image token after each deep block. Nine of our twelve baselines train nothing in these blocks. Each expert is initialized from its predecessor, which adds no parameters or computation and makes later experts more accurate on earlier domains. Inheritance thus adds more ACC as more images reach later experts. Against twelve frozen-backbone baselines, Lamarck is first on CDDB by 5.2 points, second on DN and OH, and highest in mean ACC.

Limitations. On DN, SOYO’s experts are more accurate than Lamarck’s on every domain, and SOYO’s ACC higher. Under the Mahalanobis router, inheritance changes ACC against reset adapters by −0.41 to +0.21 points; its gain becomes clear only under weaker routers or injected routing errors. Without adapters, inherited experts trail independent ones in ACC on CDDB (Table 4). The second backbone, DINOv2 ViT-B/14, is tested only on OH and CDDB, and the domain order is reversed only on CDDB. Storage exceeds that of the other prompt learners and grows by one expert per domain; routing adds an unprompted pass.

LLM Usage Statement

The authors used OpenAI tools (ChatGPT and Codex) and Anthropic tools (Claude and Claude Code) to support research and manuscript preparation. The research hypotheses were formulated by the authors based on limitations they identified in prior work. The tools supported the critical examination of these hypotheses and the proposed methodology, providing suggestions and feedback on the methodological rationale and the empirical support for the reported claims. They also facilitated experimental implementation through code generation and refactoring, including routines for data loading, preprocessing, and dataset splitting. During manuscript preparation, the tools assisted with English translation, language revision, preparation of visual material, and the organization and refinement of research and related work material prepared by the authors.

The authors critically assessed the outputs and suggestions provided by these tools and incorporated or revised them as appropriate. All final scientific decisions, including those concerning the research direction, methodology, experimental evaluation, and interpretation of results, were made by the authors. The authors reviewed the technical implementation and complete manuscript and take full responsibility for the accuracy and integrity of the submitted work.

Algorithm S1 Lamarck: training and inference

Input: domains D1,…,DT\mathcal{D}_1,\ldots,\mathcal{D}_T in sequence; frozen backbone Θ\Theta

Training

  1. for t=1,…,Tt=1,\ldots,T do

  2. if t>1t>1: ψt←ψt−1\psi_t \leftarrow\psi_{t-1} else Ub←0U_b \leftarrow0 ▷ inheritance

  3. add the prompt rows of stage tt to the pool

  4. train them, all {Db,Ub}\{D_b,U_b\} and (ffc,w)(f_{\mathrm{fc}},w) on Dt\mathcal{D}_t ▷ Θ\Theta frozen

  5. freeze and store ψt\psi_t

  6. μt←\mu_t \leftarrow mean of z(x)z(x) over Dt\mathcal{D}_t; update SS and Σ\Sigma

Inference on a test image xx after stage tt

  1. z←z(x)z \leftarrow z(x) ▷ unprompted pass

  2. e⋆←arg⁡min⁡e≤t(z−μe)⊤Σ−1(z−μe)e^\star\leftarrow\arg\min_{e\leq t}(z-\mu_e)^\top\Sigma^{-1}(z-\mu_e) ▷ Eq. (8)

  3. return arg⁡max⁡ylogits⁡(x)y\arg\max_y \operatorname{logits}(x)_y of ψe⋆\psi_{e^\star} ▷ prompted pass

As in the main paper, ACC and AIA are the final and the average incremental accuracy (Eq. (3)), and TRAIN, VAL and TEST are the training, validation and test splits. Tables, figures, sections and equations without the prefix S are those of the main paper.

Implementation details

All learners use the same ViT-B/16 timm checkpoint: vit_base_patch16_224.augreg_in21k_ft_in1k.

Lamarck. Lamarck follows the protocol of § 6.1 with a cosine learning-rate schedule, and the last epoch is kept. In DomainNet, where T=6T=6 does not divide ZZ, the last stage also receives the four remaining rows. In independent experts, the prompt rows of earlier stages stay visible at their initial values. Two readouts were compared on prompt-only experts: the adopted one (Eq. (7)) and one with an additional cosine classifier and cosine loss. They differ by at most 0.2 points in ACC and AIA on VAL and TEST. Algorithm S1 summarizes training and inference.

Splits. DomainNet uses the official test set as TEST and holds out 10% of the official training set as VAL. OfficeHome, which has no official split, is divided at random into 70/14/16% TRAIN/VAL/TEST. For CDDB-Hard we pool the official training and validation images of each domain and re-split them 80/10/10%, keeping duplicate images within one split. All methods and seeds share these splits and the same preprocessing.

Hardware. Lamarck and its prompt-only experts were trained on a V100 (PyTorch 2.4.1) and an RTX PRO 6000 (PyTorch 2.9.1). The GPU has little effect: it changes the adapter gain by −0.02 points on OfficeHome (Mahalanobis router) and −0.27 on CDDB-Hard (prototype router), and Lamarck’s OfficeHome ACC by 0.08 points (88.39±0.20 on the RTX PRO 6000, 88.47±0.25 on the V100).

Baseline settings

For a fair comparison, every baseline of Table 2 runs inside one harness that fixes the backbone, the splits, the domain order and the per-domain budget; Table S1 lists the settings of each method. The harness also fixes the prompt budget of the prompt learners at the values released with CODA-Prompt, OS-Prompt and CPS-Prompt: prompts of 8 tokens (4 key and 4 value tokens for prefix prompts) and, where a method keeps a pool, 100 prompts; UniPrompt-CL keeps its released pool. The Adam setting of § 6.1 also trains SOYO’s router, and SLCA and DUCT, which fine-tune the backbone, keep their released SGD settings within the same budget. SOYO runs the model, loss and inference code of its release. The all-expert evaluation of § 5 reproduces each learner’s own routed predictions exactly in every evaluated cell.

Own-domain accuracy of S-iPrompt

S-iPrompt is within 0.35 points of Lamarck’s prompt-only experts on every DomainNet domain except Sketch (1.78 points; Table S2).

Prompt retrieval

Table S3 gives, per prompted block, the retrieval statistics summarized in § 6.3. For Table S4, every pool is queried with the router’s frozen feature: QiQ_i in Eq. (2) is replaced by z(x)−μˉz(x)-\bar{\mu}, with μˉ\bar{\mu} the mean of the first domain’s router prototypes, and γ\gamma by γ/τ\gamma/\tau, with τ=0.3\tau=0.3 chosen on VAL; the sharpest retrieval (τ=0.03\tau=0.03) has the lowest VAL ACC.

Fine-tuned backbones: SLCA and DUCT

SLCA [40] fine-tunes the backbone with a reduced learning rate and aligns the classifier with class-wise feature statistics after each stage. Averaged over the three benchmarks, SLCA and DUCT reach 82.49 and 72.14 ACC, against 83.62 for Lamarck (Table S5).

Other views of CDDB-Hard

Table S6 adds to the CDDB-Hard ACC of Table 2 three other aggregates and the accuracy on each domain; Lamarck is also first on every domain except SAN. With the domains learned in reverse (SAN to GauGAN), Lamarck reaches 93.15 ACC, against 92.94 in the original order, and leads Hash and HiDe-Prompt in every seed, by 4.70 and 6.54 points (Table S7); its AIA in that order is also above theirs (90.12 vs. 86.30 and 83.71).

Baselines on DomainNet

On DomainNet, the gap between shared-pool and per-domain learners is forgetting, not weaker training. Averaged over the six domains and five seeds, the accuracy on a domain right after learning it is 71.34% for CODA-Prompt and 68.36–71.20% for OS-Prompt, OS-Prompt++, KA-Prompt, CPS-Prompt and UniPrompt-CL (UniPrompt-CL over four seeds), against 71.01% for SOYO, 69.46% for CONEC-LoRA, 68.87% for Hash, 67.03% for S-iPrompt, 66.19% for HiDe-Prompt and 64.15% for EASE. After the last domain, the six shared-pool learners keep at most 50.62% (Table 2); CODA-Prompt’s largest losses are on QuickDraw (68.39 to 21.22) and Infograph (45.02 to 24.40). Lamarck is more accurate than EASE on every domain and than CONEC-LoRA on four of six, most on QuickDraw (69.76% vs. 50.32% and 60.66%), while CONEC-LoRA leads on Infograph (44.92% vs. 44.16%) and Painting (73.35% vs. 70.79%; Table S8).

DomainEASE [41]CONEC-LoRA [20]Lamarck (Ours)
Clipart69.19 ± 0.0877.10 ± 0.1777.58 ± 0.27
Infograph35.84 ± 0.0744.92 ± 0.1244.16 ± 0.44
Painting67.52 ± 0.0473.35 ± 0.1870.79 ± 0.16
QuickDraw50.32 ± 0.0960.66 ± 0.6569.76 ± 0.09
Real82.99 ± 0.0383.70 ± 0.0985.00 ± 0.04
Sketch62.33 ± 0.0269.16 ± 0.0869.47 ± 0.23
ACC↑61.36 ± 0.0268.15 ± 0.1369.46 ± 0.14

Table S8.

Test-time and training cost

Table S9 gives the cost behind Figure 3, measured on one Tesla V100 (32 GB) in FP32 with batch size 64 and synthetic 224×224 inputs, each method in its released environment. EASE passes each test image through its six adapter stacks; SOYO’s released inference runs the encoder once per domain and keeps the output of the selected expert. GFLOPs per image are counted with PyTorch’s FLOP counter (matrix products, convolutions and attention; two FLOPs per multiply–add), and training GFLOPs include the backward pass. Param. excludes the 85.80M parameters of the backbone. Inference throughput is the mean over timed repeats (s.d. at most 0.90 images/s); Lamarck splits each batch evenly over its six experts and reaches 160.3 images/s when one expert serves the whole batch. A training step is one optimizer step of the last stage, timed over three repeats (s.d. at most 0.35 images/s) on a V100 of the same machine; Lamarck trains with one pass, through the expert being trained. Memory is the peak allocated by PyTorch.

Adapter rank and placement

Lamarck’s accuracy depends little on the rank of its adapters (Table S10): among r=8r=8, 32 (Lamarck’s rank) and 64, the rank that is best on VAL, r=64r=64 on all three benchmarks, changes TEST ACC by at most 0.4 points. VAL selects the adapted tokens: on DomainNet’s first domain, clipart (one seed), the first expert reaches an own-domain VAL accuracy of 80.04 with adapters on every image token after blocks 6–12 (Lamarck) and 78.43 when they correct only the CLS token (r=64r=64), against 77.54 as a prompt-only expert. Adapters in all twelve blocks (r=16r=16) reach 80.61 in the same setting, 0.57 points higher; we keep blocks 6–12 so that prompts and adapters occupy separate blocks, and report the twelve-block adapters as a control. With Lamarck’s prompts and inheritance kept, they are within 0.3 TEST ACC points of Lamarck on all three benchmarks; on DomainNet they reach 69.49, below SOYO’s 69.91 with its @page 14/21 starts_mid=0 ends_mid=1

Table S5. SLCA and DUCT, which fine-tune the backbone, lead on OfficeHome but trail Lamarck on CDDB-Hard and DomainNet. Five-seed TEST accuracy (%, mean±s.d.) on the splits, domain orders and seeds of Table 2; settings in Table S1.

Table S6. Lamarck is first on CDDB-Hard under every weighting of the five domains. TEST accuracy after the last domain (%, mean±s.d. over five seeds), each method with its own inference rule. Equal: domain mean (ACC of Table 2); Images: all 2,140 TEST images; w/o SAN: without the last domain; in parentheses, TEST images per domain. Best and second best among the learners of Table 2 in bold and underlined.

Table S7. Reversing the CDDB-Hard domain order changes Lamarck’s ACC by 0.2 points and keeps its lead over Hash and HiDe-Prompt. TEST accuracy (%, mean±s.d. over five seeds) with the domains learned in the order of Table 2 (GauGAN, BigGAN, Wild, WFIR, SAN) and in reverse; each baseline with its own inference rule, both Lamarck rows with the Mahalanobis router. * AIA is not comparable across the two orders: the first reversed stage is SAN alone (33 TEST images).

Table S8. On DomainNet, Lamarck is more accurate than EASE and CONEC-LoRA on every domain except Infograph and Painting, where CONEC-LoRA leads. TEST accuracy on each domain after the last domain (%, mean±s.d. over five seeds), each method with its own inference rule, Lamarck with the Mahalanobis router; ACC: mean over the domains (Table 2).

Further results on inheritance

OfficeHome. In Figure S1, Lamarck is 8.58 points more accurate than independent experts above the diagonal (every cell and seed) and 2.04 points more accurate below it, so on OfficeHome inheritance helps mainly, but not only, the later experts.

Figure S1. Figure 4 on OfficeHome. Panels, cells and dot areas as in Figure 4; TEST, five-seed means.

DomainNet. In Figure S2, Lamarck is 6.98 points more accurate than independent experts above the diagonal (every cell and seed) and 3.47 points more accurate below it.

Figure S2. Figure 4 on DomainNet. Panels, cells and dot areas as in Figure 4, with the colour scale starting at 30%; TEST, five-seed means.

Against reset adapters, it is 2.29 points more accurate above the diagonal and 1.13 points below it, so on DomainNet, too, inheritance helps mainly the later experts. The router misroutes 11.9% of DomainNet’s TEST images but only 0.1% of QuickDraw images, on which every other expert reaches at most 17.0% accuracy. Of the misrouted images, 63% either come from Real, the largest domain, or are sent to its expert; the most frequent case is a Painting image sent to the Real expert (17.5% of all misroutes). On the misrouted images, Lamarck’s selected experts reach 60.1% accuracy, against 76.9% for the own-domain experts.

Routing conditions. Table S11 completes the differences of § 6.4 with domain-label routing, the share of misrouted images and the s.d. over seeds. On DomainNet, inheritance lowers both the later- and the earlier-expert cost against reset adapters in every seed (Table 6), yet its ACC under the Mahalanobis router is 69.46 against 69.87, because reset adapters are 0.41 points more accurate on their own domains, in all five seeds, and the router loses 2.47 points with either bank (Tabs. 1 and S11). On CDDB-Hard, inheritance lowers the later-expert cost but not the earlier-expert cost (38.93 vs. 37.84 for reset adapters; Tab. 6).

Routing-error injection. Table S12 gives the values behind the lines of Figure 5, together with errors injected toward earlier experts, and repeats the injection, with the same draws, against independent experts and for prompt-only experts. Against independent experts too, the advantage of inheritance grows with pp when the extra errors reach later experts and grows far less, or not at all, when they reach earlier experts. Inherited prompt-only experts trail independent ones on CDDB-Hard at the router’s own error rate, p=0p=0 (−0.85-0.85 points), but lead at p=30%p=30\% by 4.09 points when the extra errors go to later experts and by 2.76 when they go to any wrong expert. When the last expert classifies every image, Lamarck reaches 48.38% on DomainNet, against 47.06% with reset adapters and 42.21% with independent experts; the bottom rows of Tab. S12 give these values on OfficeHome and CDDB-Hard. Figure 5 draws each injected share pp at the FF it gives in expectation from each seed’s Mahalanobis routes; errors injected toward earlier experts do not raise FF and are drawn against MM in Figure S3.

Figure S3. Against the share of all misrouted images, routers and injected errors align less well than against FF. Figure 5 with MM, the share of TEST images routed to any other expert (mean over domains), on the horizontal axis, and with errors injected toward earlier experts (dashed).

Routers that make their own errors. Routers that send more TEST images to later experts raise the advantage of inheritance as injected errors do (Tab. S13, markers of Figure 5). All routers are training-free, use only TRAIN statistics of the frozen feature z(x)z(x) of Eq. (8) and give every expert bank the same routes: the Mahalanobis router fitted on all TRAIN images or on k∈{2,4,8,16,32,64,128,256}k \in\{2, 4, 8, 16, 32, 64, 128, 256\} TRAIN images per domain, drawn without replacement (20 draws per seed and kk, averaged within each seed), Euclidean and cosine nearest-mean routers, the prototype router of Sec. 5 and the last expert alone. The routers, the kk grid and the slope comparison were fixed before any expert bank was scored on TEST. On CDDB-Hard, every router that sends more images to later experts than the Mahalanobis router gives Lamarck a lead over reset adapters in all five seeds; under the kk-shot routers it rises from +0.70 at k=256k=256 to +2.32 at k=2k=2. Images routed to their own expert contribute +0.24 to +0.42 under every router, so the growth comes from images at later experts (+0.05 under the Mahalanobis router, +2.06 at $k=2). The least-squares slope of the lead on FF, fitted per seed over the kk-shot draws and the Mahalanobis router, is 0.129±0.050 points per point of FF (0.072 to 0.201 across seeds), against 0.105 for errors injected toward later experts and 0.121 predicted by the 12.06-point gap in later-expert cost (Tab. 4); three of five seeds lie at or above both. On OfficeHome, the lead is +0.18 at $k=256

and +0.49 at k=2k=2, positive in all five seeds at every kk, and its slope, 0.016±0.006, is close to the 0.015 that the 1.51-point gap in later-expert cost predicts. On DomainNet, the difference from reset adapters also rises with FF, by 0.031±0.006 points per point of FF (0.022 to 0.037 across seeds), close to the 0.026 implied by the 2.55-point gap in later-expert cost (Tab. 6). It is −0.41 under the Mahalanobis router (F=9.18%F=9.18\%) and −0.13 to −0.36 under every other router except the k=2k=2 router (+0.13, F=20.70%F=20.70\%, four of five seeds) and the last expert (+1.32); images routed to their own expert lower it by 0.15 to 0.38 points under every router, as reset adapters are more accurate on their own domains. Against independent experts, the lead grows from +1.18 to +4.42 on CDDB-Hard, from +1.27 to +3.07 on OfficeHome and from +0.44 to +1.67 on DomainNet between k=256k=256 and k=2k=2 (slopes 0.259±0.049, 0.108±0.018 and 0.105±0.004); on DomainNet it is +0.08 under the Mahalanobis router (four of five seeds) and positive in every seed under every other router. On CDDB-Hard, the lead over reset adapters rises alike when ACC is taken over all TEST images instead of the domain mean or without SAN, from +0.09 and +0.11 under the Mahalanobis router to +2.67 and +2.82 at k=2k=2, and is positive under every router. At k=128k=128, where the pooled covariance is fitted from 640 (CDDB-Hard) and 512 (OfficeHome) TRAIN images in 768 dimensions, FF exceeds that at k=64k=64 (9.36% vs. 7.27% and 20.56% vs. 19.82%), and so does the CDDB-Hard lead (+1.09 vs. +0.81). Against MM, the share of images routed to any other expert, routers and injected errors align less well (Fig. S3), because the routers also send many images to earlier experts (31.1% on OfficeHome at k=2k=2), which change the lead little (+0.09).

Reversed domain order. With the CDDB-Hard domains learned from SAN to GauGAN, inheritance still lowers the later-expert cost in every seed, by 6.40±1.15 points (27.85 vs. 34.25) instead of 12.06±1.62, and adds 3.29 instead of 7.94 points when the last expert classifies every image, also in every seed (Tab. S14). Under the Mahalanobis router it changes ACC by +0.03±0.26 (two of five seeds positive) instead of +0.21±0.39, under the prototype router by +0.34 instead of +0.77, DRA by −0.10 instead of +0.31, and the earlier-expert cost by +0.83 instead of +1.08. On the same domain pairs, an expert trained after the image’s domain costs less than one trained before it, and inheritance enlarges this drop 2.5 to 3.1 times (Tab. S15). The two orders give each pair (image domain, expert domain) once with the expert trained after the image’s domain and once before it; with inherited adapters, the drop is 21.74 points on the ten pairs whose expert follows the image’s domain in the original order (U) and 11.08 on the ten converse pairs (L), against 8.85 and 3.60 with reset adapters, positive in every seed.

to 94.01 on CDDB-Hard (Tab. S16). Against reset adapters, inheritance lowers the later-expert cost (5.93 vs. 9.01 on OfficeHome, 10.02 vs. 18.31 on CDDB-Hard) and raises the accuracy of the last expert alone by 1.50 and 7.36 points, both in every seed, while under the Mahalanobis router it changes ACC by +0.20 on OfficeHome and −0.16 on CDDB-Hard (Table S17). We compare with UniPrompt-CL, S-iPrompt, the authors’ design and evaluate on DINOv2-B, and with S-iPrompt, the per-domain prompt learner of our diagnosis (§ 5); Hash, HiDe-Prompt, CONEC-LoRA, EASE and SOYO were not run on DINOv2.

Second backbone: DINOv2 ViT-B/14

On DINOv2 ViT-B/14, Lamarck runs on OfficeHome and CDDB-Hard with the configuration used on ViT-B/16, without any tuning on DINOv2, over five seeds. Its adapters raise ACC over Lamarck’s prompt-only experts in every seed, from 88.56 to 89.34 on OfficeHome and from 91.16

Table S13. Lamarck’s ACC difference from reset adapters and from independent experts rises as routers send more TEST images to later experts. Top: values behind the markers of Figure 5. MM, FF: share (%) of TEST images routed to another or to a later expert, mean over domains. −reset, −indep.: TEST ACC of Lamarck minus that with reset adapters or of independent experts; points, mean±s.d. over five seeds, in parentheses seeds with a positive difference. Own, Later: parts of −reset from images routed to their own or to a later expert (the rest is from earlier experts). k-shot: the Mahalanobis router fitted on kk TRAIN images per domain, averaged over 20 draws per seed; Euclidean, cosine: nearest TRAIN mean. Slope: per-seed least-squares slope of the difference on FF over the k-shot draws and the Mahalanobis router (points per point of FF). Mis. in Table S11 is MM weighted by images instead of domains. Bottom: DomainNet.

Routerkk\multicolumn{6}{c}{OfficeHome}\multicolumn{6}{c}{CDDB-Hard}
MMFF−resetOwnLater−indep.MMFF−resetOwnLater−indep.
Mahalanobisall17.479.65+0.15 ± 0.15 (4/5)+0.04+0.06+0.54 ± 0.28 (5/5)6.451.28+0.21 ± 0.39 (3/5)+0.32+0.05+0.58 ± 0.75 (4/5)
k-shot25635.8818.72+0.18 ± 0.10 (5/5)0.00+0.13+1.27 ± 0.12 (5/5)13.426.44+0.70 ± 0.46 (5/5)+0.29+0.53+1.18 ± 0.77 (4/5)
12839.5920.56+0.29 ± 0.11 (5/5)−0.01+0.23+1.46 ± 0.15 (5/5)18.619.36+1.09 ± 0.39 (5/5)+0.29+0.91+1.88 ± 0.70 (5/5)
6436.9319.82+0.30 ± 0.15 (5/5)0.00+0.23+1.20 ± 0.13 (5/5)15.807.27+0.81 ± 0.37 (5/5)+0.32+0.63+1.49 ± 0.56 (5/5)
3240.0421.73+0.29 ± 0.13 (5/5)+0.01+0.22+1.25 ± 0.17 (5/5)17.087.88+0.90 ± 0.36 (5/5)+0.31+0.71+1.74 ± 0.65 (5/5)
1646.0724.28+0.25 ± 0.14 (5/5)+0.01+0.17+1.49 ± 0.12 (5/5)20.469.86+1.24 ± 0.34 (5/5)+0.36+0.99+2.34 ± 0.44 (5/5)
852.0626.90+0.38 ± 0.15 (5/5)+0.01+0.27+1.97 ± 0.16 (5/5)25.3412.63+1.71 ± 0.48 (5/5)+0.33+1.40+3.20 ± 0.53 (5/5)
458.5929.80+0.40 ± 0.10 (5/5)+0.01+0.31+2.46 ± 0.41 (5/5)30.1815.70+1.94 ± 0.62 (5/5)+0.30+1.67+3.75 ± 0.89 (5/5)
263.0732.02+0.49 ± 0.14 (5/5)+0.01+0.39+3.07 ± 0.25 (5/5)37.4718.37+2.32 ± 0.78 (5/5)+0.27+2.06+4.42 ± 0.55 (5/5)
Prototypeall37.2719.31+0.12 ± 0.14 (4/5)0.00+0.10+1.10 ± 0.34 (5/5)14.355.09+0.77 ± 0.47 (5/5)+0.32+0.51+1.47 ± 0.81 (5/5)
Euclideanall36.5616.90+0.17 ± 0.17 (4/5)+0.01+0.06+0.80 ± 0.21 (5/5)18.547.65+1.07 ± 0.30 (5/5)+0.42+0.84+2.26 ± 0.44 (5/5)
Cosineall35.7916.91+0.27 ± 0.13 (5/5)−0.03+0.18+0.93 ± 0. …19.388.84+1.15 ± 0.71 (5/5)+0.42+1.02+2.41 ± 0.66 (5/5)
Last expert only−75.0075.00+0.73 ± 0.42 (5/5)+0.01+0.72+4.80 ± 0.66 (5/5)80.0080.00+7.94 ± 3.38 (5/5)+0.24+7.70+17.75 ± 2.38 (5/5)
Slope on FF0.016 ± 0.0060.108 ± 0.0180.129 ± 0.0500.259 ± 0.049

Table S13.

Routerkk\multicolumn{6}{c}{DomainNet}
MMFF−resetOwnLater−indep.
Mahalanobisall15.439.18−0.41 ± 0.16 (0/5)−0.38−0.02+0.08 ± 0.07 (4/5)
k-shot25620.7012.07−0.30 ± 0.16 (0/5)−0.32+0.04+0.44 ± 0.09 (5/5)
12834.8617.12−0.13 ± 0.17 (2/5)−0.28+0.14+0.97 ± 0.08 (5/5)
6429.9714.81−0.23 ± 0.16 (0/5)−0.30+0.07+0.67 ± 0.07 (5/5)
3227.6413.59−0.27 ± 0.15 (0/5)−0.31+0.04+0.49 ± 0.08 (5/5)
1628.4014.23−0.28 ± 0.15 (0/5)−0.31+0.04+0.51 ± 0.08 (5/5)
832.0415.56−0.23 ± 0.15 (0/5)−0.29+0.06+0.67 ± 0.09 (5/5)
437.2417.82−0.15 ± 0.15 (0/5)−0.27+0.11+0.96 ± 0.11 (5/5)
245.9720.70+0.13 ± 0.17 (4/5)−0.23+0.26+1.67 ± 0.12 (5/5)
Prototypeall22.5811.33−0.32 ± 0.16 (0/5)−0.33+0.03+0.33 ± 0.06 (5/5)
Euclideanall21.7712.01−0.36 ± 0.16 (0/5)−0.33−0.01+0.22 ± 0.06 (5/5)
Cosineall21.6111.26−0.35 ± 0.16 (0/5)−0.32−0.01+0.25 ± 0.08 (5/5)
Last expert only−83.3383.33+1.32 ± 0.61 (5/5)−0.15+1.48+6.17 ± 0.58 (5/5)
Slope on FF0.031 ± 0.0060.105 ± 0.004

Table S13.

Table S14. In both CDDB-Hard domain orders, inheritance lowers the later-expert cost in every seed and adds the most ACC when the last expert classifies every image. Lamarck and Lamarck with reset adapters in the original and the reversed domain order (Table S7); Δ: Lamarck minus reset adapters, paired by seed, in parentheses seeds with a positive difference, so 0/5 on a cost row means that Lamarck’s cost is lower in every seed. Later- and earlier-expert cost as in Eq. (5); TEST, mean±s.d. over five seeds.

Metric\multicolumn{3}{c}{Original order (GauGAN to SAN)}\multicolumn{3}{c}{Reversed order (SAN to GauGAN)}
Reset adaptersLamarck (Ours)ΔReset adaptersLamarck (Ours)Δ
ACC, Mahalanobis router92.73 ± 0.6392.94 ± 0.62+0.21 ± 0.39 (3/5)93.12 ± 0.8993.15 ± 0.96+0.03 ± 0.26 (2/5)
ACC, prototype router90.84 ± 0.7191.61 ± 0.73+0.77 ± 0.47 (5/5)90.96 ± 0.8791.30 ± 1.23+0.34 ± 0.47 (3/5)
ACC, last expert only69.12 ± 2.2277.06 ± 2.15+7.94 ± 3.38 (5/5)66.93 ± 1.0070.22 ± 2.21+3.29 ± 1.81 (5/5)
DRA94.47 ± 0.8794.78 ± 0.53+0.31 ± 0.99 (2/5)94.91 ± 0.7194.81 ± 0.50−0.10 ± 0.25 (1/5)
Later-expert accuracy70.06 ± 1.4182.12 ± 0.87+12.06 ± 1.43 (5/5)57.64 ± 1.3063.94 ± 1.05+6.30 ± 1.38 (5/5)
Later-expert cost↓27.81 ± 1.4215.75 ± 0.84−12.06 ± 1.62 (0/5)34.25 ± 1.1027.85 ± 1.43−6.40 ± 1.15 (0/5)
Earlier-expert cost↓37.84 ± 1.4938.93 ± 1.46+1.08 ± 2.13 (3/5)36.66 ± 0.7137.49 ± 1.19+0.83 ± 0.79 (5/5)

Table S14.

Table S15. On the same domain pairs, an expert trained after the image’s domain costs less than one trained before it, and inheritance enlarges this drop 2.5 to 3.1 times. Cost (Eq. (5)) of the ten CDDB-Hard pairs (image domain, expert domain) in which the expert’s domain follows the image’s domain in the original order (U) and of the ten converse pairs (L), with the expert trained after the image’s domain (original order for U, reversed order for L) or before it. Drop: before minus after, paired by seed, in parentheses seeds with a positive drop; TEST, mean±s.d. over five seeds.

PairsExpert bankCost, expert trained after↓Cost, expert trained before↓Drop↑
UReset adapters27.81 ± 1.4236.66 ± 0.71+8.85 ± 1.43 (5/5)
Lamarck (Ours)15.75 ± 0.8437.49 ± 1.19+21.74 ± 1.73 (5/5)
LReset adapters34.25 ± 1.1037.84 ± 1.49+3.60 ± 1.91 (5/5)
Lamarck (Ours)27.85 ± 1.4338.93 ± 1.46+11.08 ± 1.44 (5/5)

Table S15 (PDF p. 20).

Table S16. On DINOv2 ViT-B/14, the adapters raise AIA and ACC over Lamarck’s prompt-only experts, and Lamarck is above S-iPrompt and UniPrompt-CL. TEST AIA and ACC (%), mean±s.d. over five seeds, with the protocol and configuration of ViT-B/16 and no tuning on DINOv2. Lamarck rows use the Mahalanobis router; S-iPrompt and UniPrompt-CL predict as in Table 2. Inherited and reset adapters are within 0.2 ACC points of each other (Table S17).

MethodOfficeHome AIA↑OfficeHome ACC↑CDDB-Hard AIA↑CDDB-Hard ACC↑
S-iPrompt85.52 ± 0.7787.55 ± 0.2592.39 ± 0.2886.28 ± 0.60
UniPrompt-CL83.81 ± 0.4485.69 ± 0.1991.65 ± 0.3180.89 ± 2.42
Lamarck, prompts only85.81 ± 0.8188.56 ± 0.2195.33 ± 0.3291.16 ± 1.11
Lamarck, reset adapters87.65 ± 0.5589.15 ± 0.4097.82 ± 0.2094.17 ± 0.98
Lamarck (Ours)87.57 ± 0.1389.34 ± 0.0997.68 ± 0.1794.01 ± 0.99

Table S16 (PDF p. 20).

References

  1. [1]Rahaf Aljundi, Punarjay Chakravarty, and Tinne Tuytelaars. Expert Gate: Lifelong learning with a network of experts. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017. 1, 3arxiv.org/abs/1611.06194
  2. [2]Anonymous. When better routing hides a worse prompt update: Classifier-conditioned evaluation in domain-incremental learning. ICLR 2027 submission, OpenReview, 2026. 3, 5
  3. [3]Yewon Byun, Sanket Vaibhav Mehta, Saurabh Garg, Emma Strubell, Michael Oberst, Bryan Wilder, and Zachary C. Lipton. Expert routing with synthetic data for domain incremental learning. Transactions on Machine Learning Research, 2025. 1, 2, 5
  4. [4]Shoufa Chen, Chongjian Ge, Zhan Tong, Jiangliu Wang, Yibing Song, Jue Wang, and Ping Luo. AdaptFormer: Adapting vision transformers for scalable visual recognition. In Advances in Neural Information Processing Systems, 2022. 2, 3arxiv.org/abs/2205.13535
  5. [5]Tianqi Chen, Ian Goodfellow, and Jonathan Shlens. Net2Net: Accelerating learning via knowledge transfer. In International Conference on Learning Representations, 2016. 3arxiv.org/abs/1511.05641
  6. [6]Dipam Goswami, Yuyang Liu, Bartłomiej Twardowski, and Joost van de Weijer. FeCAM: Exploiting the heterogeneity of class distributions in exemplar-free continual learning. In Advances in Neural Information Processing Systems, 2023. 3, 4arxiv.org/abs/2309.14062
  7. [7]Junxian He, Chunting Zhou, Xue zhe Ma, Taylor Berg-Kirkpatrick, and Graham Neubig. Towards a unified view of parameter-efficient transfer learning. In International Conference on Learning Representations, 2022. 3arxiv.org/abs/2110.04366
  8. [8]Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin de Laroussilhe, Andrea Gesmundo, Mona Attarian, and Sylvain Gelly. Parameter-efficient transfer learning for NLP. In International Conference on Machine Learning, 2019. 2, 3, 4
  9. [9]Edward J. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen-Zhu, Yuanzhi Li, Shean Wang, Lu Wang, and Weizhu Chen. LoRA: Low-rank adaptation of large language models. In International Conference on Learning Representations, 2022. 3arxiv.org/abs/2106.09685
  10. [10]Wei-Cheng Huang, Chun-Fu Chen, and Hsiang Hsu. OVOR: OnePrompt with virtual outlier regularization for rehearsal-free class-incremental learning. In International Conference on Learning Representations, 2024. 2arxiv.org/abs/2402.04129
  11. [11]Menglin Jia, Luming Tang, Bor-Chun Chen, Claire Cardie, Serge Belongie, Bharath Hariharan, and Ser-Nam Lim. Visual prompt tuning. In European Conference on Computer Vision, 2022. 3
  12. [12]Gyuhak Kim, Changnan Xiao, Tatsuya Konishi, Zixuan Ke, and Bing Liu. A theoretical study on solving continual learning. In Advances in Neural Information Processing Systems, 2022. 3arxiv.org/abs/2211.02633
  13. [13]Youngeun Kim, Yuhang Li, and Priyadarshini Panda. One-stage prompt-based continual learning. In European Conference on Computer Vision, 2024. 1, 2, 3, 4, 5, 6, 15, 16arxiv.org/abs/2402.16189
  14. [14]Kimin Lee, Kibok Lee, Honglak Lee, and Jinwoo Shin. A simple unified framework for detecting out-of-distribution samples and adversarial attacks. In Advances in Neural Information Processing Systems, 2018. 2, 3, 4arxiv.org/abs/1807.03888
  15. [15]Chuqiao Li, Zhiwu Huang, Handa Pan, Pavel Zadnuei, Yabin Wang, Mohamad Shahbazi, Xiaopeng Hong, and Luc Van Gool. A continual deepfake detection benchmark: Dataset, methods, and essentials. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2023. 1, 5arxiv.org/abs/2205.05467
  16. [16]Jiangyang Li, Chenhao Ding, Songlin Dong, Qiang Wang, Jianchao Zhao, Yuhang He, and Yihong Gong. Is parameter isolation better for prompt-based continual learning? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2026. 3, 6, 15, 16arxiv.org/abs/2601.20894
  17. [17]Wonseon Lim, Jaesung Lee, and Dae-Won Kim. Critical patch-aware sparse prompting with decoupled training for continual learning on the edge. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2026. 2, 6, 15, 16
  18. [18]Mark D. McDonnell, Dong Gong, Amin Parveneh, Ehsan Abbasnejad, and Anton van den Hengel. RanPAC: Random projections and pre-trained models for continual learning. In Advances in Neural Information Processing Systems, 2023. 3
  19. [19]Gyutae Oh and Jitae Shin. UniPrompt-CL: Efficient single-inference continual learning for medical image classification. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision, 2027. To appear. 2, 3, 4, 6, 15, 16
  20. [20]Naeem Paeedeh, Mahardhika Pratama, Weiping Ding, Jimmy Cao, Wolfgang Mayer, Ryszard Kowalczyk, and Ary Shiddiqi. Continual knowledge consolidation LORA for domain incremental learning. arXiv preprint arXiv:2510.16077v2, 2026. 2, 3, 5, 6, 15, 16arxiv.org/abs/2510.16077
  21. [21]Naeem Paeedeh, Mahardhika Pratama, Wolfgang Mayer, Mukesh Prasad, Weiping Ding, and Yew-Soon Ong. Few-shot domain incremental learning via continual vision-language consolidation. arXiv preprint arXiv:2606.30190, 2026. 3arxiv.org/abs/2606.30190
  22. [22]Seoyoung Park, Haemin Lee, and Hankook Lee. Is prompt selection necessary for task-free online continual learning? In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, Findings Track, 2026. 2arxiv.org/abs/2604.04420
  23. [23]Xingchao Peng, Qinxun Bai, Xide Xia, Zijun Huang, Kate Saenko, and Bo Wang. Moment matching for multi-source domain adaptation. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2019. 5arxiv.org/abs/1812.01754
  24. [24]Aleksandar Petrov, Philip H. S. Torr, and Adel Bibi. When do prompting and prefix-tuning work? A theory of capabilities and limitations. In International Conference on Learning Representations, 2024. 3arxiv.org/abs/2310.19698
  25. [25]James Seale Smith, Leonid Karlinsky, Vyshnavi Gutta, Paola Cascante-Bonilla, Donghyun Kim, Assaf Arbelle, Rameswar Panda, Rogerio Feris, and Zsolt Kira. CODA-Prompt: Continual decomposed attention-based prompting for rehearsal-free continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2023. 1, 2, 4, 5, 6, 15, 16arxiv.org/abs/2211.13218
  26. [26]Hai-Long Sun, Da-Wei Zhou, Hanbin Zhao, Le-Chuan Zhang, and Han-Jia Ye. MOS: Model surgery for pre-trained model-based class-incremental learning. In Proceedings of the AAAI Conference on Artificial Intelligence, 2025. 3arxiv.org/abs/2412.09441
  27. [27]Yuwen Tan, Qin­hao Zhou, Xiang Xiang, Ke Wang, Yuchuan Wu, and Yongbin Li. Semantically-shifted incremental adapter-tuning is a continual ViTransformer. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024. 3DOI
  28. [28]Longxiang Tang, Zhuotao Tian, Kai Li, Chunming He, Hantao Zhou, Hengshuang Zhao, Xiu Li, and Jiaya Jia. Mind the interference: Retaining pre-trained knowledge in parameter efficient continual learning of vision-language models. In European Conference on Computer Vision, 2024. 3arxiv.org/abs/2407.05342
  29. [29]Gido M. van de Ven, Tinne Tuytelaars, and Andreas S. Tolias. Three types of incremental learning. Nature Machine Intelligence, 4(12):1185–1197, 2022. 1DOI
  30. [30]Hemanth Venkateswara, Jose Eusebio, Shayok Chakraborty, and Sethuraman Panchanathan. Deep hashing network for unsupervised domain adaptation. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2017. 5
  31. [31]Liyuan Wang, Jingyi Xie, Xingxing Zhang, Mingyi Huang, Hang Su, and Jun Zhu. Hierarchical decomposition of prompt-based continual learning: Rethinking obscured sub-optimality. In Advances in Neural Information Processing Systems, 2023. 2, 3, 6, 15, 16arxiv.org/abs/2310.07234
  32. [32]Liyuan Wang, Xingxing Zhang, Hang Su, and Jun Zhu. A comprehensive survey of continual learning: Theory, method and application. IEEE Transactions on Pattern Analysis and Machine Intelligence, 46(8):5362–5383, 2024. 1DOI
  33. [33]Qiang Wang, Yuhang He, Songlin Dong, Xinyuan Gao, Shaokun Wang, and Yihong Gong. Non-exemplar domain incremental learning via cross-domain concept integration. In European Conference on Computer Vision, 2024. 3DOI
  34. [34]Qiang Wang, Xiang Song, Yuhang He, Jizhou Han, Chenhao Ding, Xinyuan Gao, and Yihong Gong. Boosting domain incremental learning: Selecting the optimal parameters is all you need. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025. 1, 2, 3, 5, 6, 15, 16arxiv.org/abs/2505.23744
  35. [35]Yabin Wang, Zhiwu Huang, and Xiaopeng Hong. S-Prompts learning with pre-trained transformers: An Occam’s razor for domain incremental learning. In Advances in Neural Information Processing Systems, 2022. 1, 2, 3, 5, 6, 15, 16
  36. [36]Zifeng Wang, Zizhao Zhang, Sayna Ebrahimi, Ruoxi Sun, Han Zhang, Chen-Yu Lee, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, and Tomas Pfister. DualPrompt: Complementary prompting for rehearsal-free continual learning. In European Conference on Computer Vision, 2022. 1, 2, 3arxiv.org/abs/2204.04799
  37. [37]Zifeng Wang, Zizhao Zhang, Chen-Yu Lee, Han Zhang, Ruoxi Sun, Xiaoqi Ren, Guolong Su, Vincent Perot, Jennifer Dy, and Tomas Pfister. Learning to prompt for continual learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2022. 1, 2arxiv.org/abs/2112.08654
  38. [38]Martin Wistuba, Prabhu Teja Sivaprasad, Lukas Balles, and Giovanni Zappella. Continual learning with low rank adaptation. arXiv preprint arXiv:2311.17601, 2023. 3arxiv.org/abs/2311.17601
  39. [39]Kunlun Xu, Xu Zhou, Gang Hua, and Jiahuan Zhou. Componential prompt-knowledge alignment for domain incremental learning. In International Conference on Machine Learning, 2025. 2, 3, 6, 15, 16arxiv.org/abs/2505.04575
  40. [40]Gengwei Zhang, Liyuan Wang, Guoliang Kang, Ling Chen, and Yunchao Wei. SLCA: Slow learner with classifier alignment for continual learning on a pre-trained model. In Proceedings of the IEEE/CVF International Conference on Computer Vision, 2023. 6, 13, 15arxiv.org/abs/2303.05118
  41. [41]Da-Wei Zhou, Hai-Long Sun, Han-Jia Ye, and De-Chuan Zhan. Expandable subspace ensemble for pre-trained model-based class-incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2024. 3, 6, 15, 16arxiv.org/abs/2403.12030
  42. [42]Da-Wei Zhou, Zi-Wen Cai, Han-Jia Ye, De-Chuan Zhan, and Ziwei Liu. Revisiting class-incremental learning with pre-trained models: Generalizability and adaptivity are all you need. International Journal of Computer Vision, 133(3): 1012–1032, 2025. 3arxiv.org/abs/2303.07338
  43. [43]Da-Wei Zhou, Zi-Wen Cai, Han-Jia Ye, Lijun Zhang, and De-Chuan Zhan. Dual consolidation for pre-trained model-based domain-incremental learning. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, 2025. 3, 5, 6, 15arxiv.org/abs/2410.00911

Paper details

vlnc:W_q4s529w8
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

ChatGPT, Codex, Claude, Claude Code

View statement
Looking for related papers…
Contents