betaFeedback

WorksMathematics

Modular Bayesian Foundation Models: Bayesian time series encoders for OOD detection, routing and forecasting

Abstract

Industrial systems continuously generate time series that vary in sampling frequency, scale, dynamics, and operating regime. Consequently a single time-series foundation model may not be able to represent all regimes effectively and retraining a large model for every new dataset or regime shift is computationally expensive. Modular foundation models address this issue by reusing specialised encoders trained on diverse datasets and by allowing the expansion of capabilities through the addition of new experts. This approach requires compact experts to minimize storage cost and a proper selection mechanism of experts for unseen data. We propose a modular foundation model framework that combines end-to-end Bayesian encoder training with uncertainty based routing. Using Singular Bayesian Neural Networks (SBNNs), we build a collection of compact Bayesian encoders whose epistemic uncertainty guides label-free selection. At the selected ranks, our Bayesian collection uses 63.8% less storage than its deterministic counterpart. We evaluate this framework on time-series data through distribution shift, encoder selection, and probabilistic forecasting. Across seven incoming datasets, uncertainty-based routing is competitive with distance-based routing in terms of forecast MSE, while reducing average ECE from 0.086 to 0.072 and average NLL from 0.907 to 0.836. It also increases PICP90 from 82.28% to 89.03%. In a separate model-family comparison, we evaluate the SBNN pipeline against full-rank Bayes by Backprop, deep ensembles, and MC Dropout. The SBNN pipeline achieves an average MSE of 0.390, comparable with 0.396 for deep ensembles, 0.389 for MC dropout, and 0.443 for the full-rank Bayesian model. It uses 63.9% fewer parameters than MC dropout, 81.9% fewer than the full-rank model, and 92.8% fewer than the deep ensemble.

find that uncertainty-based routing helps shortlist useful experts for the downstream task. However, the reconstruction ranking within this shortlist does not necessarily carry over to forecasting.

Introduction

Foundation models learn representations that are reusable across multiple datasets and tasks. For time series, TimesFM, Chronos, and Moirai provide pretrained forecasters that transfer across datasets [11, 2, 47], while MOMENT extends representation reuse to tasks beyond forecasting [19]. However, time-series data can vary significantly in terms of sampling frequency, dynamics, or scale. Model rankings also vary across time-series domains, so strong aggregate benchmark performance need not identify the best model for a given application [40]. Modular foundation models allow for the separate training of specialized encoders, which can subsequently be reused in combination. This enables the collection to expand without the need to retrain existing encoders [32]. However, the usefulness of an expanding collection depends on two questions: which experts should be used for unfamiliar data and how much does each expert cost to store? Selection is consequential: an encoder trained successfully on its source dataset need not provide a useful representation of a new one. Storing every encoder also incurs a cost that grows with the collection, even when only a small subset is deployed for a task.

In our framework, each expert is a Bayesian encoder, trained independently on a source dataset. Our contribution focuses on the experts themselves and the signal used to select them: our encoders carry weight posteriors and their epistemic uncertainty determines both their selection and their aggregation weights. The forecasting head is also Bayesian. Each encoder stores its uncertainty on its source domain as a reference. Thus, the selection and weighting of the frozen encoders requires no target labels or router training. We build on Singular Bayesian Neural Networks (SBNNs) [42]. For a layer weight matrix WW with mm rows and nn columns, SBNNs use the factorization W=ABTW = AB^{\mathsf{T}}. Here, A∈Rm×rA \in\mathbb{R}^{m \times r} and B∈Rn×rB \in\mathbb{R}^{n \times r} are Bayesian factor matrices, and rr is the chosen rank. The variational parameter count scales as O(r(m+n))O(r(m+n)) rather than O(mn)O(mn). For r<min⁡(m,n)r < \min(m,n), the induced weight posterior is singular with respect to Lebesgue measure and introduces dependencies between weights even when the factor posterior is mean-field. Shared factors propagate uncertainty across connected weights. Empirically, SBNNs improve out-of-distribution (OOD) detection over full-rank Bayes by Backprop (BBB) across the evaluated architectures. In time-series forecasting, they also achieve better interval coverage and calibration than deep ensembles and Rank-1 models, while using substantially fewer parameters [42]. We inherit this parameterization and posterior construction from SBNNs. Our contribution is to adapt them to a modular collection of encoders and establish how their uncertainty serves shift assessment, expert selection, and downstream prediction.

Our framework relates to work on modular model reuse and expert selection [32, 33]. Appendix § 6 discusses these connections, Bayesian compression, uncertainty-based routing and downstream transfer in detail.

Contributions. We develop a modular framework in which compact Bayesian encoders are trained end to end, retained as reusable experts, and combined with a Bayesian downstream head (§2). SBNN supplies the low-rank Bayesian parameterization; the present work studies how such experts can be selected and reused on incoming datasets. Our SBNN collection uses 63.8%1 fewer stored parameters than its deterministic counterpart with both posterior means and scales included (§3.3).

The Bayesian router uses each encoder’s source-normalized epistemic uncertainty to select the top-KK experts and assign their aggregation weights (§2.2), where KK is the number of selected encoders. These decisions require no target labels, router training, or encoder retraining. Across seven incoming datasets, uncertainty-based routing remains competitive with source-distance routing in forecasting mean squared error (MSE), while improving expected calibration error (ECE) [22], negative log-likelihood (NLL) [16], and prediction interval coverage probability at the nominal 90% level (PICP@90). Coverage increases from 82.28% to 89.03% (§ 4.4).

In a separate model-family comparison, we evaluate the SBNN pipeline against a five-member ensemble (Lakshminarayanan et al., [23]), MC Dropout (Gal & Ghahramani, [15]), and full-rank mean-field Bayes by Backprop (Blundell et al., [5]) (§ 4.3). SBNN maintains competitive forecasting accuracy and uncertainty calibration with substantially fewer stored parameters. Finally, the policy comparisons (§ 4.5) show that uncertainty helps identify a useful subset of experts for forecasting. Within this subset, lower reconstruction error does not consistently imply better forecasting performance.

An Uncertainty-Aware Modular Bayesian Framework

Problem formulation and pipeline overview

We consider a collection of MM encoders pretrained independently on different source datasets. Each encoder fm(⋅;θm)f_m(\cdot;\theta_m) maps an input time-series window to a representation and has a variational posterior qm(θm)q_m(\theta_m) over its weights. Here, m∈{1,…,M}m \in\{1,\ldots,M\} indexes the encoders, and θm\theta_m denotes the weight parameters of encoder mm. Several encoders may share a source dataset while differing in architecture or training configuration. Their learned representations do not necessarily have the same dimension. Source pretraining uses self-supervised learning (SSL): training targets are constructed from the inputs themselves. The pretext objective combines masked reconstruction (Vincent et al., [44]), learned with a decoder, and a contrastive term that compares representations of different views of the inputs (Chen et al., [8]). Following Toure & Stephens [42], we learn Gaussian means and scales for the low-rank factors, with factor-space Kullback–Leibler (KL) regularization. For a layer with an a×ba \times b weight matrix and rank rr, the factors require 2r(a+b)2r(a+b) stored parameters rather than abab; biases and other parameters are counted separately. We select compact rank configurations that remain competitive in reconstruction accuracy and calibration (Appendix § 9). After pretraining, we retain the encoder posterior and compute its source-domain uncertainty reference for routing. Appendix § 8 gives the loss definitions, Bayesian parameterization, initialization and training settings.

Given an incoming dataset, our goal is to select and combine encoders for downstream forecasting. We distinguish the unlabeled input windows XrouteX_{\mathrm{route}} used for selection from the supervised training pairs Dtrain={(xi,yi)}i=1N\mathcal{D}_{\mathrm{train}} = \{(x_i,y_i)\}_{i=1}^{N} used to fit the forecasting head. Here, NN is the number of training pairs, and yiy_i contains the future observations to predict from input window xix_i. Routing uses unlabeled input windows, which may come from Dtrain\mathcal{D}_{\mathrm{train}}. Expert selection is therefore label-free; downstream head training is supervised.

The routing step returns a subset S⊆{1,…,M}\mathcal{S} \subseteq\{1,\ldots,M\} of KK encoders and nonnegative aggregation weights {αm}m∈S\{\alpha_m\}_{m\in\mathcal{S}}, with ∑m∈Sαm=1\sum_{m\in\mathcal{S}}\alpha_m=1. Under our main policy, selection and weighting use each encoder’s epistemic uncertainty on the incoming inputs relative to a stored source-domain reference. The reference is computed before downstream reuse. Routing requires no target labels, router training, or access to the original source examples.

The selected representations are projected into a common space, combined using the routing weights, and passed to a Bayesian forecasting head. During head training, the encoder posteriors, selected subset, and aggregation weights remain fixed. At prediction, each draw samples the weights of all selected encoders and the head. The resulting forecasts are combined to estimate the predictive mean and uncertainty. The reconstruction decoders are excluded from forecasting; they are used only for pretraining and reconstruction evaluation. Figure 1 summarizes the pipeline.

Figure 1. The modular Bayesian pipeline. Padlocks mark frozen posteriors whose weights are sampled. The dashed boundary marks NMCN_{\mathrm{MC}} joint encoder–head draws; selection, projections and aggregation weights remain fixed.

Training-free routing with epistemic uncertainty

For an incoming dataset, we select experts by comparing each encoder’s uncertainty on the new inputs with its uncertainty on its own source data. The resulting ratios determine both the selected set and its aggregation weights. Figure 2 shows this procedure.

Figure 2. Training-free uncertainty routing. Posterior draws estimate uncertainty for each of the MM encoders. The KK lowest source-normalized ratios receive inverse-ratio weights; others receive zero weight. Subscript (j)(j) denotes rank jj; grids show array dimensions. Routing uses no target labels, decoder or head.

For each encoder mm, we draw SS weight samples from its frozen variational posterior. The representation of an input window xx is

zm(s)(x)=fm(Tm(x);θm(s)),θm(s)∼qm,s=1,…,S,(1)z_m^{(s)}(x) = f_m\left(T_m(x); \theta_m^{(s)}\right), \qquad\theta_m^{(s)} \sim q_m,\quad s=1,\ldots,S, \tag*{(1)}

where TmT_m applies the formatting and preprocessing required by encoder mm. The input stays fixed across draws, so changes in the representation come from the sampled weights. We measure this variation by the standard deviation of each representation coordinate, then average over coordinates and the input windows in a set X\mathcal{X}:

Um(X)=1∣X∣dm∑x∈X∑j=1dmStd⁡^s=1,…,S[zm,j(s)(x)],(2)U_m(\mathcal{X}) = \frac{1}{|\mathcal{X}|d_m}\sum_{x\in\mathcal{X}}\sum_{j=1}^{d_m}\widehat{\operatorname{Std}}_{s=1,\ldots,S}\left[z_{m,j}^{(s)}(x)\right], \tag*{(2)}

where dmd_m is the representation dimension. This statistic measures encoder epistemic uncertainty. We compute it directly from the encoder representations.

Raw uncertainty levels differ across encoders. We therefore compare each encoder with its own source reference, UmsrcU_m^{\mathrm{src}}, computed using the same statistic before downstream reuse. For incoming windows Xroute\mathcal{X}_{\mathrm{route}}, the ratio is

rm=Um(Xroute)Umsrc.(3)r_m = \frac{U_m(\mathcal{X}_{\mathrm{route}})}{U_m^{\mathrm{src}}}. \tag*{(3)}

A ratio above one indicates greater uncertainty than on the source data; a ratio below one indicates less uncertainty. Only a scalar source uncertainty reference per encoder is needed at routing time.

Our main policy selects the KK encoders with the smallest ratios. For the selected set S\mathcal{S}, normalized inverse-ratio weights are

αm={rm−1∑j∈Srj−1,m∈S,0,m∉S.(4)\alpha_m = \begin{cases} \dfrac{r_m^{-1}}{\sum_{j\in\mathcal{S}}r_j^{-1}}, & m\in\mathcal{S},\\ 0, & m\notin\mathcal{S}. \end{cases} \tag*{(4)}

Lower ratios receive larger weights. These expressions assume positive source references and ratios. The choices of KK and the comparison policies are given in § 3.2.

Routing is performed once per incoming dataset. It requires posterior forward passes through every candidate encoder, without parameter updates or target labels. Selection uses the uncertainty ratios. Reconstruction errors and forecasting targets are used for evaluation. The selected set and its weights then remain fixed during downstream training and prediction.

Remark (family comparison). Each uncertainty family routes independently; the deterministic pipeline reuses SBNN selections and weights (§ 3.2). Its heteroscedastic regression head predicts the target yy and an input-dependent standard deviation σ\sigma.

Representation aggregation and downstream prediction

The selected encoders may produce representations of different dimensions. For each encoder, we compute a coordinate-wise mean and a single global standard deviation from its posterior-mean representations on the target training split. We subtract the mean and divide by the standard deviation.

We then project each representation into 64 dimensions using a fixed random matrix. Its entries are drawn independently from a standard Gaussian distribution and divided by the square root of the encoder’s output dimension. An encoder-specific seed fixes this matrix. These data-independent projections give the head a fixed 64-dimensional input (Appendix § 16.§ 3). The normalization and projection remain fixed during head training and test-time sampling.

Let PmP_m denote this standardization and projection for encoder mm. For the selected encoder weights θS={θm:m∈S}\theta_S = \{\theta_m : m \in S\}, the aggregated representation is

h(x;θS)=∑m∈SαmPm(fm(Tm(x);θm)).(5)h(x; \theta_S) = \sum_{m \in S} \alpha_m P_m(f_m(T_m(x); \theta_m)). \tag*{(5)}

The reconstruction decoders are not used in this stage.

During head training, the encoders provide fixed representations computed with their posterior-mean weights. A Bayesian forecasting head, gg, learns from these aggregated representations and the target values in Dtrain\mathcal{D}_{\mathrm{train}}. Each training window produces one aggregated representation, paired with its own forecast target. The head’s weight parameters ω\omega have variational posterior qϕ(ω)q_{\phi}(\omega), where ϕ\phi collects the learned posterior means and scales. The encoder posteriors, projections, and routing weights remain fixed. The lower strip of Figure 1 shows this training stage. The final head has two outputs for each forecast coordinate: a predictive mean μ\mu and a positive standard deviation σ\sigma. This second output represents aleatoric uncertainty. We train both outputs with β\beta-NLL [37], using β=0.5\beta= 0.5. For the Bayesian head, the training objective also includes factor-space KL regularization, as in §C.1. Appendix § 15 gives the full objective and the motivation for replacing the initial MSE-trained head.

At test time, we sample the entire pipeline inside the dashed boundary in Figure 1. For each draw tt, we sample encoder weights θm(t)∼qm\theta_m^{(t)} \sim q_m, m∈Sm \in S, and head weights ω(t)∼qϕ\omega^{(t)} \sim q_{\phi}. We compute h(t)(x)h^{(t)}(x) using Equation (5) with these sampled encoder weights. The head returns (μ(t)(x),σ(t)(x))=g(h(t)(x);ω(t))(\mu^{(t)}(x), \sigma^{(t)}(x)) = g(h^{(t)}(x); \omega^{(t)}). We repeat this procedure for NMCN_{\mathrm{MC}} Monte Carlo draws. The selected experts and their aggregation weights do not change between draws. The point forecast is the mean of the sampled predictions, μ^(x)=NMC−1∑t=1NMCμ(t)(x)\hat{\mu}(x) = N_{\mathrm{MC}}^{-1} \sum_{t=1}^{N_{\mathrm{MC}}} \mu^{(t)}(x).

Variation in μ(t)(x)\mu^{(t)}(x) estimates the epistemic uncertainty propagated through the encoders and the head:

v^epi(x)=1NMC∑t=1NMC(μ(t)(x)−μ^(x))2.(6)\hat{v}_{\mathrm{epi}}(x)=\frac{1}{N_{\mathrm{MC}}}\sum_{t=1}^{N_{\mathrm{MC}}}\left(\mu^{(t)}(x)-\hat{\mu}(x)\right)^2 . \tag*{(6)}

The aleatoric variance comes from the head’s standard-deviation output, v^alea(x)=NMC−1∑t=1NMC(σ(t)(x))2\hat{v}_{\mathrm{alea}}(x)=N_{\mathrm{MC}}^{-1}\sum_{t=1}^{N_{\mathrm{MC}}}\left(\sigma^{(t)}(x)\right)^2. These calculations are performed separately for each forecast coordinate. The total predictive variance is

v^pred(x)=v^epi(x)+v^alea(x).(7)\hat{v}_{\mathrm{pred}}(x)=\hat{v}_{\mathrm{epi}}(x)+\hat{v}_{\mathrm{alea}}(x). \tag*{(7)}

Algorithm 1 in Appendix § 16 gives the complete prediction procedure after routing and head training.

Experimental Setup

We evaluate encoder storage, source reconstruction, distribution-shift response and downstream forecasting. Routing-policy comparisons use SBNN encoders throughout.

Source domains, encoder collections, and incoming datasets

We pretrain encoders on four source datasets: Acoustic, HAR (human activity recognition), Torque45 and ETT (electricity transformer temperature). We chose these datasets to cover different measured quantities, operating regimes, and temporal dynamics. The SBNN collection contains 17 encoders (Table 1). Several experts share a source dataset but differ in architecture or training configuration.

SourceSignalEncoders
AcousticSound recordings7
HARHuman motion3
Torque45Torque3
ETTOil temperature4
Total17

Table 1. SBNN encoder inventory by source.

Incoming datasets were not used during source pretraining. The seven incoming datasets are Torque10, Torque90, WTH (weather), ETTh2, ECL (electricity consumption), vibration, and EMG (electromyographic signals). Torque10 and Torque90 represent operating conditions different from Torque45; ETTh2 belongs to the ETT dataset family. Acoustic, Torque45, Torque10, Torque90 and vibration were recorded by an industrial partner; the remaining datasets are public. Appendix § 7 gives dataset descriptions, origins and references. Storage comparisons use the same source and architecture composition across families, following § 3.3.

Baselines and evaluation protocol

Model families. We compare SBNN with deterministic models, MC Dropout, full-rank Bayes by Backprop (BBB), and five-member deep ensembles. Full-rank refers to the weight parameterization; BBB uses a mean-field Gaussian posterior over these weights. For each arrival, SBNN, MC Dropout, BBB and deep ensembles select their own KK experts and compute their own aggregation weights. The deterministic pipeline reuses SBNN selections and weights. Each family supplies its corresponding encoders and forecasting head, with the same head architecture across families and the family’s weight parameterization. The comparison covers reconstruction, forecasting accuracy, predictive uncertainty, and storage.

Source-distance score. We represent each dataset by a statistical vector ϕ(X)\phi(\mathcal{X}) containing its mean, standard deviation, 10th, 50th and 90th percentiles, and autocorrelations at lags 1, 5 and 10.

For source ss, let Xs\mathcal{X}_s denote its reference windows. Let cjc_j be the standard deviation of descriptor coordinate jj across the four source datasets. We compare the source and incoming descriptors using

Ds(X)=[18∑j=18(ϕj(X)−ϕj(Xs)cj)2]1/2.(8)D_s(\mathcal{X}) = \left[\frac{1}{8}\sum_{j=1}^{8}\left(\frac{\phi_j(\mathcal{X})-\phi_j(\mathcal{X}_s)}{c_j}\right)^2\right]^{1/2}. \tag*{(8)}

The coordinate scales remain fixed across comparisons. Lower distances indicate more similar dataset statistics, and a dataset’s distance to itself is zero. Appendix § 14 details the descriptor construction and scaling.

Routing policies. We compare uncertainty-based routing with a source-distance baseline using SBNN encoders in both pipelines. Both select KK experts from the same eligible pool, ranked by source-normalized epistemic ratios (§ 2.2) or statistical-feature distance, with normalized inverse-score weights. The distance baseline does not reproduce the full two-stage router of Pham et al. [33]. Appendix § 12 gives the source-distance calculation and tie-handling rule. The comparison changes selection and weighting together and therefore evaluates each routing policy as a whole.

We also compare source-diverse selection, random selection, and higher-uncertainty controls to examine whether a policy that supports reconstruction also supports forecasting. Random selection from the full candidate pool and from a low-uncertainty shortlist are separate controls. Table 3 gives the selection and weighting rules. Values of KK, thresholds, and candidate pools are specified with the corresponding results.

PolicySelection ruleWeights
Top-KK uncertaintyKK lowest source-normalized uncertainty ratiosInverse-ratio
Top-KK distanceKK lowest source-distance scoresInverse-distance
Source-diverse top-KKLowest-ratio encoder per source, then KK lowest ratiosInverse-ratio
Random-KK, full poolSample KK encoders without replacementInverse-ratio
Random-KK, shortlistSample KK encoders without replacementUniform
Bottom-KK, shortlistKK highest uncertainty ratios within the shortlistUniform
Bottom-KK uncertaintyKK highest uncertainty ratios in the eligible poolUniform

Table 3. Selection policies and weights. Each policy selects KK encoders. Top and bottom denote the lowest and highest scores. The shortlist contains ratios rm≤τrouter_m \le\tau_{\mathrm{route}}. Weights are normalized over selected encoders.

Head training and prediction. For the distance-versus-uncertainty comparison, both pipelines use the same input windows, target splits, head architecture, training settings, and seeds. Neither router uses forecast targets. All final forecasting heads use β\beta-NLL with β=0.5\beta= 0.5; Bayesian heads also include KL regularization. Forecasting is evaluated over three seeds: 40, 41, and 42. SBNN prediction uses NMC=100N_{\mathrm{MC}} = 100 joint encoder–head draws, following § 2.3.

Metrics and parameter accounting

We evaluate reconstruction, forecasting accuracy and predictive uncertainty separately. For reconstruction on incoming data, the policy score is the weighted average of the selected encoders’ individual reconstruction errors, RS=∑m∈SαmRmR_S = \sum_{m\in S}\alpha_m R_m. We compute each encoder’s error before taking the weighted average. Reconstruction comparisons are made within each incoming dataset.

Forecasting metrics use the standardized target scale. We report MSE for point accuracy, expected calibration error (ECE), predictive negative log-likelihood (NLL), prediction interval coverage probability at 90% (PICP@90), and mean prediction interval width at 90% (MPIW@90). Lower MSE, ECE and NLL are preferred. Coverage should approach the nominal 90%; interval width is interpreted alongside coverage. Forecasting intervals and NLL use a Gaussian approximation with the total predictive variance from § 2.3. The reported NLL is distinct from the β\beta-NLL training loss. For distribution shift, we report Spearman correlations between dataset distance and the uncertainty ratio. Exact definitions and averaging conventions are given in Appendix § 10.

Parameter counts include frozen encoder weights, Bayesian posterior means and scales, and all five ensemble members. Complete forecasting-pipeline counts also include the head and exclude reconstruction decoders and training state. At equal numerical precision, parameter-count ratios equal parameter-storage ratios. Appendix § 11 gives the detailed accounting.

Forecasting results report means and sample standard deviations over three seeds. Overall means give equal weight to the seven incoming datasets; routing comparisons pair runs by dataset and seed.

Results

We evaluate encoder storage and the use of uncertainty for expert selection. We first compare storage, reconstruction accuracy and uncertainty calibration on the source datasets. We then examine how encoder uncertainty responds to distribution shift. The downstream experiments first compare forecasting quality and parameter cost across model families, then compare uncertainty-based routing with source-distance routing. Finally, we compare selection and weighting policies under both reconstruction and forecasting to examine whether their rankings carry over between tasks.

Storage and source-task performance of compact encoders

Our starting point was an existing collection of deterministic encoders. We used SBNN to build compact Bayesian counterparts that reduce storage and provide uncertainty for expert selection. The comparison includes the corresponding deterministic, MC Dropout, full-rank BBB and five-member ensemble models on Acoustic, HAR, Torque45 and ETT. Reconstruction uses each encoder’s decoder. The decoder is subsequently discarded from the forecasting pipeline.

Encoder storage. Table 2 counts the stored encoder parameters. Bayesian counts include posterior means and scales, and ensemble counts include all five members. Decoders, forecasting heads and optimizer state are excluded.

SourceDeterministic / MC DropoutSBNNFull-rank BBBEnsemble (5 members)Reduction
Acoustic3,064,2561,048,0026,124,09815,321,28065.8%
HAR1,082,824571,8442,165,1885,414,12047.2%
Torque45555,28960,0881,110,5862,776,44589.2%
ETT116,36863,788232,736581,84045.2%
Total4,818,7371,743,7229,632,60824,093,68563.8%

Table 2. Stored encoder parameters. Counts sum the 17 matched encoders per family, grouped by source. Bayesian counts include posterior means and scales; ensemble counts include all five members. Decoders and heads are excluded. Reductions are relative to deterministic encoders. Bold marks the smallest count.

At the selected ranks, the 17-encoder SBNN collection contains 1,743,722 parameters, compared with 4,818,737 for the matched deterministic collection: a reduction of 63.8%. At equal numerical precision, parameter storage falls by the same proportion. MC Dropout stores the same number of parameters as the deterministic model. SBNN is smaller than every compared family on each source.

Figure 8 in Appendix § 11 compares all five families. A deterministic model with the same architecture and ranks would use fewer parameters than SBNN. SBNN also supplies the epistemic uncertainty used for expert selection, addressing both storage and routing. Relative to full-rank deterministic encoders, the reduction ranges from 45.2% to 89.2%. It depends on the architecture and the retained rank profile. The rank controls the tradeoff between compression, reconstruction accuracy and calibration. The ETT rank sweep in Appendix § 9 shows how different rank configurations affect these metrics.

Figure 8. Stored encoder parameters by source and family. Percentages give SBNN reductions relative to deterministic encoders. Panels use separate scales. Decoders and forecasting heads are excluded.

Reconstruction accuracy and calibration. On source reconstruction, the selected SBNN encoders achieve the lowest ECE and coverage closest to the nominal 90% level on three of the four source domains; HAR is the exception. This comes with a source-dependent increase in reconstruction error. SBNN therefore offers better uncertainty calibration at a lower storage cost. Low reconstruction error can coexist with overconfident uncertainty estimates, so we prioritize uncertainty quality alongside accuracy. Appendix § 13 gives the complete source comparison and reconstruction plots.

Epistemic response to distribution shift

Source-task calibration evaluates uncertainty on familiar data. Figure 3 compares statistical-feature distance (Equation (8)) with the source-normalized epistemic uncertainty of encoders trained on Torque45, Acoustic and ETT across eight domains. Uncertainty increases with distance overall, with local reversals in the ordering. Acoustic saturates faster. No decoder or forecasting head is involved. Appendix I gives the protocol, correlations, and per-domain values.

Figure 3. Encoder uncertainty under shift. Colours denote datasets; diamonds mark sources. Horizontal scale: log⁡(1+D)\log(1 + D); vertical scale: logarithmic. Dashed lines mark uncertainty ratio one.

Shift assessment and routing make different comparisons. Here, we vary the incoming dataset while holding the encoder fixed. Routing compares several encoders on the same incoming dataset. Each encoder has its own learned response to unfamiliar inputs.

End-to-end forecasting across model families

We now use the frozen encoders to forecast incoming time series. The comparison uses top-4 uncertainty routing (Equation (3)) and inverse-ratio aggregation weights (Equation (4)). Routing follows the family-specific protocol in § 3.2. All final heads use β\beta-NLL. Prediction follows the joint sampling procedure in § 2.3. Table 4 reports means and seed variability over the seven incoming datasets and seeds 40, 41 and 42. Appendix § 16 gives prediction details and forecast examples.

FamilyMSE ↓\downarrowECE ↓\downarrowNLL ↓\downarrowPICP90 (%) →\rightarrow 90%MPIW90Params (M) ↓\downarrow
SBNN0.3896±0.00310.3896 \pm0.00310.0687±0.0025\mathbf{0.0687 \pm0.0025}0.814±0.0210.814 \pm0.02190.47±0.0990.47 \pm0.091.771±0.0181.771 \pm0.0181.759\mathbf{1.759}
Deterministic0.4056±0.00080.4056 \pm0.00080.0816±0.00260.0816 \pm0.00261.353±0.4221.353 \pm0.42282.60±0.3682.60 \pm0.361.476±0.0071.476 \pm0.0074.8684.868
MC Dropout0.3894±0.0041\mathbf{0.3894 \pm0.0041}0.0767±0.00080.0767 \pm0.00080.817±0.0040.817 \pm0.00487.78±0.0787.78 \pm0.071.647±0.0071.647 \pm0.0074.8684.868
Full-rank BBB0.4426±0.00160.4426 \pm0.00160.0877±0.00180.0877 \pm0.00180.899±0.0020.899 \pm0.00292.29±0.0692.29 \pm0.062.100±0.0022.100 \pm0.0029.7319.731
Deep ensemble (5)0.3965±0.00280.3965 \pm0.00280.0734±0.00390.0734 \pm0.00390.881±0.0080.881 \pm0.00884.55±0.4484.55 \pm0.441.517±0.0071.517 \pm0.00724.34224.342

Table 4. Forecasting results and stored parameters. Mean ±\pm sample SD over three seeds, each averaged over seven datasets. Counts include the full encoder collection and forecasting head. Bold marks the best mean or smallest count.

Forecasting accuracy and storage. SBNN retains competitive forecasting accuracy after compressing the encoders and the head. With the forecasting head included, SBNN uses 63.9% fewer parameters than the deterministic and MC Dropout models, 81.9% fewer than BBB and 92.8% fewer than the ensemble. At equal numerical precision, the same reductions apply to parameter storage. Figure 4 compares forecasting MSE, ECE and parameter count. SBNN remains close to the best forecasting MSE while using the fewest parameters.

Figure 4. Forecasting accuracy, calibration and cost. Seven-dataset means over three seeds: MSE versus stored encoder–head parameters (left, log scale) and ECE (right). Circle area is proportional to parameter count.

Predictive calibration and coverage. The intervals and calibration metrics use the total predictive variance in Equation (7). The learned σ\sigma contributes to interval coverage in every family, including the deterministic pipeline, whose weights remain fixed. SBNN has the lowest mean ECE and NLL, and coverage closest to the nominal 90% level, although its NLL difference from MC Dropout is small. Coverage must be considered together with interval width: SBNN’s improved coverage comes with wider intervals than the under-covering MC Dropout and ensemble models, and narrower intervals than BBB.

Figure 5 summarizes forecasting quality and parameter cost, including seed variability.

Figure 5. Forecasting quality and compactness. Top-4 uncertainty routing; seven datasets, three seeds. Left: raw metrics and encoder–head counts; error bars show sample SD of dataset-averaged seed results. Dashed line: 90% coverage. Right: higher is better. MSE, ECE and parameter scores use the lowest family value divided by each family’s value; coverage uses 1−∣PICP90−0.90∣/0.901 - \lvert\mathrm{PICP90} - 0.90\rvert/0.90.

Appendix § 16.§ 2 compares observed coverage with nominal levels of 70% and 90%.

Forecast examples and nominal 90% prediction bands for all seven incoming datasets are shown in Figure 12 in Appendix § 16.

Figure 12. Forecasts and uncertainty bands across all seven incoming datasets. Columns show incoming datasets and rows show model families. Each column uses the same saved forecast window across all five families. Dashed curves show the true signal, solid curves the forecast, and shaded regions nominal 90% prediction bands. Panels retain their original vertical scales.

Remark. The MSE differences between SBNN, MC Dropout and deep ensembles are small. SBNN has slightly lower mean ECE and coverage closer to the nominal 90% level. These gains are modest. The main advantage is that SBNN maintains competitive forecasting accuracy and uncertainty quantification with substantially fewer parameters and lower storage cost.

Appendix § 15.§ 1 reports the initial MSE-trained comparison using epistemic uncertainty alone.

Downstream comparison of uncertainty and distance routing

We now compare uncertainty-based and distance-based expert selection through downstream forecasting. Both policies select three SBNN encoders from the same eligible pool, using incoming inputs without forecast targets. The source-distance baseline selects the encoders with the lowest distances to their pretraining domains and assigns normalized inverse-distance weights. Uncertainty routing selects the encoders with the lowest source-normalized epistemic ratios and assigns normalized inverse-ratio weights. The comparison therefore evaluates each routing policy as a whole, including selection and weighting.

Both routing policies are training-free and label-free. The distance baseline can cache its source statistical descriptors; uncertainty routing stores a scalar source reference per encoder.

We compare top-3 distance with top-3 uncertainty routing because adding a fourth distance-selected encoder reduced forecasting accuracy and worsened calibration. Uncertainty routing with three or four encoders gave similar results on most incoming datasets. Using three experts in both policies therefore keeps the comparison matched in the number of selected encoders.

Both policies use the matched head-training and prediction protocol described in § 3.2. We evaluate seven incoming datasets with seeds 40, 41 and 42, giving 21 paired comparisons.

Table 16 in Appendix § 17 reports the aggregate results. Means give equal weight to the seven datasets and three seeds; the table also reports variability across seeds. Uncertainty routing achieves a mean MSE of 0.3976, compared with 0.4062 for distance routing, a reduction of 2.1%. The MSE changes vary across datasets, with both improvements and increases. The main advantage concerns predictive uncertainty.

MetricTop-3 distanceTop-3 uncertainty
MSE ↓\downarrow0.4062 ± 0.00590.3976 ± 0.0070
ECE ↓\downarrow0.0857 ± 0.00130.0721 ± 0.0038
NLL ↓\downarrow0.907 ± 0.0390.836 ± 0.041
PICP90 (%) → 90%82.28 ± 0.5889.03 ± 0.35
MPIW901.795 ± 0.0071.715 ± 0.032

Table 16. Forecasting with top-3 distance and uncertainty routing. Mean ± sample SD over three seeds, each averaged over seven datasets. Bold marks the better mean.

Mean ECE falls from 0.086 to 0.072 and mean NLL from 0.907 to 0.836. Both metrics improve on all seven datasets after averaging over seeds. Across individual runs, ECE improves in 19 of the 21 comparisons and NLL in 20. Figure 6 shows the per-dataset ECE and coverage results.

Figure 6. Calibration under distance and uncertainty routing. Bars average three seeds per dataset; the dashed line marks nominal 90% coverage.

Mean PICP90 increases from 82.28% to 89.03%, a gain of 6.75 percentage points. Coverage is closer to the nominal 90% level in every paired dataset–seed comparison. The figure also shows the remaining undercoverage and overcoverage. Mean interval width decreases from 1.795 to 1.715: the coverage improvement is accompanied by slightly narrower intervals on average.

With the same number of selected experts, uncertainty routing retains competitive forecasting accuracy and improves predictive calibration and likelihood. Per-dataset summaries, together with the MSE, NLL and interval-width plots, are provided in Appendix § 17.

Does uncertainty ranking predict downstream utility?

Uncertainty routing selects experts before the forecasting head is trained. We compare the nine policies from § 3.2 on reconstruction and forecasting, using SBNN encoders throughout.

In § 4.1, each encoder was evaluated on its own source domain. Here, the frozen encoders and their decoders reconstruct the seven incoming datasets. The MSE, ECE and coverage below describe reconstruction on the seven incoming datasets.

Reconstruction favors the most confident experts. Table 5 reports the average reconstruction metrics. Each policy combines individual encoder–decoder errors as defined in Equation (13).

PolicyMSE ↓ECE ↓PICP90 (%) → 90%
Top-3 uncertainty0.01140.277763.10
Top-4 uncertainty0.12790.312356.04
Source-diverse top-30.36470.386335.24
Source-diverse top-40.35790.364139.83
Random-4 of top-80.39260.375437.94
Bottom-4 of top-80.39190.385041.09
Random-4, full pool0.54840.406528.91
Bottom-3 uncertainty0.44730.348539.28
Bottom-4 uncertainty0.52010.376333.80

Table 5. Average reconstruction metrics on seven incoming datasets.

Top-3 and top-4 have the lowest mean reconstruction MSE and ECE, and coverage closest to 90%. Top-4 achieves an MSE of 0.1279, compared with 0.3926 for random-4 and 0.3919 for bottom-4 from the shortlist. Its ECE is also lower, and its coverage is closer to nominal. The confidence ranking therefore works well for reconstruction. Mean coverage nevertheless remains below 90% on these incoming datasets.

Forecasting separates the shortlist from the controls. Table 6 reports the corresponding forecasting results. The low-uncertainty shortlist contains eight encoders in this experiment. Top-3, top-4, random-4 of top-8 and bottom-4 of top-8 all select within this shortlist. Random-4 from the full pool and the bottom-3 and bottom-4 uncertainty policies provide the controls.

PolicyMSE ↓ECE ↓PICP90 (%) → 90%NLL ↓MPIW90
Top-3 uncertainty0.39760.072189.030.83571.715
Top-4 uncertainty0.38960.068790.470.81351.771
Source-diverse top-30.37970.068291.020.79091.789
Source-diverse top-40.37600.068291.020.78931.779
Random-4 of top-80.37870.067690.610.80111.767
Bottom-4 of top-80.37910.070290.980.78661.808
Random-4, full pool0.50850.088380.860.90022.069
Bottom-3 uncertainty0.40360.084887.520.83791.871
Bottom-4 uncertainty0.40380.084987.170.84581.861

Table 6. Average forecasting metrics on seven incoming datasets.

The shortlist policies have lower mean forecasting MSE, ECE and NLL than all three controls. The clearest separation is in calibration and coverage: their ECE ranges from 0.0676 to 0.0721, with PICP90 between 89.03% and 90.98%. The controls have ECE between 0.0848 and 0.0883, and PICP90 between 80.86% and 87.52%. Random-4 from the full pool also produces wider intervals despite its lower coverage. The MSE gap is larger for this random policy and smaller for the bottom-ranked controls.

The reconstruction ranking does not transfer. Within the shortlist, random-4 and bottom-4 have lower mean forecasting MSE than top-3 and top-4, despite their higher reconstruction errors. Source-diverse top-4 gives the lowest forecasting MSE overall, although its reconstruction MSE is higher than top-4’s.

The differences within the shortlist are modest, and no policy leads every forecasting metric. Random-4 has the lowest ECE, bottom-4 has the lowest NLL, and top-4 has coverage closest to 90%. Forecasting therefore shows a useful separation between candidate groups, without preserving the reconstruction ordering within the low-uncertainty group.

Figure 15 in Appendix § 18 compares the two tasks using the same policy order.

Figure 15. Reconstruction and forecasting under the selection policies. The upper row reports reconstruction on incoming datasets; the lower row reports downstream forecasting. Columns show MSE, ECE and PICP90. Lower MSE and ECE are preferred; dashed lines mark nominal 90% coverage. Policies retain the same colours and ordering.

Pretext-side encoder uncertainty helps exclude weak candidates. Within the shortlist, reconstruction and forecasting favor different combinations, and their calibration rankings also differ. Reconstruction averages individual encoder errors; forecasting evaluates combined representations through a trained head. We compare complete policies, including their aggregation weights. Detailed results are given in Appendix § 18. Appendix § 18.§ 2 also compares fixed-KK selection with a fixed uncertainty-ratio cutoff.

Discussion and Conclusion

We presented a modular Bayesian framework for time series foundation models that addresses the two main issues in expanding encoder collections: storage cost and expert selection for incoming datasets. Building on SBNN, we trained compact Bayesian encoders whose epistemic uncertainty supports distribution-shift assessment and label-free routing. The resulting pipeline maintains competitive forecasting accuracy and uncertainty calibration while requiring substantially fewer stored parameters than the deterministic, MC Dropout, full-rank Bayes by Backprop, and five-member deep-ensemble pipelines. At the selected ranks, our Bayesian encoders require 63.8% less parameter storage than their deterministic counterparts.

The source-task results show that compactness can be achieved alongside improved uncertainty calibration, with an accuracy–calibration tradeoff that depends on the source and selected rank and KL-scale configurations. On unfamiliar inputs, encoder uncertainty increases with measured source distance, supporting its use for shift assessment. Uncertainty routing remains competitive with source-distance routing in forecasting MSE while improving mean ECE, NLL and agreement with nominal coverage. These decisions use incoming inputs without target labels, router training or encoder retraining.

The policy comparison also identifies a limit of this selection. The most confident experts reconstruct incoming data better. Within the low-uncertainty shortlist, random, source-diverse or bottom-ranked combinations can outperform the top-ranked experts in forecasting. Policies using the full pool or its least confident experts generally give poorer forecasting results and uncertainty calibration. Encoder uncertainty therefore helps narrow the choice to a useful subset without determining the best combination within that subset for the downstream task. In our experiments, better pretext reconstruction does not guarantee better forecasting.

Several limitations remain. Compression depends on the retained rank profiles, while rank and KL scale require tuning to balance reconstruction and calibration. The uncertainty response can also saturate under large shifts, reducing its ability to distinguish very distant domains. We use a fixed projection dimension of 64; its effect on forecasting accuracy and calibration remains to be studied. Final forecasting intervals combine epistemic uncertainty with the head’s learned aleatoric contribution. Including this contribution improves coverage across model families and narrows the larger calibration differences observed in the initial epistemic-only comparison. The final gains in forecasting calibration are therefore modest; the strongest advantage is maintaining predictive quality at substantially lower storage cost. Our evaluation remains limited to the studied source domains, incoming datasets and tasks.

The separation between pretext and downstream performance connects this empirical finding to a broader theoretical question. Lee et al. [24] establish conditions under which reconstruction-based learning produces useful downstream representations, including approximate conditional independence and an informative reconstruction target. Wasserer et al. [46] relate joint-embedding objectives to the risk of an optimal downstream linear classifier.

Extending such results to compact Bayesian representations would require accounting for the rank constraint and posterior learning, as well as the relationship between the pretext and downstream tasks.

Future work could build on these analyses by designing pretext objectives around the information required downstream. The goal is to establish conditions under which learning a pretext task bounds downstream risk. Masked reconstruction and forecasting provide one setting for this investigation; other pretext and downstream tasks raise the same question. Such guarantees would give a theoretical basis for deciding when a compact Bayesian representation can be reused successfully, beyond the empirical shortlisting criterion established here.

Related Work

Modular models and expert selection

Modular foundation models reuse pretrained components across datasets and tasks ([32, 33]). This allows new specialists to be added while retaining the components already trained. Reuse then requires deciding how to combine knowledge acquired on different sources.

Syed et al. [40] examine why choosing models by aggregate benchmark scores can be unreliable. They evaluate seven time-series foundation models on 72 datasets spanning six application domains and find that model rankings vary across domains and datasets. They call for domain-stratified evaluation and methods for assessing which source domains help or harm transfer to a target domain. This provides empirical motivation for retaining specialised encoders and selecting them according to incoming data. Their study evaluates zero-shot point forecasting; here, we study how an encoder collection can remain compact while providing an uncertainty signal for selection.

The reuse of pre-trained modules has been studied in language models. AdapterFusion ([31]) decouples the training of task-specific adapters from a subsequent composition step, in which an attention mechanism learns to combine their representations.

During composition, both the adapters and their shared language-model backbone remain frozen. Our encoder collection follows this separation between source training and downstream reuse, with whole encoders as the reusable components. Ostapenko et al. [30] build libraries of low-rank adaptation (LoRA) adapters and propose Arrow, a mechanism that selects relevant adapters for new inputs without requiring additional training. Thus, routing within a library of modules does not always require supervision on the target task. Modularity can also be applied to heterogeneous architectures; Manticore combines blocks from pre-trained models using projections and learned mixing weights ([35]). These methods differ in the components they reuse and the training needed to combine them.

Guard learns to combine forecasts from pretrained time-series models for distillation into a compact student ([12]). Teacher forecasting errors provide supervision for its contextual router, while an uncertainty-dependent temperature controls distillation strength. We retain the encoders as reusable experts and compute their selection scores from unlabeled incoming windows.

Selection also depends on what is known about the target task. Log Expected Empirical Prediction (LEEP) uses labeled target examples to score the transferability of a pretrained classifier ([29]). Our router uses unlabeled inputs. For time series, Pham et al. [33] provide the closest comparison to our encoder collection. Their self-supervised encoders remain frozen during downstream adaptation. Selection first uses statistical and spectral descriptors to measure source–target dataset distance, then ranks candidates by how well their representations preserve input similarities. Selected representations are aligned and combined by a Transformer aggregation module. We share this separation between pretraining, selection, and task adaptation. Our selection score comes from the Bayesian encoders’ own uncertainty on incoming data. Our downstream comparison between uncertainty based routing and distance based routing uses a nearest-source distance baseline inspired by this first criterion. A different approach is to train expert selection together with the experts. The learned gate in Jacobs et al. [20] assigns inputs to experts during training. Time-MoE and Moirai-MoE use sparse expert activation within time-series models ([38, 26]). Our unit of selection is a whole pretrained encoder. We select a fixed set for each incoming dataset, and new encoders can enter the collection without retraining existing experts or a gate.

Storage cost of Bayesian encoders

The storage cost of an encoder collection grows as specialists are added, even when the models are deterministic. The additional storage required by conventional Bayesian neural networks has long been an obstacle to scalable uncertainty quantification ([14]). Our goal is to reduce the storage cost of the collection while providing epistemic uncertainty for distribution-shift assessment and expert selection.

Under full-rank mean-field Gaussian inference, each Bayesian weight requires a mean and a scale. Bayes by Backprop learns both [5]. This doubles the stored parameters for those weights relative to a deterministic parameterization. Deep ensembles estimate uncertainty from several independently trained networks [23]. Our five-member baseline therefore stores five complete copies of the network. MC Dropout uses stochastic forward passes through one network and retains its deterministic weight matrices [15].

Low-rank methods have been developed to address the scalability of Bayesian deep learning [41, 14, 45]. Stochastic weight averaging-Gaussian (SWAG) and structured Gaussian variational inference use compact covariance representations while retaining full-dimensional weight means [27, 41]. Rank-1 BNNs learn a shared dense weight matrix and distributions over multiplicative vectors [14]. The rank-one modulation reduces Bayesian overhead while retaining the shared dense weight matrix. Bayesian Low-Rank Adaptation by Backpropagation (BLoB) learns a Bayesian low-rank adapter to a frozen backbone [45]. These approaches reduce posterior or adaptation overhead while retaining full-dimensional weight means, a shared dense matrix, or a frozen backbone. SBNNs parameterize the layer weights themselves with Bayesian low-rank factors [42]. We use this parameterization to build Bayesian encoders that cost less to store than their deterministic counterparts and provide epistemic uncertainty for expert selection. Appendix F compares the stored components of these methods.

Bayesian-MoE applies a post-hoc Laplace approximation to the second linear layer of each expert in an MoE language model [13]. Its low-rank Kronecker-factored approximation concerns posterior curvature around retained dense weights. The learned gate is kept, and the posterior supplies predictive uncertainty estimates within expert pathways.

The original SBNN study reports comparisons with both Rank-1 BNNs and SWAG [42]. Those experiments complement the modular forecasting evaluation presented here.

Uncertainty for routing and distribution-shift detection

Our routing signal comes from the pretrained encoders before downstream training.

Mixture-of-Gaussians with Uncertainty-based Gating (MoGU) predicts a Gaussian mean and variance for each expert and computes mixture weights in inverse proportion to those variances [4]. It requires no separate gating network; the variances are learned with the forecasting objective. Zhou et al. (2026) combine label-distribution learning with multiple forecasting experts and an MLP gate. Their continuous variant learns Gaussian-mixture forecasts by matching predicted and enhanced target distributions with maximum mean discrepancy. Evaluated on aggregated M5 sales data, this approach learns the gate and predictive distributions for the forecasting task.

For sparse language models, Nahshan et al. (2026) supervise routing with realized next-token cross-entropy. One formulation learns expert-indexed error scores that modify affinities before top-KK selection; another directly supervises affinity concentration. The observed loss provides supervision during training, while inference uses the learned routing mechanism without requiring the true next token. Our router requires no such target-task supervision or router fitting.

Uncertainty-Aware Routing (UAR) uses router entropy and free energy derived from router logits to adapt expert activation and load-balancing regularization [9]. These signals describe the learned gate. Our score measures variation in each encoder’s representations under posterior weight draws.

Variational Routing learns probabilistic expert assignments within pretrained MoE language models [25]. Its inference networks model routing-logit distributions or learn input-dependent temperatures for stochastic selection. Cajas Ordóñez et al. (2025) use MC Dropout in a learned router and increase the weights of higher-precision quantized experts when uncertainty rises. Both methods train routing components for their tasks. Our routing statistic is computed directly from posterior representation draws of frozen encoders and their stored source references.

Grassmannian MoE represents experts through learned subspaces and computes routing weights from subspace affinities and Bingham concentration parameters [39]. An amortization network allows token-dependent assignment distributions. These low-dimensional subspaces parameterize the router, with the expert weight matrices retained separately. GrMoE models uncertainty in expert assignment. Our score measures representation variation under the encoder weight posterior.

VI-MoLE learns the counterfactual residual risk of nested LoRA-expert prefixes, calibrates risk estimates, and allocates a shared compute budget using predicted risk reduction per unit cost [36]. It also supports abstention. A committee-only target can omit labels; task-risk supervision and answer-risk calibration use labeled data. Its acquisition policy is learned and calibrated, while our incoming-task routing scores come directly from the frozen encoders and their stored source references.

In image restoration, Zhang et al. [49] use MC Dropout within a learned router to estimate and normalize its routing logits. We select and weight pretrained encoders using their epistemic uncertainty before fitting the forecasting head.

Using an encoder’s uncertainty to assess unfamiliar data also connects routing to distribution-shift detection. Arnez et al. [3] compare entropy statistics of latent representations sampled through dropout or DropBlock with a reference fitted on source data. We obtain uncertainty through the encoder’s variational weight posterior and use the source-normalized score to select and weight experts as well as assess shift. The response to shift depends on the model: D’Angelo & Henning [10] show that even exact Bayesian inference can give poor OOD detection for common neural-network priors. We therefore evaluate the score against measured distribution shift and assess its downstream usefulness separately.

Downstream utility and calibration

Ensemble selection makes the dependence on the target task explicit by choosing combinations according to held-out performance [7]. Individual accuracy also does not determine the value of a combination. The analysis of Wood et al. [48] relates ensemble error to bias, variance, and diversity, while Gontijo-Lopes et al. [18] show that models with complementary features can improve downstream performance even when one is less accurate on its own. These results motivate examining the selected group alongside its individual encoders. In our setting, we compare reconstruction and forecasting under the same policies to establish when a selection that works for the pretext task also works downstream.

VI-MoLE separates current predictive uncertainty from the risk reduction expected from acquiring another expert [36]. In our experiments, uncertainty identifies a useful shortlist, and forecasting rankings vary within it. VI-MoLE analyzes task-specific expert acquisition. Our future work asks when pretext learning gives guarantees on downstream performance.

Forecast error and predictive calibration answer different questions [22]. Adler et al. [1] assess calibration in time-series foundation models, including its dependence on prediction heads and forecast length. We report forecasting MSE for point accuracy and calibration error, NLL, and interval coverage for the predictive distribution.

Dataset and evaluation protocol

Incoming datasets are new to the encoder collection; they were not used during source pretraining. Dataset descriptions and origins are given in Section B.1; routing settings are given in Appendix § 12.

Dataset descriptions

We use four source datasets to pretrain the encoders and seven incoming datasets to evaluate their reuse. Table 7 identifies the signals and the role of each dataset. Acoustic, the torque datasets and vibration contain recordings collected by an industrial partner.

DatasetMeasured signalOrigin
Source datasets
AcousticGenerator acoustic signalIndustrial partner
HARHuman-activity signalsPublic [34]
Torque45Shaft torque signal at 45% of maximum unit powerIndustrial partner
ETT (ETTh1)Transformer oil temperaturePublic [50]
Incoming datasets
Torque10Shaft torque signal at 10% of maximum unit powerIndustrial partner
Torque90Shaft torque signal at 90% of maximum unit powerIndustrial partner
WTHWeather measurements: wet-bulb temperaturePublic [50]
ETTh2Transformer oil temperaturePublic [50]
ECLElectricity consumptionPublic [43]
VibrationGenerator vibration signalIndustrial partner
EMGElectromyographic signalsPublic (PhysioNet) [17]

Table 7. Source and incoming datasets. Source datasets support encoder pretraining. Incoming datasets support routing, reconstruction evaluation and downstream forecasting.

The ETT source encoder is trained on oil-temperature windows from ETTh1. Incoming ETTh2 contains measurements from a different transformer. Both belong to the hourly Electricity Transformer Dataset [50].

Each experiment uses a single signal channel. For WTH, ECL and ETTh2, we retain the wet-bulb temperature, MT_320 electricity-consumption and oil-temperature channels, respectively. The stored EMG windows are concatenated in their saved order to construct the series used for forecasting.

For all seven incoming datasets, the forecasting head predicts the next 96 time steps from an input window of 96 past time steps. For encoder reuse, each incoming window is linearly interpolated on a normalized time grid to the encoder’s native input length.

Source pretraining objectives and settings

Encoder construction and training

Pretext objective. The general objective combines reconstruction and contrastive learning. Masked reconstruction trains the encoder and decoder to recover a clean signal from an incomplete input, following the denoising autoencoder principle [44]. For an input window xx, a binary mask mm retains some values and hides others. The encoder receives x~=m⊙x\tilde{x} = m \odot x, and the decoder gdecg_{\mathrm{dec}} returns x^=gdec(f(x~))\hat{x} = g_{\mathrm{dec}}(f(\tilde{x})). We score reconstruction against the clean signal over the whole window:

Lrec=1dx∑j=1dx(xj−x^j)2,(9)\mathcal{L}_{\mathrm{rec}} = \frac{1}{d_x} \sum_{j=1}^{d_x} (x_j - \hat{x}_j)^2, \tag*{(9)}

where dxd_x is the number of scalar values in the window. The decoder and its parameters are omitted from the weight notation below.

Contrastive learning encourages similar representations for two views of the same window and separates them from representations of other windows in the batch. Views are constructed through input augmentations, without class labels. We use cosine similarity between normalized representations and a temperature parameter, following Chen et al. [8]. With Lcon\mathcal{L}_{\mathrm{con}} denoting the contrastive loss, the combined loss is

LSSL=(1−α)Lrec+αλ~(t)Lcon,(10)\mathcal{L}_{\mathrm{SSL}} = (1-\alpha)\mathcal{L}_{\mathrm{rec}} + \alpha\tilde{\lambda}(t)\mathcal{L}_{\mathrm{con}}, \tag*{(10)}

where α\alpha controls the mixture and λ~(t)\widetilde{\lambda}(t) balances the magnitudes of the two terms at training step tt. This scale is detached from gradients. The contrastive variants, masking settings and loss coefficients for each source are specified below.

Bayesian low-rank weights. Let ℓ\ell index the layers. The weight matrix WℓW_{\ell} has aℓa_{\ell} rows and bℓb_{\ell} columns, and rℓr_{\ell} is our chosen rank. SBNN writes

Wℓ=AℓBℓT,Aℓ∈Raℓ×rℓ,Bℓ∈Rbℓ×rℓ.(11)W_{\ell} = A_{\ell}B_{\ell}^{\mathsf{T}}, \qquad A_{\ell} \in\mathbb{R}^{a_{\ell}\times r_{\ell}}, \qquad B_{\ell} \in\mathbb{R}^{b_{\ell}\times r_{\ell}} . \tag*{(11)}

Both factors are stochastic. We learn a Gaussian mean and scale for each factor entry. For the factors, q(Θ)=∏ℓq(Aℓ)q(Bℓ)q(\Theta) = \prod_{\ell} q(A_{\ell})q(B_{\ell}), with Θ={Aℓ,Bℓ}ℓ\Theta= \{A_{\ell}, B_{\ell}\}_{\ell}. Every sampled matrix has rank at most rℓr_{\ell}. Its entries share factors, so independent factor entries induce dependencies between weights. Convolutional kernels are reshaped into matrices for the same factorization.

A deterministic matrix requires aℓbℓa_{\ell}b_{\ell} parameters, and full-rank mean-field Bayesian weights require 2aℓbℓ2a_{\ell}b_{\ell}. SBNN stores 2rℓ(aℓ+bℓ)2r_{\ell}(a_{\ell}+b_{\ell}) factor means and scales. It is smaller than the deterministic matrix when 2rℓ(aℓ+bℓ)<aℓbℓ2r_{\ell}(a_{\ell}+b_{\ell}) < a_{\ell}b_{\ell}, and smaller than the full-rank Bayesian parameterization when rℓ(aℓ+bℓ)<aℓbℓr_{\ell}(a_{\ell}+b_{\ell}) < a_{\ell}b_{\ell}. Biases and other parameters are counted separately.

Training and configuration choices. Following Toure & Stephens [42], we initialize the factor means from a truncated singular value decomposition of the corresponding deterministic model’s weights, then learn the factor means and scales on the source task. The Bayesian training objective is

Jsource(q)=Eq[LSSL]+λKL(t)K(q,p),(12)\mathcal{J}_{\mathrm{source}}(q) = \mathbb{E}_{q}[\mathcal{L}_{\mathrm{SSL}}] + \lambda_{\mathrm{KL}}(t)\mathcal{K}(q,p), \tag*{(12)}

where K(q,p)\mathcal{K}(q,p) sums the KL terms between the factor posteriors q(Aℓ)q(A_{\ell}) and q(Bℓ)q(B_{\ell}) and their respective priors p(Aℓ)p(A_{\ell}) and p(Bℓ)p(B_{\ell}). Other Bayesian parameters, where present, contribute their own KL terms. The coefficient λKL(t)\lambda_{\mathrm{KL}}(t) sets their strength at training step tt. We estimate the expected loss with weight samples and differentiate through Gaussian reparameterization (Kingma & Welling [21]).

The rank profile determines storage cost. Lower ranks use fewer parameters but restrict the weight maps. We treat rank as a hyperparameter. Through rank ablations, we select the smallest tested rank that remains competitive with the deterministic counterpart on the validation metric of interest, within the storage budget. The reported compression therefore depends on the ranks selected for each encoder. The KL scale changes posterior regularization without changing that count. A storage budget can therefore constrain the admissible ranks, while rank and KL settings are chosen for the chosen validation metric. In our source comparisons, SBNN (reconstruction) prioritizes low validation MSE. SBNN (calibration) prioritizes low ECE and coverage close to its nominal level. SBNN (balanced) considers reconstruction and calibration together. These labels identify selection priorities within the SBNN family. Rank and KL settings vary by encoder. Appendix § 9 shows how the tested rank combinations and KL scales affect reconstruction performance and calibration on ETT.

The pretext loss and Bayesian objective are defined in Equations (10) and (12). The contrastive and KL coefficients vary across retained configurations. For SBNN, KL regularization acts on the factor posteriors (Toure & Stephens [42]); for full-rank BBB, it acts on the weight posteriors (Blundell et al. [5]). Deterministic and MC Dropout training omit this variational KL term, as does each independently trained ensemble member.

Encoder-specific coefficients. Table 8 gives the retained SBNN training coefficients and the full-rank BBB coefficients used for each source family. SBNN configurations with identical coefficients are grouped.

SBNN configurationα\alphaKL coefficient SBNNKL coefficient Full-rank BBB
Acoustic balanced0.110−410^{-4}5×10−65 \times10^{-6}
Acoustic lean0.17.5×10−57.5 \times10^{-5}5×10−65 \times10^{-6}
Acoustic counterpart0.11.5×10−41.5 \times10^{-4}5×10−65 \times10^{-6}
Acoustic strong variants (3 encoders)0.15×10−65 \times10^{-6}5×10−65 \times10^{-6}
Acoustic CNN03×10−43 \times10^{-4}5×10−65 \times10^{-6}
HAR variants (3 encoders)0.50.030.03
Torque45 variants (3 encoders)05×10−55 \times10^{-5}10−310^{-3}
ETT base (KL 5×10−75 \times10^{-7})0.55×10−75 \times10^{-7}10−610^{-6}
ETT base (KL 10−610^{-6})0.510−610^{-6}10−610^{-6}
ETT wider0.52×10−62 \times10^{-6}10−610^{-6}
ETT compact0.510−610^{-6}10−610^{-6}

Table 8. Source-pretraining coefficients. α\alpha controls the reconstruction–contrastive mixture. KL columns give the final regularization coefficients. Full-rank BBB coefficients are specified per source family.

Masking settings. Table 9 gives the input-masking settings used during source pretraining.

Encoder configurationConfigured masking
Acoustic TransformerPoint fraction 0.05; span fraction 0.25
HARMask ratio 0.50
ETTMask ratio 0.25
Torque45Mask ratio 0
Acoustic CNNMask ratio 0

Table 9. Source-pretraining masking settings. Point and span masking are separate operations. A zero ratio means that no reconstruction mask is applied.

Contrastive settings. The contrastive losses use a temperature of 0.2. Acoustic Transformer encoders use symmetric cross-view InfoNCE; HAR and ETT encoders use a polynomial variant of NT-Xent. The loss-balancing factor is detached from gradients. For Acoustic, it is updated during training using an exponential moving average. For HAR and ETT, it is computed from the first batch and then held fixed.

Rank and KL ablation on ETT

We compare three joint encoder-decoder rank profiles at three KL scales, giving nine configurations with seed 40. The posterior scale settings are fixed across candidates. Figure 7 shows their reconstruction MSE, ECE, and PICP90. Increasing rank reduces reconstruction MSE in this sweep, while calibration and coverage depend on both rank and KL scale. The configuration with the lowest MSE does not have the lowest ECE or the coverage closest to 90%.

Figure 7. Rank–KL tradeoffs in ETT source reconstruction. Each circle represents one tested configuration. Colours identify the encoder and decoder ranks; point labels give the KL scale. Encoder ranks follow the order convolution 1, convolution 2, intermediate projection, and latent layer. Decoder ranks follow the order fully connected layer 1, fully connected layer 2, and output layer. MSE is evaluated over the whole window; ECE and PICP90 use the masked-point calibration evaluation. Dark outlines mark non-dominated configurations for MSE and ECE (left), or MSE and absolute coverage error from 90% (right).

Evaluation metrics

Reconstruction. We evaluate each selected encoder with its decoder on the incoming dataset. Let RmR_m denote its reconstruction MSE, averaged over input windows and their coordinates. For the selected set of encoders SS, the policy score is

RS=∑m∈SαmRm,(13)R_S = \sum_{m \in S} \alpha_m R_m, \tag*{(13)}

where αm\alpha_m is the weight assigned to encoder mm by the evaluated policy. We compute each encoder’s reconstruction MSE separately, then take their weighted average. Policies are compared within each incoming dataset, since raw errors from signals with different scales are not directly comparable.

Forecasting accuracy. Let NtestN_{\mathrm{test}} be the number of test windows and HH the number of forecast coordinates per window. Using the predictive mean μ^\hat{\mu} from § 2.3, we compute

MSE=1NtestH∑i=1Ntest∑h=1H(yi,h−μ^i,h)2.(14)\mathrm{MSE} = \frac{1}{N_{\mathrm{test}H}} \sum_{i=1}^{N_{\mathrm{test}}} \sum_{h=1}^{H} \left(y_{i,h} - \hat{\mu}_{i,h}\right)^2. \tag*{(14)}

Forecasting metrics use the standardized target scale.

Predictive negative log-likelihood. We evaluate a Gaussian approximation with predictive mean μ^\hat{\mu} and total variance v^pred\hat{v}_{\mathrm{pred}}. For numerical stability, let vi,h=max⁡{v^pred,i,h,10−12}v_{i,h} = \max\{\hat{v}_{\mathrm{pred},i,h}, 10^{-12}\}. The reported negative log-likelihood is

NLL=12NtestH∑i=1Ntest∑h=1H[log⁡(2πvi,h)+(yi,h−μ^i,h)2vi,h].(15)\mathrm{NLL} = \frac{1}{2N_{\mathrm{test}}H} \sum_{i=1}^{N_{\mathrm{test}}} \sum_{h=1}^{H} \left[\log(2\pi v_{i,h}) + \frac{\left(y_{i,h} - \hat{\mu}_{i,h}\right)^2}{v_{i,h}}\right]. \tag*{(15)}

Lower NLL indicates a better predictive density at the observed targets. This evaluation metric uses the total predictive variance and is distinct from the β\beta-NLL training objective.

Calibration and interval coverage. For downstream forecasting, we construct pointwise Gaussian prediction intervals using σ^=v^pred\hat{\sigma} = \sqrt{\hat{v}_{\mathrm{pred}}}. For nominal coverage qq, the interval is

Ii,h(q)=[μ^i,h−cqσ^i,h,μ^i,h+cqσ^i,h],cq=Φ−1(1+q2),(16)I_{i,h}(q) = [\hat{\mu}_{i,h} - c_q\hat{\sigma}_{i,h}, \hat{\mu}_{i,h} + c_q\hat{\sigma}_{i,h}], \qquad c_q = \Phi^{-1}\left(\frac{1+q}{2}\right), \tag*{(16)}

where Φ\Phi is the standard normal cumulative distribution function. Prediction interval coverage probability (PICP) is the observed fraction of targets within these intervals:

PICP⁡(q)=1NtestH∑i=1Ntest∑h=1H1{yi,h∈Ii,h(q)}.(17)\operatorname{PICP}(q) = \frac{1}{N_{\mathrm{test}}H}\sum_{i=1}^{N_{\mathrm{test}}}\sum_{h=1}^{H} \mathbf{1}\{y_{i,h} \in I_{i,h}(q)\}. \tag*{(17)}

We report PICP@90, corresponding to q=0.9q = 0.9. Calibration at this level is assessed by proximity to 90%.

Mean prediction interval width (MPIW) measures the average width of these intervals. At the 90% level,

MPIW⁡@90=1NtestH∑i=1Ntest∑h=1H2c0.9σ^i,h.(18)\operatorname{MPIW}@90 = \frac{1}{N_{\mathrm{test}}H}\sum_{i=1}^{N_{\mathrm{test}}}\sum_{h=1}^{H} 2c_{0.9}\hat{\sigma}_{i,h}. \tag*{(18)}

We assess MPIW@90 alongside PICP@90: narrow intervals are useful when their coverage remains close to the nominal level.

Our regression ECE averages the absolute coverage error over Q={0.1,0.2,…,0.9}Q = \{0.1, 0.2, \ldots, 0.9\}:

ECE⁡=1∣Q∣∑q∈Q∣PICP⁡(q)−q∣.(19)\operatorname{ECE} = \frac{1}{\lvert Q\rvert}\sum_{q\in Q} \lvert\operatorname{PICP}(q) - q\rvert. \tag*{(19)}

Lower ECE indicates closer agreement between nominal and empirical coverage across these levels.

Distribution shift. For downstream forecasts on incoming datasets, we report prediction interval coverage, width, and calibration using PICP@90, MPIW@90, and ECE. To assess the response to shift severity, we compute Spearman correlations between measured dataset distance and the uncertainty ratio, separately for each encoder and experiment. The distance definition and evaluation panels are given in § 4.2.

Aggregation across runs. For each incoming dataset, forecasting results report the mean and sample standard deviation over 3 seeds 40, 41, and 42. Overall means give equal weight to the seven dataset means. Routing comparisons pair runs by dataset and seed. For the family coverage-error summary, we first average coverage across seeds within each dataset. We then compute the absolute deviation from 90% and average these deviations across datasets.

Parameter and storage accounting

The source-specific encoder counts are reported in Table 2.

Forecasting head and pipeline. Table 10 separates the 17-encoder collection totals from the forecasting head counts used in the storage comparison. All heads use the same architecture and predict both the forecast and its conditional standard deviation. Their stored parameter counts differ because of their weight parameterizations.

FamilyEncodersHeadTotal
SBNN1,743,72214,7841,758,506
Deterministic4,818,73749,6004,868,337
MC Dropout4,818,73749,6004,868,337
Full-rank BBB9,632,60898,7529,731,360
Deep ensemble (5)24,093,685248,00024,341,685

Table 10. Encoder and forecasting-head parameter counts. Encoder totals cover the full 17-encoder collection in each family. The total includes the collection and forecasting head. Bayesian counts include posterior means and scales; ensemble counts include all five members. Bold indicates the smallest count in each column.

Parameter accounting. We count stored scalar model parameters, including frozen learned parameters. Bayesian counts include posterior means and scales. Decoder parameters are counted separately and excluded from forecasting. Fixed projection matrices and training-set standardization statistics are preprocessing state and are excluded from these counts. Their storage is separate from the reported model-parameter storage. Optimizer state and metric accumulators are excluded.

Let pm,fp_{m,f} be the parameter count of encoder mm in family ff, and let SfS_f denote that family’s selected set of KK experts. The deployed pipeline count is

Pf(Sf)=∑m∈Sfpm,f+phead,f.(20)P_f(S_f) = \sum_{m \in S_f} p_{m,f} + p_{\mathrm{head},f}. \tag*{(20)}

Here, phead,fp_{\mathrm{head},f} counts the deployed forecasting head parameters. Ensemble encoder and head counts include all five members. Each uncertainty family selects its own experts; the deterministic pipeline reuses the SBNN selection (§ 3.2). Collection storage sums the corresponding encoder costs over every expert in the collection, regardless of routing selections. These collection totals compare matched counterparts across families. At the same numerical precision, parameter-storage ratios equal parameter-count ratios.

The encoder-only reduction relative to the deterministic counterparts is 63.8%. Including the forecasting head gives reductions of 63.9% relative to the deterministic and MC Dropout models, 81.9% relative to full-rank BBB, and 92.8% relative to the five-member ensemble.

A rank-matched deterministic factorization requires fewer stored parameters, but does not itself provide the epistemic uncertainty used by our router. Our objective is to obtain compact experts whose uncertainty also supports label-free selection. The reported storage reductions compare these Bayesian experts with their full-rank deterministic counterparts.

Stored components of alternative Bayesian methods. Table 11 summarizes what each parameterization retains for prediction, including frozen weights.

MethodStored components
SBNN [42]Means and scales of the low-rank weight factors; no retained dense weight matrix for the factorized layer.
Rank-1 BNNs [14]Shared dense weight matrix and posterior parameters for multiplicative vectors.
SWAG [27]Dense weight mean, diagonal variance statistics and retained low-rank covariance deviation vectors.
Low-rank Gaussian VI [41]Dense weight mean and low-rank covariance factors, with a fixed diagonal constant or a learned diagonal variance vector.
BLoB [45]Frozen dense backbone, one deterministic adapter factor, and the mean and scale of the other adapter factor.

Table 11. Stored components of Bayesian parameterizations. The comparison describes the published constructions, including frozen backbone weights.

Routing protocol details

Routing is performed once for each incoming dataset. The selected encoders and their aggregation weights then remain fixed during head training and forecasting. Neither selection method uses forecast targets.

Incoming inputs and source references. We compute routing scores from 192 unlabeled windows, each containing 96 observations, sampled at evenly spaced positions within the first 70% of the incoming series. These windows form a separate routing probe. They are mapped to each encoder’s input format using the preprocessing described in § 2.2.

For each encoder, the source uncertainty reference is computed with the statistic in Equation (2). The incoming uncertainty is divided by this stored reference, as defined in Equation (3). Only this scalar reference is retained for routing.

Each source reference is computed from 192 windows from the encoder’s own source dataset. We use 40 posterior weight draws to estimate both the source reference and the incoming routing uncertainty.

Uncertainty-based selection. Selection and weighting follow Table 3.

The nine-policy comparison uses a pool of 17 encoders. Its low-uncertainty shortlist contains the eight encoders with the lowest uncertainty ratios.

Source-distance selection. The distance baseline uses the statistical-feature distance defined in Equation (8). All encoder variants trained on the same source receive that source’s distance score. The KK lowest-scoring candidates are selected and receive normalized inverse-distance weights.

When several candidates have the same distance, their selection follows a fixed ordering of encoder variants. The selected set can therefore contain several encoders from the same source. Candidates with equal distance receive equal weights.

This baseline tests source-distance selection. It does not reproduce the full two-stage routing method of Pham et al. [33].

Candidate eligibility. The routing comparisons use a pool of 17 encoders.

The distance-versus-uncertainty experiment compares both selection and weighting. Its results therefore assess the two routing policies as a whole.

Additional source reconstruction results

Reconstruction accuracy and calibration. Table 12 reports reconstruction accuracy and uncertainty calibration on the source domains, before routing or downstream forecasting. The evaluated configurations and additional SBNN selection roles are given in Table 13.

Table 12: Source reconstruction and calibration. MSE uses validation windows; uncertainty metrics follow each source’s protocol described below. Bold marks the lowest MSE or ECE, or coverage closest to 90%. Dashes denote unreported values.

SourceModelMSE ↓\downarrowECE ↓\downarrowPICP90 (%) →\to 90%MPIW90
AcousticDeterministic0.07613———
AcousticMC Dropout0.076130.362029.480.2880
AcousticDeep ensemble (5)0.055150.339130.860.2865
AcousticFull-rank BBB0.074920.125171.090.8453
AcousticSBNN0.106010.045380.721.3125
HARDeterministic0.08142———
HARMC Dropout0.081890.430114.860.0491
HARDeep ensemble (5)0.080640.060272.460.2012
HARFull-rank BBB0.080760.068678.270.2816
HARSBNN0.100050.111477.620.3291
Torque45Deterministic0.03571———
Torque45MC Dropout0.153390.152066.760.5721
Torque45Deep ensemble (5)0.035510.439613.220.0660
Torque45Full-rank BBB0.035790.328737.080.1842
Torque45SBNN0.035940.009388.600.6147
ETTDeterministic0.09003———
ETTMC Dropout0.118590.283644.920.4341
ETTDeep ensemble (5)0.084580.395119.790.1800
ETTFull-rank BBB0.082570.442212.580.0867
ETTSBNN0.095670.143572.490.8026

Table 12.

ModelMSE ↓\downarrowECE ↓\downarrowPICP90 (%) →90%\to90\%MPIW90
Acoustic
Deterministic0.07613———
MC Dropout0.076130.362029.480.2880
Deep ensemble (5)0.055150.339130.860.2865
Full-rank BBB0.074920.125171.090.8453
SBNN (reconstruction)0.090970.089874.791.0428
SBNN (calibration)0.106010.045380.721.3125
SBNN (balanced)0.101640.057279.101.2267
HAR
Deterministic0.08142———
MC Dropout0.081890.430114.860.0491
Deep ensemble (5)0.080640.060272.460.2012
Full-rank BBB0.080760.068678.270.2816
SBNN (reconstruction)0.093310.118976.560.3205
SBNN (calibration)0.100050.111477.620.3291
SBNN (balanced)0.102840.114173.100.2967
Torque45
Deterministic0.03571———
MC Dropout0.153390.152066.760.5721
Deep ensemble (5)0.035510.439613.220.0660
Full-rank BBB0.035790.328737.080.1842
SBNN (reconstruction)0.035940.103877.490.4604
SBNN (calibration)0.035940.009388.600.6147
SBNN (balanced)0.035940.064794.390.7698
ETT
Deterministic0.09003———
MC Dropout0.118590.283644.920.4341
Deep ensemble (5)0.084580.395119.790.1800
Full-rank BBB0.082570.442212.580.0867
SBNN (retained)0.095670.143572.490.8026

Table 13. Source reconstruction results across model families and SBNN configurations. Bold indicates the lowest MSE or ECE, or coverage closest to 90%, within each source. Dashes indicate unreported uncertainty metrics.

For source reconstruction, Acoustic and ETT use empirical quantiles of sampled reconstructions to form prediction intervals. HAR and Torque45 use Gaussian intervals computed from the sampled reconstruction means and standard deviations. Calibration uses masked positions for Acoustic and ETT and the full target windows for HAR and Torque45.

Figure 9 compares source reconstructions and uncertainty bands across model families.

Figure 9. Source reconstructions and uncertainty bands. Selected validation windows from the four source domains. The first row shows the original signal. Subsequent rows show reconstructions (solid), the original signal (dashed) and nominal 90% uncertainty bands. Models share the same window and axes within each column. The deterministic model has no uncertainty band.

The selected SBNN encoders achieve the lowest ECE and coverage closest to the nominal 90% level on three of the four source domains. HAR is the exception; its data-quality note is given in Appendix § 13.§ 3. SBNN does not achieve the lowest reconstruction MSE. Its advantage is better uncertainty calibration at a lower storage cost, with a source-dependent increase in reconstruction error. The wider SBNN intervals cover more of the signal, while several baselines give narrow intervals that miss much of it despite low MSE.

This accuracy-calibration tradeoff matters for expert selection. Low reconstruction MSE can coexist with overconfident uncertainty estimates. We therefore prioritize uncertainty quality alongside reconstruction accuracy. The following shift and routing experiments evaluate how the encoders’ epistemic uncertainty supports expert selection. Rank and KL scale are tuned for compression with competitive reconstruction and calibration (Appendix § 9).

The table reports the source configurations evaluated in this reconstruction comparison.

Source results across SBNN configurations

Table 13 extends the source-domain comparison in Table 12. These results concern reconstruction on the source datasets, before routing or downstream forecasting.

For Acoustic, HAR and Torque45, we report three SBNN selection priorities. The reconstruction-selected configuration prioritizes reconstruction accuracy, the calibration-selected configuration prioritizes uncertainty quality, and the balanced configuration considers both. ETT reports its retained rank-KL configuration.

The additional configurations show how the selection priority changes the accuracy-calibration balance. On Acoustic, prioritizing calibration lowers ECE and improves coverage at the cost of higher reconstruction MSE. On Torque45, the three SBNN configurations have the same MSE to the displayed precision, but different interval widths and coverage. The calibration-selected configuration is closest to nominal coverage. On HAR, the ensemble and BBB retain stronger calibration results than the reported SBNN configurations.

Source reconstruction examples

The following figures compare original signals, reconstructions and uncertainty intervals on selected source windows.

Figure 10. Acoustic source reconstruction. Each column shows the same window across all model families. The first row displays the original signal. Subsequent rows show reconstructions with nominal 90% uncertainty bands; dashed curves indicate the original signal. The deterministic model has no uncertainty band.

Figure 11. HAR source reconstruction. Each column shows the same window across all model families. The first row displays the original signal. Subsequent rows show reconstructions with nominal 90% uncertainty bands; dashed curves indicate the original signal. The deterministic model has no uncertainty band.

HAR data quality

We use the supplied HAR windows for signal reconstruction. Activity labels do not enter the pretext objective. The data preparation removes 26 windows identified by an outlier filter before selecting the source-training subset. All model families use the same prepared inputs.

The displayed windows contain varying-amplitude, pulse-like patterns followed by flat segments at zero. Their structure should be considered when interpreting reconstruction errors and prediction intervals.

On this source, the deep ensemble achieves the lowest ECE, while full-rank BBB has coverage closest to the nominal 90% level.

Additional distribution-shift results

This section provides the values supporting the distribution-shift analysis in § 4.2.

Statistical-feature distance

For each dataset, we construct a statistical descriptor from 192 input windows of length 96. It contains the mean, population standard deviation, 10th, 50th and 90th percentiles, and autocorrelations at lags 1, 5 and 10:

ϕ(X)=[μ,σ,q0.10,q0.50,q0.90,ACF1,ACF5,ACF10].(21)\phi(\mathcal{X}) = [\mu, \sigma, q_{0.10}, q_{0.50}, q_{0.90}, \mathrm{ACF}_{1}, \mathrm{ACF}_{5}, \mathrm{ACF}_{10}]. \tag*{(21)}

The mean, standard deviation and quantiles are computed over all values in the selected windows. Autocorrelations are computed within each window and then averaged across windows. For each lag, we centre the window and divide the sum of lagged products by its total centred sum of squares.

The distance follows Equation (8). Each coordinate scale cjc_j is the population standard deviation of that descriptor across Acoustic, HAR, Torque45 and ETT. These scales remain fixed across comparisons.

Descriptors use the stored window values without additional per-window standardization. The score therefore remains sensitive to differences in level and amplitude as well as temporal dependence.

Distance and encoder uncertainty

We evaluate one fixed encoder from each of the Torque45, Acoustic and ETT sources across eight domains, including each encoder’s source. For each encoder–dataset pair, we compute the uncertainty ratio defined in § 2.2 using 50 posterior draws. No decoder or forecasting head is involved. We compare the statistical descriptors of the source and incoming datasets using the distance defined in Equation (8). Distance is used here to assess the uncertainty response. Uncertainty-based routing uses the epistemic ratios.

Figure 3 shows a positive association between statistical-feature distance and encoder uncertainty. For Torque45 and ETT, uncertainty increases with distance overall, with local reversals in the ordering. Acoustic saturates faster: its uncertainty ratio changes little between Torque10 and Torque90 and decreases slightly on ECL despite the larger distance. Spearman correlations across the eight domains are 0.929 for Torque45, 0.810 for Acoustic and 0.905 for ETT.

Table 14 reports the statistical-feature distance and source-normalized uncertainty ratio for each evaluated domain. The ratio follows Equation (3).

For the Acoustic encoder, Torque45 and HAR produce uncertainty ratios below one. These values indicate less variation across posterior representations than on Acoustic inputs. Their statistical-feature distances remain positive; the Acoustic source has distance zero.

Why uncertainty ratios can fall below one

The source reference compares the dispersion of representations under a fixed weight posterior. The same posterior can produce more stable representations on a new dataset than on its source. For example, the Acoustic ratios below one in Table 14 indicate lower posterior representation dispersion on those inputs. The following construction establishes that this behaviour is possible within the SBNN variational family.

DatasetTorque45 encoderAcoustic encoderETT encoder
DDrrDDrrDDrr
Torque450.0001.0001.0550.7511.7391.538
Acoustic1.0553.1770.0001.0001.9392.000
HAR1.0590.9621.8740.7801.8631.437
ETT1.7397.7591.9391.9820.0001.000
WTH1.1303.9131.0631.2891.5471.627
Torque104.03518.6483.6992.9765.3975.749
Torque9030.820143.83330.4533.03832.17234.601
ECL419.5651878.404419.8082.756418.055407.874

Table 14. Statistical-feature distance and encoder uncertainty across eight domains. DD is the distance from the encoder’s source dataset; rr is its source-normalized uncertainty ratio.

For a fixed posterior qq and a finite input set X\mathcal{X}, define the population counterpart of Equation (2) as

Uq(X)=1∣X∣d∑x∈X∑j=1dVar⁡q[zj(x)].U_q(\mathcal{X}) = \frac{1}{|\mathcal{X}|d} \sum_{x\in\mathcal{X}}\sum_{j=1}^{d}\sqrt{\operatorname{Var}_q[z_j(x)]}.

where dd is the representation dimension. With independent posterior draws and finite second moments, the sample-based statistic converges almost surely to this quantity as the number of draws increases.

A rank-one SBNN construction. Consider z(x)=ABTxz(x)=AB^{\mathsf{T}}x, with A=(a1,a2)TA=(a_1,a_2)^{\mathsf{T}} and B=(b1,b2)TB=(b_1,b_2)^{\mathsf{T}}. Let all four factor entries be independent, with a1,a2,b1∼N(1,1)a_1,a_2,b_1\sim\mathcal{N}(1,1) and b2∼N(2,4)b_2\sim\mathcal{N}(2,4); the second Gaussian argument is the variance. This is a mean-field Gaussian factor posterior with strictly positive variances, and ABTAB^{\mathsf{T}} has rank one almost surely. Let the source and incoming input distributions be uniform on Xsrc={e2,−e2}\mathcal{X}_{\mathrm{src}}=\{e_2,-e_2\} and Xin={e1,−e1}\mathcal{X}_{\mathrm{in}}=\{e_1,-e_1\}, respectively, where e1=(1,0)Te_1=(1,0)^{\mathsf{T}} and e2=(0,1)Te_2=(0,1)^{\mathsf{T}}. Their supports are disjoint, and every input has unit norm.

For independent scalar factors aa and bb,

Var⁡(ab)=(Var⁡(a)+E[a]2)Var⁡(b)+Var⁡(a)E[b]2.\operatorname{Var}(ab) = \left(\operatorname{Var}(a)+\mathbb{E}[a]^2\right)\operatorname{Var}(b)+\operatorname{Var}(a)\mathbb{E}[b]^2.

For each coordinate j∈{1,2}j\in\{1,2\}, this gives Var⁡(ajb1)=3\operatorname{Var}(a_jb_1)=3 and Var⁡(ajb2)=12\operatorname{Var}(a_jb_2)=12. Input sign changes leave these variances unchanged. Consequently, Uq(Xin)=3U_q(\mathcal{X}_{\mathrm{in}})=\sqrt{3} and Uq(Xsrc)=12U_q(\mathcal{X}_{\mathrm{src}})=\sqrt{12}, so their ratio is exactly 1/21/2. Since the source quantity is positive, the corresponding Monte Carlo ratio converges almost surely to 1/21/2 as the posterior draw counts for both sets increase.

The construction uses equal-norm inputs and a linear encoder, so neither a change in input amplitude nor activation saturation is required for a ratio below one. The weight posterior stays fixed; its induced representation dispersion depends on the input. This establishes theoretical compatibility with the observed sub-unit ratios. Identifying the particular mechanism in the trained Acoustic encoder would require a separate analysis. The broader dependence of Bayesian uncertainty on function-space properties is discussed by D’Angelo & Henning [10].

Forecasting loss and temporal fidelity

Initial forecasting behaviour. Our initial forecasting head was trained with MSE and returned only a point prediction. Although aggregate errors were low, inspection of the forecast windows showed overly smooth predictions. They stayed near the signal’s average level and missed much of its variation. We describe this behaviour as flattening toward the mean. This observation motivated the change to a probabilistic head trained with β\beta-NLL.

Learning the forecast and its uncertainty. The final head predicts a value μ\mu and a positive standard deviation σ\sigma for each forecast coordinate. The learned σ\sigma represents the conditional aleatoric uncertainty. For a scalar target yy, the loss is

Lβ-NLL(y,μ,σ)=stopgrad⁡((σ2)β)12[log⁡σ2+(y−μ)2σ2],β=0.5.(22)\mathcal{L}_{\beta\text{-NLL}}(y,\mu,\sigma)=\operatorname{stopgrad}\left((\sigma^2)^\beta\right)\frac{1}{2}\left[\log\sigma^2+\frac{(y-\mu)^2}{\sigma^2}\right], \qquad\beta=0.5. \tag*{(22)}

The loss is averaged over forecast coordinates and training examples. The stop-gradient operation holds the multiplicative weight fixed when differentiating the loss. This weight reduces the downweighting of mean-fitting errors by large predicted variances under Gaussian NLL.

Under ordinary Gaussian NLL, the gradient with respect to the point prediction is

∂LNLL∂μ=μ−yσ2.(23)\frac{\partial\mathcal{L}_{\mathrm{NLL}}}{\partial\mu} = \frac{\mu-y}{\sigma^2}. \tag*{(23)}

A large predicted variance therefore reduces the contribution of an observation to fitting the prediction. Seitzer et al. [37] show how this weighting can hinder learning in poorly fitted regions.

The detached variance weight in β\beta-NLL changes this gradient to

∂Lβ-NLL∂μ=(μ−y)(σ2)β−1.(24)\frac{\partial\mathcal{L}_{\beta\text{-NLL}}}{\partial\mu} = (\mu-y)(\sigma^2)^{\beta-1}. \tag*{(24)}

For β=0.5\beta=0.5, the gradient is (μ−y)/σ(\mu-y)/\sigma. Compared with Gaussian NLL, high predicted variance suppresses the prediction-fitting gradient less strongly.

MSE has no variance-dependent weighting. Our transition therefore changes both the training objective and the head’s outputs. The final head learns an observation-noise estimate, which contributes to the prediction intervals alongside epistemic uncertainty. Temporal fidelity is examined through forecast windows; accuracy and uncertainty quality are evaluated separately using MSE, ECE, coverage and Gaussian NLL.

Bayesian head objective. For the Bayesian head, the objective is

Jhead(qϕ)=1N∑i=1NEω∼qϕ[Lβ-NLL(yi,μi(ω),σi(ω))]+λheadKhead.(25)\mathcal{J}_{\mathrm{head}}(q_{\phi}) = \frac{1}{N}\sum_{i=1}^{N}\mathbb{E}_{\omega\sim q_{\phi}}\left[\mathcal{L}_{\beta\text{-NLL}}(y_i,\mu_i(\omega),\sigma_i(\omega))\right] + \lambda_{\mathrm{head}}\mathcal{K}_{\mathrm{head}}. \tag*{(25)}

Head training uses a fixed-feature approximation: the expectation is over head weights only, with encoder weights fixed at their posterior means. Encoder uncertainty is propagated at prediction. The encoder and head posteriors are fitted in separate stages. The head uses a KL-regularized β\beta-NLL surrogate objective.

The term Khead\mathcal{K}_{\mathrm{head}} sums the head’s factor-space KL terms, as in §C.1, and λhead\lambda_{\mathrm{head}} controls their strength.

Sampling encoder weights at prediction can change the feature distribution received by the head. Through nonlinear transformations, this can affect both forecasts and predicted conditional scales. The reported calibration metrics evaluate the complete predictive distribution obtained with this procedure. The effect of encoder sampling was not isolated experimentally.

Let QQ denote the sampling distribution over the frozen encoder posteriors and the fitted head posterior. For each forecast coordinate YY, the Gaussian mixture defined by our prediction procedure satisfies the law of total variance:

Var⁡(Y∣x)=Var⁡Q[μ(x)]+EQ[σ2(x)].\operatorname{Var}(Y\mid x)=\operatorname{Var}_{Q}[\mu(x)]+\mathbb{E}_{Q}[\sigma^2(x)].

This identity gives the variance of the constructed predictive distribution; its calibration is assessed empirically.

Initial forecasting without a learned aleatoric output

The initial forecasting heads were trained with MSE and returned only a point prediction. Prediction intervals used epistemic uncertainty, without a learned aleatoric contribution.

In that setup, SBNN had substantially lower mean ECE and coverage much closer to the nominal 90% level than MC Dropout and deep ensembles. With β\beta-NLL, the head also predicts an aleatoric standard deviation σ\sigma. Including this contribution in the predictive intervals improves coverage across these families and substantially narrows the calibration gap, as shown in the final comparison.

Table 15 reports the average results across the seven incoming datasets. The deterministic model produced no uncertainty intervals in this initial setup.

FamilyMSE ↓ECE ↓PICP90 (%) → 90%MPIW90
SBNN0.4560.08880.601.874
Deterministic0.407———
MC Dropout0.3920.34132.770.363
Deep ensemble (5)0.3940.38524.610.283

Table 15. Initial forecasting with epistemic-only intervals. Results are averaged across seven incoming datasets. Heads were trained with MSE. A dash indicates an unavailable interval metric.

SBNN provides substantially better average calibration and coverage in this initial comparison, at the cost of higher forecasting MSE. Its ECE is lowest on six of the seven datasets, and its coverage is closest to 90% on all seven. The much narrower MC Dropout and ensemble intervals cover fewer than one third of the targets on average.

Section 4.3 reports the final forecasting results with β\beta-NLL heads and learned aleatoric uncertainty.

Detailed model-family forecasting results

Comparison objectives. We separate the comparison of model families from the comparison of routing policies. The first compares the storage and predictive quality of complete family-specific pipelines, including their routing. The second tests how expert selection affects downstream forecasting.

Matched model families. Our routing collection contains SBNN encoders. For comparison, each SBNN expert is matched to a counterpart trained on the same source dataset in the deterministic, MC Dropout, full-rank Bayes by Backprop, and five-member deep ensemble families. Each uncertainty family performs its own routing and computes its own aggregation weights. The deterministic pipeline uses the SBNN selections and weights. Each family uses its own encoders and a corresponding forecasting head. We compare source reconstruction and downstream forecasting, including predictive uncertainty across the full encoder–head pipeline. The deterministic pipeline uses fixed weights. Its heteroscedastic regression head predicts the target yy and an input-dependent standard deviation σ\sigma.

Table 4 summarizes forecasting accuracy and storage; Appendix F gives the encoder and head counts separately.

Predictive calibration and coverage. The intervals and calibration metrics use the total predictive variance in Equation (7). It combines variation across sampled forecasts with the average squared standard deviation predicted by the head. The learned σ\sigma contributes to interval coverage in every family, including the deterministic pipeline, whose weights remain fixed. It can improve coverage even when the epistemic component is small.

SBNN has the lowest mean ECE, 0.0687, compared with 0.0734 for the ensemble, 0.0767 for MC Dropout and 0.0877 for BBB. Its mean NLL is also lowest, although the difference from MC Dropout is small: 0.814 versus 0.817. PICP90 reaches 90.47%, closest to the nominal 90% level. MC Dropout and the ensemble reach 87.78% and 84.55%, while BBB reaches 92.29%.

Coverage must be considered together with interval width. SBNN’s mean MPIW90 is 1.771, above MC Dropout’s 1.647 and the ensemble’s 1.517, and below BBB’s 2.100. Its improved coverage comes with wider intervals than the under-covering MC Dropout and ensemble models. The deterministic pipeline has the narrowest mean intervals. They cover 82.60% of the targets.

Algorithm 1 summarizes prediction after routing and head training. The selected encoders, routing weights and alignment maps remain fixed; encoder and head weights are sampled at each iteration.

Algorithm 1 Monte Carlo prediction through the selected encoder–head pipeline

Require: Input window xx, selected set of encoders SS, weights {αm}m∈S\{\alpha_m\}_{m\in S}, number of draws NMCN_{\mathrm{MC}}

Require: Frozen encoder posteriors {qm}m∈S\{q_m\}_{m\in S}, preprocessing maps TmT_m, alignment maps PmP_m

Require: Trained head gg with weight posterior qϕq_\phi

Ensure: Predictive mean and epistemic, aleatoric, and total variances

  1. for t=1,…,NMCt = 1, \ldots, N_{\mathrm{MC}} do

  2. h(t)←0h^{(t)} \leftarrow0

  3. for each m∈Sm \in S do

  4. Sample θm(t)∼qm\theta_m^{(t)} \sim q_m

  5. zm(t)←fm(Tm(x);θm(t))z_m^{(t)} \leftarrow f_m(T_m(x); \theta_m^{(t)})

  6. h(t)←h(t)+αmPm(zm(t))h^{(t)} \leftarrow h^{(t)} + \alpha_m P_m(z_m^{(t)})

  7. end for

  8. Sample ω(t)∼qϕ\omega^{(t)} \sim q_\phi

  9. (μ(t),σ(t))←g(h(t);ω(t))(\mu^{(t)}, \sigma^{(t)}) \leftarrow g(h^{(t)}; \omega^{(t)})

  10. end for

  11. μ^←NMC−1∑t=1NMCμ(t)\hat{\mu} \leftarrow N_{\mathrm{MC}}^{-1} \sum_{t=1}^{N_{\mathrm{MC}}} \mu^{(t)}

  12. v^epi←NMC−1∑t=1NMC(μ(t)−μ^)2\hat{v}_{\mathrm{epi}} \leftarrow N_{\mathrm{MC}}^{-1} \sum_{t=1}^{N_{\mathrm{MC}}}(\mu^{(t)}-\hat{\mu})^2

  13. v^alea←NMC−1∑t=1NMC(σ(t))2\hat{v}_{\mathrm{alea}} \leftarrow N_{\mathrm{MC}}^{-1} \sum_{t=1}^{N_{\mathrm{MC}}}(\sigma^{(t)})^2

  14. v^pred←v^epi+v^alea\hat{v}_{\mathrm{pred}} \leftarrow\hat{v}_{\mathrm{epi}}+\hat{v}_{\mathrm{alea}}

  15. return μ^,v^epi,v^alea,v^pred\hat{\mu},\hat{v}_{\mathrm{epi}},\hat{v}_{\mathrm{alea}},\hat{v}_{\mathrm{pred}}

Forecast means and variances are computed coordinatewise.

Forecast examples across incoming datasets

Additional predictive-coverage comparisons

SBNN has the lowest average absolute deviation from nominal 90% coverage: 3.15 percentage points, compared with 3.32 for full-rank BBB, 4.89 for MC Dropout and 6.07 for deep ensembles. We compute this deviation from each dataset’s seed-averaged coverage before averaging across datasets, so undercoverage and overcoverage do not cancel. Coverage varies across incoming datasets; near-nominal average coverage does not imply nominal coverage on every dataset. SBNN combines competitive forecasting and lower average coverage error with substantially fewer stored parameters.

Figure 13 compares observed coverage at nominal levels of 70% and 90%. SBNN is closest to nominal coverage at 90%, while MC Dropout is closest at 70%.

Figure 13. Forecasting coverage at two nominal levels. Points show observed coverage at nominal 70% and 90%, averaged over seven incoming datasets and three seeds. The dashed diagonal marks agreement between nominal and observed coverage.

Fixed projection and representation fusion

For this derivation, index the selected experts by m=1,…,Km = 1,\ldots,K. Let um∈Rdmu_m \in\mathbb{R}^{d_m} denote encoder mm’s standardized representation and Rm∈R64×dmR_m \in\mathbb{R}^{64 \times d_m} its fixed projection matrix, so Pm(zm)=RmumP_m(z_m) = R_m u_m. The aggregation in (5) can be written as

h=Rαu,Rα=[α1R1,…,αKRK],u=[u1⊤,…,uK⊤]⊤.h = R_\alpha u,\qquad R_\alpha= [\alpha_1 R_1,\ldots,\alpha_K R_K],\qquad u = [u_1^\top,\ldots,u_K^\top]^\top.

Thus, fusion is a fixed linear compression of the concatenated, standardized representations. It gives the head a constant input dimension across heterogeneous encoders and selected sets, without learning additional projection parameters. The head is fitted to this combined feature map; latent coordinates from different encoders need not have matching meanings. Compression can discard useful information. Comparisons with concatenation or learned projections remain future work.

Covariance of the fused representation. Condition on the selected set, routing weights, standardization statistics and realized projection matrices. For a specified source of variation with finite second moments, write Cmn=Cov⁡(um,un)C_{mn} = \operatorname{Cov}(u_m, u_n). Linearity gives the exact identity

Cov⁡(h)=∑m=1K∑n=1KαmαnRmCmnRn⊤.\operatorname{Cov}(h) = \sum_{m=1}^{K} \sum_{n=1}^{K} \alpha_m \alpha_n R_m C_{mn} R_n^{\top}.

Across input windows, the cross-covariances can be nonzero because all encoders process the same inputs. When variation comes only from independently sampled encoder weights at a fixed input xx, Cmn(x)=0C_{mn}(x) = 0 for m≠nm \ne n. Writing Vm(x)=RmCmm(x)Rm⊤V_m(x) = R_m C_{mm}(x) R_m^{\top}, the covariance then becomes

Cov⁡q(h∣x)=∑m=1Kαm2Vm(x).\operatorname{Cov}_q(h \mid x) = \sum_{m=1}^{K} \alpha_m^2 V_m(x).

The weight identity ∑mαm2∈[1/K,1]\sum_m \alpha_m^2 \in[1/K,1] holds for nonnegative weights summing to one. If all projected expert covariances equal V(x)V(x), the last expression reduces to (∑mαm2)V(x)(\sum_m \alpha_m^2)V(x). Independence of the random matrix draws alone does not establish that equality for the realized projected expert covariances. Changing the selected experts, KK or their weights can therefore change feature scale. Each policy fits its forecasting head to its own fixed feature map.

Detailed distance-versus-uncertainty routing results

Aggregate routing comparison. Table 16 summarizes the top-3 routing comparison in § 4.4.

Table 17 gives the per-dataset results for the comparison in § 4.4. Both policies select three SBNN encoders. Entries report the mean and sample standard deviation across seeds 40, 41 and 42.

PolicyMSE ↓\downarrowECE ↓\downarrowNLL ↓\downarrowPICP90 (%) →90%\to90\%MPIW90
Torque10
Top-3 distance0.5531 ± 0.00980.0624 ± 0.00861.108 ± 0.01482.62 ± 1.811.997 ± 0.013
Top-3 uncertainty0.5004 ± 0.01590.0419 ± 0.00711.067 ± 0.02085.45 ± 0.562.038 ± 0.010
Torque90
Top-3 distance0.3164 ± 0.00410.0436 ± 0.00360.889 ± 0.00583.52 ± 0.221.999 ± 0.024
Top-3 uncertainty0.3319 ± 0.00290.0195 ± 0.00230.808 ± 0.00191.46 ± 0.211.914 ± 0.010
WTH
Top-3 distance0.3112 ± 0.00260.0142 ± 0.00070.800 ± 0.00184.35 ± 0.991.615 ± 0.017
Top-3 uncertainty0.2869 ± 0.00830.0055 ± 0.00150.744 ± 0.01688.74 ± 0.331.501 ± 0.014
ETTh2
Top-3 distance0.3219 ± 0.02890.0926 ± 0.00100.990 ± 0.09274.39 ± 0.821.550 ± 0.007
Top-3 uncertainty0.3516 ± 0.03660.0874 ± 0.01120.951 ± 0.07179.44 ± 1.331.452 ± 0.017
ECL
Top-3 distance0.4700 ± 0.00910.0096 ± 0.00351.009 ± 0.00284.85 ± 0.232.036 ± 0.029
Top-3 uncertainty0.4708 ± 0.00130.0067 ± 0.00221.003 ± 0.00188.69 ± 0.211.971 ± 0.013
Vibration
Top-3 distance0.1060 ± 0.00070.1580 ± 0.00540.137 ± 0.00481.94 ± 0.421.214 ± 0.008
Top-3 uncertainty0.0853 ± 0.00020.1378 ± 0.0066−0.067 ± 0.00497.33 ± 0.071.041 ± 0.007
EMG
Top-3 distance0.7645 ± 0.01210.2198 ± 0.00971.418 ± 0.17584.32 ± 2.252.154 ± 0.096
Top-3 uncertainty0.7563 ± 0.00060.2061 ± 0.02651.344 ± 0.25492.12 ± 1.492.092 ± 0.222

Table 17. Forecasting under distance and uncertainty routing. PICP90 is expressed as a percentage.

Figure 14 shows the corresponding forecasting error, likelihood and interval-width comparisons.

Figure 14. Forecasting under distance and uncertainty routing. Both policies select three SBNN encoders. Bars show averages across seven incoming datasets. Whiskers show sample standard deviations across three seeds, after averaging each seed across datasets. The dashed line marks nominal 90% coverage.

Additional selection-policy results

For reconstruction on incoming datasets, each encoder’s intervals use Gaussian moments from 40 posterior reconstruction draws. We compute ECE and PICP90 separately for each selected encoder, then average these metrics using the policy’s aggregation weights, as for reconstruction MSE in Equation (13).

Detailed reconstruction and forecasting results

Table 18 gives the per-dataset reconstruction MSE underlying Table 5.

PolicyTorque10Torque90WTHETTh2ECLVibrationEMG
Top-3 uncertainty0.00152.43 × 10⁻⁶0.02680.01340.01040.01840.0090
Top-4 uncertainty0.20690.12600.08320.18990.26020.01930.0098
Source-diverse top-30.52470.34610.24050.54370.71430.04780.1360
Source-diverse top-40.49260.34240.20600.45920.71360.16860.1226
Random-4 of top-80.59890.40880.20920.63030.87130.02190.0079
Bottom-4 of top-80.59750.40880.20680.62920.87130.01960.0098
Random-4, full pool0.77640.54330.22990.85731.15850.16250.1109
Bottom-3 uncertainty0.55270.36500.31770.63180.77810.47090.0152
Bottom-4 uncertainty0.63470.40840.35290.71040.87350.39660.2640

Table 18.

Table 18: Reconstruction MSE on the incoming datasets. Bold marks the lowest error within each dataset.

Table 19 reports reconstruction ECE on the same incoming datasets.

PolicyTorque10Torque90WTHETTh2ECLVibrationEMG
Top-3 uncertainty0.37520.23160.16690.19360.34040.29120.3451
Top-4 uncertainty0.40440.29330.21730.25960.37520.29310.3432
Source-diverse top-30.46600.42500.36490.40450.45310.31290.2775
Source-diverse top-40.45680.42370.30010.34590.45290.31290.2566
Random-4 of top-80.46880.41790.30650.40820.45060.29960.2759
Bottom-4 of top-80.46880.41790.30760.40810.45060.29430.3477
Random-4, full pool0.50000.50000.30790.49990.50000.26750.2700
Bottom-3 uncertainty0.41450.41280.34740.34660.40630.37010.1419
Bottom-4 uncertainty0.43590.43460.38290.38500.42980.38140.1848

Table 19.

Table 19: Calibration of incoming-data reconstructions. Bold marks the lowest ECE within each dataset.

Forecasting calibration. Table 20 reports forecasting ECE for each selection policy.

PolicyTorque10Torque90WTHETTh2ECLVibrationEMG
Top-3 uncertainty0.04190.01950.00550.08740.00670.13780.2061
Top-4 uncertainty0.01660.03270.01010.06430.00680.13450.2158
Source-diverse top-30.00950.03260.01610.04850.01350.12730.2296
Source-diverse top-40.01290.04010.01080.05140.00980.12400.2285
Random-4 of top-80.01180.03450.00520.05930.01860.11710.2263
Bottom-4 of top-80.01810.03470.00810.05160.01320.11460.2509
Random-4, full pool0.09740.05600.00940.04260.03860.12980.2447
Bottom-3 uncertainty0.03820.06950.02290.03800.02230.13680.2656
Bottom-4 uncertainty0.03570.07660.01400.05520.02330.12440.2649

Table 20.

Table 20: Forecasting calibration by incoming dataset and selection policy. Lower ECE is better. Bold values indicate the lowest ECE within each dataset.

Routing with a fixed uncertainty-ratio cutoff

We also evaluate the policy ratio_le_1p10. From the pool of 17 encoders, it selects every encoder whose source-normalized uncertainty ratio satisfies rm≤1.10r_m \le1.10. Selected encoders receive normalized inverse-ratio weights, following Equation (4).

The cutoff was fixed from reconstruction-stage uncertainty ratios before forecasting. This policy allows the number of selected encoders to vary across incoming datasets. It is separate from the eight-member shortlist used in the nine-policy comparison. The forecasting evaluation uses seeds 40, 41 and 42. Table 21 compares the cutoff policy with fixed top-3 and top-4 uncertainty routing.

PolicyMSE ↓\downarrowECE ↓\downarrowPICP90 (%) →\to 90%NLL ↓\downarrowMPIW90
Top-30.3976 ±\pm 0.00700.0721 ±\pm 0.003889.03 ±\pm 0.350.8357 ±\pm 0.04131.715 ±\pm 0.032
Top-40.3896 ±\pm 0.00310.0687 ±\pm 0.002590.47 ±\pm 0.090.8135 ±\pm 0.02131.771 ±\pm 0.018
Cutoff 1.100.3972 ±\pm 0.00700.0719 ±\pm 0.004589.03 ±\pm 0.230.8240 ±\pm 0.01251.716 ±\pm 0.020

Table 21. Fixed-KK and fixed-cutoff uncertainty routing. The cutoff policy selects all encoders with rm≤1.10r_m \le1.10. Entries are means and sample standard deviations across the three seed-specific averages over seven incoming datasets.

Incoming datasetSelected encoders
Torque103
Torque903
WTH6
ETTh23
ECL3
Vibration10
EMG10

Table 22. Expert selection with a fixed uncertainty cutoff. The eligible pool contains 17 encoders. The policy retains encoders with rm≤1.10r_m \le1.10.

References

  1. [1]Coen Adler, Yuxin Chang, Felix Draxler, Samar Abdi, and Padraic Smyth. Beyond accuracy: Are time series foundation models well-calibrated? In International Conference on Learning Representations, pp. 95913–95944, 2026. URL https://proceedings.iclr.cc/paper_files/paper/2026/file/9af2b1d6acf561af9c4cf70d52c7a49d-Paper-Conference.pdf.arxiv.org/abs/2510.16060
  2. [2]Abdul Fatir Ansari, Lorenzo Stella, Caner Turkmen, Xiyuan Zhang, Pedro Mercado, Huibin Shen, Oleksandr Shchur, Syama Sundar Rangapuram, Sebastian Pineda Arango, Shubham Kapoor, Jasper Zschiegner, Danielle C. Maddix, Hao Wang, Michael W. Mahoney, Kari Torkkola, Andrew Gordon Wilson, Michael Bohlke-Schneider, and Yuyang Wang. Chronos: Learning the language of time series. Transactions on Machine Learning Research, 2024. ISSN 2835-8856. URL https://openreview.net/forum?id=gernCVqT8.arxiv.org/abs/2403.07815
  3. [3]Fabio Arnez, Daniel Alfonso Montoya Vasquez, Ansgar Radermacher, and François Terrier. Latent representation entropy density for distribution shift detection. In Negar Kiyavash and Joris M. Mooij (eds.), Proceedings of the Fortieth Conference on Uncertainty in Artificial Intelligence, volume 244 of Proceedings of Machine Learning Research, pp. 110–137. PMLR, 15–19 Jul 2024. URL https://proceedings.mlr.press/v244/arnez24a.html.
  4. [4]Gilad Aviv, Jacob Goldberger, and Yoli Shavit. MoGU: Mixture-of-gaussians with uncertainty-based gating for time series forecasting. arXiv preprint arXiv:2510.07459, 2025. URL https://arxiv.org/abs/2510.07459.
  5. [5]Charles Blundell, Julien Cornebise, Koray Kavukcuoglu, and Daan Wierstra. Weight uncertainty in neural networks. In Proceedings of the 32nd International Conference on Machine Learning, volume 37 of Proceedings of Machine Learning Research, pp. 1613–1622. PMLR, 2015. URL https://proceedings.mlr.press/v37/blundell15.html.arxiv.org/abs/1505.05424
  6. [7]Rich Caruana, Alexandru Niculescu-Mizil, Geoff Crew, and Alex Ksikes. Ensemble selection from libraries of models. In Proceedings of the Twenty-First International Conference on Machine Learning, pp. 18. ACM, 2004. doi: 10.1145/1015330.1015432. URL https://doi.org/10.1145/1015330.1015432.
  7. [8]Ting Chen, Simon Kornblith, Mohammad Norouzi, and Geoffrey Hinton. A simple framework for contrastive learning of visual representations. In Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pp. 1597–1607. PMLR, 2020. URL https://proceedings.mlr.press/v119/chen20j.html.arxiv.org/abs/2002.05709
  8. [9]Yilong Chen, Junyuan Shang, Zhenyu Zhang, Yuchen Feng, Naibin Gu, Ziqi Wang, Tingwen Liu, Shuohuan Wang, Yu Sun, Hua Wu, and Haifeng Wang. Uncertainty-aware routing for principled alignment with MoE dynamics. In Maria Liakata, Viviane P. Moreira, Jiajun Zhang, and David Jurgens (eds.), Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 38865–38880, San Diego, California, United States, July 2026. Association for Computational Linguistics. doi: 10.18653/v1/2026.acl-long.1801. URL https://aclanthology.org/2026.acl-long.1801/.DOI
  9. [10]Francesco D’Angelo and Christian Henning. On out-of-distribution detection with bayesian neural networks, 2022. URL https://arxiv.org/abs/2110.06020v2. Version 2, revised February 2022; first submitted October 2021.
  10. [11]Abhimanyu Das, Weihao Kong, Rajat Sen, and Yichen Zhou. A decoder-only foundation model for time-series forecasting. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 10148–10167. PMLR, 2024. URL https://proceedings.mlr.press/v235/das24c.html.arxiv.org/abs/2310.10688
  11. [12]Rupasree Dey, Abdul Matin, Nathan Orwick, Yao Zhang, Shrideep Pallickara, and Sangmi Lee Pallickara. When to trust, how to distill: Multi-foundation model guidance for lightweight, robust scientific time series forecasting. In Proceedings of the 32nd ACM SIGKDD Conference on Knowledge Discovery and Data Mining V.2, pp. 10807–10818. Association for Computing Machinery, 2026. doi: 10.1145/3770855.3819018. URL https://doi.org/10.1145/3770855.3819018.
  12. [13]Maryam Dialameh, Hossein Rajabzadeh, Weiwei Zhang, Walid Ahmed, and Hyock Ju Kwon. Bayesian mixture of experts for large language models, 2025. URL https://arxiv.org/abs/2511.08968.
  13. [14]Michael W. Dusenberry, Ghassen Jerfel, Yeming Wen, Yi-An Ma, Jasper Snoek, Katherine Heller, Balaji Lakshminarayanan, and Dustin Tran. Efficient and scalable Bayesian neural nets with rank-1 factors. In Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pp. 2782–2792. PMLR, 2020. URL https://proceedings.mlr.press/v119/dusenberry20a.html.arxiv.org/abs/2005.07186
  14. [15]Yarin Gal and Zoubin Ghahramani. Dropout as a Bayesian approximation: Representing model uncertainty in deep learning. In Proceedings of The 33rd International Conference on Machine Learning, volume 48 of Proceedings of Machine Learning Research, pp. 1050–1059. PMLR, 2016. URL https://proceedings.mlr.press/v48/gal16.html.arxiv.org/abs/1506.02142
  15. [16]Tilmann Gneiting and Adrian E Raftery. Strictly proper scoring rules, prediction, and estimation. Journal of the American Statistical Association, 102(477):359–378, March 2007. doi: 10.1198/016214506000001437. URL https://doi.org/10.1198/016214506000001437.
  16. [17]Ary L. Goldberger, Luis A. N. Amaral, Leon Glass, Jeffrey M. Hausdorff, Plamen Ch. Ivanov, Roger G. Mark, Joseph E. Mietus, George B. Moody, Chung-Kang Peng, and H. Eugene Stanley. PhysioBank, PhysioToolkit, and PhysioNet: Components of a new research resource for complex physiologic signals. Circulation, 101(23):e215–e220, 2000. doi: 10.1161/01.CIR.101.23.e215. URL https://doi.org/10.1161/01.CIR.101.23.e215.
  17. [18]Raphael Gontijo-Lopes, Yann Dauphin, and Ekin D. Cubuk. No one representation to rule them all: Overlapping features of training methods. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=BK-4qbGgIE3.
  18. [19]Mononito Goswami, Konrad Szafer, Arjun Choudhry, Yifu Cai, Shuo Li, and Artur Dubrawski. MOMENT: A family of open time-series foundation models. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 16115–16152. PMLR, 2024. URL https://proceedings.mlr.press/v235/goswami24a.html.arxiv.org/abs/2402.03885
  19. [20]Robert A. Jacobs, Michael I. Jordan, Steven J. Nowlan, and Geoffrey E. Hinton. Adaptive mixtures of local experts. Neural Computation, 3(1):79–87, 1991. doi: 10.1162/neco.1991.3.1.79. URL https://doi.org/10.1162/neco.1991.3.1.79.
  20. [21]Diederik P. Kingma and Max Welling. Auto-encoding variational bayes. In International Conference on Learning Representations, 2014. URL https://arxiv.org/abs/1312.6114.
  21. [22]Volodymyr Kuleshov, Nathan Fenner, and Stefano Ermon. Accurate uncertainties for deep learning using calibrated regression. In Proceedings of the 35th International Conference on Machine Learning, volume 80 of Proceedings of Machine Learning Research, pp. 2796–2804. PMLR, 2018. URL https://proceedings.mlr.press/v80/kuleshov18a.html.arxiv.org/abs/1807.00263
  22. [23]Balaji Lakshminarayanan, Alexander Pritz el, and Charles Blundell. Simple and scalable predictive uncertainty estimation using deep ensembles. In Advances in Neural Information Processing Systems, volume 30. Curran Associates, Inc., 2017. URL https://proceedings.neurips.cc/paper_files/paper/2017/file/9ef2ed4b7fd2c810847ffa5fa85bce38-Paper.pdf.arxiv.org/abs/1612.01474
  23. [24]Jason D. Lee, Qi Lei, Nikunj Saunshi, and JIACHENG ZHUO. Predicting what you already know helps: Provable self-supervised learning. In M. Ranzato, A. Beygelzimer, Y. Dauphin, P.S. Liang, and J. Wortman Vaughan (eds.), Advances in Neural Information Processing Systems, volume 34, pp. 309–323. Curran Associates, Inc., 2021. URL https://proceedings.neurips.cc/paper_files/paper/2021/file/02e656adee09f8394b402d9958389b7d-Paper.pdf.arxiv.org/abs/2008.01064
  24. [25]Albus Li and Matthew Robert Wicker. Variational routing: A scalable Bayesian framework for calibrated mixture-of-experts transformers. In Proceedings of the 43rd International Conference on Machine Learning, volume 306 of Proceedings of Machine Learning Research, pp. 71187–71215. PMLR, 2026. URL https://proceedings.mlr.press/v306/li26hj.html.arxiv.org/abs/2603.09453
  25. [26]Xu Liu, Juncheng Liu, Gerald Woo, Taha Aksu, Yuxuan Liang, Roger Zimmermann, Chenghao Liu, Junnan Li, Silvio Savarese, Caiming Xiong, and Doyen Sahoo. Moirai-MoE: Empowering time series foundation models with sparse mixture of experts. In Proceedings of the 42nd International Conference on Machine Learning, volume 267 of Proceedings of Machine Learning Research, pp. 38940–38962. PMLR, 2025. URL https://proceedings.mlr.press/v267/liu25an.html.arxiv.org/abs/2410.10469
  26. [27]Wesley J. Maddox, Timur Garipov, Pavel Izmailov, Dmitry Vetrov, and Andrew Gordon Wilson. A simple baseline for Bayesian uncertainty in deep learning. In Advances in Neural Information Processing Systems, volume 32. Curran Associates, Inc., 2019. URL https://proceedings.neurips.cc/paper_files/paper/2019/file/118921efba23fc329e6560b27861f0c2-Paper.pdf.arxiv.org/abs/1902.02476
  27. [29]Cuong V. Nguyen, Tal Hassner, Matthias Seeger, and Cedric Archambeau. LEEP: A new measure to evaluate transferability of learned representations. In Proceedings of the 37th International Conference on Machine Learning, volume 119 of Proceedings of Machine Learning Research, pp. 7294–7305. PMLR, 2020. URL https://proceedings.mlr.press/v119/nguyen20b.html.arxiv.org/abs/2002.12462
  28. [30]Oleksiy Ostapenko, Zhan Su, Edoardo Ponti, Laurent Charlin, Nicolas Le Roux, Lucas Caccia, and Alessandro Sordoni. Towards modular LLMs by building and reusing a library of LoRAs. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 38885–38904. PMLR, 2024. URL https://proceedings.mlr.press/v235/ostapenko24a.html.arxiv.org/abs/2405.11157
  29. [31]Jonas Pfeiffer, Aishwarya Kamath, Andreas Rücklé, Kyunghyun Cho, and Iryna Gurevych. Adapter-Fusion: Non-destructive task composition for transfer learning. In Proceedings of the 16th Conference of the European Chapter of the Association for Computational Linguistics: Main Volume, pp. 487–503. Association for Computational Linguistics, 2021. doi: 10.18653/v1/2021.eacl-main.39. URL https://aclanthology.org/2021.eacl-main.39/.DOI
  30. [32]Jonas Pfeiffer, Sebastian Ruder, Ivan Vulić, and Edoardo Maria Ponti. Modular deep learning. Transactions on Machine Learning Research, 2023. URL https://openreview.net/forum?id=z9EkXfvxta.arxiv.org/abs/2302.11529
  31. [33]Quang Hung Pham, Ryad Zemouri, Martin Gagnon, and Luc Vouigny. Modular foundation models for time-series perception in digital twins, 2026. URL https://arxiv.org/abs/2607.03585.
  32. [34]Jorge Reyes-Ortiz, Davide Anguita, Alessandro Ghio, Luca Oneto, and Xavier Parra. Human activity recognition using smartphones. UCI Machine Learning Repository, 2013. URL https://doi.org/10.24432/C54S4K.
  33. [35]Nicholas Roberts, Samuel Guo, Zhiqi Gao, Satya Sai Srinath Namburi GNVV, Sonia Cromp, Chengjun Wu, Chengyu Duan, and Frederic Sala. Pretrained hybrids with MAD skills. In Conference on Language Modeling, 2025. URL https://openreview.net/forum?id=8xSbwT3763.arxiv.org/abs/2406.00894
  34. [36]Tom Saliencro, Rohan Desai, Priya Nair, Maya Lindqvist, and Daniel Whitmore. Uncertainty is not enough: Value-of-information routing for mixtures of LoRA experts, 2026. URL https://arxiv.org/abs/2608.02528.
  35. [37]Maximilian Seitzer, Arash Tavakoli, Dimitrije Antic, and Georg Martius. On the pitfalls of heteroscedastic uncertainty estimation with probabilistic neural networks. In International Conference on Learning Representations, 2022. URL https://openreview.net/forum?id=aP0pXlnV1T.arxiv.org/abs/2203.09168
  36. [38]Xiaoming Shi, Shiyu Wang, Yuqi Nie, Dianqi Li, Zhou Ye, Qingsong Wen, and Ming Jin. Time-MoE: Billion-scale time series foundation models with mixture of experts. In International Conference on Learning Representations, pp. 34635–34667, 2025. URL https://proceedings.iclr.cc/paper_files/paper/2025/file/558d48c1f08675daa636e09bfe94a89e-Paper-Conference.pdf.
  37. [39]Ibne Farabi Shihab, Sanjeda Akter, and Anuj Sharma. Grassmannian mixture-of-experts: Concentration-controlled routing on subspace manifolds, 2026. URL https://arxiv.org/abs/2602.17798.
  38. [40]Md Asif Bin Syed, Md Younus Ahamed, and Azmine Toushik Wasi. Position: Time-series foundation models require explicit domain-level benchmarks. In Tong Zhang, Miroslav Dudik, Martin Jaggi, Alekh Agarwal, Sharon Li, Dale Schuurmans, Jerry Zhu, Felix Berkenkamp, Hanze Dong, and Alberto Bietti (eds.), Proceedings of the 43rd International Conference on Machine Learning, volume 306 of Proceedings of Machine Learning Research, pp. 172221–172247. PMLR, 06–11 Jul 2026. URL https://proceedings.mlr.press/v306/syed26a.html.
  39. [41]Marcin Tomczak, Siddharth Swaroop, and Richard Turner. Efficient low rank Gaussian variational inference for neural networks. In Advances in Neural Information Processing Systems, volume 33, pp. 4610–4622. Curran Associates, Inc., 2020. URL https://proceedings.neurips.cc/paper_files/paper/2020/file/310cc7ca5a76a446f85c1a0d641ba96d-Paper.pdf.
  40. [42]Mame Diarra Toure and David A. Stephens. Singular Bayesian neural networks. In Proceedings of the 43rd International Conference on Machine Learning, volume 306 of Proceedings of Machine Learning Research, pp. 121841–121911. PMLR, 2026. URL https://proceedings.mlr.press/v306/toure26a.html.arxiv.org/abs/2602.00387
  41. [43]Artur Trindade. ElectricityLoadDiagrams20112014. UCI Machine Learning Repository, 2015. URL https://doi.org/10.24432/C58C86.
  42. [44]Pascal Vincent, Hugo Larochelle, Yoshua Bengio, and Pierre-Antoine Manzagol. Extracting and composing robust features with denoising autoencoders. In Proceedings of the 25th International Conference on Machine Learning, pp. 1096–1103. ACM, 2008. doi: 10.1145/1390156.1390294. URL https://doi.org/10.1145/1390156.1390294.
  43. [45]Yibin Wang, Haizhou Shi, Ligong Han, Dimitris Metaxas, and Hao Wang. BLoB: Bayesian low-rank adaptation by backpropagation for large language models. In Advances in Neural Information Processing Systems, volume 37, pp. 67758–67794. Curran Associates, Inc., 2024. doi: 10.52202/079017-2164. URL https://proceedings.neurips.cc/paper_files/paper/2024/file/7d53575463291ea65ba23cf6e571159b-Paper-Conference.pdf.DOI
  44. [46]Theresa Wasserer, Maximilian Fleissner, and Debarghya Ghoshdastidar. Pac-bayesian analysis of the surrogate relation between joint embedding and supervised downstream losses. In Matus Telgarsky and Jonathan Ullman (eds.), Proceedings of the 37th International Conference on Algorithmic Learning Theory, volume 313 of Proceedings of Machine Learning Research, pp. 1–33. PMLR, 23–26 Feb 2026. URL https://proceedings.mlr.press/v313/wasserer26a.html.
  45. [47]Gerald Woo, Chenghao Liu, Akshat Kumar, Caiming Xiong, Silvio Savarese, and Doyen Sahoo. Unified training of universal time series forecasting transformers. In Proceedings of the 41st International Conference on Machine Learning, volume 235 of Proceedings of Machine Learning Research, pp. 53140–53164. PMLR, 2024. URL https://proceedings.mlr.press/v235/woo24a.html.arxiv.org/abs/2402.02592
  46. [48]Danny Wood, Tingting Mu, Andrew M. Webb, Henry W. J. Reeve, Mikel Luján, and Gavin Brown. A unified theory of diversity in ensemble learning. Journal of Machine Learning Research, 24(359):1–49, 2023. URL https://jmlr.org/papers/v24/23-0041.html.arxiv.org/abs/2301.03962
  47. [49]Rongyu Zhang, Yulin Luo, Jiaming Liu, Huanrui Yang, Zhen Dong, Denis Gudovskiy, Tomoyuki Okuno, Yohei Nakata, Kurt Keutzer, Yuan Du, and Shanghang Zhang. Efficient deweather mixture-of-experts with uncertainty-aware feature-wise linear modulation. Proceedings of the AAAI Conference on Artificial Intelligence, 38(15):16812–16820, March 2024. ISSN 2159-5399. doi: 10.1609/aaai.v38i15.29622. URL https://ojs.aaai.org/index.php/AAAI/article/view/29622.DOI
  48. [50]Haoyi Zhou, Shanghang Zhang, Jieqi Peng, Shuai Zhang, Jianxin Li, Hui Xiong, and Wancai Zhang. Informer: Beyond efficient transformer for long sequence time-series forecasting. In The Thirty-Fifth AAAI Conference on Artificial Intelligence, AAAI 2021, Virtual Conference, volume 35, pp. 11106–11115. AAAI Press, 2021.arxiv.org/abs/2012.07436

Paper details

vlnc:W_asvkxsuq
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 1

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Mathematics

Not recorded for this paper

View statement
Looking for related papers…
Contents