CATF: A Manager-Worker Framework for Context-Aware Time-Series Forecasting

Introduction

Time-series forecasting is critical to applications such as weather prediction, electricity load management, traffic analysis, financial forecasting, and industrial monitoring [3, 5, 44, 58, 62]. However, growing temporal complexity and rich contextual dependencies make accurate forecasting increasingly difficult and computationally expensive. Contextual variables are often heterogeneous and dynamic, evolving abruptly or nonlinearly over time. For example, hourly bike-sharing demand depends on weather, temperature, and the calendar: on working days it peaks during the morning and evening commutes, whereas on weekends and holidays it follows a single midday peak, and a holiday on a weekday can switch the demand to the weekend pattern. In this work, we focus on such context-aware forecasting: the goal is to forecast a single target variable from its history and the contextual variables that drive it, a setting also known as forecasting with exogenous variables. This differs from the setting of most multivariate forecasting benchmarks [1], in which all variables are forecast jointly; here, the contextual variables serve only as inputs.

Recent advances in deep learning have led to impressive results in time-series forecasting. Convolutional networks [60], graph neural networks [30], and Transformer-based models [42] have been adapted to capture complex temporal dependencies. More recently, large-scale time-series foundation models (TSFMs) [32, 48] have shown strong performance in zero-shot and few-shot settings. However, these models are expensive to fine-tune, struggle with distribution shifts, and often fail to generalize to domain-specific dynamics. To improve adaptability, researchers have explored alternatives such as retrieval-augmented forecasting [40], meta learning [46], and adaptive policies [22].

Hybrid forecasting approaches offer another promising direction by allowing model specialization under diverse conditions. Ensemble methods [23, 26] train specialized models on partitioned subsets and aggregate their predictions. However, they incur inference overhead and require

Authors’ Contact Information: Yueqiao Chen, UC Davis, Davis, CA, USA, [email protected]; Dongyu Liu, UC Davis, Davis, CA, USA, [email protected]. manual segmentation and weighting strategies. Mixture-of-Experts (MoE) models [33, 54] improve efficiency through sparse activation but demand careful gating design and are prone to expert under-utilization without strong regularization.

To address these challenges, we propose CATF, a _Context-Aware Time-Series Forecasting_ framework based on a Manager-Worker (MW) architecture. CATF consists of a high-level transformer-based _Manager_ and a set of _Worker_ models trained via joint optimization. The Manager dynamically routes each input to a single specialized Worker based on contextual features, enabling each Worker to focus on a distinct subset of conditions. Inspired by economic specialization [29], we formulate training as a competitive yet cooperative process: the Manager learns to assign inputs to the most suitable Worker, while Workers specialize in minimizing error within their assigned contexts.

Unlike conventional MoEs, CATF employs _hard routing_, assigning each input to a single Worker during both training and inference. This improves interpretability, encourages Worker specialization, and eliminates the need for manual partitioning or complex gating mechanisms. In addition, CATF supports heterogeneous architectures, enabling seamless integration of state-of-the-art forecasting models with minimal tuning. We evaluated CATF on five real-world context-rich datasets across four forecasting horizons, observed improvements over strong single-model baselines, and analyze its routing decisions. Our key contributions are as follows.

• Modular framework. We propose CATF, a Manager-Worker framework in which a Transformer-based Manager routes each input to one of several Worker models through hard routing, trained with a joint mechanism that lets the Workers specialize without manual data partitioning (§ 4). CATF is model-agnostic: any single-model forecaster can serve as a Worker without modification, in homogeneous or heterogeneous configurations, and we instantiate it with five representative architectures, Autoformer [61], Informer [67], FEDformer [68], TimesNet [60], and iTransformer [42].

• Context-aware interpretation. Because each prediction is produced by a single, identifiable Worker, the routing decisions can be inspected and related to the context. In a case study on a real-world bike-sharing dataset, we show that the Manager learns to route inputs by the most relevant contextual variables (e.g., working days vs. weekends) and that the Workers specialize accordingly (§ 6); on synthetic data with known context-driven regimes, the same analysis recovers the ground truth (Appendix § 12.§ 2).

• Performance improvement. On five context-rich real-world datasets, four horizons, and two normalization schemes, CATF improves over its standalone counterpart in 162 of the 200 comparisons (81%), the best CATF-enhanced model outperforms the best standalone baseline in 19 of the 20 settings, CATF reduces the run-to-run variance of unstable baselines, and it outperforms the zero-shot Time-MoE foundation model [54] in 15 of the 20 settings (§ 5.2).

Related Work

Time Series Forecasting

Single-Model Approaches. Early forecasting models relied on convolutional (CNN) [8, 19, 41, 60] and recurrent (RNN) [50, 53, 66] architectures to capture temporal dependencies. CNNs are efficient and effective for local patterns, while RNNs model autoregressive behavior. However, both struggle with long-range dependencies. Graph neural networks (GNNs) [30, 63] address inter-variable relationships but incur high computational cost in high-dimensional or long-sequence settings. More recently, multilayer perceptron (MLP) models [37, 57] have emerged as efficient alternatives, relying on token mixing or linear projections. Transformer-based models [42, 45, 61, 67, 68] have become more popular due to their ability to model global dependencies through self-attention. Despite their strengths, single-model architectures often struggle to generalize across heterogeneous and dynamic temporal regimes. Models tuned for specific conditions may underperform under unexpected shifts such as holidays or extreme events. Our proposed framework addresses this limitation by enabling dynamic model selection based on contextual cues.

Time-Series Foundation Models (TSFMs). These approaches aim to build universal forecasting models pretrained on large-scale datasets [32, 38]. These models vary in architecture (e.g., Transformer, diffusion, or MLP), pretraining objectives (supervised or self-supervised), and downstream adaptation strategies (fine-tuning, prompting, zero-shot) [4, 9, 17, 18, 24, 25, 43]. While TSFMs offer general-purpose representations and cross-domain transferability, they are computationally expensive and often brittle under distribution shifts, particularly in domains with limited or highly specialized data.

Hybrid Forecasting Architectures

Hybrid frameworks combine multiple models to leverage their complementary strengths.

Ensemble learning. Ensemble learning methods—including bagging, boosting, stacking, and hybrid approaches—consistently improve prediction accuracy in time series forecasting by reducing variance, mitigating bias, and modeling complex non-linear dependencies [13, 21, 23, 26, 34, 52]. These methods combine predictions from multiple independent models, typically through averaging or weighting, which enhances robustness but often overlooks input-specific context. Their effectiveness can be constrained by the need for manual design of segmentation and aggregation strategies, and performance gains tend to diminish when base models are highly correlated. Moreover, ensemble methods are computationally more demanding and less interpretable compared to individual models. Achieving superior performance over the best single model requires careful tuning of integration strategies, such as model weighting and diversity enforcement.

Mixture-of-Experts (MoE). MoE architectures [15, 31, 33, 35, 36, 51, 54] dynamically route inputs to specialized expert models via a gating mechanism. While sparse activation improves computational efficiency, these models require careful gating design and are prone to issues such as expert collapse or under-utilization without strong regularization. In time-series forecasting, recent MoE variants like Time-MOE [54] have shown promising results. Time-MOE is a decoder-only autoregressive Transformer trained on over 300 billion time points. It employs point-wise tokenization, multi-resolution attention heads, and top-$k$ expert activation with load-balancing to achieve scalability with minimal inference overhead. However, such models remain challenging to optimize and are often tightly coupled to specific architectural choices, limiting their flexibility across forecasting domains.

Our framework builds on the core principle of specialization, but replaces soft gating with hard routing implemented through a learned Transformer-based Manager. This design eliminates the instability commonly associated with gating mechanisms and enables transparent, deterministic input-to-expert assignments. Unlike prior MoE approaches, our framework naturally supports heterogeneous Workers and allows seamless integration of SOTA forecasting models. As a result, it achieves both scalability and interpretability—while also mitigating the challenges faced by ensemble methods, such as manual data partitioning and extensive tuning.

Preliminaries

Data Description

This study focuses on context-aware multivariate time-series forecasting: the model observes the history of a target variable together with multiple contextual variables, and predicts the future values of the target variable.

Target Time-Series Data. Let $x_t \in\mathbb{R}$ denote the value of the target variable at time $t$ (e.g., the PM2.5 concentration or the number of rented bikes). Its past observations up to time $t$ are

where $\Delta t$ is the time interval between consecutive observations and $L$ is the length of the historical (lookback) window.

Multivariate Contextual Data. In addition to the target, we observe $N$ contextual variables that capture external factors influencing the target dynamics, such as weather conditions, co-pollutants, or calendar indicators. Let $\mathbf{c}_t \in\mathbb{R}^{N}$ denote the contextual observations at time $t$; the contextual history over the same window is

The target and contextual variables are temporally aligned, so the model input at each time step is the $(N+1)$-dimensional vector $(x_t, \mathbf{c}_t)$, and the full input is an $L \times(N+1)$ matrix.

Problem Formulation

Given the historical target sequence $X = X_{t,\Delta t,L}$ and the associated contextual sequence $C = C_{t,\Delta t,L}$, the objective is to forecast the future values of the target variable over a horizon of length $L_y$:

A forecasting model $f$ learns the mapping

that is, a multivariate-input, univariate-output mapping. The contextual variables are only used as inputs; their future values are neither predicted nor assumed to be known.

Normalization and Evaluation. Because the target and contextual variables have very different scales, the inputs are normalized by a transform $g$ before being fed to the model, and the model is trained to minimize the MSE between its output and $g(Y)$ in the normalized space. We consider two choices of $g$: global normalization, which standardizes every variable with the mean and standard deviation of the training split, and sample normalization, which standardizes each input window with its own per-variable statistics and scales the target horizon with the statistics of the target in the input window. At test time, predictions are mapped back to the original scale with the inverse transform, and the error is computed against the raw ground truth:

where $\mathcal{T}$ is the set of forecast origins in the test split. Reporting the error in the original units of the target makes the results directly interpretable (e.g., squared rentals per hour for Bike Share) and comparable across normalization schemes, but not across datasets.

While recent forecasting models have achieved strong performance by modeling complex temporal dependencies, their effectiveness often degrades in the presence of heterogeneous and context-dependent patterns. Different temporal regimes may exhibit distinct dynamics, making it difficult for a single model to capture all forecasting behaviors effectively. This motivates model specialization, where different forecasting models focus on different subsets of the data distribution.

To address this challenge, the proposed CATF framework adopts a Manager–Worker architecture. A manager model first extracts contextual characteristics from the target history $X$ and the context $C$ and dynamically assigns each instance to one of several specialized worker models. By routing each instance to the most suitable worker, CATF enables adaptive specialization without requiring manual data partitioning, thereby improving forecasting accuracy across diverse temporal patterns, while the routing decisions reveal which patterns each worker specializes in.

CATF Framework

This section presents the technical details of CATF framework, a context-aware time-series forecasting approach based on a Manager–Worker (MW) architecture. We describe its model structure and training algorithm.

Modeling

CATF consists of a manager model $\mathcal{M}$ and $K$ worker models $\mathcal{W}_1,\ldots,\mathcal{W}_K$, as illustrated in Figure 1a. The manager $\mathcal{M}$ takes the target history $X$ and the context history $C$ as input and estimates, for each worker $\mathcal{W}_i$, the probability $\hat{P}_i$ that it is best suited to predict the future target sequence. The worker with the highest probability, $\mathcal{W}_{k^*}$ with $k^*=\arg\max_i\hat{P}_i$, is selected and generates the forecast $\hat{Y}=\mathcal{W}_{k^*}(X,C)$ from the same input; at inference time, the other workers are not executed.

Manager Model

Manager Model. The manager performs a $K$-class classification task, mapping input $(X,C)$ to a probability distribution over the $K$ worker models. As shown in Figure 1b, the manager follows a Transformer-based architecture [56], consisting of:

• Feature Encoding (L1): A linear projection followed by layer normalization, ReLU activation, and dropout transforms raw input into high-dimensional embeddings. This preprocessing ensures stable training and expressive representations, which are essential for downstream temporal modeling.

• Transformer Encoder Stack (L2): A stack of $L_m$ Transformer encoder layers models long-range and complex temporal-contextual dependencies through self-attention mechanisms. Each layer consists of multi-head self-attention and position-wise feedforward sub-layers, equipped

with GELU activation [27] and residual connections. GELU is chosen for its smooth, non-linear behavior, which preserves gradient flow and has been empirically shown to enhance optimization stability and performance in deep Transformer-based networks.

• Output Layer (L3–L4): Following Transformer-based classification designs [14, 65], last-timestep pooling summarizes the sequence into a fixed-length representation. This is passed through a decoder (L3) and softmax (L4) to produce the output distribution over workers. Last-timestep pooling preserves the most recent temporal information, which is critical for decision-making in time-sensitive forecasting scenarios.

4.1.2 Worker Model. The selected worker predicts the future target sequence $\hat{Y}$ given the input sequence $X$ and covariates $C$, as illustrated in Figure 1. CATF allows each worker to be instantiated with any time-series forecasting model, supporting both homogeneous and heterogeneous ensembles. In most experiments, we use multiple instances of the same base model to assess the effectiveness of the framework, while heterogeneous mixtures are examined in ablation studies.

We incorporate several established models as workers, including Informer [67], Autoformer [61], FedFormer [68], TimesNet [60], and i-Transformer [42]. These models also serve as baselines for evaluation. Their base implementations are consistent with the official versions provided in the Time-Series-Library [1], and we use the same hyper-parameter settings to ensure fair comparisons. For each base model, we implement a corresponding CATF variant within our framework. Users can specify the base models to use—either homogeneous or mixed—along with the number of workers of each type, through a simple configuration file.

Training Mechanism

We introduce an iterative training algorithm to jointly train the manager and workers, shown as Figure 2 and Algorithm 1. The manager learns to assign each input to the most suitable worker, and the workers are trained based on the data routed from the manager, specializing in certain context conditions.

Main Training Process

4.2.1 Main Training Process. Each training iteration consists of three phases (Figure 2). (a) Every worker $W_i$ forecasts the target, $\hat{Y}_i = W_i(X,C)$ (step ①; line 4 in Algorithm 1), and we compute its loss $L_i^W$ (step ②). This forward pass only evaluates the workers and does not update them. (b) The worker losses, together with the training volumes of the workers, define a pseudo ground truth $P$ for the manager output (step ③; line 5), which supervises the update of the manager $\mathcal{M}$ (step ④; line 6). (c) The updated manager predicts the routing probabilities $\hat{P} = \mathcal{M}(X,C)$ and selects the worker $W_{k^*}$ with $k^* = \operatorname*{arg\,max}_i \hat{P}_i$ (step ⑤; lines 7–8). Only this worker is trained on the sample (step ⑥; lines 9–13), while the other workers are not updated.

This selective training strategy ensures that each worker is exposed to and learns from samples within specific contextual regimes. Over time, this encourages the emergence of functional specialization—each worker becomes proficient in forecasting under particular conditions. § 6 analyzes this specialization empirically. This approach is inspired by the principle of economic specialization [29], where agents maximize collective productivity by focusing on tasks they are best suited to. Similarly, CATF enables each worker to “do what it does best,” thereby improving overall forecasting accuracy and robustness in heterogeneous environments.

4.2.2 Training Details. The following details describe key components and design choices of the training process described above.

Detail #1: Training Order. We train the manager model first and then use it to select the appropriate worker for training. In the initial iteration, untrained workers make predictions on a set of samples, and their losses are used to generate a pseudo ground truth distribution for the manager.

This allows the manager to learn how to assign tasks based on relative worker performance from the outset. Once trained, the manager routes future samples to the most suitable worker.

This process can be thought of as talent scouting before assigning roles: the manager first observes each worker’s raw potential under various contexts—without prior bias—and learns who is naturally inclined to handle which kind of data. In contrast, if we train the workers first, the untrained manager must randomly assign tasks, akin to blindly assigning roles before knowing who’s good at what. The randomly selected worker would then receive more training and, as a result, dominate in performance—not because of superior aptitude, but because of an early, unearned advantage. This skews the pseudo ground truth, leading the manager to reinforce this imbalance and reducing overall diversity and specialization. Alternatively, training all workers on every sample from the beginning contradicts the core design of specialization, where workers are meant to focus on distinct contextual regimes.

By training the manager first—using the raw, untrained losses as an initial signal—we provide it with a fair and unbiased starting point for assessing each worker’s relative strengths. While this is only the beginning of an iterative co-training process, it lays the foundation for a more principled and effective division of labor. This approach encourages the emergence of specialized expertise across workers, without introducing early random bias that could skew the learning dynamics.

Detail #2: Pseudo Ground Truth Calculation. A core challenge in training the manager is the absence of a ground-truth probability distribution $P$ that indicates which worker should handle each sample. To address this, we adopt the intuitive principle that a worker with lower prediction loss should be more likely to be selected. Concretely, each worker first makes a prediction for the given input (step ① in Figure 2), and we compute its prediction loss $L_i^W$ (e.g., $MSE_i$) (step ②). These losses are then used to construct a soft target distribution over workers. The simplest form of this pseudo ground truth is defined as:

Algorithm 1 Iterative Training Process of CATF

Require: Training data $\mathcal{D} = \{(X_i, C_i, Y_i)\}$, Manager $\mathcal{M}$, Workers $\{\mathcal{W}_1, \ldots, \mathcal{W}_K\}$, epochs $E$, worker update steps $\alpha$

1. Initialize $\mathcal{M}$ and $\{\mathcal{W}_i\}_{i=1}^{K}$

2. for epoch $= 1$ to $E$ do

3.     for each mini-batch $(X, C, Y) \in\mathcal{D}$ do

4.     Compute all worker losses $L_i^{W} = \mathcal{L}(\mathcal{W}_i(X,C),Y)$

5.     Compute target weights $P$

6.     Train manager $\mathcal{M}$

7.     $\hat{P} = \mathcal{M}(X,C)$ # predict routing probabilities

8.     Select worker index $k^* = \arg\max\hat{P}$ for each sample

9.     for step $= 1$ to $\alpha$ do

10.         for $i = 1$ to $K$ do

11.         Select sub-batch $\{(x_j,y_j)\}$ where $k_j^* = i$

12.         if selection not empty then

13.         Train Worker $\mathcal{W}_i$ on selected sub-batch

14.     Validate and save best checkpoint if improved

where lower losses result in higher scores. This distribution $P$ serves as a stand-in for the missing ground truth, guiding the training of the manager to produce probability outputs that align with worker performance (step ④ in Figure 2).

Detail #3: Training Regularization. A critical challenge in training CATF is maintaining a balanced learning process across all workers. Without proper regulation, a competitive imbalance may emerge: if one worker $\mathcal{W}_i$ happens to perform better than others—even if not inherently more capable—the manager will increasingly assign it more samples in subsequent iterations. This feedback loop can result in $\mathcal{W}_i$ becoming overly dominant, while other workers are starved of data and fail to specialize. Ultimately, this degrades model diversity and undermines the utility of contextual information. This phenomenon resembles the classic exploration–exploitation dilemma in Monte-Carlo Tree Search (MCTS) [11], where the challenge lies in choosing between high-performing nodes and less-explored ones. Inspired by MCTS, we introduce a regularization term (step ③ in Figure 2, line 5 in Algorithm 1, Equation (3)) that considers the training volume $V_i$—the number of data samples allocated to each worker $\mathcal{W}_i$ (line 13):

where $e^{-L^W/MaxL}$ represents the new loss-based term (TNewLoss), and $(MaxV - V)/MaxV$ is the regularization term (TReg). This formulation differs from the original loss-based probability in Equation (2), where $MaxL/L^W$ (TOriginLoss) may produce a much wider range and overshadow the contribution of the regularization term.

We design TReg to be large during early training—encouraging the manager to explore underutilized workers—and to diminish over time as training volumes equalize. This dynamic is supported by the fact that $MaxV$ generally grows faster than $MaxV - V$, naturally reducing TReg’s effect. The hyperparameter $\beta$ controls the strength of the regularization.

TNewLoss is preferred over TOriginLoss because its output range ($1/e$ to $1$) aligns better with TReg (which ranges from 0 to 1). This ensures that both terms meaningfully influence the resulting probability distribution $\hat{P}$ and that the manager balances performance with exploration throughout training.

Detail #4: Batch Training. For clarity, Figure 2 depicts the training process on a single sample. However, our implementation is designed to support efficient batch-wise training. Specifically, in each training iteration (Algorithm 1, lines 3–13), we first evaluate all workers and update the manager on the current mini-batch (lines 4–6), and then compute the updated manager’s output probabilities for all samples (line 7). Each sample is then assigned to the worker with the highest predicted probability via arg max selection (line 8). Next, we partition the batch into $K$ sub-batches, one for each worker, based on the manager’s assignments. Each worker $W_i$ is then trained only on its corresponding sub-batch—i.e., the samples for which it was selected—allowing it to focus on the specific types of inputs it is most suited for.

To improve training stability and accelerate convergence, we repeat the worker update step $\alpha$ times for each batch (lines 9–13 in Algorithm 1).

Detail #5: Loss Function Selection. For the workers, we adopt a standard loss metric for time-series prediction: the Mean Squared Error (MSE). This loss quantifies the average squared difference between the predicted target sequence $\hat{Y}$ and the ground truth $Y$, and is defined as:

We denote the MSE loss of worker $W_i$ as $L_i^W$, and use it as the primary feedback signal during worker training.

For the manager, we use a hybrid objective that combines the Wasserstein distance [6] and the Kullback–Leibler (KL) divergence to measure the discrepancy between the manager’s predicted probability distribution $\hat{P}$ and the pseudo ground truth $P$.

The Wasserstein distance is more continuous and geometry-aware than KL or Jensen–Shannon divergence [7], offering smoother gradients and greater stability during optimization. It is defined as:

where $f(x)$ is a 1-Lipschitz function satisfying $\lvert f(x_1)-f(x_2)\rvert\leq\lvert x_1-x_2\rvert$. This constraint ensures smooth alignment between probability distributions.

While Wasserstein distance enables fine-grained distribution matching, using it alone often leads to over-concentration of samples on a single dominant worker, which undermines diversity. To address this, we incorporate the KL divergence:

KL divergence penalizes mismatches where high-probability targets receive low attention, helping the manager distribute samples more evenly—particularly important during the early stages of training when $P$ is relatively uniform.

By combining both terms, we enable the manager to balance diversity and precision throughout training. The KL term encourages broad worker participation, while the Wasserstein term refines the accuracy of the assignment. The resulting hybrid loss for the manager is as Equation (7):

, where $\theta_j$ denotes the $j$-th parameter tensor (the weight matrix or bias vector of a layer) of the manager model. The KL divergence is much larger in magnitude than the Wasserstein distance. We therefore set $\lambda_{\mathrm{wass}} = 10$ and $\lambda_{\mathrm{KL}} = 0.01$, which bring the two terms to a comparable scale so that neither dominates the manager update, and $\lambda_{\mathrm{reg}} = 1 \times10^{-8}$. The last term is a group-lasso penalty [64] on the manager’s parameters, where $\theta_j$ denotes the $j$-th parameter tensor and $\lambda_{\mathrm{reg}} = 10^{-8}$. In our ablation, removing this term has no significant effect (§ 5.3).

Experiments

We implemented CATF and evaluated its performance through both comparative experiments and ablation studies. All models were trained using PyTorch [47] with CUDA acceleration on a server equipped with an Nvidia RTX 6000 Ada GPU (48 GB). To ensure fair comparison, each experiment was repeated across 5 independent runs per model, dataset, forecasting horizon, and normalization mode. The code is available at https://github.com/via-cs/catf.

Setup and Baselines

We compare CATF with five widely used multivariate time-series (MTS) forecasting models and a pretrained time-series foundation model on five real-world datasets, each evaluated at four forecasting horizons and under two normalization schemes. The experimental setup and baseline details are outlined below.

Datasets

We evaluate on five datasets in which a single target variable is driven by contextual variables of a different nature, such as weather and calendar variables for bike demand. We do not use standard multivariate benchmarks such as ETT or Electricity: they are designed for joint forecasting of variables of the same kind (e.g., the loads of different clients) and contain no such contextual variables. For each dataset, we chronologically partition the samples into 80% for training, 10% for validation, and 10% for testing. The input length (lookback window) is fixed to $L = 96$, and the output length is $L_y \in\{96, 192, 336, 720\}$, following the long-term forecasting protocol of prior work [42, 60, 61]. Dataset statistics are listed in Appendix Table 4.

• Weather [59] (21 variables): Meteorological measurements recorded every 10 minutes during 2020 at a weather station in Germany, including pressure, temperature, humidity, wind, and radiation. Following our code, the last column, CO$_2$ concentration (ppm), is used as the target and the other 20 variables as context. As in the original data, the target contains 50 missing-value codes (−9999) in the training period, which we keep unchanged.

• PM2.5-LA and PM2.5-RV [28, 55] (15 variables each): Hourly PM2.5 concentrations from 2022 to 2025 at two air-quality monitoring sites in Southern California (Los Angeles and Riverside). The context consists of on-site meteorology (wind speed, temperature, relative humidity, solar and UV radiation, barometric pressure), ERA5 reanalysis variables (surface pressure, skin temperature, boundary-layer height, total precipitation), and co-pollutants (CO, SO$_2$, NO$_2$, O$_3$).

• Bike Share [20] (12 variables): Hourly counts of rented bikes in the Capital Bikeshare system (Washington, D.C.) in 2011–2012. The context consists of calendar indicators (season, year, month, holiday, weekday, working day) and weather conditions (weather situation, temperature, feels-like temperature, humidity, wind speed).

• Appliances [12] (26 variables): Energy use of appliances in a low-energy house, recorded every 10 minutes from January to May 2016. The context consists of lighting energy, temperature and humidity in nine rooms, and outdoor weather from a nearby station.

Baselines

We evaluate CATF against a diverse set of baseline models, including both widely adopted MTS forecasting models and a recent time-series foundation model. This allows us to assess CATF’s effectiveness across a broad range of model designs and forecasting settings.

Worker Models and Their CATF-Enhanced Variants. CATF is a flexible framework that allows users to freely select the architecture of each worker model. In our experiments, we incorporate five established forecasting models as workers: four Transformer-based models—Informer [67], Autoformer [61], FedFormer [68], and i-Transformer [42]—and one CNN-based model, TimesNet [60]. These models are chosen for their strong empirical performance and computational efficiency across a wide range of multivariate time-series forecasting tasks.

Each of these baseline models can be easily integrated as a worker within the CATF framework with minimum efforts. CATF supports both homogeneous (identical) and heterogeneous (mixed) worker configurations. In our main experiments, we instantiate CATF with three identical baseline models as workers (i.e., number of workers = 3). These variants are denoted as CATF-<Baseline> (such as, CATF-TimesNet, CATF-iTransformer).

Time-Series Foundation Model. We additionally compare against Time-MoE [54], a large-scale time-series foundation model based on a sparse Mixture-of-Experts (MoE) architecture that routes each token to a subset of experts through a learned gating network. Time-MoE is a natural reference point for CATF, as both rely on routing inputs to specialized sub-models, but Time-MoE uses soft token-level routing inside a single pretrained network whereas CATF uses hard sample-level routing across independently trained workers. We run the publicly released Time-MoE checkpoint (Time-MoE-50M) in a zero-shot setting on exactly the same test windows, horizons, and normalization schemes as all other models, so that its results are directly comparable.

5.1.3 Hyperparameters. All baselines use the default hyperparameters of their official implementations in the Time-Series Library [1] (Appendix Table 5). Each worker in CATF uses exactly the same configuration as the corresponding baseline, so that differences in accuracy reflect the framework rather than model tuning.

Training. We train all models with Adam and a batch size of 32. Following the standard TSLib settings, baselines are trained for up to 10 epochs with a learning rate of $10^{-4}$ and early stopping on the validation MSE (patience 3). In CATF, the workers use the same learning rate as their baselines, and the manager uses $10^{-5}$ ($10^{-3}$ for CATF-FEDformer), both with a weight decay of $10^{-5}$. Because each sample trains only one of the $K = 3$ workers per epoch, a worker sees about one third of the training data in each epoch. We therefore train CATF for up to 30 epochs with early stopping (patience 5), three times the epoch budget of the baselines. All settings are listed in Appendix Table 8.

Normalization and evaluation. Because the scale of the input variables differs by orders of magnitude across datasets (e.g., bike counts vs. PM2.5 concentration), we evaluate every model under two common normalization schemes: (i) global normalization, where each variable is standardized with the mean and standard deviation of the training split, and (ii) sample normalization, where each input window is standardized with its own per-variable mean and standard deviation and the target window is scaled with the statistics of the target variable in the input window. For both schemes, predictions are mapped back to the original scale before computing the error, so all reported MSE values are in the original units of the target variable and are comparable across models within a dataset, but not across datasets.

Number of workers. We use $K = 3$ workers for all datasets and models. The suitable number of workers is dataset dependent (§ 5.3): on PM2.5-RV, $K = 6$ slightly reduces the MSE ($-1.4\%$) and $K = 9$ increases it ($+2.6\%$), and on Bike Share, neither improves the MSE (Appendix Table 17); none of these differences are statistically significant except for $K = 9$ on Bike Share. Since each additional worker adds a forward pass to every training iteration, we fix $K = 3$ as a trade-off between accuracy and training cost.

Evaluation Results

Table 1 reports the MSE of each baseline and its CATF-enhanced version for every dataset and horizon under both normalization schemes, averaged over 5 independent runs. Lower MSE values indicate better predictive accuracy. The full results, i.e., the mean and standard deviation (SD) over 5 runs for every setting, are given in Appendix Tables 9 and 10 (global normalization) and 10 (sample normalization), and the per-setting MSE reduction of each CATF-enhanced model over its baseline in Appendix Table 11. Table 1 also reports the mean MSE reduction of each CATF-enhanced model over its baseline. Because the error of a collapsed baseline run can be several orders of magnitude larger than that of a stable one (e.g., FEDformer and Autoformer on Weather), this mean is dominated by a few settings; we therefore focus on win counts.

Dataset	$L_y$	iTransformer (2024a) Global Base	+CATF	iTransformer (2024a) Sample Base	+CATF	TimesNet (2023a) Global Base	+CATF	TimesNet (2023a) Sample Base	+CATF	FEDformer (2022) Global Base	+CATF	FEDformer (2022) Sample Base	+CATF	Autoformer (2021) Global Base	+CATF	Autoformer (2021) Sample Base	+CATF	Informer (2021) Global Base	+CATF	Informer (2021) Sample Base	+CATF
Weather	96	204.7	194.9	447.0	178.3	245.6	215.9	418.6	158.5	28,741	351.3	1,208,418	181.7	39,860	419.5	1,933,907	210.1	1,733	442.7	4,402	155.7
Weather	192	249.3	226.7	637.6	231.6	271.1	234.8	410.6	226.1	28,712	548.9	10,628,690	236.8	230,293	380.4	151,937	285.4	6,645	426.5	7,309	220.8
Weather	336	254.6	232.3	749.2	254.8	264.8	243.3	456.7	241.8	50,859	559.7	380,781	267.3	70,468	279.8	157,994	270.1	77,472	460.2	3,729	240.1
Weather	720	270.1	235.9	748.3	330.0	269.1	241.7	963.3	291.8	41,001	640.8	798,364	268.3	209,899	259.7	29,367	296.1	4,335	527.5	3,637	290.7
PM2.5-LA	96	37.05	37.02	36.08	35.20	37.32	44.42	40.18	35.43	57.75	42.29	60.91	42.11	86.25	71.18	57.60	58.89	45.34	45.04	41.30	37.90
PM2.5-LA	192	40.64	41.17	40.93	40.81	40.16	42.52	42.62	39.25	65.63	45.25	64.07	43.77	66.43	64.94	61.37	65.62	46.80	42.66	42.06	39.62
PM2.5-LA	336	48.71	49.35	48.89	49.82	45.91	48.54	46.42	44.67	67.49	53.96	74.34	52.00	70.55	69.27	70.79	70.81	48.22	44.79	52.84	48.27
PM2.5-LA	720	62.55	64.17	63.62	63.88	59.76	61.11	59.23	58.73	75.79	66.34	80.34	64.62	106.7	72.36	142.7	72.23	52.81	51.37	62.88	63.56
PM2.5-RV	96	83.31	81.07	78.83	78.43	80.60	76.56	78.84	73.83	105.6	87.11	105.0	87.67	140.4	132.5	125.5	130.1	82.47	83.02	86.27	81.69
PM2.5-RV	192	92.20	90.23	90.67	89.73	87.78	83.44	99.05	84.37	131.5	98.74	146.2	96.72	142.6	133.5	158.9	163.9	94.80	97.51	101.8	91.40
PM2.5-RV	336	95.85	96.27	94.49	95.08	93.31	91.94	98.14	92.40	128.1	103.8	129.9	109.9	136.3	136.9	166.5	137.6	100.1	100.6	102.6	102.7
PM2.5-RV	720	87.58	87.89	87.28	86.70	87.86	87.73	85.81	86.12	115.2	94.93	104.5	95.63	128.4	110.8	122.7	112.5	92.79	88.98	94.57	94.62
Bike Share	96	15,889	14,815	15,134	14,258	16,967	16,450	15,511	14,288	117.5	159.7	254.2	148.6	241.1	215.6	251.9	241.6	158.1	158.9	158.0	153.8
Bike Share	192	21,640	20,864	21,140	20,356	22,654	22,035	20,500	20,508	25,283	19,930	37,106	21,570	32,384	29,645	33,966	34,204	22,758	23,480	22,524	22,916
Bike Share	336	27,939	27,025	27,477	26,907	28,267	27,661	27,702	27,672	28,721	23,498	43,632	27,528	39,929	37,735	48,489	41,074	28,818	32,018	29,075	33,134
Bike Share	720	37,411	37,451	36,714	37,845	36,838	38,760	35,968	38,228	34,864	32,858	53,511	39,902	44,139	39,982	51,277	47,750	34,917	35,416	37,201	36,790
Appliances	96	8,095	8,084	17,088	10,267	9,864	8,350	16,593	11,356	10,091	9,253	17,445	16,584	16,935	9,941	76,717	12,599	8,921	7,950	38,936	11,884
Appliances	192	8,123	8,129	12,000	11,150	10,121	8,468	22,152	12,909	9,463	9,020	26,191	12,535	36,460	10,650	97,072	13,558	9,217	7,939	52,661	12,043
Appliances	336	7,847	7,761	14,408	10,362	9,843	8,044	22,152	10,624	9,147	8,559	46,912	12,294	11,739	10,198	80,913	12,261	8,329	7,681	47,692	15,531
Appliances	720	8,046	8,048	16,877	11,478	8,861	8,280	18,186	11,308	10,063	9,080	42,264	12,637	13,574	9,706	53,329	11,398	8,261	7,870	31,365	12,709
CATF Win		12/20		16/20		15/20		17/20		20/20		20/20		19/20		14/20		14/20		15/20
Impr. (%)		+2.4		+18.4		+4.1		+21.9		+32.6		+47.8		+30.9		+40.8		+20.0		+34.1

Baseline vs CATF-Enhanced

Table 1 presents the performance of each baseline model and its corresponding CATF-enhanced variant side by side for direct comparison.

CATF-<baseline> consistently improves over the baseline models. Across the 100 baseline–variant pairs per normalization scheme (5 baselines × 5 datasets × 4 horizons), CATF-<baseline> achieves a lower MSE than its baseline in 80 pairs under global normalization and 82 pairs under sample normalization. The gains are consistent across horizons: CATF wins in 23, 20, 19, and 18 of the 25 pairs at $L_y = 96, 192, 336, 720$ under global normalization (23, 20, 20, and 19 under sample normalization), so the benefit only slightly decreases as the horizon grows. Moreover, when each model uses its better normalization scheme, the best CATF-enhanced model outperforms the best standalone baseline in 19 of the 20 settings (Table 2).

Dataset	$L_y$	Best baseline		Best CATF-enhanced		Time-MoE (2025)
		Model	MSE	Model	MSE	MSE
Weather	96	iTransformer	$204.7^{\mathrm{G}}$	CATF-Informer	$155.7^{\mathrm{S}}$	$164.9^{\mathrm{S}}$
	192	iTransformer	$249.3^{\mathrm{G}}$	CATF-Informer	$220.8^{\mathrm{S}}$	$215.0^{\mathrm{S}}$
	336	iTransformer	$254.6^{\mathrm{G}}$	CATF-iTransformer	$232.3^{\mathrm{G}}$	$225.6^{\mathrm{S}}$
	720	TimesNet	$269.1^{\mathrm{G}}$	CATF-iTransformer	$235.9^{\mathrm{G}}$	$233.5^{\mathrm{S}}$
PM2.5-LA	96	iTransformer	$36.08^{\mathrm{S}}$	CATF-iTransformer	$35.20^{\mathrm{S}}$	$39.66^{\mathrm{S}}$
	192	TimesNet	$40.16^{\mathrm{G}}$	CATF-TimesNet	$39.25^{\mathrm{S}}$	$43.16^{\mathrm{S}}$
	336	TimesNet	$45.91^{\mathrm{G}}$	CATF-TimesNet	$44.67^{\mathrm{S}}$	$51.37^{\mathrm{G}}$
	720	Informer	$52.81^{\mathrm{G}}$	CATF-Informer	$51.37^{\mathrm{G}}$	$60.59^{\mathrm{G}}$
PM2.5-RV	96	iTransformer	$78.83^{\mathrm{S}}$	CATF-TimesNet	$73.83^{\mathrm{S}}$	$78.68^{\mathrm{G}}$
	192	TimesNet	$87.78^{\mathrm{G}}$	CATF-TimesNet	$83.44^{\mathrm{G}}$	$87.28^{\mathrm{G}}$
	336	TimesNet	$93.31^{\mathrm{G}}$	CATF-TimesNet	$91.94^{\mathrm{G}}$	$91.45^{\mathrm{G}}$
	720	TimesNet	$85.81^{\mathrm{S}}$	CATF-TimesNet	$86.12^{\mathrm{S}}$	$87.41^{\mathrm{G}}$
Bike Share	96	iTransformer	$15{,}134^{\mathrm{S}}$	CATF-iTransformer	$14{,}258^{\mathrm{S}}$	$15{,}802^{\mathrm{S}}$
	192	TimesNet	$20{,}500^{\mathrm{S}}$	CATF-FEDformer	$19{,}930^{\mathrm{G}}$	$20{,}044^{\mathrm{G}}$
	336	iTransformer	$27{,}477^{\mathrm{S}}$	CATF-FEDformer	$23{,}498^{\mathrm{G}}$	$23{,}512^{\mathrm{G}}$
	720	FEDformer	$34{,}864^{\mathrm{G}}$	CATF-FEDformer	$32{,}858^{\mathrm{G}}$	$30{,}793^{\mathrm{G}}$
Appliances	96	iTransformer	$8{,}095^{\mathrm{G}}$	CATF-Informer	$7{,}950^{\mathrm{G}}$	$8{,}954^{\mathrm{G}}$
	192	iTransformer	$8{,}123^{\mathrm{G}}$	CATF-Informer	$7{,}739^{\mathrm{G}}$	$8{,}596^{\mathrm{G}}$
	336	iTransformer	$7{,}847^{\mathrm{G}}$	CATF-Informer	$7{,}681^{\mathrm{G}}$	$8{,}145^{\mathrm{G}}$
	720	iTransformer	$8{,}046^{\mathrm{G}}$	CATF-Informer	$7{,}870^{\mathrm{G}}$	$8{,}511^{\mathrm{G}}$
CATF Win		19/20		–		15/20
Impr. (%)		+5.9		–		+4.9

CATF improves every worker type, with the largest gains for the less stable baselines. Of the 20 settings (5 datasets × 4 horizons), CATF-iTransformer achieves a lower MSE than iTransformer in 12 under global and 16 under sample normalization, and CATF-TimesNet than TimesNet in 15 and 17; for these strong and recent workers, the improvements are smaller but still consistent. The gains are largest for the older and less stable workers: CATF-FEDformer outperforms FEDformer in all 20 settings under both schemes, CATF-Autoformer outperforms Autoformer in 19 and 14, and CATF-Informer outperforms Informer in 14 and 15.

CATF stabilizes training. A striking effect of CATF is the reduction of run-to-run variance. Several standalone baselines occasionally diverge on the original-scale targets, most visibly FEDformer, Autoformer, and Informer on Weather, whose MSE rises to $10^{3}$–$10^{5}$ under global and up to $10^{7}$ under sample normalization, with an SD of the same order. This instability can be traced to the missing-value codes in the training target of Weather (Appendix § 10.§ 6). Their CATF-enhanced counterparts remain within the range of the strong baselines (e.g., FEDformer on Weather at $L_y = 96$: 28,741 ± 16,160 vs. 351.3 ± 97.8 with CATF). Because each worker only needs to fit the subset of contexts routed to it, the manager–worker decomposition makes the optimization of each worker considerably easier.

The best CATF-enhanced model depends on the dataset. Across both normalization schemes, a CATF-enhanced model obtains the best result in 19 of the 20 settings of Table 1; the only exception is PM2.5-RV at $L_y = 720$, where the standalone TimesNet is best. The best worker type depends on the dataset: CATF-Informer on Appliances, the short horizons of Weather, and PM2.5-LA at $L_y = 720$; CATF-iTransformer on the long horizons of Weather and at $L_y = 96$ on PM2.5-LA and Bike Share; CATF-FEDformer on Bike Share at $L_y \ge192$; and CATF-TimesNet on PM2.5-RV and PM2.5-LA at $L_y \in\{192, 336\}$. Within each normalization scheme, a CATF-enhanced model is best in 18 of the 20 settings, both under global and under sample normalization (Appendix Tables 9 and 10).

5.2.2 CATF vs. Time-MoE. Table 2 compares, for each setting, the best standalone baseline, the best CATF-enhanced model, and zero-shot Time-MoE, each under its better normalization scheme. Time-MoE is evaluated zero-shot and is deterministic, so it is reported from a single run. The best CATF-enhanced model achieves a lower MSE than the best baseline in 19 of the 20 settings and than Time-MoE in 15 of the 20 settings (mean MSE reduction 4.9%). Time-MoE is better in the five settings, mostly at longer horizons (Weather for $L_y \ge192$, PM2.5-RV for $L_y = 336$, and Bike Share for $L_y = 720$). Overall, CATF allows comparatively small task-specific models to match or exceed a large pretrained foundation model on most context-rich forecasting tasks, while using a single worker per input at inference time.

Ablation Study

We conduct the ablation study with CATF-iTransformer on PM2.5-RV under global normalization and $L = L_y = 96$. The default configuration is identical to that of the main experiments: three homogeneous workers, a two-layer Transformer manager ($d_{model} = 512$, $d_{ff} = 2048$, 8 heads, dropout 0.05), and the combined KL and Wasserstein manager loss with group-lasso regularization. In each ablation, we change exactly one component and keep the others fixed. Each setting is repeated over 5 runs without fixed random seeds, and we test the difference from the default with Welch’s $t$-test. The default reaches an MSE of 80.52, consistent with Table 1 (81.07). Table 3 summarizes the results; Appendix § 11 reports the same ablation on Bike Share.

Setting		MSE	Δ (%)	$p$
Default		80.52 ± 1.02	–	–
Loss	KL only	81.87 ± 2.75	+1.7	0.347
	Wasserstein only	81.56 ± 1.10	+1.3	0.157
Reg.	No reg.	80.94 ± 1.19	+0.5	0.559
Manager	Small ($d_{model}=128$)	81.23 ± 0.78	+0.9	0.252
	Large ($d_{model}=768$)	81.05 ± 0.81	+0.7	0.386
Workers	$K=6$	79.37 ± 1.14	−1.4	0.132
	$K=9$	82.61 ± 2.99	+2.6	0.200
Mix	Top-3 models1 ($K=3$)	81.63 ± 2.13	+1.4	0.334
	Top-2 models$^{a}$ ×2 ($K=4$)	80.04 ± 2.70	−0.6	0.728

Overall, removing or changing any component of the manager increases the MSE by 0.5–1.7%. Changing the number or mix of workers changes the MSE by −1.4% to +2.6%; $K=6$ and the mixture of the top-2 models are slightly better than the default. None of these differences is statistically significant ($p \ge0.13$), so CATF is robust to these design choices. On Bike Share, the results are similar, except that nine workers and mixing workers with different architectures significantly degrade the MSE (Appendix Table 17).

5.3.1 Manager Model Loss Function. The manager loss (Equation (7)) consists of a divergence term, which combines the KL divergence and the Wasserstein distance, and a group-lasso regularization term on the manager’s parameters.

Divergence Term. Removing either divergence term increases the error. With the KL divergence alone, the MSE increases by 1.7% ($p = 0.35$), and its run-to-run SD is almost three times that of the default (2.75 vs. 1.02). With the Wasserstein distance alone, the MSE increases by 1.3% ($p = 0.16$). On Bike Share, the Wasserstein distance alone also over-concentrates the routing: the most used worker receives 73% of the test samples on average, compared with 49% for the default. The combined loss therefore achieves the lowest mean error, although the differences are not statistically significant, and we use it as the default.

Regularization. Removing the regularization term slightly increases the MSE (+0.5%, $p = 0.56$). We keep it in the default configuration.

5.3.2 Manager Model Architecture. We test a smaller manager ($d_{model} = 128$, $d_{ff} = 512$, 4 heads) and a larger one ($d_{model} = 768$, $d_{ff} = 3076$, 12 heads). Both slightly degrade the MSE, by 0.9% and 0.7%, although the differences are not statistically significant ($p \ge0.25$). The larger manager doubles the total number of parameters of CATF-iTransformer (15.5M vs. 7.3M) without improving the accuracy, whereas the smaller one reduces it to 1.1M at a small cost in accuracy. The default

manager capacity therefore gives the best accuracy among the tested sizes, and CATF is not highly sensitive.

Number of Workers

Increasing the number of workers from $K = 3$ to $K = 6$ reduces the MSE by 1.4% ($p = 0.13$), whereas $K = 9$ increases it by 2.6% ($p = 0.20$) and leads to the largest run-to-run variation (SD 2.99). PM2.5-RV has 14 context variables from on-site meteorology, reanalysis data, and co-pollutants, which may benefit from a moderately larger number of workers. On Bike Share, whose context consists of a small number of calendar and weather variables, neither $K = 6$ nor $K = 9$ improves the MSE. The suitable $K$ is therefore dataset dependent, and we use $K = 3$ in the main experiments as a trade-off between accuracy and cost. A promising direction for future work is to determine the number of workers adaptively, e.g., by estimating the number of latent context regimes in the data.

Mixture of Workers

We also explore heterogeneous worker configurations by combining the top-performing models on PM2.5-RV, i.e., 1×TimesNet + 1×iTransformer + 1×Informer or 2×TimesNet + 2×iTransformer. Both mixtures perform on par with the default (+1.4% and −0.6%, $p \ge0.33$), with a larger run-to-run variation (SD 2.13 and 2.70). On Bike Share, the corresponding 3-model mixture significantly degrades the MSE by 13.1% ($p < 0.001$). Thus, heterogeneous mixtures do not consistently improve forecasting accuracy, and we use homogeneous workers in the main experiments. A promising direction for future work is to explore targeted regularization or adaptive routing strategies to better leverage the complementary strengths of diverse worker models.

Case Study: Interpreting the Routing

Because each prediction of CATF is produced by a single Worker, which Worker handles which input can be observed directly, without the approximations of post-hoc explanation methods such as attention weights [39] or feature attributions [16, 49]. We examine whether these routing decisions reflect the context on Bike Share, where the effect of the context is well known: bike demand has two commuting peaks on working days and a single midday peak on weekends. We analyze

CATF-iTransformer ($L = L_y = 96$, global normalization, $K = 3$, five runs) with three questions: (i) does the learned routing improve the forecasts, (ii) which input variables drive the routing, and (iii) what has each Worker learned. As a sanity check [2], we repeat each analysis with an untrained Manager of the same architecture; a pattern that also appears for the untrained Manager reflects the input data rather than what the Manager has learned. On synthetic data with known regimes, the same analysis recovers the true regimes, the variable that determines them, and the specialization of the Workers (Appendix § 12.§ 2). Workers are labeled by test usage ($W_1$ is the most used); details of the analysis are given in Appendix § 12.§ 1.

Does the learned routing improve the forecasts? During inference, we keep the trained Workers unchanged and replace only the Manager’s routing with alternative assignments, so that any change in accuracy is due to the routing alone. The learned routing (MSE 15,000) outperforms random routing with the same Worker shares (16,175) and the untrained Manager (15,876) in all five runs, and the best single Worker for all samples (15,376) in three of five runs (Figure 3a). The gain comes from the specialization of the Workers. For the samples that the Manager routes to a given Worker, Figure 3c compares the MSE of all three Workers: the selected Worker is the most accurate, and the other two Workers have a 9–23% higher MSE. Each Worker is thus better than the others on its own samples, so the routing must send each sample to the right Worker; random routing, which ignores the context, loses this advantage.

Which input variables drive the routing? We shuffle one input variable at a time across the test samples in the Manager input only and record the share of routing decisions that change (permutation importance [10]). The day of the week is by far the most important variable (37% of the decisions change, vs. 10% for the untrained Manager), followed by the target history (22% vs. 9%) and the working-day indicator (20% vs. 6%), whereas each weather variable changes only 8–10% (Figure 3b). A counterfactual analysis shows that these calendar variables, i.e., the day of the week and the working-day indicator, cause the routing: when the day of the week of the input window is shifted by $d \in\{1,\ldots,6\}$ days, 55% of the decisions change, and 80% of the new decisions match the typical Worker of the shifted day; when instead the target history is replaced with that of $d$ days later and the calendar is kept, only 11% change (Appendix Figure 7).

What has each Worker learned? As shown above, the routing is driven by the calendar variables, i.e., the day of the week and the working-day indicator, which together determine how many working days and weekend days the forecast window contains. We therefore first examine how the Workers divide the week. The Manager splits the week by the days in the 96-hour forecast window (Figure 4a): forecasts made on Sunday to Tuesday, whose window consists mostly of working days, go to $W_1$; forecasts made on Wednesday to Friday, whose window reaches into the weekend, mostly to $W_2$; and forecasts made on Friday and Saturday, whose window starts with the weekend, mostly to $W_3$. Each Worker therefore forecasts a different demand trajectory (Figure 4b): demand recovering from the weekend, demand declining toward the weekend, and the weekend itself. The other context variables are only weakly related to the routing: the NMI between the selected Worker and the day of the week is 0.44, compared with at most 0.05 for the holiday indicator, the month, each weather variable, and the recent demand (Appendix Figure 10). The context of the samples handled by each Worker shows the same picture (Appendix Figure 9): each Worker has learned the patterns of a different context, and the differences between the Workers are largest for the context variables that drive the routing. Over the input and forecast windows, the Workers differ most in the working-day share, by 1.2 standard deviations between the Worker means, and accordingly in the demand trajectory, so each Worker is trained on and specializes in a different weekly pattern. For the weather variables, the differences are much smaller (0.2–0.4 standard deviations) and largely reflect the days on which the weather occurred; the weather has low importance because the Manager sees only the past weather, which says little about the forecast window. All five runs partition the samples by the day of the week, although the boundaries between the Workers differ between runs (Appendix § 12.§ 3).

In summary, the routing of CATF on Bike Share is consistent with domain knowledge: the Manager infers the upcoming working-day/weekend pattern from the calendar, and each phase of the week is handled by a Worker that specializes in it.

Discussion

When does CATF help? The routing is most useful when the forecasting regime is determined by the context. On Bike Share, the calendar determines whether the forecast window contains working days or the weekend, and the learned routing clearly outperforms random routing (§ 6); on synthetic data with context-driven regimes, the Manager recovers the regimes almost perfectly (Appendix § 12.§ 2). On PM2.5-RV, where the concentration varies with continuously changing meteorological conditions, the Workers barely specialize and the learned routing is on par with the best single Worker (Appendix § 12.§ 4). The routing analysis therefore also indicates whether a dataset contains context-driven regimes. Independently of the routing, splitting the data among several Workers stabilizes the training of baselines that diverge on Weather, whose training target contains missing-value codes (§ 5.2).

Interpretability. The routing decisions are an exact description of which Worker produced a forecast, and they can be related to the context with simple tools such as permutation importance and counterfactuals. They explain, however, only which Worker is used, not how this Worker arrives at its forecast. Moreover, the partition learned by the Manager can differ between runs, e.g., in where the boundaries between the Workers lie or in whether holidays are treated as weekends (Appendix § 12.§ 3); interpretations should therefore be checked across several runs.

Limitations and future work. CATF trains the Manager and all Workers jointly and therefore needs considerably more training time and memory than a single model (Appendix Table 16); at inference time, only the Manager and the selected Worker are executed. The suitable number of Workers is dataset dependent and is currently set by hand (§ 5.3), and mixtures of different worker architectures did not consistently improve the accuracy. Promising directions are to determine the number of Workers adaptively, e.g., by estimating the number of latent context regimes, to better exploit heterogeneous Workers, and to apply the Manager–Worker design to other context-dependent prediction tasks, such as trajectory forecasting or human behavior modeling.

Conclusion

We introduced CATF, a context-aware forecasting framework in which a Transformer-based Manager routes each input to one of several Worker models, trained jointly so that the Workers specialize without manual data partitioning. Experiments on five context-rich datasets, four forecasting horizons, and two normalization schemes show that CATF consistently improves its standalone counterparts, reduces their run-to-run variance, and is competitive with a pretrained foundation model, and a case study on bike-sharing data shows that its routing decisions follow the context in an interpretable way.

Experiment Details

Dataset

We conducted our comparative experiments on 5 datasets: Weather, PM2.5-LA, PM2.5-RV, Bike Share, and Appliances. For every dataset, the timestamp column is dropped, the last column is used as the forecasting target, and all remaining columns are used as context. The details of the datasets are presented in Table 4.

Dataset	#Variables	Target	Input Length ($L$)	Output Length ($L_y$)	Frequency	Dataset Size	Domain
Weather	21	CO$_2$ (ppm)	96	{96, 192, 336, 720}	10 minutes	52,696	Meteorology
PM2.5-LA	15	PM2.5 ($\mu$g/m$^3$)	96	{96, 192, 336, 720}	1 hour	35,063	Air quality
PM2.5-RV	15	PM2.5 ($\mu$g/m$^3$)	96	{96, 192, 336, 720}	1 hour	35,063	Air quality
Bike Share	12	Rental count	96	{96, 192, 336, 720}	1 hour	17,379	Transportation
Appliances	26	Energy use (Wh)	96	{96, 192, 336, 720}	10 minutes	19,735	Energy

Hyper-parameters

To ensure a much fairer comparison, we adopted baseline hyperparameters from scripts in Time-Series Library [1], which are the same as worker models, presented in Table 5. For the manager, we used the same default setting of hyperparameters (Table 6), except for fine-tuning the manager model learning rates for some experiments (Table 7).

Model	#Layers	#Model Dimension	#Feedforward Dimension	#Attention Heads	#Dropout	#Attention Factor	#Learning Rate
i-Transformer	2	128	128	8	0.1	3	0.0001
TimesNet	2	32	32	8	0.1	3	0.0001
FEDformer	2	512	2048	8	0.1	3	0.0001
Autoformer	2	512	2048	8	0.1	3	0.0001
Informer	2	512	2048	8	0.1	3	0.0001

Model Type	#Layers	#Model Dimension	#Feedforward Dimension	#Attention Heads	#Dropout
Transformer	2	512	2048	8	0.05

Model	Manager LR	Worker LR
CATF-iTransformer	0.00001	0.0001
CATF-TimesNet	0.00001	0.0001
CATF-FEDformer	0.001	0.0001
CATF-Autoformer	0.00001	0.0001
CATF-Informer	0.00001	0.0001

Worker Models

Worker Models. We adopted the same worker model hyperparameter as the baseline. All models have the same hyperparameter settings across all datasets, as follows.

Manager Models

Manager Models. For all the training of CATF-<baseline> models in comparative experiments, we use the same hyperparameters of the manager model architecture, except we fine-tuned some manager models’ learning rates for some datasets.

Here are the default settings of the hyperparameters for the manager model architecture in Table 6.

Table 7 summarizes the learning rates and optimizer settings used for the manager and the workers. The same values are used for all datasets, horizons, and normalization schemes.

Training

In Table 8, we present the shared and CATF-specific training settings, including split sizes, epochs, etc.

Hyperparameter	Baseline	CATF Models
Shared Settings
Batch Size	32	32
Training Size	0.8	0.8
Validation Size	0.1	0.1
Test Size	0.1	0.1
Epochs	10	30
Early Stopping Patience	3	5
Optimizer	Adam	Adam (weight decay $10^{-5}$)
Runs per setting	5	5
CATF-Specific Settings
Number of Workers	-	3
Gradient Clipping Value	-	1
Worker Update Steps	-	2

Complete Results of Forecasting Tasks

Full Results

Table 9 and Table 10 report the MSE of every baseline and its CATF-enhanced version for each dataset and horizon under global and sample normalization (mean $\pm$ SD over 5 runs). They complement Table 1, which reports the mean MSE only.

Dataset	$L_y$	iTransformer (2024a)	CATF-iTransformer	TimesNet (2023a)	CATF-TimesNet	FEDformer (2022)	CATF-FEDformer	Autoformer (2021)	CATF-Autoformer	Informer (2021)	CATF-Informer
Weather	96	204.7 ± 2.29	194.9 ± 5.19	245.6 ± 10.69	215.9 ± 21.53	28,741 ± 16,160	351.3 ± 97.78	39,860 ± 35,693	419.5 ± 104.1	1,733 ± 830.7	442.7 ± 35.52
	192	249.3 ± 3.11	226.7 ± 8.12	271.1 ± 13.23	234.8 ± 8.91	28,712 ± 19,726	548.9 ± 19.67	230,293 ± 340,357	380.4 ± 53.86	6,645 ± 10,171	426.5 ± 44.44
	336	254.6 ± 2.15	232.3 ± 3.39	264.8 ± 3.40	243.3 ± 6.20	50,859 ± 39,591	559.7 ± 79.00	70,468 ± 97,935	279.8 ± 13.24	77,472 ± 170,210	460.2 ± 41.99
	720	270.1 ± 2.85	235.9 ± 6.54	269.1 ± 5.97	241.7 ± 4.23	41,001 ± 22,172	640.8 ± 66.33	209,899 ± 267,336	259.7 ± 12.00	4,335 ± 4,652	527.5 ± 59.48
PM2.5-LA	96	37.05 ± 0.39	37.02 ± 0.28	37.32 ± 1.68	44.42 ± 3.64	57.75 ± 4.31	42.29 ± 1.29	86.25 ± 24.26	71.18 ± 6.13	45.34 ± 4.37	45.04 ± 6.22
	192	40.64 ± 0.30	41.17 ± 0.24	40.16 ± 1.94	42.52 ± 2.36	65.63 ± 1.73	45.25 ± 0.86	66.43 ± 8.03	64.94 ± 4.65	46.80 ± 3.08	42.66 ± 0.25
	336	48.71 ± 0.14	49.35 ± 0.76	45.91 ± 2.01	48.54 ± 1.97	67.49 ± 2.75	53.96 ± 2.78	70.55 ± 5.09	69.27 ± 2.78	48.22 ± 2.55	44.79 ± 0.57
	720	62.55 ± 0.37	64.17 ± 0.42	59.76 ± 1.06	61.11 ± 2.93	75.79 ± 5.62	66.34 ± 1.85	106.7 ± 42.71	72.36 ± 3.80	52.81 ± 0.80	51.37 ± 0.79
PM2.5-RV	96	83.31 ± 0.63	81.07 ± 1.18	80.60 ± 4.73	76.56 ± 1.93	105.6 ± 5.02	87.11 ± 2.39	140.4 ± 27.32	132.5 ± 9.88	82.47 ± 6.03	83.02 ± 0.82
	192	92.20 ± 0.71	90.23 ± 0.58	87.78 ± 4.06	83.44 ± 1.35	131.5 ± 8.30	98.74 ± 9.20	142.6 ± 26.11	133.5 ± 14.51	94.80 ± 5.01	97.51 ± 1.59
	336	95.85 ± 1.57	96.27 ± 1.15	93.31 ± 2.49	91.94 ± 1.95	128.1 ± 6.07	103.8 ± 6.66	136.3 ± 11.15	136.9 ± 9.93	100.1 ± 3.35	100.6 ± 0.74
	720	87.58 ± 0.66	87.89 ± 0.23	87.86 ± 1.28	87.73 ± 1.56	115.2 ± 16.02	94.83 ± 1.54	128.4 ± 45.85	110.8 ± 4.55	92.70 ± 3.41	89.88 ± 0.59
Bike Share	96	15,889 ± 189.8	14,815 ± 78.46	16,967 ± 583.8	16,450 ± 1,166	18,171 ± 6,192	15,597 ± 1,344	24,641 ± 3,579	21,516 ± 2,918	18,653 ± 1,798	18,214 ± 515.1
	192	21,640 ± 188.7	20,364 ± 19.94	22,654 ± 455.7	22,035 ± 731.1	25,238 ± 1,524	19,930 ± 655.7	32,384 ± 1,689	29,645 ± 1,023	27,528 ± 1,585	23,480 ± 732.0
	336	27,939 ± 175.9	27,025 ± 15.15	28,267 ± 425.4	27,661 ± 424.5	28,725 ± 2,802	23,498 ± 770.5	39,925 ± 3,912	37,735 ± 4,623	28,818 ± 6,700	32,018 ± 1,878
	720	37,411 ± 193.7	37,451 ± 257.1	36,838 ± 960.6	38,760 ± 373.4	34,864 ± 2,454	32,858 ± 2,245	44,139 ± 1,902	39,982 ± 2,296	34,947 ± 51.06	35,416 ± 3,309
Appliances	96	8,095 ± 194.7	8,084 ± 96.98	9,864 ± 703.8	8,350 ± 152.6	10,091 ± 737.9	9,235 ± 152.4	16,935 ± 1,463	9,941 ± 287.8	8,921 ± 830.9	7,950 ± 321.7
	192	8,123 ± 95.14	8,129 ± 38.71	10,121 ± 918.2	8,468 ± 187.6	9,436 ± 176.2	9,027 ± 178.2	13,646 ± 2,756	10,650 ± 449.0	9,172 ± 488.4	7,739 ± 332.7
	336	7,847 ± 36.02	7,761 ± 36.02	9,843 ± 1,338	8,044 ± 155.7	9,147 ± 520.2	8,559 ± 930.0	11,739 ± 3,494	10,198 ± 434.5	8,329 ± 365.6	7,681 ± 207.4
	720	8,046 ± 38.00	8,048 ± 63.82	8,861 ± 654.1	8,280 ± 87.46	10,063 ± 779.3	9,030 ± 118.1	13,574 ± 3,183	9,706 ± 124.9	8,261 ± 225.2	7,870 ± 175.5
CATF Win		12		15		20		19		14
Impr. (%) (mean)		+2.4		+4.1		+32.6		+30.9		+20.0

Dataset	$L_y$	iTransformer (2024a)	CATF-iTransformer	TimesNet (2023a)	CATF-TimesNet	FEDformer (2022)	CATF-FEDformer	Autoformer (2021)	CATF-Autoformer	Informer (2021)	CATF-Informer
Weather	96	447.0 ± 47.3	178.3 ± 9.8	416.0 ± 153.0	152.5 ± 10.52	1,208,418 ± 1,398,675	181.1 ± 2.3	1,933,907 ± 1,800,987	181.1 ± 24.66	4,402 ± 3,262	155.7 ± 6.54
	192	637.6 ± 71.19	231.6 ± 4.39	410.6 ± 120.8	226.1 ± 6.49	10,628,690 ± 15,360,229	236.8 ± 9.04	151,937 ± 325,720	285.4 ± 31.47	7,309 ± 2,449	220.8 ± 15.75
	336	749.2 ± 64.09	245.8 ± 8.98	456.7 ± 167.3	241.8 ± 6.77	380,781 ± 420,438	276.3 ± 16.56	157,994 ± 251,704	270.1 ± 12.01	3,729 ± 1,141	240.1 ± 5.54
	720	748.3 ± 46.15	335.0 ± 35.88	963.3 ± 360.3	291.8 ± 17.00	798,364 ± 652,673	268.3 ± 18.87	29,367 ± 28,419	296.1 ± 33.97	3,637 ± 1,223	290.7 ± 16.44
PM2.5-LA	96	36.08 ± 0.96	35.20 ± 0.96	40.18 ± 4.20	35.45 ± 3.10	60.91 ± 9.88	42.11 ± 2.02	57.60 ± 6.91	58.89 ± 3.52	41.30 ± 2.24	37.90 ± 1.28
	192	40.93 ± 0.48	40.81 ± 0.46	42.62 ± 3.67	39.25 ± 1.73	64.07 ± 2.94	43.77 ± 1.81	65.62 ± 8.96	65.82 ± 8.96	42.06 ± 1.10	39.62 ± 0.90
	336	48.89 ± 0.38	49.82 ± 0.69	46.42 ± 0.96	46.47 ± 2.05	74.34 ± 9.01	52.00 ± 2.87	70.79 ± 23.11	70.81 ± 3.65	52.84 ± 2.26	48.27 ± 3.12
	720	63.62 ± 1.45	63.88 ± 1.15	59.23 ± 0.75	58.73 ± 1.94	80.34 ± 13.98	64.62 ± 1.35	142.7 ± 82.92	72.23 ± 2.72	62.88 ± 2.04	63.56 ± 2.81
PM2.5-RV	96	78.83 ± 0.70	73.48 ± 0.42	78.84 ± 4.16	73.83 ± 1.74	105.0 ± 10.35	87.67 ± 2.04	125.5 ± 11.71	130.1 ± 13.80	86.27 ± 6.23	81.69 ± 3.16
	192	90.67 ± 1.47	89.73 ± 0.52	99.05 ± 6.72	84.37 ± 4.47	146.2 ± 3.44	96.72 ± 4.31	158.9 ± 35.83	163.9 ± 38.35	101.8 ± 5.41	91.40 ± 1.67
	336	94.49 ± 1.71	95.08 ± 1.85	98.14 ± 3.71	94.20 ± 1.92	129.9 ± 10.81	109.9 ± 3.75	166.5 ± 39.61	137.6 ± 10.42	102.6 ± 14.28	102.7 ± 2.36
	720	87.28 ± 0.80	86.70 ± 0.84	85.51 ± 1.57	86.12 ± 1.11	104.5 ± 9.92	95.63 ± 2.94	122.7 ± 15.69	112.5 ± 1.01	94.57 ± 5.98	94.62 ± 4.06
Bike Share	96	15,134 ± 148.1	14,258 ± 353	35,418 ± 861.6	14,586 ± 1,192	27,592 ± 2,130	19,444 ± 675.7	25,573 ± 3,915	25,739 ± 4,503	16,080 ± 4,465	15,038 ± 871.0
	192	21,140 ± 109.6	20,356 ± 179.0	29,500 ± 527.8	20,508 ± 713.8	37,106 ± 3,519	21,570 ± 226.8	33,966 ± 2,009	34,204 ± 3,555	22,524 ± 578.7	22,916 ± 969.5
	336	27,477 ± 214.3	26,907 ± 469.9	29,702 ± 622.4	27,672 ± 273.7	43,632 ± 3,068	27,528 ± 715.9	48,489 ± 3,288	41,474 ± 1,975	29,075 ± 676.5	33,134 ± 1.69
	720	36,714 ± 288	37,845 ± 447.9	35,968 ± 824.9	38,228 ± 1,150	53,511 ± 2,412	39,902 ± 934.9	51,277 ± 4,942	47,750 ± 2,465	37,201 ± 1,044	36,790 ± 2,374
Appliances	96	17,088 ± 2,518	10,267 ± 727.9	16,593 ± 1,273	11,356 ± 1,938	17,445 ± 2,985	13,584 ± 2,197	76,717 ± 96,329	12,599 ± 1,494	38,936 ± 15,594	11,884 ± 2,970
	192	12,600 ± 906.8	10,150 ± 1,708	15,221 ± 5,401	12,079 ± 883.9	26,191 ± 9,761	15,355 ± 2,335	97,072 ± 27,511	13,580 ± 1,081	52,661 ± 22,981	12,043 ± 3,531
	336	14,408 ± 1,905	10,362 ± 1,076	22,152 ± 7,005	10,624 ± 1,113	46,912 ± 43,302	12,294 ± 1,322	80,913 ± 63,971	12,261 ± 1,005	47,692 ± 24,051	15,531 ± 10,359
	720	16,877 ± 2,276	11,478 ± 1,603	18,836 ± 4,418	11,308 ± 393.4	42,264 ± 20,699	12,637 ± 2,454	53,329 ± 58,727	13,998 ± 1,104	31,365 ± 7,205	12,709 ± 2,409
CATF Win		16		17		20		14		15
Impr. (%) (mean)		+18.4		+21.9		+47.8		+40.8		+34.1

Reduction Percentage

Table 11 presents the percentage reduction in the MSE mean achieved by the CATF-<baseline> compared to the <baseline>. A lower MSE indicates better performance. To assess whether the improvements are statistically significant, we apply a two-sided Welch’s $t$-test to the 5 runs of each CATF-enhanced model and its standalone counterpart in every setting. Of the 200 comparisons, CATF is significantly better in 91 ($p < 0.05$) and significantly worse in 9, and the difference is not significant in the remaining 100. The share of significant improvements is highest for CATF-FEDformer (29 of 40) and lowest for CATF-Autoformer (6 of 40), whose baseline has a very large run-to-run variance, so that even large mean improvements are often not significant with 5 runs.

Dataset	$L_y$	iTransformer G	iTransformer S	TimesNet G	TimesNet S	FEDformer G	FEDformer S	Autoformer G	Autoformer S	Informer G	Informer S
Weather	96	+4.8	+60.1	+12.1	+62.1	+98.8	+100.0	+98.9	+100.0	+74.5	+96.5
	192	+9.1	+63.7	+13.4	+44.9	+98.1	+100.0	+99.8	+99.8	+93.6	+97.0
	336	+8.7	+66.0	+8.1	+47.1	+98.9	+99.9	+99.6	+99.8	+99.4	+93.6
	720	+12.7	+55.9	+10.2	+69.7	+98.4	+100.0	+99.9	+99.0	+87.8	+92.0
PM2.5-LA	96	+0.1	+2.4	<u>-19.0</u>	+11.8	+26.8	+30.9	+17.5	-2.3	+0.7	+8.2
	192	<u>-1.3</u>	+0.3	-5.9	+7.9	+31.0	+31.7	+2.2	-6.9	+8.8	+5.8
	336	-1.3	<u>-1.9</u>	-5.7	+3.8	+20.0	+30.1	+1.8	-0.0	+7.1	+8.6
	720	<u>-2.6</u>	-0.4	-2.3	+0.8	+12.5	+19.6	+32.2	+49.4	+2.7	-1.1
PM2.5-RV	96	+2.7	+0.5	+5.0	+6.4	+17.5	+16.5	+5.6	-3.6	-0.7	+5.3
	192	+2.1	+1.0	+4.9	+14.8	+24.9	+33.8	+6.4	-3.1	-2.9	+10.2
	336	-0.4	-0.6	+1.5	+5.8	+19.0	+15.4	-0.5	+17.4	-0.4	-0.0
	720	-0.4	+0.7	+0.1	-0.4	+17.7	+8.5	+13.7	+8.4	+3.0	-0.1
Bike Share	96	+6.8	+5.8	+3.0	+3.5	+14.2	+45.8	+12.7	+3.0	+2.4	+6.5
	192	+3.6	+3.7	+2.7	-0.0	+21.2	+41.9	+8.5	-0.7	-3.2	-1.7
	336	+3.3	+2.1	+2.1	+0.1	+18.2	+36.9	+5.5	+15.3	-11.1	-14.0
	720	-0.1	<u>-3.1</u>	<u>-5.2</u>	<u>-6.3</u>	+5.8	+25.4	+9.4	+6.9	<u>-1.3</u>	+1.1
Appliances	96	+0.1	+39.9	+15.3	+31.6	+8.3	+22.1	+41.3	+83.6	+10.9	+69.5
	192	-0.1	+11.5	+16.3	+45.5	+4.3	+52.8	+22.0	+86.0	+15.6	+77.1
	336	+1.1	+28.1	+18.3	+52.0	+6.4	+73.8	+13.1	+84.8	+7.8	+67.4
	720	-0.0	+32.0	+6.6	+37.8	+9.8	+70.1	+28.5	+78.6	+4.7	+59.5
#Wins		12/20	16/20	15/20	17/20	20/20	20/20	19/20	14/20	14/20	15/20
#Sig. better		9/20	10/20	8/20	10/20	17/20	12/20	4/20	2/20	7/20	12/20
#Sig. worse		2/20	2/20	2/20	1/20	0/20	0/20	0/20	0/20	1/20	1/20

MAE Results

Tables 12 and 13 report the MAE of all models under global and sample normalization, in the same format as Tables 9 and 10. The MAE results are consistent with the MSE results: CATF-enhanced models have a lower mean MAE than their standalone counterparts in 162 of the 200 comparisons (84 under global and 78 under sample normalization).

Global vs. Sample Normalization

Table 14 compares the two normalization schemes for every model. The standalone baselines are clearly sensitive to this choice: global normalization is better in 11–17 of the 20 settings, and sample normalization is drastically worse on Weather and Appliances for most baselines (e.g., +2,976% for FEDformer on Weather). With CATF, this gap shrinks considerably: the median difference over all settings lies between −1.6% and +5.5% for the CATF-enhanced models, compared with +0.5% to +52% for the baselines. Appliances is the only dataset on which global normalization is better for all models, including CATF. Time-MoE, which was pretrained with per-window normalization, is the only model that is much better with sample normalization on Weather (−76%).

Table 14. Global vs. sample normalization. For each model and dataset, the cell is the median over the four horizons of the relative difference in mean MSE, (sample − global)/global (%); positive values mean that global normalization is better. The last two columns give the number of the 20 settings (5 datasets × 4 horizons) in which global normalization is better, and the median relative difference over all 20 settings.

Model	Weather	PM2.5-LA	PM2.5-RV	Bike Share	Appliances	#Global better	Median (all)
iTransformer	+166	+0.5	−1.5	−2.1	+97	11/20	+0.5
TimesNet	+71	+3.6	+1.5	−5.7	+112	13/20	+6.9
FEDformer	+2,976	+5.7	+0.4	+52	+249	17/20	+52
Autoformer	+45	−3.6	+3.5	+11	+471	14/20	+14
Informer	−3.1	+0.3	+3.5	−0.1	+405	14/20	+5.5
CATF-iTransformer	+5.9	−0.7	−1.3	−1.4	+35	9/20	−0.4
CATF-TimesNet	−2.1	−7.8	−0.7	−4.2	+36	8/20	−1.6
CATF-FEDformer	−55	−2.9	+0.7	+13	+41	10/20	+0.1
CATF-Autoformer	−14	+0.4	+1.0	+15	+23	14/20	+5.5
CATF-Informer	−48	+0.3	+0.2	+0.5	+59	10/20	+0.2
Time-MoE (zero-shot)	−76	+1.4	+13	+12	+12	13/20	+10

Best Normalization per Model

Since the better normalization scheme differs across models and datasets, Table 15 compares each baseline and its CATF-enhanced version under their respective best normalization, i.e., the lower mean MSE of global and sample normalization. Under this comparison, CATF improves over its standalone counterpart in 82 of the 100 comparisons (iTransformer 13/20, TimesNet 17/20, FEDformer 20/20, Autoformer 16/20, Informer 16/20).

Dataset	$L_y$	iTransformer Base	iTransformer CATF	TimesNet Base	TimesNet CATF	FEDformer Base	FEDformer CATF	Autoformer Base	Autoformer CATF	Informer Base	Informer CATF
Weather	96	204.7$^{\mathrm{G}}$	178.3$^{\mathrm{S}}$	245.6$^{\mathrm{G}}$	158.5$^{\mathrm{S}}$	28,741$^{\mathrm{G}}$	181.7$^{\mathrm{S}}$	39,860$^{\mathrm{G}}$	210.1$^{\mathrm{S}}$	1,733$^{\mathrm{G}}$	155.7$^{\mathrm{S}}$
	192	249.3$^{\mathrm{G}}$	226.7$^{\mathrm{G}}$	271.1$^{\mathrm{G}}$	226.1$^{\mathrm{S}}$	28,712$^{\mathrm{G}}$	236.8$^{\mathrm{S}}$	151,937$^{\mathrm{S}}$	285.4$^{\mathrm{S}}$	6,645$^{\mathrm{G}}$	220.8$^{\mathrm{S}}$
	336	254.6$^{\mathrm{G}}$	232.3$^{\mathrm{G}}$	264.8$^{\mathrm{G}}$	241.8$^{\mathrm{S}}$	50,859$^{\mathrm{G}}$	267.3$^{\mathrm{S}}$	70,468$^{\mathrm{G}}$	270.1$^{\mathrm{S}}$	3,729$^{\mathrm{S}}$	240.1$^{\mathrm{S}}$
	720	270.1$^{\mathrm{G}}$	235.9$^{\mathrm{G}}$	269.1$^{\mathrm{G}}$	241.7$^{\mathrm{G}}$	41,001$^{\mathrm{G}}$	268.3$^{\mathrm{S}}$	29,367$^{\mathrm{S}}$	259.7$^{\mathrm{G}}$	3,637$^{\mathrm{S}}$	290.7$^{\mathrm{S}}$
PM2.5-LA	96	36.08$^{\mathrm{S}}$	35.20$^{\mathrm{S}}$	37.32$^{\mathrm{G}}$	35.43$^{\mathrm{S}}$	57.75$^{\mathrm{G}}$	42.11$^{\mathrm{S}}$	57.60$^{\mathrm{S}}$	58.89$^{\mathrm{S}}$	41.30$^{\mathrm{S}}$	37.90$^{\mathrm{S}}$
	192	40.64$^{\mathrm{G}}$	40.81$^{\mathrm{S}}$	40.16$^{\mathrm{G}}$	39.25$^{\mathrm{S}}$	64.07$^{\mathrm{S}}$	43.77$^{\mathrm{S}}$	61.37$^{\mathrm{G}}$	64.94$^{\mathrm{G}}$	42.06$^{\mathrm{S}}$	39.62$^{\mathrm{S}}$
	336	48.71$^{\mathrm{G}}$	49.35$^{\mathrm{G}}$	45.91$^{\mathrm{G}}$	44.67$^{\mathrm{S}}$	67.49$^{\mathrm{G}}$	52.00$^{\mathrm{S}}$	70.55$^{\mathrm{G}}$	69.27$^{\mathrm{G}}$	48.22$^{\mathrm{G}}$	44.79$^{\mathrm{G}}$
	720	62.55$^{\mathrm{G}}$	63.88$^{\mathrm{S}}$	59.23$^{\mathrm{S}}$	58.73$^{\mathrm{S}}$	75.79$^{\mathrm{G}}$	64.62$^{\mathrm{S}}$	106.7$^{\mathrm{G}}$	72.23$^{\mathrm{S}}$	52.81$^{\mathrm{G}}$	51.37$^{\mathrm{G}}$
PM2.5-RV	96	78.83$^{\mathrm{S}}$	78.43$^{\mathrm{S}}$	78.84$^{\mathrm{S}}$	73.83$^{\mathrm{S}}$	105.0$^{\mathrm{S}}$	87.11$^{\mathrm{G}}$	125.5$^{\mathrm{S}}$	130.1$^{\mathrm{S}}$	82.47$^{\mathrm{G}}$	81.69$^{\mathrm{S}}$
	192	90.67$^{\mathrm{S}}$	89.73$^{\mathrm{S}}$	87.78$^{\mathrm{G}}$	83.44$^{\mathrm{G}}$	131.5$^{\mathrm{G}}$	96.72$^{\mathrm{S}}$	142.6$^{\mathrm{G}}$	133.5$^{\mathrm{G}}$	94.80$^{\mathrm{G}}$	91.40$^{\mathrm{S}}$
	336	94.49$^{\mathrm{S}}$	95.08$^{\mathrm{S}}$	93.31$^{\mathrm{G}}$	91.94$^{\mathrm{G}}$	128.1$^{\mathrm{G}}$	103.8$^{\mathrm{G}}$	136.3$^{\mathrm{G}}$	136.9$^{\mathrm{G}}$	100.1$^{\mathrm{G}}$	100.6$^{\mathrm{G}}$
	720	87.28$^{\mathrm{S}}$	86.70$^{\mathrm{S}}$	85.81$^{\mathrm{S}}$	86.12$^{\mathrm{S}}$	104.5$^{\mathrm{S}}$	94.83$^{\mathrm{G}}$	122.7$^{\mathrm{S}}$	110.8$^{\mathrm{G}}$	92.70$^{\mathrm{G}}$	89.88$^{\mathrm{G}}$
Bike Share	96	15,134$^{\mathrm{S}}$	14,258$^{\mathrm{S}}$	15,418$^{\mathrm{S}}$	14,886$^{\mathrm{S}}$	18,171$^{\mathrm{G}}$	14,944$^{\mathrm{S}}$	24,641$^{\mathrm{G}}$	21,516$^{\mathrm{G}}$	16,080$^{\mathrm{S}}$	15,038$^{\mathrm{S}}$
	192	21,140$^{\mathrm{S}}$	20,356$^{\mathrm{S}}$	20,500$^{\mathrm{S}}$	20,508$^{\mathrm{S}}$	25,283$^{\mathrm{G}}$	19,930$^{\mathrm{G}}$	32,384$^{\mathrm{G}}$	29,645$^{\mathrm{G}}$	22,524$^{\mathrm{S}}$	22,916$^{\mathrm{S}}$
	336	27,477$^{\mathrm{S}}$	26,907$^{\mathrm{S}}$	27,702$^{\mathrm{S}}$	27,661$^{\mathrm{G}}$	28,721$^{\mathrm{G}}$	23,498$^{\mathrm{G}}$	39,925$^{\mathrm{G}}$	37,735$^{\mathrm{G}}$	28,818$^{\mathrm{G}}$	32,018$^{\mathrm{G}}$
	720	36,714$^{\mathrm{S}}$	37,451$^{\mathrm{G}}$	35,968$^{\mathrm{S}}$	38,228$^{\mathrm{S}}$	34,864$^{\mathrm{G}}$	32,858$^{\mathrm{G}}$	44,139$^{\mathrm{G}}$	39,982$^{\mathrm{G}}$	34,947$^{\mathrm{G}}$	35,416$^{\mathrm{G}}$
Appliances	96	8,095$^{\mathrm{G}}$	8,084$^{\mathrm{G}}$	9,864$^{\mathrm{G}}$	8,350$^{\mathrm{G}}$	10,091$^{\mathrm{G}}$	9,253$^{\mathrm{G}}$	16,935$^{\mathrm{G}}$	9,941$^{\mathrm{G}}$	8,921$^{\mathrm{G}}$	7,950$^{\mathrm{G}}$
	192	8,123$^{\mathrm{G}}$	8,129$^{\mathrm{G}}$	10,121$^{\mathrm{G}}$	8,468$^{\mathrm{G}}$	9,436$^{\mathrm{G}}$	9,027$^{\mathrm{G}}$	13,646$^{\mathrm{G}}$	10,650$^{\mathrm{G}}$	9,172$^{\mathrm{G}}$	7,739$^{\mathrm{G}}$
	336	7,847$^{\mathrm{G}}$	7,761$^{\mathrm{G}}$	9,843$^{\mathrm{G}}$	8,044$^{\mathrm{G}}$	9,147$^{\mathrm{G}}$	8,559$^{\mathrm{G}}$	11,739$^{\mathrm{G}}$	10,198$^{\mathrm{G}}$	8,329$^{\mathrm{G}}$	7,681$^{\mathrm{G}}$
	720	8,046$^{\mathrm{G}}$	8,048$^{\mathrm{G}}$	8,861$^{\mathrm{G}}$	8,280$^{\mathrm{G}}$	10,063$^{\mathrm{G}}$	9,080$^{\mathrm{G}}$	13,574$^{\mathrm{G}}$	9,706$^{\mathrm{G}}$	8,261$^{\mathrm{G}}$	7,870$^{\mathrm{G}}$
CATF Win	13/20		17/20		20/20		16/20		16/20
Impr. (%)	+2.7		+7.4		+32.9		+28.8		+21.8

Dataset

$L_y$

[42] iTransformer

CATF-iTransformer

[60] TimesNet

CATF-TimesNet

[68] FEDformer

CATF-FEDformer

[61] Autoformer

CATF-Autoformer

[67] Informer

CATF-Informer

TimeMoE (zero-shot)

Weather

96

10.31 ± 0.07

10.05 ± 0.16

11.41 ± 0.25

10.23 ± 0.49

133.1 ± 42.16

14.26 ± 1.53

154.9 ± 88.36

15.97 ± 2.15

32.48 ± 8.31

16.52 ± 0.83

21.61

192

11.64 ± 0.07

11.04 ± 0.17

12.16 ± 0.27

11.30 ± 0.17

133.1 ± 64.82

18.50 ± 0.29

241.3 ± 233.3

14.93 ± 1.06

32.27 ± 12.88

15.93 ± 1.12

24.60

336

12.03 ± 0.05

11.43 ± 0.11

12.25 ± 0.08

11.72 ± 0.10

195.6 ± 75.97

18.37 ± 1.31

150.0 ± 121.0

12.79 ± 0.34

46.55 ± 63.01

16.70 ± 0.97

26.44

720

12.71 ± 0.08

11.74 ± 0.20

12.64 ± 0.17

11.92 ± 0.11

174.9 ± 58.48

20.08 ± 1.08

274.6 ± 233.7

12.26 ± 0.26

34.14 ± 6.19

18.16 ± 1.20

28.63

PM2.5-LA

96

3.91 ± 0.01

3.93 ± 0.00

3.91 ± 0.06

4.13 ± 0.19

5.77 ± 0.24

4.72 ± 0.07

6.80 ± 1.00

6.27 ± 0.26

5.07 ± 0.26

4.97 ± 0.22

4.41

192

4.20 ± 0.02

4.24 ± 0.02

4.18 ± 0.09

4.24 ± 0.09

6.19 ± 0.09

4.88 ± 0.05

6.02 ± 0.39

5.99 ± 0.22

5.21 ± 0.14

4.96 ± 0.05

4.92

336

4.71 ± 0.01

4.74 ± 0.04

4.55 ± 0.11

4.66 ± 0.09

6.18 ± 0.16

5.34 ± 0.17

6.22 ± 0.28

6.20 ± 0.10

5.29 ± 0.12

5.12 ± 0.06

5.42

720

5.76 ± 0.02

5.87 ± 0.03

5.58 ± 0.03

5.66 ± 0.14

6.54 ± 0.26

6.04 ± 0.10

8.03 ± 1.81

6.42 ± 0.18

5.66 ± 0.06

5.62 ± 0.07

5.93

PM2.5-RV

96

6.25 ± 0.04

6.16 ± 0.06

6.08 ± 0.17

6.00 ± 0.03

7.53 ± 0.23

6.62 ± 0.05

8.36 ± 0.74

8.13 ± 0.24

6.10 ± 0.19

6.01 ± 0.07

5.97

192

6.51 ± 0.05

6.44 ± 0.03

6.28 ± 0.14

6.20 ± 0.06

8.48 ± 0.28

7.11 ± 0.27

8.31 ± 0.57

8.29 ± 0.38

6.47 ± 0.16

6.48 ± 0.02

6.26

336

6.61 ± 0.05

6.63 ± 0.02

6.57 ± 0.04

6.42 ± 0.06

8.33 ± 0.34

7.22 ± 0.32

8.41 ± 0.11

8.35 ± 0.24

6.62 ± 0.07

6.55 ± 0.04

6.40

720

6.49 ± 0.04

6.50 ± 0.03

6.43 ± 0.03

6.47 ± 0.09

7.80 ± 0.50

6.99 ± 0.14

8.32 ± 1.72

7.78 ± 0.22

6.54 ± 0.26

6.29 ± 0.04

6.36

Bike Share

96

83.38 ± 0.60

78.50 ± 0.30

81.95 ± 1.47

80.21 ± 3.90

93.44 ± 2.26

84.74 ± 2.74

116.0 ± 11.46

105.4 ± 9.40

91.06 ± 5.20

88.51 ± 1.52

78.52

192

97.71 ± 0.59

93.95 ± 0.39

96.91 ± 1.97

94.42 ± 2.02

107.0 ± 3.25

95.92 ± 1.13

135.1 ± 5.61

127.2 ± 2.66

100.9 ± 3.61

102.9 ± 2.54

87.81

336

112.8 ± 0.36

108.7 ± 0.22

108.6 ± 1.48

106.6 ± 1.29

114.1 ± 4.96

106.1 ± 3.09

154.9 ± 7.83

146.9 ± 14.25

119.0 ± 2.82

138.5 ± 10.43

97.52

720

134.9 ± 0.39

133.1 ± 0.65

126.8 ± 2.72

129.9 ± 1.29

129.1 ± 4.01

127.0 ± 3.49

167.4 ± 3.80

156.7 ± 5.79

138.1 ± 1.73

150.3 ± 0.64

118.8

Appliances

96

47.50 ± 1.34

46.53 ± 1.02

55.55 ± 2.75

47.56 ± 1.28

64.61 ± 4.50

60.66 ± 1.47

92.33 ± 5.10

61.17 ± 2.73

53.78 ± 3.13

48.72 ± 3.82

51.78

192

47.72 ± 0.88

46.13 ± 0.76

56.96 ± 3.29

48.88 ± 2.91

58.98 ± 0.92

56.28 ± 1.67

79.03 ± 11.01

61.86 ± 2.15

55.81 ± 3.22

47.32 ± 1.67

51.50

336

46.35 ± 0.66

45.16 ± 0.60

56.96 ± 4.05

48.48 ± 2.11

60.44 ± 4.10

54.25 ± 0.91

69.15 ± 4.92

63.83 ± 5.33

52.47 ± 1.49

48.81 ± 2.05

52.54

720

46.41 ± 0.42

45.68 ± 0.28

53.16 ± 3.00

47.81 ± 1.60

62.84 ± 6.14

56.70 ± 0.74

85.42 ± 23.30

61.40 ± 3.46

53.19 ± 1.30

49.15 ± 1.73

55.05

CATF Win

14

14

20

20

16

—

Impr. % (mean)

+2.1

+3.9

+26.3

+26.4

+12.1

Instability of the Baselines on Weather

The target of Weather, the CO$_2$ concentration, contains 50 missing-value codes (−9999 ppm), which we keep as in the original data. All of them lie in the training split (February to August 2020), while the validation and test splits contain none. The regular measurements have a mean of 425 ppm and a standard deviation of 18.5 ppm in the training split; with the missing-value codes, the standard deviation of the training target increases to 359 ppm. Under global normalization, the regular measurements therefore occupy a very narrow range of the normalized space, and a small error in this space becomes a large error in ppm: an error of 0.5 in the normalized space corresponds to about 180 ppm, i.e., an MSE of about $3 \times10^{4}$, the order of magnitude observed for FEDformer and Autoformer Table 9. Under sample normalization, the input windows that contain such a value are dominated by it. A model that is pulled toward these extreme training targets therefore produces large errors on the clean test data, which explains the very large MSE and SD of FEDformer, Autoformer, and Informer on Weather. Their CATF-enhanced counterparts are trained on the same data but remain within the range of the strong baselines (§ 5.2).

Computational Cost

Table 16 reports the training time per epoch, the peak GPU memory, and the number of trained epochs of the baselines and the CATF-enhanced models. In each training iteration, CATF runs a forward pass of all $K$ workers to compute the pseudo ground truth, trains the manager, and updates the selected workers $\alpha= 2$ times; in addition, the parameters and optimizer states of all $K$ workers and the manager are kept in memory. As a result, a CATF-enhanced model requires a median of 5.8× the training time per epoch and 3.8× the peak memory of its standalone counterpart across all models and horizons, and it is trained for about 27 epochs instead of about 7. The relative overhead is largest for the lightweight iTransformer and smallest for the heavier FEDformer, Autoformer, and Informer. At inference time, only the manager and the selected worker are executed for each input.

Model	$L_y=96$ Time (s/ep.)	$L_y=96$ Mem. (GB)	$L_y=192$ Time (s/ep.)	$L_y=192$ Mem. (GB)	$L_y=336$ Time (s/ep.)	$L_y=336$ Mem. (GB)	$L_y=720$ Time (s/ep.)	$L_y=720$ Mem. (GB)	Epochs
iTransformer	17	0.04	19	0.04	33	0.09	23	0.05	7.1
CATF-iTransformer	283	0.46	239	0.46	398	0.48	250	0.48	26.7
TimesNet	76	0.13	95	0.15	728	0.20	205	0.29	7.0
CATF-TimesNet	725	0.57	559	0.65	2,227	0.79	922	1.00	26.1
FEDformer	301	1.62	329	1.78	1,443	2.07	499	2.68	6.7
CATF-FEDformer	1,688	5.96	1,888	6.57	2,226	7.43	1,460	9.78	26.5
Autoformer	172	2.45	193	3.83	1,912	5.63	485	11.07	6.9
CATF-Autoformer	1,571	9.32	1,859	14.73	2,998	21.69	1,358	43.17	26.9
Informer	100	0.78	167	0.94	1,234	1.23	300	1.85	7.1
CATF-Informer	884	2.60	1,413	3.22	2,240	4.07	1,149	6.36	27.1
Time-MoE (zero-shot)	994$^{\dagger}$	0.23	1,165$^{\dagger}$	0.23	767$^{\dagger}$	0.24	1,241$^{\dagger}$	0.26	—

Table 13. Long-term forecasting results under Sample normalization. MAE is computed on the original (de-normalized) scale and reported as MEAN ± SD over 5 runs (TimeMoE: single zero-shot run). The look-back length is 96 for all settings. The better of each baseline and its CATF-enhanced version is in bold; the best result in each row is marked with a dark orange background and the second best with a light orange background. CATF Win: number of the 20 settings (5 datasets × 4 horizons) in which CATF-<baseline> has a lower mean MAE than <baseline>. Impr.: relative MAE reduction of CATF-<baseline> over <baseline>, averaged over the 20 settings.

Dataset

$L_y$

iTransformer [42]

CATF-iTransformer

TimesNet [60]

CATF-TimesNet

FEDformer [68]

CATF-FEDformer

Autoformer [61]

CATF-Autoformer

Informer [67]

CATF-Informer

TimeMoE (zero-shot)

Weather

96

14.13 ± 0.67

9.28 ± 0.20

11.84 ± 0.57

8.61 ± 0.25

725.8 ± 476.1

9.45 ± 0.19

983.5 ± 593.3

10.04 ± 0.43

28.87 ± 8.79

8.53 ± 0.20

9.05

192

17.43 ± 1.09

10.89 ± 0.12

12.80 ± 0.83

10.71 ± 0.10

1,691 ± 2,008

11.22 ± 0.19

185.3 ± 305.8

12.06 ± 0.46

32.24 ± 6.16

10.65 ± 0.25

10.63

336

19.16 ± 0.73

11.76 ± 0.17

13.89 ± 1.76

11.44 ± 0.19

380.7 ± 239.9

12.32 ± 0.37

194.7 ± 123.2

12.12 ± 0.22

29.58 ± 7.54

11.47 ± 0.10

11.14

720

19.89 ± 0.54

13.34 ± 0.57

17.53 ± 2.17

12.36 ± 0.35

588.6 ± 274.4

12.52 ± 0.49

118.6 ± 83.34

13.18 ± 0.89

28.66 ± 4.30

12.62 ± 0.35

11.55

PM2.5-LA

96

3.93 ± 0.06

3.87 ± 0.07

4.02 ± 0.12

3.85 ± 0.06

5.21 ± 0.46

4.24 ± 0.12

4.89 ± 0.24

5.07 ± 0.20

4.33 ± 0.18

4.12 ± 0.10

4.10

192

4.25 ± 0.04

4.26 ± 0.03

4.26 ± 0.15

4.14 ± 0.06

5.45 ± 0.10

4.39 ± 0.12

5.13 ± 0.48

5.37 ± 0.31

4.40 ± 0.07

4.26 ± 0.07

4.41

336

4.77 ± 0.03

4.83 ± 0.04

4.52 ± 0.03

4.49 ± 0.10

6.00 ± 0.43

4.94 ± 0.18

5.79 ± 1.05

5.85 ± 0.15

5.09 ± 0.11

4.82 ± 0.19

4.98

720

5.78 ± 0.08

5.80 ± 0.06

5.59 ± 0.08

5.62 ± 0.13

6.55 ± 0.65

5.84 ± 0.08

8.94 ± 3.09

6.34 ± 0.15

5.85 ± 0.11

5.86 ± 0.15

6.08

PM2.5-RV

96

6.08 ± 0.05

6.14 ± 0.05

6.04 ± 0.09

6.05 ± 0.09

7.26 ± 0.33

6.57 ± 0.12

7.90 ± 0.29

7.98 ± 0.54

6.25 ± 0.11

6.11 ± 0.13

6.60

192

6.47 ± 0.03

6.50 ± 0.06

6.59 ± 0.22

6.26 ± 0.15

8.65 ± 0.11

6.83 ± 0.18

8.57 ± 0.58

8.67 ± 0.65

6.72 ± 0.09

6.52 ± 0.12

6.85

336

6.69 ± 0.08

6.64 ± 0.06

6.77 ± 0.11

6.52 ± 0.10

8.14 ± 0.45

7.33 ± 0.17

8.65 ± 0.51

8.28 ± 0.38

7.07 ± 0.20

7.05 ± 0.24

7.03

720

6.53 ± 0.04

6.53 ± 0.06

6.44 ± 0.04

6.46 ± 0.08

7.20 ± 0.42

6.94 ± 0.19

8.16 ± 0.85

7.64 ± 0.10

6.90 ± 0.08

6.98 ± 0.28

6.97

Bike Share

96

83.57 ± 0.47

79.44 ± 0.16

81.15 ± 2.75

78.20 ± 4.61

116.5 ± 7.39

81.16 ± 2.43

116.8 ± 9.46

112.2 ± 9.07

83.96 ± 1.69

80.78 ± 2.91

78.94

192

100.0 ± 0.56

95.97 ± 0.83

95.55 ± 1.81

93.79 ± 1.01

135.9 ± 6.82

97.54 ± 1.40

132.4 ± 5.62

134.8 ± 7.27

103.3 ± 2.98

103.5 ± 2.60

91.42

336

115.4 ± 0.40

112.0 ± 0.80

112.3 ± 1.14

112.0 ± 1.42

139.2 ± 4.26

113.6 ± 1.89

161.7 ± 9.95

143.1 ± 4.66

122.0 ± 2.51

143.6 ± 8.16

106.2

720

136.4 ± 0.59

137.1 ± 0.79

131.2 ± 3.55

132.6 ± 1.95

157.4 ± 5.82

137.4 ± 1.74

177.8 ± 8.56

170.8 ± 3.69

153.9 ± 2.76

156.8 ± 1.93

133.1

Appliances

96

74.04 ± 5.14

60.15 ± 2.18

70.47 ± 3.88

62.76 ± 6.36

80.90 ± 4.74

72.17 ± 6.41

159.6 ± 116.5

69.60 ± 5.08

94.24 ± 14.76

63.80 ± 4.27

54.71

192

167.30 ± 3.81

62.45 ± 4.41

71.11 ± 3.60

65.18 ± 1.96

93.68 ± 8.12

68.63 ± 9.13

160.5 ± 34.00

76.08 ± 5.17

95.50 ± 12.24

63.60 ± 5.92

54.24

336

72.78 ± 5.10

62.01 ± 2.70

72.69 ± 6.57

59.66 ± 5.79

117.1 ± 49.33

67.62 ± 5.90

151.8 ± 65.44

72.94 ± 4.62

91.48 ± 15.24

64.74 ± 8.34

54.76

720

74.95 ± 2.20

62.68 ± 4.62

68.41 ± 6.09

64.19 ± 2.82

118.9 ± 29.06

68.76 ± 9.08

148.9 ± 102.5

67.81 ± 7.98

95.17 ± 7.05

66.78 ± 3.87

57.15

CATF Win

13

20

6

14

15

—

Impr. % (mean)

+10.6

+7.7

+35.7

+31.8

+19.0

—

Ablation Study

Table 17 reports the full results of the ablation study in § 5.3 on PM2.5-RV, together with the same ablation on Bike Share and the number of parameters. On Bike Share, the default setting is again an independent re-run of the configuration in Table 1. For the mixture of workers, we combine the top-performing CATF-enhanced models of each dataset.

Setting	Bike Share MSE	Bike Share MAE	Bike Share $\Delta$ (%)	Bike Share $p$	PM2.5-RV MSE	PM2.5-RV MAE	PM2.5-RV $\Delta$ (%)	PM2.5-RV $p$	#Params (M)
Default (KL + Wass., reg., $d_{\mathrm{model}}=512$, $K=3$)	15,097 ± 241	78.56 ± 0.67	–	–	80.52 ± 1.02	6.18 ± 0.03	–	–	7.3
Manager loss
KL only	15,452 ± 282	79.36 ± 0.71	+2.4	0.065	81.87 ± 2.75	6.15 ± 0.03	+1.7	0.347	7.3
Wasserstein only	15,407 ± 372	80.48 ± 0.83	+2.1	0.163	81.56 ± 1.10	6.21 ± 0.04	+1.3	0.157	7.3
Regularization
No reg.	15,169 ± 184	79.01 ± 0.40	+0.5	0.609	80.94 ± 1.19	6.14 ± 0.03	+0.5	0.559	7.3
Manager arch.
Small ($d_{\mathrm{model}}=128$, $d_{ff}=512$, 4 heads)	15,155 ± 298	79.32 ± 0.77	+0.4	0.743	81.23 ± 0.78	6.19 ± 0.04	+0.9	0.252	1.1
Large ($d_{\mathrm{model}}=768$, $d_{ff}=3076$, 12 heads)	15,042 ± 167	78.21 ± 0.60	−0.4	0.691	81.05 ± 0.81	6.17 ± 0.05	+0.7	0.386	15.5
Worker number
$K=6$	15,321 ± 302	79.26 ± 1.09	+1.5	0.232	79.37 ± 1.14	6.08 ± 0.10	−1.4	0.132	7.9
$K=9$	15,467 ± 172	79.93 ± 0.75	+2.5	0.026	82.61 ± 2.99	6.11 ± 0.07	+2.6	0.200	8.6
Worker mix
Top-3 models1 ($K=3$)	17,067 ± 579	82.88 ± 1.15	+13.1	<0.001	81.63 ± 2.13	6.13 ± 0.04	+1.4	0.334	24.9
Top-2 models1 ×2 ($K=4$)	14,728 ± 837	79.98 ± 1.52	−2.4	0.390	80.04 ± 2.70	6.07 ± 0.06	−0.6	0.728	40.8

Details of the Case Study

This section describes the analysis methods (Appendix § 12.§ 1), validates them on synthetic data with known regimes (Appendix § 12.§ 2), gives further results of the Bike Share case study in § 6 (Appendix § 12.§ 3), and repeats the analysis on PM2.5-RV (Appendix § 12.§ 4).

Analysis Methods

All analyses use CATF-iTransformer ($L=L_y=96$, global normalization, $K=3$). The figures show a single trained model; quantitative results are averaged over all runs (3 on the synthetic data, 5 on Bike Share).

Routing by context. We compare the Worker selected for each sample with a property of its context, e.g., the true regime or the day of the week, and quantify the agreement with the normalized mutual information (NMI) between the two labelings, $\operatorname{NMI}(A,B)=2I(A;B)/(H(A)+H(B))$, where $I$ is the mutual information and $H$ the entropy. The NMI is 1 if the two labelings match one-to-one and 0 if the routing is independent of the property.

Permutation importance [10]. For one variable at a time, we replace its input window in each test sample with that of a randomly chosen other test sample, while all other variables stay unchanged. This is done in the Manager input only; the Workers still receive the original input. We record the share of test samples whose selected Worker changes, averaged over three random shuffles. “All context” shuffles all context variables at once.

Sanity check. Following the model-randomization test of Adebayo et al. [2], we repeat the analyses with an untrained Manager of the same architecture. A pattern that also appears for the untrained Manager reflects the input data rather than what the Manager has learned.

Intervention. This analysis is performed during inference. We keep the trained Workers unchanged (they are not retrained) and, when forecasting the test set, replace only the learned routing with (a) a random assignment with the same Worker shares, (b) the best single Worker for all samples, and (c) the untrained Manager. If the routing is meaningful, each replacement should increase the MSE. We also record how often the Manager selects the Worker with the lowest error on a sample, and, for the samples routed to each Worker, the MSE of every Worker relative to the lowest one, which shows whether the Workers specialize.

Counterfactual (Bike Share). We change selected context variables in the Manager input and keep all other inputs fixed. The calendar variables (the day of the week and the working-day indicator) are changed in their original units and then normalized as in training. When the day of the week of the input window is shifted by $d \in\{1,\ldots,6\}$ days, the working-day indicator is recomputed accordingly. The typical Worker of a day is the majority Worker of the real forecasts made on that day at the same hour. Unlike the two previous analyses, this tests whether a variable causes the routing decision rather than whether it is associated with it.

Synthetic Data: Recovering Known Regimes

On real data, the correct assignment of samples to Workers is unknown, so an interpretation can only be checked indirectly, e.g., against domain knowledge. We therefore construct a synthetic dataset in which the regime of the target is determined by the context, so that the interpretation can be compared with the ground truth: if CATF is context-aware, the routing should follow the regimes, the Manager should rely on the variable that announces the regime, and each Worker should handle one regime.

Data. The dataset contains $T = 20{,}000$ hourly steps $t = 0,\ldots,T - 1$ and is generated in three steps.

(1) Regime sequence. We first draw a regime $r_t \in\{1,2,3\}$ for every step: the time axis is split into consecutive segments whose lengths are drawn uniformly from 72 to 240 hours (3 to 10 days), and each segment is assigned a regime drawn uniformly from the two regimes that differ from the previous segment. The regime sequence is random and does not depend on any variable; each regime covers about one third of the steps.

(2) Target. The target is generated from the regime of the current step:

where $a_t = 0.97a_{t-1} + \eta_t$, $\eta_t \sim\mathcal{N}(0,2^2)$, $a_0 = 0$, is a first-order autoregressive (AR(1)) process, which drifts slowly around zero without any periodicity. The three regimes thus have clearly different dynamics: one peak per day, two peaks per day, and no cycle.

(3) Context. There are six context variables. Only one of them, the regime signal, carries information about the regime: it equals the regime 48 hours ahead plus small Gaussian noise,

similar to a forecast of an upcoming condition. It does not enter the generation of the target; the target and the regime signal are only linked through the regime sequence, the target through the current regime $r_t$ and the regime signal through the regime 48 hours ahead. The other five context variables are uninformative about the regime: four independent random walks $z_t^{(k)} = z_{t-1}^{(k)} + \zeta_t^{(k)}$, $\zeta_t^{(k)} \sim\mathcal{N}(0,1)$, $k = 1,\ldots,4$, and the hour of the day encoded as $h_t = \sin(2\pi(t \bmod24)/24)$. The hour of the day can help to predict the values of the two periodic regimes, but it does not indicate which regime is active.

Task and ground truth. As in the main experiments, the model receives the past $L = 96$ hours of the target and the six context variables and forecasts the next $L_y = 96$ hours of the target. The true regime of a sample is the regime that covers most of its forecast window. The target history reveals the current regime, but not an upcoming switch; the regime signal announces a switch 48 hours in advance. A context-aware Manager should therefore route the samples by regime and rely on the regime signal. We use the same splits (80/10/10), global normalization, and hyperparameters as for CATF-iTransformer in the main experiments, and run each model 3 times.

Does the routing recover the regimes? Figure 5 shows the test series together with the true regime and the selected Worker: the selected Worker switches whenever the regime switches, and the routing agrees with the regime for 99.4% of the test samples. Over all runs, 99%, 94%, and 94% of the samples of the three regimes are sent to one Worker each (Figure 6a), and the NMI between routing and regime is 0.85. An untrained Manager with the same input reaches an NMI of only 0.04 (Figure 6b).

Does the Manager use the right variable? Shuffling the regime signal in the Manager input changes 65% of the routing decisions, while shuffling any other variable, including the target history, changes at most 5.3% on average (Figure 6d). For the untrained Manager, the regime signal is no more important than the distractors. Inspecting the Manager alone therefore reveals which context variable drives the regimes, without access to the data-generating process.

What has each Worker learned? $W_1$ and $W_2$ each specialize in one of the two periodic regimes, and $W_3$ absorbs the drift regime: on the daily cycle, the MSE of $W_1$ is 17.9 while that of the other Workers is 58–61; on the half-day cycle, the MSE of $W_2$ is 14.4 while that of the other Workers is 58–62 (Figure 6c). The drifting regime is not predictable 96 hours ahead, and all Workers have a high MSE on it (64–75); routing it to a separate Worker keeps it out of the training data of $W_1$ and $W_2$.

Does the learned routing improve the forecasts? With the same trained Workers, the learned routing (MSE 33.4) is much better than random routing (50.7), the best single Worker (43.8), and an untrained Manager (58.1), and CATF-iTransformer reduces the MSE of the standalone iTransformer by 7.2% (33.4 vs. 36.0; Table 18, which also gives the standard deviations over the 3 runs).

Model / routing	MSE	NMI
iTransformer (baseline)	35.98 ± 0.34	–
CATF-iTransformer	33.39 ± 0.51	0.85 ± 0.15
Same trained workers, different routing
Random routing (same worker shares)	50.69 ± 4.18	–
Best single worker for all samples	43.77 ± 1.76	–
Untrained manager	58.13 ± 6.06	0.04 ± 0.04

In summary, when the regime depends on the context, the routing analysis recovers the true structure of the data: which samples belong together (the regimes), which variable determines them (the regime signal), and what each Worker has learned (one periodic regime each for $W_1$ and $W_2$, the drift regime for $W_3$).

Bike Share: Additional Results

Routing accuracy. The Manager selects the most accurate Worker for 54% of the test samples, compared with 36% for the untrained Manager and 33% by chance. On the samples routed to a Worker, that Worker has the lowest MSE in 80% of the cases.

Counterfactual analysis. Figure 7 shows the counterfactual analysis of § 6 for each shift $d$. Shifting the calendar changes 35% of the routing decisions for $d = 1$ or 6 and about 70% for $d = 3$ or 4: a shift by one day often stays within the days handled by the same Worker, whereas a shift by half a week moves most forecasts to the days of another Worker. For every $d$, 78–82% of the decisions after the shift match the typical Worker of the new day, compared with 23–60% for the original routing. Replacing the target history instead changes only 8–13% of the decisions, and the routing does not follow the day from which the history is taken: its match with the typical Worker of that day is the same as for the original routing. The Manager therefore routes by the calendar rather than by the shape of the recent demand.

Consistency across runs. All five runs partition the samples by the day of the week (NMI between the selected Worker and the day of the week 0.13–0.48, vs. at most 0.10 for the hour of the day), although the boundaries between the Workers differ between runs. In the counterfactual analysis, the share of new decisions that match the typical Worker of the shifted day is 64–87% across runs (40% before the shift), and replacing the target history changes 5–23% of the decisions; both results hold in all five runs.

Routing over time and holidays. Figure 8 shows the routing of one run over four weeks of the test period. The routing follows the weekly cycle and does not depend on the hour of the day (NMI 0.00). Thanksgiving (Thursday, November 22, 2012), on which demand drops to a weekend level, is routed entirely to $W_3$, the Worker for Saturdays, whereas 86% of the forecasts on regular Thursdays are routed to $W_2$. The counterfactual analysis confirms that this decision is caused by the holiday indicators: setting the holiday and working-day indicators of Thanksgiving to those of a regular Thursday moves 79% of its forecasts to $W_2$, and marking regular Thursdays as holidays moves 64% of their forecasts to $W_3$ (from 8%). This behavior, however, is specific to this run: in three of the other four runs, Thanksgiving is routed mostly like a regular Thursday. With only a handful of holidays in the training data, the Manager does not consistently learn a holiday regime, in contrast to the weekly cycle.

Context of the Workers. Figure 9 shows the context of the samples routed to each Worker around the forecast origin. To compare the variables on a common scale, we compute, at each hour, the range of the three Worker means divided by the standard deviation of the variable over the test windows, and average it over the 192 hours. The Workers differ most in the variable that drives the routing, the working-day share (1.2 standard deviations), and accordingly face different demand trajectories (0.4). For the weather variables, which have low importance, the differences are much smaller than for the working-day share (temperature 0.4, humidity 0.4, wind speed 0.3, mist/rain 0.2; mean temperature within 2 °C, humidity within 10%). Figure 10 relates the routing to every input variable at the forecast origin. Only the day of the week (NMI 0.44) and the working-day indicator (0.15) are strongly related to the routing; for all other variables, the NMI is at most 0.05. Some weather bins deviate from the overall Worker shares, e.g., 67% of the forecasts made during rain or snow go to $W_1$ (42% overall), but these bins are small and fall mostly on particular days: in the test period, most rainy hours occur on Sunday to Tuesday, the days of $W_1$. Together with the counterfactual analysis, this indicates that such deviations reflect the calendar rather than the weather itself.

Example forecasts. Figure 11 compares typical forecasts of the three Workers with those of the standalone iTransformer. All three Workers capture the double commuting peak on working days and the single midday peak on weekends. The largest difference occurs for the forecast starting on Saturday: the standalone model underestimates the midday weekend peak, whereas $W_3$ captures it and reduces the MSE from 12,035 to 7,797.

PM2.5-RV

We repeat the analysis of § 6 for the five runs of CATF-iTransformer on PM2.5-RV (global normalization, $L = L_y = 96$). Here, the routing has little effect. With the same trained Workers, the learned routing (MSE 80.5 ± 1.0) is only slightly better than random routing with the same Worker shares (81.1 ± 0.7; better in four of five runs) and an untrained Manager (81.6 ± 1.2), and it is on par with the best single Worker (80.4 ± 1.0). The Manager selects the most accurate Worker for 36% of the test samples (untrained Manager: 34%), and on the samples routed to a Worker, the other Workers are at most 5% less accurate, i.e., the Workers barely specialize. Shuffling all context variables in the Manager input changes 47% of the routing decisions, compared with 41% for the untrained Manager, and no single variable stands out (19–27% vs. 13–21%). Unlike bike demand, which follows distinct patterns on working days and weekends, PM2.5 concentration varies with continuously changing meteorological conditions, so there may be no discrete context-driven regimes for the Workers to specialize in. The routing analysis therefore also indicates whether a dataset contains such regimes.