betaFeedback

WorksMathematics

Continuous Horizon Forecasting

Abstract

Multi-horizon forecasting has been dominated by discrete forecast strategies that treat model outputs as fixed-vectors or recurrently create them. In this paper, we introduce the Continuous Horizon (CH) strategy that generalizes upon recursive, direct, and multi-step strategies by redefining forecasts as a continuous operator and the forecast horizon as a query variable rather than an index. We present the theoretical foundations for the CH strategy, a Universal Operator Approximation Theorem, and a continuous-horizon bias-variance decomposition that formally proves the estimation variance improvements of CH over the discrete strategies. We validate the CH strategy empirically through a simulation study, and on 16 large-scale datasets from the M-series benchmark, showing how CH achieves competitive RMSE accuracy, while uniquely enabling parallel inference, targeted horizon querying, and interpolation and extrapolation across unseen horizons.

Introduction

Multi-horizon forecasting has been dominated by discrete strategies: multi-step (MS; [7, 4]), direct (DIR; [15]) and recurrent (REC) [6]. Although tree-based machine learning models [35], neural architectures [49, 10, 44], and foundation models [20, 18, 2, 64, 26] have advanced rapidly, multi-horizon forecast strategies have remained fixed for nearly a decade: multi-step fixed-grid outputs or recurrent rollouts [54, 57], with their known trade-offs in error accumulation, output dimension, and inability to query arbitrary horizons [5].

PropertyRecursiveDirectMulti-StepContinuous Horizon
Low Bias✗✓✓✓
Low Variance✗✗✗✓
Inference parallelism✗✓✓✓
Targeted horizon querying✗✓✗✓
Unseen Horizon inter/extrapolation✗✗✗✓
Joint horizon modeling✗✗✓✓

Table 1. Desirable properties of forecasting strategies.

We argue that this discrete view of multi-horizon strategies is a historical artifact rather than a necessity. Time series forecasting has long recognized future trajectories as inherently continuous phenomena [51, 33, 12, 34]. In this paper, we elevate this continuous data approach into a forecast strategy: Continuous Horizon (CH). We reveal that discrete strategies are coarse discretizations of CH’s continuous-horizon forecast operator, we unlock theoretical understanding and showcase the strategy’s empirical advantages:

  1. Universal Approximation & Unseen Horizons: We theoretically and empirically show that the Continuous Horizon (CH) strategy can approximate any continuous function across arbitrary horizons, including fractional horizons and those unseen during training.

  1. Continuous Bias-Variance Dominance: We extend the discrete MSE decomposition framework of Ben Taieb and Atiya [5] to continuous time, formally proving how CH leverages continuous effective horizon information to reduce estimation variance over traditional multi-step (MS) and direct (DIR) strategies.

  1. Accuracy, Computational Efficiency & Parallelism: CH achieves competitive accuracy results on the M-series benchmark (16 large-scale datasets), while enabling flexible inference that supports both targeted single-horizon queries and batched parallel inference.

The remainder of this paper is structured as follows. § 2 introduces the multi-horizon forecasting notation. § 3 presents the theoretical foundations of the CH strategy. § 4 conducts simulation of the strategies’ bias-variance behavior. § 5 investigates the strategies’ horizon interpolation and extrapolation capabilities. § 6 presents the empirical results of the M-series and inference computational complexity analysis. § 7 hosts the discussion and § 8 concludes.

Figure 1. Multi-horizon forecast strategies and the continuous-horizon paradigm.

Multi-Horizon Forecasting

Here we introduce general multi-horizon forecasting task notation. Let the forecast creation date be denoted by the index tt. We denote the functional random variable as YY, we denote its historical discrete observations at FCD tt as y:t=[y1,y2,…,yt]\mathbf{y}_{:t} = [y_1, y_2, \ldots, y_t], for a single horizon hh the variable of interest is the future behavior of Yt+hY_{t+h}, for HH horizons Yt+1:t+H=[Yt+1,Yt+2,…,Yt+H]∈RH\mathbf{Y}_{t+1:t+H} = [Y_{t+1}, Y_{t+2}, \ldots, Y_{t+H}] \in\mathbb{R}^{H}.

Following the functional time series view [33, 51], we assume that the target variable comes from a continuous process:

Yt+h=f(h∣y:t)+ϵt+h,where f(⋅∣y:t)∈C([0,H])(1)Y_{t+h} = f(h \mid\mathbf{y}_{:t}) + \epsilon_{t+h}, \qquad\text{where } f(\cdot\mid\mathbf{y}_{:t}) \in C([0,H]) \tag*{(1)}

with ff a continuous data generating process (DGP), and ϵt+h\epsilon_{t+h} is a zero-mean noise term. The DGP coincides with the conditional expectation, E[Yt+h∣y:t]=f(h∣y:t)=μt+h∣t\mathbb{E}[Y_{t+h} \mid\mathbf{y}_{:t}] = f(h \mid\mathbf{y}_{:t}) = \mu_{t+h \mid t}.

We denote the hh-step ahead forecast y^t+h=g(h∣y:t)\hat{y}_{t+h} = g(h \mid\mathbf{y}_{:t}), where gg is the model. We denote vector outputs y^t+1:t+H=[y^t+1,y^t+2,…,y^t+H]∈RH\hat{\mathbf{y}}_{t+1:t+H} = [\hat{y}_{t+1}, \hat{y}_{t+2}, \ldots, \hat{y}_{t+H}] \in\mathbb{R}^{H}. The model’s input handling and the structure of its forecast output y^t+1:t+H\hat{\mathbf{y}}_{t+1:t+H} define the overarching multi-horizon forecasting strategy.

Multi-Horizon Forecast Strategies

Inspired by the taxonomy of Ben Taieb et al. [6], we distinguish the primary variants of the multi-horizon forecast strategy: recursive, direct, multi-step and continuous horizon.

  • Recursive (REC): A single one-step-ahead model, that is, iterated H-times, by feeding each prediction back as input for the next step. It is a simple and parameter efficient approach that suffers from error accumulation.

y^t+h=g(1∣y:t,y^t+1,…,y^t+h−1),h=1,…,H.(2)\hat{y}_{t+h} = g\left(1 \mid\mathbf{y}_{:t}, \hat{y}_{t+1}, \ldots, \hat{y}_{t+h-1}\right), \qquad h = 1, \ldots, H. \tag*{(2)}
  • Direct (DIR): H separate models are trained, one per horizon. Each model directly predicts the value of step h without dependencies on intermediate predictions. It avoids error accumulation, but assumes conditional independence and is less statistically efficient.

y^t+1:t+H=[g1(y:t),…,gH(y:t)].(3)\hat{\mathbf{y}}_{t+1:t+H} = \left[g_1\left(\mathbf{y}_{:t}\right), \ldots, g_H\left(\mathbf{y}_{:t}\right)\right]. \tag*{(3)}
  • Multi-Step (MS): Also known as multiple-input multiple output. A single model outputs all H-forecasts simultaneously as a vector. It preserves the stochastic dependencies between horizons, as it does not assume conditional independence.

y^t+1:t+H=g(y:t)∈RH.(4)\hat{\mathbf{y}}_{t+1:t+H} = g\left(\mathbf{y}_{:t}\right) \in\mathbb{R}^{H}. \tag*{(4)}
  • Continuous Horizon (CH): A single model produces the forecast on horizon h through a continuous function conditioned on h.

y^t+h=g(h∣y:t),g(⋅∣y:t)∈C([0,H]),g(h∣y:t)=∑k=1Kθk(y:t)ϕk(h).(5)\begin{aligned} \hat{y}_{t+h} &= g\left(h \mid\mathbf{y}_{:t}\right), \qquad g\left(\cdot\mid\mathbf{y}_{:t}\right) \in C([0,H]), \\ g\left(h \mid\mathbf{y}_{:t}\right) &= \sum_{k=1}^{K} \theta_k\left(\mathbf{y}_{:t}\right)\phi_k(h). \tag*{(5)} \end{aligned}

The Continuous Horizon strategy generalizes the other three: recursive, direct, and multi-step forecasts are all discretizations of a continuous horizon operator, obtained by evaluating an operator at fixed integer grid points or by iterating a one-step map. Redefining the forecast from a discrete mapping to an operator that returns a forecast function, that can be thought as the asymptotic limit of a forecast horizon grid. In this work we use a flexible bilinear parametrization of CH, similar to that of the Hyndman and Shahid Ullah [33] and the branch-trunk architectures [39].

Theoretical Foundations

Parameter Learning

Estimation of a forecasting model gg is carried out according to

w^:=arg⁡min⁡w1T∑t=1T∫0HL(yt+h,g(h∣y:t,w)) dρ(h).(6)\hat{\mathbf{w}} := \arg\min_{\mathbf{w}} \frac{1}{T}\sum_{t=1}^{T}\int_{0}^{H} L(y_{t+h}, g(h \mid\mathbf{y}_{:t}, \mathbf{w}))\,\mathrm{d}\rho(h). \tag*{(6)}

where the forecaster parameters w\mathbf{w} are selected to minimize loss LL. Strategies can be identified by a way to select horizons on which to train. We can describe a common learning objective using a horizon-selector measure ρ\rho on the horizon interval [0,H][0,H] that specifies the lead times entering the objective: a point mass δh\delta_h for the REC and DIR strategies, the empirical measure 1H∑j=1Hδhj\frac{1}{H}\sum_{j=1}^{H}\delta_{h_j} for MS, and the uniform measure 1H dh\frac{1}{H}\,\mathrm{d}h for the CH strategy.

Mean Squared Error Decomposition Analysis

A deep understanding of the performance of forecasting strategies can be gained through the analysis of the mean squared error decomposition into bias and variance components.

Lemma 1 (MSE decomposition). The forecast MSE at horizon hh can be decomposed as

MSE⁡h=Ey:t[Eϵ,D[(yt+h−g(h∣y:t,w^))2∣y:t]⏟MSE⁡h(y:t)]\operatorname{MSE}_h = \mathbb{E}_{\mathbf{y}_{:t}}\left[\underbrace{\mathbb{E}_{\epsilon,\mathcal{D}}\left[(y_{t+h}-g(h\mid\mathbf{y}_{:t},\hat{\mathbf{w}}))^2\mid\mathbf{y}_{:t}\right]}_{\operatorname{MSE}_h(\mathbf{y}_{:t})}\right]
=Ey:t[Eϵ[(yt+h−μt+h∣t)2∣y:t]⏟Nh(y:t)+(μt+h∣t−ED[g(h∣y:t,w^)])2⏟Bh(y:t)= \mathbb{E}_{\mathbf{y}_{:t}}\left[\underbrace{\mathbb{E}_{\epsilon}\left[(y_{t+h}-\mu_{t+h\mid t})^2\mid\mathbf{y}_{:t}\right]}_{N_h(\mathbf{y}_{:t})} + \underbrace{\left(\mu_{t+h\mid t}-\mathbb{E}_{\mathcal{D}}[g(h\mid\mathbf{y}_{:t},\hat{\mathbf{w}})]\right)^2}_{B_h(\mathbf{y}_{:t})}\right.
+ED[(g(h∣y:t,w^)−ED[g(h∣y:t,w^)])2∣y:t]⏟Vh(y:t)].(7)\left. + \underbrace{\mathbb{E}_{\mathcal{D}}\left[(g(h\mid\mathbf{y}_{:t},\hat{\mathbf{w}})-\mathbb{E}_{\mathcal{D}}[g(h\mid\mathbf{y}_{:t},\hat{\mathbf{w}})])^2\mid\mathbf{y}_{:t}\right]}_{V_h(\mathbf{y}_{:t})}\right]. \tag*{(7)}

This decomposition separates the error at each horizon into an irreducible noise term NhN_h, a squared bias term BhB_h, and an estimation variance term VhV_h. The noise NhN_h is the conditional variance of yt+hy_{t+h} around the optimal forecast μt+h∣t\mu_{t+h\mid t}; it is irreducible, set by the DGP rather than the model, and typically grows with the horizon. The squared bias BhB_h measures how far the average fitted forecast ED[g(h∣y:t,w^)]\mathbb{E}_{\mathcal{D}}[g(h\mid\mathbf{y}_{:t},\hat{\mathbf{w}})] sits from μt+h∣t\mu_{t+h\mid t}, and reflects the model’s lack

of flexibility to represent the target variable. The estimation variance VhV_h measures the sensitivity of the fitted forecast to the particular training samples, and shrinks as the effective number of observations informing w^\hat{\mathbf{w}} grows.

Lemma 2 (Bias decomposition). The squared bias decomposes as

Bh=Ey:t[(μt+h∣t−ED[g(h∣y:t,w^)])2]B_h = \mathbb{E}_{\mathbf{y}_{:t}}\left[\left(\mu_{t+h|t} - \mathbb{E}_{D}[g(h \mid\mathbf{y}_{:t}, \hat{\mathbf{w}})]\right)^2\right]
=Ey:t[(μt+h∣t−g(h∣y:t,w⋆)⏟approximation+g(h∣y:t,w⋆)−ED[g(h∣y:t,w^)]⏟estimation)2].(8)= \mathbb{E}_{\mathbf{y}_{:t}}\left[\left(\underbrace{\mu_{t+h|t} - g(h \mid\mathbf{y}_{:t}, \mathbf{w}^{\star})}_{\text{approximation}} + \underbrace{g(h \mid\mathbf{y}_{:t}, \mathbf{w}^{\star}) - \mathbb{E}_{D}[g(h \mid\mathbf{y}_{:t}, \hat{\mathbf{w}})]}_{\text{estimation}}\right)^2\right]. \tag*{(8)}

The approximation term measures the discrepancy between the optimal forecast μt+h∣t\mu_{t+h|t} and the best forecast attainable by the model, independent of any estimation of a finite-sample; the estimation term measures how the training-average estimator departs from that best-in-class forecast under a finite sample.

Model selection trades bias BhB_h against variance VhV_h; the two results below show how the Continuous Horizon strategy controls each in turn.

Theorem 1 (Universal Operator Approximation Theorem). Let Y⊂Rt\mathcal{Y} \subset\mathbb{R}^{t} be a compact input domain of historical observations y:t∈Y\mathbf{y}_{:t} \in\mathcal{Y} and [0,H][0, H] be a compact forecast horizon interval. Let the DGP process of the target variable be governed by a continuous operator such as F:Y→C([0,H])F : \mathcal{Y} \to C([0, H]), with F(y:t)=f(⋅∣y:t)F(\mathbf{y}_{:t}) = f(\cdot\mid\mathbf{y}_{:t}). Then for any accuracy ϵ>0\epsilon> 0, there exists a neural network approximation g(h∣y:t,w⋆)=∑k=1Kθk(y:t)⋅ϕk(h)g(h \mid\mathbf{y}_{:t}, \mathbf{w}^{\star}) = \sum_{k=1}^{K}\theta_k(\mathbf{y}_{:t}) \cdot\phi_k(h), composed of a finite set of continuous neural basis functions {ϕk(h)}k=1K\{\phi_k(h)\}_{k=1}^{K} and coefficient mappings {θk(y:t)}k=1K\{\theta_k(\mathbf{y}_{:t})\}_{k=1}^{K}, such that:

∫0H(f(h∣y:t)−g(h∣y:t,w⋆))2 dh\int_{0}^{H}\left(f(h \mid\mathbf{y}_{:t}) - g(h \mid\mathbf{y}_{:t}, \mathbf{w}^{\star})\right)^2 \, \mathrm{d}h
=∫0H(f(h∣y:t)−∑k=1Kθk(y:t)⋅ϕk(h))2 dh<ϵ,∀y:t∈Y.(9)= \int_{0}^{H}\left(f(h \mid\mathbf{y}_{:t}) - \sum_{k=1}^{K}\theta_k(\mathbf{y}_{:t}) \cdot\phi_k(h)\right)^2 \, \mathrm{d}h < \epsilon,\quad\forall\mathbf{y}_{:t} \in\mathcal{Y}. \tag*{(9)}

Theorem 1 resembles the operator approximation results of Chen and Chen [14], which also support the DeepONet [39] and Fourier neural operators [38], and the neural basis approximation theorem of NHITS [10]. We include Theorem 1 to show that the separable bilinear decoder in Eq. (5) loses no expressiveness and to bound the approximation term of BhB_h (Lemma 2). Proof available in Appendix § 9.

Theorem 2 (Continuous Horizon Variance Dominance). Consider residuals (yt+h−y^t+h)(y_{t+h} - \hat{y}_{t+h}) and let K(h,h′)∈[0,1]K(h,h') \in[0,1] be a stationary, non-increasing, strictly convex kernel that describes the residual covariance at horizons h,h′h,h', so that K(h,h′)=K(∣h−h′∣)K(h,h') = K(|h-h'|). For the multi-step discrete forecast grid [1,2,…,H]⊂[0,H][1,2,\ldots,H] \subset[0,H], we define the effective horizon information ratio as:

IeffMS=H2∑h=1H∑h′=1HK(h,h′)andIeffCH=H2∫0H∫0HK(h,h′) dhdh′.(10)\mathcal{I}_{\mathrm{eff}}^{\mathrm{MS}} = \frac{H^2}{\sum_{h=1}^{H}\sum_{h'=1}^{H}K(h,h')} \qquad\text{and} \qquad\mathcal{I}_{\mathrm{eff}}^{\mathrm{CH}} = \frac{H^2}{\int_{0}^{H}\int_{0}^{H}K(h,h')\,\mathrm{d}h\mathrm{d}h'}. \tag*{(10)}

Then, the effective horizon information of the CH, MS, and DIR strategies obeys the inequality:

1=IeffDIR<IeffMS<IeffCH.(11)1 = \mathcal{I}_{\mathrm{eff}}^{\mathrm{DIR}} < \mathcal{I}_{\mathrm{eff}}^{\mathrm{MS}} < \mathcal{I}_{\mathrm{eff}}^{\mathrm{CH}}. \tag*{(11)}

Consequently, for a matched capacity of shared parameters PP, the estimation variance satisfies the following:

VCH(h)<VMS(h)<VDIR(h),∀h∈[0,H],(12)V_{\mathrm{CH}}(h) < V_{\mathrm{MS}}(h) < V_{\mathrm{DIR}}(h), \quad\forall h \in[0,H], \tag*{(12)}

The proof of the variance dominance theorem is available in Appendix B.

Bias and Variance Analysis by Simulation

To validate Theorem 2, we do two simulation studies.

Controlled Basis Experiment.

We replace the learned neural components with a fixed basis of polynomials and harmonics, and we use a Gaussian process as the DGP. Thus, the experiment measures only the estimation variance. Figure 2 shows that, with the matched parameters, CH is confirmed to have a lower estimation variance V(h)V(h) than MS and DIR on all horizons.

Figure 2. Effective information and estimation variance for different MS resolution and forecast horizon. The estimation variance of CH dominates both MS and DIR.

Classical Autoregressive DGPs.

The second study uses the DGPs and the Monte Carlo protocol of Ben Taieb and Atiya [5]. We use an MQForecaster [63] with a CH decoder. The model must learn both the coefficient mappings and the basis functions ϕ^k(h)\hat{\phi}_{k}(h). We use three autoregressive processes: a linear autoregression (LAR; 9), a nonlinear autoregression (NLAR; 43, 37), and a smooth-transition autoregression (STAR; 19, 59). Appendix § 11 gives the details of the DGPs.

Figure 3(a) shows the MSE decomposition of the LAR process for different training lengths TT. On this linear process, MS, DIR, and CH have a bias near zero. DIR has a high estimation variance, because it fits independent parameters for each horizon. MS decreases the variance, because it shares parameters across a discrete grid of horizons. CH has the lowest estimation variance and the lowest total error at all horizons hh. Appendix § 11.§ 2 and Appendix § 11.§ 3 repeat the bias–variance analysis for NLAR and STAR.

Figure 3. (a) MSE decomposition for a LAR DGP. (b) Bias–variance components across noise levels for NLAR, STAR, and LAR processes. CH improves the bias–variance tradeoff compared to MS and DIR strategies, an advantage that grows in higher-noise regimes. The dashed Naive-1 reference (bias2+variancebias^{2}+\mathrm{variance} row) bounds the useful noise range: the learned strategies stay below it where the process is predictable; its collapse toward the irreducible noise floor marks the loss of predictability.

Figure 3(b) shows the bias–variance components when we increase the observation noise, which is a proxy for a weaker correlation between horizons. When noise increases, the estimation variance of DIR and MS increases rapidly. The shared continuous basis of CH regularizes the fit across horizons. At the highest noise level, CH has approximately 40 to 55% lower variance than DIR on all three DGPs. CH also has the lowest total error on NLAR and STAR, but on LAR, REC has a lower total error.

Interpolation and Extrapolation to Unseen Horizons

Discrete strategies such as MS and DIR give forecasts only at fixed horizons. CH can query its basis functions ϕk(h)\phi_k(h) at any real hh. Thus, CH can interpolate at fractional horizons, for example h=πh = \pi, and it can extrapolate to unseen horizons. Of the discrete strategies, only the recurrent rollout of REC can also extrapolate. To show these capabilities, we simulate continuous-time Gaussian processes in five persistence regimes: mean-reverting, aperiodic, periodic, and slow and fast quasi-periodic (Figure 4, Appendix § 12).

Figure 4. Top: A sampled history (gray) and the exact posterior mean with its 10–90% band. The dotted line shows the end of the training grid (h=12h = 12). Bottom: The RMSE regret at each horizon (Eq. (14)). A regret of zero is optimal. CH has the lowest regret on almost all seen and unseen horizons. MS and DIR are shown only on their training grid.

We train the MQForecaster model on the integer horizons h≤12h \le12. We evaluate it on three sets of horizons: the seen integers, the unseen half-integers (interpolation), and the unseen integers up to h=24h = 24 (extrapolation). For the half-integer horizons, MS and DIR use linear interpolation between their two adjacent output horizons. We measure the accuracy with the RMSE and with the regret relative to the exact analytical posterior.

RMSE⁡(y,y^)=1T×∣H∣∑t=1T∑h∈H(yt+h−y^t+h)2(13)\operatorname{RMSE}(\mathbf{y}, \hat{\mathbf{y}}) = \sqrt{\frac{1}{T \times|\mathcal{H}|} \sum_{t=1}^{T} \sum_{h \in\mathcal{H}} (y_{t+h} - \hat{y}_{t+h})^{2}} \tag*{(13)}
Regret⁡(y,y^,μ)=RMSE⁡(y,y^)−RMSE⁡(y,μt+h∣t)(14)\operatorname{Regret}(\mathbf{y}, \hat{\mathbf{y}}, \boldsymbol{\mu}) = \operatorname{RMSE}(\mathbf{y}, \hat{\mathbf{y}}) - \operatorname{RMSE}(\mathbf{y}, \boldsymbol{\mu}_{t+h|t}) \tag*{(14)}

where H\mathcal{H} is a set of uniformly spaced query horizons, and μt+h∣t=f(h∣y:t)\boldsymbol{\mu}_{t+h|t} = f(h \mid\mathbf{y}_{:t}) is the exact conditional mean of § 2. For the seen and the interpolation sets, H⊂[0,H]\mathcal{H} \subset[0, H]. For the extrapolation set, H⊂(H,2H]\mathcal{H} \subset(H, 2H].

Figure 4 (bottom) and Table 2 show that CH has the lowest error on all five processes and on all three sets of horizons. Under interpolation, CH has a lower error than the interpolated MS and DIR forecasts. Appendix § 13 explains this result as manifestation of aliasing of a discrete horizon grid.

ProcessSeen (h≤12h \le12) CHMSDIRRECInterpolation CHMSDIRRECExtrapolation CHREC
Mean-reverting0.8730.9340.9150.9390.8720.9320.9080.9371.2111.280
Aperiodic0.7330.7590.7540.7630.7300.7530.7460.7581.0541.110
Periodic0.5680.5810.5810.6170.5680.5760.5750.6130.6070.649
QP (slow)0.6230.6400.6390.6710.6230.6360.6330.6700.7560.780
QP (fast)0.6890.7160.7120.7430.6880.7120.7050.7400.9400.975

Table 2. Interpolation and extrapolation on five Gaussian-process regimes (RMSE, lower is better). All strategies train on the horizons h≤12h \le12. We evaluate on the seen integers, the unseen half-integers (interpolation), and the unseen integers up to h=24h = 24 (extrapolation). For the half-integers, MS and DIR use linear interpolation between their output horizons. MS and DIR have no output for h>12h > 12, but CH and REC give forecasts for these horizons. The best value in each block is bold.

In extrapolation, CH reverts to the unconditional mean on the aperiodic processes and follows the cyclic dynamics with small ripples. The REC rollout has a higher error than CH on all five processes. For periodic targets, Theorem 3 (Appendix J) extends the approximation guarantee beyond the training horizons [0,H][0, H].

Empirical Evaluation

Neural Forecast Architecture

We evaluate all strategies within the unified MQForecaster architecture [63, 46] sequence-to-sequence model [16]. Its modular encoder-decoder design provides a natural testbed to isolate the impact of the forecasting strategy while holding architecture constant (Figure 5). Historical realizations y:ty_{:t} are encoded into a hidden state θk(y:t)\theta_k(y_{:t}).

Figure 5. Continuous Horizon (CH) decoder within the MQForecaster architecture. Instead of outputting a discrete vector, the CH forecast strategy outputs a function g(h∣y:t)g(h \mid y_{:t}), capable of evaluating continuous horizons h∈[0,H]h \in[0,H] on demand. This enables direct evaluation at arbitrary continuous lead times (including unseen interpolated and extrapolated steps) while maintaining parameter sharing for lower estimation variance. Holding the backbone fixed isolates the strategy, confirming that the CH advantages generalize across encoders.

θk(y:t)=Encoder⁡(y:t)(15)\theta_k(y_{:t}) = \operatorname{Encoder}(y_{:t}) \tag*{(15)}

A decoder then maps the hidden state to the forecast. We keep the encoder fixed and change only the decoder to realize each forecast strategy.

The strategies MS and DIR instantiate the decoder exactly as their taxonomy definitions (§ 2) prescribe, replacing the abstract function gg with an MLP acting on the hidden state; therefore, we omit their repetition here. For the REC strategy, we employ a similar approach to that of the DeepAR decoding strategy [54].

To enable the CH strategy, we use a bilinear expansion approximator informed by the universal approximation theorem (Theorem 1):

g(h∣y:t)=∑k=0Kθk(y:t)⋅ϕk(h),ϕ0(h)=1,(16)g(h \mid\mathbf{y}_{:t}) = \sum_{k=0}^{K} \boldsymbol{\theta}_k(\mathbf{y}_{:t}) \cdot\phi_k(h), \qquad\phi_0(h) = 1, \tag*{(16)}

where the constant basis function ϕ^0=1\hat{\phi}_0 = 1 gives a level θ^t,0\hat{\theta}_{t,0} that is the same for all horizons. The horizon basis functions are a learned embedding table; see the ablation study in Appendix F, where we explore basis variants. Based on this ablation, we use tabular embedding basis functions (K=16K = 16). Appendix H.§ 4 describes the controlled learning objective of the M-series experiments.

Benchmark Datasets

We evaluate our approach on the M-series benchmark, comprising 16 large-scale datasets with more than 100,000 series from the competitions M1 [40], M3 [41], M4 [42], and Tourism [3]. Following recent literature [47, 50], we use temporal cross-validation by holding out the final 3H3H observations per series (Details in Appendix § 15): the first HH form the validation window, and the final 2H2H comprise the test window (Figure 6). Rolling the test FCD forward yields HH overlapping evaluations covering [T−2H+1,T][T - 2H + 1, T]. Denoting the final series index by TT, the data partitions are defined as:

Figure 6. Temporal cross-validation on an M4 hourly series, with training (blue), validation (yellow, HH), and test (salmon, 2H2H) regions.

ytrain=y:T−3H,yvalidation=yT−3H+1:T−2H,ytest=yT−2H+1:T.(17)\mathbf{y}_{\mathrm{train}} = \mathbf{y}_{:T-3H}, \qquad\mathbf{y}_{\mathrm{validation}} = \mathbf{y}_{T-3H+1:T-2H}, \qquad\mathbf{y}_{\mathrm{test}} = \mathbf{y}_{T-2H+1:T}. \tag*{(17)}

Main Empirical Results

Accuracy Metrics. Following Hyndman and Koehler [32], we report the Relative Average RMSE (relAvgRMSE), defined as the ratio of the mean per-series RMSE to that of the Naive baseline:

relAvgRMSE⁡=1N∑i=1NRMSE⁡i(y(i),y^(i))1N∑i=1NRMSE⁡i(y(i),y~(i))(18)\operatorname{relAvgRMSE} = \frac{\frac{1}{N}\sum_{i=1}^{N}\operatorname{RMSE}_{i}(\mathbf{y}^{(i)},\hat{\mathbf{y}}^{(i)})} {\frac{1}{N}\sum_{i=1}^{N}\operatorname{RMSE}_{i}(\mathbf{y}^{(i)},\tilde{\mathbf{y}}^{(i)})} \tag*{(18)}

where NN is the number of series and y(i)\mathbf{y}^{(i)}, y^(i)\hat{\mathbf{y}}^{(i)}, and y~(i)\tilde{\mathbf{y}}^{(i)} denote the target, model forecast and Naive baseline for series ii. Computing the root per series before aggregation prevents large-scale series from dominating the metric. Values below 1 indicate an improvement over Naive.

Forecast Baselines. We vary MQForecaster’s encoder (Section 6.1): CNN [48], LSTM [24, 11], a Transformer [62], and S4 [22]. We include ETS [25] and ARIMA [31, 29] as baselines. Appendix H.1 gives the network hyperparameters. Appendix H.2 discusses the parameter matching across strategies.

Point Forecast Results. Table 3 compares the strategies at matched parameter counts (Appendix H.2). Thus, a difference in accuracy comes from the forecast strategy, not from the capacity of the model. CH has a lower relAvgRMSE than MS in 40 of the 64 encoder and frequency pairs, with a mean improvement of 2.6%. In 12 pairs, the improvement is greater than two standard errors and in 8 pairs MS is better by more than two standard errors. The advantage of CH increases with the horizon length, as predicted by § 3. CH is better in all M4 Hourly pairs (H=48H = 48), with a decrease of 38% (CNN) and 49% (Transformer), and in 14 of the 16 monthly pairs. In the short yearly horizons (H≤6H \le6), the two strategies are approximately equal, and on M4 Daily, MS is lower in all four pairs. CH is also better than REC in 58 of the 64 encoder and dataset pairs, with a mean improvement of 19%.

DatasetFreqLSTM CHLSTM MSLSTM DIRLSTM RECCNN CHCNN MSCNN DIRCNN RECTransformer CHTransformer MSTransformer DIRTransformer RECS4 CHS4 MSS4 DIRS4 RECStatsForecast ETSStatsForecast ARIMA
M1M0.8098 (0.0060)0.8784 (0.0061)0.9244 (0.0048)0.9281 (0.0149)0.8226 (0.0050)0.9023 (0.0078)0.9150 (0.0222)0.9928 (0.0417)0.9022 (0.0226)0.9373 (0.0095)0.9715 (0.0215)0.9741 (0.0082)0.8705 (0.0120)0.8781 (0.0078)0.9366 (0.0277)0.9891 (0.0159)0.7347 (-)0.7625 (-)
M1Q0.9305 (0.0264)0.9441 (0.0207)0.9616 (0.0314)0.9172 (0.0268)0.9185 (0.0053)0.9359 (0.0179)0.9969 (0.0244)1.2225 (0.1147)1.0004 (0.0809)1.0238 (0.0178)1.0329 (0.0515)0.9454 (0.0251)0.9160 (0.0214)0.9031 (0.0047)1.0435 (0.0234)0.9264 (0.0235)0.8998 (-)0.9071 (-)
M1Y0.8426 (0.0245)0.7564 (0.0207)0.7958 (0.0240)0.9285 (0.0805)0.8014 (0.0122)0.8323 (0.0289)0.7367 (0.0473)0.7786 (0.0353)0.6896 (0.0267)0.8307 (0.0770)0.8448 (0.0720)0.8032 (0.0530)0.7384 (0.0328)0.7950 (0.0281)0.7750 (0.0434)0.8660 (0.0559)0.7854 (-)0.7696 (-)
M3O0.2348 (0.0020)0.2429 (0.0214)0.2414 (0.0069)0.2651 (0.0159)0.1844 (0.0012)0.2192 (0.0033)0.2422 (0.0078)0.3834 (0.0712)0.2507 (0.0121)0.2798 (0.0099)0.4130 (0.1321)0.2930 (0.0233)0.1870 (0.0030)0.1979 (0.0040)0.2568 (0.0153)0.3134 (0.0200)0.7146 (-)0.7259 (-)
M3M0.6973 (0.0042)0.7134 (0.0071)0.7807 (0.0062)0.8662 (0.0261)0.7222 (0.0011)0.7445 (0.0028)0.7465 (0.0036)1.1237 (0.0447)0.7618 (0.0169)0.7904 (0.0098)0.8213 (0.0168)1.0170 (0.0674)0.7115 (0.0034)0.7353 (0.0059)0.7503 (0.0046)0.9843 (0.0255)0.8380 (-)0.8490 (-)
M3Q0.7962 (0.0095)0.8061 (0.0111)0.8685 (0.0113)0.9332 (0.0224)0.7849 (0.0074)0.8035 (0.0069)0.8541 (0.0126)1.0807 (0.0890)0.8780 (0.0481)0.9300 (0.0333)0.9650 (0.0137)0.9288 (0.0454)0.7561 (0.0111)0.7390 (0.0038)0.9114 (0.0040)0.9702 (0.0142)0.9206 (-)0.9279 (-)
M3Y0.8879 (0.0130)0.7854 (0.0104)0.8758 (0.0108)0.9264 (0.0458)0.8230 (0.0100)0.7939 (0.0084)0.7772 (0.0152)0.9710 (0.0678)0.8685 (0.0674)0.8596 (0.0428)0.8605 (0.0236)0.8324 (0.0236)0.7996 (0.0240)0.7793 (0.0133)0.8778 (0.0114)0.9198 (0.0113)0.9678 (-)1.0116 (-)
M4H0.1719 (0.0169)0.1810 (0.0160)0.3916 (0.0033)0.6788 (0.0468)0.2850 (0.0057)0.4610 (0.0105)0.4739 (0.0072)0.7736 (0.0536)0.2471 (0.0117)0.4877 (0.0522)0.6908 (0.0161)0.9976 (0.0237)0.2698 (0.0063)0.3016 (0.0066)0.2773 (0.0059)0.7000 (0.0122)0.4158 (-)0.1987 (-)
M4D0.8362 (0.0711)0.7923 (0.0712)0.8536 (0.0133)0.9311 (0.0347)0.8344 (0.0067)0.8293 (0.0057)0.8531 (0.0052)0.9065 (0.0053)0.8687 (0.0270)0.8401 (0.0183)0.8717 (0.0203)1.0212 (0.0880)0.8178 (0.0125)0.8068 (0.0068)0.8458 (0.0173)1.0401 (0.0491)0.9815 (-)0.9879 (-)
M4W0.7839 (0.0124)0.7240 (0.0120)0.8100 (0.0090)0.8047 (0.0016)0.7067 (0.0072)0.7894 (0.0046)0.7787 (0.0072)0.8640 (0.0604)0.8288 (0.0128)0.8333 (0.0363)0.8363 (0.0163)0.8882 (0.0233)0.8055 (0.0158)0.7943 (0.0049)0.8271 (0.0078)0.9093 (0.0399)0.9507 (-)0.9320 (-)
M4M0.7114 (0.0133)0.7307 (0.0057)0.7482 (0.0110)0.8337 (0.0219)0.7278 (0.0072)0.7305 (0.0053)0.7322 (0.0055)1.0405 (0.0541)0.7198 (0.0550)0.7490 (0.0522)0.7841 (0.0161)1.0314 (0.0237)0.7054 (0.0159)0.7343 (0.0043)0.7297 (0.0059)0.9806 (0.0122)0.8802 (-)0.8904 (-)
M4Q0.7426 (0.0073)0.7373 (0.0093)0.7904 (0.0081)0.8072 (0.0124)0.7349 (0.0065)0.7398 (0.0053)0.7520 (0.0144)0.8950 (0.0541)0.8274 (0.0550)0.8158 (0.0522)0.8448 (0.0161)0.9164 (0.0237)0.7580 (0.0159)0.7591 (0.0102)0.7961 (0.0059)0.8868 (0.0122)0.8714 (-)0.9143 (-)
M4Y0.8034 (0.0092)0.7940 (0.0066)0.8831 (0.0013)0.8642 (0.0467)0.7515 (0.0065)0.7230 (0.0054)0.7412 (0.0144)0.8591 (0.0541)0.7566 (0.0550)0.8044 (0.0522)0.7979 (0.0161)0.7406 (0.0237)0.7133 (0.0159)0.7720 (0.0102)0.8074 (0.0059)0.8417 (0.0122)0.8578 (-)0.9376 (-)
TourismM0.5856 (0.0355)0.6011 (0.0061)0.7637 (0.0274)0.8260 (0.0419)0.6533 (0.0074)0.6772 (0.0074)0.6764 (0.0039)1.1139 (0.0583)0.8286 (0.0071)0.8171 (0.0070)0.8317 (0.0116)1.0842 (0.1034)0.6790 (0.0063)0.6217 (0.0034)0.6433 (0.0075)1.2160 (0.0622)0.4476 (-)0.5026 (-)
TourismQ0.6702 (0.0241)0.6691 (0.0098)0.8084 (0.0164)0.8133 (0.0813)0.6304 (0.0056)0.6563 (0.0051)0.7421 (0.0145)0.9466 (0.0319)0.8464 (0.0848)0.8733 (0.0850)0.8508 (0.0850)0.8725 (0.0472)0.6389 (0.0127)0.6064 (0.0034)0.7636 (0.0176)0.9239 (0.0343)0.6395 (-)0.7091 (-)
TourismY1.0035 (0.0133)0.9265 (0.0295)0.9803 (0.0168)0.9683 (0.0216)0.8991 (0.0171)0.9517 (0.0315)0.9778 (0.0219)0.9480 (0.0248)0.9138 (0.0277)0.9556 (0.0456)0.9518 (0.0181)0.9431 (0.0413)0.9562 (0.0359)0.9523 (0.0122)1.0954 (0.0356)1.0042 (0.0022)0.8924 (-)1.0213 (-)

Table 3. Empirical evaluation of point forecasts. Relative Average RMSE (relAvgRMSE) with the Naive baseline, averaged over 5 runs. Lower measurements are preferred; values below 1 beat the Naive. The methods without standard deviation have deterministic solutions. For each encoder we compare four forecasting strategies: our Continuous Horizon (CH), multi-step (MS), direct (DIR), and recurrent (REC).

Appendix I complements the RMSE results with sMAPE and MASE, making our results closer to the M competitions.

Inference Complexity and Parallelism

Beyond accuracy, strategies differ in the decoder cost of one forecast on a queried horizon hh (Table 4, Figure 7). MS computes the full grid of HH horizons, so a query costs O(H)\mathcal{O}(H). REC rolls out hh sequential steps, so one query costs O(h)\mathcal{O}(h). DIR runs only the head of horizon hh, so one query costs O(1)\mathcal{O}(1), but DIR must train and store HH heads.

h≤Hh \le Hh>Hh > H
RECO(h)\mathcal{O}(h)O(h)\mathcal{O}(h)
DIRO(1)\mathcal{O}(1)O(1)\mathcal{O}(1)†
MSO(H)\mathcal{O}(H)O(h)\mathcal{O}(h)†
CHO(1)\mathcal{O}(1)O(1)\mathcal{O}(1)

Table 4. Decoder cost of one query

Figure 7. Targeted query decoder cost (schematic). MS forecasts all HH horizons. DIR runs horizon-hh’s model. REC rolls out hh sequential steps. CH needs one readout of the cached basis. For h>Hh > H, MS and DIR need a new model (dashed lines and †).

The CH basis functions ϕk(h)\phi_k(h) do not depend on the history, so we cache them before inference. Then a query on any horizon needs only one readout, at a cost of O(1)\mathcal{O}(1). For h>Hh > H, CH uses the same model. When the complete trajectory is necessary, CH evaluates all horizons in parallel, as MS does.

Discussion and Related Work

CH extends the functional time series approach [51, 33] in three ways. It uses the function directly as an end-to-end forecast strategy, without an in-sample decomposition. It learns the full mapping with a neural network, without smoothing or functional principal components. It learns its basis with MLPs and embeddings, in place of a fixed basis. A future direction is to constrain the learned basis, for example, with orthonormality [51].

Other neural forecast architectures scratched the surface of the CH forecast strategy, examples include DAM [17] and NBEATS [49, 45] that uses harmonics and polynomial trends, NHITS [10] that use wavelets. In contrast to these approaches, we do not pre-conditioning the function family and rather learn it via embeddings, but more importantly we allow for variable length horizon in contrast to these architectures that have MS fixed outputs.

Neural DEs [13, 52, 36] learn vector fields that are integrated to obtain continuous-horizon trajectories. In contrast, CH retains a discrete-time encoder and uses a horizon-separable basis for direct, parallel evaluation without numerical integration. Inspired by ODEs and integrated time-series models (e.g., ARIMA), a promising direction is to apply CH to first differences and integrate predictions to recover forecast levels, potentially improving robustness to non-stationary trends.

Our simulations decompose only the point-forecast errors. A probabilistic extension can use the decomposition of Hersbach [23], to study the uncertainty, calibration, and reliability tradeoffs of different forecast strategies.

Ben Taieb and Atiya [5] proposed discrete hybrid strategies, such as DirRec, that combine the low variance of recursive forecasts with the low bias of direct forecasts. A hybrid of discrete strategies and CH is a further direction for very long horizons.

Current foundation forecasting models use a fixed output grid or a recursive rollout [20, 2, 18, 64, 26] and truncate a long output to get a shorter horizon. CH gives only the necessary horizons, so its integration with foundation models is a promising direction.

Conclusion

We introduced the Continuous Horizon (CH) forecast strategy, and conducted a comprehensive theoretical investigation of the properties of CH, showing that it provides universal approximation guaranties for forecasting operators. Furthermore, we proved that, under reasonable assumptions, the estimation variance of CH improves on that of discrete strategies because CH makes a more efficient use of the information from the horizon.

We validated these theoretical findings through simulation studies and bias-variance decompositions, demonstrating that CH achieves a bias profile comparable to DIR and MS with a lower estimation variance. Beyond fixed grid training, CH demonstrates strong accuracy in both interpolation and extrapolation regimes, forecasting horizons unseen during training.

CH achieves competitive accuracy. It has a lower relAvgRMSE than MS in 40 of the 64 encoder and frequency pairs of the M-series benchmark. Finally, we highlight the substantial computational efficiency of CH, particularly its ability to execute targeted horizon queries in constant time.

Universal Operator Approximation Theorem

In this appendix, we prove the universal approximation theorem introduced in Section 3. We use the Stone-Weierstrass theorem to prove the density of finite sum approximation functions, and complement with the universal approximation theorem applied to its components.

Lemma 3. Let Y⊂Rt\mathcal{Y} \subset\mathbb{R}^{t} be a compact domain and [0,H]⊂R+[0,H] \subset\mathbb{R}^{+} a compact horizon interval. Let f:Y×[0,H]→Rf:\mathcal{Y} \times[0,H] \to\mathbb{R} given by (y:t,h)↦f(h∣y:t)(\mathbf{y}_{:t},h) \mapsto f(h \mid\mathbf{y}_{:t}) be a continuous forecast function. Then, for any ϵ>0\epsilon> 0, there exists a finite integer K∈NK \in\mathbb{N} and continuous functions θk∗:Y→R\theta_k^{*}:\mathcal{Y} \to\mathbb{R} and ϕk∗:[0,H]→R\phi_k^{*}:[0,H] \to\mathbb{R} for k=1,…,Kk=1,\ldots,K such that:

sup⁡y:t∈Y, h∈[0,H]∣f(h∣y:t)−∑k=1Kθk∗(y:t)ϕk∗(h)∣<ϵ\sup_{\mathbf{y}_{:t}\in\mathcal{Y},\,h\in[0,H]} \left|f(h\mid\mathbf{y}_{:t})-\sum_{k=1}^{K}\theta_k^{*}(\mathbf{y}_{:t})\phi_k^{*}(h)\right|<\epsilon

Proof. The Cartesian product of two compact sets Y×[0,H]\mathcal{Y}\times[0,H] is compact in Rt+1\mathbb{R}^{t+1}. Consider the set of finite separable functions:

A={∑k=1Kθk(y:t)ϕk(h):K<∞, θk∈C(Y), ϕk∈C([0,H])}\mathcal{A}=\left\{\sum_{k=1}^{K}\theta_k(\mathbf{y}_{:t})\phi_k(h):K<\infty,\ \theta_k\in C(\mathcal{Y}),\ \phi_k\in C([0,H])\right\}

We verify that A\mathcal{A} forms a subalgebra of C(Y×[0,H])C(\mathcal{Y}\times[0,H]) that satisfies the conditions of the Stone-Weierstrass Theorem.

  1. Contains Constants: Taking θ(y:t)=1\theta(\mathbf{y}_{:t})=1 and ϕ(h)=c\phi(h)=c shows c∈Ac\in\mathcal{A}.

  1. Closed under Addition and Multiplication: For any g1,g2∈Ag_1,g_2\in\mathcal{A}, g1+g2∈Ag_1+g_2\in\mathcal{A} and g1g2∈Ag_1g_2\in\mathcal{A} by expansion of finite sums.

  1. Point Separation: For (y,h)≠(y′,h′)(\mathbf{y},h)\ne(\mathbf{y}',h'), if y≠y′\mathbf{y}\ne\mathbf{y}', choose θ(y)=yi\theta(\mathbf{y})=y_i (where coordinate yi≠yi′y_i\ne y_i') and ϕ(h)=1\phi(h)=1. If h≠h′h\ne h', choose θ(y)=1\theta(\mathbf{y})=1 and ϕ(h)=h\phi(h)=h.

By the Stone-Weierstrass Theorem, A\mathcal{A} is dense in C(Y×[0,H])C(\mathcal{Y}\times[0,H]) equipped with the uniform norm. Therefore, for any f∈C(Y×[0,H])f\in C(\mathcal{Y}\times[0,H]) and any ϵ>0\epsilon>0, there exists some element in A\mathcal{A} within the distance ϵ\epsilon of ff, establishing the existence of finite KK and continuous functions {θk∗}k=1K,{ϕk∗}k=1K\{\theta_k^{*}\}_{k=1}^{K},\{\phi_k^{*}\}_{k=1}^{K}.

:::

Lemma 4. Let {ϕk⋆}k=1K\{\phi_k^\star\}_{k=1}^{K} be a finite set of continuous functions

ϕk⋆:[0,H]→R,\phi_k^\star: [0,H] \to\mathbb{R},

where [0,H]⊂R+[0,H] \subset\mathbb{R}^{+} is compact. Then, for all ϵ>0\epsilon> 0, there exist neural networks

{ϕk}k=1K\{\phi_k\}_{k=1}^{K}

such that

sup⁡h∈[0,H]∣ϕk⋆(h)−ϕk(h)∣<ϵ,k=1,…,K.(19)\sup_{h\in[0,H]} \left|\phi_k^\star(h)-\phi_k(h)\right| < \epsilon,\qquad k=1,\ldots,K. \tag*{(19)}

Proof. This lemma is a direct consequence of the neural universal approximation theorem [27]. Since every ϕk⋆\phi_k^\star is continuous in the compact domain [0,H][0,H], a non-polynomial activated feedforward neural network can approximate ϕk⋆\phi_k^\star uniformly to arbitrary precision. As the number of functions KK is finite, the approximation can be constructed for every k∈{1,…,K}k \in\{1,\ldots,K\} using the same target precision ϵ\epsilon.

Lemma 5. Let {θk⋆}k=1K\{\theta_k^\star\}_{k=1}^{K} be a finite set of continuous coefficient mappings

θk⋆:Y→R,\theta_k^\star: \mathcal{Y} \to\mathbb{R},

where Y⊂Rt\mathcal{Y} \subset\mathbb{R}^{t} is compact. Then, for all ϵ>0\epsilon> 0, there exist neural networks

{θk}k=1K\{\theta_k\}_{k=1}^{K}

such that

sup⁡y:t∈Y∣θk⋆(y:t)−θk(y:t)∣<ϵ,k=1,…,K.(20)\sup_{\mathbf{y}:t\in\mathcal{Y}} \left|\theta_k^\star(\mathbf{y}:t)-\theta_k(\mathbf{y}:t)\right| < \epsilon,\qquad k=1,\ldots,K. \tag*{(20)}

Proof. As in Lemma 4, this follows from the neural universal approximation theorem. Since every θk⋆\theta_k^\star is continuous in the compact domain Y\mathcal{Y}, there exists a neural network θk\theta_k that approximates it uniformly with arbitrary precision. Since KK is finite, the approximation can be constructed simultaneously for all coefficient mappings.

Theorem 1. Let Y⊂Rt\mathcal{Y} \subset\mathbb{R}^{t} be a compact domain of historical observations y:t\mathbf{y}_{:t} and let [0,H]⊂R+[0,H] \subset\mathbb{R}^{+} be a compact forecast horizon interval. Let a continuous forecast operator be

F:Y→C([0,H]),F(y:t)=f(⋅∣y:t),F:\mathcal{Y}\to\mathcal{C}([0,H]), \qquad F(\mathbf{y}_{:t})=f(\cdot\mid\mathbf{y}_{:t}),

where C([0,H])\mathcal{C}([0,H]) is equipped with the uniform norm. Then, for all ϵ>0\epsilon>0, there exists a finite K∈NK\in\mathbb{N}, a set of continuous neural network basis functions {ϕk(h)}k=1K\{\phi_k(h)\}_{k=1}^{K}, and neural coefficient mappings {θk(y:t)}k=1K\{\theta_k(\mathbf{y}_{:t})\}_{k=1}^{K}, collected in the parameter vector w⋆\mathbf{w}^{\star}, such that the neural forecaster g(h∣y:t,w⋆)=∑k=1Kθk(y:t)ϕk(h)g(h\mid\mathbf{y}_{:t},\mathbf{w}^{\star})=\sum_{k=1}^{K}\theta_k(\mathbf{y}_{:t})\phi_k(h) approximates ff uniformly over the horizon,

sup⁡h∈[0,H]∣f(h∣y:t)−g(h∣y:t,w⋆)∣<ϵ,∀y:t∈Y.(21)\sup_{h\in[0,H]}\left|f(h\mid\mathbf{y}_{:t})-g(h\mid\mathbf{y}_{:t},\mathbf{w}^{\star})\right|<\epsilon,\qquad\forall\mathbf{y}_{:t}\in\mathcal{Y}. \tag*{(21)}

Because [0,H][0,H] has finite length, this uniform bound implies the squared-error bound

∫0H(f(h∣y:t)−g(h∣y:t,w⋆))2 dh<Hϵ2,∀y:t∈Y,(22)\int_{0}^{H}\left(f(h\mid\mathbf{y}_{:t})-g(h\mid\mathbf{y}_{:t},\mathbf{w}^{\star})\right)^{2}\,\mathrm{d}h<H\epsilon^{2},\qquad\forall\mathbf{y}_{:t}\in\mathcal{Y}, \tag*{(22)}

which is the form that controls the squared approximation bias BhB_h of Lemma 1.

Proof. Since

F:Y→C([0,H])F:\mathcal{Y}\to\mathcal{C}([0,H])

is continuous under the uniform norm, the associated joint forecast function

f:Y×[0,H]→R,(y:t,h)↦f(h∣y:t),f:\mathcal{Y}\times[0,H]\to\mathbb{R},\qquad(\mathbf{y}_{:t},h)\mapsto f(h\mid\mathbf{y}_{:t}),

is continuous. Using Lemma 3, for all ϵ1>0\epsilon_1>0, there exist finite KK and continuous functions θk⋆\theta_k^{\star} and ϕk⋆\phi_k^{\star} such that

sup⁡y:t∈Ysup⁡h∈[0,H]∣f(h∣y:t)−∑k=1Kθk⋆(y:t)ϕk⋆(h)∣<ϵ1.(23)\sup_{\mathbf{y}_{:t}\in\mathcal{Y}}\sup_{h\in[0,H]}\left|f(h\mid\mathbf{y}_{:t})-\sum_{k=1}^{K}\theta_k^{\star}(\mathbf{y}_{:t})\phi_k^{\star}(h)\right|<\epsilon_1. \tag*{(23)}

Let

gK⋆(h∣y:t)=∑k=1Kθk⋆(y:t)ϕk⋆(h)g_K^{\star}(h\mid\mathbf{y}_{:t})=\sum_{k=1}^{K}\theta_k^{\star}(\mathbf{y}_{:t})\phi_k^{\star}(h)

denote this finite separable approximation. We now approximate the two ingredients of gK⋆g_K^\star. Using Lemma 4, for all ϵ2>0\epsilon_2 > 0, there exist basis functions ϕk\phi_k such that

sup⁡h∈[0,H]∣ϕk⋆(h)−ϕk(h)∣<ϵ2.(24)\sup_{h \in[0,H]} \left|\phi_k^\star(h) - \phi_k(h)\right| < \epsilon_2. \tag*{(24)}

Using Lemma 5, for all ϵ3>0\epsilon_3 > 0, there exist coefficients θk\boldsymbol{\theta}_k such that

sup⁡yt∈Y∣θk⋆(yt)−θk(yt)∣<ϵ3.(25)\sup_{\mathbf{y}_t \in\mathcal{Y}} \left|\boldsymbol{\theta}_k^\star(\mathbf{y}_t) - \boldsymbol{\theta}_k(\mathbf{y}_t)\right| < \epsilon_3. \tag*{(25)}

Define the neural forecast approximation as

g(h∣yt,w⋆)=∑k=1Kθk(yt)ϕk(h),g(h \mid\mathbf{y}_t,\mathbf{w}^\star) = \sum_{k=1}^{K} \theta_k(\mathbf{y}_t)\phi_k(h),

where the single finite parameter vector w⋆=(wθ,1⋆,…,wθ,K⋆,wϕ,1⋆,…,wϕ,K⋆)\mathbf{w}^\star= (\mathbf{w}_{\theta,1}^\star,\ldots,\mathbf{w}_{\theta,K}^\star,\mathbf{w}_{\phi,1}^\star,\ldots,\mathbf{w}_{\phi,K}^\star) concatenates the weights of the KK coefficient networks {θk}k=1K\{\theta_k\}_{k=1}^{K} (each with weights wθ,k⋆\mathbf{w}_{\theta,k}^\star) and the KK basis networks {ϕk}k=1K\{\phi_k\}_{k=1}^{K} (each with weights wϕ,k⋆\mathbf{w}_{\phi,k}^\star). Since KK is finite and each network has finitely many weights, w⋆\mathbf{w}^\star is finite-dimensional.

To conclude, we combine the finite separable approximation, the neural basis approximation, and the neural coefficient approximation. Using the triangular inequality,

sup⁡h∈[0,H]∣f(h∣yt)−g(h∣yt,w⋆)∣≤sup⁡h∈[0,H]∣f(h∣yt)−∑k=1Kθk⋆(yt)ϕk⋆(h)∣+sup⁡h∈[0,H]∣∑k=1Kθk⋆(yt)ϕk⋆(h)−∑k=1Kθk⋆(yt)ϕk(h)∣+sup⁡h∈[0,H]∣∑k=1Kθk⋆(yt)ϕk(h)−∑k=1Kθk(yt)ϕk(h)∣.(26)\begin{aligned} \sup_{h \in[0,H]} \left|f(h \mid\mathbf{y}_t) - g(h \mid\mathbf{y}_t,\mathbf{w}^\star)\right| &\leq\sup_{h \in[0,H]} \left|f(h \mid\mathbf{y}_t) - \sum_{k=1}^{K}\theta_k^\star(\mathbf{y}_t)\phi_k^\star(h)\right| \\ &\quad+ \sup_{h \in[0,H]} \left|\sum_{k=1}^{K}\theta_k^\star(\mathbf{y}_t)\phi_k^\star(h) - \sum_{k=1}^{K}\theta_k^\star(\mathbf{y}_t)\phi_k(h)\right| \\ &\quad+ \sup_{h \in[0,H]} \left|\sum_{k=1}^{K}\theta_k^\star(\mathbf{y}_t)\phi_k(h) - \sum_{k=1}^{K}\theta_k(\mathbf{y}_t)\phi_k(h)\right|. \tag*{(26)} \end{aligned}

Bounding each supremum through the triangle inequality over the sum,

sup⁡h∈[0,H]∣f(h∣y:t)−g(h∣y:t,w⋆)∣≤ϵ1+∑k=1K∣θk⋆(y:t)∣sup⁡h∈[0,H]∣ϕk⋆(h)−ϕk(h)∣+∑k=1K∣θk⋆(y:t)−θk(y:t)∣sup⁡h∈[0,H]∣ϕk(h)∣.(27)\begin{aligned} \sup_{h\in[0,H]} \left|f(h\mid\mathbf{y}_{:t})-g(h\mid\mathbf{y}_{:t},\mathbf{w}^{\star})\right| &\le\epsilon_1+\sum_{k=1}^{K}\left|\theta_k^{\star}(\mathbf{y}_{:t})\right|\sup_{h\in[0,H]}\left|\phi_k^{\star}(h)-\phi_k(h)\right|\\ &\quad+\sum_{k=1}^{K}\left|\theta_k^{\star}(\mathbf{y}_{:t})-\theta_k(\mathbf{y}_{:t})\right|\sup_{h\in[0,H]}\left|\phi_k(h)\right|. \tag*{(27)} \end{aligned}

Using continuity in compact domains, the finite collections {θk⋆}\{\theta_k^{\star}\} and {ϕk}\{\phi_k\} are uniformly bounded. Let

Mθ=max⁡1≤k≤Ksup⁡y:t∈Y∣θk⋆(y:t)∣,andMϕ=max⁡1≤k≤Ksup⁡h∈[0,H]∣ϕk(h)∣.M_{\theta}=\max_{1\leq k\leq K}\sup_{\mathbf{y}_{:t}\in\mathcal{Y}}\left|\theta_k^{\star}(\mathbf{y}_{:t})\right|,\qquad\text{and}\qquad M_{\phi}=\max_{1\leq k\leq K}\sup_{h\in[0,H]}\left|\phi_k(h)\right|.

Then

sup⁡h∈[0,H]∣f(h∣y:t)−g(h∣y:t,w⋆)∣<ϵ1+KMθϵ2+KMϕϵ3.(28)\sup_{h\in[0,H]}\left|f(h\mid\mathbf{y}_{:t})-g(h\mid\mathbf{y}_{:t},\mathbf{w}^{\star})\right|<\epsilon_1+KM_{\theta}\epsilon_2+KM_{\phi}\epsilon_3. \tag*{(28)}

Finally, the three approximation accuracies can be chosen

ϵ1=ϵ3,ϵ2=ϵ3KMθ,ϵ3=ϵ3KMϕ,\epsilon_1=\frac{\epsilon}{3},\qquad\epsilon_2=\frac{\epsilon}{3KM_{\theta}},\qquad\epsilon_3=\frac{\epsilon}{3KM_{\phi}},

which gives the uniform bound

sup⁡h∈[0,H]∣f(h∣y:t)−∑k=1Kθk(y:t)ϕk(h)∣<ϵ,∀y:t∈Y.(29)\sup_{h\in[0,H]}\left|f(h\mid\mathbf{y}_{:t})-\sum_{k=1}^{K}\theta_k(\mathbf{y}_{:t})\phi_k(h)\right|<\epsilon,\qquad\forall\mathbf{y}_{:t}\in\mathcal{Y}. \tag*{(29)}

Since [0,H][0,H] has finite length HH, the uniform bound controls both the L1L_1 error and the squared error,

∫0H∣f(h∣y:t)−g(h∣y:t,w⋆)∣ dh<Hϵ,\int_{0}^{H}\left|f(h\mid\mathbf{y}_{:t})-g(h\mid\mathbf{y}_{:t},\mathbf{w}^{\star})\right|\,\mathrm{d}h<H\epsilon,
∫0H(f(h∣y:t)−g(h∣y:t,w⋆))2 dh<Hϵ2,(30)\int_{0}^{H}\left(f(h\mid\mathbf{y}_{:t})-g(h\mid\mathbf{y}_{:t},\mathbf{w}^{\star})\right)^2\,\mathrm{d}h<H\epsilon^2, \tag*{(30)}

for all y:t∈Y\mathbf{y}_{:t}\in\mathcal{Y}. The squared-error bound is the form that controls the squared approximation bias BhB_h of Lemma 1.

Therefore, the continuous operator can be arbitrarily approximated by a finite neural basis expansion whose basis functions depend on the forecast horizon and whose coefficients depend on the historical observations.

Continuous Horizon Variance Dominance

In this appendix, we prove the variance dominance theorem introduced in § 3, which bounds the estimation variance VhV_h of Lemma 1 across forecast strategies. The argument proceeds in two stages. First, we characterize how the horizon-selector measure ρ\rho shapes the covariance of the fitted parameters: a first-order expansion of the empirical-risk minimizer yields a sandwich covariance Cov⁡(w^)≈1TA−1Ω(ρ)A−1\operatorname{Cov}(\hat{\mathbf{w}}) \approx\frac{1}{T} A^{-1}\Omega(\rho)A^{-1}, whose variability matrix Ω(ρ)\Omega(\rho) aggregates the cross-horizon residual covariances selected by ρ\rho, and the delta method propagates the parameter covariance to the horizon-hh forecast covariance. Second, we turn the forecast covariance into a scalar ordering: under a separable score covariance with a correlation kernel KK, the variability collapses to a single effective horizon information Ieff(ρ)\mathcal{I}_{\mathrm{eff}}(\rho), the number of independent horizons the forecast strategy’s objective supplies during estimation. Direct forecasting draws on one horizon, multi-step on a discrete grid, and Continuous Horizon on the whole interval, so their effective information obeys IeffDIR<IeffMS<IeffCH\mathcal{I}_{\mathrm{eff}}^{\mathrm{DIR}} < \mathcal{I}_{\mathrm{eff}}^{\mathrm{MS}} < \mathcal{I}_{\mathrm{eff}}^{\mathrm{CH}} at any finite grid resolution. Because variance scales inversely with Ieff\mathcal{I}_{\mathrm{eff}} at matched shared-parameter dimension, this chain gives the variance ordering VCH(h)<VMS(h)<VDIR(h)V_{\mathrm{CH}}(h) < V_{\mathrm{MS}}(h) < V_{\mathrm{DIR}}(h).

Parameters Covariance Characterization

The variance term VhV_h of Lemma 1 is governed by the covariance of the fitted parameters, which the following lemma characterizes for REC, DIR, MS, and CH strategies at once. Each strategy is identified by the horizon-selector measure ρ\rho of Eq. (6): a point mass δh\delta_h for the direct strategy, the empirical measure 1H∑jδhj\frac{1}{H}\sum_j\delta_{h_j} for multi-step, and the uniform measure 1H dh\frac{1}{H}\,dh for the Continuous Horizon strategy.

Lemma 6 (Parameters’ covariance under a horizon selector). Let w^\hat{\mathbf{w}} minimize the ρ\rho-averaged empirical risk of Eq. (6) over the TT tuples of D\mathcal{D}. Define the per-horizon score at the population optimum w⋆\mathbf{w}^{\star} and its ρ\rho-aggregate,

s(h)=∇wL(yt+h,g(h∣y:t,w⋆)),S=∫0Hs(h) dρ(h).s(h) = \nabla_{\mathbf{w}} L(y_{t+h}, g(h \mid\mathbf{y}_{:t}, \mathbf{w}^{\star})), \qquad S = \int_{0}^{H} s(h)\,d\rho(h).

The estimator w^\hat{\mathbf{w}} solves the first-order condition, linearizing around w⋆\mathbf{w}^{\star},

1T∑t=1TSt(w^)=0,0≈1T∑t=1TSt(w⋆)+A(w^−w⋆),\frac{1}{T}\sum_{t=1}^{T} S_t(\hat{\mathbf{w}}) = 0, \qquad0 \approx\frac{1}{T}\sum_{t=1}^{T} S_t(\mathbf{w}^{\star}) + A(\hat{\mathbf{w}}-\mathbf{w}^{\star}),

where St=∇wRtS_t = \nabla_w R_t is the score of tuple tt, the gradient of its per-tuple risk Rt(w)=∫0HL(yt+h,g(h∣y:t,w)) dρ(h)R_t(\mathbf{w}) = \int_0^H L(y_{t+h}, g(h \mid y_{:t}, \mathbf{w}))\,d\rho(h), and A=E[∇w2Rt(w⋆)]A = \mathbb{E}[\nabla_w^2 R_t(\mathbf{w}^{\star})] is the expected Hessian of the risk at the optimum. Assuming (a) w^\hat{\mathbf{w}} is consistent for w⋆\mathbf{w}^{\star}, (b) AA is invertible, and (c) the expansion remainder is asymptotically negligible, we solve the linear system

w^−w⋆≈−A−11T∑tSt(w⋆).\hat{\mathbf{w}} - \mathbf{w}^{\star} \approx-A^{-1}\frac{1}{T}\sum_t S_t(\mathbf{w}^{\star}).

Its right-hand side is an average of TT mean-zero scores with covariance 1TΩ(ρ)\frac{1}{T}\Omega(\rho), Ω(ρ)=Cov⁡(S)\Omega(\rho) = \operatorname{Cov}(S), by bilinearity of the covariance operator. Propagating through A−1A^{-1} [28, 60], the fitted parameters therefore have covariance

Cov⁡(w^)≈1TA−1Ω(ρ)A−1,(31)\operatorname{Cov}(\hat{\mathbf{w}}) \approx\frac{1}{T}A^{-1}\Omega(\rho)A^{-1}, \tag*{(31)}

with variability matrix Ω(ρ)\Omega(\rho) given by

Ω(ρ)=Cov⁡(S)=∫0H∫0HCov⁡(s(h),s(h′)) dρ(h) dρ(h′).(32)\Omega(\rho) = \operatorname{Cov}(S) = \int_0^H \int_0^H \operatorname{Cov}\left(s(h),s(h')\right)\,d\rho(h)\,d\rho(h'). \tag*{(32)}

Substituting each selector ρ\rho into Eq. (B.2) yields its variability matrix.

Proposition 1 (Per-strategy variability). With the horizon selectors of Eq. (6), the variability matrix Ω(ρ)\Omega(\rho) takes the forms

Ω(ρDIR)=Var⁡(s(h)),(33)\Omega(\rho_{\mathrm{DIR}}) = \operatorname{Var}(s(h)), \tag*{(33)}
Ω(ρMS)=1H2∑h=1H∑h′=1HCov⁡(s(h),s(h′)),(34)\Omega(\rho_{\mathrm{MS}}) = \frac{1}{H^2}\sum_{h=1}^{H}\sum_{h'=1}^{H}\operatorname{Cov}\left(s(h),s(h')\right), \tag*{(34)}
Ω(ρCH)=1H2∫0H∫0HCov⁡(s(h),s(h′)) dh dh′.(35)\Omega(\rho_{\mathrm{CH}}) = \frac{1}{H^2}\int_0^H\int_0^H\operatorname{Cov}\left(s(h),s(h')\right)\,dh\,dh'. \tag*{(35)}

The direct strategy estimates an independent parameter block wh\mathbf{w}_h for each horizon, so its selector ρ=δh\rho= \delta_h retains only the diagonal term and no cross-horizon covariance enters; multi-step and the Continuous Horizon strategy pool a single w\mathbf{w} across horizons, so their Ω(ρ)\Omega(\rho) accumulate the off-diagonal score covariances over the discrete grid and the continuum, respectively.

The three forms include progressively more off-diagonal score covariance: none for direct, a discrete double sum over the HH grid points for multi-step, and the full double integral over the continuum for Continuous Horizon. The estimator covariance of Lemma 6, with the per-strategy Ω(ρ)\Omega(\rho) of Proposition 1, propagates to the forecast through the delta method, giving the horizon-hh estimation variance VhV_h of Lemma 1.

Corollary 1 (Forecast variance). Let ψ=∇wg(h∣y:t,w⋆)\psi= \nabla_w g(h \mid\mathbf{y}_{:t}, \mathbf{w}^{\star}) be the forecast gradient at the optimum. Linearization of the forecast around w⋆\mathbf{w}^{\star},

g(h∣y:t,w^)≈g(h∣y:t,w⋆)+ψT(w^−w⋆),g(h \mid\mathbf{y}_{:t}, \hat{\mathbf{w}}) \approx g(h \mid\mathbf{y}_{:t}, \mathbf{w}^{\star}) + \psi^{\mathsf{T}}(\hat{\mathbf{w}}-\mathbf{w}^{\star}),

so that, substituting the covariance of Eq. (B.1),

Vh(y:t)≈ψTCov⁡(w^)ψ=1TψTA−1Ω(ρ)A−1ψ.(36)V_h(\mathbf{y}_{:t}) \approx\psi^{\mathsf{T}} \operatorname{Cov}(\hat{\mathbf{w}}) \psi= \frac{1}{T}\psi^{\mathsf{T}} A^{-1}\Omega(\rho)A^{-1}\psi. \tag*{(36)}

The forecast variance thus inherits the sandwich structure, quadratic in the forecast gradient ψ\psi and scaled by the parameter covariance that the selector ρ\rho shapes through AA and Ω(ρ)\Omega(\rho).

Effective horizon information

To turn the variability matrix into a scalar measure of how much independent information the horizons provide, we impose a separable structure on the score covariance.

Assumption 1 (Separable score covariance). The horizon-score process has covariance

Cov⁡(s(h),s(h′))=GK(h,h′),(37)\operatorname{Cov}\left(s(h),s(h')\right) = G K(h,h'), \tag*{(37)}

where G⪰0G \succeq0 is a fixed parameter-space matrix and KK is a correlation kernel: continuous, positive semidefinite, with K(h,h)=1K(h,h)=1 and 0≤K(h,h′)≤10 \le K(h,h') \le1, decaying in ∣h−h′∣\lvert h-h'\rvert with finite correlation length.

The nonnegative, decaying correlation captures that nearby forecast horizons provide overlapping rather than independent information. This is the score-space analogue of the effective-sample-size construction in spatial statistics [21, 61], and of its functional extension [1], where correlated observations carry less information than their nominal count.

Lemma 7 (Effective horizon information). Under Assumption 1, define the effective horizon information of a strategy with selector ρ\rho as

Ieff(ρ)=[∫0H∫0HK(h,h′) dρ(h) dρ(h′)]−1.(38)\mathcal{I}_{\mathrm{eff}}(\rho) = \left[\int_{0}^{H}\int_{0}^{H} K(h,h')\,\mathrm{d}\rho(h)\,\mathrm{d}\rho(h')\right]^{-1}. \tag*{(38)}

Then the variability matrix factors as Ω(ρ)=G/Ieff(ρ)\Omega(\rho) = G / \mathcal{I}_{\mathrm{eff}}(\rho), and the shared-parameter covariance of Lemma 6 becomes

Cov⁡(w^ρ)≈1TIeff(ρ)Aρ−1GAρ−1.(39)\operatorname{Cov}(\widehat{w}_{\rho}) \approx\frac{1}{T\mathcal{I}_{\mathrm{eff}}(\rho)} A_{\rho}^{-1} G A_{\rho}^{-1}. \tag*{(39)}

Corollary 2 (Effective information per strategy). Substituting the selectors of Eq. (6) into Eq. (B.8),

IeffDIR=1,IeffMS=H2∑h∑h′K(h,h′),IeffCH=H2∫0H∫0HK(h,h′) dh dh′.(40)\mathcal{I}_{\mathrm{eff}}^{\mathrm{DIR}} = 1,\qquad\mathcal{I}_{\mathrm{eff}}^{\mathrm{MS}} = \frac{H^{2}}{\sum_{h}\sum_{h'} K(h,h')},\qquad\mathcal{I}_{\mathrm{eff}}^{\mathrm{CH}} = \frac{H^{2}}{\int_{0}^{H}\int_{0}^{H} K(h,h')\,\mathrm{d}h\,\mathrm{d}h'}. \tag*{(40)}

These satisfy two limits. When the horizons are fully correlated (K≡1K \equiv1), every strategy collapses to Ieff=1\mathcal{I}_{\mathrm{eff}} = 1: all horizons carry the information of one. When the multi-step horizons are independent (K(h,h′)=1{h=h′}K(h,h') = \mathbb{1}\{h = h'\}), IeffMS=H\mathcal{I}_{\mathrm{eff}}^{\mathrm{MS}} = H, the nominal count underlying the residuals-per-parameter argument of Ben Taieb and Atiya [5]; for positively correlated horizons 1≤IeffMS≤H1 \leq\mathcal{I}_{\mathrm{eff}}^{\mathrm{MS}} \leq H. The independent case is thus a special case of multi-step, not an assumption imposed on Continuous Horizon.

For a stationary kernel K(h,h′)=K(u)K(h,h') = K(u) with lag u=∣h−h′∣u = |h-h'|, when HH is large relative to the correlation length the Continuous Horizon effective information admits the closed form

IeffCH≈H2ℓint,ℓint=∫0∞K(u) du.(41)\mathcal{I}_{\mathrm{eff}}^{\mathrm{CH}} \approx\frac{H}{2\ell_{\mathrm{int}}},\qquad\ell_{\mathrm{int}} = \int_{0}^{\infty} K(u)\,\mathrm{d}u. \tag*{(41)}

Information grows with the interval length divided by the correlation length; note that a denser horizon sampling does not drive the variance to zero while KK remains positively correlated.

Remark 1 (Residuals per parameter). Under the additional isotropy assumption that the horizon-averaged sensitivity factor ψ⊤Aρ−1GAρ−1ψ\psi^{\top} A_{\rho}^{-1} G A_{\rho}^{-1}\psi scales with the number of shared parameters PρP_{\rho}, the forecast variance takes the residuals-per-parameter form

Vρ(h)∝PρTIeff(ρ),RPP⁡(ρ)∝TIeff(ρ)Pρ.(42)V_{\rho}(h) \propto\frac{P_{\rho}}{T\mathcal{I}_{\mathrm{eff}}(\rho)},\qquad\operatorname{RPP}(\rho) \propto\frac{T\mathcal{I}_{\mathrm{eff}}(\rho)}{P_{\rho}}. \tag*{(42)}

The sharpest comparison is with multi-step, since both strategies pool a single ww and differ only in how the horizon objective is sampled: a discrete

grid of HH points for multi-step, the whole interval for Continuous Horizon. Their effective information is governed by the same kernel KK, so IeffMS\mathcal{I}_{\mathrm{eff}}^{\mathrm{MS}} is the Riemann sum that IeffCH\mathcal{I}_{\mathrm{eff}}^{\mathrm{CH}} integrates, and refining the grid past the correlation length adds only correlated scores, so IeffMS\mathcal{I}_{\mathrm{eff}}^{\mathrm{MS}} saturates at the continuum ceiling rather than growing with the nominal count. This is the continuous form of the classical bias–variance trade-off: the pooling that lowers the variance is bounded by the horizon correlation structure, and Continuous Horizon realizes that bound without committing to a fixed output grid.

Theorem 2. (Continuous Horizon Variance Dominance).

Suppose, in addition to Assumption 1, that the stationary correlation kernel K(u)K(u) is nonincreasing and strictly convex in ∣u∣|u|, and that the multi-step horizons form a regular grid on [0,H][0,H]. Then the continuous horizon, multi-step, and direct forecast strategies’ effective information satisfy

1=IeffDIR<IeffMS<IeffCH,(43)1 = \mathcal{I}_{\mathrm{eff}}^{\mathrm{DIR}} < \mathcal{I}_{\mathrm{eff}}^{\mathrm{MS}} < \mathcal{I}_{\mathrm{eff}}^{\mathrm{CH}}, \tag*{(43)}

with equality between multi-step and continuous horizon only in the fine-grid limit. Consequently, for a matched capacity of shared parameters PP (the shared trunk parameters, held fixed between multi-step and Continuous Horizon so that only the horizon sampling differs), the forecast estimation variance satisfies

VCH(h)<VMS(h)<VDIR(h),∀h∈[0,H],(44)V_{\mathrm{CH}}(h) < V_{\mathrm{MS}}(h) < V_{\mathrm{DIR}}(h), \qquad\forall h \in[0,H], \tag*{(44)}

so the continuous-horizon estimator attains strictly more variance reduction than any regular multi-step grid at any finite resolution.

Proof sketch. The direct strategy carries IeffDIR=1\mathcal{I}_{\mathrm{eff}}^{\mathrm{DIR}} = 1 by Corollary 2, while any positively correlated grid of more than one horizon gives IeffMS>1\mathcal{I}_{\mathrm{eff}}^{\mathrm{MS}} > 1, establishing the strict left inequality. For the right inequality, by Eq. (B.8) IeffMS<IeffCH\mathcal{I}_{\mathrm{eff}}^{\mathrm{MS}} < \mathcal{I}_{\mathrm{eff}}^{\mathrm{CH}} is equivalent to the grid average of KK exceeding its continuum average. For a strictly convex, nonincreasing KK on a regular grid, the rectangle rule strictly underestimates the integral of KK over each cell, so the grid double-average strictly dominates the double integral; inverting reverses the inequality. As the grid spacing vanishes, the rectangle rule converges to the integral, so equality holds only in the fine-grid limit. The variance ordering of Eq. (B.14) then follows from Eq. (B.9): at the matched shared-parameter capacity PP, the forecast variance scales as 1/Ieff(ρ)1/\mathcal{I}_{\mathrm{eff}}(\rho), so the information ordering inverts into the stated variance ordering. □

MSE Decomposition Experiment Details

Bias and Variance Analysis by Simulation

The bias and variance study of § 4 uses the three autoregressive processes of Ben Taieb and Atiya [5]. Each process has i.i.d. Gaussian innovations εt\varepsilon_t.

Smooth Transition Autoregression (STAR). This process has two regimes. A logistic function of yt−1y_{t-1} moves it smoothly between them [59]:

yt=0.3yt−1+0.6yt−2+(0.1−0.9yt−1+0.8yt−2)[1+e−10yt−1]−1+εt,(45)y_t = 0.3y_{t-1} + 0.6y_{t-2} + (0.1 - 0.9y_{t-1} + 0.8y_{t-2})\left[1 + e^{-10y_{t-1}}\right]^{-1} + \varepsilon_t, \tag*{(45)}

with εt∼N(0,σ2)\varepsilon_t \sim\mathcal{N}(0,\sigma^2) and σ=0.01\sigma= 0.01. Thus, the process is almost deterministic.

Nonlinear Autoregression (NLAR). This process is a neural network with two hidden units, fit to the annual sunspot series [59]. It has a linear part and two logistic gates:

yt=−0.17+0.85yt−1+0.14yt−2−0.31yt−3+0.08yt−7+12.80G1(yt−1)+2.44G2(yt−1)+εt,(46)\begin{aligned} y_t ={}& -0.17 + 0.85y_{t-1} + 0.14y_{t-2} - 0.31y_{t-3} + 0.08y_{t-7} \\ &+ 12.80G_1(y_{t-1}) + 2.44G_2(y_{t-1}) + \varepsilon_t, \tag*{(46)} \end{aligned}
G1(yt−1)=(1+exp⁡{−0.46(0.29yt−1−0.87yt−2+0.40yt−7−6.68)})−1,(47)G_1(y_{t-1}) = \left(1 + \exp\{-0.46(0.29y_{t-1} - 0.87y_{t-2} + 0.40y_{t-7} - 6.68)\}\right)^{-1}, \tag*{(47)}
G2(yt−1)=(1+exp⁡{−1.17×103(0.83yt−1−0.53yt−2−0.18yt−7+0.38)})−1,(48)G_2(y_{t-1}) = \left(1 + \exp\{-1.17 \times10^3(0.83y_{t-1} - 0.53y_{t-2} - 0.18y_{t-7} + 0.38)\}\right)^{-1}, \tag*{(48)}

with εt∼N(0,1)\varepsilon_t \sim\mathcal{N}(0,1).

Linear Autoregression (LAR). This process is an AR(6) fit to the annual sunspot series [9]:

yt=1.32yt−1−0.52yt−2−0.16yt−3+0.18yt−4−0.26yt−5+0.19yt−6+εt,(49)y_t = 1.32y_{t-1} - 0.52y_{t-2} - 0.16y_{t-3} + 0.18y_{t-4} - 0.26y_{t-5} + 0.19y_{t-6} + \varepsilon_t, \tag*{(49)}

with εt∼N(0,1)\varepsilon_t \sim\mathcal{N}(0,1). The process is linear, so σ\sigma only scales the series. It shows the cost of a flexible decoder when the true process is linear.

Training. Each training sample has N=8N = 8 short series. For each process, we discard the first 300 simulated values. We train each strategy for 2,500 to 3,000 steps on each sample. We use L=80L = 80 independent samples for the bias and variance estimates, and we report the horizons up to H=20H = 20. The series length changes as in Ben Taieb and Atiya [5].

NLAR MSE Decomposition

MSE decomposition plots for the NLAR process across training-series lengths and horizons

Figure C.8. MSE decomposition on the NLAR process [5] across training-series lengths TT and horizons hh. Direct and multi-step strategies reproduce the classical bias–variance trade-off, while Continuous Horizon (CH) attains the lowest total error, matching multi-step at short lengths and pulling ahead as TT increases.

STAR MSE Decomposition

MSE decomposition plots for the STAR process

Figure C.9. MSE decomposition on the STAR process [5], in the same layout as Figure C.8. The recurrent strategy exhibits visible bias from rollout limitations on the smooth-transition nonlinearity, whereas all other strategies—including Continuous Horizon (CH)—are near-optimal on this nearly deterministic process.

Unseen Horizons Experiment Details

Data. The experiment in Section 5 uses zero-mean Gaussian processes. A Gaussian process has a value at each real time, so it gives true targets at fractional and unseen horizons. We use three kernels, with τ=t−t′\tau= t - t':

KSE(τ)=exp⁡(−τ22ℓse2),KPer(τ)=exp⁡(−2sin⁡2(π∣τ∣/p)ℓ2),K_{\mathrm{SE}}(\tau) = \exp\left(-\frac{\tau^2}{2\ell_{\mathrm{se}}^2}\right), \qquad K_{\mathrm{Per}}(\tau) = \exp\left(-\frac{2\sin^2(\pi|\tau|/p)}{\ell^2}\right),
KLP(τ)=KPer(τ)KSE(τ).K_{\mathrm{LP}}(\tau) = K_{\mathrm{Per}}(\tau)K_{\mathrm{SE}}(\tau).

Table D.5 gives the kernel and the parameters of each regime. We add observation noise with σn=0.5\sigma_n = 0.5 to each series.

RegimeKernelℓ\ellppℓse\ell_{\mathrm{se}}
Mean-revertingKLPK_{\mathrm{LP}}31212
AperiodicKSEK_{\mathrm{SE}}––16
PeriodicKPerK_{\mathrm{Per}}312–
QP (slow)KLPK_{\mathrm{LP}}31248
QP (fast)KLPK_{\mathrm{LP}}31224

Table D.5. Kernels and parameters of the five Gaussian-process regimes.

Exact Conditional Mean. For a context yty_t, the posterior Gaussian-process gives the exact conditional mean μt+h∣t\mu_{t+h|t} at each real horizon hh. The posterior includes the same observation noise, so the regret of Eq. (14) is exact. The forecasters do not know the kernel.

Models. All strategies use the same two-layer encoder in a context window of 16 observations. The CH decoder is the branch–trunk expansion of Eq. (16). MS uses one shared output layer for all horizons, DIR uses one output layer for each horizon, and REC applies a one-step output layer recursively. MS and CH have approximately 5,000 parameters each.

Training and evaluation. Each training sample has 64 series. We train each strategy for 8,000 steps on the integer horizons h≤12h \le12. We repeat the training on R=40R = 40 independent samples. All strategies use pinball loss on quantile levels {0.05,0.1,0.25,0.5,0.75,0.9,0.95}\{0.05, 0.1, 0.25, 0.5, 0.75, 0.9, 0.95\}, and the point forecast is the median. For a Gaussian posterior, the median is equal to the conditional mean μt+h∣t\mu_{t+h|t}. We evaluate the integer horizons seen, the half-integer horizons, and the integer horizons 12<h≤2412 < h \le24.

Qualitative Forecast Exploration

Figure E.10. Forecasts of an oscillating Gaussian Process signal. Black: GP truth. Blue: CH. Red dashed: MS with linear interpolation, with markers at its output horizons. Left to right: coarse, intermediate, and full MS grid.

Here we complement the accuracy results of § 5, with a qualitative exploration of the outputs of discrete and CH forecast strategy.

The strategies MS, REC, DIR produce forecasts discretely with the output grid {0,1,…,H}\{0, 1, \ldots, H\}. For any other horizon h∈[0,H]h \in[0, H], to obtain the forecast, we must interpolate between two adjacent horizons.

Horizons therefore in the interpolation regime can be thought of as interpolant knots. The forecast accuracy between the knots depends on the distance Δ\Delta between them. For an output grid with nn intervals on [0,H][0, H], Δ=H/n\Delta= H/n.

For linear interpolation, the error between two knots is not more than Δ2/8\Delta^2/8 times the maximum curvature max⁡h∣f′′(h∣y:t)∣\max_h |f''(h \mid\mathbf{y}_{:t})| of the true forecast. Thus, if the grid has half the number of horizons, Δ\Delta doubles, and the error bound becomes four times larger [8]. A discrete forecast strategy can show an oscillation only if the distance between knots is less than half of the target signal period [56]. If the horizon distance is larger, the discrete strategies cannot contain the cycles. Then, no interpolation method can find the cycle again, and the result is aliasing. The coarse panel of Figure E.10 shows this result: the MS interpolant is an almost straight line.

The strategy CH bypasses discrete horizons. It learns the forecast as a continuous function of hh, so the interpolation error of the MS grid does not occur. But CH also trains only on integer horizons. Its accuracy between these horizons depends on the smoothness of the learned horizon basis functions ϕk(h)\phi_k(h), not on the density of the grid.

Horizon-Basis Ablation

The horizon basis ϕ^(h)\hat{\phi}(h) lets CH forecast at unseen horizons. All variants use the same encoder, parameter budget, training schedule, and branch coefficients. Only ϕ^(h)\hat{\phi}(h) changes.

Protocol. We use the Gaussian-process data of Appendix D, with 48 series in each training sample, 3,000 training steps, and R=24R = 24 replicates. Some kernel parameters are different from Table D.5. The mean-reverting regime uses KSEK_{\mathrm{SE}} with ℓse=3\ell_{\mathrm{se}} = 3, and the periodic regime uses ℓ=1.5\ell= 1.5. The slow and the fast quasi-periodic regimes use ℓse=24\ell_{\mathrm{se}} = 24 and ℓse=8\ell_{\mathrm{se}} = 8. The metric is the RMSE of Eq. (13).

Basis blocks. The basis uses five blocks. Each block is a function of the horizon hh, or a constant:

blin(h)=h/H,bquad(h)=(h/H)2,\mathbf{b}_{\mathrm{lin}}(h) = h/H, \qquad\mathbf{b}_{\mathrm{quad}}(h) = (h/H)^2,
bfour(h)=[sin⁡(ωkh), cos⁡(ωkh)]k=1m,btbl(h)=(1−λ)E⌊h⌋(tbl)+λE⌈h⌉(tbl),\mathbf{b}_{\mathrm{four}}(h) = [\sin(\omega_k h),\ \cos(\omega_k h)]_{k=1}^{m}, \qquad\mathbf{b}_{\mathrm{tbl}}(h) = (1-\lambda)\mathbf{E}^{(\mathrm{tbl})}_{\lfloor h \rfloor} + \lambda\mathbf{E}^{(\mathrm{tbl})}_{\lceil h \rceil},
bfreq=E(freq).\mathbf{b}_{\mathrm{freq}} = \mathbf{E}^{(\mathrm{freq})}.

The blocks blin\mathbf{b}_{\mathrm{lin}} and bquad\mathbf{b}_{\mathrm{quad}} give a polynomial trend. The Fourier block bfour\mathbf{b}_{\mathrm{four}} has fixed or learnable frequencies ωk\omega_k. The table block btbl\mathbf{b}_{\mathrm{tbl}} interpolates a learned embedding table E(tbl)∈RH×dt\mathbf{E}^{(\mathrm{tbl})} \in\mathbb{R}^{H \times d_t}, with λ=h−⌊h⌋\lambda= h - \lfloor h \rfloor. For h>Hh > H, it keeps the last value EH(tbl)\mathbf{E}^{(\mathrm{tbl})}_{H}. The block bfreq\mathbf{b}_{\mathrm{freq}} is a learned frequency embedding, and it does not depend on hh.

Composite basis. The basis is a linear combination of the blocks:

ϕ^(h)=∑b∈BAbbb(h),B={lin,quad,four,tbl,freq}.(50)\hat{\phi}(h) = \sum_{b \in\mathcal{B}} \mathbf{A}_b \mathbf{b}_b(h), \qquad\mathcal{B} = \{\mathrm{lin},\mathrm{quad},\mathrm{four},\mathrm{tbl},\mathrm{freq}\}. \tag*{(50)}

Each matrix Ab∈RK×db\mathbf{A}_b \in\mathbb{R}^{K \times d_b} is learned, with dbd_b the dimension of block bb. A variant removes block bb with Ab=0\mathbf{A}_b = 0. The basis does not depend on the history, so it is separable, as in Eq. (16).

Non-separable variants. Two variants use the encoder state θ^t\hat{\theta}_t in the basis. The gated variant multiplies the quadratic and the Fourier blocks by gates from the history, with Afreq=0\mathbf{A}_{\mathrm{freq}} = 0:

ϕ^(h)=Alinblin(h)+γquadAquadbquad(h)+γfourAfourbfour(h)+Atblbtbl(h),\hat{\phi}(h) = \mathbf{A}_{\mathrm{lin}}\mathbf{b}_{\mathrm{lin}}(h) + \gamma_{\mathrm{quad}}\mathbf{A}_{\mathrm{quad}}\mathbf{b}_{\mathrm{quad}}(h) + \gamma_{\mathrm{four}}\mathbf{A}_{\mathrm{four}}\mathbf{b}_{\mathrm{four}}(h) + \mathbf{A}_{\mathrm{tbl}}\mathbf{b}_{\mathrm{tbl}}(h),
(γquad,γfour)=σgate(θ^t).(51)(\gamma_{\mathrm{quad}},\gamma_{\mathrm{four}}) = \sigma_{\mathrm{gate}}(\hat{\theta}_t). \tag*{(51)}

The per-series variant uses the frequency embedding (Afreq≠0\mathbf{A}_{\mathrm{freq}} \ne0) and replaces E(freq)\mathbf{E}^{(\mathrm{freq})} with E(freq)(θ^t)\mathbf{E}^{(\mathrm{freq})}(\hat{\theta}_t), which the encoder gives. In both variants, the basis depends on the history, so it is not separable.

Ablations. Each ablation changes one part of Eq. (F.1) or Eq. (F.2). The anchor uses all blocks except the frequency embedding (Afreq=0\mathbf{A}_{\mathrm{freq}} = 0), with learnable ωk\omega_k and K=16K = 16. The ablations are:

  • Composition (Table F.6): all seven combinations of the polynomial (lin and quad), the Fourier, and the table blocks, with Afreq=0\mathbf{A}_{\mathrm{freq}} = 0 and Ab=0\mathbf{A}_b = 0 for each unused block.

  • Basis Dimensionality (Table F.7): K∈{2,4,8,16,32}K \in\{2, 4, 8, 16, 32\}.

  • Frequency representation (Table F.8): fixed or learnable ωk\omega_k, the frequency embedding with and without the Fourier block (Afour=0\mathbf{A}_{\mathrm{four}} = 0), the per-series embedding, and the gate.

In Table F.8, the Sep. column shows the separable variants, to which Appendix § 9 and Appendix B apply. MS and DIR are not in these tables, because they cannot forecast at unseen horizons.

Results. On the seen and the interpolation horizons, all variants have almost the same error. The exceptions are the polynomial-only basis and K=2K = 2 on the periodic regime. On the extrapolation horizons, the variants are different:

  • No composition is the best in all regimes. The table-only basis is the best in three regimes.

  • The polynomial-only basis has the highest error in three regimes.

  • From K=4K = 4 to K=32K = 32, the error marginally decreases by 5% or less.

  • Without the Fourier block, the frequency embedding has a higher error on the periodic regimes.

  • The non-separable variants have the lowest error in four regimes. Their advantage over the best separable variant is only 1.3% or less.

RegimeBasis compositionSeenInterp.Extrap.
Mean-revertingpolynomial only1.0921.1001.610
Mean-revertingFourier only1.0921.1011.226
Mean-revertingtable only1.0911.0991.153
Mean-revertingpolynomial + Fourier1.0941.1031.263
Mean-revertingpolynomial + table1.0911.0991.333
Mean-revertingFourier + table1.0911.0991.188
Mean-revertingfull composite (anchor)1.0921.1011.217
Aperiodicpolynomial only0.7150.7131.077
AperiodicFourier only0.7150.7131.085
Aperiodictable only0.7150.7121.030
Aperiodicpolynomial + Fourier0.7150.7131.057
Aperiodicpolynomial + table0.7150.7121.054
AperiodicFourier + table0.7150.7131.065
Aperiodicfull composite (anchor)0.7150.7121.054
Periodicpolynomial only0.6740.6585.339
PeriodicFourier only0.6430.6380.740
Periodictable only0.6420.6330.925
Periodicpolynomial + Fourier0.6420.6370.762
Periodicpolynomial + table0.6420.6331.138
PeriodicFourier + table0.6420.6360.691
Periodicfull composite (anchor)0.6410.6350.729
Quasi-periodic (slow)polynomial only0.7000.6981.391
Quasi-periodic (slow)Fourier only0.6830.6830.943
Quasi-periodic (slow)table only0.6900.6880.935
Quasi-periodic (slow)polynomial + Fourier0.6830.6830.929
Quasi-periodic (slow)polynomial + table0.6910.6890.989
Quasi-periodic (slow)Fourier + table0.6830.6830.926
Quasi-periodic (slow)full composite (anchor)0.6830.6820.919
Quasi-periodic (fast)polynomial only0.9830.9841.228
Quasi-periodic (fast)Fourier only0.9810.9831.280
Quasi-periodic (fast)table only0.9810.9821.175
Quasi-periodic (fast)polynomial + Fourier0.9820.9841.210
Quasi-periodic (fast)polynomial + table0.9810.9831.218
Quasi-periodic (fast)Fourier + table0.9800.9821.242
Quasi-periodic (fast)full composite (anchor)0.9820.9831.202

Table F.6. Ablation of the basis composition. RMSE on the seen, interpolation, and extrapolation horizons of the five regimes. The anchor is the full composite basis with learnable frequencies and K=16K = 16. Bold shows the lowest extrapolation error.

RegimeKKSeenInterp.Extrap.
Mean-revertingK=2K = 21.0911.1021.267
K=4K = 41.0931.1031.234
K=8K = 81.0931.1021.229
K=16K = 161.0921.1011.217
K=32K = 321.0921.1001.203
AperiodicK=2K = 20.7160.7161.059
K=4K = 40.7150.7131.054
K=8K = 80.7150.7131.055
K=16K = 160.7150.7121.054
K=32K = 320.7150.7121.052
PeriodicK=2K = 20.6650.7041.529
K=4K = 40.6410.6400.748
K=8K = 80.6410.6380.740
K=16K = 160.6410.6350.729
K=32K = 320.6430.6360.713
Quasi-periodic (slow)K=2K = 20.6990.7010.971
K=4K = 40.6840.6840.934
K=8K = 80.6830.6830.930
K=16K = 160.6830.6820.919
K=32K = 320.6830.6820.915
Quasi-periodic (fast)K=2K = 20.9820.9871.241
K=4K = 40.9810.9831.213
K=8K = 80.9810.9831.201
K=16K = 160.9820.9831.202
K=32K = 320.9810.9821.199

Table F.7. Ablation of the number KK of basis functions, with the full composite basis. RMSE on the seen, interpolation, and extrapolation horizons of the five regimes.

RegimeFrequency representationSeenInterp.Extrap.Sep.
Mean-revertingfixed Fourier dictionary1.0921.1011.234✓
Mean-revertinglearnable Fourier (anchor)1.0921.1011.217✓
Mean-revertingfrequency embedding, shared1.0931.1021.213✓
Mean-revertingfrequency embedding, per-series1.0921.1001.204✗
Mean-revertingfrequency embedding only (no Fourier)1.0911.0991.312✓
Mean-revertinggated composite1.0921.1001.212✗
Aperiodicfixed Fourier dictionary0.7150.7131.047✓
Aperiodiclearnable Fourier (anchor)0.7150.7121.054✓
Aperiodicfrequency embedding, shared0.7150.7131.049✓
Aperiodicfrequency embedding, per-series0.7160.7131.052✗
Aperiodicfrequency embedding only (no Fourier)0.7150.7121.046✓
Aperiodicgated composite0.7150.7131.036✗
Periodicfixed Fourier dictionary0.6420.6370.746✓
Periodiclearnable Fourier (anchor)0.6410.6350.729✓
Periodicfrequency embedding, shared0.6420.6350.703✓
Periodicfrequency embedding, per-series0.6420.6350.701✗
Periodicfrequency embedding only (no Fourier)0.6420.6331.098✓
Periodicgated composite0.6420.6350.733✗
Quasi-periodic (slow)fixed Fourier dictionary0.6830.6830.917✓
Quasi-periodic (slow)learnable Fourier (anchor)0.6830.6820.919✓
Quasi-periodic (slow)frequency embedding, shared0.6830.6820.914✓
Quasi-periodic (slow)frequency embedding, per-series0.6840.6830.918✗
Quasi-periodic (slow)frequency embedding only (no Fourier)0.6900.6880.981✓
Quasi-periodic (slow)gated composite0.6850.6840.938✗
Quasi-periodic (fast)fixed Fourier dictionary0.9810.9831.195✓
Quasi-periodic (fast)learnable Fourier (anchor)0.9820.9831.202✓
Quasi-periodic (fast)frequency embedding, shared0.9810.9831.199✓
Quasi-periodic (fast)frequency embedding, per-series0.9800.9821.200✗
Quasi-periodic (fast)frequency embedding only (no Fourier)0.9810.9831.205✓
Quasi-periodic (fast)gated composite0.9820.9831.180✗

Table F.8. Ablation of the frequency representation. RMSE on the seen, interpolation, and extrapolation horizons of the five regimes. A check mark in the Sep. column shows a separable basis. Bold shows the lowest extrapolation error in each regime.

Dataset Details

FrequencySeasonalityHorizonSeriesMin LengthMax Length% Erratic
M1Monthly1218617481500
M1Quarterly48203181140
M1Yearly1618115580
M3Other48174711040
M3Monthly12181428661442
M3Quarterly4875624721
M3Yearly16645204710
M4Hourly2448414748100817
M4Daily1144,22710799332
M4Weekly11335993261016
M4Monthly121848,0006028126
M4Quarterly4824,0002487411
M4Yearly1623,0001984118
TourismMonthly12183669133351
TourismQuarterly484273013039
TourismYearly16518114723

Table G.9. Summary of forecasting datasets used in our main empirical study.

M1. The M1 competition [40] has 1,001 monthly, quarterly, and yearly series from demography, industry, and economics. In this competition, simple methods such as ETS [25] were frequently more accurate than complex methods. The M1 competition initiated a research legacy that emphasized forecast accuracy.

M3. The M3 competition [41] has 3,003 series of business, demography, finance, and economics. All values are positive, and no series is intermittent [58]. The Theta method [30] had the best overall accuracy.

M4. The M4 competition [42] has 100,000 series at six frequencies, from demography, finance, industry, and economics. It added the evaluation of prediction intervals. ESRNN [57], a hybrid of exponential smoothing and a neural network, won the competition. M4 also made cross-learning and global models popular [55].

Tourism. The Tourism dataset [3] has 1,311 monthly, quarterly, and yearly series of tourism demand. This competition was the first to use the mean absolute scaled error (MASE). Of all series, 36% are erratic or intermittent. At the yearly frequency, few methods were more accurate than the Naïve1 method.

Training Methodology and Hyperparameters

Network Hyperparameters

We train MQForecaster varying encoders between LSTM, CNN, Transformer, and S4 modules. We train models with hyperparameter tuning using Optuna. Using 5 trials, we train individual models and select the hyperparameters that minimize the validation RMSE. Training times mostly depend on the architecture; however, we restrict the SGD training steps to 45K per architecture. Tables H.10 and H.11–H.14 list the shared hyperparameters and encoder-specific hyperparameters, respectively.

HyperparameterValues
Single GPU SGD Batch Size{sup}*{4,8,16}\{4, 8, 16\}
Initial learning rateloguniform(1e-4, 5e-2)
Maximum Training steps Smax⁡S_{\max}45,000
Learning rate decay0.1
Learning rate step size15,000
Scaler typeStandard
H-Agnostic Decoder Dimension100
H-Specific Decoder Dimension20

Table H.10. Shared hyperparameters across encoder types

HyperparameterValues
Main Activation FunctionsSigmoid, tanh
Encoder Dimension{128,256}\{128, 256\}
Dilations{{[1,2],[4,8]},{[1,2],[4,8],[16,32]}}\{\{[1,2], [4,8]\}, \newline\{[1,2], [4,8], [16,32]\}\}

Table H.11. LSTM

HyperparameterValues
Main Activation FunctionReLU
Encoder Dimension{128,256}\{128, 256\}
Temporal Convolution Kernel Size{2,5}\{2, 5\}
Temporal Convolution Dilations{[1,2,4,8],[1,2,4,8,16,32]}\{[1, 2, 4, 8], [1, 2, 4, 8, 16, 32]\}

Table H.12. CNN

HyperparameterValues
Main Activation FunctionsSigmoid, tanh
Encoder Dimension{128,256}\{128, 256\}
Projection Dimension{128,256}\{128, 256\}
Input Patch Lengths{1,8,32,96}\{1, 8, 32, 96\}
Attention Dilations{[2,6,8],[2,4,8,16]}\{[2, 6, 8], [2, 4, 8, 16]\}
Attention Dropout0.1
Number of Attention Heads4

Table H.13. Transformer

HyperparameterValues
Main Activation FunctionGELU
Encoder Dimension{128,256}\{128, 256\}
Projection Hidden Size{128,256}\{128, 256\}

Table H.14. State Space S4

Parameter-Matched Strategy Comparison

We compare the strategies at the same number of parameters, so that an accuracy difference comes from the horizon strategy, not from the model size. The decoders of MS and CH scale differently with the horizon HH. The MS decoder has one output block for each horizon, so its size increases with HH. The CH decoder uses a basis ϕ(h)\phi(h) with a fixed size. Without matching, MS has more parameters than CH, and the difference increases with HH.

To match parameters, we marginally decrease the width of the encoder output for MS, so that the totals agree. For example, the width is 32 for CH and 19 for MS with the CNN encoder. Then the totals are different by 3.1% or less (Table H.15). We count the parameters directly, because the same width gives different counts for different decoders.

ModelLSTMCNNTransformerS4
CHMSDIR*RECCHMSDIR*RECCHMSDIR*RECCHMSDIR*REC
Global model78,48277,8657,17878,48263,15465,1259,47763,15481,74680,8688,64181,74685,71486,67413,96185,714

Table H.15. Trainable MQForecaster parameters for each encoder and strategy. A dash marks a strategy not trained with that encoder; counts are functional (CH basis/embeddings disabled for DIR and MS). CH, MS, and REC are single global models. ∗DIR trains one model per horizon; the listed count is per horizon, split as encoder + decoder (CNN: 8,006+1,174). The encoder matches the global model; the decoder is 1/H1/H of the global decoder, so over H=18H=18 horizons the DIR decoders sum to ≈\approx one global decoder (18×1,174=21,13218 \times1,174=21,132, vs. 19,221) while the encoder is replicated (18×8,00618 \times8,006).

Regret-Based Dataset Sampling for Cross-Learning

We train one global model on the 16 datasets of § 6, by cross-learning [55]. The datasets are very different in size, so proportional sampling gives the small datasets few gradient updates. Thus, we change the sampling probability of each dataset during training. A dataset gets a larger probability when its validation error is high relative to a reference for that dataset. This method is a variant of group distributionally robust optimization [53] and of data-mixture reweighting of Xie et al. [65].

The Continuous-Horizon Learning Objective

MS Learning Objective. Theorem 2 shows the benefit of training on fractional horizons. But the M-series experiments compare the inductive bias of the decoders, with the learning objective fixed. Thus, we train CH on the integer horizons h∈{1,…,H}h \in\{1,\ldots,H\}, with the traditional MS objective. The uniform measure 1H dh\frac{1}{H}\,\mathrm{d}h of Eq. (6) then becomes the uniform measure on the integer grid:

w^:=arg⁡min⁡w1T×H∑t=1T∑h=1HL(yt+h,g(h∣y:t,w)).(52)\hat{\mathbf{w}} := \arg\min_{\mathbf{w}} \frac{1}{T \times H} \sum_{t=1}^{T}\sum_{h=1}^{H} L\left(y_{t+h}, g\left(h \mid\mathbf{y}_{:t}, \mathbf{w}\right)\right). \tag*{(52)}

The loss LL is the MSE. MS minimizes Eq. (H.1) with all HH horizons in each training step. For CH, we used the same loss and the same pooled data, but we sampled the horizons.

CH Learning Objective. In each training step, we sample a random subset S⊆{1,…,H}S \subseteq\{1,\ldots,H\} of the horizons for the full mini-batch. We drop each horizon independently with probability 0.2 and we keep at least 10 horizons. Each horizon has the same probability to be in SS, so

ES[1∣S∣∑h∈SL(yt+h,g(h∣y:t,w))]=1H∑h=1HL(yt+h,g(h∣y:t,w)).(53)\mathbb{E}_{S}\left[\frac{1}{\lvert S\rvert}\sum_{h\in S} L\left(y_{t+h}, g\left(h \mid\mathbf{y}_{:t}, \mathbf{w}\right)\right)\right] = \frac{1}{H}\sum_{h=1}^{H} L\left(y_{t+h}, g\left(h \mid\mathbf{y}_{:t}, \mathbf{w}\right)\right). \tag*{(53)}

Thus, a training step with SS minimizes Eq. (H.1) in expectation. In our experiments, the number of sampled horizons changed the validation accuracy by less than 2%.

Additional Point Forecast Metrics

We complement the relAvgRMSE of Table 3 with the two absolute-error metrics most used in the M-competitions, so that our results can be read against previous work on these datasets. Both metrics use the mean forecast y^t+h\hat{y}_{t+h} and the pool over all series ii, the forecast creation dates t∈[0,1,…,T]t \in[0,1,\ldots,T] and the horizons hh of the temporal cross-validation of § 6.

LSTM CHLSTM MSLSTM DIRLSTM RECCNN CHCNN MSCNN DIRCNN RECTransformer CHTransformer MSTransformer DIRTransformer RECS4 CHS4 MSS4 DIRS4 RECStatsForecast ETSStatsForecast ARIMA
M1M0.8838 (0.1518)0.6001 (0.0921)0.2027 (0.0007)0.8644 (0.1172)0.1914 (0.0009)0.1945 (0.0007)0.1974 (0.0027)0.2308 (0.0064)0.1953 (0.0041)0.2104 (0.0050)0.2126 (0.0023)0.2335 (0.0073)0.1954 (0.0004)0.1975 (0.0013)0.2077 (0.0029)0.2315 (0.0047)0.1693 (-)0.1807 (-)
M1Q1.0103 (0.1186)0.5547 (0.0914)0.1508 (0.0023)0.9659 (0.1832)0.1410 (0.0006)0.1369 (0.0010)0.1431 (0.0018)0.1479 (0.0017)0.1456 (0.0028)0.1505 (0.0062)0.1615 (0.0046)0.1574 (0.0041)0.1456 (0.0010)0.1425 (0.0021)0.1533 (0.0011)0.1596 (0.0053)0.1624 (-)0.1881 (-)
M1Y1.2673 (0.2036)1.0504 (0.2436)0.1622 (0.0021)1.2674 (0.2578)0.1601 (0.0022)0.1490 (0.0026)0.1504 (0.0023)0.1565 (0.0041)0.1690 (0.0019)0.1773 (0.0051)0.1761 (0.0072)0.1616 (0.0045)0.1626 (0.0025)0.1612 (0.0023)0.1785 (0.0042)0.1674 (0.0041)0.2188 (-)0.1908 (-)
M3O0.4495 (0.1494)0.2826 (0.0485)0.0469 (0.0013)0.3481 (0.0925)0.0360 (0.0003)0.0416 (0.0003)0.0471 (0.0013)0.0505 (0.0023)0.0477 (0.0015)0.0527 (0.0031)0.0846 (0.0267)0.0572 (0.0064)0.0417 (0.0008)0.0408 (0.0030)0.0510 (0.0043)0.0593 (0.0036)0.0433 (-)0.0438 (-)
M3M0.5901 (0.1733)0.4287 (0.0836)0.1583 (0.0008)0.5309 (0.1017)0.1486 (0.0002)0.1510 (0.0009)0.1510 (0.0007)0.1887 (0.0029)0.1542 (0.0026)0.1605 (0.0017)0.1663 (0.0038)0.1904 (0.0068)0.1488 (0.0003)0.1487 (0.0011)0.1513 (0.0014)0.1854 (0.0036)0.1627 (-)0.1645 (-)
M3Q0.8036 (0.1348)0.3693 (0.0364)0.1073 (0.0014)0.7295 (0.1438)0.0986 (0.0010)0.0985 (0.0006)0.1039 (0.0010)0.1099 (0.0017)0.1047 (0.0028)0.1122 (0.0062)0.1214 (0.0046)0.1147 (0.0041)0.0981 (0.0010)0.0920 (0.0021)0.1115 (0.0011)0.1156 (0.0053)0.1014 (-)0.1020 (-)
M3Y1.1225 (0.1953)0.8359 (0.1592)0.1170 (0.0020)1.1435 (0.1755)0.1658 (0.0018)0.1573 (0.0019)0.1508 (0.0018)0.1677 (0.0043)0.1624 (0.0046)0.1773 (0.0058)0.1753 (0.0073)0.1687 (0.0049)0.1658 (0.0024)0.1547 (0.0028)0.1680 (0.0042)0.1769 (0.0048)0.1971 (-)0.1953 (-)
M4H0.8672 (0.1765)0.6679 (0.1299)0.2222 (0.0022)0.8421 (0.1158)0.2236 (0.0019)0.2228 (0.0028)0.2327 (0.0034)0.3135 (0.0094)0.2084 (0.0018)0.2328 (0.0174)0.2972 (0.0039)0.3703 (0.0170)0.2274 (0.0008)0.1921 (0.0014)0.1788 (0.0031)0.3492 (0.0066)0.2336 (-)0.1426 (-)
M4D0.2972 (0.1323)0.1540 (0.0170)0.0319 (0.0039)0.1878 (0.1878)0.0302 (0.0002)0.0303 (0.0003)0.0304 (0.0004)0.0934 (0.0094)0.0322 (0.0019)0.0334 (0.0174)0.0334 (0.0034)0.0370 (0.0170)0.0308 (0.0008)0.0301 (0.0014)0.0318 (0.0031)0.0371 (0.0066)0.0280 (-)0.0285 (-)
M4W0.4587 (0.1493)0.2629 (0.0558)0.0979 (0.0014)0.3781 (0.0862)0.0887 (0.0007)0.0930 (0.0010)0.0929 (0.0019)0.0942 (0.0017)0.0956 (0.0017)0.0974 (0.0126)0.1003 (0.0025)0.1052 (0.0054)0.0925 (0.0013)0.0914 (0.0014)0.0971 (0.0031)0.1031 (0.0066)0.0903 (-)0.0878 (-)
M4M0.5515 (0.1717)0.5817 (0.1170)0.1445 (0.0044)0.4866 (0.0866)0.1397 (0.0013)0.1402 (0.0012)0.1397 (0.0017)0.1597 (0.0017)0.1370 (0.0017)0.1428 (0.0128)0.1533 (0.0025)0.1785 (0.0170)0.1357 (0.0013)0.1401 (0.0014)0.1394 (0.0031)0.1726 (0.0066)0.1321 (-)0.1381 (-)
M4Q0.6464 (0.1166)0.2833 (0.0199)0.1199 (0.0017)0.5502 (0.1442)0.1124 (0.0011)0.1117 (0.0017)0.1117 (0.0017)0.1220 (0.0017)0.1238 (0.0025)0.1291 (0.0127)0.1291 (0.0025)0.1327 (0.0170)0.1152 (0.0012)0.1174 (0.0014)0.1176 (0.0031)0.1273 (0.0066)0.1045 (-)0.1070 (-)
M4Y1.1435 (0.1617)0.7597 (0.0867)0.1561 (0.0021)1.1286 (0.1599)0.1503 (0.0014)0.1426 (0.0019)0.1424 (0.0019)0.1524 (0.0022)0.1466 (0.0017)0.1528 (0.0170)0.1602 (0.0025)0.1473 (0.0170)0.1555 (0.0013)0.1421 (0.0014)0.1545 (0.0031)0.1585 (0.0066)0.1546 (-)0.1616 (-)
TourismM1.0228 (0.1839)0.7346 (0.1126)0.3498 (0.0098)1.0463 (0.1921)0.3180 (0.0044)0.3200 (0.0033)0.3198 (0.0056)0.4517 (0.0214)0.3605 (0.0063)0.3727 (0.0055)0.3766 (0.0071)0.4672 (0.0291)0.3159 (0.0022)0.3024 (0.0017)0.3089 (0.0049)0.4662 (0.0133)0.2220 (-)0.2206 (-)
TourismQ0.9581 (0.1454)0.4654 (0.0601)0.2833 (0.0023)0.9156 (0.1916)0.2221 (0.0037)0.2324 (0.0024)0.2588 (0.0058)0.2839 (0.0190)0.2947 (0.0049)0.2990 (0.0029)0.3021 (0.0071)0.2975 (0.0221)0.2179 (0.0023)0.2166 (0.0022)0.2585 (0.0055)0.2856 (0.0133)0.1797 (-)0.1785 (-)
TourismY1.3570 (0.2633)1.0668 (0.1969)0.2241 (0.0033)1.1680 (0.2712)0.2076 (0.0026)0.2179 (0.0037)0.2016 (0.0033)0.2086 (0.0033)0.1865 (0.0029)0.1975 (0.0028)0.2052 (0.0049)0.2059 (0.0133)0.2077 (0.0023)0.1973 (0.0022)0.2052 (0.0055)0.2041 (0.0133)0.2870 (-)0.3030 (-)

Table I.16. Empirical evaluation of point forecasts. Symmetric Mean Absolute Percentage Error (sMAPE, Eq. (I.(1)), averaged over 5 runs. Lower measurements are preferred. The methods without standard deviation have deterministic solutions. For each encoder we compare four forecasting strategies: our Continuous Horizon (CH), multi-step (MS), direct (DIR), and recurrent (REC).

The symmetric mean absolute percentage error is

sMAPE⁡=1N×T×H∑i∑t∑h2∣yt+h(i)−y^t+h(i)∣∣yt+h(i)∣+∣y^t+h(i)∣(54)\operatorname{sMAPE} = \frac{1}{N \times T \times H} \sum_i \sum_t \sum_h \frac{2\lvert y_{t+h}^{(i)}-\hat{y}_{t+h}^{(i)}\rvert}{\lvert y_{t+h}^{(i)}\rvert+\lvert\hat{y}_{t+h}^{(i)}\rvert} \tag*{(54)}
LSTMCNNTransformerS4StatsForecast
DatasetFreqCHMSDIRRECCHMSDIRRECCHMSDIRRECCHMSDIRRECETSARIMA
M1M0.8129 (0.0062)0.8817 (0.0054)0.9279 (0.0045)0.9403 (0.0168)0.8295 (0.0048)0.9047 (0.0080)0.9196 (0.0217)1.0171 (0.0439)0.8971 (0.0231)0.9421 (0.0097)0.9732 (0.0225)1.0009 (0.0101)0.9019 (0.0032)0.8957 (0.0097)0.9437 (0.0282)1.0026 (0.0143)0.7319 (-)0.7660 (-)
Q0.9274 (0.0268)0.9426 (0.0140)0.9765 (0.0262)0.9207 (0.0211)0.9095 (0.0047)0.9353 (0.0136)0.9897 (0.0204)1.2168 (0.1247)1.0114 (0.0771)1.0400 (0.0244)1.0644 (0.0465)0.9591 (0.0228)0.9560 (0.0201)0.9363 (0.0144)1.0444 (0.0143)0.9269 (0.0204)0.8980 (-)0.8899 (-)
Y0.8187 (0.0209)0.7250 (0.0253)0.7733 (0.0274)0.9049 (0.0833)0.7830 (0.0132)0.8005 (0.0391)0.6960 (0.0475)0.7472 (0.0430)0.6603 (0.0319)0.8033 (0.0853)0.8408 (0.0841)0.7948 (0.0561)0.7373 (0.0180)0.8117 (0.0378)0.7340 (0.0509)0.8250 (0.0534)0.7838 (-)0.7534 (-)
M3O0.2018 (0.0021)0.2106 (0.0219)0.2080 (0.0065)0.2241 (0.0137)0.1552 (0.0014)0.1859 (0.0033)0.2074 (0.0068)0.3381 (0.0660)0.2191 (0.0118)0.2490 (0.0097)0.3886 (0.1386)0.2517 (0.0209)0.1786 (0.0018)0.1754 (0.0118)0.2250 (0.0163)0.2634 (0.0159)0.6915 (-)0.7040 (-)
M0.6646 (0.0049)0.6846 (0.0077)0.7538 (0.0068)0.8394 (0.0257)0.6967 (0.0012)0.7135 (0.0032)0.7131 (0.0034)1.0855 (0.0458)0.7364 (0.0182)0.7654 (0.0109)0.7978 (0.0207)0.9903 (0.0668)0.7033 (0.0022)0.7012 (0.0053)0.7147 (0.0040)0.9417 (0.0254)0.8555 (-)0.8622 (-)
Q0.7772 (0.0097)0.7874 (0.0118)0.8512 (0.0120)0.8999 (0.0217)0.7719 (0.0090)0.7832 (0.0091)0.8322 (0.0147)1.0647 (0.0942)0.8695 (0.0523)0.9260 (0.0376)0.9670 (0.0166)0.9168 (0.0434)0.7687 (0.0090)0.7247 (0.0081)0.8996 (0.0044)0.9328 (0.0118)0.9277 (-)0.9257 (-)
Y0.8905 (0.0141)0.7684 (0.0093)0.8637 (0.0104)0.9120 (0.0521)0.8189 (0.0118)0.7786 (0.0093)0.7543 (0.0143)0.9559 (0.0705)0.8759 (0.0725)0.8620 (0.0455)0.8649 (0.0300)0.8243 (0.0249)0.8637 (0.0119)0.7666 (0.0105)0.8610 (0.0106)0.8814 (0.0082)0.9716 (-)0.9868 (-)
M4H0.1617 (0.0013)0.1703 (0.0052)0.3738 (0.0031)0.6322 (0.0592)0.2715 (0.0015)0.4464 (0.0047)0.4665 (0.0063)0.6923 (0.0592)0.2363 (0.0156)0.4631 (0.0370)0.6826 (0.0295)0.9682 (0.1702)0.3018 (0.0138)0.2829 (0.0062)0.2617 (0.0058)0.5793 (0.0393)0.4148 (-)0.1899 (-)
D0.7924 (0.0031)0.7412 (0.0130)0.8025 (0.0059)0.8704 (0.0393)0.7872 (0.0075)0.7749 (0.0077)0.7940 (0.0099)0.8499 (0.0557)0.8241 (0.0204)0.7998 (0.0171)0.8317 (0.0295)0.9598 (0.0594)0.7759 (0.0276)0.7461 (0.0091)0.8034 (0.0178)0.9363 (0.0377)0.9787 (-)0.9892 (-)
W0.7441 (0.0131)0.6873 (0.0077)0.6713 (0.0113)0.7483 (0.0748)0.7249 (0.0049)0.7357 (0.0077)0.7323 (0.0074)0.8055 (0.0582)0.7847 (0.0178)0.7815 (0.0208)0.7907 (0.0297)0.8394 (0.0585)0.7427 (0.0074)0.7240 (0.0062)0.7747 (0.0178)0.8377 (0.0359)0.9470 (-)0.9179 (-)
M0.6707 (0.0069)0.6889 (0.0079)0.7013 (0.0078)0.7915 (0.0615)0.6857 (0.0047)0.6847 (0.0077)0.6791 (0.0075)0.9764 (0.0836)0.6821 (0.0191)0.7108 (0.0178)0.7503 (0.0306)0.9730 (0.0972)0.6589 (0.0074)0.6818 (0.0067)0.6799 (0.0178)0.9073 (0.0395)0.8897 (-)0.9006 (-)
Q0.7112 (0.0035)0.7054 (0.0084)0.7606 (0.0115)0.7661 (0.0248)0.7034 (0.0052)0.7037 (0.0064)0.7162 (0.0155)0.8519 (0.0558)0.8109 (0.0197)0.7952 (0.0166)0.8268 (0.0306)0.8831 (0.0704)0.7290 (0.0066)0.7425 (0.0038)0.7624 (0.0150)0.8340 (0.0283)0.8727 (-)0.9168 (-)
Y0.7954 (0.0031)0.6783 (0.0061)0.7837 (0.0051)0.8242 (0.0652)0.7364 (0.0077)0.7151 (0.0070)0.7143 (0.0155)0.8398 (0.0558)0.7959 (0.0597)0.7966 (0.0575)0.7956 (0.0294)0.7277 (0.0254)0.7744 (0.0066)0.7088 (0.0038)0.7888 (0.0049)0.8042 (0.0061)0.8700 (-)0.9138 (-)
TourismM0.5549 (0.0335)0.5803 (0.0174)0.7457 (0.0161)0.7938 (0.0465)0.6228 (0.0095)0.6564 (0.0132)0.6554 (0.0190)0.9876 (0.0936)0.8165 (0.0240)0.8057 (0.0186)0.8230 (0.0139)1.0525 (0.1068)0.6349 (0.0107)0.6108 (0.0171)0.6114 (0.0065)1.1097 (0.0244)0.4376 (-)0.5105 (-)
Q0.6349 (0.0128)0.6428 (0.0142)0.7976 (0.0227)0.7594 (0.0459)0.5982 (0.0173)0.6236 (0.0327)0.7131 (0.0241)0.8791 (0.0860)0.8221 (0.0577)0.8556 (0.0523)0.8282 (0.0282)0.8397 (0.0431)0.5811 (0.0221)0.6212 (0.0084)0.7454 (0.0122)0.8607 (0.0384)0.6232 (-)0.6894 (-)
Y0.8995 (0.0198)0.8550 (0.0295)0.9295 (0.0254)0.9524 (0.0542)0.8570 (0.0254)0.9025 (0.0279)0.8281 (0.0241)0.8323 (0.0233)0.7907 (0.0260)0.8570 (0.0377)0.9087 (0.0537)0.8869 (0.0269)0.8973 (0.0221)0.8785 (0.0221)0.9466 (0.0328)0.8508 (0.0261)0.8804 (-)1.0160 (-)

Table I.17. Empirical evaluation of point forecasts. Mean absolute scaled error (MASE, Eq. (I.(2)) with the Naive baseline, averaged over 5 runs. Lower measurements are preferred; values below 1 beat the Naive. The methods without standard deviation have deterministic solutions. For each encoder we compare four forecasting strategies: our Continuous Horizon (CH), multi-step (MS), direct (DIR), and recurrent (REC).

The mean absolute scaled error [32] scales the absolute error by that of the Naive forecast y~t+h(i)\tilde{y}^{(i)}_{t+h}, which repeats the last observation at the forecast creation date over the whole horizon:

MASE⁡=∑i∑t∑h∣yt+h(i)−y^t+h(i)∣∑i∑t∑h∣yt+h(i)−y~t+h(i)∣(55)\operatorname{MASE} = \frac{\sum_i \sum_t \sum_h \left|y^{(i)}_{t+h} - \hat{y}^{(i)}_{t+h}\right|}{\sum_i \sum_t \sum_h \left|y^{(i)}_{t+h} - \tilde{y}^{(i)}_{t+h}\right|} \tag*{(55)}

We scaled by the Naive error in the same forecast windows, as in the relAv-gRMSE, rather than by the seasonal error in-sample Naive in M4’s definition.

Universal Periodic Operator Approximation Theorem

Theorem 1 guarantees the approximation only in the training interval [0,H][0,H]. For periodic targets, a period that fits within the training interval is sufficient to extend the guaranty to all horizons h≥0h \ge0.

Theorem 3 (Universal Periodic Operator Approximation Theorem). Assume the conditions of Theorem 1. Additionally, assume that for every history y:t∈Yy_{:t} \in\mathcal{Y} the forecast function f(⋅∣y:t)f(\cdot\mid y_{:t}) extends to all h≥0h \ge0 and is periodic with a period J<HJ < H,

f(h+J∣y:t)=f(h∣y:t),h≥0,y:t∈Y.(56)f(h+J \mid y_{:t}) = f(h \mid y_{:t}), \qquad h \ge0,\quad y_{:t} \in\mathcal{Y}. \tag*{(56)}

Then for any accuracy ϵ>0\epsilon> 0, there exists a neural network approximation

gper(h∣y:t,w⋆)=∑k=1Kθk(y:t)⋅ϕk(h mod J)g_{\mathrm{per}}(h \mid y_{:t}, \mathbf{w}^{\star}) = \sum_{k=1}^{K} \theta_k(y_{:t}) \cdot\phi_k(h \bmod J)

such that, for every integer m≥0m \ge0,

∫mJ(m+1)J(f(h∣y:t)−gper(h∣y:t,w⋆))2 dh<ϵ,∀y:t∈Y.(57)\int_{mJ}^{(m+1)J} \left(f(h \mid y_{:t}) - g_{\mathrm{per}}(h \mid y_{:t}, \mathbf{w}^{\star})\right)^2 \,\mathrm{d}h < \epsilon,\qquad\forall y_{:t} \in\mathcal{Y}. \tag*{(57)}

Proof. The proof has three steps.

  1. One period approximation. By Theorem 1, there exist neural basis functions {ϕk}k=1K\{\phi_k\}_{k=1}^{K} and coefficient mappings {θk}k=1K\{\theta_k\}_{k=1}^{K} such that g(h∣y:t,w⋆)=∑k=1Kθk(y:t)ϕk(h)g(h \mid y_{:t}, \mathbf{w}^{\star}) = \sum_{k=1}^{K} \theta_k(y_{:t})\phi_k(h) satisfies ∫0H(f−g)2 dh<ϵ\int_{0}^{H}(f-g)^2\,\mathrm{d}h < \epsilon for all y:t∈Yy_{:t} \in\mathcal{Y}. Because J<HJ < H, the period [0,J][0,J] is within the training interval and the integrand is nonnegative, so ∫0J(f−g)2 dh<ϵ\int_{0}^{J}(f-g)^2\,\mathrm{d}h < \epsilon.

  1. Periodic extension. For h∈[0,J)h \in[0,J), h mod J=hh \bmod J = h, so gperg_{\mathrm{per}} equals gg in the first period and ∫0J(f−gper)2 dh<ϵ\int_{0}^{J}(f-g_{\mathrm{per}})^2\,\mathrm{d}h < \epsilon.

  1. In every period. Both ff and gperg_{\mathrm{per}} are JJ-periodic in hh. The substitution h=mJ+uh = mJ + u with u∈[0,J)u \in[0,J) gives ∫mJ(m+1)J(f−gper)2 dh=∫0J(f−gper)2 du<ϵ\int_{mJ}^{(m+1)J}(f-g_{\mathrm{per}})^2\,\mathrm{d}h = \int_{0}^{J}(f-g_{\mathrm{per}})^2\,\mathrm{d}u < \epsilon for every m≥0m \ge0.

□\square

References

  1. [1]Acosta, J., et al., 2026. Effective sample size for functional spatial data. arXiv preprint arXiv:2601.20812 .
  2. [2]Ansari, A.F., Stella, L., Turkmen, C., Zhang, X., Mercado, P., Shen, H., Schuur, O., Rangapuram, S.S., Arango, S.P., Kapoor, S., Zschiegner, J., Maddix, D.C., Mahoney, M.W., Torkkola, K., Wilson, A.G., Bohlke-Schneider, M., Wang, Y., 2024. Chronos: Learning the language of time series. arXiv:2403.07815.arxiv.org/abs/2403.07815
  3. [3]Athanasopoulos, G., Hyndman, R.J., Song, H., Wu, D.C., 2011. The Tourism forecasting competition. International Journal of Forecasting 27, 822–844. URL: https://www.sciencedirect.com/science/article/pii/S0169207010000107X, doi:https://doi.org/10.1016/j.ijforecast.2010.04.009. special Section 1: Forecasting with Artificial Neural Networks and Computational Intelligence. Special Section 2: Tourism Forecasting.
  4. [4]Bao, Y., Xiong, T., Hu, Z., 2014. Multi-step-ahead time series prediction using multiple-output support vector regression. Neurocomputing 129, 482–493. URL: https://www.sciencedirect.com/science/ article/pii/S092523121300917X, doi:https://doi.org/10.1016/j.neucom.2013.09.010.
  5. [5]Ben Taieb, S., Atiya, A.F., 2015. A bias and variance analysis for multistep-ahead time series forecasting. IEEE transactions on neural networks and learning systems 27, 62–76. URL: https://ieeexplore.ieee.org/document/7064712.DOI
  6. [6]Ben Taieb, S., Bontempi, G., Atiya, A.F., Sorjamaa, A., 2012. A review and comparison of strategies for multi-step ahead time series forecasting based on the NN5 forecasting competition. Expert Systems with Applications 39, 94–103. URL: https://arxiv.org/abs/1108.3259.
  7. [7]Ben Taieb, S., Sorjamaa, A., Bontempi, G., 2010. Multiple-output modeling for multi-step-ahead time series forecasting. Neurocomputing 73, 1950–1957. URL: https://www.sciencedirect.com/science/article/pii/S0925231210001013, doi:https://doi.org/10.1016/j.neucom.2009.11.030. subspace Learning / Selected papers from the European Symposium on Time Series Prediction.
  8. [8]de Boor, C., 1978. A Practical Guide to Splines. volume 27 of Applied Mathematical Sciences. Springer-Verlag, New York.
  9. [9]Box, G.E., Jenkins, G.M., Reinsel, G.C., Ljung, G.M., 2015. Time series analysis: forecasting and control. John Wiley & Sons.
  10. [10]Challu, C., Olivares, K.G., Oreshkin, B.N., Ramirez, F.G., Canseco, M.M., Dubrawski, A., 2023. N-HiTS: Neural hierarchical interpolation for time series forecasting, in: AAAI, pp. –. URL: https://arxiv.org/abs/2201.12886.
  11. [11]Chang, S., Zhang, Y., Han, W., Yu, M., Guo, X., Tan, W., et al., 2017. Dilated recurrent neural networks, in: 31st Conference on Neural Information Processing Systems, pp. –.
  12. [12]Chapados, N., Bengio, Y., 2007. Augmented functional time series representation and forecasting with gaussian processes, in: Platt, J., Koller, D., Singer, Y., Roweis, S. (Eds.), Advances in Neural Information Processing Systems, Curran Associates, Inc.. pp. –. URL: https://proceedings.neurips.cc/paper_files/paper/2007/ file/81e74d678581a3bb7a720b019f4f1a93-Paper.pdf.
  13. [13]Chen, R.T.Q., Rubanova, Y., Bettencourt, J., Duvenaud, D., 2018. Neural ordinary differential equations, in: Advances in Neural Information Processing Systems, pp. –.arxiv.org/abs/1806.07366
  14. [14]Chen, T., Chen, H., 1995. Universal approximation theorem for operators to neural networks. IEEE Transactions on Neural Networks 6, 911–917. doi:10.1109/72.392253.
  15. [15]Chevillon, G., 2007. Direct multi-step estimation and forecasting. Journal of Economic Surveys 21, 746–785. doi:https://doi.org/10.1111/j.1467-6419.2007.00518.x.
  16. [16]Cho, K., van Merrienboer, B., Gülçehre, Ç., Bougares, F., Schwenk, H., Bengio, Y., 2014. Learning phrase representations using RNN encoder-decoder for statistical machine translation. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) abs/1406.1078, 1724–1734. URL: http://arxiv.org/abs/1406.1078, arXiv:1406.1078.
  17. [17]Darlow, L., Deng, Q., Hassan, A., Asenov, M., Singh, R., Joosen, A., Barker, A., Storkey, A., 2024. Dam: Towards a foundation model for forecasting, in: International Conference on Learning Representations (ICLR), pp. –.arxiv.org/abs/2407.17880
  18. [18]Das, A., Kong, W., Sen, R., Zhou, Y., 2024. A decoder-only foundation model for time-series forecasting. URL: https://arxiv.org/abs/2310.10688, arXiv:2310.10688.
  19. [20]Garza, A., Mergenthaler-Canseco, M., 2023. Timegpt. URL: https://arxiv.org/abs/2310.03589, arXiv:2310.03589.
  20. [21]Griffith, D.A., 2005. Effective geographic sample size in the presence of spatial autocorrelation. Annals of the Association of American Geographers 95, 740–760.DOI
  21. [22]Gu, A., Goel, K., Ré, C., 2022. Efficiently modeling long sequences with structured state spaces, in: International Conference on Learning Representations, pp. –. URL: https://arxiv.org/abs/2111.00396.
  22. [23]Hersbach, H., 2000. Decomposition of the continuous ranked probability score for ensemble prediction systems. Weather and Forecasting 15, 559–570.doi.org/10.1175/1520-0434(2000)015<0559:dotcrp>2.0.co;2
  23. [24]Hochreiter, S., Schmidhuber, J., 1997. Long short-term memory. Neural Computation 9, 1735–1780.
  24. [25]Holt, C.C., 1957. Forecasting seasonals and trends by exponentially weighted moving averages. (O.N.R. Memorandum No. 52) URL: https://doi.org/10.1016/j.ijforecast.2003.09.015.
  25. [26]Hoo, S.B., Müller, S., Salinas, D., Hutter, F., 2024. The tabular foundation model tabPFN outperforms specialized time series forecasting models based on simple features, in: NeurIPS 2024 Third Table Representation Learning Workshop, pp. –. URL: https://openreview.net/forum?id=HO2X7R03OC.
  26. [27]Hornik, K., 1991. Approximation capabilities of multilayer feed-forward networks. Neural Networks 4, 251–257. URL: https://www.sciencedirect.com/science/article/pii/089360809190009T, doi:https://doi.org/10.1016/0893-6080(91)90009-T.
  27. [28]Huber, P.J., 1967. The behavior of maximum likelihood estimates under nonstandard conditions, in: Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, University of California Press. pp. 221–233.
  28. [29]Hyndman, R.J., Athanasopoulos, G., Garza, A., Challu, C., Mergenthaler, M., Olivares, K.G., 2026. Forecasting: Principles and Practice, the Pythonic Way. OTexts, Melbourne, Australia. Available at https://otexts.com/fpppy/.
  29. [30]Hyndman, R.J., Billah, B., 2003. Unmasking the theta method. International Journal of Forecasting 19, 287–290. URL: https://www.sciencedirect.com/science/article/pii/S0169207001001431, doi:https://doi.org/10.1016/S0169-2070(01)00143-1.
  30. [31]Hyndman, R.J., Khandakar, Y., 2008. Automatic time series forecasting: The forecast package for r. Journal of Statistical Software, Articles 27, 1–22. URL: https://www.jstatsoft.org/v027/i03, doi:10.18637/jss.v027.i03.DOI
  31. [32]Hyndman, R.J., Koehler, A.B., 2006. Another look at measures of forecast accuracy. International Journal of Forecasting 22, 679 – 688. URL: http://www.sciencedirect.com/science/article/pii/S0169207006000239, doi:https://doi.org/10.1016/j.ijforecast.2006.03.001.
  32. [33]Hyndman, R.J., Shahid Ullah, M., 2007. Robust forecasting of mortality and fertility rates: A functional data approach. Computational Statistics & Data Analysis 51, 4942–4956. URL: https://www.sciencedirect.com/science/article/pii/S0167947306002453, doi:https://doi.org/10.1016/j.csda.2006.07.028.
  33. [34]Hyndman, R.J., Shang, H.L., 2009. Forecasting functional time series. Journal of the Korean Statistical Society 38, 199–211. URL: https://www.sciencedirect.com/science/article/pii/S1226319209000398, doi:https://doi.org/10.1016/j.jkss.2009.06.002.
  34. [35]Januschowski, T., Wang, Y., Torkkola, K., Erkkilä, T., Hasson, H., Gasthaus, J., 2022. Forecasting with trees. International Journal of Forecasting 38, 1473–1481.DOI
  35. [36]Kidger, P., Morrill, J., Foster, J., Lyons, T., 2020. Neural controlled differential equations for irregular time series, in: Advances in Neural Information Processing Systems, pp. 6696–6707. URL: https://arxiv.org/abs/2005.08926.
  36. [38]Li, Z., Kovachki, N., Azizzadenesheli, K., Liu, B., Bhattacharya, K., Stuart, A., Anandkumar, A., 2021. Fourier neural operator for parametric partial differential equations. URL: https://arxiv.org/abs/2010.08895, arXiv:2010.08895.
  37. [39]Lu, L., Jin, P., Pang, G., Zhang, Z., Karniadakis, G.E., 2021. Learning nonlinear operators via DeepONet based on the universal approximation theorem of operators. Nature Machine Intelligence 3, 218–229. doi:10.1038/s42256-021-00302-5.DOI
  38. [40]Makridakis, S., Andersen, A., Carbone, R., Fildes, R., Hibon, M., Lewandowski, R., Newton, J., Parzen, E., Winkler, R., 1982. The accuracy of extrapolation (time series) methods: Results of a forecasting competition. Journal of Forecasting 1, 111–153. doi:https://doi.org/10.1002/for.3980010202.
  39. [41]Makridakis, S., Hibon, M., 2000. The M3-competition: results, conclusions and implications. International Journal of Forecasting 16, 451–476. URL: https://www.sciencedirect.com/science/article/pii/S0169207000000571, doi:https://doi.org/10.1016/S0169-2070(00)00057-1. the M3- Competition.
  40. [42]Makridakis, S., Spiliotis, E., Assimakopoulos, V., 2020. The M4 competition: 100,000 time series and 61 forecasting methods. International Journal of Forecasting 36, 54–74. URL: https://www.sciencedirect.com/science/article/pii/S0169207019301128, doi:https://doi.org/10.1016/j.ijforecast.2019.04.014. m4 Competition.
  41. [44]Nie, Y., Nguyen, N.H., Sinthong, P., Kalagnanam, J., 2023. A time series is worth 64 words: Long-term forecasting with transformers, in: International Conference on Learning Representations, pp. –.arxiv.org/abs/2211.14730
  42. [45]Olivares, K.G., Challu, C., Marcjasz, G., Weron, R., Dubrawski, A., 2023a. Neural basis expansion analysis with exogenous variables: Forecasting electricity prices with NBEATSx. International Journal of Forecasting 39, 901–921.
  43. [46]Olivares, K.G., Meetei, N., Ma, R., Reddy, R., Cao, M., Dicker, L., 2023b. Probabilistic hierarchical forecasting with deep poisson mixtures. International Journal of Forecasting Preprint version available at arXiv:2110.13179. URL: https://arxiv.org/abs/2110.13179.
  44. [47]Olivares, K.G., Wolff, M., Konstantinova, T., Ramasubramanian, S., Oreshkin, B., Wilson, A.G., Potapczynski, A., Potosnak, W., Cao, M., Mahoney, M.W., Efimov, D., 2025. A realistic evaluation of cross-frequency transfer learning and foundation forecasting models, in: Thirty-Ninth Annual Conference on Neural Information Processing Systems NeurIPS 2025, NeurIPS 2025, San Diego, USA. pp. –. URL: https://arxiv.org/abs/2411.05852, arXiv:2411.05852.
  45. [48]van den Oord, A., Dieleman, S., Zen, H., Simonyan, K., Vinyals, O., Graves, A., Kalchbrenner, N., Senior, A.W., Kavukcuoglu, K., 2016. WaveNet: A generative model for raw audio. Computer Research Repository abs/1609.03499. URL: http://arxiv.org/abs/1609.03499, arXiv:1609.03499.
  46. [49]Oreshkin, B.N., Carpov, D., Chapados, N., Bengio, Y., 2020. N-beats: Neural basis expansion analysis for interpretable time series forecasting, in: ICLR, pp. –.arxiv.org/abs/1905.10437
  47. [50]Potosnak, W., Wolff, M., Cao, M., Ma, R., Konstantinova, T., Efimov, D., Mahoney, M.W., Oreshkin, B.N., Olivares, K.G., 2026. Forking-sequences: Statistically and computationally efficient multi-horizon forecasting with reduced volatility. Transactions on Machine Learning Research URL: https://openreview.net/forum?id=dXdycy7WCX.arxiv.org/abs/2510.04487
  48. [51]Ramsay, J.O., Silverman, B.W., 2005. Functional Data Analysis. Springer Series in Statistics. 2nd ed., Springer, New York. doi:10.1007/b98888.DOI
  49. [52]Rubanova, Y., Chen, R.T.Q., Duvenaud, D., 2019. Latent ordinary differential equations for irregularly-sampled time series, in: Advances in Neural Information Processing Systems, pp. –.
  50. [53]Sagawa, S., Koh, P.W., Hashimoto, T.B., Liang, P., 2020. Distributionally robust neural networks for group shifts: On the importance of regularization for worst-case generalization, in: International Conference on Learning Representations (ICLR), pp. –.arxiv.org/abs/1911.08731
  51. [54]Salinas, D., Flunkert, V., Gasthaus, J., Januschowski, T., 2020. DeepAR: Probabilistic forecasting with autoregressive recurrent networks. International Journal of Forecasting 36, 1181–1191.arxiv.org/abs/1704.04110
  52. [55]Semenoglou, A.A., Spiliotis, E., Makridakis, S., Assimakopoulos, V., 2021. Investigating the accuracy of cross-learning time series forecasting methods. International Journal of Forecasting 37, 1072–1084. URL: https://www.sciencedirect.com/science/article/pii/S0169207020301850, doi:https://doi.org/10.1016/j.ijforecast.2020.11.009.
  53. [56]Shannon, C.E., 1949. Communication in the presence of noise. Proceedings of the IRE 37, 10–21. doi:10.1109/JRPROC.1949.232969.DOI
  54. [57]Smyl, S., 2019. A hybrid method of exponential smoothing and recurrent neural networks for time series forecasting. International Journal of Forecasting , -doi:10.1016/j.ijforecast.2019.03.017.DOI
  55. [58]Syntetos, M., Boylan, J., Croston, J., 2005. On the categorization of demand patterns. Journal of the Operational Research Society 56, -. doi:10.1057/palgrave.jors.2601841.DOI
  56. [59]Teräsvirta, T., Tjøstheim, D., Granger, C.W., 2010. Modelling Nonlinear Economic Time Series. Oxford University Press.
  57. [60]van der Vaart, A.W., 1998. Asymptotic Statistics. Cambridge Series in Statistical and Probabilistic Mathematics, Cambridge University Press.
  58. [61]Vallejos, R., Osorio, F., 2014. Effective sample size of spatial process models. Spatial Statistics 9, 66–92.DOI
  59. [62]Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I., 2017. Attention is all you need, in: Advances in Neural Information Processing Systems (NIPS), pp. 5998–6008.arxiv.org/abs/1706.03762
  60. [63]Wen, R., Torkkola, K., Narayanaswamy, B., Madeka, D., 2017. A multi-horizon quantile recurrent forecaster, in: In Proceedings of 31st Conference on Neural Information Processing Systems Time Series Workshop., pp. –. URL: https://www.amazon.science/publications/a-multi-horizon-quantile-recurrent-forecaster.arxiv.org/abs/1711.11053
  61. [64]Woo, G., Liu, C., Kumar, A., Xiong, C., Savarese, S., Sahoo, D., 2024. Unified training of universal time series forecasting transformers. URL: https://arxiv.org/abs/2402.02592, arXiv:2402.02592.
  62. [65]Xie, S.M., Pham, H., Dong, X., Du, N., Liu, H., Lu, Y., Liang, P., Le, Q.V., Ma, T., Yu, A.W., 2023. DoReMi: Optimizing data mixtures speeds up language model pretraining, in: Advances in Neural Information Processing Systems (NeurIPS), pp. –.arxiv.org/abs/2305.10429

Paper details

vlnc:W_m2856ebz
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 10

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Mathematics

Not recorded for this paper

View statement
Looking for related papers…
Contents