betaFeedback

WorksComputer Science

Not All Variables Agree: Reliability-Aware Variable-Wise Gradient Surgery for Multivariate Time-Series Forecasting

Abstract

In data-driven training, multivariate time-series forecasting is usually optimized with a scalar loss averaged over samples, variables, and horizons. This averaging is convenient, but the optimizer sees only the aggregated gradient, which does not reveal whether the variable-wise contributions align or oppose one another. To quantify how often this disagreement arises, we measure the variable-wise gradients directly and find that 30.6% of their pairwise cosine similarities are negative on average across seven datasets. However, conflict and harm are not the same thing. Under shared training 35 of the 64 variables do worse than a full-input single-target oracle, and the harmed fraction is not reliably predicted by how often gradients conflict. We propose Per-Variable Surgery (PV-Surgery), an optimizer-side training strategy for backbones with cache-compatible layers. One backward pass builds variable-wise gradient proxies from output-side signals and keeps the pointwise forecasting loss. Reliability-aware selection targets layers whose proxy sums closely approximate their shared-gradient slices. Conditional pooling forms anchor and conflict pools without dropping variables. Common-direction surgery aligns variable or pooled gradients with their normalized mean and restores input norms to avoid reweighting. In experiments across five backbones, seven datasets, and four horizons, PV-Surgery lowers MSE by 3.61% and MAE by 2.93% on average. For multivariate forecasting, this indicates that the variable-wise structure hidden by mean-loss training is a usable optimization signal.

Introduction

Multivariate time-series forecasting predicts the future values of multiple variables from a shared history of all variables ([6]; [?]). Forecasting architectures differ widely, but most are trained with a similar mean-loss objective. Prediction errors are averaged over samples, variables, and horizons to form a scalar loss, which is optimized through backpropagation ([13]; [11]; [19]). This objective is convenient and model-agnostic, but it also collapses heterogeneous variable-wise learning signals into one shared update.

The three averages behind this scalarization do not play the same role. Averaging over samples estimates an expectation over the data distribution, whereas averaging over variables and horizons assumes that heterogeneous targets and future steps can be merged into one optimization signal without losing important structure. In practice, however, variables differ in scale, noise, predictability, dynamics, and cross-variable dependency (Lai et al., 2018; Liu et al., 2024; Nochumsohn et al., 2025). Consequently, an update that helps one variable can be neutral or harmful for another, but the mean-loss gradient does not reveal which case holds (Nochumsohn et al., 2025; Liu et al., 2021).

Conceptual illustration of variable-wise gradient conflict

Figure 1. Conceptual illustration of variable-wise gradient conflict. (a) Different variables can require corrections in different directions over the same forecast horizon. (b) These corrections translate into variable-wise gradients that disagree in parameter space, while the mean-loss update shown here coincides with none of them.

More concretely, let Ld(θ)L_d(\theta) denote the loss induced by variable dd, averaged over samples and horizons. Standard training minimizes Lmean(θ)=D−1∑d=1DLd(θ)L_{\mathrm{mean}}(\theta) = D^{-1}\sum_{d=1}^{D} L_d(\theta) and updates parameters with gmean=∇θLmean(θ)=D−1∑d=1Dgdg_{\mathrm{mean}} = \nabla_\theta L_{\mathrm{mean}}(\theta) = D^{-1}\sum_{d=1}^{D} g_d, where gd=∇θLd(θ)g_d = \nabla_\theta L_d(\theta). This aggregation is many-to-one, so different sets of variable-wise gradients can produce the same update. For D=2D = 2, g1=u+vg_1 = u + v and g2=u−vg_2 = u - v produce gmean=ug_{\mathrm{mean}} = u, hiding the disagreement component vv. When ∥v∥2>∥u∥2\lVert v\rVert_2 > \lVert u\rVert_2, the inner product ⟨g1,g2⟩=∥u∥22−∥v∥22\langle g_1, g_2\rangle= \lVert u\rVert_2^2 - \lVert v\rVert_2^2 turns negative, so an update along −g1-g_1 reduces L1L_1 but increases L2L_2 to first order. Moreover, under any shared update θ←θ−ηgshared\theta\leftarrow\theta- \eta g_{\mathrm{shared}} with step size η>0\eta> 0, the first-order change of LdL_d is −η⟨gd,gshared⟩-\eta\langle g_d, g_{\mathrm{shared}}\rangle, whose sign cannot be checked from the aggregate alone. The problem is therefore not that mean loss is invalid, but that its single gradient hides the variable-wise directions needed to distinguish disagreement from actual harm.

Figure 1(a) shows three variables requiring corrections in different directions over the same horizon, and Figure 1(b) shows their parameter-space gradients, two of which oppose each other while their mean coincides with none of them. We call this variable-wise gradient conflict, following the view that a negative inner product encodes incompatible descent directions (Yu et al., 2020; Nochumsohn et al., 2025). A natural reaction is to remove every conflict, as in gradient surgery (Yu et al., 2020). In multivariate forecasting, our diagnosis suggests otherwise. Conflicts are common but not uniformly harmful, so intervention should be selective, variable-aware, and anchored to the mean-loss update.

In this paper, we propose PV-Surgery, a reliability-aware optimizer-side framework. At each training step, one backward pass builds variable-wise gradient proxies from output-side signals. PV-Surgery then selects surgery layers based on the normalized error between each proxy sum and its matching reference-gradient slice, conditionally forms anchor and conflict pools without dropping variables, and aligns variable or pooled gradients with the normalized mean of their input directions before restoring their norms to avoid reweighting. PV-Surgery requires no changes to cache-compatible forecasting backbones or per-variable losses. We make the following three contributions:

  • We analyze variable-wise conflict under standard mean-loss training using exact gradients. Although 30.6% of pairwise cosines are negative on average, conflict frequency is weakly related to harm. Shared training underperforms a full-input single-target oracle for 35 of 64 variables, while mean alignment does not reliably identify helpful partners.

  • We propose PV-Surgery, which reconstructs variable-wise gradient proxies in one backward pass, targets reconstruction-consistent layers, pools without dropping variables, and corrects directions while restoring surgery-input norms over the selected subspace.

  • We conduct experiments across five backbones, seven datasets, and four horizons, showing that PV-Surgery lowers MSE by 3.61% and MAE by 2.93% on average and that variable-wise structure hidden by mean-loss training is a usable optimization signal in these settings.

Multivariate Time-Series Forecasting

Multivariate forecasting has largely advanced through backbone design, spanning efficient attention (Zhou et al., 2021), patch tokens (Nie et al., 2023), variate tokens (Liu et al., 2024), exogenous cross-attention (Wang et al., 2024), series decomposition (Wu et al., 2021), multi-scale convolution (Wang et al., 2023), sample convolution (Liu et al., 2022), and linear baselines (Zeng et al., 2023). These approaches differ in architecture but retain the same mean-loss objective, whereas we intervene in the update it produces. MTLinear is the closest prior work, connecting multivariate forecasting to multi-task learning through variate gradient angle, correlation-based grouping, and gradient scaling for linear models (Nochumsohn et al., 2025). MTLinear fixes variable groups in advance and rescales gradient magnitudes, whereas we retain all variables, pool their gradients by directional compatibility at each step, and correct directions while restoring input norms across backbones.

Forecasting Objectives and Variable-Wise Learning

Forecasting objectives have expanded beyond point-wise MSE and MAE to include shape and time distortion losses (Guen & Thome, 2019), frequency-domain losses (Wang et al., 2025), transformation-invariant criteria (Lee et al., 2022), patch-wise structural losses (Kudrat et al., 2025), and selective timestep masking (Fu et al., 2025). These objectives change which errors are emphasized during training, but they still aggregate per-variable errors into one scalar. Reweighting the loss therefore does not reveal whether the corresponding gradients agree, cancel, or interfere.

Gradient Conflict and Multi-Task Optimization

Gradient conflict is studied in multi-task learning, where methods balance losses (Chen et al., 2018), seek Pareto-stationary descent directions (Sener & Koltun, 2018), project conflicting task gradients (Yu et al., 2020), maximize the worst-case improvement (Liu et al., 2021), or aggregate task gradients by bargaining (Navon et al., 2022). Multivariate forecasting exposes its objectives differently because variables are components scalarized inside one loss rather than separate tasks, a setting that MTLinear connects to multi-task learning (Nochumsohn et al., 2025). We therefore construct variable-wise gradient proxies for these operators rather than applying them to the aggregate.

Diagnosing Variable-Wise Gradient Conflict

We analyze the frequency of variable-wise gradient conflict under standard multivariate forecasting training and its relation to the fraction of variables harmed under shared training. We also ask whether mean pairwise alignment reliably identifies helpful training partners. Together, these analyses assess whether pairwise cosine statistics provide a sound basis for deciding when to intervene.

Diagnostic setup. We train iTransformer with the standard mean-loss objective on all seven benchmarks, at prediction length 96 for the four ETT datasets, Weather, and Exchange, and at 24 for ILI. At analysis checkpoints we decompose the scalar objective into variable-induced losses LdL_d, set gd(t)=∇θLd(θt)g_d^{(t)} = \nabla_{\theta} L_d(\theta_t), and form the pairwise cosine matrix over nonzero gradients,

Cij(t)=⟨gi(t),gj(t)⟩∥gi(t)∥2∥gj(t)∥2.(1)C_{ij}^{(t)} = \frac{\langle g_i^{(t)}, g_j^{(t)} \rangle}{\lVert g_i^{(t)} \rVert_2 \lVert g_j^{(t)} \rVert_2}. \tag*{(1)}

A pair is conflicting at checkpoint tt when Cij(t)<0C_{ij}^{(t)} < 0. Checkpoint summaries discard the first half of recorded checkpoints as warmup because early updates are unstable, but this filter does not apply to separately trained oracle or partner-group results. The diagnostics expose variable-wise structure hidden by the baseline objective and are not used to tune PV-Surgery.

Figure 3: (a) Fraction of variables for which shared training is worse than the target-specific oracle. (b) Across 162 target-and-size cells, the relative MSE difference between the two alignment groups versus the pairwise-alignment gap. Negative values favor higher alignment. The shaded ±1.1%\pm1.1\% band summarizes repeated-fit variation for fixed target-partner groups (Appendix C).

Variable-wise conflicts are frequent

Across the seven datasets, 30.6% of the pairwise cosine similarities are negative on average, with per-dataset results in Appendix B and the ETTh1 case shown in Figure 2. On the six datasets other than ILI, most post-warmup checkpoints contain at least one conflicting pair. The disagreement is also intermittent, since no pair is negative at every post-warmup checkpoint. Therefore, the standard scalar objective hides a substantial amount of variable-wise disagreement. However, this does not yet justify gradient surgery, since a negative cosine says only that two descent directions disagree at one checkpoint, not which variable is harmed or whether sharing is worse than separate training.

Pairwise gradient-cosine distribution and gradient cosine heatmaps

Figure 2. A representative ETTh1 run from the variable-wise gradient-cosine diagnostic. (a) Distribution of the post-warmup pairwise cosines, of which 30.6% fall below zero for this run. (b) An early checkpoint, which lies inside the discarded warmup half and is shown only to expose the change over training. (c) A late checkpoint. Both heatmaps show that conflict is structured across variable pairs rather than being uniform, so one summary number would hide which pairs disagree.

Conflict and harm are not the same object

To separate disagreement from harm, we compare the shared model with full-input single-target oracles, each optimized and evaluated on one target. We define the oracle gap as Δdoracle=MSEshared,d−MSEoracle,d\Delta^{\mathrm{oracle}}_{d} = \mathrm{MSE}_{\mathrm{shared},d} - \mathrm{MSE}_{\mathrm{oracle},d}. A positive Δdoracle\Delta^{\mathrm{oracle}}_{d} means variable dd is better predicted by its target-specific oracle. Across the seven datasets 35 of the 64 variables have a positive oracle gap, and fig:3a reports the fraction for every dataset. At the dataset level, the negative-cosine ratio and the harmed fraction correlate at −0.29-0.29 (p=0.53p = 0.53), so a larger conflict rate does not reliably coincide with a larger harmed fraction. ILI has the lowest conflict rate, whereas Weather has the highest harmed fraction (Table 4). A method that removes every negative cosine similarity would therefore risk destroying useful sharing for variables that improve under the shared model.

Alignment alone does not reliably identify helpful partners

For each reported target we rank the remaining variables by their post-warmup mean pairwise gradient cosine in the baseline model. For k∈{2,3,4}k \in\{2, 3, 4\}, we train with its k−1k-1 highest-ranked or lowest-ranked partners, which form disjoint sets. Both conditions use matched replicate seeds, keep the full multivariate input, restrict the loss to the target and its partners, and are scored by held-out target MSE. Appendix C gives the full protocol. Higher alignment wins 100/162 cells, and the all-cell median signed contrast is −1.4%-1.4\%, but the benchmark-averaged contrast is not distinguishable from zero. Figure 3 shows that a larger alignment gap does not reliably identify the better partner set, consistent with Appendix Figure 13, where ILI favors higher alignment at every group size but five of the other six datasets change sign across sizes. Group composition still matters, since the median absolute within-cell contrast is 3.5%, over three times the 1.1% variation observed when the same target and partner set are retrained with different random seeds.

Fraction of variables harmed by shared training and outcome gap versus alignment gap

Figure 3. (a) Harmed variables by dataset. (b) Outcome gap versus alignment gap.

Figure 13. Percentage difference between the higher- and lower-alignment partner groups’ test MSE, taken relative to the lower-alignment group and averaged over the targets of each benchmark. Negative means the higher-alignment group did better. Vertical ranges differ across panels. The grey band is that benchmark’s seed noise and the error bars are the standard error across its targets.

Together, the evidence shows that negative pairwise cosine similarities are common, but their frequency does not reveal which variables shared training harms. Partner composition can materially change a target’s error, but mean gradient alignment does not reliably predict whether the change helps or hurts. The comparisons are associational rather than causal, and they motivate conditional, reliability-aware intervention rather than removing conflicts uniformly across variable subsets.

Proposed Method

PV-Surgery is an optimizer-side gradient transformation that leaves the forecasting architecture, prediction target, and per-variable loss unchanged. Figure 4 gives an overview of PV-Surgery, which consists of four stages that form a dependency chain. Computing exact per-variable gradients would require DD costly backward passes, so PV-Surgery instead builds variable-wise gradient proxies from cached layer signals in one pass. Variable mixing can shift proxy sums from reference-gradient slices, so PV-Surgery selects layers by relative error and falls back to the output layer. Section 3.3 shows that mean alignment does not reliably identify helpful partners, which is why variables are pooled conditionally at each step without dropping any of them. Finally, surgery aligns each variable or pooled gradient with the normalized mean direction, then restores its pre-surgery norm so the correction does not reweight the surgery inputs. Appendix D gives the training pseudocode.

Figure 4. Overview of PV-Surgery. Step 1: Output-cache variable-wise gradient proxy reconstructs variable-wise proxy gradients from a single sum-loss backward pass and cached layer signals. Step 2: Reliability-aware layer selection selects slices by normalized proxy-sum error with output fallback. Step 3: Conditional variable-gradient pooling forms anchor and conflict pool gradients without dropping variables. Step 4: Magnitude-preserving common-direction surgery aligns variable or pooled gradients while restoring their pre-surgery norms to avoid reweighting.

Problem setup

For a batch of BB windows, DD variables, and horizon length HH, let Y^,Y∈RB×H×D\hat{Y}, Y \in\mathbb{R}^{B \times H \times D} be the predicted and true future values and φ\varphi be the pointwise forecasting loss. The loss induced by variable dd is

Ld(θ)=1BH∑b=1B∑h=1Hφ(Y^b,h,d,Yb,h,d).(2)L_d(\theta) = \frac{1}{BH} \sum_{b=1}^{B} \sum_{h=1}^{H} \varphi\left(\hat{Y}_{b,h,d}, Y_{b,h,d}\right). \tag*{(2)}

Let LΣ=∑d=1DLdL_{\Sigma} = \sum_{d=1}^{D} L_d and Lmean=LΣ/DL_{\mathrm{mean}} = L_{\Sigma}/D. With gd=∇θLdg_d = \nabla_{\theta} L_d and gmean=∇θLmeang_{\mathrm{mean}} = \nabla_{\theta} L_{\mathrm{mean}}, PV-Surgery’s exact single-backward reference gradient is

g0=∇θLΣ=∑d=1Dgd=Dgmean.(3)g_0 = \nabla_{\theta} L_{\Sigma} = \sum_{d=1}^{D} g_d = D g_{\mathrm{mean}}. \tag*{(3)}

PV-Surgery backpropagates LΣL_{\Sigma} once, avoiding DD backward passes and a dense D×∣θ∣D \times\lvert\theta\rvert gradient tensor. At the output boundary, variable dd’s cache row equals its gdg_d slice without 1/D1/D scaling because only that output slice enters LdL_d, and the rows sum to the matching g0g_0 slice without rescaling.

Output-cache variable-wise gradient proxy

One backward pass materializes only g0g_0, so we must estimate variable-wise gradients from same-pass layer inputs and output-side backward signals. We hook cache-compatible linear layers H\mathcal{H} to collect these quantities and convert them to a common variable-axis form by moving an explicit axis or reshaping batch-folded layouts. For a layer ℓ\ell with Wℓ∈RFout×FinW_{\ell} \in\mathbb{R}^{F_{\mathrm{out}} \times F_{\mathrm{in}}}, the caches keep a variable axis, Zℓ∈RB×D×Mℓ×FinZ_{\ell} \in\mathbb{R}^{B \times D \times M_{\ell} \times F_{\mathrm{in}}} and Eℓ∈RB×D×Mℓ×FoutE_{\ell} \in\mathbb{R}^{B \times D \times M_{\ell} \times F_{\mathrm{out}}}, and flattening the batch and position axes gives Zℓ,d∈RBMℓ×FinZ_{\ell,d} \in\mathbb{R}^{BM_{\ell} \times F_{\mathrm{in}}} and Eℓ,d∈RBMℓ×FoutE_{\ell,d} \in\mathbb{R}^{BM_{\ell} \times F_{\mathrm{out}}}. The weight and bias gradients are then

Gℓ,dW=Eℓ,dTZℓ,d,Gℓ,db=Eℓ,dT1BMℓ.(4)G^{W}_{\ell,d} = E_{\ell,d}^{\mathsf{T}} Z_{\ell,d}, \qquad G^{b}_{\ell,d} = E_{\ell,d}^{\mathsf{T}} \mathbf{1}_{BM_{\ell}}. \tag*{(4)}

When a bias is present, it is concatenated with the flattened weight gradient.

Gℓ=stack⁡d=1D(concat⁡(vec⁡(Gℓ,dW),Gℓ,db))∈RD×Nℓ.(5)G_{\ell} = \operatorname{stack}_{d=1}^{D}\left(\operatorname{concat}\left(\operatorname{vec}\left(G^{W}_{\ell,d}\right), G^{b}_{\ell,d}\right)\right) \in\mathbb{R}^{D \times N_{\ell}}. \tag*{(5)}

(4) is the per-variable chain-rule gradient at the output boundary and a proxy after variable mixing, motivating the reliability test. If a hooked layer lacks a variable-axis form for that batch, Gℓ=0D×NℓG_{\ell} = 0_{D \times N_{\ell}}. Here MℓM_{\ell} counts positions sharing WℓW_{\ell} within a variable and NℓN_{\ell} cached parameters.

Reliability-aware layer selection

Reconstruction consistency varies by layer and backbone and can shift during training, so the method selects its layer set dynamically at each step rather than fixing it in advance. For every hooked layer ℓ∈H\ell\in\mathcal{H}, we compare the summed proxy with the matching slice of the single-backward reference.

gproxy,ℓ=∑d=1DGℓ[d,:],eℓ=∥gproxy,ℓ−g0,ℓ∥2∥g0,ℓ∥2+ϵ.(6)g_{\mathrm{proxy},\ell} = \sum_{d=1}^{D} G_{\ell}[d,:], \qquad e_{\ell} = \frac{\left\|g_{\mathrm{proxy},\ell} - g_{0,\ell}\right\|_2}{\left\|g_{0,\ell}\right\|_2 + \epsilon}. \tag*{(6)}

Reconstruction reliability rℓ=(1+eℓ)−1r_{\ell} = (1+e_{\ell})^{-1} scores proxy-sum agreement, while normalized slice norm qℓ=∥g0,ℓ∥2/(∥g0∥2+ϵ)q_{\ell} = \left\|g_{0,\ell}\right\|_2/(\left\|g_0\right\|_2+\epsilon) measures reference-slice magnitude. Let Helig\mathcal{H}_{\mathrm{elig}} be the unprotected hooked layers. The median-reliability gate retains its non-output layers at or above median reliability and includes the output layer. For the resulting Hgate\mathcal{H}_{\mathrm{gate}}, we set A={ℓ∈Hgate∣qℓ>0}\mathcal{A} = \{\ell\in\mathcal{H}_{\mathrm{gate}} \mid q_{\ell} > 0\} and compute

πℓ=qℓ∑j∈Aqj,Elayer=−∑ℓ∈Aπℓlog⁡πℓ,Keff=⌈eElayer⌉.(7)\pi_{\ell} = \frac{q_{\ell}}{\sum_{j \in\mathcal{A}} q_j}, \qquad\mathcal{E}_{\mathrm{layer}} = -\sum_{\ell\in\mathcal{A}} \pi_{\ell} \log\pi_{\ell}, \qquad K_{\mathrm{eff}} = \left\lceil e^{\mathcal{E}_{\mathrm{layer}}} \right\rceil. \tag*{(7)}

It takes the KeffK_{\mathrm{eff}} largest-qℓq_{\ell} layers and the output layer that pass the reconstruction-error check, falling back to the output layer if none do (Appendix D). Here eElayere^{\mathcal{E}_{\mathrm{layer}}} is π\pi’s exponentiated Shannon entropy, or order-one Hill number [4], and KeffK_{\mathrm{eff}} is its ceiling. The selected set S\mathcal{S} then gives

GS=concat⁡ℓ∈SGℓ∈RD×NS,NS=∑ℓ∈SNℓ≤NH.(8)G_{\mathcal{S}} = \operatorname{concat}_{\ell\in\mathcal{S}} G_{\ell} \in\mathbb{R}^{D \times N_{\mathcal{S}}}, \qquad N_{\mathcal{S}} = \sum_{\ell\in\mathcal{S}} N_{\ell} \leq N_{\mathcal{H}}. \tag*{(8)}

Here NH=∑ℓ∈HNℓN_{\mathcal{H}} = \sum_{\ell\in\mathcal{H}} N_{\ell} counts all cached parameters. Fixing the support before pooling or direction change keeps surgery out of layers where the cache decomposition is not reconstruction-consistent, and leaves later stages operating on NSN_{\mathcal{S}} rather than NHN_{\mathcal{H}} parameters.

Conditional variable-gradient pooling

Treating every variable gradient as a surgery objective would make direction surgery reconcile all pairs, although Section 3 shows that a negative cosine alone does not reliably imply harmful sharing. We keep every variable but reduce the objective count. With Gpre=GSG_{\mathrm{pre}} = G_S, each row contains one variable’s proxy gradient over the selected parameter slices, and g0,Sg_{0,S} is the exact sum-loss gradient over those same slices:

gref=g0,S,ad=cos⁡(Gpre[d,⋅],gref).(9)g_{\mathrm{ref}} = g_{0,S}, \qquad a_d = \cos(G_{\mathrm{pre}}[d,\cdot], g_{\mathrm{ref}}). \tag*{(9)}

Pooling requires two finite, nonzero rows with a negative cosine, not just a row opposed to grefg_{\mathrm{ref}}. Rows with ad≥0a_d \ge0 are aligned to the reference and the rest conflict with it. An anchor combines at least two aligned rows whose sum has nonnegative cosine with each. Each conflicting row greedily joins the pool whose current sum is most aligned with it, provided their cosine is positive. The partition Mpool∈{0,1}P×DM_{\mathrm{pool}} \in\{0,1\}^{P \times D} maps DD rows to PP objectives while preserving their aggregate gradient:

Gpool=MpoolGpre∈RP×NS,∑k=1PGpool[k,⋅]=∑d=1DGpre[d,⋅].(10)G_{\mathrm{pool}} = M_{\mathrm{pool}}G_{\mathrm{pre}} \in\mathbb{R}^{P \times N_S}, \qquad\sum_{k=1}^{P} G_{\mathrm{pool}}[k,\cdot] = \sum_{d=1}^{D} G_{\mathrm{pre}}[d,\cdot]. \tag*{(10)}

Without pairwise conflict, the pooled branch skips direction surgery and remains Gppost=GpreG_p^{\mathrm{post}} = G_{\mathrm{pre}}. The subsequent safe candidate-selection step may retain it over the corrected unpooled candidate.

Magnitude-preserving common-direction surgery

Pooling reduces the objective count, but pooled gradients can still disagree, so their directions must be adjusted and the unpooled and pooled branches remain candidates. For unpooled uu and pooled pp, the inputs are Gin(u)=GpreG_{\mathrm{in}}^{(u)} = G_{\mathrm{pre}} and, when pooling is active, Gin(p)=GpoolG_{\mathrm{in}}^{(p)} = G_{\mathrm{pool}}. For c∈{u,p}c \in\{u,p\}, let PcP_c be its objective count and decompose each nonzero objective into its magnitude and unit direction:

mi(c)=∥Gin(c)[i,⋅]∥2,g^i(c)=Gin(c)[i,⋅]mi(c).(11)m_i^{(c)} = \left\lVert G_{\mathrm{in}}^{(c)}[i,\cdot]\right\rVert_2, \qquad\hat{g}_i^{(c)} = \frac{G_{\mathrm{in}}^{(c)}[i,\cdot]}{m_i^{(c)}}. \tag*{(11)}

Objectives with mi(c)=0m_i^{(c)} = 0 take g^i(c)=0\hat{g}_i^{(c)} = 0. Let gˉ(c)=Pc−1∑ig^i(c)\bar{g}^{(c)} = P_c^{-1}\sum_i \hat{g}_i^{(c)}. The operator aligns each objective with gˉ(c)\bar{g}^{(c)} while preserving its input magnitude, and makes no change when ∥gˉ(c)∥2=0\lVert\bar{g}^{(c)}\rVert_2 = 0.

G~in(c)[i,⋅]={mi(c)gˉ(c)/∥gˉ(c)∥2,∥gˉ(c)∥2>0,Gin(c)[i,⋅],∥gˉ(c)∥2=0,andGupost=G~in(u).(12)\widetilde{G}_{\mathrm{in}}^{(c)}[i,\cdot] = \begin{cases} m_i^{(c)} \bar{g}^{(c)} / \lVert\bar{g}^{(c)}\rVert_2, & \lVert\bar{g}^{(c)}\rVert_2 > 0, \\ G_{\mathrm{in}}^{(c)}[i,\cdot], & \lVert\bar{g}^{(c)}\rVert_2 = 0, \end{cases} \qquad\text{and} \qquad G_u^{\mathrm{post}} = \widetilde{G}_{\mathrm{in}}^{(u)}. \tag*{(12)}

Pool restoration and safe candidate selection. For active pooling, pool kk distributes the change made by surgery equally across its member set MkM_k.

Gppost[d,⋅]=Gpre[d,⋅]+G~in(p)[k,⋅]−Gpool[k,⋅]∣Mk∣,d∈Mk.(13)G_p^{\mathrm{post}}[d,\cdot] = G_{\mathrm{pre}}[d,\cdot] + \frac{\widetilde{G}_{\mathrm{in}}^{(p)}[k,\cdot] - G_{\mathrm{pool}}[k,\cdot]}{\lvert M_k\rvert}, \qquad d \in M_k. \tag*{(13)}

This restores DD variable rows while preserving each post-surgery pool sum, but not individual row norms. The descent-lexicographic selector compares GupostG_u^{\mathrm{post}} with GppostG_p^{\mathrm{post}}. For candidate cc, let gc=∑dGcpost[d,⋅]g_c = \sum_d G_c^{\mathrm{post}}[d,\cdot]. For each finite, nonzero variable-wise gradient, the selector computes the following scores:

γc,d=cos⁡(Gpre[d,⋅],gc),vc,d=1[γc,d<0].(14)\gamma_{c,d} = \cos(G_{\mathrm{pre}}[d,\cdot], g_c), \qquad v_{c,d} = 1[\gamma_{c,d} < 0]. \tag*{(14)}

It first minimizes ∑dvc,d\sum_d v_{c,d}, then compares the sorted cosine vectors from worst to best. Exact or numerically ambiguous ties retain the unpooled candidate. This ranks candidates and does not guarantee a descent-compatible update for every variable when both contain violations.

Final gradient assembly. For selected GpostG_{\mathrm{post}}, ΔS\Delta_S is the change in the aggregate selected-subspace gradient:

ΔS=∑d=1DGpost[d,⋅]−∑d=1DGpre[d,⋅].(15)\Delta_S = \sum_{d=1}^{D} G_{\mathrm{post}}[d,\cdot] - \sum_{d=1}^{D} G_{\mathrm{pre}}[d,\cdot]. \tag*{(15)}

For each hooked layer ℓ\ell, let Ωℓ\Omega_\ell be its flattened parameter indices, and let ΩS=⋃ℓ∈SΩℓ\Omega_S = \bigcup_{\ell\in S}\Omega_\ell. The full-sum assembly inserts every hooked-slice proxy sum and then adds the selected-subspace correction.

gfinal←g0,gfinal[Ωℓ]←∑d=1DGℓ[d,⋅] (ℓ∈H),gfinal[ΩS]←gfinal[ΩS]+ΔS.(16)g_{\mathrm{final}} \leftarrow g_0,\quad g_{\mathrm{final}}[\Omega_\ell] \leftarrow\sum_{d=1}^{D}G_\ell[d,\cdot]\ (\ell\in\mathcal{H}),\quad g_{\mathrm{final}}[\Omega_S] \leftarrow g_{\mathrm{final}}[\Omega_S] + \Delta_S. \tag*{(16)}

Thus, non-hooked parameters retain g0g_0. Unselected hooked slices use their variable-wise proxy sums, and selected slices add the corresponding ΔS\Delta_S entries.

Experiments

Experimental setup

Datasets. We evaluate PV-Surgery on ETTh1, ETTh2, ETTm1, ETTm2, Weather, and Exchange using an input length of 96 and prediction lengths {96,192,336,720}\{96, 192, 336, 720\}. For ILI, the input length is 36 and the prediction lengths are {24,36,48,60}\{24, 36, 48, 60\}. All methods share chronological splits. Appendices E and A give settings and dataset statistics.

Baselines. We compare standard MSE training and PV-Surgery on DLinear [23], iTransformer [11], MICN [18], SCINet [10], and TimeXer [19]. We additionally compare four forecasting-specific methods, TILDE-Q [7], FreDF [17], PSLoss [5], and Selective Learning [2]. All use the common protocol of Appendix E, with iTransformer for this comparison.

Metrics. We report test mean squared error (MSE) and mean absolute error (MAE), both averaged over test windows, horizons, and variables. Exact definitions are provided in Appendix E.

Main results

Tables 1 and 6 report all 140 standard-MSE comparisons, with Weather and ILI in Appendix F due to the page limit. Separately, Table 2 compares PV-Surgery with four forecasting-specific methods using iTransformer across five datasets. Appendix G covers all backbones and benchmarks.

PV-Surgery obtains lower MSE in 111 of the 140 settings and lower MAE in 119, both counted on unrounded scores, and averaging the relative change within each setting gives 3.61% in MSE and 2.93% in MAE. The improvement is largest on MICN, at 8.50% in MSE and 5.67% in MAE, and on DLinear, at 5.38% and 5.01%. PV-Surgery therefore improves the majority of settings for every backbone and dataset, although the direction and size of the average change depend on both.

Across the five datasets in Table 2, PV-Surgery ranks first or second in 19/20 MSE cells and 17/20 MAE cells. No method is best throughout. Some competing methods perform better in individual ETT settings, while PV-Surgery is particularly competitive at longer horizons and on Exchange. This pattern supports the diagnostic premise that intervention should preserve useful sharing rather than treat every disagreement as harmful. PV-Surgery acts on the update rather than the loss and can therefore complement these methods instead of replacing them.

DatasetETTh1ETTh2ETTm1ETTm2Exchange
Forecast length96 192 336 72096 192 336 72096 192 336 72096 192 336 72096 192 336 720
MSE MSE0.454 0.517 0.561 0.6710.239 0.303 0.356 0.4570.407 0.480 0.520 0.6180.157 0.205 0.252 0.3220.108 0.222 0.397 1.099
MSE MAE0.447 0.485 0.513 0.5830.323 0.366 0.401 0.4600.415 0.457 0.477 0.5310.258 0.295 0.328 0.3730.236 0.344 0.463 0.800
TILDE-Q MSE0.469 0.528 0.564 0.6690.233 0.287 0.354 0.4730.402 0.465 0.537 0.6220.156 0.200 0.247 0.3110.111 0.219 0.428 1.109
TILDE-Q MAE0.455 0.489 0.510 0.5770.315 0.353 0.400 0.4660.404 0.440 0.483 0.5340.251 0.285 0.320 0.3620.239 0.341 0.481 0.806
FreDF MSE0.451 0.512 0.560 0.6620.235 0.295 0.350 0.4500.410 0.475 0.553 0.6030.154 0.197 0.245 0.3090.112 0.223 0.462 1.132
FreDF MAE0.445 0.478 0.508 0.5760.317 0.360 0.397 0.4550.408 0.452 0.492 0.5240.250 0.284 0.319 0.3620.241 0.344 0.502 0.819
PSLoss MSE0.458 0.513 0.558 0.6650.234 0.291 0.360 0.4730.429 0.499 0.538 0.6260.153 0.196 0.239 0.3100.113 0.213 0.413 1.146
PSLoss MAE0.448 0.480 0.509 0.5770.316 0.355 0.406 0.4690.421 0.459 0.482 0.5350.247 0.281 0.314 0.3610.241 0.338 0.476 0.825
SL MSE0.465 0.527 0.560 0.6650.229 0.284 0.348 0.4380.404 0.458 0.521 0.5900.153 0.199 0.243 0.3120.108 0.211 0.393 1.228
SL MAE0.449 0.485 0.501 0.5680.312 0.349 0.393 0.4490.400 0.432 0.465 0.5050.250 0.285 0.317 0.3620.232 0.332 0.460 0.856
PV(Ours) MSE0.451 0.508 0.550 0.6610.229 0.287 0.334 0.4460.402 0.461 0.517 0.5750.150 0.195 0.240 0.3100.105 0.211 0.390 1.075
PV(Ours) MAE0.442 0.476 0.504 0.5750.314 0.355 0.388 0.4520.404 0.439 0.473 0.5100.247 0.283 0.317 0.3650.231 0.335 0.460 0.792

Table 2. Comparison of PV-Surgery with four forecasting-specific methods on iTransformer for five of the seven benchmarks. The MSE row denotes standard MSE training. Best results are bolded and second-best results are underlined using unrounded scores.

Ablation study

The full configuration gives the lowest MSE and MAE. Removing the output-cache proxy or conflict pooling causes the largest degradations. Because the output-cache ablation also disables cache-dependent layer selection, pooling, and candidate comparison, its large performance drop reflects several coupled components being removed together. Removing conflict pooling degrades both metrics much more than removing anchor pooling, consistent with its role in grouping variables into fewer, directionally coherent objectives when conflicts are frequent. Without layer selection, surgery runs on every hooked layer, lowering accuracy and raising mean wall-clock training time from 332.45 to 642.98 seconds. Appendix H reports dataset-level results.

Mechanism evidence

We directly test the mechanism by comparing exact per-variable gradients with cache-reconstructed proxies across the main-result settings (Figure 5). Proxies are nearly exact at the output boundary. Fidelity declines after variable mixing, but nonzero selected-slice proxies remain close. Appendix J details the protocol and tests whether surgery improves first-order update behavior through conflict mass, which measures pairwise disagreement among reconstructed proxy rows, and descent violations, which count variable objectives pushed uphill by the update. PV-Surgery lowers violations in most runs without increasing conflict mass. The appendix also analyzes oracle-gap reduction, where 23 of 29 harmed variables close part of their oracle gap (Figure 17). Appendix K gives qualitative results.

Figure 5. Cached-to-exact gradient cosine by backbone. Selected-slice cosine excludes zero proxies.

Conclusion

Multivariate forecasters optimize a loss averaged across variables, leaving the effect of the shared update on each variable unclear. We find that pairwise gradient conflict is common and most variables underperform a full-input single-target oracle. But conflict frequency is weakly related to harm, and mean alignment does not reliably identify helpful partners. PV-Surgery addresses this problem with a single-backward optimizer-side transformation. The method constructs variable-wise gradient proxies, restricts surgery to a reconstruction-consistent subspace, pools variables conditionally, and corrects directions while preserving surgery-input norms. Across matched settings, it reduces MSE by 3.61% and MAE by 2.93% on average, with every module contributing to both metrics. Mechanism analyses show that selected proxies track variable-wise gradients and forecasts for harmed variables often improve. Together, the results support selective, reliability-aware intervention over uniform conflict removal. PV-Surgery has two main limitations. The method raises median per-epoch cost to 1.54× the mean-loss baseline (Appendix L), and gains can weaken when proxy fidelity is low, selected layers mix variables, or variable relationships differ across datasets. Future work should reduce this overhead and improve proxy construction under stronger variable mixing and more diverse cross-variable structures without sacrificing useful sharing.

AI use statement

In this work, we used generative AI tools to edit prose, translate draft passages for internal review, provide feedback on the method and experiments, assist with parts of the method implementation, analysis code, and plotting code, and help interpret experimental results. We did not use these tools to originate the research question or central research idea. We reviewed all AI-assisted text and code and take responsibility for the final text, claims, and artifacts.

Ethics statement

The experiments use established public forecasting benchmarks and do not collect personal data or make individual-level decisions. Nevertheless, benchmark accuracy does not establish reliability under distribution shift or safety in operational use.

Reproducibility statement

Section 4 and Algorithm 1 specify the loss scaling, cache construction, layer-selection rule, pooling and restoration operations, candidate comparison, and final-gradient assembly. Appendix E reports the forecasting protocol and optimization settings. Tables 1 and 6 cover every one of the 5×7×4=1405 \times7 \times4 = 140 main comparisons, while Table 3 and Appendix H jointly cover all 140 matched ablation settings per variant. Except for the multi-seed analyses in Appendix C, all reported runs use the single seed 42, as specified in Appendix E.

Dataset Diagnostics

Table 4 summarizes the datasets and the diagnostic quantities used to contextualize PV-Surgery. In addition to common dataset metadata, we report conflict-oriented indicators tied to the optimization issue studied in this paper. The negative cosine ratio measures how often variable-wise gradients disagree, the mean cosine measures average alignment, and the harmed-variable fraction is the share of variables for which shared training performs worse than the full-input single-target oracle. Diagnostics are computed with iTransformer at prediction length 96, except for ILI where prediction length 24 is used.

DatasetFrequencyTTDDNeg. cos. (%)Mean cos.Harmed vars. (%)
ETTh11 hour17,420730.60.17814.3
ETTh21 hour17,420730.40.28157.1
ETTm115 min69,680730.80.17228.6
ETTm215 min69,680729.90.24657.1
Weather10 min52,6962137.50.13381.0
Exchange1 day7,588839.30.14825.0
ILI1 week966715.90.50871.4

Table 4. Dataset statistics and conflict-oriented diagnostics. TT is the number of timestamps and DD is the number of target variables. Neg. cos. is the percentage of negative pairwise variable-gradient cosines and Mean cos. is their average. Both follow the protocol of Section 3 and are computed after dropping the warmup half of the recorded checkpoints. Harmed vars. is the percentage of variables for which shared training is worse than the full-input single-target oracle on the held-out test split.

Gradient-Conflict Figures for All Datasets

Figure 2 presents ETTh1 in the main text. Figures 6–11 use the same three-panel layout, consisting of the post-warmup cosine distribution and heatmaps from an early and a late checkpoint, for the remaining six benchmarks. ETTh1, ETTh2, ETTm1, ETTm2, Weather, and Exchange use prediction length 96, and ILI uses 24.

Conflict is intermittent. No variable pair carries a negative cosine at every post-warmup checkpoint in any of the seven benchmarks. The most persistent pair is on ETTh2 and is negative at 72% of its checkpoints, while the median pair ranges from 11% to 43% across the seven benchmarks (Table 5). When each pair is averaged across its post-warmup checkpoints, the resulting cosines span [−0.12,0.98][-0.12, 0.98]. Thus, even the lowest-alignment pairs are only weakly opposed on average, and this average hides changes in sign over training. A static pair-level rule could therefore act after the relation has changed. To account for this behavior, the method in Section 4 instead recomputes the relevant gradient relations from the current batch and reevaluates layer support at every step.

DatasetCheckpointsPairsPersistence (%)
Most persistentMedian
ETTh1462150.028.3
ETTh2392171.820.5
ETTm1482150.035.4
ETTm2372148.640.5
Weather4421068.240.9
Exchange382855.343.4
ILI1502137.311.3

Table 5. Persistence of variable-pair conflict over training. The Checkpoints column gives the number of post-warmup analysis points, and the Pairs column gives the number of variable pairs in each benchmark. Persistence is the share of those checkpoints at which a pair has a negative cosine, reported for the most persistent pair and for the median pair. No entry reaches 100%, so no pair conflicts throughout.

Group-Composition Diagnostics for All Datasets

Group-composition protocol. For each reported target variable, we rank the remaining variables by their post-warmup mean pairwise gradient cosine in the baseline run. For each total group size k∈{2,3,4}k \in\{2, 3, 4\}, we train one model with the k−1k - 1 highest-ranked partners and another with the k−1k - 1 lowest-ranked partners. The two partner sets are disjoint in every reported cell. Each model still

Pairwise gradient-cosine distribution and early and late gradient cosine heatmaps for ETTh2

Figure 6. (a) Post-warmup cosine distribution. (b) Early checkpoint. (c) Late checkpoint. Figure 6: A representative ETTh2 run from the variable-wise gradient-cosine diagnostic at prediction length 96. (a) Distribution of the post-warmup pairwise cosines, of which 30.4% fall below zero for this run. (b) An early checkpoint, which lies inside the discarded warmup half and is shown only to expose the change over training. (c) A late checkpoint. Both heatmaps show that conflict is structured across variable pairs rather than being uniform, so one summary number would hide which pairs disagree.

Pairwise gradient-cosine distribution and early and late gradient cosine heatmaps for ETTm1

Figure 7. (a) Post-warmup cosine distribution. (b) Early checkpoint. (c) Late checkpoint. Figure 7: A representative ETTm1 run from the variable-wise gradient-cosine diagnostic at prediction length 96. (a) Distribution of the post-warmup pairwise cosines, of which 30.8% fall below zero for this run. (b) An early checkpoint, which lies inside the discarded warmup half and is shown only to expose the change over training. (c) A late checkpoint. Both heatmaps show that conflict is structured across variable pairs rather than being uniform, so one summary number would hide which pairs disagree.

Pairwise gradient-cosine distribution and early and late gradient cosine heatmaps for ETTm2

Figure 8. (a) Post-warmup cosine distribution. (b) Early checkpoint. (c) Late checkpoint. Figure 8: A representative ETTm2 run from the variable-wise gradient-cosine diagnostic at prediction length 96. (a) Distribution of the post-warmup pairwise cosines, of which 29.9% fall below zero for this run. (b) An early checkpoint, which lies inside the discarded warmup half and is shown only to expose the change over training. (c) A late checkpoint. Both heatmaps show that conflict is structured across variable pairs rather than being uniform, so one summary number would hide which pairs disagree.

Pairwise gradient-cosine distribution and gradient cosine heatmaps for a representative Weather run

Figure 9. A representative Weather run from the variable-wise gradient-cosine diagnostic at prediction length 96. (a) Distribution of the post-warmup pairwise cosines, of which 37.5% fall below zero for this run. (b) An early checkpoint, which lies inside the discarded warmup half and is shown only to expose the change over training. (c) A late checkpoint. Both heatmaps show that conflict is structured across variable pairs rather than being uniform, so one summary number would hide which pairs disagree.

Pairwise gradient-cosine distribution and gradient cosine heatmaps for a representative Exchange run

Figure 10. A representative Exchange run from the variable-wise gradient-cosine diagnostic at prediction length 96. (a) Distribution of the post-warmup pairwise cosines, of which 39.3% fall below zero for this run. (b) An early checkpoint, which lies inside the discarded warmup half and is shown only to expose the change over training. (c) A late checkpoint. Both heatmaps show that conflict is structured across variable pairs rather than being uniform, so one summary number would hide which pairs disagree.

Pairwise gradient-cosine distribution and gradient cosine heatmaps for a representative ILI run

Figure 11. A representative ILI run from the variable-wise gradient-cosine diagnostic at prediction length 24. (a) Distribution of the post-warmup pairwise cosines, of which 15.9% fall below zero for this run. (b) An early checkpoint, which lies inside the discarded warmup half and is shown only to expose the change over training. (c) A late checkpoint. Both heatmaps show that conflict is structured across variable pairs rather than being uniform, so one summary number would hide which pairs disagree.

receives every variable as input. Only the supervised loss is restricted to the target and its selected partners, and evaluation uses the target’s held-out per-variable test MSE. The 162 target-and-size cells cover every target in the four ETT benchmarks, Exchange, and ILI, plus 11 of Weather’s 21 targets. Higher- and lower-alignment conditions use matched replicate seeds, three per cell, which totals 972 group-training runs. We report the seed-averaged signed contrast 100(MSEhigh−MSElow)/MSElow100(\mathrm{MSE}_{\mathrm{high}}-\mathrm{MSE}_{\mathrm{low}})/\mathrm{MSE}_{\mathrm{low}}. Negative values favor the higher-alignment set. A cell counts as a higher-alignment win when its signed contrast is negative. The median signed contrast retains each cell’s direction, whereas the median absolute within-cell contrast removes the sign and measures the size of the partner-set difference. For every fixed dataset, target, group size, and partner condition, we express the standard deviation of test MSE across the three seeds as a percentage of their mean. To express seed variation on the scale of a two-run difference, we multiply this coefficient of variation by 2\sqrt{2}, which is the standard deviation of the difference under an equal-variance independence approximation. We use it as a descriptive reference rather than as a paired uncertainty estimate. The median over the 324 fixed partner conditions is 1.1%.

Figure 12 splits Figure 3(b) by dataset, and Figure 13 reports the higher- minus lower-alignment contrast for every benchmark and group size. Higher-alignment sets win 100 of the 162 cells, and the median signed contrast across cells is −1.4%-1.4\%. A two-sided one-sample tt-test over the seven benchmark-level mean contrasts does not reject a zero mean (p=0.45p = 0.45), although five of the seven means are negative. Excluding ILI shifts the pooled contrast from −0.9%-0.9\% to +0.4%+0.4\%. The median absolute within-cell contrast is 3.5%, which exceeds the 1.1% descriptive seed-variation reference above, so partner choice can materially change target error even though mean alignment does not reliably identify the better set. Across all 162 cells, Spearman’s correlation between the alignment gap and MSE difference is −0.19-0.19. The arithmetic mean of the seven within-benchmark correlations is −0.16-0.16, and a two-sided one-sample tt-test of those correlations against zero gives p=0.07p = 0.07. Figure 14 reports the seed-42 Weather extension at the six tested group sizes k∈{2,3,4,5,8,11}k \in\{2, 3, 4, 5, 8, 11\}, including the disjoint-set limit k=11k = 11 for D=21D = 21.

Figure 12. Figure 3(b) split by dataset, with the three group sizes marked separately. Each plot holds the 21 to 33 target-and-size cells of one benchmark.

Figure 14. Weather at the six tested group sizes k∈{2,3,4,5,8,11}k \in\{2, 3, 4, 5, 8, 11\}, using seed 42. Bars show the mean higher- versus lower-alignment MSE contrast across 11 targets, and error bars show the standard error across targets. Negative values favor the higher-alignment group. The contrast does not change monotonically with group size.

PV-Surgery Training Procedure

Algorithm 1 makes the four stages in Figure 4 explicit for one training batch. The reference gradient and output-cache signals come from the same single backward pass on the summed variable loss. Because no rescaling is applied to this sum loss, its raw reference gradient satisfies ∥g0∥2=D∥gmean∥2\lVert g_0 \rVert_2 = D\lVert g_{\mathrm{mean}} \rVert_2. Baseline and PV-Surgery runs use the same AdamW hyperparameters. The remaining operations transform cached gradient matrices and assemble the selected-slice correction. Let Hrun⊆H\mathcal{H}_{\mathrm{run}} \subseteq\mathcal{H} denote the layers whose proxy rows materialize for the current batch. We construct their observed proxy matrices and set Gℓ=0D×NℓG_\ell= 0_{D \times N_\ell} for ℓ∈H∖Hrun\ell\in\mathcal{H} \setminus\mathcal{H}_{\mathrm{run}}. Layer scoring and final assembly use this zero-extended collection over H\mathcal{H}, while parameters outside the hooked set retain their reference-gradient values.

Algorithm 1 PV-Surgery Training

Input: batch (X,Y)(X,Y), DD variables, backbone fθf_\theta, cache-compatible hooked layers H\mathcal{H} Output: optimizer gradient gfinalg_{\mathrm{final}}

// Step 1: Output-cache variable-wise gradient proxy

Y^←fθ(X)\hat{Y} \leftarrow f_\theta(X) {hooks cache {Zℓ}ℓ∈H\{Z_\ell\}_{\ell\in\mathcal{H}}}

LΣ←∑d=1DLd(Y^:,:,d,Y:,:,d)L_\Sigma\leftarrow\sum_{d=1}^{D} L_d(\hat{Y}_{:,:,d},Y_{:,:,d}); g0←Backward⁡(LΣ)g_0 \leftarrow\operatorname{Backward}(L_\Sigma) {(2), (3)}

Hrun←{ℓ∈H:Zℓ and Eℓ materialize for this batch}\mathcal{H}_{\mathrm{run}} \leftarrow\{\ell\in\mathcal{H}:Z_\ell\text{ and }E_\ell\text{ materialize for this batch}\}

Gℓ[d,⋅]←concat⁡(vec⁡(Eℓ,dZℓ,d),Eℓ,dT1)G_\ell[d,\cdot]\leftarrow\operatorname{concat}(\operatorname{vec}(E_{\ell,d}Z_{\ell,d}),E_{\ell,d}^{\mathsf{T}}\mathbf{1}) for ℓ∈Hrun\ell\in\mathcal{H}_{\mathrm{run}} {(4), (5)}

Gℓ←0D×NℓG_\ell\leftarrow0_{D\times N_\ell} for ℓ∈H∖Hrun\ell\in\mathcal{H}\setminus\mathcal{H}_{\mathrm{run}}

// Step 2: Reliability-aware layer selection

(eℓ,rℓ,qℓ)←LayerScores⁡(Gℓ,g0,ℓ,g0)(e_\ell,r_\ell,q_\ell)\leftarrow\operatorname{LayerScores}(G_\ell,g_{0,\ell},g_0) for ℓ∈H\ell\in\mathcal{H} {(6)}

Helig←{ℓ∈H:ℓ is not protected}\mathcal{H}_{\mathrm{elig}}\leftarrow\{\ell\in\mathcal{H}:\ell\text{ is not protected}\}

ρ←median⁡{rℓ:ℓ∈Helig}\rho\leftarrow\operatorname{median}\{r_\ell:\ell\in\mathcal{H}_{\mathrm{elig}}\}

Hgate←{ℓ∈Helig:rℓ≥ρ or ℓ=ℓout}\mathcal{H}_{\mathrm{gate}}\leftarrow\{\ell\in\mathcal{H}_{\mathrm{elig}}:r_\ell\geq\rho\text{ or }\ell=\ell_{\mathrm{out}}\}

A←{ℓ∈Hgate:qℓ>0}A\leftarrow\{\ell\in\mathcal{H}_{\mathrm{gate}}:q_\ell>0\}

if A≠∅A\ne\varnothing then

πℓ←qℓ/∑j∈Aqj\pi_\ell\leftarrow q_\ell/\sum_{j\in A}q_j

Keff←⌈exp⁡(−∑ℓ∈Aπℓlog⁡πℓ)⌉K_{\mathrm{eff}}\leftarrow\left\lceil\exp\left(-\sum_{\ell\in A}\pi_\ell\log\pi_\ell\right)\right\rceil

T←TopK⁡Keff(A,q)∪(A∩{ℓout})T\leftarrow\operatorname{TopK}_{K_{\mathrm{eff}}}(A,q)\cup(A\cap\{\ell_{\mathrm{out}}\})

S←{ℓ∈T:eℓ≤1}S\leftarrow\{\ell\in T:e_\ell\leq1\} {(7)}

else

S←∅S\leftarrow\varnothing

end if

S←OutputFallback⁡(S,Helig,ℓout)S\leftarrow\operatorname{OutputFallback}(S,\mathcal{H}_{\mathrm{elig}},\ell_{\mathrm{out}})

GS←concat⁡ℓ∈SGℓG_S\leftarrow\operatorname{concat}_{\ell\in S}G_\ell {(8)}

// Step 3: Conditional variable-gradient pooling

Gpre←GSG_{\mathrm{pre}}\leftarrow G_S; gref←g0,Sg_{\mathrm{ref}}\leftarrow g_{0,S}

V←{d:Gpre[d] is finite and ∥Gpre[d]∥2>ϵ}V\leftarrow\{d:G_{\mathrm{pre}}[d]\text{ is finite and }\lVert G_{\mathrm{pre}}[d]\rVert_2>\epsilon\}

(a,Mpool,active⁡)←ConditionalAnchorConflict⁡(Gpre,gref;valid⁡=V,order⁡=(ad,d))(a,M_{\mathrm{pool}},\operatorname{active})\leftarrow\operatorname{ConditionalAnchorConflict}(G_{\mathrm{pre}},g_{\mathrm{ref}};\operatorname{valid}=V,\operatorname{order}=(a_d,d)) {(9)}

if active then

Gpool←MpoolGpreG_{\mathrm{pool}}\leftarrow M_{\mathrm{pool}}G_{\mathrm{pre}} {(10)}

end if

// Step 4: Magnitude-preserving common-direction surgery

Gin(u)←GpreG_{\mathrm{in}}^{(u)}\leftarrow G_{\mathrm{pre}}, and Gin(p)←GpoolG_{\mathrm{in}}^{(p)}\leftarrow G_{\mathrm{pool}} when pooling is active

for each available candidate c∈{u,p}c\in\{u,p\} do

G~in(c)←CommonDirection⁡preserve(Gin(c))\widetilde{G}_{\mathrm{in}}^{(c)}\leftarrow\operatorname{CommonDirection}_{\mathrm{preserve}}(G_{\mathrm{in}}^{(c)}) {(11), (12)}

end for

Gupost←G~in(u)G_u^{\mathrm{post}}\leftarrow\widetilde{G}_{\mathrm{in}}^{(u)}

if active then

Gppost←PoolRestore⁡(Gpre,Mpool,G~in(p))G_p^{\mathrm{post}}\leftarrow\operatorname{PoolRestore}(G_{\mathrm{pre}},M_{\mathrm{pool}},\widetilde{G}_{\mathrm{in}}^{(p)}) {(13)}

else

Gppost←GpreG_p^{\mathrm{post}}\leftarrow G_{\mathrm{pre}}

end if

Gpost←DescentLexicographicSelect⁡u-tie(Gpre,Gupost,Gppost;V)G_{\mathrm{post}}\leftarrow\operatorname{DescentLexicographicSelect}_{u\text{-tie}}(G_{\mathrm{pre}},G_u^{\mathrm{post}},G_p^{\mathrm{post}};V) {(14)}

ΔS←∑dGpost[d,⋅]−∑dGpre[d,⋅]\Delta_S\leftarrow\sum_d G_{\mathrm{post}}[d,\cdot]-\sum_d G_{\mathrm{pre}}[d,\cdot] {(15)}

gfinal←Assemble⁡(g0,{Gℓ}ℓ∈H,S,ΔS)g_{\mathrm{final}}\leftarrow\operatorname{Assemble}(g_0,\{G_\ell\}_{\ell\in\mathcal{H}},S,\Delta_S) {(16)}

Pooling and candidate-selection conventions. Cosine and pairwise-conflict tests use only rows that are finite and have norm greater than the numerical tolerance. Cosines within a machine-scale tolerance of zero are treated as zero. A zero or non-finite row is assigned ad=0a_d=0, remains a singleton objective, and does not activate pooling. A zero or non-finite reference yields the identity pooling plan. When pooling is active, conflict rows are processed in ascending reference-alignment order, with the variable index breaking ties. Each row joins the existing conflict pool with the largest cosine only when that cosine is positive. Otherwise, it starts a new pool. Candidate scoring uses the same valid pre-surgery rows. A candidate with a zero or non-finite aggregate is invalid. If only one candidate is valid, it is selected. If both are invalid or numerically tied, the unpooled candidate is retained.

Numerical tolerance for common-direction surgery. Equation 12 defines the ideal map using an exact zero test. In the implementation, we use the first branch only when ∥g~(c)∥2>ϵ\lVert\tilde{g}^{(c)}\rVert_2>\epsilon and otherwise retain the input, where ϵ>0\epsilon> 0 is a small numerical tolerance. This avoids normalizing a near-zero mean direction.

Proxy-supported hooked-block updates. PV-Surgery treats the hooked parameter blocks as the region in which an update should remain attributable to variable-indexed signals. When ℓ∈Hrun\ell\in\mathcal{H}_{\mathrm{run}}, every hooked-block contribution is constructed from observed proxy rows and can therefore pass through the same reconstruction, pooling, and direction checks as the other variable-wise inputs. When ℓ∉Hrun\ell\notin\mathcal{H}_{\mathrm{run}}, the single-backward slice g0,ℓg_{0,\ell} is still available, but it contains only the aggregate of the unresolved variable contributions. Substituting that slice would reintroduce an update whose variable-wise sources cannot be inspected by the method. We therefore impose a blockwise evidence-support constraint that permits a nonzero hooked-block data-gradient contribution only when its variable-indexed rows materialize. Under this constraint, zero is the only admissible current data-gradient contribution on an unsupported block and is also the minimum-norm choice. This is a conservative abstention rule rather than a claim that a zero data-gradient contribution is a better descent direction than g0,ℓg_{0,\ell}. Parameters outside H\mathcal{H} are not assigned to the variable-aware region and continue to use g0g_0. A zero-extended block remains zero even if it enters SS, because its columns are zero for every surgery input and the norm-preserving direction operator cannot create support on those columns.

This zero data-gradient contribution is injected as an explicit zero loss-gradient tensor. It is therefore not the same as removing the parameters from AdamW or setting their gradients to None. AdamW still advances their optimizer state and applies decoupled weight decay. If a block has accumulated momentum from earlier nonzero updates, that state can continue to affect later parameter values while it decays. The rule should consequently be understood as withholding the current data-gradient update on an unsupported hooked block, not as permanently freezing the block.

Layer-selection thresholds. Although the cache supports explicit variable axes and supported batch-folded layouts, the configured protection rule excludes patch and value embedding parameters from direction-surgery selection to avoid changing the shared input representation before downstream variable mixing. If registered as hooked blocks, these parameters still follow the proxy-sum assembly rule above. Excluding them from SS prevents direction-surgery correction on those slices but does not restore g0g_0. Their data-gradient contribution is the sum of the variable-indexed proxy rows when those rows materialize for the current batch, and zero when no such variable-axis proxy materializes. Among the remaining candidates the reliability gate keeps every non-output layer whose rℓr_\ell reaches the median over the eligible hooked layers, and the output layer is exempt from this gate. Equation 7 is evaluated on qℓq_\ell, which also orders the layers before the KeffK_{\mathrm{eff}} cut. A layer that survives the cut is eligible when its relative reconstruction error satisfies ϵℓ≤1\epsilon_\ell\le1, equivalently rℓ≥1/2r_\ell\ge1/2. This means that its proxy-sum error does not exceed ∥g0,ℓ∥2+ϵ\lVert g_{0,\ell}\rVert_2+\epsilon, which marks, up to numerical tolerance, the boundary at which the reconstruction error begins to exceed the magnitude of the gradient being reconstructed. If nothing is left, PV-Surgery falls back to the output layer.

Properties of effective coverage. The unrounded quantity eϵlayere^{\epsilon_{\mathrm{layer}}} in Equation 7 is the exponential of the Shannon entropy of π\pi and therefore the order-one Hill number (Hill, 1973) of the normalized slice-norm distribution. Because πℓ=qℓ/∑j∈Aqj\pi_\ell=q_\ell/\sum_{j\in A}q_j, it represents the fraction of the total gated slice norm carried by layer ℓ\ell. The entropy proxy eϵlayere^{\epsilon_{\mathrm{layer}}} measures how evenly these fractions are distributed, and exponentiation converts this log-scale measure into an effective number of contributing layers. This number is one when a single layer carries all of the slice norm and mm when the norm is uniform over mm layers. An intermediate value is the number of equally contributing layers that would produce the same concentration as the observed distribution. For example, π=(0.8,0.1,0.1)\pi=(0.8,0.1,0.1) gives eϵlayer≈1.89e^{\epsilon_{\mathrm{layer}}}\approx1.89 and therefore Keff=2K_{\mathrm{eff}}=2 after taking the ceiling. The selector uses this ceiling as an entropy-derived cutoff for the largest-norm layers, while the reliability and reconstruction criteria still determine which layers are eligible. The Hill number is invariant to a common rescaling of the layer slice norms, so a backbone whose layer gradients are uniformly larger does not receive a wider support. It is recomputed at each step, allowing the support to contract when one layer dominates and to widen when the slice-norm distribution becomes diffuse. A layer whose share vanishes contributes πℓlog⁡πℓ→0\pi_\ell\log\pi_\ell\to0 and has a vanishing effect on the effective count. The choice of order also matters. Order zero counts every gated layer equally, while order two emphasizes dominant layers more strongly. Order one retains the Shannon weighting in which each entropy contribution is weighted by its normalized slice-norm share πℓ\pi_\ell. We avoid a fixed top-kk because the number of hooked layers differs by backbone, from one to seventeen in our experiments, so a new backbone would otherwise need its own value.

Detailed Experimental Configuration

Forecasting protocol. For ETTh1, ETTh2, ETTm1, ETTm2, Weather, and Exchange, we use an input length of 96, a decoder-label length of 48, and prediction lengths {96,192,336,720}\{96, 192, 336, 720\}. ILI uses input and decoder-label lengths of 36 and 18. Its main, ablation, mechanism, and objective-comparison runs use prediction lengths {24,36,48,60}\{24, 36, 48, 60\}, while the diagnostic, oracle, and partner-group analyses in Section 3 use 24. Every method uses the chronological train, validation, and test splits provided by the forecasting framework. Training windows are not shuffled because Selective Learning tracks residual statistics across consecutive windows [2].

Optimization. Unless otherwise noted for the multi-seed analyses in Appendix C, each reported run uses seed 42. Training uses a batch size of 64 for at most 30 epochs and stops after seven consecutive validation epochs without improvement. We use AdamW with a fixed learning rate of 10−410^{-4}, weight decay 5×10−45 \times10^{-4}, and a gradient-norm limit of 1.

Specialized-objective settings. In our objective comparison, TILDE-Q uses α=0.5\alpha= 0.5 and γ=0\gamma= 0, which removes the amplitude term. FreDF weights the temporal MSE and frequency-domain terms by 0.5 each. Its frequency MAE is the mean modulus of the complex RFFT difference. PSLoss uses λ=3\lambda= 3, limits the adaptive patch length to 24, and enables Gradient-based Dynamic Weighting with automatic target-layer selection and an equal-weight fallback. Selective Learning uses (ru,ra)=(0.3,0.3)(r_u, r_a) = (0.3, 0.3), (0.1,0.6)(0.1, 0.6), (0.2,0.2)(0.2, 0.2), (0.2,0.5)(0.2, 0.5), (0.1,0.2)(0.1, 0.2), (off,0.9)(\mathrm{off}, 0.9), and (0.1,0.1)(0.1, 0.1) for ETTh1, ETTh2, ETTm1, ETTm2, Weather, Exchange, and ILI, respectively. Its anomaly mask uses a pretrained DLinear estimator for each dataset and horizon. TILDE-Q, FreDF, and PSLoss select checkpoints using their configured validation objectives, whereas Selective Learning uses validation MSE. Early stopping monitors validation MSE with patience 7 for all four methods.

Diagnostic logging. The shared-model diagnostics in Section 3 record gradients every max⁡(1,⌊30S/200⌋)\max(1, \lfloor30S/200 \rfloor) optimizer steps, where SS is the number of steps in one epoch. This interval targets approximately 200 snapshots over a complete 30-epoch run. Early stopping reduces the total in most benchmarks, while ILI has an interval of one step because each epoch is short. Before removing the warmup half, the runs contain 74 to 300 snapshots. The post-warmup counts reported in Appendix B are therefore 37 to 150.

Backbone hyperparameters. DLinear uses a moving-average window of 25 and shared rather than variable-specific trend and seasonal heads. iTransformer uses dmodel=512d_{\mathrm{model}} = 512, dff=512d_{\mathrm{ff}} = 512, 8 attention heads, 3 encoder layers, GELU activations, and dropout 0.1. TimeXer uses dmodel=256d_{\mathrm{model}} = 256, dff=512d_{\mathrm{ff}} = 512, 8 attention heads, 2 encoder layers, patch length 16, GELU activations, dropout 0.1, and input normalization. MICN uses dmodel=512d_{\mathrm{model}} = 512, dff=2048d_{\mathrm{ff}} = 2048, 8 heads, 1 decoder layer, convolution kernels {12,16}\{12, 16\}, and dropout 0.05. SCINet uses one stack and dropout 0.1. Standard MSE training and PV-Surgery use the same backbone settings.

Hardware. We run the experiments on NVIDIA TITAN RTX GPUs with 24 GB of memory.

Metric definitions. We compute MSE and MAE on standardized forecasts and targets without applying the inverse transformation. Let NtestN_{\mathrm{test}} denote the number of test windows. The reported metrics are

MSE=1NtestHD∑n=1Ntest∑h=1H∑d=1D(Y^n,h,d−Yn,h,d)2,MAE=1NtestHD∑n=1Ntest∑h=1H∑d=1D∣Y^n,h,d−Yn,h,d∣.(17)\begin{aligned} \mathrm{MSE} &= \frac{1}{N_{\mathrm{test}}HD} \sum_{n=1}^{N_{\mathrm{test}}} \sum_{h=1}^{H} \sum_{d=1}^{D} \left(\widehat{Y}_{n,h,d} - Y_{n,h,d}\right)^2, \\ \mathrm{MAE} &= \frac{1}{N_{\mathrm{test}}HD} \sum_{n=1}^{N_{\mathrm{test}}} \sum_{h=1}^{H} \sum_{d=1}^{D} \left\lvert\widehat{Y}_{n,h,d} - Y_{n,h,d} \right\rvert. \tag*{(17)} \end{aligned}

Backbone Results for Weather and ILI

The page limit restricts Table 1 to five of the seven benchmarks. Table 6 reports the omitted Weather and ILI results, completing the same comparison over five backbones, seven datasets, and four prediction lengths. Each MSE and +PV pair uses the same data split, backbone configuration, and optimization protocol.

ModelsDLineariTransformerMICNSCINetTimeXer
Training setupMSE+PVMSE+PVMSE+PVMSE+PVMSE+PV
MetricMSEMAEMSEMAEMSEMAEMSEMAEMSEMAE
ETTh1 960.4620.4440.4610.4400.4540.4470.4510.4420.5520.530
ETTh1 1920.5140.4760.5140.4710.5170.4850.5080.4760.5940.553
ETTh1 3360.5590.5060.5560.5010.5610.5130.5500.5010.6880.623
ETTh1 7200.6510.5730.6440.5610.6710.5830.7000.5880.7530.671
ETTh1 Avg0.5460.5000.5430.4930.5510.5070.5430.4990.6470.594
ETTh2 960.2390.3290.2290.3130.2390.3230.2290.3200.2380.333
ETTh2 1920.3290.3930.2900.3570.3030.3660.2870.3550.3150.378
ETTh2 3360.4370.4620.3410.3960.3560.4010.3340.3880.4520.472
ETTh2 7200.6500.5820.4700.4840.4570.4600.4460.4520.6720.583
ETTh2 Avg0.4140.4420.3320.3880.3390.3880.3240.3770.4190.444
ETTm1 960.3880.4470.3830.3890.4070.4150.4020.4040.4450.472
ETTm1 1920.4470.4260.4450.4210.4800.4570.4610.4390.4950.511
ETTm1 3360.5070.4590.5050.4540.5200.4770.5140.4430.5670.570
ETTm1 7200.5750.5020.5730.4990.6180.5310.5750.5100.6270.596
ETTm1 Avg0.4790.4700.4760.4410.5060.4700.4880.4490.5330.505
ETTm2 960.1600.2680.1550.2530.1570.2580.1500.2470.1660.276
ETTm2 1920.2090.3080.1970.2870.2050.2950.1950.2830.2020.290
ETTm2 3360.2620.3500.2390.3190.2520.3280.2400.3170.2660.355
ETTm2 7200.3480.4080.3060.3650.3220.3730.3100.3650.3890.434
ETTm2 Avg0.2450.3340.2250.3060.2340.3130.2240.3030.2560.342
Exchange 960.1050.2340.1000.2260.1080.2360.1050.2310.1230.262
Exchange 1920.2240.3490.1910.3230.2220.3440.2110.3350.2190.352
Exchange 3360.4250.4980.3210.4250.3970.4630.3900.4600.4980.525
Exchange 7200.6520.6400.8470.7241.0990.8001.0750.7922.9311.400
Exchange Avg0.3520.4300.3650.4240.4570.4610.4460.4540.9430.635

Table 1. Backbone comparison between standard MSE training and PV-Surgery on five of the seven benchmarks. Within each backbone and metric, the better value between MSE and +PV is bolded using unrounded scores. Weather and ILI use the same protocol and are reported in Table 6.

ModelsDLineariTransformerMICNSCINetTimeXer
Training setupMSE+PVMSE+PVMSE+PVMSE+PVMSE+PV
MetricMSEMAEMSEMAEMSEMAEMSEMAEMSEMAE
Weather 960.2030.2660.2030.2450.1830.2220.1820.2220.3980.4420.3180.3830.1660.2120.1700.2150.1640.2080.1800.223
1920.2410.3020.2410.2920.2310.2630.2310.2650.4030.4470.4350.4600.2180.2590.2180.2570.2080.2510.2330.269
3360.2860.3380.2830.3250.2850.3000.2810.2990.3800.4220.4360.4680.2800.3030.2780.3000.2610.2900.2830.302
7200.3490.3850.3450.3680.3580.3490.3530.3450.4960.5010.4790.4930.3610.3560.3570.3520.3400.3420.3560.350
Avg0.2700.3230.2680.3080.2640.2830.2620.2830.4190.4530.4170.4510.2560.2820.2560.2810.2430.2730.2630.286
ILI 243.8281.4643.5901.4142.0180.8782.1900.9053.1691.2373.1901.2433.9461.4053.9451.4002.7980.9902.7231.047
363.8011.4353.4901.3582.1730.9452.2930.9432.9961.2282.9851.2203.9701.4173.7781.3692.2040.9312.6191.056
483.8081.4203.4461.3302.0990.9402.2830.9413.2691.2843.2281.2723.9771.4343.6981.3712.2500.9542.5421.041
604.1751.4463.7601.3592.5791.1072.1360.9323.2731.2673.3641.2833.9611.4333.6391.3682.5721.0582.4061.009
Avg3.9031.4413.5711.3652.2170.9672.2250.9303.1771.2543.1911.2543.9631.4223.7651.3772.4560.9832.5721.038

Table 6. Backbone comparison between standard MSE training and PV-Surgery on Weather and ILI. Within each backbone and metric, the better value between MSE and +PV is bolded using unrounded scores. The protocol and horizons follow Table 1.

Loss-Baseline Results for All Backbones

The page limit restricts Table 2 to iTransformer on five of the seven benchmarks. Table 7–Table 11 report all seven benchmarks separately for each backbone, completing 280 backbone, dataset, prediction-length, and metric cells.

DatasetMetricETTh1 96192336720ETTh2 96192336720ETTm1 96192336720ETTm2 96192336720Weather 96192336720Exchange 96192336720ILI 24364860
MSEMSE0.4620.5140.5590.6510.2390.3290.4370.6500.3880.4470.5070.5750.1600.2090.2620.3480.2030.2410.2860.3490.1050.2240.4250.6523.8283.8013.8084.175
MAE0.4440.4760.5060.5730.3290.3930.4620.5820.3950.4260.4590.5020.2680.3080.3500.4080.2660.3020.3380.3850.2340.3490.4980.6401.4641.4351.4201.446
TILDE-QMSE0.4670.5160.5580.6470.2330.2930.3540.5240.3830.4450.5100.5760.1580.2030.2460.3170.2000.2380.2820.3430.1160.2130.4320.8884.0143.6853.5383.777
MAE0.4430.4720.5010.5670.3170.3610.4080.5140.3870.4200.4550.4960.2580.2940.3250.3750.2420.2810.3210.3710.2500.3480.5090.7601.5291.4101.3521.365
FreDFMSE0.4590.5110.5550.6440.2350.3080.3910.5910.3860.4460.5060.5740.1570.2020.2480.3190.2000.2380.2810.3440.1070.2080.4300.7903.8103.7343.7974.146
MAE0.4390.4710.5010.5660.3230.3780.4370.5500.3920.4240.4580.5010.2600.2950.3310.3800.2490.2860.3200.3700.2390.3450.4970.7001.4661.4221.4271.456
PSLossMSE0.4620.5130.5570.6480.2380.3080.3900.6260.3850.4450.5050.5730.1560.1970.2420.3170.2020.2380.2810.3410.1200.2260.3880.7003.9613.7753.5773.763
MAE0.4410.4740.5040.5700.3250.3760.4330.5600.3910.4210.4550.4970.2540.2860.3220.3770.2420.2770.3110.3590.2560.3580.4780.6551.5031.4341.3621.368
SLMSE0.4600.5140.5580.6500.2440.3230.3950.6230.3920.4520.5110.5780.1600.2160.2750.3780.2000.2400.2880.3480.1050.2190.4060.6753.8723.8193.7534.158
MAE0.4420.4760.5060.5740.3330.3890.4370.5630.3970.4280.4600.5050.2680.3150.3610.4280.2590.3010.3410.3860.2360.3480.4870.6491.4591.4251.3961.437
PV(Ours)MSE0.4610.5140.5560.6410.2290.2900.3410.4700.3830.4450.5050.5730.1550.1970.2390.3060.2030.2410.2830.3450.1000.1910.3210.8473.5903.4903.4463.760
MAE0.4400.4710.5010.5610.3130.3570.3960.4840.3890.4210.4540.4990.2530.2870.3190.3650.2450.2920.3250.3680.2260.3230.4250.7241.4141.3581.3301.359

Table 7. Loss-baseline comparison on DLinear for all seven benchmarks. The MSE row denotes standard MSE training. Best results are bolded and second-best results are underlined using unrounded scores.

DatasetETTh1ETTh2ETTm1ETTm2WeatherExchangeILI
Forecast length96192336720961923367209619233672096192336720961923367209619233672024364860
MSEMSE0.4600.5110.5640.6930.2330.2940.3410.4490.4320.4770.5210.5800.1510.2000.2390.3120.1640.2080.2610.3400.1130.2160.3961.1052.7982.2042.2502.572
MAE0.4540.4850.5110.5900.3190.3600.3940.4570.4210.4530.4790.5150.2490.2870.3160.3670.2080.2510.2900.3420.2390.3370.4590.7980.9900.9310.9541.058
TILDE-QMSE0.4740.5250.5740.6930.2380.2950.3380.4360.4170.4600.5170.5660.1480.1940.2370.3050.1630.2160.2750.3520.1220.2270.4561.1712.6152.5032.5202.547
MAE0.4550.4880.5140.5920.3230.3590.3890.4480.4070.4370.4710.5100.2440.2800.3120.3570.2090.2580.2980.3480.2500.3480.5020.8361.0351.0221.0281.040
FreDFMSE0.4750.5560.6580.8020.2430.3000.3480.4480.4520.5100.5750.6610.1550.2020.2440.3160.1650.2240.2810.3610.1330.2560.4781.2262.6592.5262.6402.658
MAE0.4610.5100.5620.6420.3300.3660.3980.4560.4400.4710.5010.5520.2570.2920.3200.3670.2160.2700.3070.3600.2620.3730.5150.8571.0101.0311.0851.090
PSLossMSE0.4680.5220.5790.6990.2370.2950.3420.4380.4170.4650.5170.5720.1520.1960.2380.3080.1650.2190.2820.3570.1370.2540.4521.1922.7102.3162.6712.541
MAE0.4590.4940.5200.6020.3210.3600.3910.4480.4160.4450.4770.5120.2520.2850.3150.3600.2160.2620.3040.3520.2680.3720.4990.8441.0440.9731.0901.081
SLMSE0.4750.5290.5720.7030.2260.2830.3290.4380.4070.4550.5220.6000.1530.1980.2410.3220.1620.2110.2630.3420.1080.2220.4121.2082.5392.5842.4502.392
MAE0.4480.4800.5070.5860.3110.3500.3800.4540.4070.4250.4600.5020.2520.2840.3150.3680.2040.2490.2880.3410.2290.3400.4720.8450.9861.0381.0181.001
PV(Ours)MSE0.4610.5180.5640.6910.2310.2950.3420.4460.3960.4550.5050.5640.1490.1930.2350.3070.1800.2330.2830.3560.1130.2130.3821.0102.7232.6192.5422.406
MAE0.4500.4850.5150.5810.3160.3580.3900.4510.4030.4290.4620.5030.2470.2820.3120.3600.2230.2690.3020.3500.2390.3350.4520.7631.0471.0561.0411.009

Table 11. Loss-baseline comparison on TimeXer for all seven benchmarks. The MSE row denotes standard MSE training. Best results are bolded and second-best results are underlined using unrounded scores.

No method is best in every cell. Counting a tie for each tied method, PV-Surgery gives the lowest value in 91 of the 280 cells and Selective Learning in 77. These tables compare the methods as alternatives and do not evaluate combined objectives. The specialized methods change the forecasting loss, whereas PV-Surgery retains pointwise MSE and modifies the optimizer update. The two mechanisms can therefore be combined in principle, but their joint effect is not evaluated here.

Dataset-Wise Component Ablations

Table 3 reports the component ablation for MICN on Exchange. Table 12–Table 18 give the corresponding result for each dataset. Every entry averages the five backbones and four prediction lengths, yielding 20 matched settings per dataset. Relative changes are computed against the full configuration using the unrounded averages.

CategoryModuleMSE valueMSE Δ\Delta%MAE valueMAE Δ\Delta%Time (s)
Gradient proxyw/o output-cache proxy0.937+107.70.629+35.7338.92
Layer selectionw/o layer selection0.538+19.10.482+4.0642.98
Variable poolingw/o conflict pooling0.858+90.00.607+31.0253.42
Variable poolingw/o anchor pooling0.506+12.10.478+3.2272.47
Variable poolingw/o variable pooling0.497+10.00.474+2.2277.40
Direction surgeryw/o magnitude decoupling0.512+13.50.493+6.3333.05
Direction surgeryw/o safe candidate selection0.478+6.00.478+3.1312.27
PV-Surgery (full)0.451–0.463–332.45

Table 3. Core component ablation on MICN–Exchange, averaged over four horizons. Δ\Delta is the relative change against the full recipe. Appendix H reports the same table for every dataset.

CategoryModuleMSE valueMSE Δ%MAE valueMAE Δ%Time (s)
Gradient proxyw/o output-cache proxy20.574+0.80.525+0.9192.97
Layer selectionw/o layer selection0.569−0.20.520−0.0288.11
Variable poolingw/o conflict pooling0.576+1.10.524+0.8197.07
Variable poolingw/o anchor pooling0.568−0.20.519−0.0213.68
Variable poolingw/o variable pooling0.568−0.30.519−0.1198.92
Direction surgeryw/o magnitude decoupling0.568−0.30.519−0.2225.96
Direction surgeryw/o safe candidate selection0.578+1.50.525+0.9194.06
PV-Surgery (full)0.570−0.520−213.65

Table 12. Core component ablation on ETTh1, averaged over 20 matched backbone–prediction-length settings. Δ is the relative change against the full configuration. Bold indicates the best score and underlining the second best using unrounded averages.

CategoryModuleMSEMAETime (s)
valueΔ\Delta%valueΔ\Delta%
Gradient proxyw/o output-cache proxya^{a}<u>3.05</u>−0.41.18−0.712.80
Layer selectionw/o layer selection3.04−0.8<u>1.19</u>−0.519.48
Variable poolingw/o conflict pooling3.08+0.41.20+0.415.07
w/o anchor pooling3.09+0.71.20+0.315.03
w/o variable pooling3.09+0.71.20+0.314.43
Direction surgeryw/o magnitude decoupling3.13+2.01.22+2.013.79
w/o safe candidate selection3.09+0.71.20+0.614.12
PV-Surgery (full)3.07−1.19−15.21

Table 18. Core component ablation on ILI, averaged over 20 matched backbone–prediction-length settings. Δ\Delta is the relative change against the full configuration. Bold indicates the best score and underlining the second best using unrounded averages.

For every removal, the mean of the seven dataset-level relative MSE changes is positive, so each module contributes on average. Removing a component improves MSE on some datasets, but the largest such gain is 0.9%. Every ablated variant is also at least 2.1% worse than the full configuration on another dataset. Overall, the full configuration has lower MSE in 37 of the 49 dataset-variant comparisons. It is not uniformly best, but retaining all modules avoids the larger and more frequent degradations and provides the most stable performance across datasets.

DatasetMetricETTh1 96192336720ETTh2 96192336720ETTm1 96192336720ETTm2 96192336720Weather 96192336720Exchange 96192336720ILI 24364860
MSEMSE0.4540.5170.5610.6710.2390.3030.3560.4570.4070.4800.5200.6180.1570.2050.2520.3220.1830.2310.2850.3580.1080.2220.3971.0992.0182.1732.2932.579
MAE0.4470.4850.5130.5830.3230.3660.4010.4600.4150.4570.4770.5310.2580.2950.3280.3730.2220.2630.3000.3450.2360.3440.4630.8000.8780.9450.9401.107
TILDE-QMSE0.4690.5280.5640.6690.2330.2870.3540.4730.4020.4650.5370.6220.1560.2020.2500.3730.1800.2210.2870.3500.1110.2190.4281.1091.9692.0352.1432.548
MAE0.4580.4890.5100.5790.3150.3550.4000.4600.4100.4460.4930.5520.2510.2850.3220.3710.2170.2610.2970.3400.2190.3410.4281.0960.9700.9940.9321.099
FreDFMSE0.4510.5120.5600.6620.2350.2950.3500.4500.4100.4750.5530.6030.1540.1970.2450.3090.1780.2270.2870.3340.1120.2230.4621.1322.0472.0792.2722.507
MAE0.4450.4780.5080.5760.3170.3500.3900.4550.4080.4520.4920.5340.2500.2840.3190.3610.2160.2570.2970.3420.2340.3380.4760.8250.9720.7090.9441.091
PSLossMSE0.4580.5130.5580.6650.2340.2910.3600.4730.4290.4990.5380.6260.1530.1990.2390.3100.1770.2140.2780.3520.1130.2130.4131.1461.8802.0252.3002.562
MAE0.4480.4800.5090.5770.3160.3550.4060.4690.4210.4590.4820.5350.2470.2810.3140.3610.2140.2540.2930.3420.2410.3380.4760.8250.8990.9271.0031.119
SLMSE0.4650.5270.5600.6650.2290.2840.3480.4480.4040.4580.5210.5900.1500.1990.2430.3120.1820.2360.2640.3050.1080.2110.3931.2282.3242.3122.3242.348
MAE0.4490.4850.5010.5680.3120.3490.3930.4490.4000.4320.4650.5050.2500.2850.3170.3620.2190.2640.2970.3460.2320.3320.4600.8560.9720.9850.9821.018
PV(Ours)MSE0.4510.5080.5500.6610.2290.2870.3440.4460.4020.4610.5170.5750.1500.1950.2400.3100.1820.2310.2810.3500.1050.2110.3901.0752.1902.2932.2832.136
MAE0.4420.4760.5040.5750.3140.3550.3880.4460.4040.4390.4730.5100.2470.2830.3170.3650.2220.2650.2990.3450.2230.3350.4600.7920.9050.9430.9410.932

Table 8. Loss-baseline comparison on iTransformer for all seven benchmarks. The MSE row denotes standard MSE training. Best results are bolded and second-best results are underlined using unrounded scores.

DatasetMetricETTh1 96192336720ETTh2 96192336720ETTm1 96192336720ETTm2 96192336720Weather 96192336720Exchange 96192336720ILI 24364860
MSEMSE0.5520.5940.6680.7530.2380.3150.4520.6720.4450.4950.5670.6270.1600.2020.2660.3890.3980.4430.4200.4960.1230.2190.4981.2931.3692.2963.2693.273
MAE0.5300.5530.6230.6710.3330.3880.4720.5820.4720.5110.5700.5970.2660.3030.3550.4390.4420.4740.4220.5020.2620.3520.4191.8601.2271.2281.2841.267
TILDE-QMSE0.4800.5290.5560.9160.2370.3000.4030.6560.4080.4490.5050.5480.1530.2000.2410.3080.2990.4000.5063.2894.1093.0271.3453.475
MAE0.4850.5180.5340.7470.3250.3710.4450.5790.4340.4640.5010.5330.2570.2950.3240.3730.3640.4310.5010.5120.3570.3491.4261.1111.2451.2651.303
FreDFMSE0.5010.5480.5790.6990.2410.3260.3960.7350.4230.4490.5160.5700.1480.1910.2360.3010.3250.3220.3400.4510.2100.4163.0673.5852.9993.0543.286
MAE0.5040.5330.5520.6900.3320.3960.4460.6180.4540.4710.5100.5540.2520.2860.3220.3680.3860.3840.3880.4690.2190.3251.4261.3251.2361.2261.274
PSLossMSE0.4920.5290.5580.7530.2360.3090.4020.6300.4280.4750.5160.9240.1490.1870.3030.3020.2820.3370.4230.4800.1290.2430.4461.6493.2312.9913.1343.347
MAE0.4930.5170.5340.6660.3280.3810.4500.5660.4500.4830.5050.7610.2660.2830.3200.3710.3570.3940.4230.4860.2680.3730.5050.9391.2551.2301.2501.271
SLMSE0.4890.5320.5640.6690.2360.3010.4310.6150.3970.4440.4680.6110.1530.2090.3450.3840.2470.3210.3761.0373.2171.2951.2571.310
MAE0.4660.4940.5860.6590.3260.3710.4030.5650.4080.4420.4680.5550.2570.2980.3450.4310.3180.3780.4360.4760.2490.3430.4811.0541.2511.2081.2431.310
PV(Ours)MSE0.4920.5610.7000.7980.2300.2910.3520.5130.4300.4700.5080.5760.1560.1980.2360.3080.3180.4350.4761.1063.2902.9852.2283.364
MAE0.4970.5430.6300.7000.3200.3690.4120.5200.4600.4870.5130.5620.2480.2850.3160.3890.3830.4600.4680.4930.2420.3480.4700.7931.2431.2201.2721.283

Table 9. Loss-baseline comparison on MICN for all seven benchmarks. The MSE row denotes standard MSE training. Best results are bolded and second-best results are underlined using unrounded scores.

DatasetMetricETTh1 96192336720ETTh2 96192336720ETTm1 96192336720ETTm2 96192336720Weather 96192336720Exchange 96192336720ILI 24364860
MSEMSE0.4900.5450.5900.7000.2540.3110.3560.4700.4480.4880.5480.6140.1550.2020.2480.3210.1660.2180.2800.3610.1200.2230.4051.1013.9463.9703.9773.961
MAE0.4640.4950.5210.5880.3340.3700.4020.4720.4280.4530.4830.5210.2530.2900.3220.3680.2120.2590.3030.3560.2120.2590.3700.8041.4051.4711.4341.433
TILDE-QMSE0.4850.5400.5850.6960.2380.3010.3550.4540.4410.4750.5340.6030.1480.1940.2400.3120.1620.2170.2780.3520.1160.2120.3991.0924.1014.1483.9693.921
MAE0.4570.4880.5140.5830.3200.3620.3970.4570.4140.4370.4680.5110.2430.2790.3130.3590.2080.2560.2980.3460.2460.3420.4640.8041.4631.4481.4321.428
FreDFMSE0.4700.5260.5710.6800.2340.2930.3450.4470.4440.4810.5310.5890.1460.1920.2360.3070.1610.2130.2710.3350.1120.2160.3951.0933.7131.3695.1843.359
MAE0.4470.4770.5080.5840.3160.3290.3690.4540.4190.4450.4710.5040.2430.2780.3090.3580.2060.2510.2930.3450.2340.3270.4560.7971.3991.3681.3541.339
PSLossMSE0.4770.5330.5770.6830.2350.2970.3480.4530.4220.4400.5120.5090.2460.2810.3110.3580.2050.2500.2930.3450.2530.3470.4660.7971.3991.3681.3551.355
MAE0.4530.4850.5120.5800.3200.3590.3980.4570.4220.4400.4720.5090.2460.2810.3110.3580.2050.2500.2930.3450.2530.3470.4660.7971.3991.3681.3551.355
SLMSE0.4760.5330.5770.6830.2350.2970.3480.4530.3890.4490.5130.5850.1520.1960.2410.3140.1600.2120.2700.3530.1810.2120.3851.1813.9803.9593.9483.888
MAE0.4490.4820.5070.5760.3180.3590.3950.4570.3910.4400.4550.5810.2460.2810.3110.3580.2050.2500.2930.3450.2380.3360.4550.7911.3991.3681.3551.355
PV(Ours)MSE0.4730.5300.5740.6870.2370.2970.3460.4470.4210.4900.5510.6180.1500.1930.2360.3080.1700.2180.2780.3610.1430.1970.3591.1033.9453.7763.6983.639
MAE0.4490.4830.5100.5810.3170.3980.3910.4540.4140.4460.4750.5160.2460.2780.3090.3570.2150.2570.3000.3570.2440.3460.4730.8061.4051.3691.3711.368

Table 10. Loss-baseline comparison on SCINet for all seven benchmarks. The MSE row denotes standard MSE training. Best results are bolded and second-best results are underlined using unrounded scores.

CategoryModuleMSE valueMSE Δ%MAE valueMAE Δ%Time (s)
Gradient proxyw/o output-cache proxy30.367+10.50.407+5.6238.37
Layer selectionw/o layer selection0.343+3.00.391+1.5338.14
Variable poolingw/o conflict pooling0.340+2.20.390+1.2265.51
Variable poolingw/o anchor pooling0.344+3.40.393+1.9252.61
Variable poolingw/o variable pooling0.343+3.00.392+1.7239.77
Direction surgeryw/o magnitude decoupling0.335+0.60.387+0.3256.50
Direction surgeryw/o safe candidate selection0.333+0.20.386+0.0258.56
PV-Surgery (full)0.333−0.386−274.73

Table 13. Core component ablation on ETTh2, averaged over 20 matched backbone–prediction-length settings. Δ is the relative change against the full configuration. Bold indicates the best score and underlining the second best using unrounded averages.

CategoryModuleMSE valueMSE Δ\Delta%MAE valueMAE Δ\Delta%Time (s)
Gradient proxyw/o output-cache proxy20.511+3.70.474+2.3792.11
Layer selectionw/o layer selection0.496+0.70.463+0.11075.03
Variable poolingw/o conflict pooling0.493+0.10.464+0.2902.63
w/o anchor pooling0.498+1.00.467+0.8915.37
w/o variable pooling0.494+0.40.465+0.4822.81
Direction surgeryw/o magnitude decoupling0.498+1.10.463+0.0906.55
w/o safe candidate selection0.497+0.80.464+0.3919.78
PV-Surgery (full)0.493–0.463–857.04

Table 14. Core component ablation on ETTm1, averaged over 20 matched backbone–prediction-length settings. Δ\Delta is the relative change against the full configuration. Bold indicates the best score and underlining the second best using unrounded averages.

CategoryModuleMSE valueMSE Δ\Delta%MAE valueMAE Δ\Delta%Time (s)
Gradient proxyw/o output-cache proxy20.226+1.40.308+1.6925.66
Layer selectionw/o layer selection0.223+0.20.303+0.11126.85
Variable poolingw/o conflict pooling0.222−0.70.302−0.4984.73
w/o anchor pooling0.228+2.20.308+1.61077.18
w/o variable pooling0.223−0.00.304+0.3968.47
Direction surgeryw/o magnitude decoupling0.222−0.50.303−0.11105.34
w/o safe candidate selection0.221−0.90.301−0.61000.83
PV-Surgery (full)0.223–0.303–938.45

Table 15. Core component ablation on ETTm2, averaged over 20 matched backbone–prediction-length settings. Δ\Delta is the relative change against the full configuration. Bold indicates the best score and underlining the second best using unrounded averages.

CategoryModuleMSE valueMSE Δ\Delta%MAE valueMAE Δ\Delta%Time (s)
Gradient proxyw/o output-cache proxy20.301+2.80.327+1.81138.48
Layer selectionw/o layer selection0.291−0.60.321−0.41887.39
Variable poolingw/o conflict pooling0.298+1.60.326+1.31212.13
w/o anchor pooling0.307+4.80.332+3.11094.43
w/o variable pooling0.308+4.90.332+3.1993.80
Direction surgeryw/o magnitude decoupling0.301+2.70.326+1.31103.28
w/o safe candidate selection0.292−0.40.321−0.21132.23
PV-Surgery (full)0.293–0.322–1286.21

Table 16. Core component ablation on Weather, averaged over 20 matched backbone–prediction-length settings. Δ\Delta is the relative change against the full configuration. Bold indicates the best score and underlining the second best using unrounded averages.

CategoryModuleMSEMAETime (s)
valueΔ\Delta%valueΔ\Delta%
Gradient proxyw/o output-cache proxya^{a}0.528+22.60.486+7.8106.10
Layer selectionw/o layer selection0.461+7.00.461+2.2172.68
Variable poolingw/o conflict pooling0.516+19.80.484+7.297.14
w/o anchor pooling0.436+1.20.452+0.1101.68
w/o variable pooling0.434+0.70.450−0.2101.73
Direction surgeryw/o magnitude decoupling0.448+3.90.461+2.2113.14
w/o safe candidate selection0.440+2.10.457+1.3106.53
PV-Surgery (full)0.431−<u>0.451</u>−115.19

Table 17. Core component ablation on Exchange, averaged over 20 matched backbone–prediction-length settings. Δ\Delta is the relative change against the full configuration. Bold indicates the best score and underlining the second best using unrounded averages.

Multi-Task Gradient Operators

Multi-task optimization offers several ways to combine gradients that disagree across objectives. We apply five operators discussed in Section 2 to the variable-wise proxies constructed by PV-Surgery. These methods normally act on a small set of task gradients exposed directly by a multi-task model, while a forecaster combines DD variable objectives into one scalar loss. This comparison asks whether native multi-task operators can use the reconstructed variable-wise rows without PV-specific pooling or rescaling. Its purpose is to test transferability rather than establish a definitive ranking.

Native-operator protocol. Each native operator receives the same unpooled variable-wise gradient proxies on the selected subspace. We hold output-cache reconstruction, effective-coverage layer selection, selected-layer concatenation, and full-sum assembly fixed. Pooling, candidate selection, magnitude restoration, and PV-specific wrappers or rescaling are disabled, so each operator retains its native aggregation rule. Table 19 reports the 28 matched MICN settings from seven datasets and four prediction lengths using seed 42. Standard MSE training and the unchanged full PV-Surgery configuration provide the two reference rows.

DatasetETTh1ETTh1ETTh1ETTh1ETTh2ETTh2ETTh2ETTh2ETTm1ETTm1ETTm1ETTm1ETTm2ETTm2ETTm2ETTm2WeatherWeatherWeatherWeatherExchangeExchangeExchangeExchangeILIILIILIILI
Forecast length96192336720961923367209619233672096192336720961923367209619233672024364860
MSE MSE0.5520.5940.6880.7530.2380.3150.4520.6720.4450.4950.5670.6270.1660.2020.2660.3890.3980.4030.3800.4960.1230.2190.4982.9313.1692.9963.2693.273
MAE0.5300.5530.6230.6710.3330.3880.4720.5830.4720.5110.5700.5960.2760.3030.3550.4340.4420.4470.4220.5010.2620.3520.5251.4001.2371.2281.2841.267
MGDA MSE0.4970.6410.7180.7900.2360.2950.3490.4360.4470.4730.4950.5510.1510.1890.2370.3200.3550.3750.3850.5550.1220.2260.4010.8573.4542.9913.2243.394
MAE0.4960.5990.6410.6970.3240.3700.4130.4750.4720.4870.4940.5380.2530.2800.3150.3830.3980.4240.4260.5390.2610.3610.4910.7431.3001.2151.2711.286
PCGrad MSE0.4970.5590.7700.7730.2300.2910.3650.6030.4270.4750.5170.5990.1470.1930.2370.2950.3120.4460.4300.5230.1080.2170.3861.1433.4832.9833.2343.324
MAE0.5020.5420.6710.6860.3210.3660.4240.5620.4540.4890.5220.5820.2510.2910.3240.3610.3780.4710.4590.5250.2420.3520.4750.8221.3061.2211.2741.271
GradVac MSE0.4950.5560.7670.7280.2330.2960.3690.6000.4340.4680.5130.5870.1480.1950.2330.2960.3040.4220.4240.5120.1080.2150.3851.1423.4772.9803.2323.321
MAE0.5000.5400.6700.6560.3250.3710.4270.5610.4620.4850.5190.5730.2530.2900.3190.3610.3700.4490.4580.5180.2410.3500.4790.8141.3051.2201.2731.272
CAGrad MSE0.5100.5550.7700.7690.2340.3180.3860.6470.4190.4840.5150.6040.1510.2090.2430.3030.3410.4470.4370.5120.1150.2200.4271.3993.2812.9853.2333.185
MAE0.5060.5390.6710.6820.3250.3920.4510.5870.4480.4980.5210.5880.2560.3050.3290.3720.4020.4700.4640.5170.2510.3540.4910.8701.2611.2191.2741.240
Aligned-MTL MSE0.4820.5870.7210.8100.2320.2950.3600.6010.4170.4470.4960.5780.1480.1890.2280.3930.4010.3780.4640.5130.1050.2450.4362.7293.4302.9843.2513.429
MAE0.4870.5660.6450.7080.3210.3740.4180.5630.4480.4530.4930.5580.2500.2830.3100.4450.4280.4630.4840.5180.2360.3780.5011.3671.2931.2161.2781.299
PV-Surgery (Ours) MSE0.4920.5610.7900.7980.2300.2910.3520.5130.4300.4700.5080.5760.1460.1900.2310.3310.3180.4350.4360.4790.1090.2120.3781.1063.1902.9853.2263.364
MAE0.4970.5430.6300.7000.3200.3690.4120.5200.4600.4870.5130.5620.2480.2850.3160.3890.3830.4600.4680.4930.2420.3480.4700.7931.2431.2201.2721.283

Table 19. Native multi-task gradient operators on MICN over seven benchmarks and four prediction lengths. Each native row receives the same unpooled variable-wise proxies on the selected subspace and retains its own aggregation rule. PV-Surgery uses the unchanged full configuration. The MSE row denotes standard MSE training. Best results are bolded and second-best results are underlined using unrounded scores.

The operators. MGDA treats the objectives as a multi-objective problem and takes the minimum-norm point of the convex hull of their gradients, min⁡α∈Δ∥∑iαigi∥22\min_{\alpha\in\Delta} \left\lVert\sum_i \alpha_i g_i \right\rVert_2^2, which descends on every objective at once unless that minimum is zero, in which case the current point is already Pareto stationary [?]. PCGrad works pair by pair. For each objective, it compares its gradient with the others in random order and, whenever the cosine is negative, projects the gradient onto the normal plane of the conflicting one, leaving non-conflicting pairs untouched [22]. GradVac generalizes PCGrad’s rule from “repair negative cosines” to “reach a target cosine”, tracking an exponential moving average of each pair’s observed similarity and using it as the target, so PCGrad is the special case of a zero target applied only under conflict [20]. CAGrad maximizes the worst-case objective improvement inside a ball of radius c∥gˉ∥c\lVert\bar{g}\rVert around the average gradient gˉ=D−1∑igi\bar{g} = D^{-1}\sum_i g_i, which keeps the update anchored to the mean-loss direction, recovers plain descent at c=0c = 0, and approaches MGDA as c→∞c \to\infty [9]. We use c=0.5c = 0.5. Aligned-MTL reads the condition number of the gradient matrix as a joint measure of conflict and dominance, eigendecomposes the objective-by-objective Gram matrix, and rescales its singular values to the smallest one, leaving principal components orthogonal and equal in magnitude [16].

Native-operator settings. PCGrad and GradVac visit peers in random order, and GradVac uses an exponential-moving-average coefficient of 0.01. MGDA runs at most 250 minimum-norm iterations with tolerance 10−510^{-5}. CAGrad uses c=0.5c = 0.5, rescale rate 1, and SLSQP with at most 250 iterations and tolerance 10−1010^{-10}. Aligned-MTL uses the smallest positive singular value and has no tuned hyperparameter.

The operators differ in both direction and aggregation scale. PCGrad and GradVac sum their corrected rows, MGDA keeps its minimum-norm convex-combination scale, CAGrad uses its mean-anchored native rescaling, and Aligned-MTL uses its spectral scale. We preserve these native rules, so this is a method-level comparison rather than a direction-only ablation. Our common-direction operator restores the norm of each surgery input after changing its direction. The inputs are variable rows in the unpooled branch and pool rows in the pooled branch, so this restoration does not preserve every individual variable-row norm after pooling.

All five native operators reduce MSE on average relative to standard training over the 28 settings. The mean per-setting reductions are 7.59% for MGDA, 6.87% for PCGrad, 7.55% for GradVac, 4.82% for CAGrad, and 3.51% for Aligned-MTL. The unchanged PV-Surgery configuration gives the largest average reduction at 8.50%. The native operators outperform standard training in 19, 21, 22, 19, and 19 settings, respectively, but each also outperforms PV-Surgery in between 5 and 11 settings. The comparison therefore does not establish a uniform ranking. Instead, the average gains of all five operators show that the reconstructed variable-wise proxies provide a meaningful training signal that methods designed for explicit task gradients can use, while the per-setting differences show that the way this signal is aggregated still affects the outcome.

Mechanism Diagnostics

Test error alone does not show whether the internal steps of PV-Surgery behave as intended. We therefore examine the variable-wise gradient proxies reconstructed from cached layer signals and the optimizer update assembled from these proxies. The first analysis measures proxy fidelity by comparing the reconstructed proxy rows with exact gradients obtained by separate per-variable backward passes. The second measures pairwise conflict and descent violations before and after surgery. A descent violation occurs when the update passed to the optimizer has a negative inner product with a reconstructed variable gradient, indicating that a small step along the negative update would increase that variable’s loss to first order. A separate test-set analysis then asks whether PV-Surgery narrows the oracle gaps observed under standard shared training.

Measurement protocol. We run two diagnostic versions of the 140 settings in Table 1. Both use the full PV-Surgery configuration and the same training protocol as the reported runs. They enable different measurements at fixed intervals, but the diagnostic results are never used to form the optimizer update. For proxy fidelity, which measures directional agreement between a proxy and its exact gradient, we compute exact per-variable gradients every twenty optimizer steps and compare them with the reconstructed proxy rows using cosine similarity. These additional backward passes provide exact reference gradients only for measurement, while training continues to use the proxy-based update. We report fidelity at the output boundary, over all hooked parameters ΩH\Omega_{\mathcal{H}}, and on the selected subspace ΩS\Omega_{\mathcal{S}}. A separate set of runs records conflict mass, the average negative-magnitude across reconstructed variable pairs, and descent violations every fifty steps.

At 12,256 of the 45,897 logged steps, the selected-slice proxy rows have zero norm and the conditional gate skips the intervention. Cosine similarity is undefined for these zero vectors. Our primary selected-subspace statistic is therefore conditional on a nonzero selected-slice proxy. Consequently, five runs have no defined selected-subspace cosine after conditioning, leaving 135 runs for the primary statistic. If the zero entries are instead assigned a cosine of zero, the selected-subspace average is 0.739 over all 140 runs. The output-boundary and all-parameter statistics use all runs without this conditioning.

Proxy fidelity. The average cosine with the exact gradient is 0.990 at the output boundary, 0.936 on the selected subspace when its proxy is nonzero, and 0.698 over all parameters. Figure 15 shows the same conditional comparison for each benchmark. Output-boundary fidelity ranges from 0.981 to 0.996. The gap is larger inside the backbone, where variable-mixing operations make the cache an approximation. Weather has the lowest all-parameter fidelity at 0.508, which rises to 0.862 after layer selection. ETTh1 rises from 0.806 to 0.977. The conditional selected-subspace value exceeds the all-parameter value on every benchmark. Layer selection therefore concentrates active interventions on parameters for which the reconstructed gradients are more faithful, although the selected rows are not exact in every model.

Figure 15. Cosine similarity between reconstructed proxy rows and exact per-variable gradients for each benchmark. Each bar first averages logged steps within a run and then averages the available runs for that benchmark. Each benchmark contributes 20 runs from five backbones and four prediction lengths. Output-boundary and all-parameter bars use all 20 runs. Selected-subspace bars use 17 Exchange runs, 18 ILI runs, and all 20 runs elsewhere after conditioning on nonzero proxy entries. The smaller counts occur when every logged selected-slice proxy in a run has zero norm, leaving no defined selected-subspace cosine.

Effect on the reconstructed gradients. Figure 16 compares the sum-loss update with the final update in each of the 140 mechanism runs. Conflict mass is the mean of max⁡(−cos⁡ij,0)\max(-\cos_{ij}, 0) over all variable pairs, so a cosine of −0.4-0.4 contributes 0.4 while a nonnegative cosine contributes zero. A larger conflict mass therefore indicates stronger overall directional opposition among the reconstructed variable gradients. The descent violation ratio is the fraction of reconstructed variable gradients that have a negative inner product with the update. A ratio of 0.2, for example, means that a small step along the negative update would increase the losses of two out of ten reconstructed variable objectives to first order. These are optimization diagnostics defined on the reconstructed rows. They should not be read as direct measurements of test-set harm.

Figure 16. Descent violations and conflict mass before and after surgery over the 140 matched settings. (a) shows the descent-violation ratio, and (b) shows conflict mass. Each point is one run, with the horizontal and vertical axes giving the values before and after surgery. Points below the diagonal indicate a reduction.

Mean conflict mass falls from 0.075 to 0.009. It decreases in 129 runs, remains unchanged in 11, and increases in none. This one-sided shift directly verifies the intended operation of PV-Surgery. The mean descent violation ratio falls from 0.097 to 0.049. It decreases in 103 runs, remains unchanged in 29, and increases in 8. Descent violations are a stricter downstream test because they also depend on how the corrected rows combine into the final update. Their mean is nearly halved and they decrease in most runs, showing that conflict correction usually carries through to an update that is better aligned with the reconstructed variable objectives.

Test-set oracle gap. Figure 17 (a) returns to the oracle gap from Section 3. The comparison contains 57 iTransformer variables from the six benchmarks with matched baseline, PV-Surgery, and full-input single-target oracle runs. Shared training harms 29 of these variables. PV-Surgery reduces the gap for 23 and closes it for 12. Among the remaining 28 variables, whose baseline gap is not positive, PV-Surgery increases test MSE for 10 and creates a positive oracle gap for 3. Across all 57 variables, the mean gap decreases by 0.0040 MSE. The aggregate improvement therefore includes both repaired gaps and a smaller number of regressions.

Bar charts showing the effect on the Section 3 oracle gap and frozen variable-pair subsets

Figure 17. Test-set evidence for the effect of PV-Surgery. (a) reports changes in the oracle gap. (b) reports PV-Surgery MSE minus baseline MSE on fixed variable-pair subsets. Positive values in (b) favor the baseline.

Frozen variable-pair subsets. Figure 17 (b) tests a stricter hypothesis using separately trained fixed variable-pair subsets. PV-Surgery improves test MSE on only 5 of 22 conflicting pairs, compared with 16 of 28 aligned pairs. Its mean MSE change relative to the baseline is also unfavorable for both groups, at +0.0085 for conflicting pairs and +0.0021 for aligned pairs. Persistent pairwise conflict is therefore not a reliable marker of the subsets on which surgery will help. This negative result agrees with Section 3, where conflict identifies disagreement but not harm under shared training.

Qualitative Case Studies

Aggregate MSE does not show how two forecasts differ within a window. We therefore inspect four settings with prediction length 96, namely MICN on ETTm2, iTransformer on ETTm2, MICN on ETTh1, and DLinear on ETTh2. Together they cover three backbones and two ETT benchmarks. Each figure shows the six load variables HUFL, HULL, MUFL, MULL, LUFL, and LULL together with oil temperature OT.

Protocol. For each setting, the baseline and PV-Surgery runs use the same data split and test windows. Their saved ground-truth arrays are identical. Since the variables span different vertical ranges, we summarize the error in each panel using a range-normalized mean absolute error. For window nn and variable dd, let y^MSE\hat{y}^{\mathrm{MSE}} and y^PV\hat{y}^{\mathrm{PV}} denote the forecasts from standard MSE training and PV-Surgery. For either forecast y^\hat{y}, the panel distance is

δn,d(y^)=1H∑h=1H∣y^n,h,d−yn,h,d∣max⁡{y,y^MSE,y^PV}n,⋅,d−min⁡{y,y^MSE,y^PV}n,⋅,d.(18)\delta_{n,d}(\hat{y}) = \frac{\frac{1}{H}\sum_{h=1}^{H}\left|\hat{y}_{n,h,d}-y_{n,h,d}\right|}{\max\{y,\hat{y}^{\mathrm{MSE}},\hat{y}^{\mathrm{PV}}\}_{n,\cdot,d}-\min\{y,\hat{y}^{\mathrm{MSE}},\hat{y}^{\mathrm{PV}}\}_{n,\cdot,d}}. \tag*{(18)}

The denominator is the range spanned by the ground truth and both forecasts within the same panel. A value of 0.10 means that the average absolute error is one tenth of this range. The plotted vertical axes retain the standardized target and forecast values. We use δn,d\delta_{n,d} only as a scale-normalized scalar summary to compare the two forecasts within the same window and variable. The smaller of δn,d(y^MSE)\delta_{n,d}(\hat{y}^{\mathrm{MSE}}) and δn,d(y^PV)\delta_{n,d}(\hat{y}^{\mathrm{PV}}) identifies the forecast with the lower average vertical error relative to their common displayed range. We also report the cosine between the predicted and true trajectories after subtracting their respective horizon means. This direction cosine measures temporal shape independently of the mean level. A negative value indicates that the predicted trajectory moves in the opposite direction from the target.

Window selection. We manually select one representative example from each setting in which the effect of PV-Surgery is clearly visible. Together, the examples illustrate level correction, reduced drift, and cases in which panel distance improves without a higher direction cosine. They are descriptive and do not enter any aggregate estimate.

MICN on ETTm2. Across the full test set, PV-Surgery lowers MSE by 12.52% and raises the mean direction cosine from 0.422 to 0.510. Window-level MSE improves in 19,377 of 23,409 windows. The number of variables with negative direction cosine decreases in 7,726 windows and increases in 2,534. In Figure 18, the baseline forecast is generally shifted above the ground truth, while PV-Surgery reduces this offset. The panel distance falls from 0.25 to 0.13 for HUFL, from 0.24 to 0.10 for HULL, and from 0.35 to 0.18 for MULL. HULL and MULL also change from negative to positive direction cosine. OT has the smallest corrected distance at 0.08, and its direction cosine rises to 0.97.

Figure 18. MICN on ETTm2. Panel titles report the direction cosine for standard MSE training and PV-Surgery. The final panel collects the same values.

iTransformer on ETTm2. On the same dataset, iTransformer obtains a smaller MSE reduction of 4.61%. Window-level MSE improves in 15,752 of 23,409 windows. Negative direction cosines become less frequent in 5,020 windows and more frequent in 4,671. Figure 19 again shows a substantial level correction. The panel distance falls from 0.28 to 0.11 for HUFL, from 0.32 to 0.14 for MUFL, and from 0.17 to 0.03 for HULL. The improvement is small for LUFL and LULL. MULL provides a useful counterexample. Its distance decreases from 0.31 to 0.16, while its direction cosine decreases from 0.61 to 0.55. A forecast can therefore move closer in level without matching the temporal shape more closely.

Figure 19. iTransformer on ETTm2. Panel titles report the direction cosine for standard MSE training and PV-Surgery. The final panel collects the same values.

MICN on ETTh1. PV-Surgery lowers MSE by 10.85% for MICN on ETTh1. It improves window-level MSE in 4,188 of 5,709 windows. The number of negative direction cosines decreases in 1,156 windows and increases in 221. In Figure 20, the baseline tends to overshoot the target and PV-Surgery moves the forecasts downward. The largest distance reductions occur for LULL, from 0.37 to 0.21, and LUFL, from 0.35 to 0.24. HUFL and MUFL retain direction cosines near 0.92. Direction does not improve for every variable. MULL falls from 0.30 to 0.27, and LULL becomes more negative, even though both panel distances decrease.

Figure 20. MICN on ETTh1. Panel titles report the direction cosine for standard MSE training and PV-Surgery. The final panel collects the same values.

DLinear on ETTh2. PV-Surgery lowers MSE by 4.55% in the DLinear setting. Window-level MSE decreases in 4,174 of 5,709 windows. Negative direction cosines become less frequent in 1,073 windows and more frequent in 264. The clearest change in Figure 21 occurs for HULL. The baseline drifts away from an almost flat target, while the PV-Surgery forecast remains closer throughout the horizon. Its panel distance falls from 0.45 to 0.15. MULL shows a smaller reduction from 0.28 to 0.18. LUFL and OT again show that the two diagnostics need not agree. Their panel distances decrease, but their direction cosines fall from 0.66 to 0.59 and from 0.53 to 0.45.

Figure 21. DLinear on ETTh2. Panel titles report the direction cosine for standard MSE training and PV-Surgery. The final panel collects the same values.

Across these four settings, the mean direction cosine changes by 0.088, 0.034, 0.041, and 0.028. The selected windows show larger changes in level and amplitude than in temporal shape. They also show why the mechanism results in Appendix J should not be interpreted as a guarantee for every variable. PV-Surgery can reduce aggregate MSE and pairwise conflict while the direction cosine of an individual variable remains unchanged or becomes worse. These examples add a window-level view of the aggregate results and show that lower error does not always come with a higher direction cosine.

Computational Cost and Training Time

PV-Surgery leaves the forecasting architecture unchanged but requires additional computation at each training step. We describe this additional computation and measure its wall-clock cost over the main experimental grid.

Analytical cost. Like standard MSE training, PV-Surgery uses one backward pass for each optimizer step. Its additional cost comes from reconstructing and processing the variable-wise gradient rows. Let NHN_{\mathcal{H}} denote the number of parameters in the hooked layers and NSN_{\mathcal{S}} the number in the selected layers. Dense zero-extended proxy storage and later row-wise processing scale as O(DNH)O(DN_{\mathcal{H}}). Computing the observed weight outer products costs O(∑ℓ∈HrunBDMℓFin,ℓFout,ℓ)O(\sum_{\ell\in\mathcal{H}_{\mathrm{run}}}BDM_{\ell}F_{\mathrm{in},\ell}F_{\mathrm{out},\ell}), where Fin,ℓF_{\mathrm{in},\ell} and Fout,ℓF_{\mathrm{out},\ell} are the layer widths and the bias term is lower order. Pairwise cosine similarities on the selected subspace cost O(D2NS)O(D^2N_{\mathcal{S}}), and evaluating the unpooled candidate plus a pooled candidate with PP pools costs O((D+P)NS)O((D+P)N_{\mathcal{S}}).

Layer selection cannot reduce the cache-construction term because the cache is needed to score the layers. It does reduce the pairwise and candidate computations because NSN_{\mathcal{S}} is no larger than NHN_{\mathcal{H}}. Candidate selection adds another direction evaluation when pooling is active. These operations affect training only. The forward pass used at inference is identical to that of the original backbone.

Protocol. We pair each PV-Surgery run with the standard MSE run from the same backbone, benchmark, and prediction length. All 140 pairs use seed 42, the same batch size, and the same GPU type. Early stopping gives different training lengths, so total time alone mixes computational overhead with the number of completed epochs. We divide each run’s logged training time by its completed epoch count and form the PV-Surgery to MSE ratio within each pair. The baseline completes 22.1 epochs on average and PV-Surgery completes 23.3. These timings cover complete training epochs, including data loading, model computation, optimizer steps, and logging. They therefore measure practical training time rather than the isolated cost of the PV-Surgery operator.

Measured overhead. Table 20 reports the paired results. Across all settings, the per-epoch ratio has a median of 1.54 and a mean of 1.71. PV-Surgery is slower in 118 of the 140 pairs. Summing the logged time over the grid gives 12.84 hours for standard MSE training and 20.56 hours for PV-Surgery, a total-time ratio of 1.60. On the six benchmarks with prediction lengths 96, 192, 336, and 720, the median per-epoch ratio remains between 1.51 and 1.54. We do not observe a systematic increase with forecast length in this grid.

BackboneEpochsSeconds / epochPer-epoch ratioSlower
MSE+PVMSE+PVmedianmean
DLinear27.223.46.25.50.881.156/28
MICN17.617.846.162.11.351.3728/28
SCINet28.527.718.728.51.521.5128/28
iTransformer15.625.46.415.32.092.2928/28
TimeXer21.822.36.614.82.222.2428/28
All22.123.316.825.31.541.71118/140

Table 20. Training cost of PV-Surgery against standard MSE training over the 140 matched settings of Table 1. Epochs and seconds per epoch are means over the 28 settings of each backbone. The ratio columns are computed per setting and then aggregated, so they are not the quotient of the two preceding columns.

The overhead varies more across backbones. The median ratio is 2.09 for iTransformer and 2.22 for TimeXer, both of which expose several cache-compatible linear layers. MICN has a lower median ratio of 1.35, although its baseline epoch is much slower in absolute time. DLinear has a median ratio below one at 0.88, but its mean ratio is 1.15 and six of its twenty-eight settings are slower with PV-Surgery. The DLinear result should therefore be read as low overhead with substantial timing variation, not as evidence that surgery generally accelerates training.

Implementation considerations. The measured ratios depend on the hardware and current implementation and should not be treated as architecture-independent constants. Caching a separate variable axis and, on some execution paths, materializing dense D×NHD \times N_H proxy rows increase memory traffic, while evaluating both unpooled and pooled candidates adds computation. Compact slice-wise assembly and reuse of intermediate quantities could reduce this overhead. Reducing the number of cached layers offers another cost-coverage trade-off. PV-Surgery changes only training-time gradient construction, so inference remains unchanged.

References

  1. [2]Yisong Fu, Zhezhi Shao, Chengqing Yu, Yujie Li, Zhulin An, Qi (Cheems) Wang, Yongjun Xu, and Fei Wang. Selective learning for deep time series forecasting. In Danielle Belgrave, Cheng Zhang, Laura N. Montoya, Hsuan-Tien Lin, Razvan Pascanu, Piotr Koniusz, Marzieh Ghassemi, Nancy Chen, Iván Vladimir Meza Ruíz, and Arturo Loaiza-Bonilla (eds.), Advances in Neural Information Processing Systems 38: Annual Conference on Neural Information Processing Systems 2025, NeurIPS 2025, San Diego, CA, USA, December 2-7, 2025 / Mexico City, Mexico, November 30 - December 5, 2025, 2025. URL http://papers.nips.cc/paper_files/paper/2025/hash/8cf54ff53f44835b9bdabc54a61aca6d-Abstract-Conference.html.
  2. [4]Mark O Hill. Diversity and evenness: a unifying notation and its consequences. Ecology, 54(2): 427–432, 1973.DOI
  3. [5]Dilfira Kudrat, Zongxia Xie, Yanru Sun, Tianyu Jia, and Qinghua Hu. Patch-wise structural loss for time series forecasting. In Aarti Singh, Maryam Fazel, Daniel Hsu, Simon Lacoste-Julien, Felix Berkenkamp, Tegan Maharaj, Kiri Wagstaff, and Jerry Zhu (eds.), Forty-second International Conference on Machine Learning, ICML 2025, Vancouver, BC, Canada, July 13-19, 2025, volume 267 of Proceedings of Machine Learning Research. PMLR / OpenReview.net, 2025. URL https://proceedings.mlr.press/v267/kudrat25a.html.arxiv.org/abs/2503.00877
  4. [6]Guokun Lai, Wei-Cheng Chang, Yiming Yang, and Hanxiao Liu. Modeling long- and short-term temporal patterns with deep neural networks. In Kevyn Collins-Thompson, Qiaozhu Mei, Brian D. Davison, Yiqun Liu, and Emine Yilmaz (eds.), The 41st International ACM SIGIR Conference on Research & Development in Information Retrieval, SIGIR 2018, Ann Arbor, MI, USA, July 08-12, 2018, pp. 95–104. ACM, 2018. doi: 10.1145/3209978.3210006. URL https://doi.org/10.1145/3209978.3210006.
  5. [7]Hyunwook Lee, Chunggi Lee, Hongkyu Lim, and Sungahn Ko. TILDE-Q: A transformation invariant loss function for time-series forecasting. CoRR, abs/2210.15050, 2022. doi: 10.48550/ARXIV.2210.15050. URL https://doi.org/10.48550/arXiv.2210.15050.
  6. [9]Bo Liu, Xingchao Liu, Xiaojie Jin, Peter Stone, and Qiang Liu. Conflict-averse gradient descent for multi-task learning. In Marc’Aurelio Ranzato, Alina Beygelzimer, Yann N. Dauphin, Percy Liang, and Jennifer Wortman Vaughan (eds.), Advances in Neural Information Processing Systems 34: Annual Conference on Neural Information Processing Systems 2021, NeurIPS 2021, December 6-14, 2021, virtual, pp. 18878–18890, 2021. URL https://proceedings.neurips.cc/paper/2021/hash/9d27fd2477efbf837d73ef7ae23db9-Abstract.html.arxiv.org/abs/2110.14048
  7. [10]Minhao Liu, Ailing Zeng, Muxi Chen, Zhijian Xu, Oiuixia Lai, Lingna Ma, and Qiang Xu. Scinet: Time series modeling and forecasting with sample convolution and interaction. In Sanmi Koyejo, S. Mohamed, A. Agarwal, Danielle Belgrave, K. Cho, and A. Oh (eds.), Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022, NeurIPS 2022, New Orleans, LA, USA, November 28 - December 9. 2022, 2022. URL http://papers.nips.cc/paper_files/paper/2022/hash/266983d0949aed78a16fa4782237dea7-Abstract-Conference.html.
  8. [11]Yong Liu, Tengge Hu, Haoran Zhang, Haixu Wu, Shiyu Wang, Lintao Ma, and Mingsheng Long. itransformer: Inverted transformers are effective for time series forecasting. In The Twelfth International Conference on Learning Representations, ICLR 2024, Vienna, Austria, May 7-11, 2024. OpenReview.net, 2024. URL https://openreview.net/forum?id=JePfAI8fah.
  9. [13]Yuqi Nie, Nam H. Nguyen, Phanwadee Sinthong, and Jayant Kalagnanam. A time series is worth 64 words: Long-term forecasting with transformers. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023. URL https://openreview.net/forum?id=Jbdc0vTOcoL.arxiv.org/abs/2211.14730
  10. [16]Dmitry Senushkin, Nikolay Patakin, Arseny Kuznetsov, and Anton Konushin. Independent component alignment for multi-task learning. In IEEE/CVF Conference on Computer Vision and Pattern Recognition, CVPR 2023, Vancouver, BC, Canada, June 17-24, 2023, pp. 20083–2023. IEEE, 2023. doi: 10.1109/CVPR52729.2023.01923. URL https://doi.org/10.1109/CVPR52729.2023.01923.
  11. [17]Hao Wang, Lichen Pan, Yuan Shen, Zhichao Chen, Degui Yang, Yifei Yang, Sen Zhang, Xinggao Liu, Haoxuan Li, and Dacheng Tao. Fredf: Learning to forecast in the frequency domain. In The Thirteenth International Conference on Learning Representations, ICLR 2025, Singapore, April 24-28, 2025. OpenReview.net, 2025. URL https://openreview.net/forum?id=4A9D1Sa1ul.
  12. [18]Huiqiang Wang, Jian Peng, Feihu Huang, Jince Wang, Junhui Chen, and Yifei Xiao. MICN: multi-scale local and global context modeling for long-term series forecasting. In The Eleventh International Conference on Learning Representations, ICLR 2023, Kigali, Rwanda, May 1-5, 2023. OpenReview.net, 2023. URL https://openreview.net/forum?id=zt53IDUR1U.
  13. [19]Yuxuan Wang, Haixu Wu, Jiaxiang Dong, Guo Qin, Haoran Zhang, Yong Liu, Yunzhong Qiu, Jianmin Wang, and Mingsheng Long. Timexer: Empowering transformers for time series forecasting with exogenous variables. In Amir Globerson, Lester Mackey, Danielle Belgrave, Angela Fan, Ulrich Paquet, Jakub M. Tomczak, and Cheng Zhang (eds.), Advances in Neural Information Processing Systems 37: Annual Conference on Neural Information Processing Systems 2024, NeurIPS 2024, Vancouver, BC, Canada, December 10 - 15, 2024, 2024. URL https://papers.nips.cc/paper_files/paper/2024/hash/0113ef4642264adc2e6924a3cbdf532-Abstract-Conference.html.arxiv.org/abs/2402.19072
  14. [20]Zirui Wang, Yulia Tsvetkov, Orhan Firat, and Yuan Cao. Gradient vaccine: Investigating and improving multi-task optimization in massively multilingual models. In 9th International Conference on Learning Representations, ICLR 2021, Virtual Event, Austria, May 3-7, 2021. OpenReview.net, 2021. URL https://openreview.net/forum?id=F1vE jWK-1H_.
  15. [22]Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn. Gradient surgery for multi-task learning. In Hugo Larochelle, Marc’Aurelio Ranzato, Raia Hadsell, Maria-Florina Balcan, and Hsuan-Tien Lin (eds.), Advances in Neural Information Processing Systems 33: Annual Conference on Neural Information Processing Systems 2020, NeurIPS 2020, December 6-12, 2020, virtual, 2020. URL https://proceedings.neurips.cc/paper/2020/hash/3fe78a8acf5fda99de95303940a2420c-Abstract.html.
  16. [23]Ailing Zeng, Muxi Chen, Lei Zhang, and Qiang Xu. Are transformers effective for time series forecasting? In Brian Williams, Yiling Chen, and Jennifer Neville (eds.), Thirty-Seventh AAAI Conference on Artificial Intelligence, AAAI 2023, Thirty-Fifth Conference on Innovative Applications of Artificial Intelligence, IAAI 2023, Thirteenth Symposium on Educational Advances in Artificial Intelligence, EAAI 2023, Washington, DC, USA, February 7-14, 2023, pp. 11121–11128. AAAI Press, 2023. doi: 10.1609/AAAI.V37I9.26317. URL https://doi.org/10.1609/aaai.v37i9.26317.

Paper details

vlnc:W_djgp82pq
Version
v1 · latest
Published
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents