WorksComputer Science
Perceive, Solve, Reason: Progressive Capability Formation in Time Series-LLMs
Abstract
Large language models (LLMs) can now perceive, describe, and reason about images and audio, yet time series understanding remains underestimated: most work uses LLMs as forecasters and classifiers, reasoning studies are few and narrow, and no suite trains and evaluates the full range of capabilities. In this paper, we build the first such suite, with 337K training and 44K evaluation samples across three progressive capabilities, Perceive, Solve, and Reason, and use it to ask how time series should enter an LLM that must master all three, and in what order. We find that a dedicated encoder is essential: with one, a 4B model surpasses GPT-4o on every perception attribute we test, whereas textualizing the series costs 8–10 points in reasoning accuracy. The capabilities then build on one another: learning them in order beats mixing the same data and rivals reasoning-only training with over three times the reasoning data, but only when an encoder is used. Analyses further reveal that capabilities are transferred through distinct network depths rather than simply stacked, and that a stage converged on its own benchmark is not necessarily ready for the next. Distilled into a simple recipe, these findings yield 4B TS-LLMs that, without any target-domain data, outperform four recent 7B TS-LLMs on seven of eight out-of-domain metrics, by up to 11 points. We therefore advocate treating time series as a first-class modality for LLMs, read through dedicated encoders and taught to perceive before they reason.
Introduction
Time series record how the world evolves, from power grids and patient vitals to markets and traffic, and reading them in context underlies many critical decisions [10]. In vision and audio, large language models (LLMs) have become general assistants that perceive, describe, and reason about their inputs, built on training and evaluation suites that span these capabilities [6, 18, 20, 21, 28]. Time series understanding, by contrast, remains underestimated. Most work on time series LLMs (TS-LLMs) applies them to traditional tasks such as forecasting, classification, and anomaly detection [11, 14, 16, 38], reducing understanding to curve fitting [26]. Studies of reasoning are few and narrow, often stopping at surface-level question answering [7, 31], and benchmarks for generalist models largely overlook time series [34]. Above all, no off-the-shelf suite trains and evaluates the full range of time series capabilities.
We view comprehensive time series understanding as three progressive capabilities: perceiving basic attributes such as trend, seasonality, and local patterns; solving standard tasks such as forecasting and anomaly detection; and reasoning about a series and its context to explain, extrapolate, and decide. Each capability has been studied on its own [2, 9, 13, 17, 22, 30, 37], and so has the way time series enter the LLM: as text, as plotted images, or through a dedicated encoder with different alignment designs [8, 19, 26, 31, 32]. These comparisons, however, are each made in a separate setting with its own tasks and data. When one model must acquire all three capabilities, a question arises that isolated settings cannot answer: how should time series enter the LLM, and in what order should the capabilities be learned? The answer hinges on whether what is learned for one capability carries over to the next.
To answer this question, we build, to our knowledge, the first suite that trains and evaluates all three capabilities. It curates four public corpora, covering synthetic series and real-world series from over ten domains [13, 26, 27, 31]. The data are organized into three stages, Perceive, Solve, and Reason, each with its own training set and held-out evaluation set, totaling 337K training and 44K evaluation samples. Three external benchmarks are reserved for out-of-domain testing [2, 4, 37]. On this suite, we compare textualization, which writes the series as text, with two encoder-based designs, pre-alignment and post-alignment, which align encoder outputs before or within the LLM [13, 26]. For each design, we contrast progressive training, which learns the three stages in order, with mixed training on all the data at once and with reasoning-only training. We further examine which perceptual attributes matter, how gradients evolve under each training scheme, and how representations transfer across layers.
In this setting, a dedicated encoder proves essential. Textualization trails both encoder-based designs by 8–10 points in reasoning accuracy, although it fine-tunes the LLM while pre-alignment keeps the LLM frozen. Borrowed formats miss fine-grained structure. Even GPT-4o, reading a series as text or as a plot, scores about half as high as our perception-trained 4B models on local patterns and inter-variable relations. Ablating the perception data shows that these are exactly the attributes reasoning relies on most. Between the two alignment designs, the lighter pre-alignment performs on par with post-alignment. Its encoder can even be frozen after perception with little loss, so later capabilities form mainly by re-aligning fixed temporal features rather than by refining the encoder.
Given such an encoder, we reach four findings on how the capabilities form. (1) The capabilities build on one another. Perception alone improves all four standard tasks, and perception and task solving strengthen complementary aspects of reasoning: the former chiefly helps read the signal and ground it in context, the latter chiefly helps infer dependencies and choose actions. (2) Order matters beyond the data, but only with an encoder. On identical data, learning the stages in order beats mixing them, most of all when reasoning data are scarce: with progressive training, 15% of the reasoning data roughly matches reasoning-only training on 50%. Under mixed training, reasoning gradients dominate the updates while aligning poorly with the overall objective, and ordering the stages more than doubles this alignment. Without an encoder, however, the advantage nearly vanishes, an interaction that only a multi-capability setting can reveal. (3) Capabilities are transferred through distinct network depths rather than simply stacked. Perception-to-solving transfer dominates the shallower layers and solving-to-reasoning transfer the deeper ones, while direct perception-to-reasoning transfer stays weakest, so reasoning is built through task solving. (4) Local convergence does not mean readiness. Each preceding stage saturates on its own benchmark with about 70% of its data, yet training on the rest still benefits the stages that follow, so local convergence alone should not decide when to advance.
These findings translate into a simple recipe: we should encode time series with a dedicated encoder and learn the three stages in order, each on its full data. With only 4B parameters, the resulting TS-LLMs achieve the best result on seven of eight out-of-domain metrics, ahead of 7B TS-LLMs such as TimeOmni-1, ITFormer, ChatTS, and PATRA [7, 22, 30, 31].
In summary, our contributions are as follows: (1) A comprehensive suite. We build the first training and evaluation suite that covers perception, task solving, and reasoning for time series. (2) How time series should enter the LLM. In this multi-capability setting, we show that a dedicated encoder is essential, that a pre-aligned encoder trained once suffices, and that later capabilities form by re-alignment. (3) How capabilities form. We show that the capabilities build on one another and are best learned in order, explain this through gradient alignment, depth-wise representation transfer, and stage readiness, and turn the findings into a recipe for strong, compact TS-LLMs.
Related Work
LLM for Time Series Analysis. Recent studies have explored the potential of LLMs for time series analysis across various tasks, including forecasting, classification, anomaly detection, and QA [13, 24, 30, 32]. These works investigate how the semantic understanding and instruction-following capabilities of LLMs can be adapted to temporal data [8, 11, 25]. For instance, CrisisTS integrates textual crisis reports with meteorological time series to improve multilingual urgency assessment beyond text-only approaches [23]. Time-MQA extends LLMs to multi-task temporal QA by combining language reasoning with time series information [13], while ITFormer develops an end-to-end framework that jointly models temporal patterns and language generation for multimodal QA in engine maintenance scenarios [30]. Existing studies mainly target individual downstream tasks, while general temporal capability formation remains underexplored.
QA with Language and Time Series. Some studies have explored the integration of language models with time series data to enable interactive temporal QA [7, 13, 30, 31, 34]. These works construct time series QA datasets and leverage instruction tuning, chain-of-thought reasoning, and alignment strategies to extend TS-LLMs from numerical prediction toward temporal understanding [7, 13, 30, 31]. For example, Time-MQA introduces multi-task temporal QA by combining language reasoning with time series information [13], while recent frameworks further investigate complex temporal analysis through instruction following and reasoning-oriented training [7, 30, 34]. Existing approaches mainly optimize QA performance, while the temporal perception and task-solving foundations of reasoning remain underexplored. Direct QA supervision may therefore yield strong responses without grounded temporal understanding.
A Progressive Three-Stage Capability Dimensions for TS-LLMs
Despite recent progress in applying LLMs to time series perception, task solving, and reasoning, these capabilities are often studied independently. To analyze the relationships among different tasks, we organize the capabilities of TS-LLMs into three stages: Perceive, Solve, and Reason. More data details are provided in Appendix § 8 & § 10.
Model Architecture. Existing studies have developed three main paradigms in Figure 1. (1) Time Series Textualization directly converts time series into textual sequences that can be processed by LLMs. (2) Pre-align architecture extracts temporal representations with an independent time series encoder and projects them into the LLM embedding space, enabling joint processing with text inputs. (3) Post-align architecture first encodes raw time series into continuous temporal embeddings and feeds them into the LLM as additional inputs. In our experiments, we adopt TSAlign [26] and Time-MQA [13] as representative Pre-align and Post-align architectures, with Qwen3-4B [33] as the LLM backbone. In Pre-align, we train only the encoder and alignment module; in Post-align, we train the encoder and LLM backbone; and in the textualization setting, only the LLM is fine-tuned via SFT.
Figure 1. Three Model Architecture.
Stage 1: Fundamental Attribute Perception. Advanced time series analysis relies on understanding fundamental temporal dynamics. For example, forecasting requires capturing trend patterns, while anomaly analysis requires identifying local behaviors. Therefore, Stage 1 aims to equip LLMs with basic temporal perception. The questions focus on trends, seasonality, anomalies, local patterns, and multivariate relationships. Training Set. The training data is mainly collected from ChatTS, with approximately K samples [31]. Evaluation Set. We use two datasets to assess perception. Dataset A contains 525 constructed questions. Dataset B contains 1,616 questions. The evaluation includes categorical and numerical tasks. Categorical tasks are evaluated by F1-score, while numerical tasks are evaluated by relative accuracy.
Stage 2: Temporal Task Solving. Many real-world time series applications require models to solve specific tasks. Therefore, Stage 2 investigates whether models can transfer basic temporal perception abilities to downstream task solving. Training Set. We use TSQA [4] and TS Instruct [27] to train task-solving abilities. TSQA contains 155,214 samples, including 42,557 forecasting, 38,657 imputation, 37,000 anomaly detection, and 37,000 classification samples. TS Instruct provides an additional 2,945 classification samples, resulting in 158,159 samples in total. All samples are split into training and testing sets with an 8:2 ratio. Evaluation Set. We evaluate four standard tasks. Forecasting and imputation are evaluated using Mean Squared Error (MSE), while anomaly detection and classification are evaluated using Accuracy.
Stage 3: High-level Reasoning. Stage 3 aims to advance TS-LLMs to intelligent systems capable of analysis. Training Set. TSQA reasoning QA provides 26,163 samples, ChatTS TSEvol and Instruct Follow [31] provide 29,320 samples, the TSCognition [26] training set provides 32,869 samples, and TS Instruct provides 12,852 samples, resulting in 105,751 samples in total. Evaluation Set. We use two independent sets. Set A contains 4,547 samples covering binary, multiple-choice, and open-ended questions, evaluated using Accuracy, ROUGE-L, and BLEU. Set B contains 6,163 samples and evaluates five dimensions: Decoding, Grounding, Inferring, Extrapolating, and Acting. The metric is the accuracy of multiple-choice questions.
Investigating Capability Formation in TS-LLMs
We formulate five research questions (RQs) to investigate the mechanisms underlying capability relationships in TS-LLMs. They examine cross-stage transfer, progressive supervision, input format, stage-wise convergence, and the conditions that support high-level capability formation. The LLM backbone is primarily Qwen3-4B, with additional Qwen3-1.7B results reported in Appendix E.
RQ1: Can Temporal Perception Facilitate to Task Solving? We first evaluate the perception ability of models after Stage 1 training. As shown in Figure 2 & Table 16, we compare two TS-LLM architectures, Pre-align and Post-align on categorical and numerical tasks, with general multimodal LLMs, GPT-4o (Text) and GPT-4o (Vision), where the GPT-4o results are adopted from [31]. Both Pre-align and Post-align TS-LLMs outperform GPT-4o on most tasks, showing that single Stage 1 training enables TS-LLMs to develop effective fundamental perception. We compare different learning paradigm on the Stage 2 task solving evaluation set to investigate whether perception abilities can facilitate task solving. As shown in Table 1, *Stage 1
Figure 2. Categorical and numerical perception comparison across different methods.
| Model | Training Setting | Fore. MSE | Impu. MSE | Ano Det. Acc. | Class. Acc. |
| Qwen3-4B | No fine-tuning | 1.970 | 0.0286 | 0.333 | 0.296 |
| Pre-align | Stage 1 only | 1.842 | 0.0194 | 0.587 | 0.459 |
| Pre-align | Stage 2 only | 1.368 | 0.0136 | 0.682 | 0.567 |
| Pre-align | Stage (1 2) | 1.328 | 0.0132 | 0.739 | 0.628 |
| Post-align | Stage 1 only | 1.716 | 0.0217 | 0.612 | 0.431 |
| Post-align | Stage 2 only | 1.423 | 0.0146 | 0.660 | 0.529 |
| Post-align | Stage (1 2) | 1.286 | 0.0159 | 0.720 | 0.597 |
Table 1. Results of four task solving evaluation sets.
| Method | Trend Cate. | Trend Num. | Season Cate. | Season Num. | Noise Cate. | Noise Num. | Local Cate. | Local Num. | Corr. Cate. | Clus. Cate. |
| Pre-align | 0.937 | 0.865 | 0.911 | 0.872 | 0.857 | 0.534 | 0.799 | 0.700 | 0.831 | 0.783 |
| GPT-4o (Text) | 0.664 | 0.839 | 0.751 | 0.546 | 0.654 | 0.287 | 0.237 | 0.165 | 0.361 | 0.294 |
| GPT-4o (Vision) | 0.691 | 0.423 | 0.907 | 0.582 | 0.659 | 0.361 | 0.415 | 0.302 | 0.394 | 0.411 |
| Post-align | 0.922 | 0.880 | 0.931 | 0.834 | 0.900 | 0.479 | 0.824 | 0.733 | 0.771 | 0.804 |
Table 16. Detailed categorical and numerical perception results.
only denotes training only on the training set of Temporal Perception. Stage 2 only directly trains the model on the training set of Task Solving, and Stage (1 ) denotes sequential training first on Temporal Perception and then on Task Solving.
Conclusion: (1) Stage 1 provides effective fundamental perception. Compared with the non-fine-tuned model, Stage 1 improves all tasks, indicating effective acquisition of fundamental temporal attributes. (2) Stage 2 is the primary source of performance improvement. Training on task-solving objectives enhances the corresponding downstream capabilities. (3) Temporal perception can effectively facilitate downstream task solving. Stage (1 ) obtains the best results on most tasks.
RQ2: Do Perceive and Solve Facilitate Reason Formation? Recent studies have advanced TS-LLMs toward complex QA reasoning [7, 30]. However, QA supervision mainly optimizes final answers and may overlook critical temporal evidence. We therefore compare four capability development paths across five dimensions. Experimental results in Table 2 and Table 3 show that introducing preceding stages improves reasoning performance compared with directly training on Reasoning datasets. On Evaluation Set B, Stage (1 \to 3)* achieves higher accuracy than Stage 3 Only, indicating that perception helps models better utilize evidence. *Stage (2 \to 3) also improves multiple reasoning dimensions, suggesting that task solving experience provides richer temporal dynamics for subsequent reasoning. Similar trends are observed on Evaluation Set A, where both preceding-stage strategies improve True/False, multiple-choice, and open-ended QA performance in most cases. Overall, Stage (1 $\to 2 \to 3) achieves the best results on most metrics.
| Model | Training Setting | Decoding | Grounding | Inferring | Extrapolating | Acting |
| Pre-align | Stage 3 Only | 72.2 | 68.5 | 61.3 | 57.2 | 50.7 |
| Pre-align | Stage (1 3) | 77.8 | 81.5 | 63.3 | 62.2 | 51.8 |
| Pre-align | Stage (2 3) | 75.1 | 72.0 | 65.8 | 59.5 | 53.9 |
| Pre-align | Stage (1 2 3) | 82.5 | 79.1 | 70.7 | 65.4 | 57.2 |
| Post-align | Stage 3 Only | 68.5 | 66.2 | 58.9 | 58.8 | 49.2 |
| Post-align | Stage (1 3) | 75.6 | 73.8 | 61.2 | 64.1 | 54.3 |
| Post-align | Stage (2 3) | 73.9 | 72.7 | 66.0 | 68.9 | 58.8 |
| Post-align | Stage (1 2 3) | 79.0 | 79.4 | 64.2 | 66.7 | 61.4 |
Table 2. Accuracy (%) of different strategies on the Evaluation Set B of high-level reasoning.
| Model | Training Setting | Ture/False QA Acc. ↑ | Multi-Choice QA Acc. ↑ | Open-end QA ROUGE-L↑ | Open-end QA BLEU↑ |
| Pre-align | Stage 3 Only | 71.4 | 58.6 | 43.8 | 24.7 |
| Pre-align | Stage (1 → 3) | 78.3 | 62.2 | 45.5 | 25.8 |
| Pre-align | Stage (2 → 3) | 74.5 | 65.1 | 47.2 | 26.7 |
| Pre-align | Stage (1 → 2 → 3) | 80.6 | 68.8 | 51.6 | 31.5 |
| Post-align | Stage 3 Only | 66.3 | 60.3 | 42.1 | 22.3 |
| Post-align | Stage (1 → 3) | 70.4 | 63.6 | 40.8 | 24.5 |
| Post-align | Stage (2 → 3) | 68.6 | 67.6 | 44.4 | 25.9 |
| Post-align | Stage (1 → 2 → 3) | 72.3 | 71.5 | 49.7 | 28.9 |
Table 3. Performance on the Evaluation Set A of high-level reasoning.
Conclusion: (1) Perceive and Solve jointly facilitate the formation of Reason. Temporal perception provides fundamental knowledge, while task-solving experience exposes models to richer dynamics, together supporting more effective high-level reasoning. (2) The complete Stage (1 $\to 2 \to 3) pathway achieves the best overall performance.
RQ3: Does Reason Benefit from Progressive Capability Accumulation? RQ2 has shown that both Perceive and Solve provide effective foundations for Reason. We further investigate whether these capabilities need to be accumulated progressively following the stage order. To this end, we compare three training strategies: Stage 3 Only, Mixed Training, and
Stage (1 → 2 → 3). We gradually vary the proportion of high-level reasoning data from 15% to 100%, while keeping the perception and task-solving training data unchanged. Figure 3 & Table 17 shows that progressive capability accumulation achieves stronger Reason performance on the Evaluation Set B of high-level reasoning across most data scales. When only 15% of Stage 3 data is used, Stage (1 → 2 → 3) reaches 0.528 and 0.520 on Pre-align and Post-align, clearly outperforming both Mixed Training and Stage 3 Only. As more Stage 3 data is introduced, all strategies improve, while the progressive pathway remains highly competitive.
Figure 3. Accuracy across different training data scales.
| Model | Training Setting | 15% | 25% | 50% | 75% | 100% |
| Pre-align | Stage 3 Only | 0.414 | 0.464 | 0.527 | 0.586 | 0.619 |
| Pre-align | Mixed Training | 0.479 | 0.520 | 0.583 | 0.673 | 0.689 |
| Pre-align | Stage () | 0.528 | 0.574 | 0.598 | 0.664 | 0.710 |
| Post-align | Stage 3 Only | 0.403 | 0.453 | 0.524 | 0.585 | 0.617 |
| Post-align | Mixed Training | 0.461 | 0.502 | 0.570 | 0.664 | 0.678 |
| Post-align | Stage () | 0.520 | 0.561 | 0.614 | 0.680 | 0.701 |
| Directly | Stage 3 Only | 0.402 | 0.436 | 0.478 | 0.517 | 0.546 |
| Directly | Mixed Training | 0.439 | 0.497 | 0.530 | 0.588 | 0.608 |
| Directly | Stage () | 0.451 | 0.490 | 0.542 | 0.579 | 0.614 |
| Directly | Stage () | 0.429 | 0.482 | 0.519 | 0.572 | 0.599 |
| Directly | Stage () | 0.447 | 0.469 | 0.535 | 0.563 | 0.604 |
| Directly | No Training | 0.351 | 0.351 | 0.351 | 0.351 | 0.351 |
Table 17. Detailed reasoning accuracy under different Stage 3 training data scales.
Conclusion: (1) Ordered capability accumulation is more effective than mixed supervision for Reason formation. Under the same supervision sources, progressive training consistently outperforms mixed training. (2) Progressive capability accumulation improves Reason supervision efficiency. Its advantage becomes more pronounced when Stage 3 data is limited.
RQ4: Does Time Series Input Format Affect Progressive Formation? We adopt the Textualization paradigm, which directly feeds time series representations into the LLM, and compare Stage 3 Only, Mixed Training, Stage (1 → 3), Stage (2 → 3), and Stage (1 → 2 → 3) under the same training paths. Reason performance is evaluated on the Evaluation Set A of high-level reasoning. Results are shown in Figure 4 & Table 17. No Training denotes direct evaluation of the original Qwen3-4B without any additional training. We observe that under the Textualization paradigm, all training strategies improve steadily as more training data is introduced. However, the gaps between different capability paths are much smaller than those under Pre-align
Figure 4. Reason under the Textualization paradigm across training data scales of high-level reasoning.
and Post-align. In addition, the complete Stage (1 → 2 → 3) pathway provides only limited gains and performs similarly to Mixed Training, Stage (1 → 3), and Stage (2 → 3). Our results indicate that progressive capability formation is more evident in TS-LLMs with dedicated cross-modal alignment, while the cross-stage accumulation effect is weakened under direct textualized input.
Conclusion: (1) Time series input format affects progressive capability formation. Input strategies with temporal modeling exhibit clearer progressive gains, while direct textualization weakens this effect. (2) Explicit temporal integration facilitates cross-stage capability transfer. It better preserves and transfers preceding capabilities across progressive stages.
RQ5: Does Stage-wise Convergence Ensure TS-LLMs Progression? During progressive learning, a key question is whether convergence at a preceding stage provides a sufficiently stable foundation for subsequent capability development. Similar to how pretraining states affect post-training in LLMs, local convergence does not necessarily imply sufficient preparation for later capability formation [1, 3]. We therefore examine two processes: (i) when increasing data in a preceding stage causes its stage-specific performance to enter a saturation regime; (ii) whether additional data from that stage can still provide gains for subsequent capability formation after its local performance has largely stabilized. As shown in Table 4 & Table 18, local performance at each stage reaches a clear saturation regime before using the full training set. Stage 1 only and Stage (1→2) achieve 98.3% and 98.8% of their full-stage performance at around 70% data, while Stage (1→2→3) reaches 97.2% at 85%. This indicates that each stage appears nearly converged under its own evaluation. However, local saturation does not imply that the model is fully prepared for subsequent capability development. Full I→II→III training still yields additional gains, especially for Solve and Reason.
| Training Setting | Training Data | Evaluation Performance | Relative Perf. | I→II→III |
| Stage 1 only | (40% → 100%) I | 0.613, 0.662, 0.738, 0.745, 0.751 | 98.3% | 0.743 |
| Stage (1 → 2) | 70% I, (40% → 100%) II | 0.542, 0.603, 0.644, 0.649, 0.652 | 98.8% | 0.684 |
| Stage (1 → 2 → 3) | 70% I, 70% II, (40% → 100%) III | 0.537, 0.569, 0.616, 0.663, 0.682 | 97.2% | 0.710 |
Table 4. Stage-wise performance of the Pre-align model under increasing training data scales. For each stage, we progressively increase the training data while fixing preceding stages at their identified saturation points. Stage 1 uses Numerical Perception Relative Accuracy; Stage 2 uses Anomaly Detection and Classification Accuracy; Stage 3 uses the average Accuracy on Evaluation Set B. For convenient evaluation, we select accuracy-based tasks as measures. Bold denotes the local saturation points.
| Training Setting | Data Scale | Evaluation Performance | Relative Perf. | Full Progressive |
| Stage 1 | 40% | 0.613 | – | |
| 55% | 0.662 | – | ||
| 70% | 0.738 | 98.3% | 0.743 | |
| 85% | 0.745 | – | ||
| 100% | 0.751 | 100% | ||
| Stage () | 40% | 0.542 | – | |
| 55% | 0.603 | – | ||
| 70% | 0.644 | 98.8% | 0.684 | |
| 85% | 0.649 | – | ||
| 100% | 0.652 | 100% | ||
| Stage () | 40% | 0.537 | – | |
| 55% | 0.569 | – | ||
| 70% | 0.616 | – | 0.710 | |
| 85% | 0.663 | 97.2% | ||
| 100% | 0.682 | 100% |
Table 18. Detailed stage-wise performance under increasing training data scales. For Stage (), Stage 1 is fixed at its 70% saturation point; for Stage (), Stages 1 and 2 are both fixed at 70%. Bold denotes the identified local saturation points.
Conclusion: Stage-wise convergence does not necessarily indicate capability sufficiency for subsequent progression. A capability can approach its local performance ceiling while additional training at that stage still provides valuable foundations for later capability formation, suggesting that local saturation alone is insufficient to determine when progressive training should advance.
Mechanisms of Progressive Capability Formation
To understand why progressive learning facilitates capability development, we examine the internal mechanisms underlying the three-stage pathway. We conduct analyses from the perspectives of optimization dynamics (§ 5.1), temporal attribute contribution (§ 5.2), representation transfer across stages (§ 5.4), and alignment-driven capability formation (§ 5.3).
Optimization Dynamics
To understand the optimization mechanism behind progressive capability formation, we analyze the training dynamics from the perspective of data-induced gradient evolution.
Formulation. We consider 3 capability stages, with data distributions denoted as , , and . All stages share the same objective: . For each capability stage , the corresponding optimization objective is defined over its data distribution: . The gradient induced by each capability objective is given by: . Under Mixed Training, multiple capability objectives are optimized simultaneously [29]. The overall update direction is determined by the aggregated gradient: , where denotes the sampling proportion of each stage. Balancing gradients from different objectives is important for stable multi-task optimization [5]. Such joint optimization may introduce competition among capability objectives, causing different stages to follow inconsistent optimization trajectories [15, 35]. Progressive Training follows the ordered capability pathway: , where each stage is optimized sequentially. This separates the optimization process and provides stage-specific updates during capability formation.
Definition. (1) Optimization Strength. For each capability stage , the optimization strength is measured by the squared gradient norm: , where denotes the gradient induced by stage data. A larger indicates stronger optimization signals from the corresponding capability objective. Under mixed training, the relative gradient contribution is computed as . (2) Optimization Alignment. The directional compatibility between capability-specific optimization and the overall training objective is quantified by: , where denotes the aggregated gradient across all capability stages. A larger indicates that the capability-specific gradient follows the global optimization direction more closely. (3) Gradient Update Efficiency. The efficiency of converting gradients from each capability stage into parameter updates is: , where denotes the accumulated parameter update during optimization. A larger indicates that gradients from a capability stage are converted into parameter updates more effectively. We further compare the relative update efficiency across capability stages by normalizing with respect to the overall optimization efficiency: .
Analysis. As shown in Figure 5, Optimization Strength reveals distinct optimization dynamics between Mixed Training and progressive training. We quantify the optimization strength of each capability stage using the squared gradient norm. Mixed Training jointly optimizes multiple capability objectives, leading to imbalanced optimization signals, where Reason supervision may dominate parameter updates. In contrast, progressive training allocates optimization strength according to the capability development order, allowing each stage to receive more concentrated optimization signals. TS Textualization fails to achieve optimization dynamics comparable to architectures with Pre-align and Post-align. Therefore, structured temporal representations provide more stable optimization signals. Building on the analysis of optimization signal magnitude, we compute the alignment score of each capability stage during training. This metric measures the directional consistency between capability-specific gradients and the aggregated gradient, providing a comparison of optimization directions. As shown in Table 5, Progressive Training improves gradient alignment under both Pre-align and Post-align architectures. The improvement is particularly evident in the Reason stage, suggesting that progressive training enables high-level capability optimization. Finally, we compute the relative efficiency gain of Progressive Training over Mixed Training to evaluate how effectively optimization signals are utilized under different temporal representations. As shown in Figure 6, Progressive Training improves optimization efficiency across all temporal representation paradigms. The gain is most significant in the Reason stage, where Pre-align and Post-align achieve efficiency improvements of 112.5% and 96.7%, and TS Textualization achieves only 48.6%.
Figure 5. Optimization strength dynamics across capability stages.
| Model | Setting | Perceive | Solve | Reason |
| Pre-align | Mixed | 0.463 | 0.382 | 0.214 |
| Progressive | 0.612 | 0.574 | 0.593 | |
| Post-align | Mixed | 0.421 | 0.347 | 0.236 |
| Progressive | 0.534 | 0.536 | 0.565 | |
| TS Textualization | Mixed | 0.318 | 0.219 | 0.241 |
| Progressive | 0.301 | 0.254 | 0.226 |
Table 5. Optimization alignment between specific and overall optimization directions.
Figure 6. Gradient update efficiency.
Impact of Temporal Attributes on Reasoning Capability
To analyze the role of perception abilities, we conduct an ablation study over six Stage 1 attributes: Trend, Seasonality, Noise, Local Pattern, Correlation, and Clustering. Specifically, we remove one attribute category at a time while keeping the same three-stage progressive strategy. The final models are evaluated on Evaluation Set A of high-level reasoning to measure their Reason performance.
As shown in Figure 7, removing different attributes from Stage 1 leads to varying impacts on the final Reason capability. Among them, removing Correlation, Local Pattern, and Clustering causes the most significant performance degradation. In contrast, Trend and Seasonality have relatively smaller impacts, but still contribute to specific reasoning dimensions. In contrast, removing Noise causes little degradation, showing that Reason capability relies less on noise-related temporal patterns.
Figure 7. Temporal attribute ablation analysis.
Alignment-Driven Capability Formation
Under the Pre-align architecture, the Temporal Encoder is updated only during Stage 1 and then frozen throughout Stages 2 and 3, while only the input-side alignment modules continue to be optimized. Compared with full progressive training, freezing the encoder causes only limited degradation on task solving in Table 6, with forecasting, imputation, and anomaly detection showing only small changes and classification remaining comparable. A similar pattern is observed for high-level reasoning in Figure 9, where the performance gap between frozen and non-frozen settings remains small across capability stages. Our observation suggests that Stage 1 already provides a sufficiently mature temporal feature space to support subsequent capability development. The formation therefore appears to rely mainly on how fixed temporal features are progressively aligned and reorganized for complex downstream capabilities. In this sense, later-stage capability growth is driven more by the evolution of the alignment process than by further modification of the underlying temporal encoder.
| Metric | Non Frozen | Frozen |
| Fore. MSE | 1.328 | 1.336 |
| Impu. MSE | 0.0132 | 0.0142 |
| Ano. Acc. | 0.739 | 0.718 |
| Class. Acc. | 0.628 | 0.643 |
Table 6. Effect of freezing the encoder in task solving evaluation.

Figure 9. Effect of freezing the encoder in reasoning evaluation.
Representation Transfer across Stages
We further investigate capability transfer from the representation perspective. Following the three-stage learning process , we adopt the representation projection analysis in [12, 36] to measure the proportion of later-stage representations preserved in the representation space of preceding stages. This transfer ratio quantifies the information transmission between stages, where a higher value indicates stronger representation retention across stages.
We conduct a cross-layer analysis on Qwen3-4B to examine representation transfer patterns. We explore three capability transitions: Perceive-to-Solve (PS), Solve-to-Reason (SR), and Perceive-to-Reason (PR). As shown in Figure 8, different capability transitions exhibit distinct hierarchical transfer patterns. PS transfer mainly occurs in intermediate layers, suggesting that fundamental temporal structures are transformed into task-solving representations within middle layers. In contrast, SR transfer reaches its peak in deeper layers, indicating that Reason primarily inherits high-level task-oriented representations. PR transfer remains relatively limited, suggesting that advanced reasoning is more likely constructed through the intermediate Solve stage rather than direct capability transfer. Our findings reveal the representation evolution mechanism of Progressive Training: different capabilities are not simply stacked together but are progressively transformed across different network depths, forming a three-stage representation development pathway.
Figure 8. Representation transfer across layers.
Application: Train a Powerful TS-LLM
Based on the above analysis, we adopt a three-stage paradigm, Perceive Solve Reason, to train TS-LLM (Pre-align) and TS-LLM (Post-align). Similarly, Qwen3-4B is used as the backbone. Stage 1 uses about 105K training samples to learn fundamental temporal attributes, including trends, seasonality, anomalies, local patterns, and multivariate relationships. Stage 2 is built on 158,159 samples, split 8:2 for training and testing, covering forecasting, imputation, anomaly detection, and classification. Stage 3 introduces multi-source data to develop evidence-based temporal analysis. For the Pre-align architecture, the Encoder is updated only in Stage 1 and frozen in Stages 2 and 3. Finally, we obtain two powerful TS-LLMs.
We conduct out-of-domain (OOD) experiments on MTBench [4] while ensuring that no evaluation data are exposed during training. We compare our models with representative TS-LLMs, including ITFormer, ChatTS, TimeOmni-1, and PATRA [7, 22, 30, 31]. More detailed descriptions of the baselines and datasets are provided in Appendix § 9 & Appendix § 10. As shown in Table 7, our two TS-LLMs achieve the best or second-best results on most metrics across both Weather and Finance domains. TS-LLM (Pre-align) performs particularly well on Weather MCQA, weather prediction, and 7-day finance trend prediction, while TS-LLM (Post-align) achieves the strongest results on 7-day finance MSE and most 30-day finance metrics. In addition, we evaluate temporal perception on TimeSeriesExam [2] and CaTS-Bench [37]. As shown in Table 8, TS-LLM (Pre-align) achieves the best performance on most metrics. our results show that the proposed progressive training strategy generalizes effectively to unseen domains and maintains strong performance across different tasks.
| Models | Weather | Finance (7 days) | Finance (30 days) | |||||
| MCQ. ↑ | Pre. ↑ | 3-way ↑ | 5-way ↑ | MSE ↓ | 3-way ↑ | 5-way ↑ | MSE ↓ | |
| TimeOmni-1-7B | 41.67 | 51.40 | 40.76 | 34.97 | 0.622 | 37.39 | 20.00 | 1.490 |
| ITFormer-7B | 57.14 | 36.26 | 40.14 | 33.57 | 0.174 | 51.03 | 31.72 | 0.907 |
| ChatTS-7B | 56.21 | 28.79 | 19.71 | 16.09 | 2.047 | 19.38 | 13.46 | 3.342 |
| PATRA-7B | 60.46 | 52.04 | 63.33 | 60.80 | 0.191 | 54.39 | 43.70 | 0.822 |
| TS-LLM (Pre-align) | 67.24 | 57.17 | 68.19 | 57.82 | 0.187 | 60.29 | 41.51 | 0.837 |
| TS-LLM (Post-align) | 65.92 | 52.76 | 59.88 | 55.43 | 0.160 | 65.78 | 47.16 | 0.744 |
Table 7. Out-of-domain results on reasoning tasks. Red/blue indicate the 1st/2nd best results.
| Method | TimeSeriesExam | CaTS-Bench | ||||||||
| Ano. Det. | Causality | Noise | Pattern | Similarity | Caption Ret. | TS Ret. | Amp. Comp. | Mean Comp. | ||
| Original Paper | 28.0 | 52.0 | 26.0 | 47.0 | 45.0 | 44.0 | 29.0 | 43.0 | 35.0 | |
| TimeOmni-1 | 42.6 | 37.5 | 56.0 | 56.4 | 46.7 | 57.0 | 37.0 | 92.5 | 67.5 | |
| TS-LLM (Pre-align) | 49.1 | 47.2 | 60.7 | 61.6 | 55.0 | 52.0 | 46.0 | 82.5 | 77.5 |
Table 8. Results on TimeSeriesExam and CaTS-Bench. Bold indicates the best results.
Conclusion
Our work investigates how temporal capabilities progressively form in TS-LLMs. We organize model abilities into three stages: Perceive, Solve, and Reason, and show that ordered capability accumulation provides a stronger foundation for high-level reasoning than isolated or mixed training. Further analyses reveal that capability formation is influenced by training order, input format, stage maturity, and cross-layer representation transfer. We also find that local stage convergence does not necessarily indicate sufficient preparation for subsequent capability development. These observations provide a clearer understanding of how lower-level temporal abilities support more advanced reasoning. Based on these findings, we develop a progressive training strategy for TS-LLMs, which achieves strong out-of-domain performance and offers a practical pathway for building more comprehensive temporal reasoning models.
Statistics Overview
Stage 1: Fundamental Perception. Table 9 summarizes the data used for temporal perception. The training set contains approximately 105K samples from ChatTS, while two independent evaluation sets contain 525 manually constructed questions and 1,616 synthetic questions, respectively. The evaluation covers both categorical and numerical perception tasks.
| Split | Source | Samples | Task Type | Metric |
| Training | ChatTS | 105K | Temporal Perception | – |
| Evaluation A | Manual | 525 | Categorical / Numerical | F1 / Relative Acc. |
| Evaluation B | Synthetic | 1,616 | Categorical / Numerical | F1 / Relative Acc. |
Table 9. Statistics of the Stage 1 perception datasets.
Stage 2: Temporal Task Solving. Table 10 reports the detailed statistics of the task-solving data. The dataset covers forecasting, imputation, anomaly detection, and classification, with 158,159 samples in total. All samples are split into training and testing sets with an 8:2 ratio, and task-specific metrics are used for evaluation.
| Source | Task | Total Samples | Train Samples | Test Samples | Metric |
| TSQA | Forecasting | 42,557 | 34,046 | 8,511 | MSE |
| TSQA | Imputation | 38,657 | 30,926 | 7,731 | MSE |
| TSQA | Anomaly Detection | 37,000 | 29,600 | 7,400 | Accuracy |
| TSQA | Classification | 37,000 | 29,600 | 7,400 | Accuracy |
| TS Instruct | Classification | 2,945 | 2,356 | 589 | Accuracy |
| Total | – | 158,159 | 126,528 | 31,631 | – |
Table 10. Detailed statistics of the Stage 2 task-solving datasets.
Stage 3: High-level Reasoning. Table 11 presents the composition of the reasoning data. The training set is collected from multiple sources, including TSQA, ChatTS, TSCognition, and TS Instruct. Two independent evaluation sets are used to assess both QA formats and five reasoning dimensions: Decoding, Grounding, Inferring, Extrapolating, and Acting.
| Split | Source | # Samples | Metric |
| Training | TSQA Reasoning QA | 26,163 | – |
| Training | ChatTS TSEvol + Instruct Follow | 29,320 | – |
| Training | TSCognition | 32,869 | – |
| Training | TS Instruct | 12,852 | – |
| Evaluation A | Binary QA | 4,547 | Accuracy |
| Evaluation A | Multiple-Choice QA | 4,547 | Accuracy |
| Evaluation A | Open-ended QA | 4,547 | ROUGE-L / BLEU |
| Evaluation B | Decoding | 6,163 | Accuracy |
| Evaluation B | Grounding | 6,163 | Accuracy |
| Evaluation B | Inferring | 6,163 | Accuracy |
| Evaluation B | Extrapolating | 6,163 | Accuracy |
| Evaluation B | Acting | 6,163 | Accuracy |
Table 11. Detailed statistics of the Stage 3 high-level reasoning datasets.
Baseline
ChatTS [31]. ChatTS is a time series understanding framework that enhances LLMs with instruction-based temporal data covering fundamental patterns such as trends, seasonality, anomalies, and local variations. It converts time series into representations compatible with language models and performs instruction tuning for temporal question answering. We use ChatTS as a representative baseline for evaluating whether models trained on temporal understanding data can generalize to more comprehensive perception and reasoning tasks.
TimeOmni-1 [7]. TimeOmni-1 is a unified TS-LLM designed to support multiple time series understanding and reasoning tasks within a single framework. It integrates temporal information with language instructions and is trained on diverse time series tasks to improve cross-task generalization. We include TimeOmni-1 as a representative general-purpose TS-LLM baseline for comparing comprehensive temporal perception and reasoning capabilities.
ITFormer [30]. ITFormer is an end-to-end multimodal framework for time series question answering that jointly models temporal representations and language generation. It directly integrates time series features with the language model to support temporal understanding and reasoning over multimodal inputs. We use ITFormer as a representative alignment-based TS-LLM baseline for evaluating high-level temporal reasoning and out-of-domain generalization.
PATRA [22]. PATRA is a recent reasoning-oriented TS-LLM developed for complex temporal analysis and question answering. It emphasizes the integration of time series representations with language reasoning to improve performance on diverse temporal reasoning tasks. We include PATRA as a strong recent baseline for comparing the reasoning and generalization capabilities of our progressively trained TS-LLMs.
Qwen3-4B [33]. Qwen3-4B is the base large language model used in our framework. We adopt it as a non-specialized LLM baseline by directly applying the same input format without progressive time series capability training. This baseline is used to evaluate the temporal understanding and reasoning ability provided by the pretrained language model itself, and to measure the gains introduced by our stage-wise Perceive Solve Reason training strategy.
Dataset
ChatTS [31]. ChatTS constructs fully synthetic time series–text data for temporal understanding and reasoning. Its training data consist of five subsets: UTS, MTS-Shape, MTS-Local, TSEvol, and Instruct Follow. UTS contains 35K samples for basic univariate temporal attributes, including global and local patterns; MTS-Shape contains 35K multivariate samples focusing on global trend correlations; and MTS-Local contains 35K samples emphasizing correlated local fluctuations across variables. In addition, TSEvol provides 24,270 diverse question–answer pairs for temporal reasoning, while Instruct Follow contains 5,050 samples for instruction-following ability. The generated time series cover lengths from 64 to 1024 and include numerical supervision such as extrema, segmented averages, local fluctuation positions and amplitudes, seasonality, trends, and point-wise values. In our framework, the three alignment subsets are mainly used for Stage 1 temporal perception, while TSEvol and Instruct Follow are incorporated into Stage 3 reasoning training.
TSQA [13]. TSQA is a large-scale time series QA dataset introduced with Time-MQA, containing approximately 200K QA pairs across 12 real-world application domains, including healthcare, environment, energy, finance, transportation, IoT, AIOps, and human activities. It covers five task categories: forecasting, imputation, anomaly detection, classification, and open-ended reasoning. Specifically, it contains 42,557 forecasting, 38,657 imputation, 37,000 anomaly detection, 37,000 classification, and 37,629 open-ended QA samples. The reasoning subset further includes true/false, multiple-choice, and open-ended questions. In our framework, the numerical task subsets are mainly used in Stage 2, while the reasoning QA subset is incorporated into Stage 3.
TS Instruct [27]. TS Instruct is a multimodal instruction-tuning dataset introduced with Chat-TS, pairing diverse time series with natural-language instructions and responses. The training set contains 18,306 samples covering four categories: 9,726 reasoning samples, 3,126 decision-making samples, 2,945 classification samples, and 2,509 mathematical reasoning samples. The dataset is generated using time series properties, visual representations, task descriptors, and synthetic conversations to support joint reasoning over temporal and textual information. In our framework, the classification subset is used in Stage 2, while the remaining reasoning-oriented samples are incorporated into Stage 3.
TSCognition [26]. TSCognition is constructed from 15 public real-world datasets spanning eight domains, including Energy (Buildings-900K1 and UMass Smart*2), Retail (Favorita Sales3 and Dominick4), Healthcare (Project Tycho5 and JHU CSSE COVID-196), Transportation (LargeST7 and METR-LA8), Climate (SubseasonalClimateUSA9 and Oikolab Weather10), Finance (Bitcoin11), Web (Web Traffic12), and Industry (WADI13, C-MAPSS14, and Paderborn15). It organizes time series cognition into five hierarchical tasks: Decoding, Grounding, Inferring, Extrapolating, and Acting. Decoding focuses on extracting fundamental temporal patterns, Grounding aligns temporal signals with contextual information, Inferring models dependencies and relational structures, Extrapolating extends learned patterns to future dynamics, and Acting integrates preceding reasoning results to support decision making. The sample distribution across domains and tasks is shown in Figure 10, while detailed task-level statistics, including variable numbers, time series lengths, and answer choices, are summarized in Table 12. In our framework, the TSCognition training split is incorporated into Stage 3 to strengthen high-level temporal reasoning capability.
Figure 10. Distribution of TSCognition samples across eight domains and five hierarchical tasks.
| Tasks | Variable Numbers | Time Series Length | Choices | ||||||
| Min | Max | Avg. | Min | Max | Avg. | Min | Max | Avg. | |
| Decoding Task | 4 | 5 | 4.7 | 437 | 982 | 742 | 4 | 5 | 4.7 |
| Grounding Task | 3 | 6 | 4.4 | 656 | 1018 | 871 | 4 | 5 | 4.3 |
| Inferring Task | 3 | 6 | 5.1 | 882 | 1059 | 912 | 4 | 5 | 4.6 |
| Extrapolating Task | 3 | 6 | 5.2 | 601 | 1052 | 846 | 4 | 5 | 4.5 |
| Acting Task | 4 | 6 | 5.5 | 468 | 1047 | 795 | 4 | 5 | 4.2 |
| Overall | 3 | 6 | 4.9 | 437 | 1059 | 785 | 4 | 5 | 4.5 |
Table 12. Task-level statistics of time series in TSCognition.
TimeSeriesExam [2]. TimeSeriesExam is a configurable multiple-choice benchmark designed to evaluate fundamental time series understanding in LLMs. It covers five categories: pattern recognition, noise understanding, similarity analysis, anomaly detection, and causality analysis. The benchmark is procedurally generated from 104 expert-designed question templates and contains more than 700 questions. Its synthetic time series are constructed from diverse baseline signal components, including linear, nonlinear, and cyclic patterns, allowing controlled evaluation of different temporal properties. The detailed category statistics are summarized in Table 13.
| Category | Samples |
| Anomaly Detection | 108 |
| Causality | 72 |
| Noise Understanding | 84 |
| Pattern Recognition | 362 |
| Similarity Analysis | 120 |
| Total | 746 |
Table 13. Statistics of TimeSeriesExam.
CaTS-Bench [37]. CaTS-Bench is a real-world benchmark for context-aware time series captioning and reasoning across 11 diverse domains. Each sample combines a numeric time series segment with contextual metadata, a line-chart visualization, and a natural-language caption. In addition to caption generation, CaTS-Bench provides diagnostic multiple-choice tasks for evaluating fine-grained temporal understanding, including caption retrieval, time series retrieval, amplitude comparison, and mean comparison in Table 14. In our experiments, we use these diagnostic tasks to evaluate whether the learned temporal perception capability generalizes to unseen real-world data and cross-modal reasoning settings.
| Task | Samples |
| Caption Retrieval | 100 |
| Time Series Retrieval | 100 |
| Amplitude Comparison | 40 |
| Mean Comparison | 40 |
| Total | 280 |
Table 14. Statistics of CaTS-Bench used in our evaluation.
MTBench [4]. MTBench is a multimodal time series benchmark for evaluating temporal reasoning over numerical signals and textual context in the weather and finance domains. It includes weather question answering and prediction, as well as short- and long-horizon financial trend prediction and forecasting tasks. We use MTBench exclusively for out-of-domain evaluation to assess model generalization to unseen domains and reasoning settings. The evaluation tasks used in our experiments are summarized in Table 15.
| Domain | Evaluation Tasks |
| Weather | MCQA / Prediction |
| Finance (7 days) | 3-way / 5-way / Forecasting |
| Finance (30 days) | 3-way / 5-way / Forecasting |
Table 15. Evaluation settings of MTBench.
Detailed Results
Perception Results. Table 16 reports the detailed categorical and numerical perception results across different methods. It compares performance on trend, seasonality, noise, local patterns, correlation, and clustering, corresponding to the perception analysis shown in Figure 2.
Reasoning Scaling Results. Table 17 reports the detailed reasoning accuracy under different proportions of Stage 3 training data, corresponding to Figure 3. For both Pre-align and Post-align architectures, progressive Stage () training consistently outperforms Stage 3 Only and Mixed Training, with a larger advantage under limited reasoning supervision. The Directly setting shows weaker overall performance, further indicating that explicit temporal representation learning benefits progressive capability formation.
Stage-wise Saturation Results. Table 18 provides the detailed results corresponding to Table 4. Stage 1 and Stage () reach local saturation at approximately 70% of the corresponding training data, achieving 98.3% and 98.8% of their full-stage performance, respectively. Stage () reaches a similar saturation region at 85%, with 97.2% of its full-stage performance. These results further show that local convergence can occur before the complete progressive pathway reaches its best performance.
Supplementary Experiments
To examine whether the observed progressive capability formation is specific to the Qwen3-4B backbone, we further conduct supplementary experiments using Qwen3-1.7B. We follow the same three-stage training paradigm and evaluation protocol as the main experiments. The results across perception, task solving, and high-level reasoning show similar trends, indicating that the progressive capability relationships remain consistent under a smaller LLM backbone.
Stage 1: Fundamental Attribute Perception. Table 19 reports the Stage 1 perception results of Pre-align and Post-align. They show strong performance across categorical and numerical temporal attributes, including trend, seasonality, noise, local patterns, correlation, and clustering.
| Model | Trend | Season | Noise | Local | Multivariate | |||||
| Cate. | Num. | Cate. | Num. | Cate. | Num. | Cate. | Num. | Corr. | Clus. | |
| Pre-align | 0.894 | 0.843 | 0.859 | 0.843 | 0.822 | 0.558 | 0.786 | 0.727 | 0.788 | 0.761 |
| Post-align | 0.873 | 0.846 | 0.892 | 0.834 | 0.861 | 0.506 | 0.752 | 0.709 | 0.755 | 0.747 |
Table 19. Stage 1 perception results on Qwen3-1.7B.
Stage 2: Temporal Task Solving. Table 20 presents the Stage 2 results. Stage 1-only training already improves all four downstream tasks over the non-fine-tuned Qwen3-1.7B baseline. Further task-specific supervision provides substantial gains, while Stage () achieves competitive or better performance on most metrics.
| Model | Training Setting | Fore. MSE | Impu. MSE | Ano. Det. Acc. | Class. Acc. |
| Qwen3-1.7B | No Fine-tuning | 1.884 | 0.0312 | 0.303 | 0.234 |
| Pre-align | Stage 1 Only | 1.813 | 0.0218 | 0.548 | 0.421 |
| Stage 2 Only | 1.412 | 0.0151 | 0.651 | 0.542 | |
| Stage () | 1.438 | 0.0144 | 0.684 | 0.576 | |
| Post-align | Stage 1 Only | 1.782 | 0.0235 | 0.574 | 0.408 |
| Stage 2 Only | 1.469 | 0.0162 | 0.648 | 0.507 | |
| Stage () | 1.391 | 0.0150 | 0.626 | 0.559 |
Table 20. Stage 2 task-solving results on Qwen3-1.7B.
Stage 3: High-level Reasoning. Table 21 reports the reasoning results on the Stage 3 evaluation set B. Compared with Stage 3 Only, incorporating preceding perception or task-solving stages consistently improves reasoning performance. The complete Stage () pathway achieves the strongest overall results across the five reasoning dimensions.
Overall, the Qwen3-1.7B exhibit trends consistent with the main Qwen3-4B results. Fundamental perception provides transferable temporal knowledge for downstream task solving, while the
| Model | Training Setting | Decoding | Grounding | Inferring | Extrapolating | Acting |
| Pre-align | Stage 3 Only | 65.8 | 63.7 | 54.2 | 53.6 | 46.6 |
| Stage (13) | 73.8 | 76.9 | 60.7 | 58.9 | 49.4 | |
| Stage (23) | 74.9 | 69.5 | 65.6 | 58.1 | 52.1 | |
| Stage (123) | 78.1 | 74.2 | 63.1 | 65.2 | 55.6 | |
| Post-align | Stage 3 Only | 62.9 | 62.8 | 53.1 | 55.4 | 47.7 |
| Stage (13) | 72.5 | 70.3 | 59.4 | 61.5 | 50.9 | |
| Stage (23) | 70.4 | 69.8 | 64.2 | 63.4 | 57.3 | |
| Stage (123) | 75.4 | 74.3 | 61.6 | 67.7 | 53.2 |
caption: Stage 3 reasoning results on Evaluation Set B using Qwen3-1.7B.
combination of perception and task-solving supervision further supports high-level reasoning. These results suggest that the observed capability formation is not restricted to a specific model scale.
Reproducibility
We adopt unified implementation, training, and evaluation settings whenever possible to ensure fair and reproducible comparisons in Table 22. Unless otherwise specified, all experiments are conducted on two NVIDIA A800 GPUs using PyTorch, HuggingFace Transformers, DeepSpeed, and Accelerate. We use Qwen3-4B as the primary LLM backbone and additionally conduct supplementary experiments on Qwen3-1.7B. The same data splits, stage definitions, evaluation protocols, and prompt formats are maintained within each group of comparative experiments. We fix the random seed to 2026 and use bfloat16 mixed-precision training for all LLM-based experiments.
| Configuration | Setting |
| Primary backbone | Qwen3-4B |
| Additional backbone | Qwen3-1.7B |
| GPUs | 2 NVIDIA A800 |
| Framework | PyTorch + HuggingFace Transformers |
| Distributed training | DeepSpeed + Accelerate |
| Precision | bfloat16 |
| Optimizer | AdamW |
| Learning rate | 2 10 |
| Weight decay | 1 10 |
| Batch size per GPU | 4 |
| Gradient accumulation steps | 4 |
| Effective batch size | 32 |
| Gradient clipping | 1.0 |
| Learning rate scheduler | Cosine decay |
| Warmup ratio | 0.0 |
| Random seed | 2026 |
caption: Implementation and training configurations used in our experiments.
All capability stages are trained using supervised fine-tuning with the same optimization configuration unless otherwise specified. For the Pre-align architecture, the Temporal Encoder and alignment module are optimized in Stage 1, while the Temporal Encoder is frozen in subsequent stages and the remaining alignment components are updated. For the Post-align architecture, the Temporal Encoder and LLM backbone are jointly optimized. For the textualization setting, the numerical time series are directly converted into textual sequences and only the LLM is fine-tuned. Across different capability-development paths, we keep the model architecture, data split, initialization, and evaluation procedure unchanged so that performance differences mainly reflect the training stage and supervision strategy. This ensures a controlled comparison across different training paradigms.
For evaluation, Stage 1 separately measures categorical and numerical temporal perception, Stage 2 evaluates forecasting, imputation, anomaly detection, and classification, and Stage 3 evaluates high-level temporal reasoning. Forecasting and imputation use MSE, anomaly detection and classification use accuracy, while the reasoning dimensions are evaluated using multiple-choice accuracy when applicable. All reported comparative results are obtained under the same evaluation protocol within each experiment. This provides consistent evaluation criteria across all capability stages.
Case Study and Failure Analysis
Example of the Decoding task
Instruction: You are an expert in time series analysis and temporal reasoning. A temporal token representing the input time series is provided to the model. Please answer the multiple-choice question based on the temporal evidence, context, and candidate answers. Only one answer is correct.
Variable Information: : Load Demand; : Thermal Generation; : Renewable Output; : Battery Power; : Electricity Price; : Frequency Deviation.

Figure 11 (PDF p. 23).
Context: The data describe the operation of a regional power system. The operator monitors load demand, thermal generation, renewable output, battery behavior, electricity price, and frequency deviation to evaluate supply-demand balance and dispatch decisions. Battery power reflects storage operation, while electricity price and frequency deviation may indicate system stress. The actual temporal state should be inferred from the provided time series representation.
Question: Which statement best characterizes the joint temporal behavior of renewable output and battery power?
Candidate Answers:
A. Renewable output stays low throughout the period, while the battery remains inactive with no clear charging or discharging phase.
B. Renewable output reaches its highest level only during the final high-demand period, while the battery mainly charges at the same time.
C. Renewable output and battery power increase monotonically together, indicating that storage output always follows renewable generation.
D. Renewable output first increases to a mid-period peak and then declines, while the battery tends to charge during the renewable-rich phase and discharge later.
E. Renewable output fluctuates randomly without a recognizable phase, while battery behavior shows no relation to the system state.
Output: Please output the correct candidate answer.
Answer: D. Renewable output forms a clear mid-period peak before decreasing. Battery power shifts from charging during the renewable-rich phase to discharging afterward.
Example of the Grounding task
Instruction: You are an expert in time series analysis and temporal reasoning. A temporal token representing the input time series is provided to the model. Please answer the multiple-choice question based on the temporal evidence, context, and candidate answers. Only one answer is correct.
Variable Information: : Load Demand; : Thermal Generation; : Renewable Output; : Battery Power; : Electricity Price; : Frequency Deviation.

Figure 12.
Context: The data describe the operation of a regional power system. The operator monitors load demand, thermal generation, renewable output, battery behavior, electricity price, and frequency deviation to evaluate supply-demand balance and dispatch decisions. Battery power reflects storage operation, while electricity price and frequency deviation may indicate system stress. The actual temporal state should be inferred from the provided time series representation.
Question: Which interpretation best grounds the observed temporal patterns in the operational context of power dispatch?
Candidate Answers:
A. A renewable-rich period is reflected by higher renewable output together with battery charging, suggesting that surplus renewable energy is being absorbed.
B. A renewable-rich period is reflected by high electricity price and large frequency deviation, because these always indicate abundant renewable supply.
C. A peak-load period is reflected by low demand and strong battery charging, suggesting that the system is under severe supply shortage.
D. A stable operating period is reflected by simultaneous increases in load demand, electricity price, and frequency deviation, regardless of renewable output.
E. The storage behavior cannot be grounded in the power-system context because battery power is unrelated to renewable output or demand.
Output: Please output the correct candidate answer.
Answer: A. The high-renewable interval coincides with sustained battery charging. This pattern indicates that the battery is absorbing surplus renewable energy.
Example of the Inferring task
Instruction: You are an expert in time series analysis and temporal reasoning. A temporal token representing the input time series is provided to the model. Please answer the multiple-choice question based on the temporal evidence, context, and candidate answers. Only one answer is correct.
Variable Information: : Load Demand; : Thermal Generation; : Renewable Output; : Battery Power; : Electricity Price; : Frequency Deviation.

Figure 1 (PDF p. 25).
Context: The data describe the operation of a regional power system. The operator monitors load demand, thermal generation, renewable output, battery behavior, electricity price, and frequency deviation to evaluate supply-demand balance and dispatch decisions. Battery power reflects storage operation, while electricity price and frequency deviation may indicate system stress. The actual temporal state should be inferred from the provided time series representation.
Question: What is the most plausible explanation for the system pressure indicated by electricity price and frequency deviation during the high-load phase?
Candidate Answers:
A. The pressure is mainly caused by decreasing demand, since lower load usually forces the system to increase electricity price and frequency deviation.
B. The pressure is likely caused by the coincidence of high load demand, reduced renewable support, and increased dependence on dispatchable resources.
C. The pressure is mainly caused by excessive renewable generation, because abundant renewable output always increases electricity price and frequency deviation.
D. The pressure is unrelated to supply-demand balance and is only caused by random fluctuations in the battery power curve.
E. The pressure disappears because battery discharge and thermal generation fully eliminate all frequency deviation.
Output: Please output the correct candidate answer.
Answer: B. Load demand rises while renewable output weakens during the high-pressure phase. The concurrent increases in thermal generation, electricity price, and frequency deviation indicate greater reliance on dispatchable resources.
Example of the Extrapolating task
Instruction: You are an expert in time series analysis and temporal reasoning. A temporal token representing the input time series is provided to the model. Please answer the multiple-choice question based on the temporal evidence, context, and candidate answers. Only one answer is correct.
Variable Information: : Load Demand; : Thermal Generation; : Renewable Output; : Battery Power; : Electricity Price; : Frequency Deviation.

Figure 13.
Context: The data describe the operation of a regional power system. The operator monitors load demand, thermal generation, renewable output, battery behavior, electricity price, and frequency deviation to evaluate supply-demand balance and dispatch decisions. Battery power reflects storage operation, while electricity price and frequency deviation may indicate system stress. The actual temporal state should be inferred from the provided time series representation.
Question: Assume that the current high-load phase continues for the next interval and renewable output does not recover. Which future evolution is most consistent with the observed temporal evidence?
Candidate Answers:
A. The system will remain risk-free even if demand keeps increasing, because frequency deviation is independent of supply-demand imbalance.
B. Renewable output will necessarily rebound immediately, so electricity price and frequency deviation should both drop to their minimum values.
C. Battery power is more likely to switch to strong charging, because a high-load phase implies surplus electricity in the system.
D. Thermal generation will become unnecessary, since declining renewable output reduces the need for dispatchable resources.
E. Electricity price and frequency deviation may further increase unless additional dispatchable generation or storage discharge is available.
Output: Please output the correct candidate answer.
Answer: E. The final interval shows persistently high demand and continuously weakening renewable support. If this trend continues without additional controllable supply, electricity price and frequency deviation are likely to increase further.
Example of the Acting task
Instruction: You are an expert in time series analysis and temporal reasoning. A temporal token representing the input time series is provided to the model. Please answer the multiple-choice question based on the temporal evidence, context, and candidate answers. Only one answer is correct.
Variable Information: : Load Demand; : Thermal Generation; : Renewable Output; : Battery Power; : Electricity Price; : Frequency Deviation.

Figure 14 (PDF p. 27).
Context: The data describe the operation of a regional power system. The operator monitors load demand, thermal generation, renewable output, battery behavior, electricity price, and frequency deviation to evaluate supply-demand balance and dispatch decisions. Battery power reflects storage operation, while electricity price and frequency deviation may indicate system stress. The actual temporal state should be inferred from the provided time series representation.
Question: Which operational decision best matches the observed system state and dispatch objective?
Candidate Answers:
A. Increase battery charging and reduce dispatchable generation, because the system appears to have large surplus supply during the high-load phase.
B. Maintain the current dispatch without intervention, because electricity price and frequency deviation provide no useful signal about operating pressure.
C. Discharge the battery and increase dispatchable generation to relieve the high-load pressure while monitoring frequency stability.
D. Shut down renewable generation completely, because removing renewable output is the most direct way to stabilize price and frequency.
E. Reduce both thermal generation and battery output, because lower controllable supply will automatically reduce demand-side pressure.
Output: Please output the correct candidate answer.
Answer: C. High demand and weak renewable output coincide with elevated electricity price and frequency deviation. Increasing dispatchable generation and battery discharge is therefore the appropriate response to relieve system pressure.
Failure Case
Instruction: You are an expert in time series analysis and temporal reasoning. A temporal token representing the input time series is provided to the model. Please answer the multiple-choice question based on the temporal evidence, context, and candidate answers. Only one answer is correct.
Variable Information: : Surface Temperature; : Relative Humidity; : Precipitation; : Wind Speed; : Solar Radiation.

Figure 15.
Context: The data describe the operation of a regional weather monitoring system. The operator monitors surface temperature, relative humidity, precipitation, wind speed, and solar radiation to evaluate atmospheric conditions and weather evolution. Temperature and solar radiation reflect thermal conditions, while humidity and precipitation characterize moisture variation. The actual temporal state should be inferred from the provided time series representation.
Question: Which statement best describes the temporal relationship between surface temperature and solar radiation over the observed period?
Candidate Answers:
A. Surface temperature gradually increases throughout the period, while solar radiation decreases after an early peak, indicating a delayed thermal response to weakening radiation.
B. Solar radiation increases toward the middle of the period and then decreases, while surface temperature follows a similar pattern with a warming phase followed by cooling.
C. Surface temperature decreases during the period, while solar radiation reaches its maximum value near the end, suggesting reduced thermal input and delayed warming.
D. Solar radiation remains relatively stable with small fluctuations, while surface temperature exhibits strong variations unrelated to radiation changes.
E. Surface temperature and solar radiation both increase continuously throughout the period, showing a consistent upward relationship between thermal conditions and radiation intensity.
Answer: E. Surface temperature and solar radiation exhibit a broadly aligned upward trend despite several local fluctuations.
Ground Truth: B.
Failure Analysis: The model captures the initial increasing trend but fails to recognize the subsequent reversal shared by surface temperature and solar radiation. It therefore overemphasizes the early local pattern and incorrectly predicts a continuous increase. This failure indicates difficulty in tracking coordinated phase transitions across variables over the full temporal horizon.
References
- [1]Syeda Nahida Akter, Shrimai Prabhumoye, Eric Nyberg, Mostofa Patwary, Mohammad Shoeybi, Yejin Choi, and Bryan Catanzaro. Front-loading reasoning: The synergy between pretraining and post-training data. In International Conference on Learning Representations, volume 2026, pages 80909–80928, 2026.arxiv.org/abs/2510.03264
- [2]Yifu Cai, Arjun Choudhry, Mononito Goswami, and Artur Dubrawski. Timeseriesexam: A time series understanding exam. arXiv preprint arXiv:2410.14752, 2024.arxiv.org/abs/2410.14752
- [3]Fan Chen, Audrey Huang, Noah Golowich, Sadhika Malladi, Adam Block, Jordan Ash, Akshay Krishnamurthy, and Dylan Foster. The coverage principle: How pre-training enables post-training. In International Conference on Learning Representations, volume 2026, pages 112225–112293, 2026.arxiv.org/abs/2510.15020
- [4]Jialin Chen, Aosong Feng, Ziyu Zhao, Juan Garza, Gaukhar Nurbek, Cheng Qin, Ali Maatouk, Leandros Tassiulas, Yifeng Gao, and Rex Ying. Mtbench: A multimodal time series benchmark for temporal reasoning and question answering. arXiv preprint arXiv:2503.16858, 2025.arxiv.org/abs/2503.16858
- [5]Zhao Chen, Vijay Badrinarayanan, Chen-Yu Lee, and Andrew Rabinovich. Gradnorm: Gradient normalization for adaptive loss balancing in deep multitask networks. In International conference on machine learning, pages 794–803. PMLR, 2018.
- [6]Yunfei Chu, Jin Xu, Xiaohuan Zhou, Qian Yang, Shiliang Zhang, Zhijie Yan, Chang Zhou, and Jingren Zhou. Qwen-audio: Advancing universal audio understanding via unified large-scale audio-language models. arXiv preprint arXiv:2311.07919, 2023.arxiv.org/abs/2311.07919
- [7]Tong Guan, Zijie Meng, Dianqi Li, Shiyu Wang, Chao-Han Huck Yang, Qingsong Wen, Zuozhuo Liu, Sabato Marco Siniscalchi, Ming Jin, and Shirui Pan. Timeomni-1: Incentivizing complex reasoning with time series in large language models. arXiv preprint arXiv:2509.24803, 2025.arxiv.org/abs/2509.24803
- [8]Zelin He, Sarah Alnegheimish, and Matthew Reimherr. Harnessing vision-language models for time series anomaly detection. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 40, pages 21690–21698, 2026.
- [9]Yuxiao Hu, Qian Li, Dongxiao Zhang, Jinyue Yan, and Yuntian Chen. Context-alignment: Activating and enhancing llms capabilities in time series. In International Conference on Learning Representations, volume 2025, pages 90696–90722, 2025.arxiv.org/abs/2501.03747
- [10]Yushan Jiang, Kanghui Ning, Zijie Pan, Xuyang Shen, Jingchao Ni, Wenchao Yu, Anderson Schneider, Haifeng Chen, Yuriy Nevmyvaka, and Dongjin Song. Multi-modal time series analysis: A tutorial and survey. In Proceedings of the 31st ACM SIGKDD Conference on Knowledge Discovery and Data Mining V. 2, pages 6043–6053, 2025.
- [11]Ming Jin, Shiyu Wang, Lintao Ma, Zhixuan Chu, James Zhang, Xiaoming Shi, Pin-Yu Chen, Yuxuan Liang, Yuan-Fang Li, Shirui Pan, and Qingsong Wen. Time-llm: Time series forecasting by reprogramming large language models. In B. Kim, Y. Yue, S. Chaudhuri, K. Fragkiadaki, M. Khan, and Y. Sun, editors, International Conference on Learning Representations, volume 2024, pages 23857–23880, 2024. URL https://proceedings.iclr.cc/paper_files/paper/2024/file/680b2a8135b9c71278a09cafb605869e-Paper-Conference.pdf.arxiv.org/abs/2310.01728
- [12]Zixuan Ke, Bing Liu, Nianzu Ma, Hu Xu, and Lei Shu. Achieving forgetting prevention and knowledge transfer in continual learning. Advances in Neural Information Processing Systems, 34:22443–22456, 2021.arxiv.org/abs/2112.02706
- [13]Yaxuan Kong, Yiyuan Yang, Yoontae Hwang, Wenjie Du, Stefan Zohren, Zhangyang Wang, Ming Jin, and Qingsong Wen. Time-mqa: Time series multi-task question answering with context enhancement. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 29736–29753, 2025.
- [14]Yaxuan Kong, Yiyuan Yang, Shiyu Wang, Chenghao Liu, Yuxuan Liang, Ming Jin, Stefan Zohren, Dan Pei, Yan Liu, and Qingsong Wen. Achieving time series reasoning requires rethinking model design, tasks formulation, and evaluation. arXiv preprint arXiv:2502.01477, 2025.arxiv.org/abs/2502.01477
- [15]Bo Liu, Xingchao Liu, Xiaojie Jin, Peter Stone, and Qiang Liu. Conflict-averse gradient descent for multi-task learning. Advances in neural information processing systems, 34: 18878–18890, 2021.arxiv.org/abs/2110.14048
- [16]Chenxi Liu, Qianxiong Xu, Hao Miao, Sun Yang, Lingzheng Zhang, Cheng Long, Ziyue Li, and Rui Zhao. Timecema: Towards llm-empowered multivariate time series forecasting via cross-modality alignment. In Proceedings of the AAAI Conference on Artificial Intelligence, volume 39, pages 18780–18788, 2025.arxiv.org/abs/2406.01638
- [17]Hao Liu, Chun Yang, Xiaobin Zhu, et al. Semantic-enhanced time-series forecasting via large language models. In International Conference on Learning Representations, volume 2026, pages 101404–101425, 2026.arxiv.org/abs/2508.07697
- [18]Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee. Visual instruction tuning. Advances in Neural Information Processing Systems, 36:34892–34916, 2023.
- [19]Haoxin Liu, Chenghao Liu, and B Aditya Prakash. A picture is worth a thousand numbers: Enabling llms reason about time series via visualization. In Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies (Volume 1: Long Papers), pages 7486–7518, 2025.arxiv.org/abs/2411.06018
- [20]Wenjie Liu, Hao Wu, Xin Qiu, Xudong Wang, Yingqi Fan, Yihan Zhang, Anhao Zhao, Yunpu Ma, and Xiaoyu Shen. Vica: Efficient multimodal llms with vision-only cross-attention. arXiv preprint arXiv:2602.07574, 2026.
- [21]Yuan Liu, Haodong Duan, Yuanhan Zhang, Bo Li, Songyang Zhang, Wangbo Zhao, Yike Yuan, Jiaqi Wang, Conghui He, Ziwei Liu, Kai Chen, and Dahua Lin. Membench: Is your multi-modal model an all-around player? In European Conference on Computer Vision, 2024.
- [22]Junkai Lu, Peng Chen, Xingjian Wu, Yang Shu, Chenjuan Guo, Christian S Jensen, and Bin Yang. Patra: Pattern-aware alignment and balanced reasoning for time series question answering. arXiv preprint arXiv:2602.23161, 2026.
- [23]Romain Meunier, Farah Benamara, Véronique Moriceau, Zhongzheng Qiao, and Savitha Ramasamy. Crisists: Coupling social media textual data and meteorological time series for urgency classification. In Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 16082–16099, 2025.
- [24]Xin Qiu, Junlong Tong, Yirong Sun, Yunpu Ma, and Xiaoyu Shen. The few govern the many: Unveiling few-layer dominance for time series models. arXiv preprint arXiv:2511.07237, 2025.arxiv.org/abs/2511.07237
- [25]Xin Qiu, Junlong Tong, Yirong Sun, Yunpu Ma, Wei Zhang, and Xiaoyu Shen. Rethinking the role of llms in time series forecasting. arXiv preprint arXiv:2602.14744, 2026.
- [26]Xin Qiu, Junlong Tong, Yao Zhang, Yunpu Ma, Wei Zhang, and Xiaoyu Shen. From recognition to understanding: Unlocking cognitive time series reasoning with llms. arXiv preprint arXiv:2606.22126, 2026.arxiv.org/abs/2606.22126
- [27]Paul Quinlan, Qingguo Li, and Xiaodan Zhu. Chat-ts: Enhancing multi-modal reasoning over time-series and natural language data. In Proceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics (Volume 1: Long Papers), pages 5621–5647, 2026.
- [28]S Sakshi, Utkarsh Tyagi, Sonal Kumar, Ashish Seth, Ramaneswaran Selvakumar, Oriol Nieto, Ramani Duraiswami, Sreyan Ghosh, and Dinesh Manocha. Mmau: A massive multi-task audio understanding and reasoning benchmark. In International Conference on Learning Representations, 2025.arxiv.org/abs/2410.19168
- [29]Ozan Sener and Vladlen Koltun. Multi-task learning as multi-objective optimization. Advances in neural information processing systems, 31, 2018.arxiv.org/abs/1810.04650
- [30]Yilin Wang, Peixuan Lei, Jie Song, Yuzhe Hao, Tao Chen, Yuxuan Zhang, Lei Jia, Yuanxiang Li, and Zhongyu Wei. Itformer: Bridging time series and natural language for multi-modal qa with large-scale multitask dataset. In International Conference on Machine Learning, pages 63324–63344. PMLR, 2025.arxiv.org/abs/2506.20093
- [31]Zhe Xie, Zeyan Li, Xiao He, Longlong Xu, Xid ao Wen, Tieying Zhang, Jianjun Chen, Rui Shi, and Dan Pei. Chatts: Aligning time series with llms via synthetic data for enhanced understanding and reasoning. Proceedings of the VLDB Endowment, 18(8):2385–2398, 2025.DOI
- [32]Xiongxiao Xu, Haoran Wang, Yueqing Liang, Philip S Yu, Yue Zhao, and Kai Shu. Can multimodal llms perform time series anomaly detection? In Proceedings of the ACM Web Conference 2026, pages 5392–5403, 2026.arxiv.org/abs/2502.17812
- [33]An Yang, Anfeng Li, Baosong Yang, Beichen Zhang, Binyuan Hui, Bo Zheng, Bowen Yu, Chang Gao, Chengen Huang, Chenxu Lv, et al. Qwen3 technical report. arXiv preprint arXiv:2505.09388, 2025.arxiv.org/abs/2505.09388
- [34]Fangxu Yu, Xingang Guo, Lingzhi Yuan, Haoqiang Kang, Hongyu Zhao, Lianhui Qin, Furong Huang, Bin Hu, and Tianyi Zhou. Tsrbench: A comprehensive multi-task multi-modal time series reasoning benchmark for generalist models. arXiv preprint arXiv:2601.18744, 2026.arxiv.org/abs/2601.18744
- [35]Tianhe Yu, Saurabh Kumar, Abhishek Gupta, Sergey Levine, Karol Hausman, and Chelsea Finn. Gradient surgery for multi-task learning. Advances in neural information processing systems, 33:5824–5836, 2020.arxiv.org/abs/2001.06782
- [36]Zhen Zhao, Zhizhong Zhang, Xin Tan, Jun Liu, Yanyun Qu, Yuan Xie, and Lizhuang Ma. Rethinking gradient projection continual learning: Stability/plasticity feature space decoupling. In 2023 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pages 3718–3727. IEEE, 2023.DOI
- [37]Luca Zhou, Pratham Yashwante, Marshall Fisher, Alessio Sampieri, Zihao Zhou, Fabio Galasso, and Rose Yu. Cats-bench: Can language models describe time series? In Findings of the Association for Computational Linguistics: ACL 2026, pages 34479–34519, 2026.arxiv.org/abs/2509.20823
- [38]Tian Zhou, Peisong Niu, Liang Sun, Rong Jin, et al. One fits all: Power general time series analysis by pretrained lm. Advances in neural information processing systems, 36: 43322–43355, 2023.arxiv.org/abs/2302.11939








