betaFeedback

WorksComputer Science

SPH-Net: A Co-Attention Hybrid Model for Accurate Stock Price Prediction

Abstract

Prediction of stock price movements presents a formidable challenge in financial analytics due to the inherent volatility, non-stationarity, and nonlinear characteristics of market data. This paper introduces SPH-Net (Stock Price Prediction Hybrid Neural Network), an innovative deep learning framework designed to enhance the accuracy of time series forecasting in financial markets. The proposed architecture employs a novel co-attention mechanism that initially processes temporal patterns through a Vision Transformer, followed by refined feature extraction via an attention mechanism, thereby capturing both global and local dependencies in market data. To rigorously evaluate the model's performance, we conduct comprehensive experiments on eight diverse stock datasets: AMD, Ebay, Facebook, FirstService Corp, Tesla, Google, Mondi ADR, and Matador Resources. Each dataset is standardized using six fundamental market indicators: Open, High, Low, Close, Adjusted Close, and Volume, representing a complete set of features for comprehensive market analysis. Experimental results demonstrate that SPH-Net consistently outperforms existing stock prediction models across all evaluation metrics. The model's superior performance stems from its ability to effectively capture complex temporal patterns while maintaining robustness against market noise. By significantly improving prediction accuracy in financial time series analysis, SPH-Net provides valuable decision-support capabilities for investors and financial analysts, potentially enabling more informed investment strategies and risk assessment in volatile market conditions.

Introduction

Stock price prediction represents a fundamental yet challenging research domain in computational finance, owing to the inherent volatility, non-linearity, and stochastic nature of financial markets [1]. While traditional statistical approaches, including ARIMA models [16] and econometric frameworks [10], have established foundational methodologies for time series analysis, their capacity to capture complex market dynamics and long-range dependencies remains limited [17, 23].

The advent of artificial intelligence, particularly deep learning, has catalyzed significant advancements in financial forecasting. Machine learning algorithms such as Support Vector Machines [6], Random Forests [14, 35], and Gradient Boosting methods [4] have demonstrated improved performance by modeling non-linear relationships in market data. Subsequent developments in recurrent neural architectures, including RNNs [8] and their advanced variants (LSTMs [23] and GRUs [24]), have further enhanced temporal pattern recognition capabilities.

The recent emergence of Transformer-based models [17] has introduced new possibilities through their attention mechanisms, which effectively capture global contextual relationships across extended time sequences. However, current approaches still exhibit limitations in processing heterogeneous financial data and adapting to real-world market volatility [9, 17].

To overcome these challenges, we present SPH-Net (Stock Price Hybrid Neural Network), an innovative hybrid architecture that synergistically combines Vision Transformer (ViT) [9] with Transformer encoder-decoder mechanisms [17]. Our framework employs co-attention mechanisms to simultaneously model temporal dependencies and cross-feature interactions in financial time series data.

The principal contributions of this work are threefold:

  • A novel hybrid deep learning architecture that integrates ViT and Transformer components for comprehensive stock price prediction

  • An innovative data preprocessing methodology that transforms temporal financial data into image-like patch representations, facilitating enhanced pattern extraction through ViT

  • Comprehensive empirical validation across eight diverse stock datasets, demonstrating SPH-Net’s superior performance over state-of-the-art models in both regression and classification scenarios

Related Work

Contemporary AI Models for Stock Price Forecasting

Early approaches to stock price prediction predominantly employed classical statistical methods, including Autoregressive Integrated Moving Average (ARIMA) models [16] and linear regression techniques [2, 5, 10]. While these methods established foundational frameworks for time series analysis, their inherent linearity constrained their capacity to model the complex, non-linear dynamics characteristic of financial markets.

The emergence of machine learning paradigms introduced more sophisticated algorithms, notably Support Vector Machines (SVMs) [6], Decision Trees [7], and Random Forests [14]. These techniques demonstrated improved predictive performance through enhanced pattern recognition capabilities. Nevertheless, their effectiveness remained limited when modeling sequential dependencies in temporal data.

Recent advances in deep learning have revolutionized the field, with Recurrent Neural Networks (RNNs) [8], Long Short-Term Memory networks (LSTMs) [11], [23], Gated Recurrent Units (GRUs) [24], and hybrid CNN-LSTM architectures [18] demonstrating superior temporal modeling performance. Specifically, LSTM and GRU variants excel at capturing long-range dependencies in financial time series, while CNN-LSTM hybrids benefit from synergistic spatial-temporal feature extraction.

The introduction of Transformer-based architectures [9], [17], [21], [29], [31] has marked a significant paradigm shift, with self-attention mechanisms enabling more effective modeling of long-range dependencies. Notably, BERT [21] has been adapted for time-series analysis, though its conventional architecture lacks the encoder-decoder structure typically required for forecasting applications.

Limitations of Current Approaches

Contemporary models have made notable progress in addressing these challenges. MagicNet [28] incorporates causal graph networks with memory-aware components to capture spatiotemporal dependencies. CMIN [31] leverages textual data and inter-stock causal relationships through innovative fusion techniques. HATR-I [34] implements a sophisticated dual-attention mechanism combined with Hawkes processes and domain-aware graph modules. DeepClue [33] enhances model interpretability by visualizing influential prediction factors derived from news and social media data. Despite these advancements, persistent limitations remain in terms of: (1) scalability to large-scale financial datasets, (2) effective integration of heterogeneous data modalities, and (3) robustness to market anomalies and regime changes.

Research Motivation

Building upon these developments, we propose SPH-Net, which synergistically combines the hierarchical feature extraction capabilities of Vision Transformers (ViT) [9] with the sequential modeling strengths of Transformer architectures [17]. Our framework introduces a novel co-attention mechanism operating across both temporal and feature dimensions, addressing key limitations in current approaches. Similar to established encoder-decoder frameworks [18], [19], [22], [30], SPH-Net employs ViT as the encoder for high-dimensional feature extraction, while utilizing a Transformer-based decoder for sequential prediction generation. This integrated approach aims to achieve superior predictive accuracy while maintaining robust generalization across diverse market conditions.

Methodology

As illustrated in Figure 1, the SPH-Net architecture comprises two principal components: the Trend Recognition Feature Extractor and the Deep Time-Series Predictor. The input data undergoes initial preprocessing to transform it into a pixel matrix compatible with Vision Transformer (ViT) input requirements [9]. Subsequently, the processed data flows through a Transformer-based architecture [17], where the outputs can be decoded into various representations including graphical visualizations, numerical outputs, or other formats as required by specific application scenarios.

Figure 1. Architectural overview of the proposed SPH-Net framework.

Trend Recognition Feature Extractor

While ViT [9] was originally designed for visual data processing, we adapt this architecture for time-series analysis by treating N-day stock price sequences as image-like patches. The transformation begins by segmenting the input time-series data X∈RT×dX \in\mathbb{R}^{T \times d} into pp distinct patches, as formalized in Eq. (1):

X=[X1,X2,…,XP],Xi∈Rp×d(1)X = [X_1, X_2, \ldots, X_P], \qquad X_i \in\mathbb{R}^{p \times d} \tag*{(1)}

where TT denotes the temporal dimension and dd represents the feature dimensionality. The segmented data is then projected into a higher-dimensional space through the embedding operation defined in Eq. (2):

Z=WEX+bE(2)Z = W_E X + b_E \tag*{(2)}

Here, WEW_E constitutes a learnable projection matrix, bEb_E represents the bias term, and ZZ denotes the resulting embeddings for ViT processing. To compensate for the Transformer’s inherent lack of sequential awareness, we incorporate learnable positional encodings as specified in Eq. (3):

Z=Z+PE(3)Z = Z + PE \tag*{(3)}

The positional encoding at time step tt is computed as:

PEt=WPEt(4)PE_t = W_{PE}t \tag*{(4)}

Deep Time-Series Predictor

The feature representations HViTH_{\mathrm{ViT}}, equivalently denoted as ViT(Z)V_i^T(Z), obtained from the ViT processing [9] are subsequently fed into a Transformer encoder [17] to capture global temporal dependencies. The attention mechanism operates as follows:

Attention⁡(Q,K,V)=softmax⁡(QKTdk)V(5)\operatorname{Attention}(Q, K, V) = \operatorname{softmax}\left(\frac{QK^T}{\sqrt{d_k}}\right)V \tag*{(5)}

where Q=HViTWQQ = H_{\mathrm{ViT}}W_Q, K=HViTWKK = H_{\mathrm{ViT}}W_K, and V=HViTWVV = H_{\mathrm{ViT}}W_V represent the query, key, and value matrices respectively. The projection matrices WQW_Q, WKW_K, and WVW_V are trainable parameters, while dkd_k denotes the dimensionality of each attention head. The softmax operation computes attention weights, effectively determining the relative importance of different temporal segments. The architecture further incorporates a multi-head self-attention mechanism (MHSA) to capture diverse temporal patterns:

MHSA⁡(HViT)=Concat⁡(head⁡1,…,head⁡h)WO(6)\operatorname{MHSA}(H_{\mathrm{ViT}}) = \operatorname{Concat}(\operatorname{head}_1, \ldots, \operatorname{head}_h)W_O \tag*{(6)}
head⁡i=Attention⁡(HViTWQi,HViTWKi,HViTWVi)(7)\operatorname{head}_i = \operatorname{Attention}(H_{\mathrm{ViT}}W_Q^i, H_{\mathrm{ViT}}W_K^i, H_{\mathrm{ViT}}W_V^i) \tag*{(7)}

To enhance training stability and mitigate gradient-related issues, the Transformer employs layer normalization (LayerNorm) followed by a feedforward network (FFN):

HViT′=LayerNorm⁡(HViT+MHSA⁡(HViT))(8)H'_{\mathrm{ViT}} = \operatorname{LayerNorm}(H_{\mathrm{ViT}} + \operatorname{MHSA}(H_{\mathrm{ViT}})) \tag*{(8)}
FFN⁡(HViT′)=σ(W1HViT′+b1)W2+b2(9)\operatorname{FFN}(H'_{\mathrm{ViT}}) = \sigma(W_1H'_{\mathrm{ViT}} + b_1)W_2 + b_2 \tag*{(9)}
HT=LayerNorm⁡(HViT′+FFN⁡(HViT′))(10)H_T = \operatorname{LayerNorm}(H'_{\mathrm{ViT}} + \operatorname{FFN}(H'_{\mathrm{ViT}})) \tag*{(10)}

The final temporal features HTH_T are subsequently transformed into stock price predictions through a fully connected layer:

y^=WyHT+by(11)\hat{y} = W_yH_T + b_y \tag*{(11)}

where WyW_y and byb_y represent the output layer’s weight matrix and bias vector, respectively.

Dataset and Implementation

Dataset

The experimental data is sourced from the Center for Research in SecurityPrices, LLC2 (CRSP). Our investigation encompasses stock market data from eight publicly traded companies spanning diverse industrial sectors, including semiconductor technology, e-commerce platforms, social media, real estate investment trusts, electric vehicle manufacturing, internet services, sustainable packaging, and energy exploration. The selected companies (AMD, Ebay, FB, FSV, Tesla, Google, Mondy, and MTDR) provide comprehensive coverage of major market sectors, ensuring robust evaluation of our model across varying market conditions and industrial landscapes.

The dataset comprises six standardized financial metrics: Open, High, Low, Close, Adj Close, and Volume. These metrics collectively provide a multidimensional view of market dynamics: the opening and closing prices bookend daily trading activity, while the high and low values delineate the intraday price range. The adjusted close price incorporates corporate actions such as dividends and stock splits, facilitating accurate historical comparisons. Trading volume serves as a crucial indicator of market liquidity and investor interest. These metrics are intrinsically interrelated - for instance, the high-low differential quantifies daily volatility, while the open-close relationship indicates prevailing market sentiment.

Figure 2 presents the statistical distributions of stock prices across all eight datasets, visualized through both price range charts and histograms. These visualizations reveal key characteristics of each stock’s price behavior, including central tendency, dispersion, and outlier patterns. The comprehensive statistical analysis of these financial metrics forms the foundation for technical analysis and predictive modeling [12, 15].

Statistical distributions of stock prices across eight datasets, with box plots and histograms for AMD, Ebay, FB, FSV, Tesla, Google, Mondy, and MTDR

Figure 2. Statistical distribution of stock prices across eight datasets.

Implementation

  1. Data Preprocessing: The implementation pipeline begins with data ingestion from eight CSV files, from which we extract the six key financial features. Missing values are addressed through forward-filling imputation, followed by MinMax normalization3 to ensure numerical stability during training. We employ a sliding window approach to convert the time-series data into sequential samples, which are then transformed into PyTorch4 tensors. The Dataset and DataLoader utilities facilitate efficient batch processing during model training.

  1. Model Configuration: The model architecture employs a patch embedding dimension of 128, with the Transformer component consisting of 4 layers and 8 parallel attention heads. The hidden dimension (dmodeld_{\mathrm{model}}) is set to 128 to effectively capture complex temporal dependencies. Optimization is performed using the Adam algorithm [26] with a learning rate of 0.001 and a batch size of 32.

  1. Evaluation Matrix: We employ two principal metrics for model evaluation:

Mean Squared Error (MSE): This metric quantifies the average squared deviation between predicted and actual stock prices, as defined in Eq. (12):

MSE=1n∑i=1n(yi−y^i)2(12)MSE = \frac{1}{n}\sum_{i=1}^{n}(y_i-\hat{y}_i)^2 \tag*{(12)}

where yiy_i denotes the observed stock price, y^i\hat{y}_i represents the model’s prediction, and nn indicates the number of observations. The squaring operation emphasizes larger errors, making MSE particularly sensitive to significant prediction inaccuracies.

Coefficient of Determination (R2R^2): This metric evaluates the proportion of variance in stock prices explained by the model, as formalized in Eq. (13):

R2=1−SSresSStot(13)R^2 = 1 - \frac{SS_{res}}{SS_{tot}} \tag*{(13)}

where SSresSS_{res} quantifies the residual variation unexplained by the model, and SStotSS_{tot} represents the total variation in the observed data. An R2R^2 value approaching 1 indicates excellent explanatory power, while values near 0 suggest limited predictive capability.

To evaluate the directional accuracy of our predictions (i.e., whether the model correctly predicts upward or downward price movements compared to actual market movements), we frame this assessment as a binary classification task. This approach allows us to measure the model’s ability to correctly identify market trends using a confusion matrix, which provides a comprehensive performance evaluation through the following metrics:

  • True Positive (TP): the TP indicates that Class A is

correctly identified as belonging to Class A.

  • True Negative (TN): this matrix indicates that Class A is correctly identified as not belonging to Class A.

  • False Positive (FP): it indicates that Class A is not correctly identified as belonging to Class A.

  • False Negative (FN): it indicates that Class A is not correctly identified as not belonging to Class A.

Especially in classification tasks, three key evaluation metrics are precision, accuracy, and recall. For precision, it measures how many of the predicted positive cases are actually positive. High precision means fewer FP. Its formula is shown below in Eq. (14):

Precision=TPTP+FP(14)\mathrm{Precision} = \frac{TP}{TP + FP} \tag*{(14)}

For accuracy, it measures the overall correctness of the model by calculating the percentage of correctly classified samples (both positive and negative). Its formula is shown below in Eq. (15):

Accuracy=TP+TNTP+TN+FP+FN(15)\mathrm{Accuracy} = \frac{TP + TN}{TP + TN + FP + FN} \tag*{(15)}

For recall, it measures how many actual positive cases were correctly identified by the model. Its formula is shown below in Eq. (16):

Recall=TPTP+FN(16)\mathrm{Recall} = \frac{TP}{TP + FN} \tag*{(16)}

4) Hardware: the hardware specifications for training and testing include 4 4090-Ti GPUs (4×244 \times24GB), 64GB of RAM, 8 CPU cores per node, and a total of 6 nodes.

Experimental Result

In this section, we conduct comprehensive experiments to evaluate the effectiveness and robustness of the proposed SPH-Net model. Our evaluation focuses on two key aspects: (1) regression performance for precise price prediction and (2) classification accuracy for trend direction identification. Through systematic comparisons with state-of-the-art baselines, detailed ablation studies, and analytical visualizations, we demonstrate the model’s capability to capture complex market patterns.

All experiments follow a standardized protocol across eight stock datasets spanning multiple sectors (technology, energy, e-commerce, etc.). The datasets are partitioned into 70% training and 30% testing sets, with the model trained for 100 epochs using the Adam optimizer (learning rate = 0.001) and a batch size of 32. This consistent setup ensures fair comparison and reproducible results.

Comparative Experiment

We first compare SPH-Net with traditional statistical methods (Linear Regression [2], Logistic Regression [5]), machine learning baselines (Decision Tree [7], Random Forest [14]), classical deep learning models (RNN [8], LSTM [23], GRU [24], CNN-LSTM [18], Bi-LSTM [27]), and recent state-of-the-art models including Transformer [17], BERT [21], DeepClue [33], HATR-I [34], MagicNet [28], and CMIN [31].

Table I presents the regression results on eight datasets. SPH-Net consistently achieves the best R2R^2 values and lowest MSE scores across all cases. For instance, on the AMD dataset, SPH-Net achieves an R2R^2 of 0.9949 and MSE of 31.23, outperforming all baselines. On the Google dataset, SPH-Net is the only model with a positive R2R^2 (1.02), while most others perform poorly or even negatively due to high volatility and sparse data patterns.

ModelDatasetAMDEbayFBFSVTESLAGoogleMondyMTDR
R2^2MSER2^2MSER2^2MSER2^2MSER2^2MSER2^2MSER2^2MSER2^2MSE
Linear Regression [2]97.6957.2799.8832.4498.48628.398.2263.6791.068047-1.115032391.35301.297.8366.22
Logistic Regression [5]97.1247.8998.8559.7497.99445.998.3452.1778.6415690-7.236987394.87259.397.8945.78
RNN [8]97.1239.8998.7341.2798.12712.997.1845.8989.3615743-3.287896391.87378.496.8457.14
Bi-RNN [20]98.4638.3299.0139.6898.41814.996.8959.6982.788763-17.259889595.69433.692.75174.9
CNN-RNN [13]98.5185.6999.1442.1998.36684.398.14106.992.6511065-3.164959596.11239.896.1858.35
LSTM [23]97.5344.5898.4239.1797.25417.997.65101.393.627471-0.693295695.58271.695.8487.48
Bi-LSTM [27]98.8961.2398.7847.3897.84389.795.8698.9991.2255490.852445195.87259.491.45145.2
LSTM-seq2seq [25]96.2659.6396.8551.2595.97409.687.41102.187.999541-0.293332695.69396.592.84163.5
CNN-LSTM [18]98.6936.9598.7944.5798.14563.498.8944.1994.414502-0.032958995.82393.1595.69112.8
CMIN [31]97.99138.9698.6748.9698.03487.395.5768.8994.026998-0.985968796.12175.994.3898.74
BERT [21]99.1938.1199.3454.9898.22337.497.8832.6894.5542950.98999596.14215.799.0938.65
GRU [24]97.8645.6298.7194.1096.89786.894.3569.5978.2656326-12.31403696.02211.896.8569.38
GRU-seq2seq [30]98.1271.7598.8269.8597.13110795.6245.2993.876563-0.991560495.67210.693.5958.49
Transformer [17]98.9735.9499.1632.7498.13374.997.8946.3693.365886-0.321102094.89185.998.4951.86
DeepClue [33]98.7636.1098.73132.697.79518.998.8842.6294.2554470.951321295.72298.698.5647.85
HATR-I [34]98.7435.4198.9875.9198.39478.698.2142.1293.8568590.231195395.81132.598.7441.85
MagicNet [28]98.4739.7897.3668.9697.98419.896.8768.9681.97149800.121054992.64236.998.06143.82
Decision Tree [7]91.94197.999.7571.8895.90170294.41199.966.8929829-292.89358284.68533.495.48138.1
Random Forest [14]96.5784.2599.8638.2697.84894.097.14102.186.1912440-41.443369489.32372.197.3780.23
SPH-Net99.4931.2399.9125.6999.47323.699.0131.2694.5941031.02997896.35127.399.1238.63

Table I. Comparative experiments of SPH-Net and other models on 8 datasets. (×100%)

For the classification task of predicting price direction (up or down), we evaluate precision, accuracy, and recall (Table II). SPH-Net outperforms all competing models across all datasets. On TESLA, SPH-Net reaches a precision of 97.17%, accuracy of 96.12%, and recall of 93.21%, substantially exceeding the closest competitor, DeepClue [33]. On FB, SPH-Net maintains a high classification precision of 96.27%, demonstrating its robustness even on platforms heavily influenced by external events. These results clearly demonstrate SPH-Net’s superior generalization ability, benefiting from its hybrid co-attention structure that effectively models both feature-wise and temporal dependencies.

DatasetModelSPH-NetDeepClue [33]MagicNet [28]HATR-I [34]
PrecisionAccuracyRecallPrecisionAccuracyRecallPrecisionAccuracyRecallPrecisionAccuracyRecall
AMD96.6494.8494.2194.8193.4088.7991.5787.2684.9897.3294.0893.04
Ebay97.6496.1294.5997.3592.5589.2593.2190.4288.2589.2785.5881.57
FB96.2792.3290.9391.4390.2087.0090.7486.1983.7986.5284.9580.90
FSV98.2696.1194.5694.1192.8288.5394.1892.8691.5297.8395.3392.85
TESLA97.1796.1293.2194.7992.2990.9595.1091.8789.1796.7895.3492.37
Google95.0392.0988.1286.5485.0781.4894.7091.3786.5289.8787.7383.26
Mondy98.1597.0991.8797.6196.4492.6491.8686.8985.5992.2087.3284.23
MTDR97.2394.1191.3193.1689.8285.2285.5983.4778.6796.5793.7590.27

Table II. Comparative experiments with other models for stock price prediction classification. (×100%)

Ablation Experiment

To investigate each architectural component’s contribution, we conducted detailed ablation studies (Table III). Following standard configurations, ViT’s initial patches are set to 8 [9], and the Transformer head number is also set to 8 [17]. We examined:

ModelDatasetAMDEbayFBFSVTESLAGoogleMondyMTDR
R2R^2MSER2R^2MSER2R^2MSER2R^2MSER2R^2MSER2R^2MSER2R^2MSER2R^2MSE
ViT [9]98.2339.6798.6548.3697.35402.596.2552.3990.786754-0.361364293.14244.797.8259.22
ViT (patches = 2)97.2749.1898.9937.8897.17392.096.0038.4898.5441320.491945695.79194.397.8796.20
ViT (patches = 4)99.0834.6799.5628.9598.74341.898.4536.5994.1552230.251036495.03152.997.8645.74
ViT (patches = 16)98.9732.3699.7027.3499.31328.798.8832.3394.4545650.981608296.17135.397.9139.47
ViT + RNN [8]99.4731.3999.8825.9399.45324.498.9931.4194.574169-0.012999296.32128.498.0038.75
ViT + GRU [24]98.7134.43100.3429.5797.75330.696.9734.2094.5452010.901040996.23143.297.9240.04
ViT + LSTM [23]99.2833.1799.5428.5199.20332.498.7933.1094.3448950.951015796.05141.397.8240.07
ViT + seq2seq []99.3332.6999.6327.8099.27330.298.8532.6494.4046970.961011296.12137.997.9839.71
ViT + Bi-RNN [24]98.2135.0599.2729.6996.41325.695.2634.7095.0252440.881060496.17141.596.2839.53
ViT + Bi-LSTM [24]99.3532.5299.6627.5799.29329.598.8732.4894.4346310.971009796.15136.597.9539.59
Transformer [17]98.9735.9499.1632.7498.13374.997.8946.3693.365886-0.321102094.89185.998.0151.86
Transformer (head=2)97.6447.1898.6541.2296.75623.195.2958.3692.146345-1.521203493.74224.596.7162.17
Transformer (head=4)99.0834.6799.5628.9598.74341.898.4536.5994.1552230.251036495.03152.997.9245.74
Transformer (head=16)98.4240.3299.0335.6798.03379.297.3547.8193.015987-0.551123094.36192.197.8553.78
Bert [21] + Transformer98.8533.1198.5827.6897.92324.797.1232.9694.8146690.951028696.23134.397.2039.09
CNN [18] + Transformer97.1548.0399.5741.3895.42614.993.6159.2192.516397-1.491226094.99221.295.4861.38
Decision Tree [7] + Transformer98.4736.5998.8732.8796.78369.496.1647.0493.745934-0.311122796.15183.7697.2451.20
Random Forest [14] + Transformer99.4537.5599.8526.1699.42325.898.9731.5794.5542351.011400796.30129.598.0258.87
Linear Regression [2] + Transformer97.7648.3198.0837.7395.53397.597.7337.9294.1441080.501909898.48197.299.8297.43
Logistic Regression [5] + Transformer99.3132.8599.6028.0499.25330.898.8332.7994.3847630.961012796.10138.497.9339.83
ViT [9] + Transformer [17]99.2133.8299.4229.4599.11335.498.7233.7194.2651590.921021795.95145.698.0840.55
(patches = 2) + (head = 2)97.9341.0599.8935.8196.68373.995.6348.5194.396036-0.541144195.61189.896.6053.10
(patches = 2) + (head = 4)97.1659.3799.4139.0297.76297.498.7198.1094.315982-0.091499796.65198.697.7848.30
(patches = 2) + (head = 8)99.2633.3399.5128.7599.18333.198.7733.2594.3249610.941017296.03142.797.6640.19
(patches = 2) + (head = 16)98.8036.61100.428.3897.85326.497.0633.4294.5448690.541746496.17138.197.3239.44
(patches = 4) + (head = 2)99.3033.0199.5728.2799.22331.198.8132.9494.3648290.551714296.08139.898.0739.95
(patches = 4) + (head = 4)97.4960.04100.0262.6095.25461.794.1257.9294.389034-1.832993096.14279.395.4158.08
(patches = 4) + (head = 8)98.4757.9498.1952.1497.93474.897.5356.2694.329887-0.891902194.12285.997.8959.57
(patches = 4) + (head = 16)97.9858.9899.1052.3596.58468.295.8157.0894.508960-0.871937895.37282.196.6458.82
(patches = 8) + (head = 2)97.4960.04100.0252.5695.25462.594.1257.9295.089034-0.851974296.64279.595.4158.08
(patches = 8) + (head = 4)99.1832.4999.5337.1298.73378.298.5236.3893.4754990.281906796.01137.197.3739.15
(patches = 8) + (head = 16)99.4931.2399.9125.6999.47323.699.0131.2694.5941031.02997896.35127.398.1238.63
(patches = 16) + (head = 2)98.4736.5999.0832.8798.78369.496.1647.0494.745934-0.311122796.15183.697.2451.20
(patches = 16) + (head = 4)99.0439.7297.8236.3998.42355.897.9651.7294.5353011.001402295.27130.797.2451.99
(patches = 16) + (head = 8)98.8739.3698.7041.3498.31345.797.8842.3394.2555650.111708295.17155.197.4145.47
(patches = 16) + (head = 16)97.9459.3898.8348.5097.39973.997.1559.1794.509877-0.191994296.08197.597.4547.46
SPH-Net99.4931.2399.9125.6999.49323.699.0131.2694.5941031.02997896.35127.398.1238.63

Table III. Ablation experiment of SPH-Net on 8 datasets. (×100%\times100\%)

  • ViT patch sizes: Testing sizes (2, 4, 8, 16) revealed optimal performance at 8, suggesting the best balance between context and granularity.

  • Transformer attention heads: Evaluating configurations (2, 4, 8, 16) showed 8 heads provide the optimal trade-off between attention coverage and computational efficiency.

  • Alternative encoders: Replacing the Transformer decoder with LSTM [23], GRU [24], seq2seq [3], and Bi-LSTM [27] consistently resulted in inferior performance, validating the advantage of attention-based decoding.

The complete SPH-Net configuration, employing 8 ViT patches and 8 attention heads, achieves the best performance across all tasks.

Visualization

As shown in Figure 3, we generated correlation heatmaps for the six key numerical features on each dataset. These heatmaps provide insights into the linear relationships between features. For instance, Open and Close prices typically exhibit strong positive correlations, reflecting their co-movement throughout trading days. Meanwhile, Volume often shows weaker or variable correlations with price features, indicating its influence may be more context-dependent. These visualizations help identify which features are most informative for predicting future stock prices and assist in understanding feature dependencies.

Correlation Heatmap of AMD

(a) Correlation Heatmap of AMD

Correlation Heatmap of Ebay

(b) Correlation Heatmap of Ebay

Correlation Heatmap of FB

(c) Correlation Heatmap of FB

Correlation Heatmap of FSV

(d) Correlation Heatmap of FSV

Correlation Heatmap of Tesla

(e) Correlation Heatmap of Tesla

Correlation Heatmap of Google

(f) Correlation Heatmap of Google

Correlation Heatmap of Mondy

(g) Correlation Heatmap of Mondy

Correlation Heatmap of MTDR

(h) Correlation Heatmap of MTDR

Figure 3. Correlation heatmap of numerical features on 8 datasets.

As shown in Figure 4, visualization of predicted versus actual closing prices directly assess the model’s forecasting accuracy. For each dataset, SPH-Net’s predicted stock prices closely align with the actual values, with minimal lag or divergence, even in volatile market conditions. This confirms that the model effectively captures both short-term fluctuations and long-term patterns. The smoothness and stability of the predicted curves also demonstrate the robustness of the co-attention mechanism in handling noisy and non-linear financial data. These plots visually validate the strong regression performance shown in quantitative metrics such as R2R^2 and MSE.

Predicated V.S. Actual Prices on AMD

(a) Predicated V.S. Actual Prices on AMD

Predicated V.S. Actual Prices on Ebay

(b) Predicated V.S. Actual Prices on Ebay

Predicated V.S. Actual Prices on FB

(c) Predicated V.S. Actual Prices on FB

Predicated V.S. Actual Prices on FSV

(d) Predicated V.S. Actual Prices on FSV

Predicated V.S. Actual Prices on Tesla

(e) Predicated V.S. Actual Prices on Tesla

Predicated V.S. Actual Prices on Google

(f) Predicated V.S. Actual Prices on Google

Predicated V.S. Actual Prices on Mondy

(g) Predicated V.S. Actual Prices on Mondy

Predicated V.S. Actual Prices on MTDR

(h) Predicated V.S. Actual Prices on MTDR

Figure 4. Comparison between predicated and actual next day closing prices on 8 datasets.

As shown in Figure 5, moving average trends (SMA and EMA) with actual stock prices can evaluate how well SPH-Net tracks underlying market trends. While moving averages provide smoothed estimates that filter out short-term noise, they often lag behind real price movements. In contrast, SPH-Net’s predictions more accurately follow the actual price trajectory while still preserving trend consistency. This indicates that SPH-Net effectively balances reactivity and stability—two crucial aspects for real-time financial forecasting. Together, these visual analyses reinforce the model’s interpretability, accuracy, and applicability in real-world trading environments.

Moving versus actual stock prices on AMD, Ebay, FB, FSV, Tesla, Google, Mondy, and MTDR

Figure 5. Moving averages V.S. actual stock prices on 8 datasets (Simple Moving Average, SMA and Exponential Moving Average, EMA).

Analysis

The comparative experimental results demonstrate that SPH-Net achieves superior performance compared to both traditional machine learning models and recent deep learning architectures across all eight datasets. As shown in the results, the model consistently attains the lowest mean squared error (MSE) and the highest coefficients of determination R2R^2, along with optimal precision, accuracy, and recall metrics. This performance advantage primarily originates from the synergistic integration of co-attention mechanisms and hybrid architectural design. In contrast to Transformer [17] and BERT [21] architectures, SPH-Net incorporates visual feature extraction through Vision Transformer (ViT) [9], which enables effective learning of subtle temporal patterns from raw time-series data converted into patch sequences. Notably, SPH-Net surpasses the performance of MagicNet [28], DeepClue [33], and HATR-I [34] models that employ textual or structural auxiliary data, thereby demonstrating that architectural innovations in SPH-Net alone provide sufficient predictive enhancement without requiring supplementary data modalities.

The ablation study systematically confirms the individual contributions of each component within SPH-Net. Experimental evidence indicates that removing the ViT module or modifying the patch size configuration results in measurable performance degradation, substantiating ViT’s critical role in capturing local temporal dependencies and short-term market dynamics. The analysis further reveals that the number of attention heads in the Transformer decoder significantly impacts model effectiveness, with eight attention heads achieving an optimal balance between computational efficiency and representational capacity. Comparative experiments replacing the Transformer decoder with recurrent architectures (GRU [24], LSTM [23]) exhibit markedly inferior performance, thereby validating the superiority of self-attention mechanisms in modeling long-range temporal dependencies and complex inter-step interactions. These findings collectively establish that the co-attention-based integration of ViT and Transformer constitutes an essential architectural configuration for high-performance stock price prediction.

The visualization results presented in Figure 3, Figure 4, and Figure 5 provide complementary insights into SPH-Net’s operational characteristics. The correlation heatmaps demonstrate the model’s enhanced capability to extract discriminative features from highly correlated financial variables, a critical requirement for robust cross-domain generalization. The temporal prediction plots reveal SPH-Net’s effectiveness in mitigating persistent challenges observed in traditional approaches: specifically, the elimination of prediction latency artifacts common in LSTM [23] implementations and the avoidance of oversmoothing tendencies characteristic of ARIMA [16] models. Comparative analysis with moving average trends [2], [4], [5] further demonstrates SPH-Net’s improved responsiveness to rapid market fluctuations and price volatility events. These visualizations collectively confirm that the hybrid architecture - combining ViT [9] for fine-grained pattern extraction and Transformer [17] for temporal dependency modeling - achieves not only numerical superiority but also produces interpretable predictions aligned with financial market dynamics, thereby fulfilling practical requirements for real-world deployment.

An interesting phenomenon worth noting is the performance of all models on Google’s dataset. As shown in Figure 2k, Figure 2l, and Figure 3f, Google’s data is particularly discrete and there is no regularity in its distribution. Figure 4f and Figure 5f are also verified, and the data generated is very scattered. Table. Table I and Table. Table III also show a large indicator gap on MSE. Since it is just a prediction of the increase (binary classification) rather than an exact value, the models perform better in Table. Table II. Based on it, Google’s dataset can be used as a good interference term to verify the robustness of the model.

Conclusion

In this paper, we introduce SPH-Net, a novel hybrid neural network with co-attention mechanisms, specifically designed for accurate stock price prediction. By integrating the Vision Transformer with a Transformer encoder-decoder architecture, SPH-Net effectively captures both local and global temporal dependencies inherent in financial time-series data. Our approach transforms conventional stock price data into a structured, image-like format, thereby facilitating more expressive feature extraction and improved trend recognition. Extensive experimentation on eight real-world stock datasets spanning multiple industries demonstrates that SPH-Net consistently outperforms state-of-the-art models across both regression and classification tasks. The model achieves superior R2R^2 scores, reduced MSE, and enhanced classification metrics such as accuracy, precision, and recall. Moreover, ablation studies substantiate the contributions of each architectural component, validating the efficacy of the co-attention mechanism and the hybrid structure. In conclusion, SPH-Net offers a promising framework for financial time-series forecasting, delivering enhanced prediction accuracy and robustness. Furthermore, its design is adaptable for potential extensions to multimodal data and other time-sensitive applications beyond stock market analysis.

Future Work

Despite SPH-Net’s strong performance across various datasets, several avenues for future research remain. Firstly, incorporating additional data modalities, such as financial news, analyst reports, or sentiment data, could further bolster the model’s ability to capture market context and external influences. Secondly, while the current model demonstrates robust results in daily-level predictions, investigating its applicability to intra-day or high-frequency trading data presents an interesting challenge. Additionally, further advancements in the scalability and interpretability of SPH-Net are needed. Specifically, developing lightweight versions of the model for deployment in real-time systems and integrating explainable AI techniques to enhance decision-making transparency represent valuable directions for future work. Finally, extending SPH-Net to other financial forecasting tasks—such as volatility prediction or portfolio optimization—could expand its applicability in practical investment scenarios.

References

  1. [1]S. B. S, N. S. Fatima and M. Abbas, ”Enhancing Stock Market Prediction with an LSTM-Attention Model Integrating Macroeconomic and Sentiment Features,” 2024 International Conference on Innovative Computing, Intelligent Communication and Smart Electrical Systems (ICSES), Chennai, India, 2024, pp. 1-6.
  2. [2]X. Chai, S. Shan, X. Chen and W. Gao, ”Locally Linear Regression for Pose-Invariant Face Recognition,” in IEEE Transactions on Image Processing, vol. 16, no. 7, pp. 1716-1725, July 2007.pubmed.ncbi.nlm.nih.gov/17605371
  3. [3]K. Kawano, S. Koide and C. Imamura, ”Seq2seq Fingerprint with Byte-Pair Encoding for Predicting Changes in Protein Stability upon Single Point Mutation,” in IEEE/ACM Transactions on Computational Biology and Bioinformatics, vol. 17, no. 5, pp. 1762-1772, 1 Sept.-Oct. 2020.DOI
  4. [4]A. B. Gumelar et al., ”Boosting the Accuracy of Stock Market Prediction using XGBoost and Long Short-Term Memory,” 2020 International Seminar on Application for Technology of Information and Communication (iSemantic), Semarang, Indonesia, 2020, pp. 609-613.DOI
  5. [5]P. A. Gutiérrez, C. Hervás-Martínez and F. J. Martínez-Estudillo, ”Logistic Regression by Means of Evolutionary Radial Basis Function Neural Networks,” in IEEE Transactions on Neural Networks, vol. 22, no. 2, pp. 246-263, Feb. 2011.DOI
  6. [6]Y. Lin, H. Guo and J. Hu, ”An SVM-based approach for stock market trend prediction,” The 2013 International Joint Conference on Neural Networks (IJCNN), Dallas, TX, USA, 2013, pp. 1-7.DOI
  7. [7]O. T. Yildiz, ”VC-Dimension of Univariate Decision Trees,” in IEEE Transactions on Neural Networks and Learning Systems, vol. 26, no. 2, pp. 378-387, Feb. 2015.DOI
  8. [8]Li, Shuai, Wanqing Li, Chris Cook, Ce Zhu, and Yanbo Gao. ”Independently recurrent neural network (indrnn): Building a longer and deeper rnn.” In Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5457-5466. 2018.arxiv.org/abs/1803.04831
  9. [9]Dosovitskiy, Alexey, Lucas Beyer, Alexander Kolesnikov, Dirk Weissenborn, Xiaohua Zhai, Thomas Unterthiner, Mostafa Dehghani et al. ”An image is worth 16x16 words: Transformers for image recognition at scale.” arXiv preprint arXiv:2010.11929 (2020).arxiv.org/abs/2010.11929
  10. [10]R. Gupta, N. Garg and S. Singh, ”Stock Market Prediction Accuracy Analysis Using Kappa Measure,” 2013 International Conference on Communication Systems and Network Technologies, Gwalior, India, 2013, pp. 635-639.DOI
  11. [11]Huang, Cheng, Weizheng Xie, Jian Zhou, Tsengdar Lee, Karanjit Kooner, and Jia Zhang. ”GlaLSTM: A Concurrent LSTM Stream Framework for Glaucoma Detection via Biomarker Mining.” arXiv preprint arXiv:2408.15555 (2024).
  12. [12]Ma, Yun, Xiaoli Ma, and George J. Jiang. ”Insider Trading Patterns During the Covid Period.” Available at SSRN 5074899.
  13. [13]O. Vinyals, A. Toshev, S. Bengio and D. Erhan, ”Show and tell: A neural image caption generator,” 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Boston, MA, USA, 2015, pp. 3156-3164.
  14. [14]M. Li et al., ”Protein-Protein Interaction Sites Prediction Based on an Under-Sampling Strategy and Random Forest Algorithm,” in IEEE/ACM Transactions on Computational Biology and Bioinformatics, vol. 19, no. 6, pp. 3646-3654, 1 Nov.-Dec. 2022.DOI
  15. [15]Karadas, Serkan, and Minh Tam Tammy Schlosky. ”What explains trading behaviors of members of congress? Evidence from over 100,000 congressional stock trades.” International Review of Economics & Finance 96 (2024): 103591.DOI
  16. [16]A. A. Ariyo, A. O. Adewumi and C. K. Ayo, ”Stock Price Prediction Using the ARIMA Model,” 2014 UKSim-AMSS 16th International Conference on Computer Modelling and Simulation, Cambridge, UK, 2014, pp. 106-112.
  17. [17]Vaswani, Ashish, Noam Shazeer, Niki Parmar, Jakob Uszkoreit, Llion Jones, Aidan N. Gomez, Lukasz Kaiser, and Illia Polosukhin. ”Attention is all you need.” Advances in neural information processing systems 30 (2017).arxiv.org/abs/1706.03762
  18. [18]Wang, Jin, Liang-Chih Yu, K. Robert Lai, and Xuejie Zhang. ”Dimensional sentiment analysis using a regional CNN-LSTM model.” In Proceedings of the 54th annual meeting of the association for computational linguistics (volume 2: Short papers), pp. 225-230. 2016.DOI
  19. [19]Y. Lin et al., ”Attention Enhanced Network with Semantic Inspector for Medical Image Report Generation,” 2023 IEEE 35th International Conference on Tools with Artificial Intelligence (ICTAI), Atlanta, GA, USA, 2023, pp. 242-249.DOI
  20. [20]Schuster, Mike, and Kuldip K. Paliwal. ”Bidirectional recurrent neural networks.” IEEE transactions on Signal Processing 45, no. 11 (1997): 2673-2681.DOI
  21. [21]Devlin, Jacob, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. ”Bert: Pre-training of deep bidirectional transformers for language understanding.” In Proceedings of the 2019 conference of the North American chapter of the association for computational linguistics: human language technologies, volume 1 (long and short papers), pp. 4171-4186. 2019.arxiv.org/abs/1810.04805
  22. [22]Huang, Cheng, Junhao Shen, Beichen Hu, Mohammad Ausaf Ali Haqqani, Tsengdar Lee, Karanjit Kooner, Ning Zhang, and Jia Zhang. ”Semantic and Visual Attention-Driven Multi-LSTM Network for Automated Clinical Report Generation.” In AI for Health Equity and Fairness: Leveraging AI to Address Social Determinants of Health, pp. 233-248. Cham: Springer Nature Switzerland, 2024.DOI
  23. [23]Hochreiter, Sepp, and Jürgen Schmidhuber. ”Long short-term memory.” Neural computation 9, no. 8 (1997): 1735-1780.
  24. [24]Cho, Kyunghyun, Bart van Merriënboer, Çağlar Gülçehre, F. Bougares, H. Schwenk, and Yoshua Bengio. ”Learning phrase representations using RNN encoder-decoder for statistical machine translation.” In Conference on Empirical Methods in Natural Language Processing (EMNLP 2014). 2014.arxiv.org/abs/1406.1078
  25. [25]Benaddi, Lamya, Charaf Ouaddi, Adnane Souha, Abdeslam Jakimi, Mohamed Rahouti, Mohammed Aledhari, Diogo Oliveira, and Brahim Ouchao. ”Seq2Seq Model-Based Chatbot with LSTM and Attention Mechanism for Enhanced User Interaction.” arXiv preprint arXiv:2501.00049 (2024).arxiv.org/abs/2501.00049
  26. [26]A. H. Khan, X. Cao, S. Li, V. N. Katsikis and L. Liao, ”BAS-ADAM: an ADAM based approach to improve the performance of beetle antennae search optimizer,” in IEEE/CAA Journal of Automatica Sinica, vol. 7, no. 2, pp. 461-471, March 2020.DOI
  27. [27]Graves, Alex, Navdeep Jaitly, and Abdel-rahman Mohamed. ”Hybrid speech recognition with deep bidirectional LSTM.” In 2013 IEEE workshop on automatic speech recognition and understanding, pp. 273-278. IEEE, 2013.DOI
  28. [28]D. Luo, S. Li, W. Liao and R. Yan, ”MagicNet: Memory-Aware Graph Interactive Causal Network for Multivariate Stock Price Movement Prediction,” in IEEE Transactions on Knowledge and Data Engineering, vol. 37, no. 4, pp. 1989-2000, April 2025.
  29. [29]Huang, Cheng, Fan Gao, Yutong Liu, Yadi Liu, Xiaoli Ma, Ye Aung Moe, Yuan Zhang et al. ”IFDA: A Large-Scale Benchmark for Insider Filing Violation Detection.” arXiv preprint arXiv:2507.20162 (2025).
  30. [30]Lee, Yi-Ting, Da-Yi Wu, Chih-Chun Yang, and Shou-De Lin. ”Exposing the Functionalities of Neurons for Gated Recurrent Unit Based Sequence-to-Sequence Model.” arXiv preprint arXiv:2303.15072 (2023).
  31. [31]Luo, Di, Weiheng Liao, Shuqi Li, Xin Cheng, and Rui Yan. ”Causality-guided multi-memory interaction network for multivariate stock price movement prediction.” In Proceedings of the 61st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 12164-12176, 2023.
  32. [33]L. Shi, Z. Teng, L. Wang, Y. Zhang and A. Binder, ”DeepClue: Visual Interpretation of Text-Based Deep Stock Prediction,” in IEEE Transactions on Knowledge and Data Engineering, vol. 31, no. 6, pp. 1094-1108, 1 June 2019.
  33. [34]H. Wang, T. Wang, S. Li and S. Guan, ”HATR-I: Hierarchical Adaptive Temporal Relational Interaction for Stock Trend Prediction,” in IEEE Transactions on Knowledge and Data Engineering, vol. 35, no. 7, pp. 6988-7002, 1 July 2023.
  34. [35]Y. Yi et al., ”Optimization of Quantum Key Distribution Parameters Based on Random Forest,” 2021 4th International Conference on Pattern Recognition and Artificial Intelligence (PRAI), Yibin, China, 2021, pp. 164-168.

Paper details

vlnc:W_289uhnve
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 11

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Computer Science

Not recorded for this paper

View statement
Looking for related papers…
Contents