betaFeedback

WorksEconomics, Econometrics and Finance

Quantformer: from attention to profit with a quantitative transformer trading strategy

Abstract

In traditional quantitative trading practice, navigating the complicated and dynamic financial market presents a persistent challenge. Fully capturing various market variables, including long-term information, as well as essential signals that may lead to profit remains a difficult task for learning algorithms. In order to tackle this challenge, this paper introduces quantformer, an enhanced neural network architecture based on transformer, to build investment factors. By transfer learning from sentiment analysis, quantformer not only exploits its original inherent advantages in capturing long-range dependencies and modeling complex data relationships, but is also able to solve tasks with numerical inputs and accurately forecast future returns over a given period. This work collects more than 5,000,000 rolling data of 4,601 stocks in the Chinese capital market from 2010 to 2023. The results of this study demonstrate the model's superior performance in predicting stock trends compared with other 100-factor-based quantitative strategies. Notably, the model's innovative use of transformer-like model to establish factors, in conjunction with market sentiment information, has been shown to enhance the accuracy of trading signals significantly, thereby offering promising implications for the future of quantitative trading strategies.

Introduction

The goal of stock trading is to optimize the return on investment in the capital market according to the process of buying or selling one or more companies’ shares. Traders obtain profit when a positive difference is generated by the fluctuation of stock price. However, stocks are influenced by a large number of factors, which constitute a complex system and make it difficult for people to make a profit. The assessment of a stock’s evolving trend is inherently challenging due to the highly volatile and interconnected nature of the market, which sets it apart from typical time series modeling (Wang et al., 2022b). As a result, many strategies and tools have been built, in parallel with the development of capital markets, and quantitative strategies have been playing an important role among them.

Some traditional quantitative tools, such as the Markowitz portfolio theory (Markowitz, 1952) and the Capital Asset Pricing Model (CAPM) (Sharpe, 1964), focus mainly on static fundamental analysis. In other words, these strategies aim to make a profit by simple calculation and analysis. Since then, along with the development of computer science, more quantitative methods and tools have been introduced. Within these methods, factor-based strategies have attracted much attention. In 1993, Fama and French (1993) introduced the Fama-French Three Factor Model (FF3), which has become an influential model in quantitative trading. In 2015, Fama and French revised their model with a Five-Factor Asset Pricing Model (FF5) (Fama and French, 2015). Besides this classical theory, numerous trading strategies have been published for decades. For example, Shi et al. (2024) built an optimal portfolio to catch Future Investment Opportunities (FIO) by multi-factor models.

Quantitative trading with factors typically follows two primary approaches, which are shown in Figure 1. The first approach involves the computation of stock factor values. Based on these calculated values, stocks are ranked to establish a pool. Once this pool is established, assets are held for a predetermined period. Adjustments to the portfolio are then made at specific time intervals, ensuring alignment with evolving market conditions and factor readings. The second method employs a fixed pool of stocks, wherein factors guide the derivation of long/short signals. Traders can execute corresponding actions when they receive the signals from factors, allowing for a dynamic response to market fluctuations based on factor insights.

Quantitative trading architecture with factors

Figure 1. Quantitative trading architecture with factors

In recent years, Machine Learning (ML) has became an instrumental tool in trading algorithms and decision-making processes. ML methods allow systems to learn from and make decisions based on data, and lend themselves particularly well to the vast and dynamic landscapes of stock markets. For example, Nayak et al. (2015) exploited machine learning algorithms based on manual indicators, but the underlying random walk hypothesis may hamper the task of understanding inherently non-stationary series. With the rise of different architectures, Feng et al. (2019) introduced Relational Stock Ranking (RSR) to catch the sentiment of the market in trading data.

Although there exist several previous experiments with ML factors that attempted to fetch market sentiment in the quantitative finance field, this research still faces two difficulties. Firstly, in the field of sentiment analysis, which is a branch of Natural Language Processing (NLP, a field of computer science that aims to understand, interpret, and generate human language), models are used to convert words in text to word vectors through word embeddings to serve as inputs. However, instead of words, financial datasets contain both categorical data, such as industry types, as well as quantitative data, such as price fluctuation, turnover rate, and financial indicators. If the input comprises only categorical data, the time series can be treated as a sentence (Gorishniy et al., 2022). In most cases, the input will involve numerical data, which cannot be transformed via word embeddings.

Example of stock prediction

Figure 2. Example of stock prediction

Secondly, most NLP tasks can be transformed into sequence-to-sequence (seq2seq) problems, such as in machine translation, dialogue systems, and speech recognition. As an example, the transformer architecture is based on the seq2seq architecture (Vaswani et al., 2017). To utilize existing outputs, decoders in transformer sequentially output samples and use masking operations to handle input sequences during training. However, in stock prediction, where the aim is often to accurately forecast future returns over a period, the transformer model is rarely used for such tasks.

To address these problems, we propose quantformer, a modified transformer architecture adapted to quantitative data, and use it as an investment factor. Quantformer is able to input numerical data directly, which refers to a method similar to sentiment analysis. Our contributions lie in following aspects:

  • We propose quantformer, whose structure adapts to rolling stock-related time series data as inputs without positioning module and word embedding. The new structure with linear embedding shows better fitting to the

numerical type inputs.

  • According to our experiments, quantformer-based factors perform better in back-tests compared with other traditional factor strategies under the same time period with different trading frequencies. These experiments indicate a potential direction for applying transformer-based models to quantitative financial tasks.

The paper is structured as follows. § 2 discusses previous quantitative financial works based on machine learning. § 3 and § 4 introduce quantformers. A factor based on quantformer will be trained and back-tested. For the practical backtest, we collected data from more than 4,600 stocks in the past 14 years (from 2010 to 2023). To comprehensively test the ability of the factor, we divide the data by different frequencies (Subsection 4.2) and trained under different training scales (Subsection 5.2). Finally, the results of the back-tests including the comparison between the quantformer-factor and other 100 factors as well as the insights gained from such comparative analysis are discussed in § 4 and § 5.

Preliminaries

This section briefly introduces the related works about stock prediction with market sentiment and the development of quantitative financial trading methods, especially those based on Machine Learning (ML) methods.

Stock data and market sentiment

It is known that, in some cases, stock data can be deemed to influence the future trend of a stock price [2]. Chen et al. (2020), P.H and Rishad (2020) have discussed the correlation between investor sentiment and stock market metrics such as turnover rate, indicating that investor feelings, both individual and institutional, significantly influence market behavior, affecting aspects such as volatility and returns. This relationship is not uniform but varies with changes in investor sentiment. It also highlights that investor psychology, especially during periods of high market stress can drastically impact market behavior and investor decisions. This link between sentiment and market performance is further affirmed by studies focusing on factors such as the turnover ratio, which are seen as reflective of market liquidity and investor behavior (Naseem et al., 2021). Ding et al. (2023) built a new factor based on sentiment analysis with the average of trading signals from technical trading strategies to benchmark stocks of the S&P 500 index and DJIA. The sentiment factor shows correlation with stock returns. These findings underscore the mutual relationship between investor sentiment and market performance, demonstrating how psychological factors can drive market dynamics.

Applications of ML methods for trading

Machine learning exploits a range of algorithms and statistical techniques used for tasks such as regression, clustering, and classification. In past decades, there have been methods that have been increasingly applied to stock data due to their ability to process vast amounts of data and make predictions based on them, offering a potential advantage in the financial markets. At the same time, the finance sector, particularly quantitative trading, has started to be aware of the potential of deep learning models for predicting stock movements, portfolio optimization, and risks.

Support Vector Machines (SVMs)

SVMs have been used in quantitative trading, as they can manage high-dimensional spaces and intricacies found in datasets. Kim (2003) used SVMs to forecast financial time series in 2003, with a focus on the South Korean stock market, exhibiting the robustness of SVM against traditional statistical methods and some other computational techniques. Similarly, Huang et al. (2005) showcased SVMs’ prowess in anticipating the movement direction of Japanese stock markets, especially when determining bullish or bearish market sentiments. Cavalcante et al. (2016) assessed SVMs’ adaptability and precision in financial tasks such as portfolio and quantitative trading.

However, SVMs can also show some disadvantages in quantitative trading. In Huck (2009)’s work, SVM was underscored for the accuracy of overfitting. When applied to financial datasets, the model’s propensity to fit too closely to the training data might lead to reduced generalization capabilities, making the strategy ineffective on unseen data. In high-frequency trading, though the model displayed competency in specific scenarios, researchers pinpointed potential limitations when grappling with certain volatile market dynamics, suggesting the need for adaptations (Kercheval and Zhang, 2015).

Long Short-Term Memory (LSTM)

LSTM is a neural network architecture designed to capture long-term dependencies in sequential data (Hochreiter and Schmidhuber, 1997). LSTM networks can accentuate and effectively model volatile financial markets, showcasing significant improvements over traditional time series models. Orsel and Yamada (2022) combined LSTM with linear Kalman filter and set experiments on different volatility stocks to build strategies. Bao et al. [5] deployed LSTM with stacked autoencoders to forecast financial time series. Fischer and Krauss (2018a) tested the efficacy of LSTM for price predictions. They observed LSTMs’ prowess in noisy data and their superiority in prediction accuracy over other conventional models. Sezer and Ozbayoglu (2018) combined LSTM with other architectures and found that these can enhance the feature extraction process, thereby improving prediction accuracy. For predicting stock prices, Zhang et al. (2017) harnessed the power of LSTM to unearth multi-frequency trading patterns. Their method underlined the adaptability of LSTMs in modeling complex situations inherent in stock prices.

Gated Recurrent Units (GRUs)

GRUs (Chung et al., 2014) are another neural network architecture which has emerged as a noteworthy method in the quest for accurate financial forecasting. GRUs can capture long-term dependencies in time series data and have seen increasing popularity in the world of quantitative trading. Bao et al. [5] introduced a comprehensive deep-learning framework for financial time series, which incorporated GRUs alongside LSTM and stacked autoencoders. Their model, when applied to diverse datasets, performs better than traditional time series models, accentuating the adaptability and robustness of GRUs. Fischer and Krauss (2018b) centralized on LSTM networks, provided comparative insights into GRUs. However, in their analysis, LSTMs showed a slight edge in prediction accuracy, indicating that while GRUs are powerful, selecting between them and LSTMs may boil down to specific use cases and computational constraints. Beyond that, Pak et al. (2018) presented a model fusing GRUs with Convolutional Neural Networks (CNN) O’Shea and Nash (2015). The strategy to combine CNNs and GRUs also exhibits advantages for stock price prediction. Sun et al. (2024) introduced the Attention-GRU model which combined attention and GRU to establish a new factor based on CVaR portfolio ([4] (Zhu and Fukushima 2009)). The Attention-GRU model fitted market return, stocks return from 28 Dow Jones Industrial Average index (DJIA) stocks, which achieved better performance than other models in eight metrics such as annual return, standard deviation and information ratio.

Transformer in time series prediction

Transformer (Vaswani et al., 2017), which was introduced in 2017, has received considerable attention in the NLP field. According to its special encoder and decoder stacks with self-attention blocks, the transformer shows its advantage in stability, speed, and long-term memory compared with other traditional models. Transformer-based models such as GPT (Radford et al., 2019) and BERT (Devlin et al., 2019) have been influential in the NLP field. More recent transformer-based models include ChatGPT (Brown et al., 2020), FinBERT [1] (Yang et al. 2020) and BloombergGPT (Wu et al., 2023).

Similar to other neural sequence transduction models, transformer also has an encoder-decoder structure. The model takes as input a sequence of context, which could be natural language sentences, and generates output with texts depending on the input, similar to a translation according to the input or a predicted sentence. Transformer consists of stacks of layers, which include self-attention and point-wise operations. These layers are used to connect the encoder and decoder in the architecture. For these encoder and decoder layers, the self-attention method plays an important role in matrix calculation. The linear and softmax functions are used in the dot-product attention module. In the vanilla transformer model, multi-head attention layers are used and the result from each layer is outputted after going through an additional normalization layer. The above-mentioned steps cover most of the progress made with the vanilla transformer.

Table 1 shows a set of advanced transformer models, with a focus on time series prediction tasks, which are tasks of interest in quantitative finance. The table highlights the modifications made to the original transformer model. These modifications can be categorized into innovations in attention mechanisms, encoding and decoding structure, positioning module, and mixture model.

Some research proposed enhancements of the attention module. For example, Adversarial Sparse Transformer (AST) (Wu et al., 2020) introduced sparse attention using the α\alpha-Entmax function within adversarial training. Autoformer (Wu et al., 2021b) replaced traditional self-attention with an auto-correlation mechanism that utilizes periodic patterns in frequency-domain time series analysis. Similarly, Informer (Zhou et al., 2021) suggested ProbSparse attention that selects only the most relevant queries and introduces generative-style decoding to handle long-term dependencies. Gated Transformer Networks (GTN) (Liu et al., 2021) used separate attention for channel-wise and temporal-wise interactions for multivariate time series. Patch Time Series Transformer (PatchTST) (Nie et al., 2022) divided time series into patches to model local dependencies, and Quatformer (Chen et al., 2022) attempted to use rotation-based attention to capture periodic patterns in time series. AirFormer (Liang et al., 2023) introduced dartboard spatial attention and causal temporal attention in air quality prediction. Rough transformer (Moreno-Pino et al., 2024) facilitated multi-view signature attention and continuous-time path encoding for financial market modeling for complex financial time series prediction.

In another direction, some works tried to make changes to the encoding or decoding of transformer. FEDformer (Zhou et al., 2022) splitted time series data into trend and seasonal components with frequency-enhanced modules. To improve computational efficiency, some models, including Multivariate Transformer (Zerveas et al., 2021) gave up using a decoder and focused solely on an encoder for feature extraction for forecasting tasks.

As a kind of revision sight, some research has proposed to replace or extend the positioning modules to better accommodate time series data. For instance, HFformer [6] gave up positioning modules, and uses activation functions with a lightweight linear decoder to revise the model. Noting that temporal order is inherently embedded in time series data, iTransformer (Liu et al., 2024) introduced a dimension-inverted structure and removes positioning. Transformer Hawkes Process (THP) (Zuo et al., 2020) put time intervals into the positioning module directly. FX-Spot Transformer (Fischer et al., 2024) used the Time to Vector (Time2Vec) function to handle irregular time intervals.

To overcome the limitations of standalone transformer models, an approach is to integrate them with other frameworks. ConvLSTM (Kim et al., 2024) merged convolutional neural networks (CNNs) and LSTM layers with transformer moduled to simultaneously model spatial and temporal patterns. ProTran (Tang and Matteson, 2021) integrated state-space models (SSMs) with transformers to address non-Markovian dynamics in time series data, providing robust long-term forecasts. These hybrid approaches improved the versatility and domain adaptability of transformer-based architectures.

Transformer in quantitative trading

In the quantitative trading domain, a few research articles have investigated the potential of transformers. Wang et al. (2022a) utilized transformer

ReferenceModelMethod
Wu et al. (2020)Adversarial Sparse Transformer (AST)Sparse attention with adversarial training
Liang et al. (2023)AirFormerDartboard spatial self-attention and causal temporal self-attention for different training stage
Wang et al. (2022b)ALSP-TFLocal interaction and global interaction between layers
Wu et al. (2021b)AutoformerAuto-correlation instead of attention
Wang (2023)BiLSTM-MTRAN-TCNCombined BiLSTM, Transformer and Temporary Revolution Network (TCN)
Kim et al. (2024)ConvLSTMCombined LSTM with transformer
Zeng et al. (2023a)LTSF-LinearDLinear and NLinear, MLP-based with transformer
So et al. (2019)Evolved TransformerNeural Architecture Search (NAS) in transformer
Zhou et al. (2022)FEDformerTimestamp encoding with frequency enhanced attention
Liu et al. (2021)Gated Transformer Networks (GTN)Two-tower transformer with channel-wise & step-wise
Ding et al. (2020)Gaussian transformerMulti-scale Gaussian prior, orthogonal regularization and trading gap splitter for self-attention
Barez et al. [6]HFformerLinear decoder and spiking activation
Zhou et al. (2021)InformerProbSparse self-attention, self-attention distilling operation and generative style decoder
Liu et al. (2024)iTransformerDimension inversion and deleted positioning
Li et al. (2019)LogSparse TransformerCausal convolution attention and query-key matching
Eisenach et al. (2020)MQ-TransformerPositional encoding from event indicators and decoder-encoder attention for context-alignment
Ramos-Pérez et al. (2021)Multi-TransformerAverage multi-head and bagging attention and positioning based on time stamps
Zerveas et al. (2021)Multivariate transformerDeleted decoder and applied learnable positioning module for multivariate time series
Liu et al. (2022b)Non-stationary TransformerSeries standardization and De-stationary attention
Nie et al. (2022)Patch Time Series Transformer (PatchTST)Divided time series into patches, applied channel-independence and instance normalization
Tang and Matteson (2021)ProTranModeled non-Markovian dynamics by State Space Models (SSMs) to
Liu et al. (2022a)PyraformerPyramidal Attention Module (PAM) and intra-scale neighboring connections model
Chen et al. (2022)QuatformerLearning-to-rotate Attention (LRA) with global memory and trend normalization
Kitaev et al. (2020)ReformerLocality-sensitive hashing attention and reversible residual layers
Moreno-Pino et al. (2024)Rough TransformerMulti-view signature attention and continuous-time paths encoding transformation
Shabani et al. (2022)ScaleformerMulti-scale framework with various models (such as FEDformer, Autoformer)
Lin et al. (2021)State Space Decomposition Neural Network (SSDNet)Transformer with SSMs
Ma et al. (2025)StockformerDual-frequency spatiotemporal encoder for data in different frequency with graph embedding and fusion attention
Chowdhury et al. (2022)TARNetAttention score for timestamps masking
Olorunnimbe and Viktor (2024)Temporal Fusion Transformer (TFT)LSTM learning positional encoding
Fischer et al. (2024)Transformer for FX-SpotTime2Vec positioning module
Zuo et al. (2020)Transformer Hawkes process (THP)Positional encoding by translating time intervals
Cirstea et al. (2022)TriformerTriangular,variable-specific patch attention

Table 1. Advanced transformer-based architectures for time series data

to predict stock market indices (including CSI 300, S&P 500, Hang Seng Index, and Nikkei 225) and concluded that the model can better catch the rules of stock market dynamics. Ding et al. (2020) exploited transformer on trading sequences to classify stock price movements. Besides using a traditional transformer, other innovative methods of transformer are worth mentioning. Zhou et al. (2021) optimized the efficiency of time complexity and memory usage of transformer on extremely long time series by informer. Zeng et al. (2023b) combined the advantages of CNNs and transformers to model short-term and long-term dependencies in financial time series. They illustrate the merits of this approach on intraday stock price prediction of S&P 500 constituents. ConvLSTM (Kim et al., 2024) has been trained on stocks from the S&P 500 between 2004 and 2021, and performs better than other models such as VAR and ARIMA.

Similar to previous work, Wang et al. (2022b) introduced Adaptive Long-Short Pattern Transformer (ALSP-TF). Their model is structurally designed for hierarchical representation and interaction of stock price series at different context scales. With the help of a learnable function, they make self-attention aware of the weighted time intervals between patterns, to adaptively adjust their dependencies beyond similarity matching. In the end, they obtained more than 10% of annual return on average.

However, the transformer model can also have some disadvantages in practice. Xu et al. (2021) mentioned that the global self-attention module focuses on point-wise token similarities without contextual insights. As fluctuations of stocks are conditioned on composite signals over manifold periods, lacking pattern-wise interaction hinders the adequate discrimination of stock tendency and is susceptible to noise points. On the other hand, Wu et al. (2021a) claimed that the basic query-key matching paradigm is position agnostic. Although position embedding is inserted into the sequential inputs, it may not be optimal because of the inability to reveal precise distances.

Methodology

Several researchers have tried to improve transformers for time series and quantitative financial applications and proposed improved models (see time series papers in Table 1), but there are still some aspects that have not been considered. For data in time series type, such as stock prices or daily returns, it is reasonable to assume that each data in the series is assigned a “position” automatically, which is different from sentences. Based on this observation, the positional encoding module on the vanilla model may not be a necessary part. Furthermore, market data are related to the market trend. For example, the return of the stock may include the market sentiment and the turnover rate reflects investor sentiment and trading enthusiasm. Therefore, integrating such market-related features as input can enhance the model’s ability to capture underlying market dynamics and improve the accuracy of future trend predictions. To implement these ideas, we propose quantformer, an enhanced transformer-based model for quantitative financial trading, and assess its performance on real-world datasets.

This section introduces the framework of the work and the steps of the establishment of the model, including data processing, quantformer construction and prediction.

Framework overview

Figure 3 gives an overview of the structure. There are three major parts: (i) Data Initialization to align the time series of stocks into a regular matrix; (ii) Unlike the transformer, embedding the training dataset and training them by a quantformer with word-embedding layer replaced by linear layer and with no Mask(opt.) layer; (iii) Lastly, using the trained model to predict the possible trend of stocks.

Overview of the work

Figure 3. Overview of the work

Problem formulation

Training input

Consider a candidate stock set StS^t containing NN stocks on the trading timestamp tt:

St={snt}n=1NS^t = \{s_n^t\}_{n=1}^{N}

For a stock snts_n^t in the stock set StS^t, where n∈{1,…,N}n \in\{1,\ldots,N\}, consider the two-dimensional historical 1-time step (such as one month) input feature matrix Xnt∈R20×2\mathcal{X}_n^t \in\mathbb{R}^{20 \times2}, consisting of 20 continuous timestamps:

Xnt={xnt−m∣m=19,18,…,0}(1)\mathcal{X}_n^t = \{x_n^{t-m} \mid m = 19,18,\ldots,0\} \tag*{(1)}

Each row vector xnt−mx_n^{t-m} for m=0,…,19m = 0,\ldots,19 contains two features. The first one is the accumulated daily profit rate rnt−mr_n^{t-m} during the time step, where pnt−mp_n^{t-m} is the close price of the last day at the timestamp t−mt-m and pnt−m−1p_n^{t-m-1} is the close price of the last day at the timestamp t−m−1t-m-1.

rnt−m=pnt−m−pnt−m−1pnt−m−1,vnt−m=∑ivnt−m,i(2)r_n^{t-m} = \frac{p_n^{t-m} - p_n^{t-m-1}}{p_n^{t-m-1}}, \qquad v_n^{t-m} = \sum_i v_n^{t-m,i} \tag*{(2)}

The second feature is the accumulated daily turnover rate vnt−mv_n^{t-m} during the time step t−mt-m for stock nn. vnt−m,iv_n^{t-m,i} is the daily turnover rate and ii is the day index in the time period. To sum up, each Xnt\mathcal{X}_n^t can be represented as:

Xnt=[xnt−19  xnt−18  …  xnt]T∈R20×2,xnt−m=[rnt−m, vnt−m](3)\mathcal{X}_n^t = [x_n^{t-19}\ \ x_n^{t-18}\ \ \ldots\ \ x_n^t]^T \in\mathbb{R}^{20 \times2}, \quad x_n^{t-m} = [r_n^{t-m},\ v_n^{t-m}] \tag*{(3)}

Normalization

The next step is to normalize the input data for each time step by zero-mean, unit-variance normalization (Z-score). The normalization equation is shown in equation (4). For the row vector xntx_n^t (equation (1)), E[xt]\mathbb{E}[x^t] and std⁡[xt]\operatorname{std}[x^t] represent the mean and standard deviation of all the row vectors xtx^t at time tt, respectively, and the resulting values x~nt\tilde{x}_n^t represent the normalized values of the row data xntx_n^t.

x~nt=xnt−E[xt]std⁡[xt](4)\tilde{x}_n^t = \frac{x_n^t - \mathbb{E}[x^t]}{\operatorname{std}[x^t]} \tag*{(4)}

So, the normalized two-dimensional historical 1-time step input feature matrix can be represented in this way:

X~nt={x~nt−m∣m=19,18,…,0}(5)\tilde{\mathcal{X}}_n^t = \{\tilde{x}_n^{t-m} \mid m = 19,18,\ldots,0\} \tag*{(5)}

In this way, the input sequences are normalized with zero mean and unit variance, which aims to reduce the influence from outlying time points and allow different features to be comparable with each other (Klambauer et al., 2017).

Target labels

For a stock set StS^{t} on the trading timestamp tt, each stock snts_{n}^{t} has a profit rate rnt+1r_{n}^{t+1} on the trading timestamp t+1t+1, where the calculation method is the same as for rntr_{n}^{t} in the sequence of inputs shown in equation (2). The set Gt+1G^{t+1} contains the next-time stamp’s profit for NN stocks in the timestamps t+1t+1. It is defined as Gt+1={rnt+1}n=1NG^{t+1}=\{r_{n}^{t+1}\}_{n=1}^{N}. Each rnt+1r_{n}^{t+1} in the list Gt+1G^{t+1} is then ranked by Ψ(rnt+1)\Psi(r_{n}^{t+1}), which is the empirical quantile CDF for rntr_{n}^{t}:

Ψ(rnt+1)=1n∑i=1n1{rit+1≤rnt+1}(6)\Psi(r_{n}^{t+1})=\frac{1}{n}\sum_{i=1}^{n}\mathbb{1}_{\{r_{i}^{t+1}\leq r_{n}^{t+1}\}} \tag*{(6)}

For the target label for each feature snts_{n}^{t} in Gt+1G^{t+1}, we have a one-hot vector ynt∈Rϱy_{n}^{t}\in\mathbb{R}^{\varrho} based on Ψ(rnt+1)\Psi(r_{n}^{t+1}). ynty_{n}^{t} is partitioned into ϱ\varrho equal bins, where ϱ\varrho is set to 3 or more. Each bin corresponds to a φ×100%\varphi\times100\% of the stocks in the set, where φϱ≤1\varphi\varrho\leq1. And the boundary term ξ\xi is used to ensure non-overlapping intervals. Then ynty_{n}^{t} is represented as:

ynt=[1{(i−1)(ϱ+ξ)≤Ψ(rnt+1)<iϱ+(i−1)ξ}∣i=1,…,ϱ]∈Rϱ,ξ=1−φϱϱ−1y_{n}^{t}=\left[\mathbb{1}_{\{(i-1)(\varrho+\xi)\leq\Psi(r_{n}^{t+1})<i\varrho+(i-1)\xi\}}\mid i=1,\ldots,\varrho\right]\in\mathbb{R}^{\varrho},\quad\xi=\frac{1-\varphi\varrho}{\varrho-1}

For the empirical quantile CDF Ψ(rnt+1)\Psi(r_{n}^{t+1}) of the stock in the range [iφ,(i+1)ξ)[i\varphi,(i+1)\xi) for i=1,…,ϱi=1,\ldots,\varrho, ynty_{n}^{t} is represented as 0ϱ\mathbf{0}_{\varrho}. For example, suppose ϱ=3\varrho=3, φ=0.2\varphi=0.2; for the stocks snts_{n}^{t} whose profit values rntr_{n}^{t} ranked in the top, middle, bottom φ×100%=20%\varphi\times100\%=20\% respectively, as (ξ=0.2(\xi=0.2 in this case):

ynt={[1,0,0]Tif Ψ(rnt+1)∈[0,0.2)[0,1,0]Tif Ψ(rnt+1)∈[0.4,0.6)[0,0,1]Tif Ψ(rnt+1)∈[0.8,1.0)[0,0,0]Totherwisey_{n}^{t}= \begin{cases} [1,0,0]^{T} & \text{if }\Psi(r_{n}^{t+1})\in[0,0.2)\\ [0,1,0]^{T} & \text{if }\Psi(r_{n}^{t+1})\in[0.4,0.6)\\ [0,0,1]^{T} & \text{if }\Psi(r_{n}^{t+1})\in[0.8,1.0)\\ [0,0,0]^{T} & \text{otherwise} \end{cases}

The stocks in the intermediate ranges (Ψ(rnt+1)∈[0.2,0.4),[0.6,0.8))(\Psi(r_{n}^{t+1})\in[0.2,0.4),[0.6,0.8)) are marked as [0,0,0]T[0,0,0]^{T}. Alternatively, when ϱ=5\varrho=5 and φ=0.2\varphi=0.2, the stocks are divided into five equally sized quantiles, with no null labels ([0,0,0,0,0]T)([0,0,0,0,0]^{T}), where each part contains 20% of the stock in StS^{t}:

ynt∈{[1,0,0,0,0]T,[0,1,0,0,0]T,…,[0,0,0,0,1]T}y_{n}^{t}\in\{[1,0,0,0,0]^{T},[0,1,0,0,0]^{T},\ldots,[0,0,0,0,1]^{T}\}

Quantformer encoder

Then, a quantformer encoder structure is designed as described in this subsection. The representation hierarchy consists of LL blocks of multi-head self-attention layers. Taking initialized stock embedding sequences X={X~n(t)}n=1N∈RN×20×2\mathcal{X} = \{\widetilde{\mathcal{X}}_{n}^{(t)}\}_{n=1}^{N} \in\mathbb{R}^{N \times20 \times2} as inputs, the canonical self-attention layers introduced in Vaswani et al. (2017) can perform information exchange between every time points for each stock snts_{n}^{t}.

To process both categorical and numerical data, the word embedding layer is replaced by a standard linear layer, utilizing linear transformations to substitute the process of word embedding. The linear embedding is shown in equation 7, where WE∈R2×d\mathbf{W}_{E} \in\mathbb{R}^{2 \times d} is the trainable weight matrix and θE∈Rd\theta_{E} \in\mathbb{R}^{d} is the bias and dd is the dimension of the hidden feature space. The resulting sequence Xi′\mathcal{X}_{i}^{\prime} is

Xi′=XiWE+θE(7)\mathcal{X}_{i}^{\prime} = \mathcal{X}_{i}\mathbf{W}_{E} + \theta_{E} \tag*{(7)}

In stock prediction, we aim to accurately forecast the return for a future period, thus the model’s output is generally a single value representing the probability of price increase or decrease. Therefore, the decoder is simplified by removing the autoregressive prediction mechanism (in the decoder) and the masking operations. For the attention head h=1,…,Hh = 1, \ldots, H, the query, key, and value metrics for each Xi∈R20×2\mathcal{X}_{i} \in\mathbb{R}^{20 \times2} are computed using separate learned linear projections:

Qi,h=Xi′WhQ,Ki,h=Xi′WhK,Vi,h=Xi′WhV(8)\mathbf{Q}_{i,h} = \mathcal{X}_{i}^{\prime}\mathbf{W}_{h}^{Q}, \quad\mathbf{K}_{i,h} = \mathcal{X}_{i}^{\prime}\mathbf{W}_{h}^{K}, \quad\mathbf{V}_{i,h} = \mathcal{X}_{i}^{\prime}\mathbf{W}_{h}^{V} \tag*{(8)}

where the trainable weights WhQ\mathbf{W}_{h}^{Q}, WhK\mathbf{W}_{h}^{K}, WhV∈Rd×ϱ\mathbf{W}_{h}^{V} \in\mathbb{R}^{d \times\varrho} for the hh-th head. Then the attention parameter for each head is computed using scaled dot-produced attention:

Attention⁡(Qi,h,Ki,h,Vi,h)=softmax⁡(Qi,hKi,hT/d)Vi,h(9)\operatorname{Attention}(\mathbf{Q}_{i,h}, \mathbf{K}_{i,h}, \mathbf{V}_{i,h}) = \operatorname{softmax}\left(\mathbf{Q}_{i,h}\mathbf{K}_{i,h}^{T}/\sqrt{d}\right)\mathbf{V}_{i,h} \tag*{(9)}

The outputs from all heads are concatenated and sent back to the original dimension using WO∈RϱH×d\mathbf{W}^{O} \in\mathbb{R}^{\varrho H \times d}, and the final multi-head attention output Fi\mathbf{F}_{i}:

Fi=Multihead⁡(Qi,h,Ki,h,Vi,h)=(∥h=1HAttention⁡(Qi,h,Ki,h,Vi,h))WO(10)\mathbf{F}_{i} = \operatorname{Multihead}(\mathbf{Q}_{i,h}, \mathbf{K}_{i,h}, \mathbf{V}_{i,h}) = \left(\mathop{\|}_{h=1}^{H} \operatorname{Attention}(\mathbf{Q}_{i,h}, \mathbf{K}_{i,h}, \mathbf{V}_{i,h})\right)\mathbf{W}^{O} \tag*{(10)}

where ∥\| represents the concatenation operator. After applying attention to all inputs, all the output from attention modules are represented in the form of:

F=[F1;F2;…FN]∈RN×20×ϱH\mathbf{F} = [\mathbf{F}_{1}; \mathbf{F}_{2}; \ldots\mathbf{F}_{N}] \in\mathbb{R}^{N \times20 \times\varrho H}

which are fed into feed-forward layers.

Output process

After processing through the multi-head self-attention and feed-forward layers, with the encoder output FiF_i, the output layer can be represented as:

Zt=Softmax⁡(FiWZ+θZ)={z1t,z2t,…zNt}∈Rϱ(11)\mathbf{Z}^{t} = \operatorname{Softmax}(\mathbf{F}_i\mathbf{W}_Z + \theta_Z) = \{z_1^t, z_2^t, \ldots z_N^t\} \in\mathbb{R}^{\varrho} \tag*{(11)}

where WZ∈Rϱ×d\mathbf{W}_Z \in\mathbb{R}^{\varrho\times d} and θZ∈Rd\theta_Z \in\mathbb{R}^{d}. With the softmax function, the predicted probability of the output can be represented as:

Y^nt={y^n,i}i=1ϱ={exp⁡(zit)∑i=1ϱexp⁡(zit)}i=1ϱ∈Rϱ(12)\hat{Y}_n^t = \{\hat{y}_{n,i}\}_{i=1}^{\varrho} = \left\{ \frac{\exp(z_i^t)}{\sum_{i=1}^{\varrho}\exp(z_i^t)} \right\}_{i=1}^{\varrho} \in\mathbb{R}^{\varrho} \tag*{(12)}

In equation (12), the Y^nt\hat{Y}_n^t take values between 0 and 1 and sum to 1, which means that they can be interpreted as the probability distribution of the stock’s performance.

Prediction

The Mean Squared Error Loss (MSELoss) is used to quantify the loss. It is defined as the average of the squares of the differences between the predicted value and the actual prices:

MSELoss⁡=1N∑i=1N∥yit−y^it∥22(13)\operatorname{MSELoss} = \frac{1}{N}\sum_{i=1}^{N}\left\lVert y_i^t - \hat{y}_i^t \right\rVert_2^2 \tag*{(13)}

where ∥⋅∥22\lVert\cdot\rVert_2^2 denotes the squared Euclidean distance between predicted and target class probability vectors.

Experiments

In this section, we test the ability of the quantformer architecture, introduced in the previous section, for stock price prediction. We are going to detail the set of experiments, including the data resource, implementation details, trading strategy and metrics.

Dataset

Our training data contains 4601 stocks listed either on the Shanghai Stock Exchange (SHSE) or on the Shenzhen Stock Exchange (SZSE). The time period ranges from January 2010 to May 2023. The data has been obtained from AKShare3 and Tushare4, which are quantitative finance terminals. The training period is from January 2010 to December 2019 and the testing period starts from January 2020.

Closing price adjustments, such as dividends, stock splits, and other corporate actions that can affect a stock’s price, are applied to the stock training prices. These adjustments are essential for stock price analysis, especially on long time period, as they provide a more accurate picture of a stock’s value and performance over time, which is more appropriate for financial backtesting. (Diamond, 1971, Wei et al., 2022).

Time frequency

Data frequency refers to the number of data points within a specific unit of time, which may reflect different market characteristics (De Prado, 2018). We consider three frequencies: monthly, weekly, and daily stock data.

In the first experiments, the time frequency is set to one month. For a stock snts_n^t from the stock set StS^t, each item in the feature sequence contains the accumulated profit and accumulated turnover rate in the month (equation (3)). Sometimes, the available data may not cover whole trading days, for example, because a stock was first listed on the market or resumed trading after an interruption. In such a situation, we would still record the stock data for this day. However, if there is a whole month during which the stock did not have any trading (perhaps due to stopped trading or not being listed in that month), this month’s data will be recorded as “NaN”, and the sequence Xnt\mathcal{X}_n^t will not be used to train the model.

Within the first experiment, three sub-experiments are set. The sub-experiments share the same inputs. The values ϱ=3\varrho= 3 and ϱ=5\varrho= 5 are used in this experiment. The value of φ\varphi is fixed for all the experiments to 0.2. In the groups of ϱ=5\varrho= 5, stocks are divided into five parts and marked respectively. Then, in the groups of ϱ=3\varrho= 3, the top 20%, middle 20% and bottom 20% of the stocks are labeled with one-hot vectors [1, 0, 0], [0, 1, 0] and [0, 0, 1], respectively. Stocks in the intermediate quantile (i.e., 20% – 40% and 60% – 80%) are marked as [0, 0, 0] (as the boundary term ξ=0.2\xi= 0.2 here, and φ=0.2\varphi= 0.2 as well). In the first group of ϱ=3\varrho= 3, the null labels ([0, 0, 0]T^{T}) are excluded from training, whereas the second group of ϱ=3\varrho= 3 includes them.

StrategyFrequencyLabel dim(ϱ\varrho)Training samplesSectionNull-label
Month_1Monthly385,490100w/o
Month_2Monthly3142,409100w/
Month_3Monthly5142,409100w/o
Week_1Weekly3455,157466w/o
Week_2Weekly3758,300466w/
Week_3Weekly5758,300466w/o
Day_1Daily33,586,4352,420w/o
Day_2Daily35,140,2792,420w/
Day_3Daily55,140,2792,420w/o

Table 2. Detailed information about experiments

Outliers of both accumulated profit and accumulated turnover rate are not removed from the dataset. These situations may happen in the future and they are expected to be predicted, though these incidents rarely happen.

In the second and third experiments, the inputs are in weekly and daily frequency, respectively. The sub-experiments under different groups are similar to the first one. All the accumulated number of parameters, the number of trained sections (such as 100 months from 2010 to 2019), and output dimensions are shown in Table 2.

Implementation details

The model and its training process are implemented with PyTorch (Paszke et al., 2019). The hyperparameters of the model are optimized by grid search, as it is simple to implement and reliable in low-dimensional spaces [7]. Here the input dimension is 2, and the dimension of the hidden feature space dd is 16. The number of multi-head attention modules is 16, and the number of layers of encoder and decoder is 6. The model was trained on an NVIDIA GeForce RTX 2070 GPU and NVIDIA A100 Tensor Core GPU using the Adam optimizer (Kingma and Ba, 2015) for 50 epochs. The learning rate is 0.001, and the batch size is 64.

Trading strategy

Algorithm 1 shows the pseudocode of the trading strategy. Before the first trade date of the timestamp tt, all sequences Xnt\mathcal{X}^{t}_{n} from the stock set StS^{t} are put in the model and the list of outputs Y^t\widehat{Y}^{t} is required. The portfolio value at time tt, denoted as PtP^{t} is updated based on the previous period’s weights and returns as follows:

Pt=Pt−1(∑n=1Nwnt−1(1+rnt−1))(14)P^{t}=P^{t-1}\left(\sum_{n=1}^{N}w^{t-1}_{n}\left(1+r^{t-1}_{n}\right)\right) \tag*{(14)}

where wntw^{t}_{n} is the weight of stock snts^{t}_{n}, rntr^{t}_{n} is the return of the stock, and ∑n=1Nwnt=1\sum_{n=1}^{N}w^{t}_{n}=1. To determine the weight of each stock, a trading strategy Φ∈Rϱ\Phi\in\mathbb{R}^{\varrho} and the decision factor b is used, where b determines how many quantile groups are selected, satisfying 1≤b<ϱ1 \leq\textbf{b} < \varrho. The strategy does not allow short selling (as the Chinese stock market does not allow short selling), so Φ\Phi only contains 0 or 1.

Φ∈{0,1}ϱ,∥Φ∥0=b\Phi\in\{0,1\}^{\varrho}, \quad\lVert\Phi\rVert_{0}=\textbf{b}

For example:

  • If ϱ=3\varrho=3, b=1\textbf{b}=1, then Φ\Phi could be one of: [1, 0, 0], [0, 1, 0], or [0, 0, 1].

  • If b=2\textbf{b}=2, then possible values of Φ\Phi include: [1, 1, 0], [1, 0, 1], or [0, 1, 1].

Recall the predicted output y^nt={y^n,it}i=1ϱ\widehat{y}^{t}_{n}=\{\widehat{y}^{t}_{n,i}\}_{i=1}^{\varrho}, we sort StS^{t} by Ψ(y^n,1t)\Psi(\widehat{y}^{t}_{n,1}), which is the empirical quantile CDF for y^n,1t\widehat{y}^{t}_{n,1}. Similar to the equation (6), Ψ(y^n,1t)\Psi(\widehat{y}^{t}_{n,1}) is computed in this way:

Ψ(y^n,1t)=1n∑i=1n1{y^i,1t≤y^n,1t}(15)\Psi(\widehat{y}^{t}_{n,1})=\frac{1}{n}\sum_{i=1}^{n}\mathbb{1}\{\widehat{y}^{t}_{i,1}\leq\widehat{y}^{t}_{n,1}\} \tag*{(15)}

The sorted predicted vector y~nt\widetilde{y}^{t}_{n} for stock snts^{t}_{n} at time tt is shown below. Here the stocks are ranked into ϱ\varrho parts and each part contains φ×100%\varphi\times100\% of the stocks, and the parameters ϱ\varrho and φ\varphi satisfy φϱ≤1\varphi\varrho\leq1:

y~nt=[1{(i−1)(ϱ+ξ)≤Ψ(y^n,1t)<iϱ+(i−1)ξ}∣i=1,…,ϱ]∈Rϱ,ξ=1−φϱϱ−1\widetilde{y}^{t}_{n}=\left[\mathbb{1}\left\{(i-1)(\varrho+\xi)\leq\Psi(\widehat{y}^{t}_{n,1})<i\varrho+(i-1)\xi\right\}\mid i=1,\ldots,\varrho\right]\in\mathbb{R}^{\varrho},\quad\xi=\frac{1-\varphi\varrho}{\varrho-1}

Then the weight for each selected stock is computed using equation (16) shown below, and all the chosen stocks are equal-weighted in this way:

wnt=1b⋅ϱ(y~nt−1ΦTy~nt)T⋅1(16)w^{t}_{n}=\frac{1}{\textbf{b}\cdot\varrho}\left(\widetilde{y}^{t-1}_{n}\Phi^{T}\widetilde{y}^{t}_{n}\right)^{T}\cdot\mathbf{1} \tag*{(16)}

The same method is run repeatedly during the subsequent periods. The backtest starts from January 2020, in other words, the result of the sequences from May 2018 to December 2019 will be used as the first stock pool to trade. The exchange fee in the Chinese stock market is 0.00384% (Shanghai stock exchange, 2023; Shenzhen stock exchange, 2023), and the brokerage commission is capped at 0.3% of the transaction amount (China securities regulatory commission, 2002). To keep our backtest results conservative, we set the transaction fee in the backtest to 0.3%.

Algorithm 1 Trading strategy

Require: The feature sequence χnt\chi_n^t for each stock snt∈Sts_n^t \in S^t, initial portfolio cash P0P^0, trading strategy Φ\Phi, strategy decision factor b\mathbf{b}.

1: run_frequency(trade, frequency, startday=1, time=‘open’) 2: function Predict(χnt\chi_n^t): return y^nt=quantformer⁡(χnt)\hat{y}_n^t = \operatorname{quantformer}(\chi_n^t) 3: end function 4: function SortLabel(y^n,1\hat{y}_{n,1}): return y~n,1=[1{i−1ϱ≤Ψ(y^n,1)<iϱ}]i=1ϱ\tilde{y}_{n,1} = \left[\mathbb{1}\left\{\frac{i-1}{\varrho} \leq\Psi(\hat{y}_{n,1}) < \frac{i}{\varrho}\right\}\right]_{i=1}^{\varrho} 5: end function 6: function ComputeWeight(y~nt−1,y~nt,Φ\tilde{y}_n^{t-1}, \tilde{y}_n^t, \Phi): 7: numerator =y~nt−1⋅Φ⊤⋅y~nt= \tilde{y}_n^{t-1} \cdot\Phi^\top\cdot\tilde{y}_n^t 8: denominator =∑k=1Ny~kt−1⋅Φ⋅y~kt= \sum_{k=1}^{N} \tilde{y}_k^{t-1} \cdot\Phi\cdot\tilde{y}_k^t 9: return wnt=1b⋅ϱ⋅numeratordenominatorw_n^t = \frac{1}{b \cdot\varrho} \cdot\frac{\text{numerator}}{\text{denominator}} 10: end function 11: function Trade(wnt,Ptw_n^t, P^t): 12: for snts_n^t in StS^t do order(snt,wnt⋅Pts_n^t, w_n^t \cdot P^t) 13: end for 14: end function

Metrics

The Sharpe Ratio (SR) (Sharpe, 1964) and the α\alpha rate will be used to test the performance of the strategy. The SR is a measure of risk-adjusted return that describes the additional earnings an investor receives for each standard deviation unit increase (equation (17)), where RpR_p is the return of the portfolio and RfR_f is the risk-free rate, chosen to be the London Interbank Offered Rate (LIBOR) which averages rate estimates submitted by banks in London5. The risk-free rate in the backtest will be calculated as the average LIBOR rate during the test period.

SR=E[Rp]−Rfstd⁡[Rp](17)\mathrm{SR} = \frac{\mathbb{E}[R_p] - R_f}{\operatorname{std}[R_p]} \tag*{(17)}

Alpha represents the excess return of an investment portfolio relative to its benchmark. It aims to measure the stock selection skills of an investment strategy. A positive alpha indicates that the investment portfolio has achieved a higher return than its benchmark after risk adjustment. Specifically, alpha is the excess return of the actual portfolio return over its expected theoretical return. Equation (18) shows the calculation method of the alpha rate, where E(Rm)E(R_m) is the market return and β\beta is the correlation between the portfolio and the benchmark’s systemic risk.

α=(Rp−Rf)−β(E[Rm]−Rf)(18)\alpha= (R_p - R_f) - \beta\left(E[R_m] - R_f\right) \tag*{(18)}

Besides the SR ratio and alpha rate, the annual return (AR) of the stock, the annual excess return (AER), the win rate (WR) and the average turnover rate (TR) of the portfolio will also be shown. The excess returns are returns achieved above and beyond the return of a proxy, chosen to be the CSI 300 index. The win rate is the percentage of trading days during which the portfolio strategy generates a positive return. The turnover rate of the portfolio is the proportion of portfolio weight changes between two consecutive rebalancing periods. The definition of TR at each time tt is:

TRt=12∑k=1N∣wnt−wnt−1∣(19)\mathrm{TR}^{t} = \frac{1}{2}\sum_{k=1}^{N}\left|w_n^{t} - w_n^{t-1}\right| \tag*{(19)}

where wntw_n^{t} and wnt−1w_n^{t-1} denote the portfolio weight of the stock sns_n in the portfolio at time tt and t−1t-1, respectively.

Value at risk (VaR) is a method to summarize the total risk in a portfolio (Hull, 2012).

VaRα=inf⁡{x:P(L>x)≤α}(20)\mathrm{VaR}_{\alpha} = \inf\{x : P(\mathcal{L} > x) \le\alpha\} \tag*{(20)}

Equation (20) shows the calculation of VaR, where L\mathcal{L} is the loss of the holding period TT, then VaR⁡(α)\operatorname{VaR}(\alpha) is the α\alpha-th upper quantile of L\mathcal{L}. In the measurement of the portfolio, 99% VaR is used to estimate the maximum loss during the period with 99% confidence.

Results and discussion

Building upon the analysis of individual strategies, this section presents an evaluation and discussion of the experimental outcomes. The discussion focuses on the comparative effectiveness of different time frequencies (monthly, weekly, and daily) and different training scales.

Overall performance

Our results are shown in Table 3. For the monthly group, Month_1 shows an annual return of 17.35% and an annual excess return of 19.43%, indicating a robust performance over that period. Conversely, Month_3 shows a diminished annual return and annual excess return of 7.37% and 9.91% respectively. The is notably high in the second month at 51.69%. The win rate ranges from 43.1% to 57.3%, with the first month showing the highest win rate. The Sharpe Ratio and Alpha vary across the strategies, with some periods showing negative values, suggesting underperformance relative to the risk taken, while others show positive values. In the comparison of 99% VaR, the strategies in monthly frequency perform better than others, where Month_1 and Month_3 are lower than 3%. This means that, with 99% confidence, the portfolio maximum daily losses are 2.81% and 2.3% respectively.

StrategyARAERTRWRSRAlphaVaR
Month_117.35%19.43%26.09%57.8%0.9150.1622.81
Month_29.91%13.86%51.69%49.3%0.2890.1023.61
Month_37.37%9.91%32.33%51.6%0.2460.0642.3
Week_1-0.83%1.31%7.13%46.4%-0.236-0.0303.05
Week_27.49%10.81%1.18%49.2%0.1600.0853.73
Week_312.3%12.73%1.39%54.4%0.3720.1163.77
Day_17.89%11.4%6.71%43.1%0.1810.0903.92
Day_210.23%10.94%6.51%44.4%0.2790.0973.91
Day_39.81%10.03%5.57%44.6%0.2810.0924.02
CSI3001.77%\\\-0.015\3.19%

Table 3. Result of experiments

In the weekly strategy results, Week_3 shows an annual return of 12.3% and an annual excess return of 12.73%, while Week_1 displays a negative annual return. For the daily strategies, Day_2 has a return of 10.23% and an annual excess return of 10.94%. The turnover rate across the daily strategies shows lesser variation than the monthly or weekly, indicating a more consistent trading frequency. The win rate for the daily strategies remain fairly stable, hovering around the mid-40% mark. The Sharpe Ratio and Alpha for weekly and daily strategies show a mix of positive and negative values, reflecting the fluctuating nature of shorter-term trading efficacy.

To demonstrate the advantages of the quantformer factor over traditional factors, 100 price-volume type factors from JoinQuant6 (one of China’s largest quantitative finance platforms) are selected for backtesting under the same trading strategy (Algorithm 1). Price-volume factors are used because these are calculated based on stocks’ prices, volumes, and turnover rates, which are similar to the training data used by quantformer factors. The detailed performance of factors and benchmark is shown in Appendix § 8. On average, these factors achieves an annual return of -3.78%, an average excess return of -2.15%, and a Sharpe ratio of -0.36, where the benchmark return is -1.77%. The average maximum drawdown of the strategy is 44.88%, and the average volatility of the strategy (σp\sigma_p) is 0.26, with the benchmark at 0.197.

Among these factor-based strategies, the quantformer factor performs well. Except for the win rate, which is slightly lower than a few strategies, it performs well in terms of annual return, annual excess return, σp\sigma_p, Sharpe ratio and Sortino ratio, ranking the best among the 101 factors. The 99% VaR of the quantformer factor is among the best 10% factors. Figure 4 illustrates the return curves of the quantformer (QF_Month_1) alongside the benchmark and some traditional factors. “25 week rank” represents the current price’s position over the past 25 weeks; “EMAC20” and “MAC20” are the 20-day index moving average and the stock’s 20-day moving average respectively; “DAVOL20” and “VOL20” represent the ratio of the 20-day average turnover rate to the 120-day average turnover rate, and the mean of the stock’s 20-day turnover rate; “ROC20” is the price rate of change over 20 days; “MAWVAD” is calculated as the product of the difference between the closing price and the opening price, divided by the range of the highest and lowest prices, all multiplied by the volume, accumulated over six days; “Variance20” is the variance of the stock’s 20-day annualized returns. Most of these factors are compared based on a window of 20 timestamps, as the quantformer factor was also trained on 20 timestamps. In comparison with these factors, the quantformer factor (blue line) demonstrates a significantly better performance in terms of returns than the benchmark (orange line) and other factor strategies. Compared to other factors by the Sharpe ratio, the quantformer (0.915) still performs better than others (the highest Sharpe ratio among these factors is 0.243). Furthermore, the quantformer also shows a lower downside risk in terms of risk assessment as it achieves a VaR of 2.81, compared to the highest VaR of 2.86 among other factors. These indicate the improvement of the quantformer factor over traditional price-volume factors.

Backtest results based on different factors

Figure 4. Backtest results based on different factors

Training under different scales

To test the stock selection capabilities of the model, three other factors “QF_10%”, “QF_5%”, “QF_1%” are trained and tested (with “QF_Month_1%”, named as “QF_20%”). The four groups are all trained under ϱ=3\varrho= 3, which include the first, middle, and tail 20%, 10%, 5%, and 1% of stocks in ranking, respectively. The backtest period and other settings are the same as in § 4.

Table 4 shows the result of the backtest. QF_10% strategy records an AR of 13.12% and an AER of 17.73%, with a slightly lower portfolio volatility of 0.159. This strategy has the highest WR at 61.8%. QF-5% strategy delivered 12.59% AR and 16.02% AW, with the lowest portfolio volatility at 0.136. It has the lowest VaR at 2.015%. QF-1% strategy outperformed the others, achieving the highest AR of 24.71% and the highest AER of 35.74%, though it takes the highest volatility at 0.214. These four factors all perform better than CSI 300 based on their return and risk. These results suggest that the quantformer model’s performance seems robust to the choice of different training sets and requirements.

StrategyARAERσp\sigma_pWRSRAlphaVaR
QF-20%17.35%19.43%0.16257.8%0.9150.1622.81%
QF-10%13.12%17.730.15961.8%0.5740.1282.023%
QF-5%12.59%16.02%0.13661%0.630.1172.015%
QF-1%24.71%35.74%0.21453.3%0.9670.2493.048%
CSI3001.77%\0.197\-0.015\3.19%

Table 4. Result of factors under different scales

Conclusion

This study proposes a new neural network architecture, quantformer, inspired by the transformer architecture, for quantitative stock prediction and trading. We address the need for handling numerical input data rather than text, and adapted the model for forecasting tasks rather than sequence-to-sequence problems common in NLP. To enable direct processing of numerical time series data, we replace the word embedding layer with a standard linear layer and removed the output masking operations. We also simplify the decoder to produce a probability distribution over future price movements rather than autoregressively generating token sequences.

We also backtest the quantformer factor on financial market data and compare it with 100 other price-volume factors. Our experimental results demonstrate the promise of this approach. The quantformer-based trading strategies are able to deliver substantial excess returns over the benchmark under a longer time period, with Sharpe Ratios indicating sound risk-adjusted performance. The positive alphas affirm the strategy’s ability to outperform expected returns after accounting for risk factors.

Overall, our work illustrates the viability of the quantformer architecture to handle financial time series, design profitable and robust trading strategies, and calls for further analysis and investigations of this proposed ML-based quantitative methods. The implementation code of quantformer is available at https://github.com/zhangmordred/QuantFormer.

Acknowledgements and declaration

This work was funded by the Natural Science Foundation of China (12271047); Guangdong Provincial/Zhuhai Key Laboratory of Interdisciplinary Research and Application for Data Science, Beijing Normal-Hong Kong Baptist University (2022B1212010006); BNBU research grants (UICR0400008-21; UICR0700041-22; R72021114); and the Guangdong College Enhancement and Innovation Program (2021ZDZX1046).

The authors have no competing interests to declare that are relevant to the content of this article.

Detailed backtest results between 100 factors and the quantformer factor

All the data of factors come from JoinQuant7, which is one of the most common quantitative finance platforms in China. Detailed descriptions of the following factors are available at https://www.joinquant.com/help/api/help#factor_values.

Ten metrics are used to evaluate the performance of each factor:

  • Annual Return (AR) and Annual Excess Return (AER) are two metrics to reflect the return of the portfolio directly. The excess returns are returns achieved above and beyond the return of a proxy and the CSI 300 index is used as the basis when calculating excess returns.

  • Win Rate (WR) is the metric defined as the fraction between the number of trading periods that generate a profit over the total number of trading periods.

  • Sharpe Ratio (SR) is a measure of risk-adjusted return that describes the additional earnings an investor receives for each unit of the standard deviation of returns. It is shown in equation (17).

  • The Alpha measures the ability of a portfolio to generate returns above the market benchmark. A positive alpha indicates that the investment portfolio has achieved a higher return than its benchmark after risk adjustment. It is shown in equation (18).

  • Beta is the correlation between the portfolio and systemic risk, reflecting the sensitivity of the strategy to changes in the market. For the daily return of the strategy, DpD_p, and the daily return of the benchmark DmD_m, the equation of beta is:

Beta⁡=βp=Cov⁡(Dp,Dm)Var⁡(Dm)\operatorname{Beta} = \beta_p = \frac{\operatorname{Cov}(D_p,D_m)}{\operatorname{Var}(D_m)}

where Cov⁡(Dp,Dm)\operatorname{Cov}(D_p,D_m) is the correlation between between DpD_p and DmD_m, and Var⁡(Dm)\operatorname{Var}(D_m) is the variance of the daily returns of the benchmark.

  • Max Drawdown (MD) means the potential worst-case scenario or the most extreme possible loss from the previous peak. For the trough value of a portfolio AtroughA_{\mathrm{trough}} and the previous peak value of the portfolio ApeakA_{\mathrm{peak}}, the MD is calculated in the following way:

MD=Apeak−AtroughApeak\mathrm{MD} = \frac{A_{\mathrm{peak}}-A_{\mathrm{trough}}}{A_{\mathrm{peak}}}
  • Portfolio volatility (σp\sigma_p) is the standard deviation of the portfolio returns. It provides a kind of weighted average deviation in which large deviations carry more weight [3].

  • Sortino Ratio (STN) differentiates harmful volatility from total overall volatility by using the standard deviation of negative portfolio returns. This metric measures the performance of the investment relative to the downward risk.

  • Value-at-Risk (VaR) estimates the potential loss in value of a risky investment, which is used to quantify the amount of potential loss and the likelihood of occurrence for that loss within a specified time frame, which is shown in equation (20). In the backtest, 99%VaR is used.

FactorAR (%)AER (%)WR (%)SRAlphaBetaMD (%)σp\sigma_pVaR (%)STN
ARBR-1.470.3254.7-0.251-0.0020.91434.460.2213.74-0.375
AR-5.87-4.3753.7-0.406-0.0480.98147.840.2603.940.612
ATR14-2.12-0.3762.7-0.2500.0061.16046.550.2504.51-0.35
ATR6-3.94-2.3161.2-0.328-0.0141.17550.110.2534.48-0.455
BBIC1.173.1450.9-0.1090.0331.03832.640.2584.16-0.155
BR-6.91-5.4753.6-0.467-0.0640.93451.720.2543.80-0.718
CCI10-18.65-17.9943.3-1.402-0.2610.81272.040.2202.79-2.026
CCI15-17.70-16.9844.2-1.293-0.2400.83271.430.2232.90-1.866
CCI20-15.52-14.6646.3-1.051-0.1960.86869.120.2353.28-1.519
CR20-4.27-2.6754.8-0.329-0.0320.92749.920.2634.26-0.511
DAVOL10-7.44-6.0454.1-0.504-0.0710.92045.450.2483.38-0.812
DAVOL20-10.95-9.7854.4-0.702-0.1190.91948.960.2462.86-1.079
DAVOL5-9.54-8.2851.8-0.612-0.0970.95046.410.2503.17-1.019
EMA52.774.8548.3-0.0470.0491.04130.700.2594.66-0.068
EMAC100.272.1750.6-0.1440.0241.03731.420.2584.19-0.203
EMAC121.583.5750.6-0.0930.0371.03231.760.2584.17-0.131
EMAC206.018.2954.10.0690.0791.04432.430.2604.320.102
EMAC263.835.9754.1-0.0080.0591.04634.870.2614.29-0.012
EMAC1201.743.7548.3-0.0880.0340.96035.810.2543.77-0.145
Kurtosis20-2.96-1.2752.5-0.395-0.0270.77227.710.1822.74-0.6
Kurtosis60-0.281.5953.9-0.240-0.0010.71622.070.1792.64-0.367
Kurtosis120-4.12-2.5149.0-0.468-0.0430.70828.280.1822.44-0.684
MAC50.542.4646.9-0.1340.0261.03530.030.2574.29-0.195
Cont’d
FactorAR (%)AER (%)WR (%)SRAlphaBetaMD (%)σp\sigma_pVaR (%)STN
MAC10-1.200.6149.6-0.2060.0081.03934.150.2564.19-0.286
MAC202.474.5253.5-0.0590.0461.04334.580.2574.09-0.083
MAC6011.1013.7253.80.2430.1200.98728.610.2544.340.399
MAC1203.115.2147.7-0.0350.0490.98735.400.2593.99-0.058
MACDC-11.12-9.9752.4-0.674-0.1171.00346.480.2613.54-1.046
MASS-10.55-9.3551.0-0.673-0.1150.88648.340.2496.19-1.032
MAWVAD7.299.6755.90.1190.0820.89136.540.2484.190.184
MFI14-6.21-4.7351.5-0.456-0.0560.91845.650.2413.61-0.699
PLRC6-14.83-13.9247.8-0.905-0.1780.96066.240.2593.85-1.368
PLRC12-7.28-5.8753.3-0.452-0.0641.01252.300.2734.20-0.682
PLRC24-5.77-4.2658.1-0.361-0.0441.03453.280.2894.88-0.549
PSY-9.11-7.8350.6-0.660-0.0960.88049.790.2233.36-0.992
Price1M-13.55-12.5553.0-0.760-0.1560.97764.850.2804.21-1.145
Price3M-6.69-5.2558.4-0.380-0.0561.02359.140.3055.44-0.555
Price1Y1.553.5466.8-0.0750.0411.11660.250.3267.28-0.105
ROC6-16.13-15.3047.4-0.982-0.2000.97868.140.2623.56-1.741
ROC20-11.63-10.5154.7-0.628-0.1231.01460.280.2914.51-0.971
ROC60-6.86-5.4359.6-0.382-0.0561.06262.160.3095.28-0.529
ROC1202.364.4166.9-0.0510.0491.11756.360.3196.79-0.071
Skewness20-5.91-4.4150.8-0.538-0.0580.82040.650.1973.040.839
Skewness60-6.47-5.0151.9-0.595-0.0680.76538.020.1902.58-0.917
Skewness120-4.37-2.7751.1-0.4590.0440.74336.010.1922.69-0.684
TRIX5-10.72-9.5452.5-0.607-0.1101.00956.070.2804.16-0.923
TRIX10-3.51-1.8558.4-0.268-0.0191.00852.920.2925.14-0.4
TVMA20-1.92-0.1664.9-0.2090.0141.26844.660.2884.63-0.31
TVMA6-0.751.0962.4-0.1660.0261.25939.460.2884.76-0.246
TVSTD20-1.83-0.0658.5-0.2140.0121.21835.890.2774.03-0.318
TVSTD6-3.77-2.1353.4-0.297-0.0101.20246.570.2724.40-0.449
VDEA-3.05-1.3757.6-0.305-0.0230.85737.010.2393.06-0.484
VDIFF-7.93-6.570.2-0.533-0.0790.89539.790.2473.08-0.842
VEMA5-0.411.4657.9-0.1860.0060.85037.310.2373.33-0.277
VEMA100.952.9158.9-0.1280.0190.84435.510.2363.36-0.19
VEMA121.012.9659.3-0.1260.0200.84035.170.2353.35-0.186
VEMA262.414.4659.3-0.0680.0330.83831.030.2313.37-0.1
VMACD-1.030.7950.5-0.2160.0010.87533.960.2353.38-0.345
VOL5-8.12-6.7655.9-0.452-0.0661.15553.860.2964.48-0.656
VOL10-7.07-5.6458.3-0.404-0.0531.14550.650.2984.35-0.59
VOL20-5.43-3.8958.6-0.337-0.0321.15650.190.2984.33-0.481
VOL600.422.3458.1-0.1250.0321.14538.350.2864.71-0.175
VOL120-1.88-0.1253.0-0.2130.0071.13642.090.2814.64-0.3
VOL240-5.51-3.9847.0-0.382-0.0381.07548.530.2653.99-0.541
VOSC-6.35-4.8851.2-0.453-0.0560.94440.400.2463.44-0.739
VROC6-16.30-15.4942.7-1.179-0.2080.89769.030.2212.91-1.778
VROC12-11.47-10.3446.7-0.793-0.1270.91551.620.2282.74-1.241
VR-8.43-7.1052.8-0.566-0.0830.93248.570.2443.72-0.834
VSTD10-0.771.0756.3-0.2050.0010.84337.750.2343.19-0.309
VSTD20-1.600.1953.7-0.245-0.0070.84437.310.2323.01-0.366
Variance20-1.94-0.1859.9-0.2010.0051.12446.380.3024.84-0.301
Variance609.9612.5165.20.1680.1221.17830.630.3135.910.241
Variance1205.407.6560.80.0400.0831.19339.540.3125.930.058
Volume1M-15.54-14.6820.7-0.867-0.1890.99163.680.2853.61-1.339
WVAD-0.711.1352.3-0.1990.0020.83642.380.2383.67-0.317
arron_down_25-7.35-5.9448.3-0.593-0.0720.88042.630.2092.84-0.626
arron_up_25-15.51-14.6546.4-1.049-0.1950.87066.730.2353.19-1.604
bear_power-13.31-12.3050.8-0.800-0.1540.93459.580.2613.61-1.182
beta10.3012.8762.40.1680.1341.34435.960.3286.760.252
book_to_price1.873.8658.5-0.1170.0310.88926.730.1812.91-0.162
bull_power-14.81-13.9151.1-0.844-0.1760.98465.990.2783.83-1.285
earnings-yield-2.41-0.6852.6-0.348-0.0290.62725.870.1882.55-0.55
growth2.904.9959.9-0.0450.0521.06837.350.2444.50-0.067
leverage2.054.0753.1-0.0890.0230.71922.570.2173.22-0.144
liquidity-2.71-1.0056.7-0.243-0.0021.13741.280.2844.56-0.35
momentum10.9313.5465.8-0.1950.1291.15943.190.3106.820.271
money_flow_20-1.720.0565.50.2020.0161.26944.100.2884.63-0.298
price_no_fq-6.839.1776.10.0950.0921.13244.860.2674.200.131
pull-up2.194.2354.5-0.0680.0441.06233.090.2653.92-0.093
pull-down-2.28-0.5447.6-0.317-0.0150.83134.460.2033.87-0.439
residual_volatility4.136.3062.30.0030.0621.04841.340.2695.250.004
sharpe_ratio_20-11.88-10.7854.0-0.663-0.1290.97957.660.2824.09-1.019
sharpe_ratio_60-7.42-6.0258.8-0.417-0.0651.02965.190.3005.17-0.583
sharpe_ratio_120-0.321.5565.7-0.1430.0191.06260.520.3036.10-0.2
size-6.36-4.8951.8-0.571-0.0580.90636.000.1962.64-0.82
turnover_volatility-10.80-9.6251.8-0.624-0.1081.07557.050.2743.70-0.905
FactorAR (%)AER (%)WR (%)SRAlphaBetaMD (%)σp\sigma_pVaR (%)STN
1day_VPT-16.45-15.6544.8-1.062-0.2110.90470.430.2483.09-1.553
1day_VPT_6-6.36-4.8953.0-0.416-0.0530.99547.060.2693.81-0.82
1day_VPT_12-0.071.8257.9-0.1460.0201.03346.510.2784.39-0.22
25week_close-11.51-10.3951.0-0.859-0.1330.81747.790.2112.44-0.684
QF_Month_117.3519.4357.80.9150.1620.58818.350.1612.810.946
QF_Month_29.9113.5349.30.2890.1020.68426.770.2053.610.372
QF_Month_37.3710.451.60.2460.0640.48317.330.1372.30.309
QF_Week_1-0.831.4146.4-0.236-0.0030.72629.310.2043.05-0.291
QF_Week_27.4910.5549.20.160.0850.79229.080.2193.730.206
QF_Week_312.312.454.40.3720.1160.79329.860.2233.770.48
QF_Day_17.8911.443.10.1810.090.83429.150.1963.920.245
QF_Day_210.2310.6844.40.2790.0970.81930.020.2233.910.361
QF_Day_39.8110.0344.60.2840.0980.80130.10.2231.020.367
QF_10%13.1217.7361.80.7540.1280.59418.320.1592.020.714
QF_5%12.5916.02610.630.1170.50413.30.1362.010.796
QF_1%24.7135.3453.30.9670.2490.66723.280.2143.351.299
Average-2.92-1.2553.28-0.32-0.020.9643.610.253.99-0.467
CSI3001.77\\0.009\\\0.1973.19\

Table 2 (continued). ’d

Comparison between different quantformer strategies

Comparison between factor performance and quantformer performance

Figure B.5. Comparison between factor performance and quantformer performance

In the provided scatterplot B.5, the QF_Month_1 factor exhibits notable performance in terms of annual return. Compared to other factors, QF_Month_1 offers better returns. Regarding risk, as measured by the Variance Covariance VaR 99, the QF_Month_1 factor’s data points are positioned in a moderately high range, indicating a balanced risk profile. The size and color intensity of the QF_Month_1 points, reflecting higher Alpha values, suggest a robust excess return or relative performance. Combining these observations, the QF_Month_1 factor likely provides a compelling investment opportunity by delivering robust returns while maintaining a moderate risk level, an attractive proposition for strategies aiming to optimize the trade-off between risk and return. Overall, most of the factors based on quantformer perform better compared with the other 100 factors.

Symbol table

SymbolDescription
Apeak,AtroughA_{\mathrm{peak}}, A_{\mathrm{trough}}peak value of the portfolio
α\alphaalpha ratio
b\mathbf{b}decision factor in the strategy
θ\thetabias in the model
βp\beta_{p}beta ratio
DpD_{p}daily return
h,Hh, Hattention head
χnt,χ~nt\chi_{n}^{t}, \widetilde{\chi}_{n}^{t}input feature matrix, normalized sequence
Y^nt\widehat{Y}_{n}^{t}predicted output
E[x]\mathbb{E}[x]mean
Gt\mathcal{G}^{t}set of the next timestamp profit
MD\mathrm{MD}max drawdown
tttimestep
mmthe number of timesteps before tt
MSELoss\mathrm{MSE}_{\mathrm{Loss}}MSE loss
nnthe index of the stock
pntp_{n}^{t}the close price of the stock
PPamount value of portfolio
Φ\Phitrading strategy
Ψ\Psithe empirical quantile CDF
Rf,Rm,RpR_{f}, R_{m}, R_{p}return of risk-free rate, market return, return of portfolio
rntr_{n}^{t}profit rate
snts_{n}^{t}stock
StS^{t}stock set
SR\mathrm{SR}Sharpe ratio
std⁡[x]\operatorname{std}[x]standard deviation
TR\mathrm{TR}turnover rate of the portfolio
vntv_{n}^{t}turnover rate of the stock over the time period tt
VaRα\mathrm{VaR}_{\alpha}value at risk at α\alpha level
wntw_{n}^{t}weight of portfolio
xnt,x~ntx_{n}^{t}, \widetilde{x}_{n}^{t}row vector, normalized vector
ynt,y~nty_{n}^{t}, \widetilde{y}_{n}^{t}target label, sorted predicted output
ϱ\varrhonumber of bins in ynty_{n}^{t}
φ\varphipercentage of stocks for each ϱ\varrho
ξ\ximiddle percentage of ynty_{n}^{t}
L\mathcal{L}loss of holding stock
F\mathbf{F}multi-head attention output
K,Q,V\mathbf{K}, \mathbf{Q}, \mathbf{V}key, query, value in quantformer
W\mathbf{W}trainable weight matrix
Z\mathbf{Z}output layer

Table 1 (PDF p. 40).

References

  1. [1]D. Araci. FinBERT: financial sentiment analysis with pre-trained language models. arXiv:1908.10063, 2019.arxiv.org/abs/1908.10063
  2. [2]C. S. Asness. The power of past stock returns to explain future stock returns. SSRN:2856769, 1995.
  3. [3]C. R. Bacon. Practical portfolio performance measurement and attribution. John Wiley & Sons, Chennai, 2023.DOI
  4. [4]G.-Y. Ban, N. El Karoui, and A. E. Lim. Machine learning and portfolio optimization. Management Science, 64(3):1136–1154, 2018.
  5. [5]W. Bao, J. Yue, and Y. Rao. A deep learning framework for financial time series using stacked autoencoders and long-short term memory. PloS One, 12(7):e0180944, 2017.DOI
  6. [6]F. Barez, P. Bilokon, A. Gervais, and N. Lisitsyn. Exploring the advantages of transformers for high-frequency trading. arXiv:2302.13850, 2023.arxiv.org/abs/2302.13850
  7. [7]J. Bergstra and Y. Bengio. Random search for hyper-parameter optimization. Journal of Machine Learning Research, 13(2), 2012.

Paper details

vlnc:W_spm6zk2q
Version
v1 · latest
Published
Fresh — added to the Hub within the last 30 days
Added
License
CC BY 4.0
Usage

views · 0

downloads · 0

A view counts once per browser per day, a download each time the PDF is requested.

  • Economics, Econometrics and Finance

Not recorded for this paper

View statement
Looking for related papers…
Contents