Stock Index Prediction using Cointegration test and Quantile Loss

09/29/2021
by   Jaeyoung Cheong, et al.
Yonsei University
0

Recent researches on stock prediction using deep learning methods has been actively studied. This is the task to predict the movement of stock prices in the future based on historical trends. The approach to predicting the movement based solely on the pattern of the historical movement of it on charts, not on fundamental values, is called the Technical Analysis, which can be divided into univariate and multivariate methods in the regression task. According to the latter approach, it is important to select different factors well as inputs to enhance the performance of the model. Moreover, its performance can depend on which loss is used to train the model. However, most studies tend to focus on building the structures of models, not on how to select informative factors as inputs to train them. In this paper, we propose a method that can get better performance in terms of returns when selecting informative factors using the cointegration test and learning the model using quantile loss. We compare the two RNN variants with quantile loss with only five factors obtained through the cointegration test among the entire 15 stock index factors collected in the experiment. The Cumulative return and Sharpe ratio were used to evaluate the performance of trained models. Our experimental results show that our proposed method outperforms the other conventional approaches.

READ FULL TEXT VIEW PDF

Authors

page 1

page 2

page 3

page 4

06/17/2021

Design and Analysis of Robust Deep Learning Models for Stock Price Prediction

Building predictive models for robust and accurate prediction of stock p...
01/26/2022

Machine Learning for Stock Prediction Based on Fundamental Analysis

Application of machine learning for stock prediction is attracting a lot...
09/20/2020

Stock Price Prediction Using Machine Learning and LSTM-Based Deep Learning Models

Prediction of stock prices has been an important area of research for a ...
07/30/2020

Prediction of stock movement using phase space reconstruction and extreme learning machines

Stock movement prediction is regarded as one of the most difficult, mean...
08/26/2020

Share Price Prediction of Aerospace Relevant Companies with Recurrent Neural Networks based on PCA

The capital market plays a vital role in marketing operations for aerosp...
08/07/2019

HATS: A Hierarchical Graph Attention Network for Stock Movement Prediction

Many researchers both in academia and industry have long been interested...
08/05/2021

Two-Stage Sector Rotation Methodology Using Machine Learning and Deep Learning Techniques

Market indicators such as CPI and GDP have been widely used over decades...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Recently, it has been actively studied to predict stock index prices using deep learning [1], [2]. This task is to predict stock index prices in the future based on historical stock index prices. Traditionally, the approach of stock index prediction is divided into Fundamental Analysis and Technical analysis. The former is the method to evaluate the value of a company by analyzing its financial statements, soundness, etc. On the other hand, the latter is the method to predict the price of a stock index (or stock price of a company) based on trends of the historical price of it [3]. The Deep Learning methods play an important role in stock (index) prediction. They are used to train the information of historical patterns. This trained model can predict stock prices in the future. However, there are many different methods of how to train models, and how to select informative factors as inputs that are also stock index such as gold, crude oil, etc [4].

To the best of our knowledge, no existing works have studied the cointegration-test and quantile loss simultaneously in stock index prediction using deep learning. Firstly, A vast majority of works have not focused on how to select factors to train a model as input data in the stock prediction. Especially, no earlier works have studied that using cointegration test to select meaningful factors can provide better performance in terms of returns than in the case it is not used. In the TA, to predict stock index, it is important to select different factors well that are stock (index) prices related to factors we will predict such as s&p500, etc. Next, as recent studies point out, the performance of deep learning models depends partly on which loss function you choose, such as quantile loss, rmse loss. In most regression tasks, mean absolute error and mean squared error are used popularly to train deep learning models. However, the quantile loss that we will reexamine in this paper gives better performance than the mse loss when evaluating it in terms of returns.

This paper aims to address that cointegration-test is excellent to select factors as inputs learning a model and also quantile loss is better than rmse loss that is commonly used to get higher returns.

Our study’s key contributions are as follows:

  • We show that when building a deep learning-based prediction model to predict the stock index, factor selection method using cointegration test improves the performance of the model when evaluating the model in terms of returns

  • We show that a regression model using Quantile Loss rather than the commonly used RMSE Loss gives better performance of the model in terms of returns

2 Related Work

2.1 Cointegration Test

Empirically, most financial time-series are non-stationary. Some researchers analyze whether there is a long-run equilibrium between this unstable financial time-series, and then perform stock prediction based on multiple regression using several informative factors as explanatory factors. In general, many of the researchers perform an Augmented Dickney-Fuller test[5], a kind of unit root test, to determine whether the data we collect is stationary or not before performing a cointegration test. The ADF test is a kind of Dickney-Fuller test, which is the process of examining whether it is or

in expressions modified from the random probability model

to . The Cointegration test [6] is performed after the Unit Root Test is done. We use the Johansen test [7], which is widely used among the variety of methods for Cointegration tests. Using this method, we can determine whether there is a long-term dependence relationship between time series without obtaining stationary via differentiation from non-stationary time series. Let’s explore how to verify the existence of cointegration with the following VAR(2) model. Equation (1) follows the process below and results in (5).

(1)
(2)
(3)
(4)
(5)

In Equation (5), it is interpreted that cointegration does not exist in time series if , and cointegration exists in time series if . If the above test method determines whether the cointegration exists, it can be seen that a long-term dependence relationship between time series holds.

2.2 Correlation

As another method for factor selection, we adopt a method that utilizes Pearson Correlation [8] to determine the correlation between financial time series. The independent data for the dependent factor will have a correlation coefficient close to zero, and the positively correlated data will be close to one. In our work, we select five arbitrary with a high positive correlation for the dependent factor and use them as the learning data of the prediction model. The formula for obtaining Pearson correlation coefficients is shown below.

(6)

where is the correlation coefficient, the closer to 1, the two factors will be positively correlated, and the closer to -1, the negatively correlated.

2.3 Quantile Loss for Stock Index Prediction

The loss function is used as an indicator to evaluate how well the designed regression model fits the data. By finding the weights that minimize the value of the loss function, we can design a regression model suitable for the data. We try to adopt a model with higher performance by comparing RMSE which is commonly used in regression models and Quantile Loss.

The Quantile loss is a loss function for predicting intervals. Since linear regression cannot be used when the data do not follow a normal distribution, quantile loss is used to predict the Q-quantiles of the outcome factor. In particular, in the case of financial products, complete risk protection is essential to form a portfolio. For this purpose, it is effective to use a quantile loss that is robust to outliers and can more accurately predict the tail of the revenue distribution

[9]. As [10] showed that information can be effectively used in the overall distribution by using quantile loss, our study also uses it to prove that it is a better loss function in terms of return.

2.4 Backtesting

Backtesting is a method of verifying the effectiveness of an investment strategy by testing it based on the predicted price information of the specific portfolio such as S&P500. The position of the S&P500 index in backtesting is based on the Buy-Sell-Hold trading strategy. This strategy is as follows: it buys stocks when the target price increases significantly, sells them when the target price decreases and holds them when the target price does not increase or decrease. The aim of this strategy maximizes the Cumulative return [11] of the portfolio S&P500 during the period of backtesting.

(7)
  • the Daily Return

  • the Period of Backtesting

  • the Cumulative Return

We need to study what is the Daily Return. With an S&P500 index, we will analyze the . Firstly, the function to set the position of the S&P500 index in backtesting is called , and the log return of the S&P500 index is called . The is defined as follows. The is the actual value of the S&P500 divided by the predicted value.

(8)

Then, we can find the position value for the S&P500 using the position function and . In the formula below, the map applies

to each element of the vector in

.

(9)

Next, the log return of the S&P500 index is defined as follows,

(10)
  • : the first value of S&P500 index

  • : the last value of S&P500 index


Using the position value for the S&P500 and the log return , the daily return can be calculated as follows,

(11)

Therefore, the Cumulative return [11] of the portfolio S&P500 is expressed as:

(12)

3 Methods

We compare the traditional methods and suggested techniques in stock prediction to evaluate the benefits of factor selection and quantile loss introduced above. The general method takes all data as inputs for learning a model without filtering all the collected data and applies commonly used RMSE loss in a forecasting task using deep learning. Our proposed ideas are as follows. In figure 1, we present the illustration of this process. Firstly, Informative factors are selected to get the higher performance of the prediction model using two methods of factor selection that are the cointegration test and Pearson correlation ordering. Next, the model is trained. We use two architectures that are LSTM, GRU, and two-loss functions that are Quantile loss, RMSE loss. Finally, the trained model is evaluated using backtesting in terms of returns.

Figure 1: A Process of our proposed idea

3.1 Model

Time-series data has time-order information. In a methodology of the deep learning, to address this sequence data, it is general to use Recurrent Neural Network

[12], [13]. Moreover, in the Stock Prediction task, it is popular to adopt the RNN architecture [1]. The Recurrent model has the form , which means that it takes at a current timestep and hidden state at as input data and outputs

. It implies that the RNN memorizes the previous information to update current cell. However, this vanila RNN cannot capture long-term dependencies. To resolve this limitation, Long Short-Term Memory(LSTM)

[14]

and (GRU) Gated recurrent unit

[15] are designed. Actually, both have been used frequently in the domain of stock prediction [1],[16]. In figure 2, we illustrate the mechanism of vanila RNN that is the foundation of LSTM, GRU in stock prediction. The length of data is 601 ranged from 2018-03-05 to 2021-03-05. For the input , d implies the day and where the is the number of selected stocks. The 5 implies the number of a weekday. At every timestep, the model predicts the S&P500 stock index price of the next day. In the final layer, there is a Fully Connected layer where the h is hidden size. Finally, the predicted stock index price in the future is output from the .

Figure 2: Architecture of the RNN model for stock index prediction

3.2 Loss

In this study, the quantile loss is calculated using a total of three quantiles: 0.1, 0.5, and 0.9. The average of the calculated loss of each quantile becomes the final quantile loss of the entire model. The definition of the calculation formula is as follows.

(13)
(14)
  • : a quantile

  • : i-th predicted value

  • : i-th actual value


3.3 Evaluation

To evaluate the performance of our trading strategy for the portfolio S&P500, we use two performance metrics that are Cumulative returns [11] and Sharpe ratio [17] [18] [19]. Thus, we can evaluate the portfolio S&P500 by comparing the two performance metrics of the portfolio S&P500 index that we predict using the model and the SPY index that is the S&P500 index in the actual world. In the Experiment, we compare both the Cumulative Return and the Sharpe Ratio between the portfolio S&P500 index (12) and the SPY index. The Cumulative return of can be obtained as follows,

(15)
  • : the log return of SPY index


We can evaluate the portfolio by comparing the Cumulative Return, which is the value of (12) and (15).

The second performance metric, the Sharpe ratio, can be expressed as follows,

(16)

Let the Sharpe ratio of the portfolio S&P500 be and the Sharpe ratio of SPY be . They can be expressed as

(17)

As well as the Cumulative return, you can evaluate the portfolio with the value them.

4 Experiments

4.1 Data

To predict the value of the S&P500 index in the future, we selected 15 stock index candidates for inputs and then trained the model using them or some of them picked by the method of factor selection. These time series were collected from March 5th, 2018 to March 5th, 2021. All of the collected data were used as inputs in the case that we perform a conventional method to predict stock index in the future, but when we do the suggested methods such as coordination test and Pearson-correlation to select important factors we use only 5 stock indexes of them.

All data were collected from Yahoo Finance except for the U.S Federal Funds Rate. We collected Close Price data from March 15, 2018, to March 15, 2021. We processed missing values found in some indexes using interpolation. To perform a cointegration test, we do the ADF test on each index to identify whether all factors (stock indices) are stationary. The table below is the results.

Results of stock selection methods
Index List Ticker ADF-test (p-value) Cointegration test (p-value)
snp500 ^GSPC 0.742 0.000
Russell2000 ^RUT 0.776 0.888
Dow Jones Industrial Average ^DJI 0.299 0.680
NASDAQ ^IXIC 0.946 0.102
USD/JPY JPY=X 0.093 0.160
Bitcoin USD BTC-USD 1.000 0.061
FTSE 100 ^FTSE 0.381 0.508
Nikkei 225 ^N225 0.766 0.740
Treasury Yield 10 Years ^TNX 0.911 0.427
EUR/USD EURUSD=X 0.694 0.647
Gold Aug 21 GC=F 0.798 0.518
Silver Jul 21 SI=F 0.144 0.700
GPB/USD GBPUSD=X 0.148 0.491
U.S Federal Funds Rate 0.909 0.229
Crude oil CL=F 0.348 0.773

4.2 Method

We compare all of the above factors as inputs to examine which model is best. However, in the case that uses Pearson Correlation, we select only five factors as inputs where we chose them in the highest order of correlation coefficient. On the other hand, in the case that uses the Cointegration test, we select five factors in the lowest order of p-value. Furthermore, we compare the utility of two using rmse loss and quantile loss respectively to see whether the model performs better in terms of returns in the case that we do not use rmse loss which is frequently used in regression tasks. Finally, we do this experiment by adopting the most popular advanced RNN architecture designed to predict the price in the future of snp500. The table below shows the results of the experiment.

Results of backtesting
Cumulative return Sharpe ratio
All+Quantile+LSTM 4.304 0.145
All+Quantile+GRU 4.397 0.147
Correlation+Quantile+LSTM 26.781 0.314
Correlation+Quantile+GRU 28.179 0.319
Cointegration+Quantile+LSTM 33.491 0.339
Cointegration+Quantile+GRU 36.779 0.351
All+RMSE+LSTM 2.382 0.089
All+RMSE+GRU 2.742 0.100
Correlation+RMSE+LSTM 11.171 0.230
Correlation+RMSE+GRU 12.963 0.245
Cointegration+RMSE+LSTM 3.838 0.128
Cointegration+RMSE+GRU 3.838 0.128

In the above table, the ‘All’ is the case that we use all given data. The ‘Quantile’ refers to the case that we use quantile loss, and the ‘LSTM’ also refers to the case that we use LSTM architecture. ‘All+Quantile+LSTM’ is the combination of these cases. ‘Correlation’ and ‘Cointegration’ are also the cases we use Pearson Correlation ordering and Cointegration test as introduced above to select informative factors.

4.3 Discussion

Analyzing based on the results of the above experiments, we can obtain the following insights: First, using factor selection methods gives us better performance in terms of returns than using all the collected data. Next, in our experiment, using the Cointegration test for factor selection gives us better performance than using correlation analysis. Third, using Quantile loss gives much better performance than using RMSE loss. Finally, in building a prediction model, LSTM and GRU do not make significant differences in performance.

In fact, in the case of the Cointegration test that tests the long-run equilibrium relationship between two factors, if the p-value is more than 0.05, some statisticians think that it can result in that the two factors do not have a long-term equilibrium relationship. However, in our experiment, we selected only five factors with the lowest p-value among them and used them as input values, even if the p-value of the cointegration test between the snp500 and all the remaining factors exceeds 0.05. Nevertheless, we suggest that the cointegration test can be a good choice for building a better prediction model for the stock index, given that the cointegration test used this test to input the selected factors, although the p-value of the test is higher than 0.05.

4.4 Limitation

Our study might have restraints. We need to point out the possible limitations of this study. Firstly, the 15 stock indexes used in this experiment can be a factor depending on the researcher’s decision. The settings of the number and components of explanatory factors for training models can vary widely. For example, the number of inputs data used in [20] and [21] differs from that of the explanatory factors adopted in our study. Next, the experimental results may vary depending on the length of the data. The data used in this experiment are collected from March 5th, 2018 to March 5th, 2021. The results of the cointegration test with snp500 and the remaining explanatory factors collected during this period do not show that p-value < 0.05. On the other hand, the results of it with snp500 and remainings collected over a longer period show that the relationship between the Dow index and the snp500 is long-term equilibrium (p < 0.05).

5 Conclusion

The target of this paper was to show that a factor selection method and Quantile loss, which have not been highlighted in the stock index prediction task using deep learning, can be used to build the prediction model with better performance in terms of returns. We found that using factor selection methods such as the Cointegration test makes the model better performance in terms of returns than using all the stock index data collected in this work. Furthermore, in this Stock Index Prediction, we found that using Quantile Loss would return better returns than using RMSE Loss, which is often used in regression or prediction tasks. In the future, we will further explore whether other architectures such as Graph Neural Networks, Transformers, etc can perform better than the variant RNNs used in this work.

References

  • [1] Ahmet Murat Ozbayoglu, Mehmet Ugur Gudelek, and Omer Berat Sezer. Deep learning for financial applications: A survey. Applied Soft Computing, 93:106384, 2020.
  • [2] Weiwei Jiang. Applications of deep learning in stock market prediction: recent progress. arXiv preprint arXiv:2003.01859, 2020.
  • [3] Andrea Picasso, Simone Merello, Yukun Ma, Luca Oneto, and Erik Cambria. Technical analysis and sentiment embeddings for market trend prediction. Expert Systems with Applications, 135:60–70, 2019.
  • [4] Fernando Jimenez, Jose Palma, Gracia Sanchez, David Marin, MD Francisco Palacios, and MD Lucía López. Feature selection based multivariate time series forecasting: An application to antibiotic resistance outbreaks prediction. Artificial Intelligence in Medicine, 104:101818, 2020.
  • [5] David A Dickey and Wayne A Fuller.

    Distribution of the estimators for autoregressive time series with a unit root.

    Journal of the American statistical association, 74(366a):427–431, 1979.
  • [6] Robert F Engle and Clive WJ Granger. Co-integration and error correction: representation, estimation, and testing. Econometrica: journal of the Econometric Society, pages 251–276, 1987.
  • [7] Gerald P Dwyer. The johansen tests for cointegration. White Paper, 2015.
  • [8] Imane Jebli, Fatima-Zahra Belouadha, Mohammed Issam Kabbaj, and Amine Tilioua.

    Prediction of solar energy guided by pearson correlation using machine learning.

    Energy, 224:120109, 2021.
  • [9] David E Allen and AK Singh. Minimizing loss at times of financial crisis: Quantile regression as a tool for portfolio investment decisions. school of accounting. Finance and Economics & FEMARC Working Paper Series, 2009.
  • [10] DE Allen, RJ Powell, and AK Singh. Quantile regression as a tool for portfolio investment decisions during times of financial distress. Annals of Financial Economics, 6(01):1150003, 2011.
  • [11] Mykola Babiak and Jozef Baruník. Deep learning, predictability, and optimal portfolio returns. arXiv preprint arXiv:2009.03394, 2020.
  • [12] David E Rumelhart, Geoffrey E Hinton, and Ronald J Williams. Learning representations by back-propagating errors. nature, 323(6088):533–536, 1986.
  • [13] Paul J Werbos. Backpropagation through time: what it does and how to do it. Proceedings of the IEEE, 78(10):1550–1560, 1990.
  • [14] Sepp Hochreiter and Jürgen Schmidhuber. Long short-term memory. Neural computation, 9(8):1735–1780, 1997.
  • [15] Kyunghyun Cho, Bart Van Merriënboer, Caglar Gulcehre, Dzmitry Bahdanau, Fethi Bougares, Holger Schwenk, and Yoshua Bengio. Learning phrase representations using rnn encoder-decoder for statistical machine translation. arXiv preprint arXiv:1406.1078, 2014.
  • [16] Kerda Varaku. Stock price forecasting and hypothesis testing using neural networks. Available at SSRN 3597684, 2020.
  • [17] J Dave Jobson and Bob M Korkie. Performance hypothesis testing with the sharpe and treynor measures. Journal of Finance, pages 889–908, 1981.
  • [18] Christoph Memmel. Performance hypothesis testing with the sharpe ratio. Available at SSRN 412588, 2003.
  • [19] Pui-Lam Leung and Wing-Keung Wong. On testing the equality of the multiple sharpe ratios, with application on the evaluation of ishares. Available at SSRN 907270, 2006.
  • [20] M Hiransha, E Ab Gopalakrishnan, Vijay Krishna Menon, and KP Soman. Nse stock market prediction using deep-learning models. Procedia computer science, 132:1351–1362, 2018.
  • [21] Shruti Shakhla, Bhavya Shah, Niket Shah, Vyom Unadkat, and Pratik Kanani. Stock price trend prediction using multiple linear regression. Int. J. Eng. Sci. Inven.(IJESI), 7(10):29–33, 2018.