Leverage Financial News to Predict Stock Price Movements Using Word Embeddings and Deep Neural Networks

06/24/2015
by   Yangtuo Peng, et al.
York University
0

Financial news contains useful information on public companies and the market. In this paper we apply the popular word embedding methods and deep neural networks to leverage financial news to predict stock price movements in the market. Experimental results have shown that our proposed methods are simple but very effective, which can significantly improve the stock prediction accuracy on a standard financial database over the baseline system using only the historical price information.

READ FULL TEXT VIEW PDF

page 1

page 2

page 3

page 4

08/23/2020

Towards Earnings Call and Stock Price Movement

Earnings calls are hosted by management of public companies to discuss t...
05/22/2017

Predicting stock market movements using network science: An information theoretic approach

A stock market is considered as one of the highly complex systems, which...
09/09/2020

Impact of News on the Commodity Market: Dataset and Results

Over the last few years, machine learning based methods have been applie...
06/15/2021

StockBabble: A Conversational Financial Agent to support Stock Market Investors

We introduce StockBabble, a conversational agent designed to support und...
09/07/2019

A Deep Learning Framework for Pricing Financial Instruments

We propose an integrated deep learning architecture for the stock moveme...
05/03/2021

The Doge of Wall Street: Analysis and Detection of Pump and Dump Cryptocurrency Manipulations

Cryptocurrencies are increasingly popular. Even people who are not exper...
01/31/2022

An Exploratory Study of Stock Price Movements from Earnings Calls

Financial market analysis has focused primarily on extracting signals fr...

1 Introduction

In the past few years, deep neural networks (DNNs) have achieved huge successes in many data modeling and prediction tasks, ranging from speech recognition, computer vision to natural language processing. In this paper, we are interested in applying the powerful deep learning methods to financial data modeling to predict stock price movements.

Traditionally neural networks have been used to model stock prices as time series for the forecasting purpose, such as in [Kaastra and Boyd1991, Adya and Collopy1991, Chan et al.2000, Skabar and Cloete2002, Zhu et al.2008]

. In these earlier work, due to the limited training data and computing power available back then, normally shallow neural networks were used to model various types of features extracted from stock price data sets, such as historical prices, trading volumes, etc, in order to predict future stock yields and market returns. More recently, in the community of natural language processing, many methods have been proposed to explore additional information (mainly online text data) for stock forecasting, such as financial news

[Xie et al.2013, Ding et al.2014], twitters sentiments [Si et al.2013, Si et al.2014], microblogs [Bar-Haim et al.2011]. For example, [Xie et al.2013]

propose to use semantic frame parsers to generalize from sentences to scenarios to detect the (positive or negative) roles of specific companies, where support vector machines with tree kernels are used as predictive models. On the other hand,

[Ding et al.2014]

propose to use various lexical and syntactic constraints to extract event features for stock forecasting, where they have investigate both linear classifiers and deep neural networks as predictive models.

In this paper, we propose to use the recent word embedding methods [Mikolov et al.2013b] to select features from on-line financial news corpora, and employ deep neural networks (DNNs) to predict the future stock movements based on the extracted features. Experimental results have shown that the features derived from financial news are very useful and they can significantly improve the prediction accuracy over the baseline system that only relies on the historical price information.

2 Our Approach

In this paper, we use deep neural networks (DNNs) as our predictive model, which takes as input the features extracted from both historical price information and on-line financial news to predict the stock movements in the future (either up or down).

2.1 Deep Neural Networks

The structure of DNNs used in this paper is a conventional multi-layer perceptron with many hidden layers. An

-layer DNN consisting of

hidden nonlinear layers and one output layer. The output layer is used to model the posterior probability of each output target. In this paper, we use the rectified linear activation function, i.e.,

, to compute from activations to outputs in each hidden layer, which are in turn fed to the next layer as inputs. For the output layer, we use the softmax function to compute posterior probabilities between two nodes, standing for stock-up and stock-down.

2.2 Features from historical price data

In this paper, for each target stock on a target date, we choose the previous five days’ closing prices and concatenate them to form an input feature vector for DNNs: , where denotes the target date, and denotes the closing price on the date

. We then normalize all prices by the mean and variance calculated from all closing prices of this stock in the training set. In addition, we also compute first and second order differences among the five days’ closing prices, which are appended as extra feature vectors. For example, we compute the first order difference as follows:

. In the same way, the second order difference is calculated by taking the difference between two adjacent values in each . Finally, for each target stock on a particular date, the feature vector representing the historical price information consists of , and .

2.3 Financial news features

In order to extract fixed-size features suitable to DNNs from financial news corpora, we need to pre-process the text data. For all financial articles, we first split them into sentences. We only keep those sentences that mention at least one stock name or one public company. Each sentence is labelled by the publication date of the original article and the mentioned stock name. It is possible that multiple stocks are mentioned in one sentence. In this case, this sentence is labeled several times for each mentioned stock. We then group these sentences by the publication dates and the underlying stock names to form the samples. Each sample contains a list of sentences that were published on the same date and mentioned the same stock or company. Moreover, each sample is labelled as positive (“price-up”) or negative (“price-down”) based on its next day’s closing price consulted from the CRSP financial database [Booth2012]. In the following, we introduce our method to extract three types of features from each sample.

(1) Bag of keywords (BoK): We first select the keywords based on the recent word embedding methods in [Mikolov et al.2013a, Mikolov et al.2013b]. Using the popular word2vec method from Google111https://code.google.com/p/word2vec/, we first compute the vector representations for all words occurring in the training set. Secondly, we manually select a small set of seed words, i.e., nine words of {surge, rise, shrink, jump, drop, fall, plunge, gain, slump} in this work, which are believed to have a strong indication to the stock price movements. Next, these seed words are used to search for other useful keywords based on the cosine distances calculated between the word vector of each seed word and that of other words occurring in the training set. For example, based on the pre-calculated word vectors, we have found other words, such as rebound, decline, tumble, slowdown, climb, which are very close to at least one of the seed words. In this way, we have searched all words occurring in training set and kept the top 1,000 words (including the nine seed words) as the keywords for our prediction task. Finally, a 1000-dimension feature vector, called bag-of-keywords or BoK, is generated for each sample. Each dimension of the BoK vector is the TFIDF score computed for each selected keyword from the whole training corpus.

(2) Polarity score (PS): We further compute so-called polarity scores [Turney and Littman2003, Turney and Pantel2010] to measure how each keyword is related to stock movements and how each keyword applies to a target stock in each sentence. To do this, we first compute the point-wise mutual information for each keyword : where denotes the frequency of the keyword occurring in all positive samples, denotes the total number of samples in the training set, denotes the total number of keyword occurring in the whole training set and denotes the total number of positive samples in the training set. Furthermore, we calculate the polarity score for each keyword as: Obviously, the above polarity score measures how (either positively or negatively) each keyword is related to stock movements and by how much.

Next, for each sentence in all samples, we need to detect how each keyword is related to the mentioned stock. To do this, we use the Stanford parser [Marneffe et al.2006] to detect whether the target stock is a subject of the keyword or not. If the target stock is not the subject of the keyword in the sentence, we assume the keyword is oppositely related to the underlying stock. As a result, we need to flip the sign of the polarity score. Otherwise, if the target stock is the subject of the keyword, we keep the keyword’s polarity score as it is. For example, in a sentence like “Apple slipped behind Samsung and Microsoft in a 2013 customer experience survey from Forrester Research”, we first identify the keyword slipped, based on the parsing result, we know Apple is the subject while Samsung and Microsoft are not. Therefore, if this sentence is used as a sample for Apple, the above polarity score of “slipped” is directly used. However, if this sentence is used as a sample for Samsung or Microsoft, the polarity score of “slipped” is flipped by multiplying .

Finally, the resultant polarity scores are multiplied to the TFIDF scores to generate another 1000-dimension feature vector for each sample.

(3) Category tag (CT): We further define a list of categories that may indicate a specific event or activity of a public company, which we call as category tags. In this paper, the defined category tags include: new-product, acquisition, price-rise, price-drop, law-suit, fiscal-report, investment, bankrupt, government, analyst-highlights. Each category is first manually assigned with a few words that are closely related to the category. For example, we have chosen released, publish, presented, unveil as a list of seed words for the category new-product, which indicates the company’s announcement of new products. Similarly, we use the above word embedding model to automatically expand the above word list by searching for more words that have closer cosine distances with the selected seed words. In this paper, we choose the top 100 words to assign to each category.

After we have collected all key words for each category, for each sample, we count the total number of occurrences of all words under each category, and then we take the logarithm to obtain a feature vector as , where denotes the total number of times the words in category appear in a sample.

2.4 Predicting Unseen Stocks via Correlation Graph

There are a large number of stocks trading in the market. However, we normally can only find a fraction of them mentioned in daily financial news. Hence, for each date, the above method can only predict those stocks mentioned in the news. In this section, we propose a new method to extend to predict more stocks that may not be directly mentioned in the financial news. Here we propose to use a stock correlation graph, shown in Figure 1, to predict those unseen stocks. The stock correlation graph is an undirected graph, where each node represents a stock and the arc between two nodes represents the correlation between these two stocks. For example, if some stocks in the graph are mentioned in the news on a particular day, we first use the above method to predict these mentioned stocks. Afterwards, the predictions are propagated along the arcs in the graph to generate predictions for those unseen stocks.

Figure 1: Illustration of a part of correlation graph

(1) Build the graph: We choose the top 5,000 stocks from the CRSP database [Booth2012] to construct the correlation graph. At each time, any two stocks in the collection are selected to align their closing prices based on the related dates (between 2006/01/01 - 2012/12/31). Then we calculate the correlation coefficient between the closing prices of these two stocks. The computed correlation coefficient (between and ) is attached to the arc connecting these two stocks in the graph, indicating their price correlation. The correlation coefficients are calculated for every pair of stocks from the collection of 5,000 stocks. In this paper we only keep the arcs with an absolute correlation value greater than , all other edges are considered to be unreliable and pruned from the graph, a tiny fraction of which is shown in Figure 1.

(2) Predict unseen stocks: In order to predict price movements of unseen stocks, we first take the prediction results of those mentioned stocks from the DNN outputs, by which we construct a 5000-dimension vector . Each dimension of corresponds to one stock and we set zeros for all unseen stocks. The above graph propagation process can be mathematically represented as a matrix multiplication: , where is a symmetric matrix denoting all correlation weights in the graph. Of course, the graph propagation, i.e. matrix multiplication, may be repeated for several times until the prediction converges.

3 Dataset

The financial news data we used in this paper are provided by [Ding et al.2014] which contains 106,521 articles from Reuters and 447,145 from Bloomberg. The news articles were published in the time period from October 2006 to December 2013. The historical stock security data are obtained from the Centre for Research in Security Prices (CRSP) database [Booth2012]. We only use the security data from 2006 to 2013 to match the time period of the financial news. Base on the samples’ publication dates, we split the dataset into three sets: a training set (all samples between 2006-10-01 and 2012-12-31), a validation set (2013-01-01 and 2013-06-15) and a test set (2013-06-16 to 2013-12-31). The training set containts 65,646 samples, the validation set 10,941 samples, and the test set 9,911 samples.

4 Experiments

4.1 Stock Prediction using DNNs

In the first set of experiments, we use DNNs to predict stock’s price movement based on a variety of features, namely producing a polar prediction of the price movement on next day (either price-up or price-down). Here we have trained a set of DNNs using different combinations of feature vectors and found that the DNN structure of 4 hidden layers (with 1024 hidden nodes in each layer) yields the best performance in the validation set. We use the historical price feature alone to create the baseline and various features derived from the financial news are added on top of it. We measure the final performance by calculating the error rate on the test set. As shown in Table 1, the features derived from financial news can significantly improve the prediction accuracy and we have obtained the best performance (an error rate of 43.13%) by using all the features discussed in Sections 2.2 and 2.3.

feature combination error rate
price 48.12%
price + BoK 46.02%
price + BoK + PS 43.96%
price + BOK + CT 45.86%
price + PS 45.00%
price + CT 46.10%
price + PS +CT 46.03%
price + BoK + PS + CT 43.13%
Table 1: Stock prediction error rates on the test set.

4.2 Predict Unseen Stocks via Correlation

Figure 2: Predict unseen stocks via correlation

Here we group all outputs from DNNs based on the dates of all samples on the test set. For each date, we create a vector based on the DNN prediction results for all observed stocks and zeros for all unseen stocks, as described in section 2.4. Then, the vector is propagated through the correlation graph to generate another set of stock movement prediction. We may apply a threshold on the propagated vector to prune all low-confidence predictions. The remaining ones may be used to predict some stocks unseen on the test set. The prediction of all unseen stocks is compared with the actual stock movement on next day. Experimental results are shown in Figure 2, where the left y-axis denotes the prediction accuracy and the right y-axis denotes the percentage of stocks predicated out of all 5000 per day under each pruning threshold. For example, using a large threshold (0.9), we may predict with an accuracy of 52.44% on 354 extra unseen stocks per day, in addition to predicting only 110 stocks per day on the test set.

5 Conclusion

In this paper, we have proposed a simple method to leverage financial news to predict stock movements based on the popular word embedding and deep learning techniques. Our experiments have shown that the financial news is very useful in stock prediction and the proposed methods can significantly improve the prediction accuracy on a standard financial data set.

Acknowledgments

This work was supported in part by an NSERC Engage grant from Canadian federal government.

References

  • [Adya and Collopy1991] Monica Adya and Fred Collopy. 1991. How effective are neural networks at forecasting and prediction? a review and evaluation. Journal of Forecasting, 17:481–495.
  • [Bar-Haim et al.2011] Roy Bar-Haim, Elad Dinur, Ronen Feldman, Moshe Fresko, and Guy Goldstein. 2011. Identifying and following expert investors in stock microblogs. In Proceedings of the 2011 Conference on Empirical Methods in Natural Language Processing, pages 1310–1319, Edinburgh, Scotland, UK., July. Association for Computational Linguistics.
  • [Booth2012] Chicago Booth. 2012. CRSP Data Description Guide for the CRSP US Stock Database and CRSP US Indices Database. Center for Research in Security Prices, The University of Chicago Graduate School of Business (https://wrds-web.wharton.upenn.edu/wrds/index.cfm).
  • [Chan et al.2000] Man-Chung Chan, Chi-Cheong Wong, and Chi-Chung Lam. 2000.

    Financial time series forecasting by neural network using conjugate gradient learning algorithm and multiple linear regression weight initialization.

    Computing in Economics and Finance, 61.
  • [Ding et al.2014] Xiao Ding, Yue Zhang, Ting Liu, and Junwen Duan. 2014. Using structured events to predict stock price movement: An empirical investigation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1415–1425, Doha, Qatar, October. Association for Computational Linguistics.
  • [Kaastra and Boyd1991] Iebeling Kaastra and Milton Boyd. 1991. Designing a neural network for forecasting financial and economic time series. Neurocomputing, 10:215–236.
  • [Marneffe et al.2006] Marie-Catherine Marneffe, Bill MacCartney, and Christopher D. Manning. 2006. Generating typed dependency parses from phrase structure parses. In Proceedings LREC.
  • [Mikolov et al.2013a] Tomas Mikolov, Kai Chen, Greg Corrado, and Jeffrey Dean. 2013a.

    Efficient estimation of word representations in vector space.

    In Proceedings of Workshop at ICLR.
  • [Mikolov et al.2013b] Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013b. Distributed representations of words and phrases and their compositionality. In Proceedings of NIPS, pages 3111–3119.
  • [Si et al.2013] Jianfeng Si, Arjun Mukherjee, Bing Liu, Qing Li, Huayi Li, and Xiaotie Deng. 2013. Exploiting topic based twitter sentiment for stock prediction. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 2: Short Papers), pages 24–29, Sofia, Bulgaria, August. Association for Computational Linguistics.
  • [Si et al.2014] Jianfeng Si, Arjun Mukherjee, Bing Liu, Sinno Jialin Pan, Qing Li, and Huayi Li. 2014. Exploiting social relations and sentiment for stock prediction. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pages 1139–1145, Doha, Qatar, October. Association for Computational Linguistics.
  • [Skabar and Cloete2002] Andrew Skabar and Ian Cloete. 2002. Neural networks, financial trading and the efficient markets hypothesis. In Proc. the Twenty-Fifth Australasian Computer Science Conference (ACSC2002), Melbourne, Australia.
  • [Turney and Littman2003] Peter D. Turney and Michael L. Littman. 2003. Measuring praise and criticism: Inference of semantic orientation from association. ACM Trans. Inf. Syst., 21(4):315–346, October.
  • [Turney and Pantel2010] Peter D. Turney and Patrick Pantel. 2010. From frequency to meaning: Vector space models of semantics. J. Artif. Int. Res., 37(1):141–188, January.
  • [Xie et al.2013] Boyi Xie, Rebecca J. Passonneau, Leon Wu, and Germán G. Creamer. 2013. Semantic frames to predict stock price movement. In Proceedings of the 51st Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pages 873–883, Sofia, Bulgaria, August. Association for Computational Linguistics.
  • [Zhu et al.2008] Xiaotian Zhu, Hong Wang, Li Xu, and Huaizu Li. 2008. Predicting stock index increments by neural networks: The role of trading volume under different horizons. Expert Systems with Applications, 34:3043–3054.