Financial Trading as a Game: A Deep Reinforcement Learning Approach

07/08/2018
by   Chien Yi Huang, et al.
0

An automatic program that generates constant profit from the financial market is lucrative for every market practitioner. Recent advance in deep reinforcement learning provides a framework toward end-to-end training of such trading agent. In this paper, we propose an Markov Decision Process (MDP) model suitable for the financial trading task and solve it with the state-of-the-art deep recurrent Q-network (DRQN) algorithm. We propose several modifications to the existing learning algorithm to make it more suitable under the financial trading setting, namely 1. We employ a substantially small replay memory (only a few hundreds in size) compared to ones used in modern deep reinforcement learning algorithms (often millions in size.) 2. We develop an action augmentation technique to mitigate the need for random exploration by providing extra feedback signals for all actions to the agent. This enables us to use greedy policy over the course of learning and shows strong empirical performance compared to more commonly used epsilon-greedy exploration. However, this technique is specific to financial trading under a few market assumptions. 3. We sample a longer sequence for recurrent neural network training. A side product of this mechanism is that we can now train the agent for every T steps. This greatly reduces training time since the overall computation is down by a factor of T. We combine all of the above into a complete online learning algorithm and validate our approach on the spot foreign exchange market.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

07/09/2019

Deep Reinforcement Learning in Financial Markets

In this paper we explore the usage of deep reinforcement learning algori...
07/08/2019

An intelligent financial portfolio trading strategy using deep Q-learning

A goal of financial portfolio trading is maximizing the trader's utility...
06/30/2017

A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem

Financial portfolio management is the process of constant redistribution...
11/09/2020

Reinforced Deep Markov Models With Applications in Automatic Trading

Inspired by the developments in deep generative models, we propose a mod...
10/18/2021

Embracing advanced AI/ML to help investors achieve success: Vanguard Reinforcement Learning for Financial Goal Planning

In the world of advice and financial planning, there is seldom one right...
01/12/2022

The Recurrent Reinforcement Learning Crypto Agent

We demonstrate an application of online transfer learning as a digital a...
08/13/2020

Adaptive Market Neutral Strategy Amid COVID-19 Regime-shifting Times, a Reinforcement Learning Approach

Pairs trading is the foundation of market neutral strategy, which is one...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.