Financial Trading as a Game: A Deep Reinforcement Learning Approach

by   Chien Yi Huang, et al.

An automatic program that generates constant profit from the financial market is lucrative for every market practitioner. Recent advance in deep reinforcement learning provides a framework toward end-to-end training of such trading agent. In this paper, we propose an Markov Decision Process (MDP) model suitable for the financial trading task and solve it with the state-of-the-art deep recurrent Q-network (DRQN) algorithm. We propose several modifications to the existing learning algorithm to make it more suitable under the financial trading setting, namely 1. We employ a substantially small replay memory (only a few hundreds in size) compared to ones used in modern deep reinforcement learning algorithms (often millions in size.) 2. We develop an action augmentation technique to mitigate the need for random exploration by providing extra feedback signals for all actions to the agent. This enables us to use greedy policy over the course of learning and shows strong empirical performance compared to more commonly used epsilon-greedy exploration. However, this technique is specific to financial trading under a few market assumptions. 3. We sample a longer sequence for recurrent neural network training. A side product of this mechanism is that we can now train the agent for every T steps. This greatly reduces training time since the overall computation is down by a factor of T. We combine all of the above into a complete online learning algorithm and validate our approach on the spot foreign exchange market.



There are no comments yet.


page 1

page 2

page 3

page 4


Deep Reinforcement Learning in Financial Markets

In this paper we explore the usage of deep reinforcement learning algori...

An intelligent financial portfolio trading strategy using deep Q-learning

A goal of financial portfolio trading is maximizing the trader's utility...

A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem

Financial portfolio management is the process of constant redistribution...

Reinforced Deep Markov Models With Applications in Automatic Trading

Inspired by the developments in deep generative models, we propose a mod...

Embracing advanced AI/ML to help investors achieve success: Vanguard Reinforcement Learning for Financial Goal Planning

In the world of advice and financial planning, there is seldom one right...

The Recurrent Reinforcement Learning Crypto Agent

We demonstrate an application of online transfer learning as a digital a...

Adaptive Market Neutral Strategy Amid COVID-19 Regime-shifting Times, a Reinforcement Learning Approach

Pairs trading is the foundation of market neutral strategy, which is one...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.