Reinforced Deep Markov Models With Applications in Automatic Trading

11/09/2020
by   Tadeu A. Ferreira, et al.
21

Inspired by the developments in deep generative models, we propose a model-based RL approach, coined Reinforced Deep Markov Model (RDMM), designed to integrate desirable properties of a reinforcement learning algorithm acting as an automatic trading system. The network architecture allows for the possibility that market dynamics are partially visible and are potentially modified by the agent's actions. The RDMM filters incomplete and noisy data, to create better-behaved input data for RL planning. The policy search optimisation also properly accounts for state uncertainty. Due to the complexity of the RKDF model architecture, we performed ablation studies to understand the contributions of individual components of the approach better. To test the financial performance of the RDMM we implement policies using variants of Q-Learning, DynaQ-ARIMA and DynaQ-LSTM algorithms. The experiments show that the RDMM is data-efficient and provides financial gains compared to the benchmarks in the optimal execution problem. The performance improvement becomes more pronounced when price dynamics are more complex, and this has been demonstrated using real data sets from the limit order book of Facebook, Intel, Vodafone and Microsoft.

READ FULL TEXT

page 13

page 14

page 19

page 20

page 21

06/21/2022

Imitate then Transcend: Multi-Agent Optimal Execution with Dual-Window Denoise PPO

A novel framework for solving the optimal execution and placement proble...
07/08/2018

Financial Trading as a Game: A Deep Reinforcement Learning Approach

An automatic program that generates constant profit from the financial m...
03/31/2019

Cooperative Multi-Agent Reinforcement Learning Framework for Scalping Trading

We explore deep Reinforcement Learning(RL) algorithms for scalping tradi...
09/16/2022

Model-based gym environments for limit order book trading

Within the mathematical finance literature there is a rich catalogue of ...
08/28/2019

Reinforcement Learning: Prediction, Control and Value Function Approximation

With the increasing power of computers and the rapid development of self...
07/08/2019

An intelligent financial portfolio trading strategy using deep Q-learning

A goal of financial portfolio trading is maximizing the trader's utility...
08/11/2022

A Modular Framework for Reinforcement Learning Optimal Execution

In this article, we develop a modular framework for the application of R...