Reinforced Deep Markov Models With Applications in Automatic Trading

by   Tadeu A. Ferreira, et al.

Inspired by the developments in deep generative models, we propose a model-based RL approach, coined Reinforced Deep Markov Model (RDMM), designed to integrate desirable properties of a reinforcement learning algorithm acting as an automatic trading system. The network architecture allows for the possibility that market dynamics are partially visible and are potentially modified by the agent's actions. The RDMM filters incomplete and noisy data, to create better-behaved input data for RL planning. The policy search optimisation also properly accounts for state uncertainty. Due to the complexity of the RKDF model architecture, we performed ablation studies to understand the contributions of individual components of the approach better. To test the financial performance of the RDMM we implement policies using variants of Q-Learning, DynaQ-ARIMA and DynaQ-LSTM algorithms. The experiments show that the RDMM is data-efficient and provides financial gains compared to the benchmarks in the optimal execution problem. The performance improvement becomes more pronounced when price dynamics are more complex, and this has been demonstrated using real data sets from the limit order book of Facebook, Intel, Vodafone and Microsoft.


page 13

page 14

page 19

page 20

page 21


Imitate then Transcend: Multi-Agent Optimal Execution with Dual-Window Denoise PPO

A novel framework for solving the optimal execution and placement proble...

Financial Trading as a Game: A Deep Reinforcement Learning Approach

An automatic program that generates constant profit from the financial m...

Cooperative Multi-Agent Reinforcement Learning Framework for Scalping Trading

We explore deep Reinforcement Learning(RL) algorithms for scalping tradi...

Model-based gym environments for limit order book trading

Within the mathematical finance literature there is a rich catalogue of ...

Reinforcement Learning: Prediction, Control and Value Function Approximation

With the increasing power of computers and the rapid development of self...

An intelligent financial portfolio trading strategy using deep Q-learning

A goal of financial portfolio trading is maximizing the trader's utility...

A Modular Framework for Reinforcement Learning Optimal Execution

In this article, we develop a modular framework for the application of R...