DeepAI AI Chat
Log In Sign Up

G-Learner and GIRL: Goal Based Wealth Management with Reinforcement Learning

by   Matthew Dixon, et al.

We present a reinforcement learning approach to goal based wealth management problems such as optimization of retirement plans or target dated funds. In such problems, an investor seeks to achieve a financial goal by making periodic investments in the portfolio while being employed, and periodically draws from the account when in retirement, in addition to the ability to re-balance the portfolio by selling and buying different assets (e.g. stocks). Instead of relying on a utility of consumption, we present G-Learner: a reinforcement learning algorithm that operates with explicitly defined one-step rewards, does not assume a data generation process, and is suitable for noisy data. Our approach is based on G-learning - a probabilistic extension of the Q-learning method of reinforcement learning. In this paper, we demonstrate how G-learning, when applied to a quadratic reward and Gaussian reference policy, gives an entropy-regulated Linear Quadratic Regulator (LQR). This critical insight provides a novel and computationally tractable tool for wealth management tasks which scales to high dimensional portfolios. In addition to the solution of the direct problem of G-learning, we also present a new algorithm, GIRL, that extends our goal-based G-learning approach to the setting of Inverse Reinforcement Learning (IRL) where rewards collected by the agent are not observed, and should instead be inferred. We demonstrate that GIRL can successfully learn the reward parameters of a G-Learner agent and thus imitate its behavior. Finally, we discuss potential applications of the G-Learner and GIRL algorithms for wealth management and robo-advising.


page 1

page 2

page 3

page 4


Inverse Reinforcement Learning from a Gradient-based Learner

Inverse Reinforcement Learning addresses the problem of inferring an exp...

Tournament selection in zeroth-level classifier systems based on average reward reinforcement learning

As a genetics-based machine learning technique, zeroth-level classifier ...

Regularized Inverse Reinforcement Learning

Inverse Reinforcement Learning (IRL) aims to facilitate a learner's abil...

Ecological Reinforcement Learning

Much of the current work on reinforcement learning studies episodic sett...

Policy Poisoning in Batch Reinforcement Learning and Control

We study a security threat to batch reinforcement learning and control w...

Adapting Behaviour via Intrinsic Reward: A Survey and Empirical Study

Learning about many things can provide numerous benefits to a reinforcem...

A Model-Based Reinforcement Learning Approach for a Rare Disease Diagnostic Task

In this work, we present our various contributions to the objective of b...