Policy Gradient for Coherent Risk Measures

02/13/2015
by   Aviv Tamar, et al.
0

Several authors have recently developed risk-sensitive policy gradient methods that augment the standard expected cost minimization problem with a measure of variability in cost. These studies have focused on specific risk-measures, such as the variance or conditional value at risk (CVaR). In this work, we extend the policy gradient method to the whole class of coherent risk measures, which is widely accepted in finance and operations research, among other fields. We consider both static and time-consistent dynamic risk measures. For static risk measures, our approach is in the spirit of policy gradient algorithms and combines a standard sampling approach with convex programming. For dynamic risk measures, our approach is actor-critic style and involves explicit approximation of value function. Most importantly, our contribution presents a unified approach to risk-sensitive reinforcement learning that generalizes and extends previous results.

READ FULL TEXT
research
01/26/2023

On the Global Convergence of Risk-Averse Policy Gradient Methods with Dynamic Time-Consistent Risk Measures

Risk-sensitive reinforcement learning (RL) has become a popular tool to ...
research
12/26/2021

Reinforcement Learning with Dynamic Convex Risk Measures

We develop an approach for solving time-consistent risk-sensitive stocha...
research
08/22/2019

Practical Risk Measures in Reinforcement Learning

Practical application of Reinforcement Learning (RL) often involves risk...
research
03/04/2021

On the Convergence and Optimality of Policy Gradient for Markov Coherent Risk

In order to model risk aversion in reinforcement learning, an emerging l...
research
06/29/2022

Conditionally Elicitable Dynamic Risk Measures for Deep Reinforcement Learning

We propose a novel framework to solve risk-sensitive reinforcement learn...
research
09/09/2021

Deep Reinforcement Learning for Equal Risk Pricing and Hedging under Dynamic Expectile Risk Measures

Recently equal risk pricing, a framework for fair derivative pricing, wa...
research
08/14/2018

A note on representation of BSDE-based dynamic risk measures and dynamic capital allocations

In this paper, we provide a representation theorem for dynamic capital a...

Please sign up or login with your details

Forgot password? Click here to reset