DeepAI AI Chat
Log In Sign Up

Risk Aware and Multi-Objective Decision Making with Distributional Monte Carlo Tree Search

02/01/2021
by   Conor F. Hayes, et al.
14

In many risk-aware and multi-objective reinforcement learning settings, the utility of the user is derived from the single execution of a policy. In these settings, making decisions based on the average future returns is not suitable. For example, in a medical setting a patient may only have one opportunity to treat their illness. When making a decision, just the expected return – known in reinforcement learning as the value – cannot account for the potential range of adverse or positive outcomes a decision may have. Our key insight is that we should use the distribution over expected future returns differently to represent the critical information that the agent requires at decision time. In this paper, we propose Distributional Monte Carlo Tree Search, an algorithm that learns a posterior distribution over the utility of the different possible returns attainable from individual policy executions, resulting in good policies for both risk-aware and multi-objective settings. Moreover, our algorithm outperforms the state-of-the-art in multi-objective reinforcement learning for the expected utility of the returns.

READ FULL TEXT
11/23/2022

Monte Carlo Tree Search Algorithms for Risk-Aware and Multi-Objective Reinforcement Learning

In many risk-aware and multi-objective reinforcement learning settings, ...
06/02/2021

Expected Scalarised Returns Dominance: A New Solution Concept for Multi-Objective Decision Making

In many real-world scenarios, the utility of a user is derived from the ...
09/06/2019

Multi-Objective Multi-Agent Decision Making: A Utility-based Analysis and Survey

The majority of multi-agent system (MAS) implementations aim to optimise...
07/01/2022

Multi-Objective Coordination Graphs for the Expected Scalarised Returns with Generative Flow Models

Many real-world problems contain multiple objectives and agents, where a...
04/20/2021

Backtesting Systemic Risk Forecasts using Multi-Objective Elicitability

Backtesting risk measure forecasts requires identifiability (for model c...
05/17/2019

Stochastically Dominant Distributional Reinforcement Learning

We describe a new approach for mitigating risk in the Reinforcement Lear...
03/25/2021

MCTSteg: A Monte Carlo Tree Search-based Reinforcement Learning Framework for Universal Non-additive Steganography

Recent research has shown that non-additive image steganographic framewo...