Situational Awareness by Risk-Conscious Skills

10/10/2016
by   Daniel J. Mankowitz, et al.
0

Hierarchical Reinforcement Learning has been previously shown to speed up the convergence rate of RL planning algorithms as well as mitigate feature-based model misspecification (Mankowitz et. al. 2016a,b, Bacon 2015). To do so, it utilizes hierarchical abstractions, also known as skills -- a type of temporally extended action (Sutton et. al. 1999) to plan at a higher level, abstracting away from the lower-level details. We incorporate risk sensitivity, also referred to as Situational Awareness (SA), into hierarchical RL for the first time by defining and learning risk aware skills in a Probabilistic Goal Semi-Markov Decision Process (PG-SMDP). This is achieved using our novel Situational Awareness by Risk-Conscious Skills (SARiCoS) algorithm which comes with a theoretical convergence guarantee. We show in a RoboCup soccer domain that the learned risk aware skills exhibit complex human behaviors such as `time-wasting' in a soccer game. In addition, the learned risk aware skills are able to mitigate reward-based model misspecification.

READ FULL TEXT
research
11/20/2017

Situationally Aware Options

Hierarchical abstractions, also known as options -- a type of temporally...
research
07/11/2022

Learning Temporally Extended Skills in Continuous Domains as Symbolic Actions for Planning

Problems which require both long-horizon planning and continuous control...
research
06/11/2015

Bootstrapping Skills

The monolithic approach to policy representation in Markov Decision Proc...
research
05/11/2018

Stochastic Approximation for Risk-aware Markov Decision Processes

In this paper, we develop a stochastic approximation type algorithm to s...
research
06/07/2022

Meta-Learning Transferable Parameterized Skills

We propose a novel parameterized skill-learning algorithm that aims to l...
research
02/21/2023

Conditioning Hierarchical Reinforcement Learning on Flexible Constraints

Safety in goal directed Reinforcement Learning (RL) settings has typical...
research
09/06/2022

Risk Aware Belief-dependent Constrained POMDP Planning

Risk awareness is fundamental to an online operating agent. However, it ...

Please sign up or login with your details

Forgot password? Click here to reset