Variational Meta Reinforcement Learning for Social Robotics

06/07/2022
by   Anand Ballou, et al.
0

With the increasing presence of robots in our every-day environments, improving their social skills is of utmost importance. Nonetheless, social robotics still faces many challenges. One bottleneck is that robotic behaviors need to be often adapted as social norms depend strongly on the environment. For example, a robot should navigate more carefully around patients in a hospital compared to workers in an office. In this work, we investigate meta-reinforcement learning (meta-RL) as a potential solution. Here, robot behaviors are learned via reinforcement learning where a reward function needs to be chosen so that the robot learns an appropriate behavior for a given environment. We propose to use a variational meta-RL procedure that quickly adapts the robots' behavior to new reward functions. As a result, given a new environment different reward functions can be quickly evaluated and an appropriate one selected. The procedure learns a vectorized representation for reward functions and a meta-policy that can be conditioned on such a representation. Given observations from a new reward function, the procedure identifies its representation and conditions the meta-policy to it. While investigating the procedures' capabilities, we realized that it suffers from posterior collapse where only a subset of the dimensions in the representation encode useful information resulting in a reduced performance. Our second contribution, a radial basis function (RBF) layer, partially mitigates this negative effect. The RBF layer lifts the representation to a higher dimensional space, which is more easily exploitable for the meta-policy. We demonstrate the interest of the RBF layer and the usage of meta-RL for social robotics on four robotic simulation tasks.

READ FULL TEXT
research
02/01/2022

A General, Evolution-Inspired Reward Function for Social Robotics

The field of social robotics will likely need to depart from a paradigm ...
research
07/29/2023

PIMbot: Policy and Incentive Manipulation for Multi-Robot Reinforcement Learning in Social Dilemmas

Recent research has demonstrated the potential of reinforcement learning...
research
02/11/2020

Hyper-Meta Reinforcement Learning with Sparse Reward

Despite their success, existing meta reinforcement learning methods stil...
research
10/26/2020

MELD: Meta-Reinforcement Learning from Images via Latent State Models

Meta-reinforcement learning algorithms can enable autonomous agents, suc...
research
03/14/2023

Learning to Adapt the Parameters of Behavior Trees and Motion Generators to Task Variations

The ability to learn new tasks and quickly adapt to different variations...
research
10/17/2017

Map-based Multi-Policy Reinforcement Learning: Enhancing Adaptability of Robots by Deep Reinforcement Learning

In order for robots to perform mission-critical tasks, it is essential t...
research
10/16/2018

Social Behavior Learning with Realistic Reward Shaping

Deep reinforcement learning has been widely applied in the field of robo...

Please sign up or login with your details

Forgot password? Click here to reset