Handling Sparse Rewards in Reinforcement Learning Using Model Predictive Control

10/04/2022
by   Murad Dawood, et al.
0

Reinforcement learning (RL) has recently proven great success in various domains. Yet, the design of the reward function requires detailed domain expertise and tedious fine-tuning to ensure that agents are able to learn the desired behaviour. Using a sparse reward conveniently mitigates these challenges. However, the sparse reward represents a challenge on its own, often resulting in unsuccessful training of the agent. In this paper, we therefore address the sparse reward problem in RL. Our goal is to find an effective alternative to reward shaping, without using costly human demonstrations, that would also be applicable to a wide range of domains. Hence, we propose to use model predictive control (MPC) as an experience source for training RL agents in sparse reward environments. Without the need for reward shaping, we successfully apply our approach in the field of mobile robot navigation both in simulation and real-world experiments with a Kuboki Turtlebot 2. We furthermore demonstrate great improvement over pure RL algorithms in terms of success rate as well as number of collisions and timeouts. Our experiments show that MPC as an experience source improves the agent's learning process for a given task in the case of sparse rewards.

READ FULL TEXT

page 1

page 2

page 4

page 5

research
12/02/2021

Hindsight Task Relabelling: Experience Replay for Sparse Reward Meta-RL

Meta-reinforcement learning (meta-RL) has proven to be a successful fram...
research
04/14/2021

GridToPix: Training Embodied Agents with Minimal Supervision

While deep reinforcement learning (RL) promises freedom from hand-labele...
research
10/01/2019

Accelerated Robot Learning via Human Brain Signals

In reinforcement learning (RL), sparse rewards are a natural way to spec...
research
06/04/2021

Online reinforcement learning with sparse rewards through an active inference capsule

Intelligent agents must pursue their goals in complex environments with ...
research
06/17/2023

The RL Perceptron: Generalisation Dynamics of Policy Learning in High Dimensions

Reinforcement learning (RL) algorithms have proven transformative in a r...
research
08/26/2022

Play with Emotion: Affect-Driven Reinforcement Learning

This paper introduces a paradigm shift by viewing the task of affect mod...
research
05/26/2023

A Reminder of its Brittleness: Language Reward Shaping May Hinder Learning for Instruction Following Agents

Teaching agents to follow complex written instructions has been an impor...

Please sign up or login with your details

Forgot password? Click here to reset