Hyper-Meta Reinforcement Learning with Sparse Reward

by   Yun Hua, et al.

Despite their success, existing meta reinforcement learning methods still have difficulty in learning a meta policy effectively for RL problems with sparse reward. To this end, we develop a novel meta reinforcement learning framework, Hyper-Meta RL (HMRL), for sparse reward RL problems. It consists of meta state embedding, meta reward shaping and meta policy learning modules: The cross-environment meta state embedding module constructs a common meta state space to adapt to different environments; The meta state based environment-specific meta reward shaping effectively extends the original sparse reward trajectory by cross-environmental knowledge complementarity; As a consequence, the meta policy then achieves better generalization and efficiency with the shaped meta reward. Experiments with sparse reward show the superiority of HMRL on both transferability and policy learning efficiency.


page 10

page 11

page 12

page 13


Exploration in Approximate Hyper-State Space for Meta Reinforcement Learning

Meta-learning is a powerful tool for learning policies that can adapt ef...

Hindsight Task Relabelling: Experience Replay for Sparse Reward Meta-RL

Meta-reinforcement learning (meta-RL) has proven to be a successful fram...

Learning Action Translator for Meta Reinforcement Learning on Sparse-Reward Tasks

Meta reinforcement learning (meta-RL) aims to learn a policy solving a s...

Variational Meta Reinforcement Learning for Social Robotics

With the increasing presence of robots in our every-day environments, im...

Importance Weighted Policy Learning and Adaption

The ability to exploit prior experience to solve novel problems rapidly ...

NavTuner: Learning a Scene-Sensitive Family of Navigation Policies

The advent of deep learning has inspired research into end-to-end learni...

Enhanced Meta Reinforcement Learning using Demonstrations in Sparse Reward Environments

Meta reinforcement learning (Meta-RL) is an approach wherein the experie...