AI Chat AI Image Generator AI Video Text to Speech

Plan-Based Relaxed Reward Shaping for Goal-Directed Tasks

07/14/2021

∙

by Ingmar Schubert, et al.

∙

∙

In high-dimensional state spaces, the usefulness of Reinforcement Learning (RL) is limited by the problem of exploration. This issue has been addressed using potential-based reward shaping (PB-RS) previously. In the present work, we introduce Final-Volume-Preserving Reward Shaping (FV-RS). FV-RS relaxes the strict optimality guarantees of PB-RS to a guarantee of preserved long-term behavior. Being less restrictive, FV-RS allows for reward shaping functions that are even better suited for improving the sample efficiency of RL algorithms. In particular, we consider settings in which the agent has access to an approximate plan. Here, we use examples of simulated robotic manipulation tasks to demonstrate that plan-based FV-RS can indeed significantly improve the sample efficiency of RL over plan-based PB-RS.

Ingmar Schubert
5 publications
Ozgur S. Oguz
11 publications
Marc Toussaint
79 publications

page 7

page 15

research

∙ 03/16/2021

Learning to Shape Rewards using a Game of Switching Controls

Reward shaping (RS) is a powerful method in reinforcement learning (RL) ...

4 David Mguni, et al. ∙

research

∙ 09/07/2020

Predicting Requests in Large-Scale Online P2P Ridesharing

Peer-to-peer ridesharing (P2P-RS) enables people to arrange one-time rid...

0 Filippo Bistaffa, et al. ∙

research

∙ 10/15/2021

Value Penalized Q-Learning for Recommender Systems

Scaling reinforcement learning (RL) to recommender systems (RS) is promi...

0 Chengqian Gao, et al. ∙

research

∙ 11/03/2019

Online Robustness Training for Deep Reinforcement Learning

In deep reinforcement learning (RL), adversarial attacks can trick an ag...

0 Marc Fischer, et al. ∙

research

∙ 02/13/2023

On Modeling Long-Term User Engagement from Stochastic Feedback

An ultimate goal of recommender systems (RS) is to improve user engageme...

0 Guoxi Zhang, et al. ∙

research

∙ 10/30/2022

Reward Shaping Using Convolutional Neural Network

In this paper, we propose Value Iteration Network for Reward Shaping (VI...

0 Hani Sami, et al. ∙

research

∙ 05/30/2022

Stock Trading Optimization through Model-based Reinforcement Learning with Resistance Support Relative Strength

Reinforcement learning (RL) is gaining attention by more and more resear...

0 Huifang Huang, et al. ∙