DeepAI AI Chat
Log In Sign Up

Efficient State Representation Learning for Dynamic Robotic Scenarios

by   Zhaorun Chen, et al.
Shanghai Jiao Tong University

While the rapid progress of deep learning fuels end-to-end reinforcement learning (RL), direct application, especially in high-dimensional space like robotic scenarios still suffers from high sample efficiency. Therefore State Representation Learning (SRL) is proposed to specifically learn to encode task-relevant features from complex sensory data into low-dimensional states. However, the pervasive implementation of SRL is usually conducted by a decoupling strategy in which the observation-state mapping is learned separately, which is prone to over-fit. To handle such problem, we present a new algorithm called Policy Optimization via Abstract Representation which integrates SRL into the original RL scale. Firstly, We engage RL loss to assist in updating SRL model so that the states can evolve to meet the demand of reinforcement learning and maintain a good physical interpretation. Secondly, we introduce a dynamic parameter adjustment mechanism so that both models can efficiently adapt to each other. Thirdly, we introduce a new prior called domain resemblance to leverage expert demonstration to train the SRL model. Finally, we provide a real-time access by state graph to monitor the course of learning. Results show that our algorithm outperforms the PPO baselines and decoupling strategies in terms of sample efficiency and final rewards. Thus our model can efficiently deal with tasks in high dimensions and facilitate training real-life robots directly from scratch.


page 1

page 5


Low-Dimensional State and Action Representation Learning with MDP Homomorphism Metrics

Deep Reinforcement Learning has shown its ability in solving complicated...

Improving Sample Efficiency in Model-Free Reinforcement Learning from Images

Training an agent to solve control tasks directly from high-dimensional ...

MELD: Meta-Reinforcement Learning from Images via Latent State Models

Meta-reinforcement learning algorithms can enable autonomous agents, suc...

Unsupervised state representation learning with robotic priors: a robustness benchmark

Our understanding of the world depends highly on our capacity to produce...

Low Dimensional State Representation Learning with Robotics Priors in Continuous Action Spaces

Autonomous robots require high degrees of cognitive and motoric intellig...