DeepAI AI Chat
Log In Sign Up

Physics-informed Dyna-Style Model-Based Deep Reinforcement Learning for Dynamic Control

by   Xin-Yang Liu, et al.

Model-based reinforcement learning (MBRL) is believed to have much higher sample efficiency compared to model-free algorithms by learning a predictive model of the environment. However, the performance of MBRL highly relies on the quality of the learned model, which is usually built in a black-box manner and may have poor predictive accuracy outside of the data distribution. The deficiencies of the learned model may prevent the policy from being fully optimized. Although some uncertainty analysis-based remedies have been proposed to alleviate this issue, model bias still poses a great challenge for MBRL. In this work, we propose to leverage the prior knowledge of underlying physics of the environment, where the governing laws are (partially) known. In particular, we developed a physics-informed MBRL framework, where governing equations and physical constraints are utilized to inform the model learning and policy search. By incorporating the prior information of the environment, the quality of the learned model can be notably improved, while the required interactions with the environment are significantly reduced, leading to better sample efficiency and learning performance. The effectiveness and merit have been demonstrated over a handful of classic control problems, where the environments are governed by canonical ordinary/partial differential equations.


Physics-Informed Model-Based Reinforcement Learning

We apply reinforcement learning (RL) to robotics. One of the drawbacks o...

Model-based Reinforcement Learning with a Hamiltonian Canonical ODE Network

Model-based reinforcement learning usually suffers from a high sample co...

Differentiable Physics Models for Real-world Offline Model-based Reinforcement Learning

A limitation of model-based reinforcement learning (MBRL) is the exploit...

Adaptive Rollout Length for Model-Based RL Using Model-Free Deep RL

Model-based reinforcement learning promises to learn an optimal policy f...

Sample-efficient Model-based Reinforcement Learning for Quantum Control

We propose a model-based reinforcement learning (RL) approach for noisy ...

Physically Consistent Neural Networks for building thermal modeling: theory and analysis

Due to their high energy intensity, buildings play a major role in the c...