DeepAI AI Chat
Log In Sign Up

Real-Time Model Calibration with Deep Reinforcement Learning

by   Manuel Arias Chao, et al.
Luleå University of Technology
ETH Zurich

The dynamic, real-time, and accurate inference of model parameters from empirical data is of great importance in many scientific and engineering disciplines that use computational models (such as a digital twin) for the analysis and prediction of complex physical processes. However, fast and accurate inference for processes with large and high dimensional datasets cannot easily be achieved with state-of-the-art methods under noisy real-world conditions. The primary reason is that the inference of model parameters with traditional techniques based on optimisation or sampling often suffers from computational and statistical challenges, resulting in a trade-off between accuracy and deployment time. In this paper, we propose a novel framework for inference of model parameters based on reinforcement learning. The contribution of the paper is twofold: 1) We reformulate the inference problem as a tracking problem with the objective of learning a policy that forces the response of the physics-based model to follow the observations; 2) We propose the constrained Lyapunov-based actor-critic (CLAC) algorithm to enable the robust and accurate inference of physics-based model parameters in real time under noisy real-world conditions. The proposed methodology is demonstrated and evaluated on two model-based diagnostics test cases utilizing two different physics-based models of turbofan engines. The performance of the methodology is compared to that of two alternative approaches: a state update method (unscented Kalman filter) and a supervised end-to-end mapping with deep neural networks. The experimental results demonstrate that the proposed methodology outperforms all other tested methods in terms of speed and robustness, with high inference accuracy.


page 3

page 7

page 12


Battery Model Calibration with Deep Reinforcement Learning

Lithium-Ion (Li-I) batteries have recently become pervasive and are used...

Online Multi-agent Reinforcement Learning for Decentralized Inverter-based Volt-VAR Control

The distributed Volt/Var control (VVC) methods have been widely studied ...

PhysiNet: A Combination of Physics-based Model and Neural Network Model for Digital Twins

As the real-time digital counterpart of a physical system or process, di...

Real-time Blind Deblurring Based on Lightweight Deep-Wiener-Network

In this paper, we address the problem of blind deblurring with high effi...

Continuous calibration of a digital twin: comparison of particle filter and Bayesian calibration approaches

Assimilation of continuously streamed monitored data is an essential com...

Physics-Informed Neural Networks for Non-linear System Identification applied to Power System Dynamics

Varying power-infeed from converter-based generation units introduces gr...

Dynamic probabilistic predictable feature analysis for high dimensional temporal monitoring

Dynamic statistical process monitoring methods have been widely studied ...