Dependability Analysis of Deep Reinforcement Learning based Robotics and Autonomous Systems

by   Yi Dong, et al.

While Deep Reinforcement Learning (DRL) provides transformational capabilities to the control of Robotics and Autonomous Systems (RAS), the black-box nature of DRL and uncertain deployment-environments of RAS pose new challenges on its dependability. Although there are many existing works imposing constraints on the DRL policy to ensure a successful completion of the mission, it is far from adequate in terms of assessing the DRL-driven RAS in a holistic way considering all dependability properties. In this paper, we formally define a set of dependability properties in temporal logic and construct a Discrete-Time Markov Chain (DTMC) to model the dynamics of risk/failures of a DRL-driven RAS interacting with the stochastic environment. We then do Probabilistic Model Checking based on the designed DTMC to verify those properties. Our experimental results show that the proposed method is effective as a holistic assessment framework, while uncovers conflicts between the properties that may need trade-offs in the training. Moreover, we find the standard DRL training cannot improve dependability properties, thus requiring bespoke optimisation objectives concerning them. Finally, our method offers a novel dependability analysis to the Sim-to-Real challenge of DRL.


page 1

page 5


Learning on Abstract Domains: A New Approach for Verifiable Guarantee in Reinforcement Learning

Formally verifying Deep Reinforcement Learning (DRL) systems is a challe...

R^3: On-device Real-Time Deep Reinforcement Learning for Autonomous Robotics

Autonomous robotic systems, like autonomous vehicles and robotic search ...

Development and Validation of an AI-Driven Model for the La Rance Tidal Barrage: A Generalisable Case Study

In this work, an AI-Driven (autonomous) model representation of the La R...

Surrogate Empowered Sim2Real Transfer of Deep Reinforcement Learning for ORC Superheat Control

The Organic Rankine Cycle (ORC) is widely used in industrial waste heat ...

First Study on Data Readiness Level

We introduce the idea of Data Readiness Level (DRL) to measure the relat...

Deploying Deep Reinforcement Learning Systems: A Taxonomy of Challenges

Deep reinforcement learning (DRL), leveraging Deep Learning (DL) in rein...

TempoRL: laser pulse temporal shape optimization with Deep Reinforcement Learning

High Power Laser's (HPL) optimal performance is essential for the succes...

Please sign up or login with your details

Forgot password? Click here to reset