Learning on Abstract Domains: A New Approach for Verifiable Guarantee in Reinforcement Learning

06/13/2021
by   Peng Jin, et al.
0

Formally verifying Deep Reinforcement Learning (DRL) systems is a challenging task due to the dynamic continuity of system behaviors and the black-box feature of embedded neural networks. In this paper, we propose a novel abstraction-based approach to train DRL systems on finite abstract domains instead of concrete system states. It yields neural networks whose input states are finite, making hosting DRL systems directly verifiable using model checking techniques. Our approach is orthogonal to existing DRL algorithms and off-the-shelf model checkers. We implement a resulting prototype training and verification framework and conduct extensive experiments on the state-of-the-art benchmark. The results show that the systems trained in our approach can be verified more efficiently while they retain comparable performance against those that are trained without abstraction.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
11/21/2022

BBReach: Tight and Scalable Black-Box Reachability Analysis of Deep Reinforcement Learning Systems

Reachability analysis is a promising technique to automatically prove or...
research
09/14/2021

Dependability Analysis of Deep Reinforcement Learning based Robotics and Autonomous Systems

While Deep Reinforcement Learning (DRL) provides transformational capabi...
research
01/10/2022

Verified Probabilistic Policies for Deep Reinforcement Learning

Deep reinforcement learning is an increasingly popular technique for syn...
research
03/15/2022

Linear-Time Verification of Data-Aware Dynamic Systems with Arithmetic

Combined modeling and verification of dynamic systems and the data they ...
research
05/25/2021

Towards Scalable Verification of RL-Driven Systems

Deep neural networks (DNNs) have gained significant popularity in recent...
research
01/27/2023

Neural Episodic Control with State Abstraction

Existing Deep Reinforcement Learning (DRL) algorithms suffer from sample...
research
09/18/2016

Towards Deep Symbolic Reinforcement Learning

Deep reinforcement learning (DRL) brings the power of deep neural networ...

Please sign up or login with your details

Forgot password? Click here to reset