Formal Methods with a Touch of Magic

05/25/2020
by   Parand Alizadeh Alamdari, et al.
0

Machine learning and formal methods have complimentary benefits and drawbacks. In this work, we address the controller-design problem with a combination of techniques from both fields. The use of black-box neural networks in deep reinforcement learning (deep RL) poses a challenge for such a combination. Instead of reasoning formally about the output of deep RL, which we call the wizard, we extract from it a decision-tree based model, which we refer to as the magic book. Using the extracted model as an intermediary, we are able to handle problems that are infeasible for either deep RL or formal methods by themselves. First, we suggest, for the first time, combining a magic book in a synthesis procedure. We synthesize a stand-alone correct-by-design controller that enjoys the favorable performance of RL. Second, we incorporate a magic book in a bounded model checking (BMC) procedure. BMC allows us to find numerous traces of the plant under the control of the wizard, which a user can use to increase the trustworthiness of the wizard and direct further training.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
07/20/2022

Deep Reinforcement Learning for Market Making Under a Hawkes Process-Based Limit Order Book Model

The stochastic control problem of optimal market making is among the cen...
research
03/24/2023

Autonomous Blimp Control via H-infinity Robust Deep Residual Reinforcement Learning

Due to their superior energy efficiency, blimps may replace quadcopters ...
research
11/17/2017

A Supervisory Control Algorithm Based on Property-Directed Reachability

We present an algorithm for synthesising a controller (supervisor) for a...
research
10/02/2018

CEM-RL: Combining evolutionary and gradient-based methods for policy search

Deep neuroevolution and deep reinforcement learning (deep RL) algorithms...
research
10/16/2022

Entropy Regularized Reinforcement Learning with Cascading Networks

Deep Reinforcement Learning (Deep RL) has had incredible achievements on...
research
09/16/2022

Model-based gym environments for limit order book trading

Within the mathematical finance literature there is a rich catalogue of ...

Please sign up or login with your details

Forgot password? Click here to reset