Provably Safe Model-Based Meta Reinforcement Learning: An Abstraction-Based Approach

by   Xiaowu Sun, et al.

While conventional reinforcement learning focuses on designing agents that can perform one task, meta-learning aims, instead, to solve the problem of designing agents that can generalize to different tasks (e.g., environments, obstacles, and goals) that were not considered during the design or the training of these agents. In this spirit, in this paper, we consider the problem of training a provably safe Neural Network (NN) controller for uncertain nonlinear dynamical systems that can generalize to new tasks that were not present in the training data while preserving strong safety guarantees. Our approach is to learn a set of NN controllers during the training phase. When the task becomes available at runtime, our framework will carefully select a subset of these NN controllers and compose them to form the final NN controller. Critical to our approach is the ability to compute a finite-state abstraction of the nonlinear dynamical system. This abstract model captures the behavior of the closed-loop system under all possible NN weights, and is used to train the NNs and compose them when the task becomes available. We provide theoretical guarantees that govern the correctness of the resulting NN. We evaluated our approach on the problem of controlling a wheeled robot in cluttered environments that were not present in the training data.


Provably Correct Training of Neural Network Controllers Using Reachability Analysis

In this paper, we consider the problem of training neural network (NN) c...

ShieldNN: A Provably Safe NN Filter for Unsafe NN Controllers

In this paper, we consider the problem of creating a safe-by-design Rect...

Neurosymbolic Motion and Task Planning for Linear Temporal Logic Tasks

This paper presents a neurosymbolic framework to solve motion planning p...

Safe-by-Repair: A Convex Optimization Approach for Repairing Unsafe Two-Level Lattice Neural Network Controllers

In this paper, we consider the problem of repairing a data-trained Recti...

A Bioinspired Synthetic Nervous System Controller for Pick-and-Place Manipulation

The Synthetic Nervous System (SNS) is a biologically inspired neural net...

Model-Assisted Probabilistic Safe Adaptive Control With Meta-Bayesian Learning

Breaking safety constraints in control systems can lead to potential ris...

Failing with Grace: Learning Neural Network Controllers that are Boundedly Unsafe

In this work, we consider the problem of learning a feed-forward neural ...

Please sign up or login with your details

Forgot password? Click here to reset