The Fundamental Limits of Interval Arithmetic for Neural Networks

by   Matthew Mirman, et al.

Interval analysis (or interval bound propagation, IBP) is a popular technique for verifying and training provably robust deep neural networks, a fundamental challenge in the area of reliable machine learning. However, despite substantial efforts, progress on addressing this key challenge has stagnated, calling into question whether interval arithmetic is a viable path forward. In this paper we present two fundamental results on the limitations of interval arithmetic for analyzing neural networks. Our main impossibility theorem states that for any neural network classifying just three points, there is a valid specification over these points that interval analysis can not prove. Further, in the restricted case of one-hidden-layer neural networks we show a stronger impossibility result: given any radius α < 1, there is a set of O(α^-1) points with robust radius α, separated by distance 2, that no one-hidden-layer network can be proven to classify robustly via interval analysis.



There are no comments yet.


page 1

page 2

page 3

page 4


On the Convergence of Certified Robust Training with Interval Bound Propagation

Interval Bound Propagation (IBP) is so far the base of state-of-the-art ...

Development of linear functional arithmetic and its applications to the solution of interval linear systems with constraints

The work is devoted to the construction of a new interval arithmetic whi...

Using Propagation for Solving Complex Arithmetic Constraints

Solving a system of nonlinear inequalities is an important problem for w...

An Interval Arithmetic for Robust Error Estimation

Interval arithmetic is a simple way to compute a mathematical expression...

Fast and Stable Interval Bounds Propagation for Training Verifiably Robust Models

We present an efficient technique, which allows to train classification ...

Universal Approximation with Certified Networks

Training neural networks to be certifiably robust is a powerful defense ...

POLAR: A Polynomial Arithmetic Framework for Verifying Neural-Network Controlled Systems

We propose POLAR, a polynomial arithmetic framework that leverages polyn...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.