Lipschitz constant estimation of Neural Networks via sparse polynomial optimization

04/18/2020
by   Fabian Latorre, et al.
0

We introduce LiPopt, a polynomial optimization framework for computing increasingly tighter upper bounds on the Lipschitz constant of neural networks. The underlying optimization problems boil down to either linear (LP) or semidefinite (SDP) programming. We show how to use the sparse connectivity of a network, to significantly reduce the complexity of computation. This is specially useful for convolutional as well as pruned neural networks. We conduct experiments on networks with random weights as well as networks trained on MNIST, showing that in the particular case of the ℓ_∞-Lipschitz constant, our approach yields superior estimates, compared to baselines available in the literature.

READ FULL TEXT

Authors

page 1

page 2

page 3

page 4

02/10/2020

Polynomial Optimization for Bounding Lipschitz Constants of Deep Networks

The Lipschitz constant of a network plays an important role in many appl...
03/02/2022

A Quantitative Geometric Approach to Neural Network Smoothness

Fast and precise Lipschitz constant estimation of neural networks is an ...
05/28/2018

Lipschitz regularity of deep neural networks: analysis and efficient estimation

Deep neural networks are notorious for being sensitive to small well-cho...
12/10/2020

Certifying Incremental Quadratic Constraints for Neural Networks

Abstracting neural networks with constraints they impose on their inputs...
04/16/2018

MaxGain: Regularisation of Neural Networks by Constraining Activation Magnitudes

Effective regularisation of neural networks is essential to combat overf...
03/16/2022

On the sensitivity of pose estimation neural networks: rotation parameterizations, Lipschitz constants, and provable bounds

In this paper, we approach the task of determining sensitivity bounds fo...
06/26/2017

Spectrally-normalized margin bounds for neural networks

This paper presents a margin-based multiclass generalization bound for n...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.