Lipschitz regularized Deep Neural Networks converge and generalize

08/28/2018
by   Adam M. Oberman, et al.
0

Lipschitz regularized neural networks augment the usual fidelity term used in training with a regularization term corresponding the excess Lipschitz constant of the network compared to the Lipschitz constant of the data. We prove that Lipschitz regularized neural networks converge, and provide a rate, in the limit as the number of data points n→∞. We consider the regime where perfect fitting of data is possible, which means the size of the network grows with n. There are two regimes: in the case of perfect labels, we prove convergence to the label function which corresponds to zero loss. In the case of corrupted labels which occurs when the Lipschitz constant of the data blows up, we prove convergence to a regularized label function which is the solution of a limiting variational problem.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
09/29/2020

Lipschitz neural networks are dense in the set of all Lipschitz functions

This note shows that, for a fixed Lipschitz constant L > 0, one layer ne...
research
10/31/2022

Lipschitz regularized gradient flows and latent generative particles

Lipschitz regularized f-divergences are constructed by imposing a bound ...
research
09/07/2020

System Identification Through Lipschitz Regularized Deep Neural Networks

In this paper we use neural networks to learn governing equations from d...
research
05/26/2019

Nonparametric Online Learning Using Lipschitz Regularized Deep Neural Networks

Deep neural networks are considered to be state of the art models in man...
research
03/20/2019

Box-constrained monotone L_∞-approximations and Lipschitz-continuous regularized functions

Let f:[0,1]→[0,1] be a nondecreasing function. The main goal of this wor...
research
02/16/2022

Learning Smooth Neural Functions via Lipschitz Regularization

Neural implicit fields have recently emerged as a useful representation ...
research
12/03/2021

Neural Pseudo-Label Optimism for the Bank Loan Problem

We study a class of classification problems best exemplified by the bank...

Please sign up or login with your details

Forgot password? Click here to reset