IMEXnet: A Forward Stable Deep Neural Network

03/06/2019
by   Eldad Haber, et al.
0

Deep convolutional neural networks have revolutionized many machine learning and computer vision tasks. Despite their enormous success, remaining key challenges limit their wider use. Pressing challenges include improving the network's robustness to perturbations of the input images and simplifying the design of architectures that generalize. Another problem relates to the limited "field of view" of convolution operators, which means that very deep networks are required to model nonlocal relations in high-resolution image data. We introduce the IMEXnet that addresses these challenges by adapting semi-implicit methods for partial differential equations. Compared to similar explicit networks such as the residual networks (ResNets) our network is more stable. This stability has been recently shown to reduce the sensitivity to small changes in the input features and improve generalization. The implicit step connects all pixels in the images and therefore addresses the field of view problem, while being comparable to standard convolutions in terms of the number of parameters and computational complexity. We also present a new dataset for semantic segmentation and demonstrate the effectiveness of our architecture using the NYU depth dataset.

READ FULL TEXT

page 4

page 7

page 8

research
08/05/2021

Deep Neural Networks and PIDE discretizations

In this paper, we propose neural networks that tackle the problems of st...
research
05/20/2018

Low-Cost Parameterizations of Deep Convolutional Neural Networks

Convolutional Neural Networks (CNNs) filter the input data using a serie...
research
01/06/2022

Nonlocal Kernel Network (NKN): a Stable and Resolution-Independent Deep Neural Network

Neural operators have recently become popular tools for designing soluti...
research
04/15/2016

Improving the Robustness of Deep Neural Networks via Stability Training

In this paper we address the issue of output instability of deep neural ...
research
02/20/2019

Spatially-Adaptive Filter Units for Compact and Efficient Deep Neural Networks

Convolutional neural networks excel in a number of computer vision tasks...
research
05/20/2018

Low-Cost Parameterizations of Deep Convolution Neural Networks

The main computational cost in the training of and prediction with Convo...
research
07/30/2021

Connections between Numerical Algorithms for PDEs and Neural Networks

We investigate numerous structural connections between numerical algorit...

Please sign up or login with your details

Forgot password? Click here to reset