Universal flow approximation with deep residual networks

10/21/2019
by   Johannes Müller, et al.
0

Residual networks (ResNets) are a deep learning architecture with the recursive structure x_k+1 = x_k + R_k(x_k) where R_k is a neural network and the copying of the input x_k is called a skip connection. This structure can be seen as the explicit Euler discretisation of an associated ordinary differential equation. We use this interpretation to show that by simultaneously increasing the number of skip connection as well as the expressivity of the networks R_k the flow of an arbitrary right hand side f∈ L^1( I; C_b^0, 1(R^d; R^d)) can be approximated uniformly by deep ReLU ResNets on compact sets. Further, we derive estimates on the number of parameters needed to do this up to a prescribed accuracy under temporal regularity assumptions. Finally, we discuss the possibility of using ResNets for diffeomorphic matching problems and propose some next steps in the theoretical foundation of this approach.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
06/10/2020

Is the Skip Connection Provable to Reform the Neural Network Loss Landscape?

The residual network is now one of the most effective structures in deep...
research
07/04/2023

Free energy of Bayesian Convolutional Neural Network with Skip Connection

Since the success of Residual Network(ResNet), many of architectures of ...
research
04/14/2016

Deep Residual Networks with Exponential Linear Unit

Very deep convolutional neural networks introduced new problems like van...
research
10/08/2018

Deep Diffeomorphic Normalizing Flows

The Normalizing Flow (NF) models a general probability density by estima...
research
07/13/2020

Implicit Euler ODE Networks for Single-Image Dehazing

Deep convolutional neural networks (CNN) have been applied for image deh...
research
05/15/2021

Rethinking Skip Connection with Layer Normalization in Transformers and ResNets

Skip connection, is a widely-used technique to improve the performance a...

Please sign up or login with your details

Forgot password? Click here to reset