A Flow Model of Neural Networks

08/21/2017
by   Zhen Li, et al.
0

Based on a natural connection between ResNet and transport equation or its characteristic equation, we propose a continuous flow model for both ResNet and plain net. Through this continuous model, a ResNet can be explicitly constructed as a refinement of a plain net. The flow model provides an alternative perspective to understand phenomena in deep neural networks, such as why it is necessary and sufficient to use 2-layer blocks in ResNets, why deeper is better, and why ResNets are even deeper, and so on. It also opens a gate to bring in more tools from the huge area of differential equations.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
10/27/2017

Beyond Finite Layer Neural Networks: Bridging Deep Architectures and Numerical Differential Equations

In our work, we bridge deep neural network design with numerical differe...
research
09/17/2023

Energy stable neural network for gradient flow equations

In this paper, we propose an energy stable network (EStable-Net) for sol...
research
11/13/2018

Data Driven Governing Equations Approximation Using Deep Neural Networks

We present a numerical framework for approximating unknown governing equ...
research
09/26/2022

Optical Neural Ordinary Differential Equations

Increasing the layer number of on-chip photonic neural networks (PNNs) i...
research
05/27/2019

Neural Stochastic Differential Equations

Deep neural networks whose parameters are distributed according to typic...
research
02/16/2021

ResNet-LDDMM: Advancing the LDDMM Framework Using Deep Residual Networks

In deformable registration, the geometric framework - large deformation ...
research
06/22/2021

Residual Networks as Flows of Velocity Fields for Diffeomorphic Time Series Alignment

Non-linear (large) time warping is a challenging source of nuisance in t...

Please sign up or login with your details

Forgot password? Click here to reset