DeepAI AI Chat
Log In Sign Up

Time Dependence in Non-Autonomous Neural ODEs

05/05/2020
by   Jared Quincy Davis, et al.
7

Neural Ordinary Differential Equations (ODEs) are elegant reinterpretations of deep networks where continuous time can replace the discrete notion of depth, ODE solvers perform forward propagation, and the adjoint method enables efficient, constant memory backpropagation. Neural ODEs are universal approximators only when they are non-autonomous, that is, the dynamics depends explicitly on time. We propose a novel family of Neural ODEs with time-varying weights, where time-dependence is non-parametric, and the smoothness of weight trajectories can be explicitly controlled to allow a tradeoff between expressiveness and efficiency. Using this enhanced expressiveness, we outperform previous Neural ODE variants in both speed and representational capacity, ultimately outperforming standard ResNet and CNN models on select image classification and video prediction tasks.

READ FULL TEXT
05/11/2023

Generalization bounds for neural ordinary differential equations and deep residual networks

Neural ordinary differential equations (neural ODEs) are a popular famil...
06/21/2021

Neural Controlled Differential Equations for Online Prediction Tasks

Neural controlled differential equations (Neural CDEs) are a continuous-...
06/19/2018

Neural Ordinary Differential Equations

We introduce a new family of deep neural network models. Instead of spec...
12/12/2020

Delay Differential Neural Networks

Neural ordinary differential equations (NODEs) treat computation of inte...
06/25/2023

A Neural RDE approach for continuous-time non-Markovian stochastic control problems

We propose a novel framework for solving continuous-time non-Markovian s...
06/03/2022

Neural Differential Equations for Learning to Program Neural Nets Through Continuous Learning Rules

Neural ordinary differential equations (ODEs) have attracted much attent...
07/07/2023

PAC bounds of continuous Linear Parameter-Varying systems related to neural ODEs

We consider the problem of learning Neural Ordinary Differential Equatio...