Deep learning as optimal control problems: models and numerical methods

04/11/2019
by   Martin Benning, et al.
0

We consider recent work of Haber and Ruthotto 2017 and Chang et al. 2018, where deep learning neural networks have been interpreted as discretisations of an optimal control problem subject to an ordinary differential equation constraint. We review the first order conditions for optimality, and the conditions ensuring optimality after discretization. This leads to a class of algorithms for solving the discrete optimal control problem which guarantee that the corresponding discrete necessary conditions for optimality are fulfilled. We discuss two different deep learning algorithms and make a preliminary analysis of the ability of the algorithms to generalise.

READ FULL TEXT

page 12

page 15

page 16

page 21

page 22

page 23

page 24

page 25

research
12/02/2021

Optimal Control of the Kirchhoff Equation

We consider an optimal control problem for the steady-state Kirchhoff eq...
research
03/29/2022

A Derivation of Nesterov's Accelerated Gradient Algorithm from Optimal Control Theory

Nesterov's accelerated gradient algorithm is derived from first principl...
research
03/04/2018

An Optimal Control Approach to Deep Learning and Applications to Discrete-Weight Neural Networks

Deep learning is formulated as a discrete-time optimal control problem. ...
research
08/18/2019

Neural Dynamics on Complex Networks

We introduce a deep learning model to learn continuous-time dynamics on ...
research
05/24/2023

Neural Lyapunov and Optimal Control

Optimal control (OC) is an effective approach to controlling complex dyn...
research
11/30/2020

Optimal exploitation of renewable resource stocks: Necessary conditions

We study a model for the exploitation of renewable stocks developed in C...
research
02/06/2020

Optimal Control of Sliding Droplets using the Contact Angle Distribution

Controlling the shape and position of moving and pinned droplets on a so...

Please sign up or login with your details

Forgot password? Click here to reset