DeepAI AI Chat
Log In Sign Up

Deep learning as optimal control problems: models and numerical methods

by   Martin Benning, et al.

We consider recent work of Haber and Ruthotto 2017 and Chang et al. 2018, where deep learning neural networks have been interpreted as discretisations of an optimal control problem subject to an ordinary differential equation constraint. We review the first order conditions for optimality, and the conditions ensuring optimality after discretization. This leads to a class of algorithms for solving the discrete optimal control problem which guarantee that the corresponding discrete necessary conditions for optimality are fulfilled. We discuss two different deep learning algorithms and make a preliminary analysis of the ability of the algorithms to generalise.


page 12

page 15

page 16

page 21

page 22

page 23

page 24

page 25


Optimal Control of the Kirchhoff Equation

We consider an optimal control problem for the steady-state Kirchhoff eq...

A Derivation of Nesterov's Accelerated Gradient Algorithm from Optimal Control Theory

Nesterov's accelerated gradient algorithm is derived from first principl...

An Optimal Control Approach to Deep Learning and Applications to Discrete-Weight Neural Networks

Deep learning is formulated as a discrete-time optimal control problem. ...

Neural Dynamics on Complex Networks

We introduce a deep learning model to learn continuous-time dynamics on ...

Neural Lyapunov and Optimal Control

Optimal control (OC) is an effective approach to controlling complex dyn...

Optimal exploitation of renewable resource stocks: Necessary conditions

We study a model for the exploitation of renewable stocks developed in C...

Optimal Control of Sliding Droplets using the Contact Angle Distribution

Controlling the shape and position of moving and pinned droplets on a so...