DeepAI AI Chat
Log In Sign Up

Deep learning as optimal control problems: models and numerical methods

04/11/2019
by   Martin Benning, et al.
0

We consider recent work of Haber and Ruthotto 2017 and Chang et al. 2018, where deep learning neural networks have been interpreted as discretisations of an optimal control problem subject to an ordinary differential equation constraint. We review the first order conditions for optimality, and the conditions ensuring optimality after discretization. This leads to a class of algorithms for solving the discrete optimal control problem which guarantee that the corresponding discrete necessary conditions for optimality are fulfilled. We discuss two different deep learning algorithms and make a preliminary analysis of the ability of the algorithms to generalise.

READ FULL TEXT

page 12

page 15

page 16

page 21

page 22

page 23

page 24

page 25

12/02/2021

Optimal Control of the Kirchhoff Equation

We consider an optimal control problem for the steady-state Kirchhoff eq...
03/29/2022

A Derivation of Nesterov's Accelerated Gradient Algorithm from Optimal Control Theory

Nesterov's accelerated gradient algorithm is derived from first principl...
03/04/2018

An Optimal Control Approach to Deep Learning and Applications to Discrete-Weight Neural Networks

Deep learning is formulated as a discrete-time optimal control problem. ...
08/18/2019

Neural Dynamics on Complex Networks

We introduce a deep learning model to learn continuous-time dynamics on ...
05/24/2023

Neural Lyapunov and Optimal Control

Optimal control (OC) is an effective approach to controlling complex dyn...
11/30/2020

Optimal exploitation of renewable resource stocks: Necessary conditions

We study a model for the exploitation of renewable stocks developed in C...
02/06/2020

Optimal Control of Sliding Droplets using the Contact Angle Distribution

Controlling the shape and position of moving and pinned droplets on a so...