Control On the Manifolds Of Mappings As a Setting For Deep Learning
share
We use a control-theoretic setting to model the process of training (deep learning) of Artificial Neural Networks (ANN), which are aimed at solving classification problems. A successful classifier is the network whose input-output map approximates well the classifying map defined on a finite or an infinite training set. A fruitful idea is substitution of a multi-layer ANN by a continuous-time control system, which can be seen as a neural network with infinite number of layers. Under certain conditions it can achieve high rate of approximation with presumably not so high computational cost. The problem of best approximation for this model results in optimal control problem of Bolza type for ensembles of points. The two issues to be studied are: i) possibility of a satisfactory approximation of complex classification profiles; ii) finding the values of parameters (controls) which provide the best approximation. In control-theoretic terminology it corresponds respectively to the verification of an ensemble controllability property and to the solution of an ensemble optimal control problem. In the present contribution we concentrate on the first type of problems; our main results include examples of control systems, which are approximately controllable in the groups of diffeomorphisms of ℝ^n, 𝕋^n, 𝕊^2.
READ FULL TEXT