Structure preserving deep learning

by   Elena Celledoni, et al.

Over the past few years, deep learning has risen to the foreground as a topic of massive interest, mainly as a result of successes obtained in solving large-scale image processing tasks. There are multiple challenging mathematical problems involved in applying deep learning: most deep learning methods require the solution of hard optimisation problems, and a good understanding of the tradeoff between computational effort, amount of data and model complexity is required to successfully design a deep learning approach for a given problem. A large amount of progress made in deep learning has been based on heuristic explorations, but there is a growing effort to mathematically understand the structure in existing deep learning methods and to systematically design new deep learning methods to preserve certain types of structure in deep learning. In this article, we review a number of these directions: some deep neural networks can be understood as discretisations of dynamical systems, neural networks can be designed to have desirable properties such as invertibility or group equivariance, and new algorithmic frameworks based on conformal Hamiltonian systems and Riemannian manifolds to solve the optimisation problems have been proposed. We conclude our review of each of these topics by discussing some open problems that we consider to be interesting directions for future research.


page 24

page 29


Opportunities and Challenges in Deep Learning Methods on Electrocardiogram Data: A Systematic Review

Objective: To conduct a systematic review of deep learning methods on El...

Deep Learning for Visual Speech Analysis: A Survey

Visual speech, referring to the visual domain of speech, has attracted i...

Deep-Learning-based Counting Methods, Datasets, and Applications in Agriculture – A Review

The number of objects is considered an important factor in a variety of ...

Model Complexity of Deep Learning: A Survey

Model complexity is a fundamental problem in deep learning. In this pape...

Machine Learning for Stuttering Identification: Review, Challenges Future Directions

Stuttering is a speech disorder during which the flow of speech is inter...

Hands-on Bayesian Neural Networks – a Tutorial for Deep Learning Users

Modern deep learning methods have equipped researchers and engineers wit...

Explainable Biometrics in the Age of Deep Learning

Systems capable of analyzing and quantifying human physical or behaviora...

Please sign up or login with your details

Forgot password? Click here to reset