Geometric Deep Learning: Grids, Groups, Graphs, Geodesics, and Gauges

04/27/2021
by   Michael M. Bronstein, et al.
288

The last decade has witnessed an experimental revolution in data science and machine learning, epitomised by deep learning methods. Indeed, many high-dimensional learning tasks previously thought to be beyond reach – such as computer vision, playing Go, or protein folding – are in fact feasible with appropriate computational scale. Remarkably, the essence of deep learning is built from two simple algorithmic principles: first, the notion of representation or feature learning, whereby adapted, often hierarchical, features capture the appropriate notion of regularity for each task, and second, learning by local gradient-descent type methods, typically implemented as backpropagation. While learning generic functions in high dimensions is a cursed estimation problem, most tasks of interest are not generic, and come with essential pre-defined regularities arising from the underlying low-dimensionality and structure of the physical world. This text is concerned with exposing these regularities through unified geometric principles that can be applied throughout a wide spectrum of applications. Such a 'geometric unification' endeavour, in the spirit of Felix Klein's Erlangen Program, serves a dual purpose: on one hand, it provides a common mathematical framework to study the most successful neural network architectures, such as CNNs, RNNs, GNNs, and Transformers. On the other hand, it gives a constructive procedure to incorporate prior physical knowledge into neural architectures and provide principled way to build future architectures yet to be invented.

READ FULL TEXT

Authors

page 8

page 9

page 14

page 23

page 26

page 36

page 39

page 41

02/20/2014

Le Cam meets LeCun: Deficiency and Generic Feature Learning

"Deep Learning" methods attempt to learn generic features in an unsuperv...
11/25/2016

Geometric deep learning on graphs and manifolds using mixture model CNNs

Deep learning has achieved a remarkable performance breakthrough in seve...
05/15/2022

What is an equivariant neural network?

We explain equivariant neural networks, a notion underlying breakthrough...
07/25/2016

gvnn: Neural Network Library for Geometric Computer Vision

We introduce gvnn, a neural network library in Torch aimed towards bridg...
03/18/2021

A deep learning theory for neural networks grounded in physics

In the last decade, deep learning has become a major component of artifi...
06/23/2020

Direct Feedback Alignment Scales to Modern Deep Learning Tasks and Architectures

Despite being the workhorse of deep learning, the backpropagation algori...

Code Repositories

JHU_AI_Journey

Artificial Intelligence, Master of Science @ JHU


view repo

ml-bookmarks

My bookmarks related to Machine Learning


view repo
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.