Implicit Deep Learning

08/17/2019

∙

by   Laurent El Ghaoui, et al.

∙

0

∙

share

We define a new class of "implicit" deep learning prediction rules that generalize the recursive rules of feedforward neural networks. These models are based on the solution of a fixed-point equation involving a single a vector of hidden features. The new framework greatly simplifies the notation of deep learning, and opens up new possibilities, for example in terms of novel architectures and algorithms, robustness analysis and design, interpretability, sparsity, and network architecture optimization.