Deep Declarative Networks: A New Hope

09/11/2019
by   Stephen Gould, et al.
38

We introduce a new class of end-to-end learnable models wherein data processing nodes (or network layers) are defined in terms of desired behavior rather than an explicit forward function. Specifically, the forward function is implicitly defined as the solution to a mathematical optimization problem. Consistent with nomenclature in the programming languages community, we name our models deep declarative networks. Importantly, we show that the class of deep declarative networks subsumes current deep learning models. Moreover, invoking the implicit function theorem, we show how gradients can be back-propagated through declaratively defined data processing nodes thereby enabling end-to-end learning. We show how these declarative processing nodes can be implemented in the popular PyTorch deep learning software library allowing declarative and imperative nodes to co-exist within the same network. We provide numerous insights and illustrative examples of declarative nodes and demonstrate their application for image and point cloud classification tasks.

READ FULL TEXT
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 3

page 4

03/03/2020

Implicitly Defined Layers in Neural Networks

In conventional formulations of multilayer feedforward neural networks, ...
05/27/2021

Optimization Induced Equilibrium Networks

Implicit equilibrium models, i.e., deep neural networks (DNNs) defined b...
07/01/2018

cilantro: a lean, versatile, and efficient library for point cloud data processing

We introduce cilantro, an open-source C++ library for geometric and gene...
08/17/2019

Implicit Deep Learning

We define a new class of "implicit" deep learning prediction rules that ...
12/12/2016

Neurogenesis Deep Learning

Neural machine learning methods, such as deep neural networks (DNN), hav...
03/23/2021

Fixed Point Networks: Implicit Depth Models with Jacobian-Free Backprop

A growing trend in deep learning replaces fixed depth models by approxim...
07/17/2020

End-to-end Deep Prototype and Exemplar Models for Predicting Human Behavior

Traditional models of category learning in psychology focus on represent...

Code Repositories

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.