A Novel Representation of Neural Networks

10/05/2016
by   Anthony Caterini, et al.
0

Deep Neural Networks (DNNs) have become very popular for prediction in many areas. Their strength is in representation with a high number of parameters that are commonly learned via gradient descent or similar optimization methods. However, the representation is non-standardized, and the gradient calculation methods are often performed using component-based approaches that break parameters down into scalar units, instead of considering the parameters as whole entities. In this work, these problems are addressed. Standard notation is used to represent DNNs in a compact framework. Gradients of DNN loss functions are calculated directly over the inner product space on which the parameters are defined. This framework is general and is applied to two common network types: the Multilayer Perceptron and the Deep Autoencoder.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
08/15/2016

A Geometric Framework for Convolutional Neural Networks

In this paper, a geometric framework for neural networks is proposed. Th...
research
03/24/2018

A Proximal Block Coordinate Descent Algorithm for Deep Neural Network Training

Training deep neural networks (DNNs) efficiently is a challenge due to t...
research
06/06/2021

Topological Measurement of Deep Neural Networks Using Persistent Homology

The inner representation of deep neural networks (DNNs) is indecipherabl...
research
01/04/2021

Frequency Principle in Deep Learning Beyond Gradient-descent-based Training

Frequency perspective recently makes progress in understanding deep lear...
research
05/16/2016

Alternating optimization method based on nonnegative matrix factorizations for deep neural networks

The backpropagation algorithm for calculating gradients has been widely ...
research
05/15/2020

Sobolev Gradients for the Möbius Energy

Aiming at optimizing the shape of closed embedded curves within prescrib...
research
02/21/2008

Testing the number of parameters with multidimensional MLP

This work concerns testing the number of parameters in one hidden layer ...

Please sign up or login with your details

Forgot password? Click here to reset