
Theoretical Issues in Deep Networks: Approximation, Optimization and Generalization
While deep learning is successful in a number of applications, it is not...
read it

Biologicallyplausible learning algorithms can scale to large datasets
The backpropagation (BP) algorithm is often thought to be biologically i...
read it

A Surprising Linear Relationship Predicts Test Performance in Deep Networks
Given two networks with the same training loss on a dataset, when would ...
read it

Theory IIIb: Generalization in Deep Networks
A main puzzle of deep neural networks (DNNs) revolves around the apparen...
read it

FisherRao Metric, Geometry, and Complexity of Neural Networks
We study the relationship between geometry and capacity measures for dee...
read it

Streaming Normalization: Towards Simpler and More Biologicallyplausible Normalizations for Online and Recurrent Learning
We systematically explored a spectrum of normalization algorithms relate...
read it

Viewtolerant face recognition and Hebbian learning imply mirrorsymmetric neural tuning to head orientation
The primate brain contains a hierarchy of visual areas, dubbed the ventr...
read it

Bridging the Gaps Between Residual Learning, Recurrent Neural Networks and Visual Cortex
We discuss relations between Residual Networks (ResNet), Recurrent Neura...
read it

Deep Convolutional Networks are Hierarchical Kernel Machines
In itheory a typical layer of a hierarchical architecture consists of H...
read it

Pruning Convolutional Neural Networks for Image Instance Retrieval
In this work, we focus on the problem of image instance retrieval with d...
read it

Do Deep Neural Networks Suffer from Crowding?
Crowding is a visual effect suffered by humans, in which an object that ...
read it

Theory II: Landscape of the Empirical Risk in Deep Learning
Previous theoretical work on deep learning and neural network optimizati...
read it

Compression of Deep Neural Networks for Image Instance Retrieval
Image instance retrieval is the problem of retrieving images from a data...
read it

Holographic Embeddings of Knowledge Graphs
Learning embeddings of entities and relations is an efficient and versat...
read it

Nested Invariance Pooling and RBM Hashing for Image Instance Retrieval
The goal of this work is the computation of very compact binary hashes f...
read it

Group Invariant Deep Representations for Image Instance Retrieval
Most image instance retrieval pipelines are based on comparison of vecto...
read it

A Deep Representation for Invariance And Music Classification
Representations in the auditory cortex might be based on mechanisms simi...
read it

Learning with a Wasserstein Loss
Learning to predict multilabel outputs is challenging, but in many prob...
read it

Learning with Group Invariant Features: A Kernel Perspective
We analyze in this paper a random feature map based on a theory of invar...
read it

On Learnability, Complexity and Stability
We consider the fundamental question of learnability of a hypotheses cla...
read it

Multiclass Learning with Simplex Coding
In this paper we discuss a novel framework for multiclass learning, defi...
read it

Learning Manifolds with KMeans and KFlats
We study the problem of estimating a manifold from random samples. In pa...
read it

Unsupervised learning of clutterresistant visual representations from natural videos
Populations of neurons in inferotemporal cortex (IT) maintain an explici...
read it

Neural tuning size is a key factor underlying holistic face processing
Faces are a class of visual stimuli with unique significance, for a vari...
read it

Unsupervised Learning of Invariant Representations in Hierarchical Architectures
The present phase of Machine Learning is characterized by supervised lea...
read it

Can a biologicallyplausible hierarchy effectively replace face detection, alignment, and recognition pipelines?
The standard approach to unconstrained face recognition in natural photo...
read it

Theory of Deep Learning III: explaining the nonoverfitting puzzle
A main puzzle of deep networks revolves around the absence of overfittin...
read it

Theory of Deep Learning IIb: Optimization Properties of SGD
In Theory IIb we characterize with a mix of theory and experiments the o...
read it

An analysis of training and generalization errors in shallow and deep networks
An open problem around deep networks is the apparent absence of overfit...
read it

Approximate inference with Wasserstein gradient flows
We present a novel approximate inference method for diffusion processes,...
read it

Theory III: Dynamics and Generalization in Deep Networks
We review recent observations on the dynamical systems induced by gradie...
read it