research
∙
05/31/2023
Mildly Overparameterized ReLU Networks Have a Favorable Loss Landscape
We study the loss landscape of two-layer mildly overparameterized ReLU n...
share
research
∙
05/24/2023
Supermodular Rank: Set Function Decomposition and Optimization
We define the supermodular rank of a function on a lattice. This is the ...
research
∙
04/12/2023
Function Space and Critical Points of Linear Convolutional Networks
We study the geometry of linear networks with one-dimensional convolutio...
research
∙
03/06/2023
Critical Points and Convergence Analysis of Generative Deep Linear Networks Trained with Bures-Wasserstein Loss
We consider a deep matrix factorization model of covariance matrices tra...
research
∙
01/17/2023
Expected Gradients of Maxout Networks and Consequences to Parameter Initialization
We study the gradients of a maxout network with respect to inputs and pa...
research
∙
11/15/2022
Characterizing the Spectrum of the NTK via a Power Series Expansion
Under mild conditions on the network initialization we derive a power se...
research
∙
11/03/2022
Geometry and convergence of natural policy gradient methods
We study the convergence of several natural policy gradient (NPG) method...
research
∙
10/21/2022
FoSR: First-order spectral rewiring for addressing oversquashing in GNNs
Graph neural networks (GNNs) are able to leverage the structure of graph...
research
∙
09/29/2022
Enumeration of max-pooling responses with generalized permutohedra
We investigate the combinatorics of max-pooling layers, which are functi...
research
∙
08/06/2022
Oversquashing in GNNs through the lens of information contraction and graph expansion
The quality of signal propagation in message-passing graph neural networ...
research
∙
06/21/2022
On the effectiveness of persistent homology
Persistent homology (PH) is one of the most popular methods in Topologic...
research
∙
06/06/2022
Spectral Bias Outside the Training Set for Deep Networks in the Kernel Regime
We provide quantitative bounds measuring the L^2 difference in function ...
research
∙
05/27/2022
Solving infinite-horizon POMDPs with memoryless stochastic policies in state-action space
Reward optimization in fully observable Markov decision processes is equ...
research
∙
04/23/2022
Continuity and Additivity Properties of Information Decompositions
Information decompositions quantify how the Shannon information about a ...
research
∙
01/12/2022
Implicit Bias of MSE Gradient Optimization in Underparameterized Neural Networks
We study the dynamics of a neural network in function space when optimiz...
research
∙
10/27/2021
Training Wasserstein GANs without gradient penalties
We propose a stable method to train Wasserstein generative adversarial n...
research
∙
10/23/2021
Learning curves for Gaussian process regression with power-law priors and targets
We study the power-law asymptotics of learning curves for Gaussian proce...
research
∙
10/14/2021
The Geometry of Memoryless Stochastic Policy Optimization in Infinite-Horizon POMDPs
We consider the problem of finding the best memoryless stochastic policy...
research
∙
08/03/2021
Geometry of Linear Convolutional Networks
We study the family of functions that are represented by a linear convol...
research
∙
07/01/2021
On the Expected Complexity of Maxout Networks
Learning with neural networks relies on the complexity of the representa...
research
∙
06/23/2021
Weisfeiler and Lehman Go Cellular: CW Networks
Graph Neural Networks (GNNs) are limited in their expressive power, stru...
research
∙
05/04/2021
Information Complexity and Generalization Bounds
We present a unifying picture of PAC-Bayesian and mutual information-bas...
research
∙
04/16/2021
Sharp bounds for the number of regions of maxout networks and vertices of Minkowski sums
We present results on the number of linear regions of the functions that...
research
∙
03/04/2021
Weisfeiler and Lehman Go Topological: Message Passing Simplicial Networks
The pairwise interaction paradigm of graph machine learning has predomin...
research
∙
12/21/2020
Tight Bounds on the Smallest Eigenvalue of the Neural Tangent Kernel for Deep ReLU Networks
A recent line of work has analyzed the theoretical properties of deep ne...
research
∙
11/30/2020
Can neural networks learn persistent homology features?
Topological data analysis uses tools from topology – the mathematical ar...
research
∙
07/18/2020
Distributed Learning via Filtered Hyperinterpolation on Manifolds
Learning mappings of data on manifolds is an important topic in contempo...
research
∙
06/12/2020
Implicit bias of gradient descent for mean squared error regression with wide neural networks
We investigate gradient descent training of wide neural networks and the...
research
∙
06/11/2020
Optimization Theory for ReLU Neural Networks Trained with Normalization Layers
The success of deep neural networks is in part due to the use of normali...
research
∙
03/15/2020
Wasserstein Distance to Independence Models
An independence model for discrete random variables is a Segre-Veronese ...
research
∙
10/22/2019
Stochastic Feedforward Neural Networks: Universal Approximation
In this chapter we take a look at the universal approximation question f...
research
∙
10/21/2019
Kernelized Wasserstein Natural Gradient
Many machine learning problems can be expressed as the optimization of s...
research
∙
10/09/2019
How Well Do WGANs Estimate the Wasserstein Metric?
Generative modelling is often cast as minimizing a similarity measure be...
research
∙
09/25/2019
HaarPooling: Graph Pooling with Compressive Haar Basis
Deep Graph Neural Networks (GNNs) are instrumental in graph classificati...
research
∙
09/15/2019
Wasserstein Diffusion Tikhonov Regularization
We propose regularization strategies for learning discriminative models ...
research
∙
06/13/2019
Factorized Mutual Information Maximization
We investigate the sets of joint probability distributions that maximize...
research
∙
10/27/2018
The Variational Deficiency Bottleneck
We introduce a bottleneck method for learning data representations based...
research
∙
07/18/2018
Ricci curvature for parametric statistics via optimal transport
We elaborate the notion of a Ricci curvature lower bound for parametrize...
research
∙
06/19/2018
Restricted Boltzmann Machines: Introduction and Review
The restricted Boltzmann machine is a network of stochastic units with u...
research
∙
03/16/2018
Natural gradient via optimal transport I
We study a natural Wasserstein gradient flow on manifolds of probability...
research
∙
09/15/2017
Mixtures and products in two graphical models
We compare two statistical models of three binary random variables. One ...
research
∙
04/06/2017
Geometry of Policy Improvement
We investigate the geometry of optimal memoryless time independent decis...
research
∙
05/31/2016
Information Theoretically Aided Reinforcement Learning for Embodied Agents
Reinforcement learning for embodied agents is a challenging problem. The...
research
∙
12/01/2015
Evaluating Morphological Computation in Muscle and DC-motor Driven Models of Human Hopping
In the context of embodied artificial intelligence, morphological comput...
research
∙
11/10/2015
Dimension of Marginals of Kronecker Product Models
A Kronecker product model is the set of visible marginal probability dis...
research
∙
08/14/2015
Hierarchical Models as Marginals of Hierarchical Models
We investigate the representation of hierarchical models in terms of mar...
research
∙
03/24/2015
Universal Approximation of Markov Kernels by Shallow Stochastic Feedforward Networks
We establish upper bounds for the minimal number of hidden units for whi...
research
∙
03/24/2015
Geometry and Determinism of Optimal Stationary Control in Partially Observable Markov Decision Processes
It is well known that for any finite state Markov decision process (MDP)...
research
∙
11/14/2014
Deep Narrow Boltzmann Machines are Universal Approximators
We show that deep narrow Boltzmann machines are universal approximators ...
research
∙
02/14/2014