research
∙
06/01/2023
Initial Guessing Bias: How Untrained Networks Favor Some Classes
The initial state of neural networks plays a central role in conditionin...
share
research
∙
05/25/2023
Dynamic Context Pruning for Efficient and Interpretable Autoregressive Transformers
Autoregressive Transformers adopted in Large Language Models (LLMs) are ...
research
∙
01/19/2023
An SDE for Modeling SAM: Theory and Insights
We study the SAM (Sharpness-Aware Minimization) optimizer which has rece...
research
∙
10/03/2022
Mastering Spatial Graph Prediction of Road Networks
Accurately predicting road networks from satellite images requires a glo...
research
∙
09/19/2022
On the Theoretical Properties of Noise Correlation in Stochastic Optimization
Studying the properties of stochastic noise to optimize complex non-conv...
research
∙
07/01/2022
Characterizing the Effect of Class Imbalance on the Learning Dynamics
Data imbalance is a common problem in the machine learning literature th...
research
∙
06/07/2022
Signal Propagation in Transformers: Theoretical Perspectives and the Role of Rank Collapse
Transformers have achieved remarkable success in several domains, rangin...
research
∙
03/14/2022
Phenomenology of Double Descent in Finite-Width Neural Networks
`Double descent' delineates the generalization behaviour of models depen...
research
∙
03/07/2022
Generalization Through The Lens Of Leave-One-Out Error
Despite the tremendous empirical success of deep learning models to solv...
research
∙
02/21/2022
A Globally Convergent Evolutionary Strategy for Stochastic Constrained Optimization with Applications to Reinforcement Learning
Evolutionary strategies have recently been shown to achieve competing le...
research
∙
02/06/2022
Anticorrelated Noise Injection for Improved Generalization
Injecting artificial noise into gradient descent (GD) is commonly employ...
research
∙
12/10/2021
Faster Single-loop Algorithms for Minimax Optimization without Strong Concavity
Gradient descent ascent (GDA), the simplest single-loop algorithm for no...
research
∙
10/25/2021
On the Second-order Convergence Properties of Random Search Methods
We study the theoretical convergence properties of random-search methods...
research
∙
06/11/2021
Neural Symbolic Regression that Scales
Symbolic equations are at the core of scientific discovery. The task of ...
research
∙
03/23/2021
Generative Minimization Networks: Training GANs Without Competition
Many applications in machine learning can be framed as minimization prob...
research
∙
02/22/2021
Direct-Search for a Class of Stochastic Min-Max Problems
Recent applications in machine learning have renewed the interest of the...
research
∙
10/30/2020
The power of quantum neural networks
Fault-tolerant quantum computers offer the promise of dramatically impro...
research
∙
10/14/2020
Scalable Graph Networks for Particle Simulations
Learning system dynamics directly from observations is a promising direc...
research
∙
07/07/2020
An Accelerated DFO Algorithm for Finite-sum Convex Functions
Derivative-free optimization (DFO) has recently gained a lot of momentum...
research
∙
06/24/2020
Randomized Block-Diagonal Preconditioning for Parallel Learning
We study preconditioned gradient-based optimization methods where the pr...
research
∙
06/13/2020
Convolutional Generation of Textured 3D Meshes
Recent generative models for 2D images achieve impressive visual results...
research
∙
04/17/2020
Emulation of cosmological mass maps with conditional generative adversarial networks
Mass maps created using weak gravitational lensing techniques play a cru...
research
∙
12/06/2019
Controlling Style and Semantics in Weakly-Supervised Image Generation
We propose a weakly-supervised approach for conditional image generation...
research
∙
11/23/2019
A Stochastic Tensor Method for Non-convex Optimization
We present a stochastic optimization method that uses a fourth-order reg...
research
∙
11/12/2019
Shadowing Properties of Optimization Algorithms
Ordinary differential equation (ODE) models of gradient-based optimizati...
research
∙
08/15/2019
Cosmological N-body simulations: a challenge for scalable generative models
Deep generative models, such as Generative Adversarial Networks (GANs) o...
research
∙
07/02/2019
The Role of Memory in Stochastic Optimization
The choice of how to retain information about past gradients dramaticall...
research
∙
05/22/2019
Ellipsoidal Trust Region Methods and the Marginal Value of Hessian Information for Neural Network Training
We investigate the use of ellipsoidal trust region constraints for secon...
research
∙
12/04/2018
PolyMapper: Extracting City Maps using Polygons
We propose a method to leapfrog pixel-wise, semantic segmentation of (ae...
research
∙
11/13/2018
Evaluating GANs via Duality
Generative Adversarial Networks (GANs) have shown great results in accur...
research
∙
10/05/2018
Continuous-time Models for Stochastic Optimization Algorithms
We propose a new continuous-time formulation for first-order stochastic ...
research
∙
06/20/2018
A Distributed Second-Order Algorithm You Can Trust
Due to the rapid growth of data and computational resources, distributed...
research
∙
05/27/2018
Towards a Theoretical Understanding of Batch Normalization
Normalization techniques such as Batch Normalization have been applied v...
research
∙
05/22/2018
Adversarially Robust Training through Structured Gradient Regularization
We propose a novel data-dependent structured gradient regularizer to inc...
research
∙
05/15/2018
Local Saddle Point Optimization: A Curvature Exploitation Approach
Gradient-based optimization methods are the most popular choice for find...
research
∙
03/15/2018
Escaping Saddles with Stochastic Gradients
We analyze the variance of stochastic gradients along negative curvature...
research
∙
01/27/2018
Fast Cosmic Web Simulations with Generative Adversarial Networks
Dark matter in the universe evolves through gravity to form a complex ne...
research
∙
01/23/2018
Fast Point Spread Function Modeling with Deep Learning
Modeling the Point Spread Function (PSF) of wide-field surveys is vital ...
research
∙
10/31/2017
Flexible Prior Distributions for Deep Generative Models
We consider the problem of training generative models with deep neural n...
research
∙
10/31/2017
Semantic Interpolation in Implicit Models
In implicit models, one often interpolates between sampled points in lat...
research
∙
07/28/2017
Generator Reversal
We consider the problem of training generative models with deep neural n...
research
∙
07/21/2017
Learning Aerial Image Segmentation from Online Maps
This study deals with semantic segmentation of high-resolution (aerial) ...
research
∙
07/17/2017
Cosmological model discrimination with Deep Learning
We demonstrate the potential of Deep Learning methods for measurements o...
research
∙
06/10/2017
An Online Learning Approach to Generative Adversarial Networks
We consider the problem of training generative models with a Generative ...
research
∙
05/25/2017
Stabilizing Training of Generative Adversarial Networks through Regularization
Deep generative models based on Generative Adversarial Networks (GANs) h...
research
∙
05/16/2017
Sub-sampled Cubic Regularization for Non-convex Optimization
We consider the minimization of non-convex functions that typically aris...
research
∙
03/07/2017
Leveraging Large Amounts of Weakly Supervised Data for Multi-Language Sentiment Classification
This paper presents a novel approach for multi-lingual sentiment classif...
research
∙
11/16/2016
A Semi-supervised Framework for Image Captioning
State-of-the-art approaches for image captioning require supervised trai...
research
∙
09/08/2015
Probabilistic Bag-Of-Hyperlinks Model for Entity Linking
Many fundamental problems in natural language processing rely on determi...
research
∙
06/11/2015