research
∙
07/18/2023
Promoting Exploration in Memory-Augmented Adam using Critical Momenta
Adaptive gradient-based optimizers, particularly Adam, have left their m...
share
research
∙
07/05/2023
Additive Decoders for Latent Variables Identification and Cartesian-Product Extrapolation
We tackle the problems of latent variables identification and "out-of-su...
research
∙
06/28/2023
Identifiability of Discretized Latent Coordinate Systems via Density Landmarks Detection
Disentanglement aims to recover meaningful latent ground-truth factors f...
research
∙
04/06/2023
PopulAtion Parameter Averaging (PAPA)
Ensemble methods combine the predictions of multiple models to improve p...
research
∙
03/07/2023
Can We Scale Transformers to Predict Parameters of Diverse ImageNet Models?
Pretraining a neural network on a large dataset is becoming a cornerston...
research
∙
01/30/2023
Unlocking Slot Attention by Changing Optimal Transport Costs
Slot attention is a powerful method for object-centric modeling in image...
research
∙
12/03/2022
CrossSplit: Mitigating Label Noise Memorization through Data Splitting
We approach the problem of improving robustness of deep learning algorit...
research
∙
11/26/2022
Synergies Between Disentanglement and Sparsity: a Multi-Task Learning Perspective
Although disentangled representations are often said to be beneficial fo...
research
∙
08/08/2022
Controlled Sparsity via Constrained Optimization or: How I Learned to Stop Tuning Penalties and Love Constraints
The performance of trained neural networks is robust to harsh levels of ...
research
∙
07/15/2022
Partial Disentanglement via Mechanism Sparsity
Disentanglement via mechanism sparsity was introduced recently as a prin...
research
∙
03/09/2022
Data-Efficient Structured Pruning via Submodular Optimization
Structured pruning is an effective approach for compressing large pre-tr...
research
∙
02/28/2022
Bayesian Structure Learning with Generative Flow Networks
In Bayesian structure learning, we are interested in inferring a distrib...
research
∙
11/23/2021
Multiset-Equivariant Set Prediction with Approximate Implicit Differentiation
Most set prediction models in deep learning use set-equivariant operatio...
research
∙
11/12/2021
Convergence Rates for the MAP of an Exponential Family and Stochastic Mirror Descent – an Open Problem
We consider the problem of upper bounding the expected log-likelihood su...
research
∙
10/27/2021
A Survey of Self-Supervised and Few-Shot Object Detection
Labeling data is often expensive and time-consuming, especially for task...
research
∙
07/21/2021
Discovering Latent Causal Variables via Mechanism Sparsity: A New Principle for Nonlinear ICA
It can be argued that finding an interpretable low-dimensional represent...
research
∙
06/30/2021
Stochastic Gradient Descent-Ascent and Consensus Optimization for Smooth Games: Convergence Analysis under Expected Co-coercivity
Two of the most prominent algorithms for solving unconstrained smooth ga...
research
∙
05/25/2021
Structured Convolutional Kernel Networks for Airline Crew Scheduling
Motivated by the needs from an airline crew scheduling application, we i...
research
∙
03/16/2021
Repurposing Pretrained Models for Robust Out-of-domain Few-Shot Learning
Model-agnostic meta-learning (MAML) is a popular method for few-shot lea...
research
∙
03/02/2021
Online Adversarial Attacks
Adversarial attacks expose important vulnerabilities of deep learning mo...
research
∙
02/18/2021
SVRG Meets AdaGrad: Painless Variance Reduction
Variance reduction (VR) methods for finite-sum minimization typically re...
research
∙
11/23/2020
Geometry-Aware Universal Mirror-Prox
Mirror-prox (MP) is a well-known algorithm to solve variational inequali...
research
∙
11/23/2020
On the Convergence of Continuous Constrained Optimization for Structure Learning
Structure learning of directed acyclic graphs (DAGs) is a fundamental pr...
research
∙
09/30/2020
Machine Learning in Airline Crew Pairing to Construct Initial Clusters for Dynamic Constraint Aggregation
The crew pairing problem (CPP) is generally modelled as a set partitioni...
research
∙
09/26/2020
Flight-connection Prediction for Airline Crew Scheduling to Construct Initial Clusters for OR Optimizer
We present a case study of using machine learning classification algorit...
research
∙
08/03/2020
Implicit Regularization in Deep Learning: A View from Function Space
We approach the problem of implicit regularization in deep learning from...
research
∙
07/08/2020
Stochastic Hamiltonian Gradient Methods for Smooth Games
The success of adversarial formulations in machine learning has brought ...
research
∙
07/03/2020
Differentiable Causal Discovery from Interventional Data
Discovering causal relationships in data is a challenging task that invo...
research
∙
07/01/2020
Adversarial Example Games
The existence of adversarial examples capable of fooling trained neural ...
research
∙
06/11/2020
Adaptive Gradient Methods Converge Faster with Over-Parameterization (and you can do a line-search)
As adaptive gradient methods are typically used for training over-parame...
research
∙
05/18/2020
An Analysis of the Adaptation Speed of Causal Models
We consider the problem of discovering the causal process that generated...
research
∙
02/24/2020
Stochastic Polyak Step-size for SGD: An Adaptive Learning Rate for Fast Convergence
We propose a stochastic variant of the classical Polyak step-size (Polya...
research
∙
01/02/2020
Accelerating Smooth Games by Manipulating Spectral Shapes
We use matrix iteration theory to characterize acceleration in smooth ga...
research
∙
10/11/2019
Fast and Furious Convergence: Stochastic Second Order Methods under Interpolation
We consider stochastic second order methods for minimizing strongly-conv...
research
∙
06/19/2019
GEAR: Geometry-Aware Rényi Information
Shannon's seminal theory of information has been of paramount importance...
research
∙
06/13/2019
A Tight and Unified Analysis of Extragradient for a Whole Spectrum of Differentiable Games
We consider differentiable games: multi-objective minimization problems,...
research
∙
06/11/2019
A Closer Look at the Optimization Landscapes of Generative Adversarial Networks
Generative adversarial networks have been very successful in generative ...
research
∙
06/05/2019
Gradient-Based Neural DAG Learning
We propose a novel score-based approach to learning a directed acyclic g...
research
∙
05/24/2019
Painless Stochastic Gradient: Interpolation, Line-Search, and Convergence Rates
Recent works have shown that stochastic gradient descent (SGD) achieves ...
research
∙
04/30/2019
Implicit Regularization of Discrete Gradient Dynamics in Deep Linear Neural Networks
When optimizing over-parameterized models, such as deep neural networks,...
research
∙
04/18/2019
Reducing Noise in GAN Training with Variance Reduced Extragradient
Using large mini-batches when training generative adversarial networks (...
research
∙
02/22/2019
Centroid Networks for Few-Shot Clustering and Unsupervised Few-Shot Classification
Traditional clustering algorithms such as K-means rely heavily on the na...
research
∙
01/22/2019
Predicting Tactical Solutions to Operational Planning Problems under Imperfect Information
This paper offers a methodological contribution at the intersection of m...
research
∙
10/26/2018
Quantifying Learning Guarantees for Convex but Inconsistent Surrogates
We study consistency properties of machine learning methods based on min...
research
∙
10/19/2018
A Modern Take on the Bias-Variance Tradeoff in Neural Networks
We revisit the bias-variance tradeoff for neural networks in light of mo...
research
∙
09/17/2018
Scattering Networks for Hybrid Representation Learning
Scattering networks are a class of designed Convolutional Neural Network...
research
∙
07/31/2018
Predicting Solution Summaries to Integer Linear Programs under Imperfect Information with Machine Learning
The paper provides a methodological contribution at the intersection of ...
research
∙
07/12/2018
Negative Momentum for Improved Game Dynamics
Games generalize the optimization paradigm by introducing different obje...
research
∙
04/09/2018
Frank-Wolfe Splitting via Augmented Lagrangian Method
Minimizing a function over an intersection of convex sets is an importan...
research
∙
02/28/2018