research
∙
05/30/2023
Identifying Spurious Biases Early in Training through the Lens of Simplicity Bias
Neural networks trained with (stochastic) gradient descent have an induc...
share
research
∙
04/27/2023
JaxPruner: A concise library for sparsity research
This paper introduces JaxPruner, an open-source JAX-based pruning and sp...
research
∙
12/27/2022
Limitations of Information-Theoretic Generalization Bounds for Gradient Descent Methods in Stochastic Convex Optimization
To date, no "information-theoretic" frameworks for reasoning about gener...
research
∙
12/01/2022
The Effect of Data Dimensionality on Neural Network Prunability
Practitioners prune neural networks for efficiency gains and generalizat...
research
∙
10/25/2022
Pruning's Effect on Generalization Through the Lens of Training and Regularization
Practitioners frequently observe that pruning improves model generalizat...
research
∙
10/06/2022
Unmasking the Lottery Ticket Hypothesis: What's Encoded in a Winning Ticket's Mask?
Modern deep learning involves training costly, highly overparameterized ...
research
∙
06/29/2022
Understanding Generalization via Leave-One-Out Conditional Mutual Information
We study the mutual information between (certain summaries of) the outpu...
research
∙
06/02/2022
Lottery Tickets on a Data Diet: Finding Initializations with Sparse Trainable Networks
A striking observation about iterative magnitude pruning (IMP; Frankle e...
research
∙
11/09/2021
Towards a Unified Information-Theoretic Framework for Generalization
In this work, we investigate the expressiveness of the "conditional mutu...
research
∙
10/22/2021
Probabilistic fine-tuning of pruning masks and PAC-Bayes self-bounded learning
We study an approach to learning pruning masks by optimizing the expecte...
research
∙
07/15/2021
Deep Learning on a Data Diet: Finding Important Examples Early in Training
The recent success of deep learning has partially been driven by trainin...
research
∙
12/14/2020
NeurIPS 2020 Competition: Predicting Generalization in Deep Learning
Understanding generalization in deep learning is arguably one of the mos...
research
∙
11/05/2020
On the Information Complexity of Proper Learners for VC Classes in the Realizable Case
We provide a negative resolution to a conjecture of Steinke and Zakynthi...
research
∙
10/28/2020
Deep learning versus kernel learning: an empirical study of loss landscape geometry and the time evolution of the Neural Tangent Kernel
In suitably initialized wide networks, small learning rates transform de...
research
∙
10/26/2020
Enforcing Interpretability and its Statistical Impacts: Trade-offs between Accuracy and Interpretability
To date, there has been no formal study of the statistical cost of inter...
research
∙
10/22/2020
In Search of Robust Measures of Generalization
One of the principal scientific challenges in deep learning is explainin...
research
∙
09/18/2020
Pruning Neural Networks at Initialization: Why are We Missing the Mark?
Recent work has explored the possibility of pruning neural networks at i...
research
∙
06/19/2020
On the role of data in PAC-Bayes bounds
The dominant term in PAC-Bayes bounds is often the Kullback–Leibler dive...
research
∙
04/27/2020
Sharpened Generalization Bounds based on Conditional Mutual Information and an Application to Noisy, Iterative Algorithms
The information-theoretic framework of Russo and J. Zou (2016) and Xu an...
research
∙
03/25/2020
RelatIF: Identifying Explanatory Training Examples via Relative Influence
In this work, we focus on the use of influence functions to identify rel...
research
∙
12/11/2019
Linear Mode Connectivity and the Lottery Ticket Hypothesis
We introduce "instability analysis," a framework for assessing whether t...
research
∙
12/09/2019
In Defense of Uniform Convergence: Generalization via derandomization with an application to interpolating predictors
We propose to study the generalization error of a learned predictor ĥ in...
research
∙
06/10/2019
Stochastic Neural Network with Kronecker Flow
Recent advances in variational inference enable the modelling of highly ...
research
∙
03/05/2019
The Lottery Ticket Hypothesis at Scale
Recent work on the "lottery ticket hypothesis" proposes that randomly-in...
research
∙
02/26/2018
Data-dependent PAC-Bayes priors via differential privacy
The Probably Approximately Correct (PAC) Bayes framework (McAllester, 19...
research
∙
12/26/2017
Entropy-SGD optimizes the prior of a PAC-Bayes bound: Data-dependent PAC-Bayes priors via differential privacy
We show that Entropy-SGD (Chaudhari et al., 2016), when viewed as a lear...
research
∙
08/02/2016
A study of the effect of JPG compression on adversarial images
Neural network image classifiers are known to be vulnerable to adversari...
research
∙
11/19/2015
Neural Network Matrix Factorization
Data often comes in the form of an array or matrix. Matrix factorization...
research
∙
05/14/2015