research
∙
09/21/2023
Smooth ECE: Principled Reliability Diagrams via Kernel Smoothing
Calibration measures and reliability diagrams are two fundamental tools ...
share
research
∙
05/30/2023
When Does Optimizing a Proper Loss Yield Calibration?
Optimizing proper loss functions is popularly believed to yield predicto...
research
∙
04/19/2023
Loss minimization yields multicalibration for large neural networks
Multicalibration is a notion of fairness that aims to provide accurate p...
research
∙
11/30/2022
A Unifying Theory of Distance from Calibration
We study the fundamental question of how to define and measure the dista...
research
∙
10/08/2022
APE: Aligning Pretrained Encoders to Quickly Learn Aligned Multimodal Representations
Recent advances in learning aligned multimodal representations have been...
research
∙
10/05/2022
The Calibration Generalization Gap
Calibration is a fundamental property of a good predictive model: it req...
research
∙
07/14/2022
Benign, Tempered, or Catastrophic: A Taxonomy of Overfitting
The practical success of overparameterized neural networks has motivated...
research
∙
06/20/2022
Limitations of the NTK for Understanding Generalization in Deep Learning
The “Neural Tangent Kernel” (NTK) (Jacot et al 2018), and its empirical ...
research
∙
04/07/2022
What You See is What You Get: Distributional Generalization for Algorithm Design in Deep Learning
We investigate and leverage a connection between Differential Privacy (D...
research
∙
03/28/2022
Knowledge Distillation: Bad Models Can Be Good Role Models
Large neural networks trained in the overparameterized regime are able t...
research
∙
02/20/2022
Deconstructing Distributions: A Pointwise Framework of Learning
In machine learning, we traditionally evaluate the performance of a sing...
research
∙
02/17/2022
Limitations of Neural Collapse for Understanding Generalization in Deep Learning
The recent work of Papyan, Han, Donoho (2020) presented an intriguin...
research
∙
11/09/2021
Turing-Universal Learners with Optimal Scaling Laws
For a given distribution, learning algorithm, and performance metric, th...
research
∙
06/14/2021
Revisiting Model Stitching to Compare Neural Representations
We revisit and extend model stitching (Lenc Vedaldi 2015) as a metho...
research
∙
10/16/2020
The Deep Bootstrap: Good Online Learners are Good Offline Generalizers
We propose a new framework for reasoning about generalization in deep le...
research
∙
09/17/2020
Distributional Generalization: A New Kind of Generalization
We introduce a new notion of generalization – Distributional Generalizat...
research
∙
05/15/2020
Learning Rate Annealing Can Provably Help Generalization, Even for Convex Problems
Learning rate schedule can significantly affect generalization performan...
research
∙
03/04/2020
Optimal Regularization Can Mitigate Double Descent
Recent empirical and theoretical studies have shown that many learning a...
research
∙
12/16/2019
More Data Can Hurt for Linear Regression: Sample-wise Double Descent
In this expository note we describe a surprising phenomenon in overparam...
research
∙
12/04/2019
Deep Double Descent: Where Bigger Models and More Data Hurt
We show that a variety of modern deep learning tasks exhibit a "double-d...
research
∙
05/28/2019
SGD on Neural Networks Learns Functions of Increasing Complexity
We perform an experimental study of the dynamics of Stochastic Gradient ...
research
∙
01/02/2019
Adversarial Robustness May Be at Odds With Simplicity
Current techniques in machine learning are so far are unable to learn cl...
research
∙
10/03/2018
Algorithmic Polarization for Hidden Markov Models
Using a mild variant of polar codes we design linear compression schemes...
research
∙
09/14/2018
The Generic Holdout: Preventing False-Discoveries in Adaptive Data Science
Adaptive data analysis has posed a challenge to science due to its abili...
research
∙
07/17/2018
Tracking the ℓ_2 Norm with Constant Update Time
The ℓ_2 tracking problem is the task of obtaining a streaming algorithm ...
research
∙
02/08/2018
General Strong Polarization
Arı kan's exciting discovery of polar codes has provided an altogether n...
research
∙
09/19/2017