Preetum Nakkiran | DeepAI

DeepAI AI Chat

Featured Co-authors

Tengyu Ma
92 publications
Venkatesan Guruswami
54 publications
Ilya Sutskever
53 publications
Mikhail Belkin
53 publications
Sham Kakade
45 publications
Behnam Neyshabur
42 publications
Saurabh Garg
39 publications
Michael Mitzenmacher
38 publications
Oncel Tuzel
36 publications
Shai Shalev-Shwartz
34 publications
Hanie Sedghi
26 publications

∙ 05/30/2023

When Does Optimizing a Proper Loss Yield Calibration?

Optimizing proper loss functions is popularly believed to yield predicto...

0 Jarosław Błasiok, et al. ∙

∙ 04/19/2023

Loss minimization yields multicalibration for large neural networks

Multicalibration is a notion of fairness that aims to provide accurate p...

0 Jarosław Błasiok, et al. ∙

∙ 11/30/2022

A Unifying Theory of Distance from Calibration

We study the fundamental question of how to define and measure the dista...

0 Jarosław Błasiok, et al. ∙

∙ 10/08/2022

APE: Aligning Pretrained Encoders to Quickly Learn Aligned Multimodal Representations

Recent advances in learning aligned multimodal representations have been...

13 Elan Rosenfeld, et al. ∙

∙ 10/05/2022

The Calibration Generalization Gap

Calibration is a fundamental property of a good predictive model: it req...

5 A. Michael Carrell, et al. ∙

∙ 07/14/2022

Benign, Tempered, or Catastrophic: A Taxonomy of Overfitting

The practical success of overparameterized neural networks has motivated...

13 Neil Mallinar, et al. ∙

∙ 06/20/2022

Limitations of the NTK for Understanding Generalization in Deep Learning

The “Neural Tangent Kernel” (NTK) (Jacot et al 2018), and its empirical ...

0 Nikhil Vyas, et al. ∙

∙ 04/07/2022

What You See is What You Get: Distributional Generalization for Algorithm Design in Deep Learning

We investigate and leverage a connection between Differential Privacy (D...

7 Bogdan Kulynych, et al. ∙

∙ 03/28/2022

Knowledge Distillation: Bad Models Can Be Good Role Models

Large neural networks trained in the overparameterized regime are able t...

0 Gal Kaplun, et al. ∙

∙ 02/20/2022

Deconstructing Distributions: A Pointwise Framework of Learning

In machine learning, we traditionally evaluate the performance of a sing...

0 Gal Kaplun, et al. ∙

∙ 02/17/2022

Limitations of Neural Collapse for Understanding Generalization in Deep Learning

The recent work of Papyan, Han, Donoho (2020) presented an intriguin...

0 Like Hui, et al. ∙

∙ 11/09/2021

Turing-Universal Learners with Optimal Scaling Laws

For a given distribution, learning algorithm, and performance metric, th...

0 Preetum Nakkiran, et al. ∙

∙ 06/14/2021

Revisiting Model Stitching to Compare Neural Representations

We revisit and extend model stitching (Lenc Vedaldi 2015) as a metho...

0 Yamini Bansal, et al. ∙

∙ 10/16/2020

The Deep Bootstrap: Good Online Learners are Good Offline Generalizers

We propose a new framework for reasoning about generalization in deep le...

0 Preetum Nakkiran, et al. ∙

∙ 09/17/2020

Distributional Generalization: A New Kind of Generalization

We introduce a new notion of generalization – Distributional Generalizat...

18 Preetum Nakkiran, et al. ∙

∙ 05/15/2020

Learning Rate Annealing Can Provably Help Generalization, Even for Convex Problems

Learning rate schedule can significantly affect generalization performan...

8 Preetum Nakkiran, et al. ∙

∙ 03/04/2020

Optimal Regularization Can Mitigate Double Descent

Recent empirical and theoretical studies have shown that many learning a...

13 Preetum Nakkiran, et al. ∙

∙ 12/16/2019

More Data Can Hurt for Linear Regression: Sample-wise Double Descent

In this expository note we describe a surprising phenomenon in overparam...

0 Preetum Nakkiran, et al. ∙

∙ 12/04/2019

Deep Double Descent: Where Bigger Models and More Data Hurt

We show that a variety of modern deep learning tasks exhibit a "double-d...

13 Preetum Nakkiran, et al. ∙

∙ 05/28/2019

SGD on Neural Networks Learns Functions of Increasing Complexity

We perform an experimental study of the dynamics of Stochastic Gradient ...

0 Preetum Nakkiran, et al. ∙

∙ 01/02/2019

Adversarial Robustness May Be at Odds With Simplicity

Current techniques in machine learning are so far are unable to learn cl...

0 Preetum Nakkiran, et al. ∙

∙ 10/03/2018

Algorithmic Polarization for Hidden Markov Models

Using a mild variant of polar codes we design linear compression schemes...

0 Venkatesan Guruswami, et al. ∙

∙ 09/14/2018

The Generic Holdout: Preventing False-Discoveries in Adaptive Data Science

Adaptive data analysis has posed a challenge to science due to its abili...

0 Preetum Nakkiran, et al. ∙

∙ 07/17/2018

Tracking the ℓ_2 Norm with Constant Update Time

The ℓ_2 tracking problem is the task of obtaining a streaming algorithm ...

0 Chi-Ning Chou, et al. ∙

∙ 02/08/2018

General Strong Polarization

Arı kan's exciting discovery of polar codes has provided an altogether n...

0 Jarosław Błasiok, et al. ∙

∙ 09/19/2017

Predicting Positive and Negative Links with Noisy Queries: Theory & Practice

Social networks and interactions in social media involve both positive a...

0 Charalampos E. Tsourakakis, et al. ∙