Jacob Steinhardt

Understanding Generalization in Adversarial Training via the Bias-Variance Decomposition

Adversarially trained models exhibit a large generalization gap: they ca...
03/17/2021 ∙ by Yaodong Yu, et al. ∙ 8 ∙ share

read it
Approximating How Single Head Attention Learns

Why do models often attend to salient words, and how does this evolve th...
03/13/2021 ∙ by Charlie Snell, et al. ∙ 4 ∙ share

read it
Limitations of Post-Hoc Feature Alignment for Robustness

Feature alignment is an approach to improving robustness to distribution...
03/10/2021 ∙ by Collin Burns, et al. ∙ 5 ∙ share

read it
Measuring Mathematical Problem Solving With the MATH Dataset

Many intellectual endeavors require mathematical problem solving, but th...
03/05/2021 ∙ by Dan Hendrycks, et al. ∙ 0 ∙ share

read it
Enabling certification of verification-agnostic networks via memory-efficient semidefinite programming

Convex relaxations have emerged as a promising approach for verifying de...
10/22/2020 ∙ by Sumanth Dathathri, et al. ∙ 6 ∙ share

read it
Measuring Massive Multitask Language Understanding

We propose a new test to measure a text model's multitask accuracy. The ...
09/07/2020 ∙ by Dan Hendrycks, et al. ∙ 28 ∙ share

read it
Aligning AI With Shared Human Values

We show how to assess a language model's knowledge of basic concepts of ...
08/05/2020 ∙ by Dan Hendrycks, et al. ∙ 13 ∙ share

read it
The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization

We introduce three new robustness benchmarks consisting of naturally occ...
06/29/2020 ∙ by Dan Hendrycks, et al. ∙ 5 ∙ share

read it
Robust estimation via generalized quasi-gradients

We explore why many recently proposed robust estimation problems are eff...
05/28/2020 ∙ by Banghua Zhu, et al. ∙ 0 ∙ share

read it
Identifying Statistical Bias in Dataset Replication

Dataset replication is a useful tool for assessing whether improvements ...
05/19/2020 ∙ by Logan Engstrom, et al. ∙ 10 ∙ share

read it
Rethinking Bias-Variance Trade-off for Generalization of Neural Networks

The classical bias-variance trade-off predicts that bias decreases and v...
02/26/2020 ∙ by Zitong Yang, et al. ∙ 0 ∙ share

read it
When does the Tukey median work?

We analyze the performance of the Tukey median estimator under total var...
01/21/2020 ∙ by Banghua Zhu, et al. ∙ 0 ∙ share

read it
A Benchmark for Anomaly Segmentation

Detecting out-of-distribution examples is important for safety-critical ...
11/25/2019 ∙ by Dan Hendrycks, et al. ∙ 26 ∙ share

read it
Generalized Resilience and Robust Statistics

Robust statistics traditionally focuses on outliers, or perturbations in...
09/19/2019 ∙ by Banghua Zhu, et al. ∙ 0 ∙ share

read it
Testing Robustness Against Unforeseen Adversaries

Considerable work on adversarial defense has studied robustness to a fix...
08/21/2019 ∙ by Daniel Kang, et al. ∙ 1 ∙ share

read it
Natural Adversarial Examples

We introduce natural adversarial examples -- real-world, unmodified, and...
07/16/2019 ∙ by Dan Hendrycks, et al. ∙ 6 ∙ share

read it
Transfer of Adversarial Robustness Between Perturbation Types

We study the transfer of adversarial robustness of deep neural networks ...
05/03/2019 ∙ by Daniel Kang, et al. ∙ 12 ∙ share

read it
FrAngel: Component-Based Synthesis with Control Structures

In component-based program synthesis, the synthesizer generates a progra...
11/13/2018 ∙ by Kensen Shi, et al. ∙ 0 ∙ share

read it
Semidefinite relaxations for certifying robustness to adversarial examples

Despite their impressive performance on diverse tasks, neural networks f...
11/02/2018 ∙ by Aditi Raghunathan, et al. ∙ 0 ∙ share

read it
Stronger Data Poisoning Attacks Break Data Sanitization Defenses

Machine learning models trained on data from the outside world can be co...
11/02/2018 ∙ by Pang Wei Koh, et al. ∙ 0 ∙ share

read it
Troubling Trends in Machine Learning Scholarship

Collectively, machine learning (ML) researchers are engaged in the creat...
07/09/2018 ∙ by Zachary C Lipton, et al. ∙ 12 ∙ share

read it
Sever: A Robust Meta-Algorithm for Stochastic Optimization

In high dimensions, most machine learning methods are brittle to even a ...
03/07/2018 ∙ by Ilias Diakonikolas, et al. ∙ 0 ∙ share

read it
The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation

This report surveys the landscape of potential security threats from mal...
02/20/2018 ∙ by Miles Brundage, et al. ∙ 2 ∙ share

read it
Certified Defenses against Adversarial Examples

While neural networks have achieved high accuracy on standard image clas...
01/29/2018 ∙ by Aditi Raghunathan, et al. ∙ 0 ∙ share

read it
Resilience: A Criterion for Learning in the Presence of Arbitrary Outliers

We introduce a criterion, resilience, which allows properties of a datas...
03/15/2017 ∙ by Jacob Steinhardt, et al. ∙ 0 ∙ share

read it
Learning from Untrusted Data

The vast majority of theoretical results in machine learning and statist...
11/07/2016 ∙ by Moses Charikar, et al. ∙ 0 ∙ share

read it
Concrete Problems in AI Safety

Rapid progress in machine learning and artificial intelligence (AI) has ...
06/21/2016 ∙ by Dario Amodei, et al. ∙ 0 ∙ share

read it
Unsupervised Risk Estimation Using Only Conditional Independence Structure

We show how to estimate a model's test error from unlabeled data, on dis...
06/16/2016 ∙ by Jacob Steinhardt, et al. ∙ 0 ∙ share

read it
The Statistics of Streaming Sparse Regression

We present a sparse analogue to stochastic gradient descent that is guar...
12/13/2014 ∙ by Jacob Steinhardt, et al. ∙ 0 ∙ share

read it

Jacob Steinhardt

Featured Co-authors