Jacob Steinhardt

Enabling certification of verification-agnostic networks via memory-efficient semidefinite programming

Convex relaxations have emerged as a promising approach for verifying de...
10/22/2020 ∙ by Sumanth Dathathri, et al. ∙ 0 ∙ share

read it
Measuring Massive Multitask Language Understanding

We propose a new test to measure a text model's multitask accuracy. The ...
09/07/2020 ∙ by Dan Hendrycks, et al. ∙ 28 ∙ share

read it
Aligning AI With Shared Human Values

We show how to assess a language model's knowledge of basic concepts of ...
08/05/2020 ∙ by Dan Hendrycks, et al. ∙ 13 ∙ share

read it
The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization

We introduce three new robustness benchmarks consisting of naturally occ...
06/29/2020 ∙ by Dan Hendrycks, et al. ∙ 5 ∙ share

read it
Robust estimation via generalized quasi-gradients

We explore why many recently proposed robust estimation problems are eff...
05/28/2020 ∙ by Banghua Zhu, et al. ∙ 0 ∙ share

read it
Identifying Statistical Bias in Dataset Replication

Dataset replication is a useful tool for assessing whether improvements ...
05/19/2020 ∙ by Logan Engstrom, et al. ∙ 10 ∙ share

read it
Rethinking Bias-Variance Trade-off for Generalization of Neural Networks

The classical bias-variance trade-off predicts that bias decreases and v...
02/26/2020 ∙ by Zitong Yang, et al. ∙ 0 ∙ share

read it
When does the Tukey median work?

We analyze the performance of the Tukey median estimator under total var...
01/21/2020 ∙ by Banghua Zhu, et al. ∙ 0 ∙ share

read it
A Benchmark for Anomaly Segmentation

Detecting out-of-distribution examples is important for safety-critical ...
11/25/2019 ∙ by Dan Hendrycks, et al. ∙ 26 ∙ share

read it
Generalized Resilience and Robust Statistics

Robust statistics traditionally focuses on outliers, or perturbations in...
09/19/2019 ∙ by Banghua Zhu, et al. ∙ 0 ∙ share

read it
Testing Robustness Against Unforeseen Adversaries

Considerable work on adversarial defense has studied robustness to a fix...
08/21/2019 ∙ by Daniel Kang, et al. ∙ 1 ∙ share

read it
Natural Adversarial Examples

We introduce natural adversarial examples -- real-world, unmodified, and...
07/16/2019 ∙ by Dan Hendrycks, et al. ∙ 6 ∙ share

read it
Transfer of Adversarial Robustness Between Perturbation Types

We study the transfer of adversarial robustness of deep neural networks ...
05/03/2019 ∙ by Daniel Kang, et al. ∙ 12 ∙ share

read it
FrAngel: Component-Based Synthesis with Control Structures

In component-based program synthesis, the synthesizer generates a progra...
11/13/2018 ∙ by Kensen Shi, et al. ∙ 0 ∙ share

read it
Semidefinite relaxations for certifying robustness to adversarial examples

Despite their impressive performance on diverse tasks, neural networks f...
11/02/2018 ∙ by Aditi Raghunathan, et al. ∙ 0 ∙ share

read it
Stronger Data Poisoning Attacks Break Data Sanitization Defenses

Machine learning models trained on data from the outside world can be co...
11/02/2018 ∙ by Pang Wei Koh, et al. ∙ 0 ∙ share

read it
Troubling Trends in Machine Learning Scholarship

Collectively, machine learning (ML) researchers are engaged in the creat...
07/09/2018 ∙ by Zachary C Lipton, et al. ∙ 12 ∙ share

read it
Sever: A Robust Meta-Algorithm for Stochastic Optimization

In high dimensions, most machine learning methods are brittle to even a ...
03/07/2018 ∙ by Ilias Diakonikolas, et al. ∙ 0 ∙ share

read it
The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation

This report surveys the landscape of potential security threats from mal...
02/20/2018 ∙ by Miles Brundage, et al. ∙ 2 ∙ share

read it
Certified Defenses against Adversarial Examples

While neural networks have achieved high accuracy on standard image clas...
01/29/2018 ∙ by Aditi Raghunathan, et al. ∙ 0 ∙ share

read it
Resilience: A Criterion for Learning in the Presence of Arbitrary Outliers

We introduce a criterion, resilience, which allows properties of a datas...
03/15/2017 ∙ by Jacob Steinhardt, et al. ∙ 0 ∙ share

read it
Learning from Untrusted Data

The vast majority of theoretical results in machine learning and statist...
11/07/2016 ∙ by Moses Charikar, et al. ∙ 0 ∙ share

read it
Concrete Problems in AI Safety

Rapid progress in machine learning and artificial intelligence (AI) has ...
06/21/2016 ∙ by Dario Amodei, et al. ∙ 0 ∙ share

read it
Unsupervised Risk Estimation Using Only Conditional Independence Structure

We show how to estimate a model's test error from unlabeled data, on dis...
06/16/2016 ∙ by Jacob Steinhardt, et al. ∙ 0 ∙ share

read it
The Statistics of Streaming Sparse Regression

We present a sparse analogue to stochastic gradient descent that is guar...
12/13/2014 ∙ by Jacob Steinhardt, et al. ∙ 0 ∙ share

read it

Jacob Steinhardt

Featured Co-authors

Percy Liang

Pushmeet Kohli

Dawn Song

Ian Goodfellow

Zachary C Lipton

Ilias Diakonikolas

Yi Ma

Jerry Li

Stefan Wager

Daniel M. Kane

Aleksander Madry

Enabling certification of verification-agnostic networks via memory-efficient semidefinite programming

Measuring Massive Multitask Language Understanding

Aligning AI With Shared Human Values

The Many Faces of Robustness: A Critical Analysis of Out-of-Distribution Generalization

Robust estimation via generalized quasi-gradients

Identifying Statistical Bias in Dataset Replication

Rethinking Bias-Variance Trade-off for Generalization of Neural Networks

When does the Tukey median work?

A Benchmark for Anomaly Segmentation

Generalized Resilience and Robust Statistics

Testing Robustness Against Unforeseen Adversaries

Natural Adversarial Examples

Transfer of Adversarial Robustness Between Perturbation Types

FrAngel: Component-Based Synthesis with Control Structures

Semidefinite relaxations for certifying robustness to adversarial examples

Stronger Data Poisoning Attacks Break Data Sanitization Defenses

Troubling Trends in Machine Learning Scholarship

Sever: A Robust Meta-Algorithm for Stochastic Optimization

The Malicious Use of Artificial Intelligence: Forecasting, Prevention, and Mitigation

Certified Defenses against Adversarial Examples

Resilience: A Criterion for Learning in the Presence of Arbitrary Outliers

Learning from Untrusted Data

Concrete Problems in AI Safety

Unsupervised Risk Estimation Using Only Conditional Independence Structure

The Statistics of Streaming Sparse Regression