Nicholas Carlini

research

∙ 03/06/2023

Students Parrot Their Teachers: Membership Inference on Model Distillation

Model distillation is frequently proposed as a technique to reduce the p...

0 Matthew Jagielski, et al. ∙

research

∙ 02/27/2023

Randomness in ML Defenses Helps Persistent Attackers and Hinders Evaluators

It is becoming increasingly imperative to design robust ML defenses. How...

0 Keane Lucas, et al. ∙

research

∙ 02/20/2023

Poisoning Web-Scale Training Datasets is Practical

Deep learning models are often trained on distributed, webscale datasets...

0 Nicholas Carlini, et al. ∙

research

∙ 02/15/2023

Tight Auditing of Differentially Private Machine Learning

Auditing mechanisms for differential privacy use probabilistic means to ...

0 Milad Nasr, et al. ∙

research

∙ 02/02/2023

Effective Robustness against Natural Distribution Shifts for Models with Different Training Data

“Effective robustness” measures the extra out-of-distribution (OOD) robu...

0 Zhouxing Shi, et al. ∙

research

∙ 01/30/2023

Extracting Training Data from Diffusion Models

Image diffusion models such as DALL-E 2, Imagen, and Stable Diffusion ha...

0 Nicholas Carlini, et al. ∙

research

∙ 12/28/2022

Publishing Efficient On-device Models Increases Adversarial Vulnerability

Recent increases in the computational demands of deep neural networks (D...

0 Sanghyun Hong, et al. ∙

research

∙ 12/13/2022

Considerations for Differentially Private Learning with Large-Scale Public Pretraining

The performance of differentially private machine learning can be booste...

0 Florian Tramèr, et al. ∙

research

∙ 10/31/2022

Preventing Verbatim Memorization in Language Models Gives a False Sense of Privacy

Studying data memorization in neural language models helps us understand...

0 Daphne Ippolito, et al. ∙

research

∙ 09/29/2022

No Free Lunch in "Privacy for Free: How does Dataset Condensation Help Privacy"

New methods designed to preserve data privacy require careful scrutiny. ...

0 Nicholas Carlini, et al. ∙

research

∙ 06/28/2022

Increasing Confidence in Adversarial Robustness Evaluations

Hundreds of defenses have been proposed to make deep neural networks rob...

0 Roland S. Zimmermann, et al. ∙

research

∙ 06/21/2022

The Privacy Onion Effect: Memorization is Relative

Machine learning models trained on private datasets have been shown to l...

0 Nicholas Carlini, et al. ∙

research

∙ 03/31/2022

Truth Serum: Poisoning Machine Learning Models to Reveal Their Secrets

We introduce a new class of attacks on machine learning models. We show ...

12 Florian Tramèr, et al. ∙

research

∙ 02/15/2022

Quantifying Memorization Across Neural Language Models

Large language models (LMs) have been shown to memorize parts of their t...

1 Nicholas Carlini, et al. ∙

research

∙ 12/24/2021

Counterfactual Memorization in Neural Language Models

Modern neural language models widely used in tasks across NLP risk memor...

25 Chiyuan Zhang, et al. ∙

research

∙ 09/28/2021

Unsolved Problems in ML Safety

Machine learning (ML) systems are rapidly increasing in size, are acquir...

0 Dan Hendrycks, et al. ∙

research

∙ 08/16/2021

NeuraCrypt is not private

NeuraCrypt (Yara et al. arXiv 2021) is an algorithm that converts a sens...

0 Nicholas Carlini, et al. ∙

research

∙ 07/14/2021

Deduplicating Training Data Makes Language Models Better

We find that existing language modeling datasets contain many near-dupli...

0 Katherine Lee, et al. ∙

research

∙ 06/28/2021

Evading Adversarial Example Detection Defenses with Orthogonal Projected Gradient Descent

Evading adversarial example detection defenses requires finding adversar...

12 Oliver Bryniarski, et al. ∙

research

∙ 06/18/2021

Indicators of Attack Failure: Debugging and Improving Optimization of Adversarial Examples

Evaluating robustness of machine-learning models to adversarial examples...

25 Maura Pintor, et al. ∙

research

∙ 06/17/2021

Poisoning and Backdooring Contrastive Learning

Contrastive learning methods like CLIP train on noisy and uncurated trai...

0 Nicholas Carlini, et al. ∙

research

∙ 06/08/2021

AdaMatch: A Unified Approach to Semi-Supervised Learning and Domain Adaptation

We extend semi-supervised learning to the problem of domain adaptation t...

0 David Berthelot, et al. ∙

research

∙ 06/08/2021

Handcrafted Backdoors in Deep Neural Networks

Deep neural networks (DNNs), while accurate, are expensive to train. Man...

0 Sanghyun Hong, et al. ∙

research

∙ 05/04/2021

Poisoning the Unlabeled Dataset of Semi-Supervised Learning

Semi-supervised machine learning models learn from a (small) set of labe...

15 Nicholas Carlini, et al. ∙

research

∙ 12/14/2020

Extracting Training Data from Large Language Models

It has become common to publish large (billion parameter) language model...

0 Nicholas Carlini, et al. ∙

research

∙ 09/30/2020

Erratum Concerning the Obfuscated Gradients Attack on Stochastic Activation Pruning

Stochastic Activation Pruning (SAP) (Dhillon et al., 2018) is a defense ...

0 Guneet S. Dhillon, et al. ∙

research

∙ 09/23/2020

A Partial Break of the Honeypots Defense to Catch Adversarial Attacks

A recent defense proposes to inject "honeypots" into neural networks in ...

0 Nicholas Carlini, et al. ∙

research

∙ 07/28/2020

Label-Only Membership Inference Attacks

Membership inference attacks are one of the simplest forms of privacy le...

8 Christopher A. Choquette-Choo, et al. ∙

research

∙ 07/01/2020

Measuring Robustness to Natural Distribution Shifts in Image Classification

We study how robust current ImageNet models are to distribution shifts a...

3 Rohan Taori, et al. ∙

research

∙ 04/01/2020

Evading Deepfake-Image Detectors with White- and Black-Box Attacks

It is now possible to synthesize highly realistic images of people who d...

4 Nicholas Carlini, et al. ∙

research

∙ 03/10/2020

Cryptanalytic Extraction of Neural Network Models

We argue that the machine learning problem of model extraction is actual...

0 Nicholas Carlini, et al. ∙

research

∙ 02/19/2020

On Adaptive Attacks to Adversarial Example Defenses

Adaptive attacks have (rightfully) become the de facto standard for eval...

9 Florian Tramèr, et al. ∙

research

∙ 02/11/2020

Fundamental Tradeoffs between Invariance and Sensitivity to Adversarial Perturbations

Adversarial examples are malicious inputs crafted to induce misclassific...

0 Florian Tramèr, et al. ∙

research

∙ 01/21/2020

FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence

Semi-supervised learning (SSL) provides an effective means of leveraging...

7 Kihyuk Sohn, et al. ∙

research

∙ 11/21/2019

ReMixMatch: Semi-Supervised Learning with Distribution Alignment and Augmentation Anchoring

We improve the recently-proposed "MixMatch" semi-supervised learning alg...

24 David Berthelot, et al. ∙

research

∙ 10/29/2019

Distribution Density, Tails, and Outliers in Machine Learning: Metrics and Applications

We develop techniques to quantify the degree to which a given (training ...

24 Nicholas Carlini, et al. ∙

research

∙ 09/03/2019

High-Fidelity Extraction of Neural Network Models

Model extraction allows an adversary to steal a copy of a remotely deplo...

10 Matthew Jagielski, et al. ∙

research

∙ 07/12/2019

Stateful Detection of Black-Box Adversarial Attacks

The problem of adversarial examples, evasion attacks on machine learning...

0 Steven Chen, et al. ∙

research

∙ 05/17/2019

A critique of the DeepSec Platform for Security Analysis of Deep Learning Models

At IEEE S&P 2019, the paper "DeepSec: A Uniform Platform for Security An...

6 Nicholas Carlini, et al. ∙

research

∙ 05/06/2019

MixMatch: A Holistic Approach to Semi-Supervised Learning

Semi-supervised learning has proven to be a powerful paradigm for levera...

20 David Berthelot, et al. ∙

research

∙ 03/29/2019

SysML: The New Frontier of Machine Learning Systems

Machine learning (ML) techniques are enjoying rapidly increasing adoptio...

0 Alexander Ratner, et al. ∙

research

∙ 03/25/2019

Exploiting Excessive Invariance caused by Norm-Bounded Adversarial Robustness

Adversarial examples are malicious inputs crafted to cause a model to mi...

6 Jörn-Henrik Jacobsen, et al. ∙

research

∙ 03/22/2019

Imperceptible, Robust, and Targeted Adversarial Examples for Automatic Speech Recognition

Adversarial examples are inputs to machine learning models designed by a...

40 Yao Qin, et al. ∙

research

∙ 02/18/2019

On Evaluating Adversarial Robustness

Correctly evaluating defenses against adversarial examples has proven to...

10 Nicholas Carlini, et al. ∙

research

∙ 02/06/2019

Is AmI (Attacks Meet Interpretability) Robust to Adversarial Examples?

No....

10 Nicholas Carlini, et al. ∙

research

∙ 09/22/2018

Unrestricted Adversarial Examples

We introduce a two-player contest for evaluating the safety and robustne...

14 Tom B. Brown, et al. ∙

research

∙ 04/10/2018

On the Robustness of the CVPR 2018 White-Box Adversarial Example Defenses

Neural networks are known to be vulnerable to adversarial examples. In t...

0 Anish Athalye, et al. ∙

research

∙ 02/22/2018

The Secret Sharer: Measuring Unintended Neural Network Memorization & Extracting Secrets

Machine learning models based on neural networks and deep learning are b...

0 Nicholas Carlini, et al. ∙

research

∙ 02/01/2018

Obfuscated Gradients Give a False Sense of Security: Circumventing Defenses to Adversarial Examples

We identify obfuscated gradients as a phenomenon that leads to a false s...

0 Anish Athalye, et al. ∙

research

∙ 01/05/2018

Audio Adversarial Examples: Targeted Attacks on Speech-to-Text

We construct targeted audio adversarial examples on automatic speech rec...

0 Nicholas Carlini, et al. ∙

Nicholas Carlini

Featured Co-authors