DeepAI AI Chat
Log In Sign Up

Witches' Brew: Industrial Scale Data Poisoning via Gradient Matching

by   Jonas Geiping, et al.

Data Poisoning attacks involve an attacker modifying training data to maliciouslycontrol a model trained on this data. Previous poisoning attacks against deep neural networks have been limited in scope and success, working only in simplified settings or being prohibitively expensive for large datasets. In this work, we focus on a particularly malicious poisoning attack that is both "from scratch" and"clean label", meaning we analyze an attack that successfully works against new, randomly initialized models, and is nearly imperceptible to humans, all while perturbing only a small fraction of the training data. The central mechanism of this attack is matching the gradient direction of malicious examples. We analyze why this works, supplement with practical considerations. and show its threat to real-world practitioners, finding that it is the first poisoning method to cause targeted misclassification in modern deep networks trained from scratch on a full-sized, poisoned ImageNet dataset. Finally we demonstrate the limitations of existing defensive strategies against such an attack, concluding that data poisoning is a credible threat, even for large-scale deep learning systems.


page 9

page 17

page 18


Sleeper Agent: Scalable Hidden Trigger Backdoors for Neural Networks Trained from Scratch

As the curation of data for machine learning becomes increasingly automa...

Not All Poisons are Created Equal: Robust Training against Data Poisoning

Data poisoning causes misclassification of test time target examples by ...

Revisiting the Threat Space for Vision-based Keystroke Inference Attacks

A vision-based keystroke inference attack is a side-channel attack in wh...

Poisoning Web-Scale Training Datasets is Practical

Deep learning models are often trained on distributed, webscale datasets...

MetaPoison: Practical General-purpose Clean-label Data Poisoning

Data poisoning–the process by which an attacker takes control of a model...

Few-shot Backdoor Attacks via Neural Tangent Kernels

In a backdoor attack, an attacker injects corrupted examples into the tr...

Defending against malicious peripherals with Cinch

Malicious peripherals designed to attack their host computers are a grow...

Code Repositories


Witches' Brew: Industrial Scale Data Poisoning via Gradient Matching

view repo