Accumulative Poisoning Attacks on Real-time Data

06/18/2021
by   Tianyu Pang, et al.
0

Collecting training data from untrusted sources exposes machine learning services to poisoning adversaries, who maliciously manipulate training data to degrade the model accuracy. When trained on offline datasets, poisoning adversaries have to inject the poisoned data in advance before training, and the order of feeding these poisoned batches into the model is stochastic. In contrast, practical systems are more usually trained/fine-tuned on sequentially captured real-time data, in which case poisoning adversaries could dynamically poison each data batch according to the current model state. In this paper, we focus on the real-time settings and propose a new attacking strategy, which affiliates an accumulative phase with poisoning attacks to secretly (i.e., without affecting accuracy) magnify the destructive effect of a (poisoned) trigger batch. By mimicking online learning and federated learning on CIFAR-10, we show that the model accuracy will significantly drop by a single update step on the trigger batch after the accumulative phase. Our work validates that a well-designed but straightforward attacking strategy can dramatically amplify the poisoning effects, with no need to explore complex techniques.

READ FULL TEXT

page 9

page 16

research
08/27/2022

Network-Level Adversaries in Federated Learning

Federated learning is a popular strategy for training models on distribu...
research
03/05/2019

Online Data Poisoning Attack

We study data poisoning attacks in the online learning setting where the...
research
08/27/2018

Data Poisoning Attacks against Online Learning

We consider data poisoning attacks, a class of adversarial attacks on ma...
research
05/10/2019

A fast online cascaded regression algorithm for face alignment

Traditional face alignment based on machine learning usually tracks the ...
research
03/26/2020

Obliviousness Makes Poisoning Adversaries Weaker

Poisoning attacks have emerged as a significant security threat to machi...
research
04/20/2023

Get Rid Of Your Trail: Remotely Erasing Backdoors in Federated Learning

Federated Learning (FL) enables collaborative deep learning training acr...
research
10/05/2014

Real-time animation of human characters with fuzzy controllers

The production of animation is a resource intensive process in game comp...

Please sign up or login with your details

Forgot password? Click here to reset