Detecting Backdoor Poisoning Attacks on Deep Neural Networks by Heatmap Clustering

by   Lukas Schulth, et al.

Predicitions made by neural networks can be fraudulently altered by so-called poisoning attacks. A special case are backdoor poisoning attacks. We study suitable detection methods and introduce a new method called Heatmap Clustering. There, we apply a k-means clustering algorithm on heatmaps produced by the state-of-the-art explainable AI method Layer-wise relevance propagation. The goal is to separate poisoned from un-poisoned data in the dataset. We compare this method with a similar method, called Activation Clustering, which also uses k-means clustering but applies it on the activation of certain hidden layers of the neural network as input. We test the performance of both approaches for standard backdoor poisoning attacks, label-consistent poisoning attacks and label-consistent poisoning attacks with reduced amplitude stickers. We show that Heatmap Clustering consistently performs better than Activation Clustering. However, when considering label-consistent poisoning attacks, the latter method also yields good detection performance.


page 3

page 4

page 5

page 8


Invisible Backdoor Attacks Against Deep Neural Networks

Deep neural networks (DNNs) have been proven vulnerable to backdoor atta...

Detecting Temporally Consistent Objects in Videos through Object Class Label Propagation

Object proposals for detecting moving or static video objects need to ad...

Selective Pseudo-label Clustering

Deep neural networks (DNNs) offer a means of addressing the challenging ...

Ensemble Generative Cleaning with Feedback Loops for Defending Adversarial Attacks

Effective defense of deep neural networks against adversarial attacks re...

Learning Discriminative Features via Label Consistent Neural Network

Deep Convolutional Neural Networks (CNN) enforces supervised information...

Class-wise Thresholding for Detecting Out-of-Distribution Data

We consider the problem of detecting OoD(Out-of-Distribution) input data...