The TrojAI Software Framework: An OpenSource tool for Embedding Trojans into Deep Learning Models

03/13/2020
by   Kiran Karra, et al.
0

In this paper, we introduce the TrojAI software framework, an open source set of Python tools capable of generating triggered (poisoned) datasets and associated deep learning (DL) models with trojans at scale. We utilize the developed framework to generate a large set of trojaned MNIST classifiers, as well as demonstrate the capability to produce a trojaned reinforcement-learning model using vector observations. Results on MNIST show that the nature of the trigger, training batch size, and dataset poisoning percentage all affect successful embedding of trojans. We test Neural Cleanse against the trojaned MNIST models and successfully detect anomalies in the trained models approximately 18% of the time. Our experiments and workflow indicate that the TrojAI software framework will enable researchers to easily understand the effects of various configurations of the dataset and training hyperparameters on the generated trojaned deep learning model, and can be used to rapidly and comprehensively test new trojan detection methods.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/14/2018

DeepMutation: Mutation Testing of Deep Learning Systems

Deep learning (DL) defines a new data-driven programming paradigm where ...
research
06/09/2022

Uncovering bias in the PlantVillage dataset

We report our investigation on the use of the popular PlantVillage datas...
research
04/14/2022

To What Extent do Deep Learning-based Code Recommenders Generate Predictions by Cloning Code from the Training Set?

Deep Learning (DL) models have been widely used to support code completi...
research
09/09/2020

Multimodal Deep Learning for Flaw Detection in Software Programs

We explore the use of multiple deep learning models for detecting flaws ...
research
06/09/2020

Tamil Vowel Recognition With Augmented MNIST-like Data Set

We report generation of a MNIST [4] compatible data set [1] for Tamil vo...
research
11/29/2020

Scaling down Deep Learning

Though deep learning models have taken on commercial and political relev...
research
05/20/2020

Reducing Overlearning through Disentangled Representations by Suppressing Unknown Tasks

Existing deep learning approaches for learning visual features tend to o...

Please sign up or login with your details

Forgot password? Click here to reset