Invariant backpropagation: how to train a transformation-invariant neural network

02/16/2015
by   Sergey Demyanov, et al.
0

In many classification problems a classifier should be robust to small variations in the input vector. This is a desired property not only for particular transformations, such as translation and rotation in image classification problems, but also for all others for which the change is small enough to retain the object perceptually indistinguishable. We propose two extensions of the backpropagation algorithm that train a neural network to be robust to variations in the feature vector. While the first of them enforces robustness of the loss function to all variations, the second method trains the predictions to be robust to a particular variation which changes the loss function the most. The second methods demonstrates better results, but is slightly slower. We analytically compare the proposed algorithm with two the most similar approaches (Tangent BP and Adversarial Training), and propose their fast versions. In the experimental part we perform comparison of all algorithms in terms of classification accuracy and robustness to noise on MNIST and CIFAR-10 datasets. Additionally we analyze how the performance of the proposed algorithm depends on the dataset size and data augmentation.

READ FULL TEXT

page 1

page 2

page 3

page 4

research
05/10/2021

Robust Training Using Natural Transformation

Previous robustness approaches for deep learning models such as data aug...
research
03/22/2021

Adversarially Optimized Mixup for Robust Classification

Mixup is a procedure for data augmentation that trains networks to make ...
research
04/26/2018

Optimal-margin evolutionary classifier

We introduce a novel approach for discriminative classification using ev...
research
09/04/2018

Chi-Square Test Neural Network: A New Binary Classifier based on Backpropagation Neural Network

We introduce the chi-square test neural network: a single hidden layer b...
research
05/16/2018

Spatial Transformer Introspective Neural Network

Natural images contain many variations such as illumination differences,...
research
10/14/2020

Ferrograph image classification

It has been challenging to identify ferrograph images with a small datas...
research
02/24/2021

On the Impact of Interpretability Methods in Active Image Augmentation Method

Robustness is a significant constraint in machine learning models. The p...

Please sign up or login with your details

Forgot password? Click here to reset