Model Doctor: A Simple Gradient Aggregation Strategy for Diagnosing and Treating CNN Classifiers

12/09/2021
by   Zunlei Feng, et al.
Zhejiang University
1

Recently, Convolutional Neural Network (CNN) has achieved excellent performance in the classification task. It is widely known that CNN is deemed as a 'black-box', which is hard for understanding the prediction mechanism and debugging the wrong prediction. Some model debugging and explanation works are developed for solving the above drawbacks. However, those methods focus on explanation and diagnosing possible causes for model prediction, based on which the researchers handle the following optimization of models manually. In this paper, we propose the first completely automatic model diagnosing and treating tool, termed as Model Doctor. Based on two discoveries that 1) each category is only correlated with sparse and specific convolution kernels, and 2) adversarial samples are isolated while normal samples are successive in the feature space, a simple aggregate gradient constraint is devised for effectively diagnosing and optimizing CNN classifiers. The aggregate gradient strategy is a versatile module for mainstream CNN classifiers. Extensive experiments demonstrate that the proposed Model Doctor applies to all existing CNN classifiers, and improves the accuracy of 16 mainstream CNN classifiers by 1

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 4

page 5

page 7

page 10

page 11

page 12

page 13

10/02/2019

Contextual Local Explanation for Black Box Classifiers

We introduce a new model-agnostic explanation technique which explains t...
05/06/2020

Towards Frequency-Based Explanation for Robust CNN

Current explanation techniques towards a transparent Convolutional Neura...
10/30/2017

Grad-CAM++: Generalized Gradient-based Visual Explanations for Deep Convolutional Networks

Over the last decade, Convolutional Neural Network (CNN) models have bee...
04/11/2022

Generalizing Adversarial Explanations with Grad-CAM

Gradient-weighted Class Activation Mapping (Grad- CAM), is an example-ba...
11/09/2020

Robust Visual Tracking via Statistical Positive Sample Generation and Gradient Aware Learning

In recent years, Convolutional Neural Network (CNN) based trackers have ...
03/24/2018

CNN Based Adversarial Embedding with Minimum Alteration for Image Steganography

Historically, steganographic schemes were designed in a way to preserve ...
11/10/2016

Landmark-based consonant voicing detection on multilingual corpora

This paper tests the hypothesis that distinctive feature classifiers anc...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

Introduction

Image classification is a widely studied topic in the computer vision area. The image classification algorithms are the essential ingredient in many applications, such as, face recognition 

2018Deep, autonomous driving DBLP:journals/corr/BojarskiTDFFGJM16, text recognition DBLP:journals/corr/abs-2005-03492; 2015Text, medical image analysis 2017A; 2021Edge-competing, product quality inspection 2018Scale and so on. In recent years, Convolutional Neural Network (CNN) based classification networks, e.g., AlexNet 2012ImageNet, VGG-Net 2014Very, ResNet 2016Deep, DenseNet 2017Densely, SimpleNet hasanpour2016lets, GoogLeNet 2014Going, MobileNet 2017MobileNets, ShuffleNet 2017ShuffleNet, SqueezeNet 2016SqueezeNet:, MnasNet tan2019mnasnet, have achieved breakthrough results across a variety of image classification tasks.

Those CNN classifiers achieve high classification performance but often lack straightforward interpretability of the classifier predictions 2018Understanding; 2018Explanation; 2018A. In other words, the classifier acts as a blackbox and does not provide details about why it reaches a specific classification decision.

Some works are developed to explain and debug the machine learning models

shouling2019survey. Most current works krause2016interacting; krause2017a; zhang2019manifold

focus on explaining and diagnosing the deep learning model’s prediction with visual analysis techniques. Based on those visualized diagnoses and analyses, researchers interactively optimize the deep learning models through improving data quality, choosing reliable features, and fine-tuning the model parameters. Another kind of works

bastani2017interpretability; lei2018understanding

adopted the explainable random forest to approximate the deep learning models and analyze the deep learning model by examining the explainable model. To sum up, all existing deep model explanation and debugging methods require humans interaction. There is a lack of an entirely automatic model diagnosing and treating tool for efficiently and effectively optimize the deep learning models.

In this paper, we put forward a completely automatic model diagnosing and treating tool, termed as Model Doctor, which is based on two discoveries. The first discovery is that the target category is highly correlated with sparse and specific convolutional kernels in the last few layers of the CNN classifier. Meanwhile, there are also some incorrect responses for those corresponding convolutional kernels in the background areas. The second discovery is that adversarial samples are isolated while normal samples are successive in the feature space. So, we devise the gradient aggregation strategy for diagnosing the deficiency of CNN classifiers and treating them automatically.

In the diagnosing stage, the absolute value of the first-order derivatives of the predict category w.r.t. each feature map of each layer are summed into a single correlation index, which denotes the correlation degree between the target category and the corresponding convolutional kernel. To alleviate the disturbance of inaccurate features, we also calculate the relevance index for the disturbed feature maps that are synthesized by adding some small noises to the original feature maps. Then, we aggregate those correlation index values in each layer of the CNN classifier for all training samples of the same category. The aggregated correlation index values for each convolutional kernel can be seen as the criterion for diagnosing the wrong prediction samples.

In the treating stage, we devise the channel-wise and space-wise constraints for treating CNN classifiers based on the above correlated convolutional kernels distribution for each category. The channel-wise constraint is adopted to restrain the wrong correlation between the convolutional kernels and the target category. The space-wise constraint is adopted to restrain the wrong correlation between the unrelated background features and the target category, which requires extra coarse annotations of the object area. Experiments demonstrate that the proposed Model Doctor can effectively diagnose the possible causes for the failure prediction of the model and improve the accuracy of the CNN classifier effectively.

Our contribution is therefore the first completely automatic model diagnosing and treating tool, termed as Model Doctor. We reveal two discoveries and analyze the relationship between the convolution kernels and categories, which can be used as the criterion for future researches on deep model diagnosing and treating. The gradient aggregation strategy combined with the channel-wise and space-wise constraints is devised for diagnosing and treating CNN classifiers. Extensive experiments demonstrate that the proposed methods effectively improve the accuracy of mainstream CNN classifiers by . It’s worth noting that the proposed Model Doctor is built on top of pre-trained CNN classifiers, which is applicable for all existing CNN classifiers.

Related Work

Model Debugging and Explanation.

Due to the complex operation mechanism and low transparency of machine learning models, there is usually a lack of reliable causes and reasoning to assist researchers in debugging the models. Some model explanation techniques are developed as tools for model debugging and analysis. cadamuro2016debugging proposed a debugging method for machine learning models to identify the training items most responsible for biasing the model towards creating this error. krause2016interacting presented an explanatory debugging method that explains to users how the system made each of its predictions. The user then explains any necessary corrections back to the learning system. kulesza2010explanatory presented an explanatory debugging approach for debugging machine-learned programs. brooks2015featureinsight presented an interactive visual analytics tool FeatureInsight for building new dictionary features (semantically related groups of words) for text classification problems. paiva2015an proposed a visual data classification methodology that supports users in tasks related to categorization such as training set selection, model creation, verification, and classifier tuning. The above methods focused on debugging traditional machine learning models. The improvement process needs human interaction.

For the deep learning models, bastani2017interpretability; lei2018understanding adopted the explainable random forests to approximate the blackbox model and debug the blackbox model by examining the explainable model. krause2016interacting proposed an interactive partial dependence diagnostics to understand how features affect the prediction overall. krause2017a proposed the visual analytics workflow that leverages ¡°instance-level explanations¡±, measures of local feature relevance that explain single instances. zhang2019manifold proposed a framework that utilizes visual analysis techniques to support the interpretation, debugging and comparison of machine learning models in an interactive manner. Those methods focused on diagnosing the possible causes for models’ prediction, based on which researchers handle the following model debugging and optimization manually. Different from the above methods, we focus on automatically treating the deep models based on the diagnosed results.

Attribution Method.

Existing attribution methods contain perturbation-based and backpropagation-based methods. Perturbation-based methods

zeiler2014visualizing; zhou2015predicting; zintgraf2017visualizing; lengerich2017visual directly compute the attribution of an input feature by removing, masking, or altering them and running a forward pass on the new input, measuring the difference with the actual output.

For the backpropagation-based method, baehrens2010how first applied the first-order derivative of the predict category w.r.t. the input to explain classification decisions of the Bayesian classification task. Furthermore, simonyan2013deep extended the same technique into the CNN classification network to extract class aware saliency maps. sundararajan2017axiomatic took the (signed) partial derivatives of the output w.r.t the input and multiplying them with the input itself (i.e., Gradient Input), which improves the sharpness of the attribution maps. shrikumar2016not computed the average gradient while the input varies along a linear path from a baseline, which is defined by the user and often chosen to be zero, to the initial input (i.e., Integrated Gradients). shrikumar2017learning

recently proposed DeepLift for decomposing the output prediction of a neural network on a specific input by backpropagating the contributions of all neurons in the network to every feature of the input.

bach2015on; feng2018joint proposed an approach for propagating importance scores called Layerwise Relevance Propagation (LRP). kindermans2016investigating showed that the original LRP rules were equivalent within a scaling factor to the gradient input.

With the above techniques, song2019deep adopted the Saliency Map, Gradient Input, and

-LRP to calculate the attribution map for estimating the transferability of deep networks.

ancona2018towards analyzed above Gradient Input, -LRP, Integrated Gradients and DeepLIFT (rescale) from theoretical and practical perspectives, which shows that these four methods, despite their different formulation, are firmly related, proving conditions of equivalence or approximation between them.

CAM and Grad-CAM.

Another related area is visual feature localization, which includes gradient-free technique and gradient-based technique. For the gradient-free technique, 2016Learning proposed the first visual feature localization technique, called Class Activation Mapping (CAM), to visualize the predicted class scores on any given image, highlighting the discriminative object parts detected by the CNN. CAM computes a weighted sum of the feature maps of the last convolutional layer to obtain the class activation maps. wang2020score proposed the score-CAM, a novel CAM variant, which uses the increase in confidence for the weight of each activation map. desai2020ablation introduced an enhanced visual explanation for visual sharpness called SS-CAM, which produces centralized localization of object features within an image through a smooth operation. desai2020ablation proposed the Ablation-CAM that uses ablation analysis to determine the importance (weights) of individual feature map units w.r.t. class, which is time-consuming.

For the gradient-based technique, selvaraju2020grad proposed the Grad-CAM, which utilizes a local gradient to represent the linear weight and can be applied to any average pooling-based CNN architectures without re-training. chattopadhay2018grad proposed the Grad-CAM++ that used a weighted combination of the positive high order derivatives of the last convolutional layer feature maps w.r.t a specific class score as weights to generate better object localization as well as explaining occurrences of multiple object instances in a single image. omeiza2019smooth proposed the Smooth Grad-CAM++ that calculates these maps by averaging gradients from many small perturbations of a given image and applying the resulting gradients in the generalized Grad-CAM algorithm.

Deconvolutional Visualization.

The deconvolutional network zeiler2010deconvolutional; zeiler2011adaptive was originally proposed to learning representation in an unsupervised manner and later applied to visualization 2014Visualizing. 2014Visualizing proposed the first deconvolution visualization approach DeConvNet to better understand what the higher layers in a given network have learned. ¡°DeConvNet¡± makes data flow from a neuron activation in the higher layers down to the image. JT2014STRIVING extended this work to guided backpropagation which helped understand the impact of each neuron in a deep network w.r.t. the input image. mahendran2016salient extended DeConvNet to a general method for architecture reversal and visualization.

Method

Discovery and Analysis

Discovery:

The target category is only correlated with sparse and specific convolutional kernels in the last few layers of the CNN classifier. Meanwhile, unrelated background features will disturb the prediction of the CNN classifier.

Figure 1: The relationship between the predicted category and feature maps (convolutional kernels) in each layer.

Analysis: In this section, we first give the analysis between the convolutional kernels with the target category. For the input image , the feature maps of the -th layer are denoted as , as shown in Fig. 1. With the convolutional kernels , feature maps of the -th layer are calculated as follows:

(1)

where denotes the convolutional operation,

denotes the following operation, such as pooling operation, activation function. With feature maps

of the -th layer, the predicted category is denoted as , where denotes the following operations: convolution, pooling, activation function, and fully connected layer, decided by the specific network architecture.

For feature maps of the -th layer, the first-order derivative of the predicted category w.r.t the -th feature map is calculated as follows:

(2)

The magnitude of the derivative indicates which feature value in needs to be changed the least to affect the class score the most.

Then, the summed value of -th first-order derivative is calculated as follows:

(3)

where denotes summing values of the matrix . The magnitude of indicates the importance of the feature map to affect the class score . From Eqn. (1), we can see that, with the same input feature maps and operation , the convolutional kernel determines the different feature maps . So, the magnitude of also indicates the degree of correlation between the -th convolutional kernel and the predicted category .

Fig. 2(a) shows the statistical correlation degree between all categories and the convolutional kernels in the last layer of GoogLeNet szegedy2015going on the CIFA10 dataset. For each category, we calculate the sum value for each convolutional kernel of images, prediction confidence of which are larger than . From Fig. 2(a), we can observe that each category is only correlated with spare and specific convolutional kernels (bright color), while most convolutional kernels are not correlated (dark color). For each image, the convolutional kernels with high correlation are almost identical with the statistical high correlation kernels in each layer of the network. Furthermore, the number of convolutional kernels with high correlation reduces along with the network layer goes deeper. More visual correlation diagrams about a single image and each layer are given in the supplements.

For the image with inaccurate prediction, the summed derivative map is mapped into the original image. Fig. 2(b) shows the high-confidence and low-confidence prediction images and the corresponding image covered with the summed derivative maps. We can find that some background areas have a relatively high correlation. For images with accurate predictions, there are hardly any background areas with high correlations. The above phenomenons indicate that those unrelated background features will disturb the prediction of the CNN classifier.

Figure 2: (a) the statistical correlation diagram between all categories and the convolution kernels in last layer of GoogLeNet. (b) the high and low-confidence images covered with summed derivative maps.
Discovery:

Adversarial samples are isolated while normal samples are successive in the feature space.

Analysis: In the experiment, we find that the performance of the normal sample with disturbances is more robust than the adversarial sample. So, we assume that adversarial samples are isolated while normal samples are successive in the feature space. For the normal and corresponding adversarial samples obtained with fast gradient sign method goodfellow2015explaining, noises of different ranges are added into those samples to verify their robustness with different disturbances. For each image, the noises are randomly added to the arbitrary layer of the network times separately.

Fig. 3 shows the accuracy curves of normal and adversarial samples with different disturbances. The accuracy curves are the average results of samples. From Fig. 3, we can see that a normal sample achieves accuracy close to when the noise range in value from to . However, with some noises, the prediction of the adversarial sample will turn back to its’ original true label and has accuracy close to . Only less than samples are still be predicted to the same false label as the adversarial sample. So, we conclude the secondary discovery that adversarial samples are isolated while normal samples are successive in the feature space.

Figure 3: The accuracy curves of normal and adversarial samples with different disturbances.

Model Doctor

Based on the above two discoveries, we put forward a simple gradient aggregation strategy for diagnosing and treating the mainstream CNN classifiers. In the diagnosing phase, we accumulate the correlation between the predicted category and the convolution kernels in each layer. In most of the case, the accurate prediction is only correlated with sparse and specific convolution kernels in the last few layers of the CNN classifier (discovery 1). Then, we can adopt the accumulated correlation distribution of each layer to diagnose the cause of why the sample is misclassified. In the treating stage, two constraints are devised for treating CNN classifiers based on diagnosed results and the discovery 2. The channel-wise constraint is proposed to restrain the incorrect correlative convolution kernels different from the accumulated correlation distribution. The space-wise constraint is proposed for restraining the incorrect correlation with the background features. The details about diagnosing and treating are elaborated as follows.

Figure 4:

The accuracy and increased accuracy of each category with different constraints on the mini-ImageNet dataset.

Diagnosing stage.

Inspired by the discovery 1, we accumulate the correlation between the target category and convolution kernels in each layer. To alleviate the disturbance of misclassified features, average statistics are adopted to accumulate the correlations between the target category and the convolution kernels. For the disturbed feature map , the noise matrix sampled from the interval will be added to the feature map . The average correlation value between the convolution kernel and the category is calculated as follows:

(4)

where denotes noise matrix sampled at the -th time.

For convolution kernels in each layer, Eqn.(4) is adopted to calculate the correlation with the target category. For each category, samples are used for calculating the average correlation distribution, which can illustrate the relation between the target category and the convolution kernels in each layer. is usually set to according to the experiment results. The illustrated relationship can be used to diagnose the possible reasons for the misclassified samples.

Constraint strategy.

Before the treating stage, two constraints are devised for treating CNN classifiers based on the accumulated correlation distribution between the target category and the convolution kernels in each layer. It’s worth noting that the treating is applied to the trained classifier. For the trained CNN classifier and training samples with high-confidence predictions, the average correlation value between each convolution kernel and each category is firstly calculated. Then, for the training image with GT label , the -th feature maps are denoted as and the predicted label is denoted . The channel-wise constraint on the -th layer for the image is denoted as follows:

(5)

where, the first term is used for restraining the wrong correlation between the predicted category and convolution kernels of -th layer that are different from average correlation, the second term is used to restrain all the correlations between the second high-confidence prediction and the convolution kernels of the -th layer, denotes the number of the convolution kernel in each layer, denotes noise matrix sampled at the -th time, equals if is less than the threshold value ; otherwise equals .

Furthermore, the space-wise constraint on the -th layer is proposed for restraining the incorrect correlation with the background features, which is formulated as follows:

(6)

where, denotes the rescaled background mask that has the same size as , denotes the Hadamard product. In the mask , the background area has value one, and the mask is eroded to preserve the object boundary features. It’s worth noting that the space-wise constraint requires additional annotations, which can be rough boundaries.

Treating Stage.

In the treating stage, all constraints can be applied to any layer of the CNN classifier. Based on the fact that deep layers of the CNN classifier usually contain high-level semantic features, the space-wise and channel-wise constraints are adopted to constrain the deep layers. The following ablation study on layer depth also verifies the practicability and effectiveness of the above constraining way. Furthermore, the above two constraints are usually appended to the original training loss as follows:

(7)

where, and denote the constrained layer sets of space-wise constraint and channel-wise constraint, respectively. The two constraints also could be appended to the original training loss separately. In the treating stage, the parameter setting and the optimizer are the same as the setting of the original CNN classifier.

Dataset MNIST Fashion-MNIST CIFAR-10 CIFAR-100 SVHN STL-10

mini-ImageNet

AlexNet / +All 99.60 /  93.32 /  86.32 /  55.04 /  93.44 /  67.59 / +0.66 76.92 / +3.84
+Space / +Channel -1.63 / +0.01 -1.44 / +0.09 -1.91 / +0.31 -6.81 / +1.87 -1.65 / +0.08 +0.59 / +0.62 +3.74 / +3.21
VGG-16 / +All 99.71 /  95.21 /  93.46 /  70.39 /  94.72 /  77.65 / +0.79 83.00 / +5.92
+Space / +Channel -0.98 / +0.03 -1.91 / +0.10 -2.52 / +0.74 -7.92 / +1.08 -1.67 / +0.07 +0.69 / +0.73 +5.58 / +5.50
ResNet-50 / +All 99.73 /  95.33 /  94.85 /  77.08 /  94.81 /  82.14 / +0.63 90.25 / +3.71
+Space / +Channel -1.20 / +0.00 -0.61 / +0.15 -3.44 / +0.54 -4.70 / +0.95 -0.92 / +0.04 +0.61 / +0.58 +3.61 / +3.59
SENet-34 / +All 99.75 /  95.35 /  94.76 /  74.95 /  94.67 /  81.67 / +0.92 89.23 / +2.73
+Space / +Channel -0.88 / +0.01 -3.63 / +0.15 -1.89 / +0.41 -6.21 / +1.22 -1.30 / +0.08 +0.87 / +0.73 +2.12 / +2.63
WideResNet-28 / +All 99.47 /  93.81 /  94.26 /  77.48 /  94.11 /  79.34 / +0.85 88.47 / +3.26
+Space / +Channel -1.72 / +0.19 -3.70 / +1.45 -2.94 / +0.35 -5.01 / +0.39 -2.30 / +0.02 +0.82 / +0.81 +3.24 / +3.15
ResNeXt-50 / +All 99.69 /  95.37 /  94.34 /  74.76 /  94.25 /  83.21 / +0.49 89.72 / +3.42
+Space / +Channel -1.34 / +0.01 -2.01 / +0.19 -1.77 / +1.21 -6.05 / +2.26 -1.81 / +0.03 +0.34 / +0.47 +3.19 / +3.21
DenseNet-121 / +All 99.72 /  95.43 /  95.22 /  76.92 /  95.18 /  84.03 / +0.91 89.83 / +3.29
+Space / +Channel -0.70 / +0.01 -3.38 / +0.00 -3.81 / +0.52 -5.79 / +0.75 -0.94 / +0.01 +0.84 / +0.69 +2.98 / +2.97
SimpleNet-v1 / +All 99.72 /  95.39 /  94.61 /  75.29 /  94.51 /  81.92 / +0.67 87.92 / +2.19
+Space / +Channel -1.28 / +0.01 -2.75 / +0.11 -3.67 / +0.80 -4.93 / +1.51 -1.36 / +0.07 +0.66 / +0.53 +1.97 / +1.82
EfficientNetV2-S / +All 99.66 /  93.82 /  91.07 /  61.01 /  92.14 /  79.32 / +0.78 86.33 / +4.84
+Space / +Channel -1.11 / +0.05 -1.21 / +0.08 -1.09 / +0.33 -3.91 / +2.65 -1.55 / +0.05 +0.55 / +0.71 +4.73 / +4.67
GoogLeNet / +All 99.75 /  95.20 /  94.36 /  75.28 /  94.20 /  83.33 / +0.93 91.33 / +2.32
+Space / +Channel -0.90 / +0.01 -2.98 / +0.17 -4.67 / +0.58 -6.34 / +0.71 -0.82 / +0.13 +0.63 / +0.92 +1.95 / +2.21
Xception / +All 99.75 /  95.23 /  93.35 /  74.15 /  92.38 /  79.62 / +0.96 84.25 / +3.81
+Space / +Channel -1.04 / +0.02 -1.40 / +0.05 -3.81 / +0.57 -5.45 / +0.42 -1.20 / +0.04 +0.84 / +0.93 +4.52 / +4.72
MobileNetV2 / +All 99.68 /  94.58 /  91.95 /  68.26 /  90.43 /  78.92 / +0.67 86.21 / +4.79
+Space / +Channel -2.83 / +0.00 -2.91 / +0.39 -2.08 / +1.07 -7.91 / +2.75 -0.73 / +0.08 +0.65 / +0.52 +4.61 / +4.51
Inception-v3 / +All 99.70 /  95.25 /  94.77 /  76.93 /  93.54 /  89.46 / +0.78 89.41 / +4.10
+Space / +Channel -1.20 / +0.06 +1.29 / +0.08 -2.56 / +0.52 -7.24 / +1.20 -1.72 / +0.01 +0.72 / +0.69 +3.83 / +3.70
ShuffleNetV2 / +All 99.69 /  94.63 /  92.28 /  67.39 /  92.47 /  75.21 / +0.64 87.08 / +2.69
+Space / +Channel -1.01 / +0.00 -3.62 / +0.19 -2.81 / +0.64 -3.23 / +0.43 -1.26 / +0.06 +0.56 / +0.43 +2.60 / +2.57
SqueezeNet / +All 99.71 /  94.74 /  91.93 /  67.87 /  92.65 /  78.75 / +0.59 88.58 / +3.52
+Space / +Channel -0.92 / +0.01 -1.45 / +0.18 -2.76 / +0.92 -6.81 / +0.60 -0.98 / +0.06 +0.57 / +0.59 +3.19 / +3.01
MnasNet / +All 99.66 /  93.53 /  85.55 /  53.60 /  91.08 /  74.32 / +0.31 87.33 / +3.49
+Space / +Channel -1.14 / +0.01 -2.53 / +0.03 -2.58 / +1.37 -3.90 / +0.50 -1.04 / +0.03 +0.29 / +0.19 +2.08 / +3.27
Table 1: The base and improved accuracy of classifiers on datasets. For the small-size image, due to the expanded receptive field of convolution operation, the feature map of the last few layers will contain mixture features of object and backgrounds, which leads to negative effective of the space-wise constraint. So, “+All” are not applied to the small-size image datasets.

Experiments

In the experiment, the adopted classifiers, datasets, and experiment settings are listed as follows.

Classifier. The selected classifiers cover mainstream classification network architectures, which are listed as follows: AlexNet 2012ImageNet, VGG-16 2014Very, ResNet-34 2016Deep, ResNet-50 2016Deep, WideResNet-28 zagoruyko2016wide, ResNeXt-50 xie2017aggregated, DenseNet-121 2017Densely, SimpleNet-v1 hasanpour2016lets, EfficientNetV2-S tan2021efficientnetv2, GoogLeNet 2014Going, Xception chollet2017xception, MobileNetV2 2017MobileNets, Inception-v3 szegedy2016rethinking, ShuffleNetV2 2017ShuffleNet, SqueezeNet 2016SqueezeNet:, and MnasNet tan2019mnasnet.

Dataset. The datasets we adopted contain MNIST lecun2001gradient, Fashion-MNIST xiao2017fashion, CIFAR-10, CIFAR-100 krizhevsky2009learning, SVHN netzer2011reading, STL-10 coates2011an and mini-ImageNet vinyals2016matching, which are commonly used datasets for the classification task.

Experiment setting. Unless stated otherwise, the default experiment settings are given as follows: , , the expanded pixel number is . More experiment details are given in the supplements.

The Effect of Model Doctor for SOTA Classifiers

In this section, we conduct massive experiments of mainstream CNN classifiers on datasets. All results are averages of three runs. For the space-wise constraint, we only annotated low-confidence images for each category. Of course, the classifier can achieve higher accuracy with more annotations. Before diagnose and treating, all classifiers have achieved the optimal classification performance in the setting of original training. From Table 1, we can see that Model Doctor helps all CNN classifiers improve accuracy by , which verifies the versatility and effectiveness of the proposed gradient aggregation strategy. For datasets with small-size images, only the channel-wise constraint can improve the accuracy. On the contrary, the space-wise constraint has negative effects on those small-size image datasets. The root reason is that the expanded receptive field of convolution operation brings the mixture features of objects and backgrounds in the feature map of the last few layers, which leads to the wrong scope constraint of the small-size image. What’s more, Model Doctor has little effect on high-performance networks such as, accuracy on MNIST dataset. For the regular size image datasets (STL-10 and mini-ImageNet), the space-wise constraint achieves a significant accuracy increase, which indicates that background area features indeed have some distractions for classifiers. Model Doctor only achieves accuracy close to on STL-10. The possible reason is that the object almost takes up the whole image.

To verify the effectiveness of Model Doctor on various categories, we conduct experiments on randomly selected categories times for the mini-ImageNet dataset, which contains categories and samples for each category. Table 1 shows the average results of times. More experiments on mini-ImageNet are given in the supplements. Fig. 4 shows the accuracy and increased accuracy of VGG16 on a category classification task. We can see that Model Doctor can improve higher accuracy increase on low accuracy categories, which indicates that Model Doctor can accurately diagnose the classifier failures and effectively treat them.

Visual Results

Furthermore, Fig. 5 shows the visual results of original images with different constraints and Grad-CAM for visualizing the differences intuitively. We can see that the space-wise constraint successfully restrains the irrelevant background areas. For the visual results of channel-wise constraint, most irrelevant areas are also restrained. The association maps calculated with Grad-CAM contain some background areas. The proposed Model Doctor not only can improve the classification accuracy but also regularize the association between the category and the input features.

Figure 5: Visual results of original images with different constraints and Grad-CAM on mini-ImageNet datasets.

The Ablation Study

This section conducts the ablation study of channel-wise constraint and space-wise constraint on mini-ImageNet with VGG16. The layer depth impact for different constraints is shown in Fig. 6

, where we can see that the increased accuracy of all constraints will increase when the layer goes deep. It reveals that the proposed Model Doctor is more suitable for deep layers of classifiers. The possible reason is that the deep and shallow layers contain more high-level semantic features and low-level features, respectively. Constraining shallow layer will disturb feature extraction ability of classifier on essential features.

Furthermore, we conduct the ablation study on the annotated sample number and the expanded pixel number of the background mask. From Table 2, we can see that annotated samples for space-wise constraint have achieved near-optimal accuracy increase, which indicates that only small part annotations are enough for the space-wise constraint. Table 3 shows that -pixel expansion achieves the best performance and the expanded mask with less than pixels achieves a negative effect, the reason of which is that boundary features are crucial for the classification task.

Discussion and Future Work

From Table 1 , we can see that the treated classifier still can’t achieve accuracy. The possible reasons contain multiple aspects: 1) hard samples have different correlations with convolution kernels from the correlations of normal samples; 2) the training samples are not sufficient for the classifier; 3) the classifier may be over-/under-fitting. We will explore actual causes in future work.

In this paper, we verify Model Doctor on mainstream CNN classifiers. Actually, Model Doctor can be extended to a unified framework for diagnosing and treating deep models for different tasks. We will focus on exploring the extension of Model Doctor on segmentation, detection, and key points identification tasks in the future.

Figure 6: The ablation study of constraints on layer depth. Red dash line denotes the original accuracy ().
Number 100 200 400 600 800 1000
VGG16 Ratio 2 4 8 12 16 20
83.00% +Space +3.22% +5.58% +6.28% +7.53% +7.96% +8.03%
Table 2: The ablation study of space-wise constraint on annotation samples.
VGG16 Pixel 0 5 10 20 30 40
83.00% +Space -7.48% +0.43% +3.45% +4.23% +5.58% +4.07%
Table 3: The ablation study of space-wise constraint on mask expansion.

Conclusion

In this paper, we put forward a universal Model Doctor for diagnosing and treating CNN classifiers in a completely automatic manner. Firstly, we explore and validate two discoveries that 1) each category is only correlated with sparse and specific convolution kernels, and 2) adversarial samples are isolated while normal samples are successive in the feature space, which can serve as grounds for future researches. Based on the above two discoveries, a simple gradient aggregation strategy is devised for effectively diagnosing and optimizing CNN classifiers. Massive experiments reveal that the devised two constraints are suitable for deep layers of the classifier. For trained classifiers, the proposed Model Doctor can effectively help them improve accuracy by . Model Doctor can be used as a convenient tool for researchers to optimize their CNN classifiers.

Acknowledgments

This work is funded by the National Key R&D Program of China (Grant No: 2018AAA0101503) and the Science and technology project of SGCC (State Grid Corporation of China): fundamental theory of human-in-the-loop hybrid-augmented intelligence for power grid dispatch and control.

References