1 Introduction
As machine learning systems start to be more widely used, we are starting to care not just about the accuracy and speed of the predictions, but also why it made its specific predictions. While we need not always care about the why of a complex system in order to trust it, especially if we observe that the system has high accuracy, such trust typically hinges on the belief that some other expert has a richer understanding of the system. For instance, while we might not know exactly how planes fly in the air, we trust some experts do. In the case of machine learning models however, even machine learning experts do not have a clear understanding of why say a deep neural network makes a particular prediction. Our work proposes to address this gap by focusing on improving the understanding of experts, in addition to lay users. In particular, expert users could then use these explanations to further finetune the system (e.g. dataset/model debugging), as well as suggest different approaches for model training, so that it achieves a better performance.
Our key approach to do so is via a representer theorem for deep neural networks, which might be of independent interest even outside the context of explainable ML. We show that we can decompose the preactivation prediction values into a linear combination of training point activations, with the weights corresponding to what we call representer values, which can be used to measure the importance of each training point has on the learned parameter of the model. Using these representer values, we select representer points – training points that have large/small representer values – that could aid the understanding of the model’s prediction.
Such representer points provide a richer understanding of the deep neural network than other approaches that provide influential training points, in part because of the metaexplanation underlying our explanation: a positive representer value indicates that a similarity to that training point is excitatory, while a negative representer value indicates that a similarity to that training point is inhibitory, to the prediction at the given test point. It is in these inhibitory training points where our approach provides considerably more insight compared to other approaches: specifically, what would cause the model to not make a particular prediction? In one of our examples, we see that the model makes an error in labeling an antelope as a deer. Looking at its most inhibitory training points, we see that the dataset is rife with training images where there are antelopes in the image, but also some other animals, and the image is labeled with the other animal. These thus contribute to inhibitory effects of small antelopes with other big objects: an insight that as machine learning experts, we found deeply useful, and which is difficult to obtain via other explanatory approaches. We demonstrate the utility of our class of representer point explanations through a range of theoretical and empirical investigations.
2 Related Work
There are two main classes of approaches to explain the prediction of a model. The first class of approaches point to important input features. Ribeiro et al. [1] provide such featurebased explanations that are modelagnostic; explaining the decision locally around a test instance by fitting a local linear model in the region. Ribeiro et al. [2]
introduce Anchors, which are locally sufficient conditions of features that “holds down” the prediction so that it does not change in a local neighborhood. Such feature based explanations are particularly natural in computer vision tasks, since it enables visualizing the regions of the input pixel space that causes the classifier to make certain predictions. There are numerous works along this line, particularly focusing on gradientbased methods that provide saliency maps in the pixel space
[3, 4, 5, 6].The second class of approaches are samplebased, and they identify training samples that have the most influence on the model’s prediction on a test point. Among modelagnostic samplebased explanations are prototype selection methods [7, 8] that provide a set of “representative” samples chosen from the data set. Kim et al. [9] provide criticism alongside prototypes to explain what are not captured by prototypes. Usually such prototype and criticism selection is modelagnostic and used to accelerate the training for classifications. Modelaware samplebased explanation identify influential training samples which are the most helpful for reducing the objective loss or making the prediction. Recently, Koh and Liang [10] provide tractable approximations of influence functions that characterize the influence of each sample in terms of change in the loss. Anirudh et al. [11] propose a generic approach to influential sample selection via a graph constructed using the samples.
Our approach is based on a representer theorem for deep neural network predictions. Representer theorems [12]
in machine learning contexts have focused on nonparametric regression, specifically in reproducing kernel Hilbert spaces (RKHS), and which loosely state that under certain conditions the minimizer of a loss functional over a RKHS can be expressed as a linear combination of kernel evaluations at training points. There have been recent efforts at leveraging such insights to compositional contexts
[13, 14], though these largely focus on connections to nonparametric estimation.
Bohn et al. [13] extend the representer theorem to compositions of kernels, while Unser [14] draws connections between deep neural networks to such deep kernel estimation, specifically deep spline estimation. In our work, we consider the much simpler problem of explaining preactivation neural network predictions in terms of activations of training points, which while less illuminating from a nonparametric estimation standpoint, is arguably much more explanatory, and useful from an explainable ML standpoint.3 Representer Point Framework
Consider a classification problem, of learning a mapping from an input space (e.g., images) to an output space (e.g., labels), given training points , and corresponding labels . We consider a neural network as our prediction model, which takes the form , where and is the last intermediate layer feature in the neural network for input . Note that is the number of classes, is the dimension of the feature, is a matrix , and is all the parameters to generate the last intermediate layer from the input . Thus are all the parameters of our neural network model. The parameterization above connotes splitting of the model as a feature model and a prediction network with parameters . Note that the feature model can be arbitrarily deep, or simply the identity function, so our setup above is applicable to general feedforward networks.
Our goal is to understand to what extent does one particular training point affect the prediction of a test point as well as the learned weight parameter . Let be the loss, and be the empirical risk. To indicate the form of a representer theorem, suppose we solve for the optimal parameters for some nondecreasing . We would then like our preactivation predictions to have the decomposition: . Given such a representer theorem, can be seen as the contribution of the training data on the testing prediction . However, such representer theorems have only been developed for nonparametric predictors, specifically where lies in a reproducing kernel Hilbert space. Moreover, unlike the typical RKHS setting, finding a global minimum for the empirical risk of a deep network is difficult, if not impossible, to obtain. In the following, we provide a representer theorem that addresses these two points: it holds for deep neural networks, and for any stationary point solution.
Theorem 3.1.
Let us denote the neural network prediction function by , where and . Suppose is a stationary point of the optimization problem: , where for some . Then we have the decomposition:
where and , which we call a representer value for given .
Proof.
Note that for any stationary point, the gradient of the loss with respect to is equal to 0. We therefore have
(1) 
where
by the chain rule. We thus have that
(2) 
We note that can be seen as the resistance for training example feature towards minimizing the norm of the weight matrix . Therefore, can be used to evaluate the importance of the training data have on . Note that for any class , holds by (2). Moreover, we can observe that for to have a significant value, two conditions must be satisfied: (a) should have a large value, and (b) should have a large value. Therefore, we interpret the preactivation value as a weighted sum for the feature similarity with the weight . When is close to with a large positive weight , the prediction score for class is increased. On the other hand, when is close to with a large negative weight , the prediction score for class is then decreased.
We can thus interpret the training points with negative representer values as inhibitory points that suppress the activation value, and those with positive representer values as excitatory examples that does the opposite. We demonstrate this notion with examples further in Section 4.2. We note that such excitatory and inhibitory points provide a richer understanding of the behavior of the neural network: it provides insight both as to why the neural network prefers a particular prediction, as well as why it does not, which is typically difficult to obtain via other samplebased explanations.
3.1 Training an Interpretable Model by Imposing L2 Regularization.
Theorem 3.1 works for any model that performs a linear matrix multiplication before the activation , which is quite general and can be applied on most neuralnetworklike structures. By simply introducing a L2 regularizer on the weight with a fixed , we can easily decompose the presoftmax prediction value as some finite linear combinations of a function between the test and train data. We now state our main algorithm. First we solve the following optimization problem:
(3) 
Note that for the representer point selection to work, we would need to achieve a stationary point with high precision. In practice, we find that using a gradient descent solver with line search or LBFGS solver to finetune after converging in SGD can achieve highly accurate stationary point. Note that we can perform the finetuning step only on , which is usually efficient to compute. We can then decompose by Theorem 3.1 for any arbitrary test point , where is the contribution of training point on the presoftmax prediction . We emphasize that imposing L2 weight decay is a common practice to avoid overfitting for deep neural networks, which does not sacrifice accuracy while achieving a more interpretable model.
3.2 Generating Representer Points for a Given Pretrained Model.
We are also interested in finding representer points for a given model that has already been trained, potentially without imposing the L2 regularizer. While it is possible to add the L2 regularizer and retrain the model, the retrained model may converge to a different stationary point, and behave differently compared to the given model, in which case we cannot use the resulting representer points as explanations. Accordingly, we learn the parameters while imposing the L2 regularizer, but under the additional constraint that be close to . In this case, our learning objective becomes instead of , and our loss can be written as .
Definition 3.1.
We say that a convex loss function
is “suitable” to an activation function
, if it holds that for any , we have = .Assume that we are given such a loss function that is “suitable to” the activation function . We can then solve the following optimization problem:
(4) 
The optimization problem can be seen to be convex under the assumptions on the loss function. The parameter controls the tradeoff between the closeness of and , and the computational cost. For a small , could be arbitrarily close to , while the convergence time may be long. We note that the learning task in Eq. (4) can be seen as learning from a teacher network and imposing a regularizer to make the student model capable of generating representer points. In practice, we may take as an initialization for and perform a simple linesearch gradient descent with respect to in (4). In our experiments, we discover that the training for (4) can converge to a stationary point in a short period of time, as demonstrated in Section 4.5.
We now discuss our design for the loss function that is mentioned in . When is the softmax activation, we choose the softmax crossentropy loss, which computes the cross entropy between and for . When
is ReLU activation, we choose
, where is the elementwise product. In the following Proposition, we show that and are convex, and satisfy the desired suitability property in Definition 3.1. The proof is provided in the supplementary material.Proposition 3.1.
The loss functions and are both convex in . Moreover, is “suitable to” the softmax activation, and is “suitable to” the ReLU activation, following Definition 3.1.
As a sanity check, we perform experiments on the CIFAR10 dataset [15] with a pretrained VGG16 network [16]. We first solve (4) with loss for , and then calculate as in (2) for all train and test points. We note that the computation time for the whole procedure only takes less than a minute, given the pretrained model. We compute the Pearson correlation coefficient between the actual output and the predicted output for multiple points and plot them in Figure 1. The correlation is almost 1 for both train and test data, and most points lie at the both ends of line.
We note that Theorem 3.1 can be applied to any hidden layer with ReLU activation by defining a subnetwork from input and the output being the hidden layer of interest. The training could be done in a similar fashion by replacing with . In general, any activation can be used with a derived "suitable loss".
4 Experiments
We perform a number of experiments with multiple datasets and evaluate our method’s performance and compare with that of the influence functions.^{1}^{1}1Source code available at github.com/yankeesrules/Representer_Point_Selection. The goal of these experiments is to demonstrate that selecting the representer points is efficient and insightful in several ways. Additional experiments discussing the differences between our method and the influence function are included in the supplementary material.
4.1 Dataset Debugging
To evaluate the influence of the samples, we consider a scenario where humans need to inspect the dataset quality to ensure an improvement of the model’s performance in the test data. Realworld data is bound to be noisy, and the bigger the dataset becomes, the more difficult it will be for humans to look for and fix mislabeled data points. It is crucial to know which data points are more important than the others to the model so that prioritizing the inspection can facilitate the debugging process.
To show how well our method does in dataset debugging, we run a simulated experiment on CIFAR10 dataset [17]
with a task of binary classification with logistic regression for the classes automobiles and horses. The dataset is initially corrupted, where 40 percent of the data has the labels flipped, which naturally results in a low test accuracy of
. The simulated user will check some fraction of the train data based on the order set by several metrics including ours, and fix the labels. With the corrected version of the dataset, we retrain the model and record the test accuracies for each metrics. For our method, we train an explainable model by mimimizing (3) as explained in section 3.1. The L2 weight decay is set to for all methods for fair comparison. All experiments are repeated for 5 random splits and we report the average result. In Figure 2 we report the results for four different metrics: “ours” picks the points with bigger for training instance and its corresponding label ; “influence” prioritizes the training points with bigger influence function value; and “random” picks random points. We observe that our method recovers the same amount of training data as the influence function while achieving higher testing accuracy. Nevertheless, both methods perform better than the random selection method.4.2 Excitatory (Positive) and Inhibitory (Negative) Examples
We visualize the training points with high representer values (both positive and negative) for some test points in Animals with Attributes (AwA) dataset [18] and compare the results with those of the influence functions. We use a pretrained Resnet50 [19] model and finetune on the AwA dataset to reach over 90 percent testing accuracy. We then generate representer points as described in section 3.2. For computing the influence functions, just as described in [10], we froze all top layers of the model and trained the last layer. We report top three points for two test points in the following Figures 3 and 4. In Figure 3, which is an image of three grizzly bears, our method correctly returns three images that are in the same class with similar looks, similar to the results from the influence function. The positive examples excite the activation values for a particular class and supports the decision the model is making. For the negative examples, just like the influence functions, our method returns images that look like the test image but are labeled as a different class. In Figure 4, for the image of a rhino the influence function could not recover useful training points, while ours does, including the similarlooking elephants or zebras which might be confused as rhinos, as negatives. The negative examples work as inhibitory examples for the model – they suppress the activation values for a particular class of a given test point because they are in a different class despite their striking similarity to the test image. Such inhibitory points thus provide a richer understanding, even to machine learning experts, of the behavior of deep neural networks, since they explicitly indicate training points that lead the network away from a particular label for the given test point. More examples can be found in the supplementary material.
4.3 Understanding Misclassified Examples
The representer values can be used to understand the model’s mistake on a test image. Consider a test image of an antelope predicted as a deer in the leftmost panel of Figure 5. Among 181 test images of antelopes, the total number of misclassified instances is 15, among which 12 are misclassified as deer. All of those 12 test images of antelopes had the four training images shown in Figure 5 among the top inhibitory examples. Notice that we can spot antelopes even in the images labeled as zebra or elephant. Such noise in the labels of the training data confuses the model – while the model sees elephant and antelope, the label forces the model to focus on just the elephant. The model thus learns to inhibit the antelope class given an image with small antelopes and other large objects. This insight suggests for instance that we use multilabel prediction to train the network, or perhaps clean the dataset to remove such training examples that would be confusing to humans as well. Interestingly, the model makes the same mistake (predicting deer instead of antelope) on the second training image shown (third from the left of Figure 5), and this suggests that for the training points, we should expect most of the misclassifications to be deer as well. And indeed, among 863 training images of antelopes, 8 are misclassified, and among them 6 are misclassified as deer.
4.4 Sensitivity Map Decomposition
From Theorem 3.1, we have seen that the presoftmax output of the neural network can be decomposed as the weighted sum of the product of the training point feature and the test point feature, or . If we take the gradient with respect to the test input for both sides, we get . Notice that the LHS is the widelyused notion of sensitivity map (gradientbased attribution), and the RHS suggests that we can decompose this sensitivity map into a weighted sum of sensitivity maps that are native to each th training point. This gives us insight into how sensitivities of training points contribute to the sensitivity of the given test image.
In Figure 6, we demonstrate two such examples, one from the class zebra and one from the class moose from the AwA dataset. The first column shows the test images whose sensitivity maps we wish to decompose. For each example, in the following columns we show top four influential representer points in the the top row, and visualize the decomposed sensitivity maps in the bottom. We used SmoothGrad [20] to obtain the sensitivity maps.
For the first example of a zebra, the sensitivity map on the test image mainly focuses on the face of the zebra. This means that infinitesimally changing the pixels around the face of the zebra would cause the greatest change in the neuron output. Notice that the focus on the head of the zebra is distinctively the strongest in the fourth representer point (last column) when the training image manifests clearer facial features compared to other training points. For the rest of the training images that are less demonstrative of the facial features, the decomposed sensitivity maps accordingly show relatively higher focus on the background than on the face. For the second example of a moose, a similar trend can be observed – when the training image exhibits more distinctive bodily features of the moose than the background (first, second, third representer points), the decomposed sensitivity map highlights the portion of the moose on the test image more compared to training images with more features of the background (last representer point). This provides critical insight into the contribution of the representer points towards the neuron output that might not be obvious just from looking at the images itself.
4.5 Computational Cost and Numerical Instabilities
Computation time is particularly an issue for computing the influence function values [10] for a large dataset, which is very costly to compute for each test point. We randomly selected a subset of test points, and report the comparison of the computation time in Table 1
measured on CIFAR10 and AwA datasets. We randomly select 50 test points to compute the values for all train data, and recorded the average and standard deviation of computation time. Note that the influence function does not need the finetuning step when given a pretrained model, hence the values being 0, while our method first optimizes for
using linesearch then computes the representer values. However, note that the finetuning step is a one time cost, while the computation time is spent for every testing image we analyze. Our method significantly outperforms the influence function, and such advantage will favor our method when a larger number of data points is involved. In particular, our approach could be used for realtime explanations of test points, which might be difficult with the influence function approach.Influence Function  Ours  

Dataset  Finetuning  Computation  Finetuning  Computation 
CIFAR10  0  
AwA  0 
Time required for computing an influence function / representer value for all training points and a test point in seconds. The computation of Hessian Vector Products for influence function alone took longer than our combined computation time.
While ranking the training points according to their influence function values, we have observed numerical instabilities, more discussed in the supplementary material. For CIFAR10, over 30 percent of the test images had all zero training point influences, so influence function was unable to provide positive or negative influential examples. The distribution of the values is demonstrated in Figure 7, where we plot the histogram of the maximum of the absolute values for each test point in CIFAR10. Notice that over 300 testing points out of 1,000 lie in the first bin for the influence functions (right). We checked that all data in the first bin had the exact value of 0. Roughly more than 200 points lie in range , the values which may create numerical instabilities in computations. On the other hand, our method (left) returns nontrivial and more numerically stable values across all test points.
5 Conclusion and Discussion
In this work we proposed a novel method of selecting representer points, the training examples that are influential to the model’s prediction. To do so we introduced the modified representer theorem that could be generalized to most deep neural networks, which allows us to linearly decompose the prediction (activation) value into a sum of representer values. The optimization procedure for learning these representer values is tractable and efficient, especially when compared against the influence functions proposed in [10]. We have demonstrated our method’s advantages and performances on several largescale models and image datasets, along with some insights on how these values allow the users to understand the behaviors of the model.
An interesting direction to take from here would be to use the representer values for data poisoning just like in [10]
. Also to truly see if our method is applicable to several domains other than image dataset with different types of neural networks, we plan to extend our method to NLP datasets with recurrent neural networks. The result of a preliminary experiment is included in the supplementary material.
Acknowledgements
We acknowledge the support of DARPA via FA87501720152, and Zest Finance.
References
 Ribeiro et al. [2016] Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. Why should i trust you?: Explaining the predictions of any classifier. In Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pages 1135–1144. ACM, 2016.
 Ribeiro et al. [2018] Marco Tulio Ribeiro, Sameer Singh, and Carlos Guestrin. Anchors: Highprecision modelagnostic explanations. 2018.
 Simonyan et al. [2013] Karen Simonyan, Andrea Vedaldi, and Andrew Zisserman. Deep inside convolutional networks: Visualising image classification models and saliency maps. arXiv preprint arXiv:1312.6034, 2013.
 Shrikumar et al. [2017] Avanti Shrikumar, Peyton Greenside, and Anshul Kundaje. Learning important features through propagating activation differences. arXiv preprint arXiv:1704.02685, 2017.
 Sundararajan et al. [2017] Mukund Sundararajan, Ankur Taly, and Qiqi Yan. Axiomatic attribution for deep networks. arXiv preprint arXiv:1703.01365, 2017.
 Bach et al. [2015] Sebastian Bach, Alexander Binder, Grégoire Montavon, Frederick Klauschen, KlausRobert Müller, and Wojciech Samek. On pixelwise explanations for nonlinear classifier decisions by layerwise relevance propagation. PloS one, 10(7):e0130140, 2015.
 Bien and Tibshirani [2011] Jacob Bien and Robert Tibshirani. Prototype selection for interpretable classification. The Annals of Applied Statistics, pages 2403–2424, 2011.
 Kim et al. [2014] Been Kim, Cynthia Rudin, and Julie A Shah. The bayesian case model: A generative approach for casebased reasoning and prototype classification. In Advances in Neural Information Processing Systems, pages 1952–1960, 2014.
 Kim et al. [2016] Been Kim, Rajiv Khanna, and Oluwasanmi O Koyejo. Examples are not enough, learn to criticize! criticism for interpretability. In Advances in Neural Information Processing Systems, pages 2280–2288, 2016.
 Koh and Liang [2017] Pang Wei Koh and Percy Liang. Understanding blackbox predictions via influence functions. In International Conference on Machine Learning, pages 1885–1894, 2017.
 Anirudh et al. [2017] Rushil Anirudh, Jayaraman J Thiagarajan, Rahul Sridhar, and Timo Bremer. Influential sample selection: A graph signal processing approach. arXiv preprint arXiv:1711.05407, 2017.

Schölkopf et al. [2001]
Bernhard Schölkopf, Ralf Herbrich, and Alex J Smola.
A generalized representer theorem.
In
International conference on computational learning theory
, pages 416–426. Springer, 2001.  Bohn et al. [2017] Bastian Bohn, Michael Griebel, and Christian Rieger. A representer theorem for deep kernel learning. arXiv preprint arXiv:1709.10441, 2017.
 Unser [2018] Michael Unser. A representer theorem for deep neural networks. arXiv preprint arXiv:1802.09210, 2018.
 Krizhevsky and Hinton [2009] Alex Krizhevsky and Geoffrey Hinton. Learning multiple layers of features from tiny images. 2009.
 Simonyan and Zisserman [2014] Karen Simonyan and Andrew Zisserman. Very deep convolutional networks for largescale image recognition. arXiv preprint arXiv:1409.1556, 2014.
 LeCun et al. [1998] Yann LeCun, Léon Bottou, Yoshua Bengio, and Patrick Haffner. Gradientbased learning applied to document recognition. Proceedings of the IEEE, 86(11):2278–2324, 1998.
 Xian et al. [2017] Yongqin Xian, Christoph H Lampert, Bernt Schiele, and Zeynep Akata. Zeroshot learninga comprehensive evaluation of the good, the bad and the ugly. arXiv preprint arXiv:1707.00600, 2017.

He et al. [2016]
Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun.
Deep residual learning for image recognition.
In
Proceedings of the IEEE conference on computer vision and pattern recognition
, pages 770–778, 2016.  Smilkov et al. [2017] Daniel Smilkov, Nikhil Thorat, Been Kim, Fernanda Viégas, and Martin Wattenberg. Smoothgrad: removing noise by adding noise. arXiv preprint arXiv:1706.03825, 2017.

Maas et al. [2011]
Andrew L Maas, Raymond E Daly, Peter T Pham, Dan Huang, Andrew Y Ng, and
Christopher Potts.
Learning word vectors for sentiment analysis.
In Proceedings of the 49th annual meeting of the association for computational linguistics: Human language technologiesvolume 1, pages 142–150. Association for Computational Linguistics, 2011.
Appendix A Proof of Proposition 3.1
Proof.
The convexity can be checked easily. If , by the first order condition we have
(5) 
By chain rule we obtain
(6) 
and thus we have
(7) 
When is not a zero vector, this reduces to
(8) 
and we show that . As a result, is “suitable to” the softmax activation.
If , we can rewrite as
(9) 
and we note that , the th row for , is only related to . Therefore, we have . We now consider the cases where and .
When ,
(10) 
obtains the minimum when , therefore .
When ,
For , the minimum for is 0. For , the minimum for is only if . Therefore, the minimum is reached again when . As a result, is “suitable to” the ReLU activation. ∎
Appendix B Relationship with the Influence Function
In this section, we compare the behaviors of our method and the influence functions [10]. Recall that for a training point and a test point , the influence function value is computed in terms of the gradients/hessians of the loss, and it reflects how the loss at a test point will change when the training point is perturbed (weighted more/less). Because the influence function is defined in terms of the loss, its value can easily become arbitrarily close to zero if the the loss is flat in some region. On the other hand, our representer values are computed using the neurons’ activation values, which may result in comparatively larger values in general. We verify this in a toy dataset in 2D with a large margin shown in Figure 8.
We train a multilayer perceptron with ReLU activations as a binary classifier. The influential points, we would expect, are the points that are closer to the decision boundary. As shown on the left of Figure
8, the influence function does not provide positive or negative examples from each class for the given test point in green square because all training points have exactly zero influence function values, marked with cyan crosses. However, our method provides correct positive and negative points near the decision boundary as we can see from the rightmost panel of Figure 8.In Figure 9, we compare the behaviors of several metrics (Euclidean distance, influence function, representer value) for selecting training points that are most similar to a test point from CIFAR10 dataset. We use a pretrained VGG16. As we can observe from the first two plots, the Euclidean distance does not reflect the class each training point is in – even if the training point is far away from the test point it may still be a similar image in the same class. On the other hand, representer and influence function values tell us about which class each training point is in. From the third plot, we observe that influence function and representer values agree on selecting images of horses as harmful and dogs as helpful.
Appendix C More Examples of Positive/Negative Reperesenter Points
More examples of positive and negative representer points are shown in Figure 10. Observe that the positive points all have the same class label as the test image with high resemblance, while the negative points, despite their similarity to the test image, have different classes.
Appendix D Representer Points of LSTM on NLP Data
We perform a preliminary experiment on an LSTM network trained on a IMDB movie review dataset [21]. Each data point is a review about a movie, and the task is to identify whether the review has a positive or negative sentiment. The pretrain model achieves accuracy. We obtain the positive and negative repesenter points with methods described in section 3.2. In Table 2, we show the test review which is predicted as a negative review by the LSTM network. We observe that both the top1 positive and negative representer contain negative connotations just like the test review, but in the negative representer (which is a positive review about the movie) the negative connotation comes from a character in the movie rather than from a reviewer.
Test Review  <START> when i first saw this movie in the theater i was so angry it completely 
blew in my opinion i didn’t see it for a decade then decided what the hell let’s see  
i’m watching all <> movies now to see where it went wrong my guess is it was with  
sequel 5 that was the first to <> the whole i am in a dream <> i see weird stuff oh <>  
it’s not a dream oh wait i see something spooky oh never mind <> storyline those  
which made it so scary in the first place nothing fantasy nothing weird the box got  
opened boom they came was the only one that could bargain her way out of it first  
Positive Representer  <START> no not the <> of <> the <> <> <> but the mini series <> <> lifetime must 
have realized what a dog this was because the series was burned off two episodes at a  
time most of them broadcast between 11 p m friday nights and 1 a m saturday <> as to  
why i watched the whole thing i can only <> to <> sudden <> attacks of <> br br most  
of the cast are <> who are likely to remain unknown the only two <> names are shirley  
jones and rachel ward who turn in the only decent performances jones doesn’t make it  
through the entire series lucky woman ward by the way is aging quite well since her  
Negative Representer  <START> this time around <> is no longer royal or even particularly close to being any 
such thing instead rather a butler to the prince <> portrayed by hugh <> who <> tim <>  
who presence is <> missed and that hole is never filled his character had an innocent charm  
was a bumbling and complete <> we can’t help but care for him which isn’t at all true of his  
<> as being <> which he apparently was according to the <> page not to mention loud  
<> and utterly non threatening <> can now do just about what he <> and does so why is  
he so frustrated and angry honestly it gets depressing at times yes his master is a <> they 
Comments
There are no comments yet.