A tool to visualize feature spaces commonly used in object detection.
We introduce algorithms to visualize feature spaces used by object detectors. Our method works by inverting a visual feature back to multiple natural images. We found that these visualizations allow us to analyze object detection systems in new ways and gain new insight into the detector's failures. For example, when we visualize the features for high scoring false alarms, we discovered that, although they are clearly wrong in image space, they do look deceptively similar to true positives in feature space. This result suggests that many of these false alarms are caused by our choice of feature space, and supports that creating a better learning algorithm or building bigger datasets is unlikely to correct these errors. By visualizing feature spaces, we can gain a more intuitive understanding of recognition systems.READ FULL TEXT VIEW PDF
We introduce algorithms to visualize feature spaces used by object detec...
Object detection is a very important function of visual perception syste...
In this paper, we propose a binarized neural network learning method cal...
Self-supervised learning (SSL) has achieved great success in a variety o...
Deep learning object detectors often return false positives with very hi...
We propose a new feature extraction method based on two dynamical system...
Adverse weather conditions such as rain and haze corrupt the quality of
A tool to visualize feature spaces commonly used in object detection.
A tool to visualize feature spaces commonly used in object detection.
A tool to visualize feature spaces commonly used in object detection.
shows a high scoring detection from an object detector with HOG features and a linear SVM classifier trained on a large database of images.Why does this detector think that sea water looks like a car?
Unfortunately, computer vision researchers are often unable to explain the failures of object detection systems. Some researchers blame the features, others the training set, and even more the learning algorithm. Yet, if we wish to build the next generation of object detectors, it seems crucial to understand the failures of our current detectors.
In this paper, we introduce a tool to explain some of the failures of object detection systems. We present algorithms to visualize the feature spaces of object detectors. Since features are too high dimensional for humans to directly inspect, our visualization algorithms work by inverting features back to natural images. We found that these inversions provide an intuitive visualization of the feature spaces used by object detectors.
Figure 2 shows the output from our visualization algorithm on the features for the false car detection. This visualization reveals that, while there are clearly no cars in the original image, there is a car hiding in the HOG descriptor. HOG features see a slightly different visual world than what we see, and by visualizing this space, we can gain a more intuitive understanding of our object detectors.
Figure 3 inverts more top detections on PASCAL for a few categories. Can you guess which are false alarms? Take a minute to study the figure since the next sentence might ruin the surprise. Although every visualization looks like a true positive, all of these detections are actually false alarms. Consequently, even with a better learning algorithm or more data, these false alarms will likely persist. In other words, the features are responsible for these failures.
The primary contribution of this paper is a general algorithm for visualizing features used in object detection. We present a method that inverts visual features back to images, and show experiments for two standard features in object detection, HOG and activations from CNNs. Since there are many images that can produce equivalent feature descriptors, our method moreover recovers multiple images that are perceptually different in image space, but map to similar feature vectors, illustrated in Figure4.
The remainder of this paper presents and analyzes our visualization algorithm. We first review a growing body of work in feature visualization for both handcrafted features and learned representations. We evaluate our inversions with both automatic benchmarks and a large human study, and we found our visualizations are perceptually more accurate at representing the content of a HOG feature than standard methods; see Figure 5 for a comparison between our visualization and HOG glyphs. We then use our visualizations to inspect the behaviors of object detection systems and analyze their features. Since we hope our visualizations will be useful to other researchers, our final contribution is a public feature visualization toolbox.111Available online at http://mit.edu/hoggles
Our visualization algorithms are part of an actively growing body of work in feature inversion. Oliva and Torralba (2001), in early work, described a simple iterative procedure to recover images given gist descriptors. Weinzaepfel et al (2011) were the first to reconstruct an image given its keypoint SIFT descriptors (Lowe, 1999). Their approach obtains compelling reconstructions using a nearest neighbor based approach on a massive database. d’Angelo et al (2012) then developed an algorithm to reconstruct images given only LBP features (Calonder et al, 2010; Alahi et al, 2012). Their method analytically solves for the inverse image and does not require a dataset. Kato and Harada (2014) posed feature inversion as a jigsaw puzzle problem to invert bags of visual words.
Since visual representations that are learned can be difficult to interpret, there has been recent work to visualize and understand learned features. Zeiler and Fergus (2013)
present a method to visualize activations from a convolutional neural network. In related work,Simonyan et al (2013) visualize class appearance models and their activations for deep networks. Girshick et al (2013) proposed to visualize convolutional neural networks by finding images that activate a specific feature. Mahendran and Vedaldi (2014) describe a general method for inverting visual features from CNNs by incorporating natural image priors.
While these methods are good at reconstructing and visualizing images from their respective features, our visualization algorithms have some advantages. Firstly, while most methods are tailored for specific features, the visualization algorithms we propose are feature independent. Since we cast feature inversion as a machine learning problem, our algorithms can be used to visualize any feature. In this paper, we focus on features for object detection, and we use the same algorithm to invert both HOG and CNN features. Secondly, our algorithms are fast: our best algorithm can invert features in under a second on a desktop computer, enabling interactive visualization, which we believe is important for real-time debugging of vision systems. Finally, our algorithm explicitly optimizes for multiple inversions that are diverse in image space, yet match in feature space.
Our method builds upon work that uses a pair of dictionaries with a coupled representation for super resolution(Yang et al, 2010; Wang et al, 2012) and image synthesis (Huang and Wang, 2013). We extend these methods to show that similar approaches can visualize features as well. Moreover, we incorporate novel terms that encourage diversity in the reconstructed image in order to recover multiple images from a single feature.
Feature visualizations have many applications in computer vision. The computer vision community has been using these visualization largely to understand object recognition systems so as to reveal information encoded by features (Zhang et al, 2014; Sadeghi and Forsyth, 2013), interpret transformations in feature space (Chen and Grauman, 2014), studying diverse images with similar features (Tatu et al, 2011; Lenc and Vedaldi, 2014), find security failures in machine learning systems (Biggio et al, 2012; Weinzaepfel et al, 2011), and fix problems in convolutional neural networks (Zeiler and Fergus, 2013; Simonyan et al, 2013; Bruckner, 2014). With many applications, feature visualizations are an important tool for the computer vision researcher.
Visualizations enable analysis that complement a recent line of papers that provide tools to diagnose object recognition systems, which we briefly review here. Parikh and Zitnick (2011, 2010) introduced a new paradigm for human debugging of object detectors, an idea that we adopt in our experiments. Hoiem et al (2012) performed a large study analyzing the errors that object detectors make. Divvala et al (2012) analyze part-based detectors to determine which components of object detection systems have the most impact on performance. Liu and Wang (2012) designed algorithms to highlight which image regions contribute the most to a classifier’s confidence. Zhu et al (2012) try to determine whether we have reached Bayes risk for HOG. The tools in this paper enable an alternative mode to analyze object detectors through visualizations. By putting on ‘HOG glasses’ and visualizing the world according to the features, we are able to gain a better understanding of the failures and behaviors of our object detection systems.
We now describe our feature inversion method. Let be a natural RGB image and be its corresponding feature descriptor. Since features are many-to-one functions, our goal is to invert the features by recovering a set of images that all map to the original feature descriptor.
We compute this inversion set by solving an optimization problem. We wish to find several that minimize their reconstruction error in feature space while simultaneously appearing diverse in image space. We write this optimization as:
The first term of this objective favors images that match in feature space and the slack variables penalize pairs of images that are too similar to each other in image space where is the similarity cost, parametrized by , between inversions and . A high similarity cost intuitively means that and
look similar and should be penalized. The hyperparametercontrols the strength of the similarity cost. By increasing , the inversions will look more different, at the expense of matching less in feature space.
There are a variety of similarity costs that we could use. In this work, we use costs of the form:
is an affinity matrix. Since we are interested in images that are diverse and not negatives of each other, we square. The identity affinity matrix, i.e. , corresponds to comparing inversions directly in the color space. However, more metrics are also possible, which we describe now.
Edges: We can design to favor inversions that differ in edges. Let where . The first rows of correspond to the convolution with the vertical edge filters and similarly the second rows are for the horizontal edge filters .
Color: We can also encourage the inversions to differ only in colors. Let where is a matrix that averages each color channel such that is the average RGB color.
Spatial: We can force the inversions to only differ in certain spatial regions. Let where is a binary diagonal matrix. A spatial region of will be only encouraged to be diverse if its corresponding element on the diagonal of is . Note we can combine spatial similarity costs with both color and edge costs to encourage color and edge diversity in only certain spatial regions as well.
Unfortunately, optimizing equation 1 efficiently is challenging because it is not convex. Instead, we will make two modifications to solve an approximation:
Modification 1: Since the first term of the objective depends on the feature function , which is often not convex nor differentiable, efficient optimization is difficult. Consequently, we approximate an image and its features with a paired, over-complete basis to make the objective convex. Suppose we represent an image and its feature in a natural image basis and a feature space basis respectively. We can estimate and such that images and features can be encoded in their respective bases but with shared coefficients :
If and have this paired representation, then we can invert features by estimating an that reconstructs the feature well. See Figure 6 for a graphical representation of the paired dictionaries.
Modification 2: However, the objective is still not convex when there are multiple outputs. We approach solving equation 1 sub-optimally using a greedy approach. Suppose we already computed the first inversions, . We then seek the inversion that is only different from the previous inversions, but still matches .
Taking these approximations into account, we solve for the inversion with the optimization:
where there is a sparsity prior on parameterized by .222We found a sparse improves our results. While our method will work when regularizing with instead, it tends to produce more blurred images. After estimating , the inversion is .
The similarity costs can be seen as adding a weighted Tikhonov regularization ( norm) on because
Since this is combined with lasso, the optimization behaves as an elastic net (Zou and Hastie, 2005). Note that if we remove the slack variables (), our method reduces to (Vondrick et al, 2013) and only produces one inversion.
As the similarity costs are in the form of equation 2, we can absorb into the norm of equation 4. This allows us to efficiently optimize equation 4 using an off-the-shelf sparse coding solver. We use SPAMS (Mairal et al, 2009) in our experiments. The optimization typically takes a few seconds to produce each inversion on a desktop computer.
The bases and can be learned such that they have paired coefficients. We first extract millions of image patches and their corresponding features from a large database. Then, we can solve a dictionary learning problem similar to sparse coding, but with paired dictionaries:
for some hyperparameters and . We optimize the above with SPAMS (Mairal et al, 2009). Optimization typically took a few hours, and only needs to be performed once for a fixed feature. See Figure 7 for a visualization of the learned dictionary pairs.
In order to evaluate our method, we also developed several baselines that we use for comparison. We first describe three baselines for single feature inversion, then discuss two baselines for multiple feature inversion.
Consider the top detections for the exemplar object detector (Hariharan et al, 2012; Malisiewicz et al, 2011) for a few images shown in Figure 8. Although all top detections are false positives, notice that each detection captures some statistics about the query. Even though the detections are wrong, if we squint, we can see parts of the original object appear in each detection.
We use this observation to produce our first baseline. Suppose we wish to invert feature . We first train an exemplar LDA detector (Hariharan et al, 2012) for this query, where and are parameters estimated with a large dataset. We then score against every sliding window in this database. The feature inverse is the average of the top detections in RGB space: where is an image of a top detection.
This method, although simple, produces reasonable reconstructions, even when the database does not contain the category of the feature template. However, it is computationally expensive since it requires running an object detector across a large database. Note that a similar nearest neighbor method is used in brain research to visualize what a person might be seeing (Nishimoto et al, 2011).
We describe a fast, parametric inversion baseline based off ridge regression. Let
be a random variable representing a gray scale image and
be a random variable of its corresponding feature. We define these random variables to be normally distributed on a-variate Gaussian with parameters and . In order to invert a feature
, we calculate the most likely image from the conditional Gaussian distribution:
It is well known that a Gaussian distribution have a closed form conditional mode:
Under this inversion algorithm, any feature can be inverted by a single matrix multiplication, allowing for inversion in under a second.
We estimate and on a large database. In practice, is not positive definite; we add a small uniform prior (i.e., ) so can be inverted. Since we wish to invert any feature, we assume that is stationary (Hariharan et al, 2012), allowing us to efficiently learn the covariance across massive datasets. For features with varying spatial dimensions, we invert a feature by marginalizing out unused dimensions.
We now provide a baseline that attempts to find images that, when we compute features on it, sufficiently match the original descriptor. In order to do this efficiently, we only consider images that span a natural image basis. Let be the natural image basis. We found using the first eigenvectors of worked well for this basis. Any image can be encoded by coefficients in this basis: . We wish to minimize:
Empirically we found success optimizing equation 8 using coordinate descent on with random restarts. We use an over-complete basis corresponding to sparse Gabor-like filters for . We compute the eigenvectors of across different scales and translate smaller eigenvectors to form .
In order to compare our ability to recover multiple inversions, we describe two baselines for multiple feature inversions. Our first method modifies paired dictionaries. Rather than incorporating similarity costs, we add noise to a feature to create a slightly different inversion by “nudging” it in random directions:
where is noise from a standard normal distribution such that
is the identity matrix andis a hyperparameter that controls the strength of the diversity.
In addition, we compare against a second baseline that modifies a paired dictionary by removing the basis elements that were activated on previous iterations. Suppose the first inversion activated the first basis elements. We obtain a second inversion by only giving the paired dictionary the other basis elements. This forces the sparse coding to use a disjoint basis set, leading to different inversions.
We evaluate our inversion algorithms using both qualitative and quantitative measures. We use PASCAL VOC 2011 (Everingham et al, 2010) as our dataset and we invert patches corresponding to objects. Any algorithm that required training could only access the training set. During evaluation, only images from the validation set are examined. The database for exemplar LDA excluded the category of the patch we were inverting to reduce the potential effect of dataset biases. Due to their popularity in object detection, we first focus on evaluating HOG features.
We show our inversions in Figure 9 for a few object categories. Exemplar LDA and ridge regression tend to produce blurred visualizations. Direct optimization recovers high frequency details at the expense of extra noise. Paired dictionary learning tends to produce the best visualization for HOG descriptors. By learning a dictionary over the visual world and the correlation between HOG and natural images, paired dictionary learning recovered high frequencies without introducing significant noise.
Although HOG does not explicitly encode color, we found that the paired dictionary is able to recover color from HOG descriptors. Figure 10 shows the result of training a paired dictionary to estimate RGB images instead of grayscale images. While the paired dictionary assigns arbitrary colors to man-made objects and indoor scenes, it frequently colors natural objects correctly, such as grass or the sky, likely because those categories are strongly correlated to HOG descriptors. We focus on grayscale visualizations in this paper because we found those to be more intuitive for humans to understand.
We also explored whether our visualization algorithm could invert other features besides HOG, such as deep features. Figure14 shows how our algorithm can recover some details of the original image given only activations from the last convolutional layer of Krizhevsky et al (2012). Although the visualizations are blurry, they do capture some important visual aspects of the original images such as shapes and colors. This suggests that our visualization algorithm may be general to the type of feature.
While our visualizations do a good job at representing HOG features, they have some limitations. Figure 11 compares our best visualization (paired dictionary) against a greedy algorithm that draws triangles of random rotation, scale, position, and intensity, and only accepts the triangle if it improves the reconstruction. If we allow the greedy algorithm to execute for an extremely long time (a few days), the visualization better shows higher frequency detail. This reveals that there exists a visualization better than paired dictionary learning, although it may not be tractable for large scale experiments. In a related experiment, Figure 12 recursively computes HOG on the inverse and inverts it again. This recursion shows that there is some loss between iterations, although it is minor and appears to discard high frequency details. Moreover, Figure 13 indicates that our inversions are sensitive to the dimensionality of the HOG template. Despite these limitations, our visualizations are, as we will now show, still perceptually intuitive for humans to understand.
We quantitatively evaluate our algorithms under two benchmarks. Firstly, we use an automatic inversion metric that measures how well our inversions reconstruct original images. Secondly, we conducted a large visualization challenge with human subjects on Amazon Mechanical Turk (MTurk), which is designed to determine how well people can infer high level semantics from our visualizations.
Pixel Level Reconstruction: We consider the inversion performance of our algorithm: given a HOG feature , how well does our inverse reconstruct the original pixels for each algorithm? Since HOG is invariant up to a constant shift and scale, we score each inversion against the original image with normalized cross correlation. Our results are shown in Table 1. Overall, exemplar LDA does the best at pixel level reconstruction.
Semantic Reconstruction: While the inversion benchmark evaluates how well the inversions reconstruct the original image, it does not capture the high level content of the inverse: is the inverse of a sheep still a sheep? To evaluate this, we conducted a study on MTurk. We sampled 2,000 windows corresponding to objects in PASCAL VOC 2011. We then showed participants an inversion from one of our algorithms and asked participants to classify it into one of the 20 categories. Each window was shown to three different users. Users were required to pass a training course and qualification exam before participating in order to guarantee users understood the task. Users could optionally select that they were not confident in their answer. We also compared our algorithms against the standard black-and-white HOG glyph popularized by (Dalal and Triggs, 2005).
Our results in Table 2 show that paired dictionary learning and direct optimization provide the best visualization of HOG descriptors for humans. Ridge regression and exemplar LDA perform better than the glyph, but they suffer from blurred inversions. Human performance on the HOG glyph is generally poor, and participants were even the slowest at completing that study. Interestingly, the glyph does the best job at visualizing bicycles, likely due to their unique circular gradients. Our results overall suggest that visualizing HOG with the glyph is misleading, and richer visualizations from our paired dictionary are useful for interpreting HOG features.
Our experiments suggest that humans can predict the performance of object detectors by only looking at HOG visualizations. Human accuracy on inversions and state-of-the-art object detection AP scores from (Felzenszwalb et al, 2010a) are correlated with a Spearman’s rank correlation coefficient of 0.77.
We also asked computer vision PhD students at MIT to classify HOG glyphs in order to compare MTurk participants with experts in HOG. Our results are summarized in the last column of Table 2. HOG experts performed slightly better than non-experts on the glyph challenge, but experts on glyphs did not beat non-experts on other visualizations. This result suggests that our algorithms produce more intuitive visualizations even for object detection researchers.
Since features are many-to-one functions, our visualization algorithms should be able to recover multiple inversions for a feature descriptor. We look at the multiple inversions from deep network features because these features appear to be robust to several invariances.
To conduct our experiments with multiple inversions, we inverted features from the AlexNet convolutional neural network (Krizhevsky et al, 2012)
trained on ImageNet(Deng et al, 2009; Russakovsky et al, 2014)
. We use the publicly available Caffe software package(Jia, 2013) to extract features. We use features from the last convolutional layer (pool5), which has been shown to have strong performance on recognition tasks (Girshick et al, 2013). We trained the dictionaries and using random windows from the PASCAL VOC 2007 training set (Everingham et al, 2010). We tested on two thousand random windows corresponding to objects in the held-out PASCAL VOC 2007 validation set.
We first look at a few qualitative results for our multiple feature inversions. Figure 15 shows a few examples for both our method (top rows) and the baselines (bottom rows). The 1st column shows the result of a paired dictionary on CNN features, while the 2nd and 3rd show the additional inversions that our method finds. While the results are blurred, they do tend to resemble the original image in rough shape and color.
The color affinity in Figure 15a is often able to produce inversions that vary slightly in color. Notice how the cat and the floor are changing slightly in hue, and the grass the bird is standing on is varying slightly. The edge affinity in Figure 15b can occasionally generate inversions with different edges, although the differences can be subtle. To better show the differences with the edge affinity, we visualize a difference matrix in Figure 16. Notice how the edges of the bird and person shift between each inversion.
The baselines tend to either produce nearly identical inversions or inversions that do not match well in feature space. Nudged dictionaries in Figure 15c frequently retrieves inversions that look nearly identical. Subset dictionaries in Figure 15d recovers different inversions, but the inversions do not match in feature space, likely because this baseline operates over a subset of the basis elements.
Although HOG is not as invariant to visual transformations as deep features, we can still recover multiple inversions from a HOG descriptor. The block-wise histograms of HOG allow for gradients in the image to shift up to their bin size without affecting the feature descriptor. Figure 17 shows multiple inversions from a HOG descriptor of a man where the person shifts slightly between each inversion.
We wish to quantify how well our inversions trade off matching in feature space versus having diversity in image space. To evaluate this, we calculated Euclidean distance between the features of the first and second inversions from each method, , and compared it to the Euclidean distance of the inversions in Lab image space, where is the Lab colorspace transformation.333We chose Lab because Euclidean distance in this space is known to be perceptually uniform (Jain, 1989), which we suspect better matches human interpretation. We consider one inversion algorithm to be better than another method if, for the same distance in feature space, the image distance is larger.
We show a scatter plot of this metric in Figure 18 for our method with different similarity costs. The thick lines show the median image distance for a given feature distance. The overall trend suggests that our method produces more diverse images for the same distance in feature space. Setting the affinity matrix to perform color averaging produces the most image variation for CNN features in order to keep the feature space accuracy small. The baselines in general do not perform as well, and baseline with subset dictionaries struggles to even match in feature space, causing the green line to abruptly start in the middle of the plot. The edge affinity produces inversions that tend to be more diverse than baselines, although this effect is best seen qualitatively in the next section.
We consider a second evaluation metric designed to determine how well our inversions match the original features. Since distances in a feature space are unscaled, they can be difficult to interpret, so we use a normalized metric. We calculate the ratio of distances that the inversions make to the original feature:where is the original feature and and are the first and second inversions. A value of implies the second inversion is just as close to as the first. We then compare the ratio to the Lab distance in image space.
We show results for our second metric in Figure 19 as a density map comparing image distance and the ratio of distances in feature space. Black is a higher density and implies that the method produces inversions in that region more frequently. This experiment shows that for the same ratio , our approach tends to produce more diverse inversions when affinity is set to color averaging. Baselines frequently performed poorly, and struggled to generate diverse images that are close in feature space.
While the goal of this paper is to visualize object detection features, in this section we will use our visualizations to inspect the behavior of object detection systems. Due to our budget for experiments, we focus on HOG features.
Our visualizations reveal that the world that features see is slightly different from the world that the human eye perceives. Figure (a)a shows a normal photograph of a man standing in a dark room, but Figure (b)b shows how HOG features see the same man. Since HOG is invariant to illumination changes and amplifies gradients, the background of the scene, normally invisible to the human eye, materializes in our visualization.
In order to understand how this clutter affects object detection, we visualized the features of some of the top false alarms from the Felzenszwalb et al. object detection system (Felzenszwalb et al, 2010b) when applied to the PASCAL VOC 2007 test set. Figure 3 shows our visualizations of the features of the top false alarms. Notice how the false alarms look very similar to true positives. While there are many different types of detector errors, this result suggests that these particular failures are due to limitations of HOG, and consequently, even if we develop better learning algorithms or use larger datasets, these will false alarms will likely persist.
Figure 23 shows the corresponding RGB image patches for the false positives discussed above. Notice how when we view these detections in image space, all of the false alarms are difficult to explain. Why do chair detectors fire on buses, or people detectors on cherries? By visualizing the detections in feature space, we discovered that the learning algorithm made reasonable failures since the features are deceptively similar to true positives.
Although HOG features are designed for machines, how well do humans see in HOG space? If we could quantify human vision on the HOG feature space, we could get insights into the performance of HOG with a perfect learning algorithm (people). Inspired by Parikh and Zitnick’s methodology (Parikh and Zitnick, 2011, 2010), we conducted a large human study where we had Amazon Mechanical Turk participants act as sliding window HOG based object detectors.
We built an online interface for humans to look at HOG visualizations of window patches at the same resolution as DPM. We instructed participants to either classify a HOG visualization as a positive example or a negative example for a category. By averaging over multiple people (we used 25 people per window), we obtain a real value score for a HOG patch. To build our dataset, we sampled top detections from DPM on the PASCAL VOC 2007 dataset for a few categories. Our dataset consisted of around windows per category and around were true positives.
Figure 21 shows precision recall curves for the Human + HOG based object detector. In most cases, human subjects classifying HOG visualizations were able to rank sliding windows with either the same accuracy or better than DPM. Humans tied DPM for recognizing cars, suggesting that performance may be saturated for car detection on HOG. Humans were slightly superior to DPM for chairs, although performance might be nearing saturation soon. There appears to be the most potential for improvement for detecting cats with HOG. Subjects performed slightly worst than DPM for detecting people, but we believe this is the case because humans tend to be good at fabricating people in abstract drawings.
We then repeated the same experiment as above on chairs except we instructed users to classify the original RGB patch instead of the HOG visualization. As expected, humans have near perfect accuracy at detecting chairs with RGB sliding windows. The performance gap between the Human+HOG detector and Human+RGB detector demonstrates the amount of information that HOG features discard.
Our experiments suggest that there is still some performance left to be squeezed out of HOG. However, DPM is likely operating very close to the performance limit of HOG. Since humans are the ideal learning agent and they still had trouble detecting objects in HOG space, HOG may be too lossy of a descriptor for high performance object detection. If we wish to significantly advance the state-of-the-art in recognition, we suspect focusing effort on building better features that capture finer details as well as higher level information will lead to substantial performance improvements in object detection. Indeed, recent advances in object recognition have been driven by learning with richer features (Girshick et al, 2013).
We found our algorithms are also useful for visualizing the learned models of an object detector. Figure 22 visualizes the root templates and the parts from (Felzenszwalb et al, 2010b) by inverting the positive components of the learned weights. These visualizations provide hints on which gradients the learning found discriminative. Notice the detailed structure that emerges from our visualization that is not apparent in the HOG glyph. Often, one can recognize the category of the detector by only looking at the visualizations.
We believe visualizations can be a powerful tool for understanding object detection systems and advancing research in computer vision. To this end, this paper presented and evaluated several algorithms to visualize object detection features. We hope more intuitive visualizations will prove useful for the community.
Acknowledgments: We thank the CSAIL Vision Group for many important discussions. Funding was provided by a NSF GRFP to CV, a Facebook fellowship to AK, and a Google research award, ONR MURI N000141010933 and NSF Career Award No. 0747120 to AT.
Biggio B, Nelson B, Laskov P (2012) Poisoning attacks against support vector machines. ICML