Visual Concepts and Compositional Voting

11/13/2017
by   Jianyu Wang, et al.
0

It is very attractive to formulate vision in terms of pattern theory Mumford2010pattern, where patterns are defined hierarchically by compositions of elementary building blocks. But applying pattern theory to real world images is currently less successful than discriminative methods such as deep networks. Deep networks, however, are black-boxes which are hard to interpret and can easily be fooled by adding occluding objects. It is natural to wonder whether by better understanding deep networks we can extract building blocks which can be used to develop pattern theoretic models. This motivates us to study the internal representations of a deep network using vehicle images from the PASCAL3D+ dataset. We use clustering algorithms to study the population activities of the features and extract a set of visual concepts which we show are visually tight and correspond to semantic parts of vehicles. To analyze this we annotate these vehicles by their semantic parts to create a new dataset, VehicleSemanticParts, and evaluate visual concepts as unsupervised part detectors. We show that visual concepts perform fairly well but are outperformed by supervised discriminative methods such as Support Vector Machines (SVM). We next give a more detailed analysis of visual concepts and how they relate to semantic parts. Following this, we use the visual concepts as building blocks for a simple pattern theoretical model, which we call compositional voting. In this model several visual concepts combine to detect semantic parts. We show that this approach is significantly better than discriminative methods like SVM and deep networks trained specifically for semantic part detection. Finally, we return to studying occlusion by creating an annotated dataset with occlusion, called VehicleOcclusion, and show that compositional voting outperforms even deep networks when the amount of occlusion becomes large.

READ FULL TEXT

page 7

page 8

page 9

page 15

page 16

page 23

page 26

page 31

research
07/25/2017

Detecting Semantic Parts on Partially Occluded Objects

In this paper, we address the task of detecting semantic parts on partia...
research
09/14/2017

DeepVoting: An Explainable Framework for Semantic Part Detection under Partial Occlusion

In this paper, we study the task of detecting semantic parts of an objec...
research
03/04/2020

Neural Kernels Without Tangents

We investigate the connections between neural networks and simple buildi...
research
05/23/2019

Hangul Fonts Dataset: a Hierarchical and Compositional Dataset for Interrogating Learned Representations

Interpretable representations of data are useful for testing a hypothesi...
research
07/29/2020

Boardroom Voting: Verifiable Voting with Ballot Privacy Using Low-Tech Cryptography in a Single Room

A boardroom election is an election that takes place in a single room – ...
research
09/22/2016

On the usability of deep networks for object-based image analysis

As computer vision before, remote sensing has been radically changed by ...
research
06/12/2021

Equivariant Networks for Pixelized Spheres

Pixelizations of Platonic solids such as the cube and icosahedron have b...

Please sign up or login with your details

Forgot password? Click here to reset