Towards Interpretable Image Synthesis by Learning Sparsely Connected AND-OR Networks

09/10/2019
by   Xianglei Xing, et al.
18

This paper proposes interpretable image synthesis by learning hierarchical AND-OR networks of sparsely connected semantically meaningful nodes. The proposed method is based on the compositionality and interpretability of scene-objects-parts-subparts-primitives hierarchy in image representation. A scene has different types (i.e., OR) each of which consists of a number of objects (i.e., AND). This can be recursively formulated across the scene-objects-parts-subparts hierarchy and is terminated at the primitive level (e.g., Gabor wavelets-like basis). To realize this interpretable AND-OR hierarchy in image synthesis, the proposed method consists of two components: (i) Each layer of the hierarchy is represented by an over-completed set of basis functions. The basis functions are instantiated using convolution to be translation covariant. Off-the-shelf convolutional neural architectures are then exploited to implement the hierarchy. (ii) Sparsity-inducing constraints are introduced in end-to-end training, which facilitate a sparsely connected AND-OR network to emerge from initially densely connected convolutional neural networks. A straightforward sparsity-inducing constraint is utilized, that is to only allow the top-k basis functions to be active at each layer (where k is a hyperparameter). The learned basis functions are also capable of image reconstruction to explain away input images. In experiments, the proposed method is tested on five benchmark datasets. The results show that meaningful and interpretable hierarchical representations are learned with better qualities of image synthesis and reconstruction obtained than state-of-the-art baselines.

READ FULL TEXT

page 2

page 6

page 7

page 8

page 9

research
10/02/2017

Interpretable Convolutional Neural Networks

This paper proposes a method to modify traditional convolutional neural ...
research
01/20/2019

Inducing Sparse Coding and And-Or Grammar from Generator Network

We introduce an explainable generative model by applying sparse operatio...
research
08/20/2018

Class2Str: End to End Latent Hierarchy Learning

Deep neural networks for image classification typically consists of a co...
research
01/22/2017

Greedy Compositional Clustering for Unsupervised Learning of Hierarchical Compositional Models

This paper proposes to integrate a feature pursuit learning process into...
research
02/21/2020

Face Phylogeny Tree Using Basis Functions

Photometric transformations, such as brightness and contrast adjustment,...
research
11/14/2016

Growing Interpretable Part Graphs on ConvNets via Multi-Shot Learning

This paper proposes a learning strategy that extracts object-part concep...

Please sign up or login with your details

Forgot password? Click here to reset