Amortized Inference and Learning in Latent Conditional Random Fields for Weakly-Supervised Semantic Image Segmentation

by   Gaurav Pandey, et al.

Conditional random fields (CRFs) are commonly employed as a post-processing tool for image segmentation tasks. The unary potentials of the CRF are often learnt independently by a classifier, thereby decoupling the inference in CRF from the training of classifier. Such a scheme works effectively, when pixel-level labelling is available for all the images. However, in absence of pixel-level labels, the classifier is faced with the uphill task of selectively assigning the image-level labels to the pixels of the image. Prior work often relied on localization cues, such as saliency maps, objectness priors, bounding boxes etc., to address this challenging problem. In contrast, we model the labels of the pixels as latent variables of a CRF. The pixels and the image-level labels are the observed variables of the latent CRF. We amortize the cost of inference in the latent CRF over the entire dataset, by training an inference network to approximate the posterior distribution of the latent variables given the observed variables. The inference network can be trained in an end-to-end fashion, and requires no localization cues for training. Moreover, unlike other approaches for weakly-supervised segmentation, the proposed model doesn't require further post-processing. The proposed model achieves performance comparable with other approaches that employ saliency masks for the task of weakly-supervised semantic image segmentation on the challenging VOC 2012 dataset.


page 4

page 6


Conditional Random Fields as Recurrent Neural Networks

Pixel-level labelling tasks, such as semantic segmentation, play a centr...

Universal Weakly Supervised Segmentation by Pixel-to-Segment Contrastive Learning

Weakly supervised segmentation requires assigning a label to every pixel...

Saliency guided deep network for weakly-supervised image segmentation

Weakly-supervised image segmentation is an important task in computer vi...

One Weird Trick to Improve Your Semi-Weakly Supervised Semantic Segmentation Model

Semi-weakly supervised semantic segmentation (SWSSS) aims to train a mod...

Tackling the Problem of Limited Data and Annotations in Semantic Segmentation

In this work, the case of semantic segmentation on a small image dataset...

Learning deep structured network for weakly supervised change detection

Conventional change detection methods require a large number of images t...

Convolutional CRFs for Semantic Segmentation

For the challenging semantic image segmentation task the most efficient ...

Please sign up or login with your details

Forgot password? Click here to reset