Conditional Random Fields as Recurrent Neural Networks

02/11/2015
by   Shuai Zheng, et al.
0

Pixel-level labelling tasks, such as semantic segmentation, play a central role in image understanding. Recent approaches have attempted to harness the capabilities of deep learning techniques for image recognition to tackle pixel-level labelling tasks. One central issue in this methodology is the limited capacity of deep learning techniques to delineate visual objects. To solve this problem, we introduce a new form of convolutional neural network that combines the strengths of Convolutional Neural Networks (CNNs) and Conditional Random Fields (CRFs)-based probabilistic graphical modelling. To this end, we formulate mean-field approximate inference for the Conditional Random Fields with Gaussian pairwise potentials as Recurrent Neural Networks. This network, called CRF-RNN, is then plugged in as a part of a CNN to obtain a deep network that has desirable properties of both CNNs and CRFs. Importantly, our system fully integrates CRF modelling with CNNs, making it possible to train the whole deep network end-to-end with the usual back-propagation algorithm, avoiding offline post-processing methods for object delineation. We apply the proposed method to the problem of semantic image segmentation, obtaining top results on the challenging Pascal VOC 2012 segmentation benchmark.

READ FULL TEXT

page 7

page 9

page 13

page 14

page 15

page 16

page 17

research
05/03/2017

Amortized Inference and Learning in Latent Conditional Random Fields for Weakly-Supervised Semantic Image Segmentation

Conditional random fields (CRFs) are commonly employed as a post-process...
research
03/09/2017

End-to-end semantic face segmentation with conditional random fields as convolutional, recurrent and adversarial networks

Recent years have seen a sharp increase in the number of related yet dis...
research
12/06/2019

End-to-end Training of CNN-CRF via Differentiable Dual-Decomposition

Modern computer vision (CV) is often based on convolutional neural netwo...
research
05/12/2018

Convolutional CRFs for Semantic Segmentation

For the challenging semantic image segmentation task the most efficient ...
research
03/14/2017

A fully end-to-end deep learning approach for real-time simultaneous 3D reconstruction and material recognition

This paper addresses the problem of simultaneous 3D reconstruction and m...
research
12/11/2019

Bipartite Conditional Random Fields for Panoptic Segmentation

We tackle the panoptic segmentation problem with a conditional random fi...
research
03/08/2020

A Multi-scale CNN-CRF Framework for Environmental Microorganism Image Segmentation

In order to assist researchers to identify Environmental Microorganisms ...

Please sign up or login with your details

Forgot password? Click here to reset