InfoSeg: Unsupervised Semantic Image Segmentation with Mutual Information Maximization

10/07/2021
by   Robert Harb, et al.
0

We propose a novel method for unsupervised semantic image segmentation based on mutual information maximization between local and global high-level image features. The core idea of our work is to leverage recent progress in self-supervised image representation learning. Representation learning methods compute a single high-level feature capturing an entire image. In contrast, we compute multiple high-level features, each capturing image segments of one particular semantic class. To this end, we propose a novel two-step learning procedure comprising a segmentation and a mutual information maximization step. In the first step, we segment images based on local and global features. In the second step, we maximize the mutual information between local features and high-level features of their respective class. For training, we provide solely unlabeled images and start from random network initialization. For quantitative and qualitative evaluation, we use established benchmarks, and COCO-Persons, whereby we introduce the latter in this paper as a challenging novel benchmark. InfoSeg significantly outperforms the current state-of-the-art, e.g., we achieve a relative increase of 26 COCO-Stuff dataset.

READ FULL TEXT

page 2

page 6

page 7

page 12

research
03/08/2021

Multimodal Representation Learning via Maximization of Local Mutual Information

We propose and demonstrate a representation learning approach by maximiz...
research
07/16/2020

Autoregressive Unsupervised Image Segmentation

In this work, we propose a new unsupervised image segmentation approach ...
research
04/11/2018

Unsupervised Pathology Image Segmentation Using Representation Learning with Spherical K-means

This paper presents a novel method for unsupervised segmentation of path...
research
07/01/2021

Unsupervised Image Segmentation by Mutual Information Maximization and Adversarial Regularization

Semantic segmentation is one of the basic, yet essential scene understan...
research
10/18/2019

A Mutual Information Maximization Perspective of Language Representation Learning

We show state-of-the-art word representation learning methods maximize a...
research
03/13/2020

Mutual Information Maximization for Effective Lip Reading

Lip reading has received an increasing research interest in recent years...
research
05/02/2023

Long-Tailed Recognition by Mutual Information Maximization between Latent Features and Ground-Truth Labels

Although contrastive learning methods have shown prevailing performance ...

Please sign up or login with your details

Forgot password? Click here to reset