Log In Sign Up

GMMSeg: Gaussian Mixture based Generative Semantic Segmentation Models

by   Chen Liang, et al.

Prevalent semantic segmentation solutions are, in essence, a dense discriminative classifier of p(class|pixel feature). Though straightforward, this de facto paradigm neglects the underlying data distribution p(pixel feature|class), and struggles to identify out-of-distribution data. Going beyond this, we propose GMMSeg, a new family of segmentation models that rely on a dense generative classifier for the joint distribution p(pixel feature,class). For each class, GMMSeg builds Gaussian Mixture Models (GMMs) via Expectation-Maximization (EM), so as to capture class-conditional densities. Meanwhile, the deep dense representation is end-to-end trained in a discriminative manner, i.e., maximizing p(class|pixel feature). This endows GMMSeg with the strengths of both generative and discriminative models. With a variety of segmentation architectures and backbones, GMMSeg outperforms the discriminative counterparts on three closed-set datasets. More impressively, without any modification, GMMSeg even performs well on open-world datasets. We believe this work brings fundamental insights into the related fields.


page 2

page 8

page 9

page 18

page 19

page 20

page 21


Global and Local Features through Gaussian Mixture Models on Image Semantic Segmentation

The semantic segmentation task aims at dense classification at the pixel...

A Gaussian mixture model representation of endmember variability in hyperspectral unmixing

Hyperspectral unmixing while considering endmember variability is usuall...

KRADA: Known-region-aware Domain Alignment for Open World Semantic Segmentation

In semantic segmentation, we aim to train a pixel-level classifier to as...

Superpixel Segmentation Using Gaussian Mixture Model

Superpixel segmentation algorithms are to partition an image into percep...

Learning a Discriminative Feature Network for Semantic Segmentation

Most existing methods of semantic segmentation still suffer from two asp...

FuSS: Fusing Superpixels for Improved Segmentation Consistency

In this work, we propose two different approaches to improve the semanti...

Discriminative out-of-distribution detection for semantic segmentation

This paper considers dense detection of out-of-distribution pixels. As a...