HyperSeg: Patch-wise Hypernetwork for Real-time Semantic Segmentation

12/21/2020
by   Yuval Nirkin, et al.
14

We present a novel, real-time, semantic segmentation network in which the encoder both encodes and generates the parameters (weights) of the decoder. Furthermore, to allow maximal adaptivity, the weights at each decoder block vary spatially. For this purpose, we design a new type of hypernetwork, composed of a nested U-Net for drawing higher level context features, a multi-headed weight generating module which generates the weights of each block in the decoder immediately before they are consumed, for efficient memory utilization, and a primary network that is composed of novel dynamic patch-wise convolutions. Despite the usage of less-conventional blocks, our architecture obtains real-time performance. In terms of the runtime vs. accuracy trade-off, we surpass state of the art (SotA) results on popular semantic segmentation benchmarks: PASCAL VOC 2012 (val. set) and real-time semantic segmentation on Cityscapes, and CamVid.

READ FULL TEXT

page 3

page 7

page 13

page 14

research
05/01/2023

PRSeg: A Lightweight Patch Rotate MLP Decoder for Semantic Segmentation

The lightweight MLP-based decoder has become increasingly promising for ...
research
06/22/2018

Efficient Semantic Segmentation using Gradual Grouping

Deep CNNs for semantic segmentation have high memory and run time requir...
research
11/27/2018

ShelfNet for Real-time Semantic Segmentation

In this project, we present ShelfNet, a lightweight convolutional neural...
research
02/17/2022

Shift-Memory Network for Temporal Scene Segmentation

Semantic segmentation has achieved great accuracy in understanding spati...
research
04/16/2021

A De-raining semantic segmentation network for real-time foreground segmentation

Few researches have been proposed specifically for real-time semantic se...
research
12/28/2022

Efficient Semantic Segmentation on Edge Devices

Semantic segmentation works on the computer vision algorithm for assigni...
research
12/03/2019

RGPNet: A Real-Time General Purpose Semantic Segmentation

We propose a novel real-time general purpose semantic segmentation archi...

Please sign up or login with your details

Forgot password? Click here to reset