SFNet: Faster, Accurate, and Domain Agnostic Semantic Segmentation via Semantic Flow

07/10/2022
by   Xiangtai Li, et al.
0

In this paper, we focus on exploring effective methods for faster, accurate, and domain agnostic semantic segmentation. Inspired by the Optical Flow for motion alignment between adjacent video frames, we propose a Flow Alignment Module (FAM) to learn Semantic Flow between feature maps of adjacent levels, and broadcast high-level features to high resolution features effectively and efficiently. Furthermore, integrating our FAM to a common feature pyramid structure exhibits superior performance over other real-time methods even on light-weight backbone networks, such as ResNet-18 and DFNet. Then to further speed up the inference procedure, we also present a novel Gated Dual Flow Alignment Module to directly align high resolution feature maps and low resolution feature maps where we term improved version network as SFNet-Lite. Extensive experiments are conducted on several challenging datasets, where results show the effectiveness of both SFNet and SFNet-Lite. In particular, the proposed SFNet-Lite series achieve 80.1 mIoU while running at 60 FPS using ResNet-18 backbone and 78.8 mIoU while running at 120 FPS using STDC backbone on RTX-3090. Moreover, we unify four challenging driving datasets (i.e., Cityscapes, Mapillary, IDD and BDD) into one large dataset, which we named Unified Driving Segmentation (UDS) dataset. It contains diverse domain and style information. We benchmark several representative works on UDS. Both SFNet and SFNet-Lite still achieve the best speed and accuracy trade-off on UDS which serves as a strong baseline in such a new challenging setting. All the code and models are publicly available at https://github.com/lxtGH/SFSegNets.

READ FULL TEXT

page 3

page 6

page 13

page 14

page 15

page 16

research
02/24/2020

Semantic Flow for Fast and Accurate Scene Parsing

In this paper, we focus on effective methods for fast and accurate scene...
research
03/15/2023

HFGD: High-level Feature Guided Decoder for Semantic Segmentation

Commonly used backbones for semantic segmentation, such as ResNet and Sw...
research
05/25/2021

Fast and Accurate Scene Parsing via Bi-direction Alignment Networks

In this paper, we propose an effective method for fast and accurate scen...
research
03/28/2019

FastFCN: Rethinking Dilated Convolution in the Backbone for Semantic Segmentation

Modern approaches for semantic segmentation usually employ dilated convo...
research
03/08/2022

Stage-Aware Feature Alignment Network for Real-Time Semantic Segmentation of Street Scenes

Over the past few years, deep convolutional neural network-based methods...
research
05/04/2020

How to Train Your Dragon: Tamed Warping Network for Semantic Video Segmentation

Real-time semantic segmentation on high-resolution videos is challenging...
research
09/03/2019

HarDNet: A Low Memory Traffic Network

State-of-the-art neural network architectures such as ResNet, MobileNet,...

Please sign up or login with your details

Forgot password? Click here to reset