Lightweight Real-time Semantic Segmentation Network with Efficient Transformer and CNN

02/21/2023
by   Guoan Xu, et al.
0

In the past decade, convolutional neural networks (CNNs) have shown prominence for semantic segmentation. Although CNN models have very impressive performance, the ability to capture global representation is still insufficient, which results in suboptimal results. Recently, Transformer achieved huge success in NLP tasks, demonstrating its advantages in modeling long-range dependency. Recently, Transformer has also attracted tremendous attention from computer vision researchers who reformulate the image processing tasks as a sequence-to-sequence prediction but resulted in deteriorating local feature details. In this work, we propose a lightweight real-time semantic segmentation network called LETNet. LETNet combines a U-shaped CNN with Transformer effectively in a capsule embedding style to compensate for respective deficiencies. Meanwhile, the elaborately designed Lightweight Dilated Bottleneck (LDB) module and Feature Enhancement (FE) module cultivate a positive impact on training from scratch simultaneously. Extensive experiments performed on challenging datasets demonstrate that LETNet achieves superior performances in accuracy and efficiency balance. Specifically, It only contains 0.95M parameters and 13.6G FLOPs but yields 72.8% mIoU at 120 FPS on the Cityscapes test set and 70.5% mIoU at 250 FPS on the CamVid test dataset using a single RTX 3090 GPU. The source code will be available at https://github.com/IVIPLab/LETNet.

READ FULL TEXT

page 1

page 3

page 8

page 10

research
04/06/2022

PP-LiteSeg: A Superior Real-Time Semantic Segmentation Model

Real-world applications have high demands for semantic segmentation meth...
research
10/13/2022

RTFormer: Efficient Design for Real-Time Semantic Segmentation with Transformer

Recently, transformer-based networks have shown impressive results in se...
research
01/11/2023

Head-Free Lightweight Semantic Segmentation with Linear Transformer

Existing semantic segmentation works have been mainly focused on designi...
research
05/30/2022

Illumination Adaptive Transformer

Challenging illumination conditions (low light, underexposure and overex...
research
02/17/2023

Model Doctor for Diagnosing and Treating Segmentation Error

Despite the remarkable progress in semantic segmentation tasks with the ...
research
03/26/2022

Semantic Segmentation by Early Region Proxy

Typical vision backbones manipulate structured features. As a compromise...
research
06/08/2020

FibeR-CNN: Expanding Mask R-CNN to Improve Image-Based Fiber Analysis

Fiber-shaped materials (e.g. carbon nano tubes) are of great relevance, ...

Please sign up or login with your details

Forgot password? Click here to reset