CAR: Class-aware Regularizations for Semantic Segmentation

by   Ye Huang, et al.

Recent segmentation methods, such as OCR and CPNet, utilizing "class level" information in addition to pixel features, have achieved notable success for boosting the accuracy of existing network modules. However, the extracted class-level information was simply concatenated to pixel features, without explicitly being exploited for better pixel representation learning. Moreover, these approaches learn soft class centers based on coarse mask prediction, which is prone to error accumulation. In this paper, aiming to use class level information more effectively, we propose a universal Class-Aware Regularization (CAR) approach to optimize the intra-class variance and inter-class distance during feature learning, motivated by the fact that humans can recognize an object by itself no matter which other objects it appears with. Three novel loss functions are proposed. The first loss function encourages more compact class representations within each class, the second directly maximizes the distance between different class centers, and the third further pushes the distance between inter-class centers and pixels. Furthermore, the class center in our approach is directly generated from ground truth instead of from the error-prone coarse prediction. Our method can be easily applied to most existing segmentation models during training, including OCR and CPNet, and can largely improve their accuracy at no additional inference overhead. Extensive experiments and ablation studies conducted on multiple benchmark datasets demonstrate that the proposed CAR can boost the accuracy of all baseline models by up to 2.23 available at


page 3

page 6

page 13

page 14

page 18

page 19


CARD: Semantic Segmentation with Efficient Class-Aware Regularized Decoder

Semantic segmentation has recently achieved notable advances by exploiti...

Semantic Segmentation via Pixel-to-Center Similarity Calculation

Since the fully convolutional network has achieved great success in sema...

Class-level Multiple Distributions Representation are Necessary for Semantic Segmentation

Existing approaches focus on using class-level features to improve seman...

Inter-Image Communication for Weakly Supervised Localization

Weakly supervised localization aims at finding target object regions usi...

On the Importance of Feature Separability in Predicting Out-Of-Distribution Error

Estimating the generalization performance is practically challenging on ...

Location-aware Upsampling for Semantic Segmentation

Many successful learning targets such as dice loss and cross-entropy los...

Prototype Learning for Out-of-Distribution Polyp Segmentation

Existing polyp segmentation models from colonoscopy images often fail to...

Please sign up or login with your details

Forgot password? Click here to reset