On Boosting Semantic Street Scene Segmentation with Weak Supervision

by   Panagiotis Meletis, et al.

Training convolutional networks for semantic segmentation requires per-pixel ground truth labels, which are very time consuming and hence costly to obtain. Therefore, in this work, we research and develop a hierarchical deep network architecture and the corresponding loss for semantic segmentation that can be trained from weak supervision, such as bounding boxes or image level labels, as well as from strong per-pixel supervision. We demonstrate that the hierarchical structure and the simultaneous training on strong (per-pixel) and weak (bounding boxes) labels, even from separate datasets, constantly increases the performance against per-pixel only training. Moreover, we explore the more challenging case of adding weak image-level labels. We collect street scene images and weak labels from the immense Open Images dataset to generate the OpenScapes dataset, and we use this novel dataset to increase segmentation performance on two established per-pixel labeled datasets, Cityscapes and Vistas. We report performance gains up to +13.2 classes, and inference speed of 20 fps on a Titan V GPU for Cityscapes at 512 x 1024 resolution. Our network and OpenScapes dataset are shared with the research community.


page 1

page 2

page 3

page 4

page 6


Training of Convolutional Networks on Multiple Heterogeneous Datasets for Street Scene Semantic Segmentation

We propose a convolutional network with hierarchical classifiers for per...

Data Selection for training Semantic Segmentation CNNs with cross-dataset weak supervision

Training convolutional networks for semantic segmentation with strong (p...

Towards holistic scene understanding: Semantic segmentation and beyond

This dissertation addresses visual scene understanding and enhances segm...

Multi-utility Learning: Structured-output Learning with Multiple Annotation-specific Loss Functions

Structured-output learning is a challenging problem; particularly so bec...

Learning with Weak Annotations for Robust Maritime Obstacle Detection

Robust maritime obstacle detection is crucial for safe navigation of aut...

Learning to detect chest radiographs containing lung nodules using visual attention networks

Machine learning approaches hold great potential for the automated detec...

Please sign up or login with your details

Forgot password? Click here to reset