Fair Latency-Aware Metric for real-time video segmentation networks

by   Evann Courdier, et al.

As supervised semantic segmentation is reaching satisfying results, many recent papers focused on making segmentation network architectures faster, smaller and more efficient. In particular, studies often aim to reach the stage to which they can claim to be "real-time". Achieving this goal is especially relevant in the context of real-time video operations for autonomous vehicles and robots, or medical imaging during surgery. The common metric used for assessing these methods is so far the same as the ones used for image segmentation without time constraint: mean Intersection over Union (mIoU). In this paper, we argue that this metric is not relevant enough for real-time video as it does not take into account the processing time (latency) of the network. We propose a similar but more relevant metric called FLAME for video-segmentation networks, that compares the output segmentation of the network with the ground truth segmentation of the current video frame at the time when the network finishes the processing. We perform experiments to compare a few networks using this metric and propose a simple addition to network training to enhance results according to that metric.



There are no comments yet.


page 7

page 8


GSVNet: Guided Spatially-Varying Convolution for Fast Semantic Segmentation on Video

This paper addresses fast semantic segmentation on video.Video segmentat...

Highway Driving Dataset for Semantic Video Segmentation

Scene understanding is an essential technique in semantic segmentation. ...

Real-time Semantic Image Segmentation via Spatial Sparsity

We propose an approach to semantic (image) segmentation that reduces the...

Fast-SCNN: Fast Semantic Segmentation Network

The encoder-decoder framework is state-of-the-art for offline semantic i...

ShuffleSeg: Real-time Semantic Segmentation Network

Real-time semantic segmentation is of significant importance for mobile ...

Real-time semantic segmentation on FPGAs for autonomous vehicles with hls4ml

In this paper, we investigate how field programmable gate arrays can ser...

Clockwork Convnets for Video Semantic Segmentation

Recent years have seen tremendous progress in still-image segmentation; ...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.