Temporal Memory Attention for Video Semantic Segmentation

02/17/2021
by   Hao Wang, et al.
17

Video semantic segmentation requires to utilize the complex temporal relations between frames of the video sequence. Previous works usually exploit accurate optical flow to leverage the temporal relations, which suffer much from heavy computational cost. In this paper, we propose a Temporal Memory Attention Network (TMANet) to adaptively integrate the long-range temporal relations over the video sequence based on the self-attention mechanism without exhaustive optical flow prediction. Specially, we construct a memory using several past frames to store the temporal information of the current frame. We then propose a temporal memory attention module to capture the relation between the current frame and the memory to enhance the representation of the current frame. Our method achieves new state-of-the-art performances on two challenging video semantic segmentation datasets, particularly 80.3 76.5

READ FULL TEXT

page 1

page 2

research
01/05/2021

Local Memory Attention for Fast Video Semantic Segmentation

We propose a novel neural network module that transforms an existing sin...
research
11/28/2019

Every Frame Counts: Joint Learning of Video Segmentation and Optical Flow

A major challenge for video semantic segmentation is the lack of labeled...
research
01/10/2023

Video Semantic Segmentation with Inter-Frame Feature Fusion and Inner-Frame Feature Refinement

Video semantic segmentation aims to generate accurate semantic maps for ...
research
03/14/2022

Attention based Memory video portrait matting

We proposed a novel trimap free video matting method based on the attent...
research
07/21/2022

Mining Relations among Cross-Frame Affinities for Video Semantic Segmentation

The essence of video semantic segmentation (VSS) is how to leverage temp...
research
02/26/2020

Efficient Semantic Video Segmentation with Per-frame Inference

In semantic segmentation, most existing real-time deep models trained wi...
research
08/04/2020

Temporal Context Aggregation for Video Retrieval with Contrastive Learning

The current research focus on Content-Based Video Retrieval requires hig...

Please sign up or login with your details

Forgot password? Click here to reset