End-to-End Compressed Video Representation Learning for Generic Event Boundary Detection

03/29/2022
by   Congcong Li, et al.
0

Generic event boundary detection aims to localize the generic, taxonomy-free event boundaries that segment videos into chunks. Existing methods typically require video frames to be decoded before feeding into the network, which demands considerable computational power and storage space. To that end, we propose a new end-to-end compressed video representation learning for event boundary detection that leverages the rich information in the compressed domain, i.e., RGB, motion vectors, residuals, and the internal group of pictures (GOP) structure, without fully decoding the video. Specifically, we first use the ConvNets to extract features of the I-frames in the GOPs. After that, a light-weight spatial-channel compressed encoder is designed to compute the feature representations of the P-frames based on the motion vectors, residuals and representations of their dependent I-frames. A temporal contrastive module is proposed to determine the event boundaries of video sequences. To remedy the ambiguities of annotations and speed up the training process, we use the Gaussian kernel to preprocess the ground-truth event boundaries. Extensive experiments conducted on the Kinetics-GEBD dataset demonstrate that the proposed method achieves comparable results to the state-of-the-art methods with 4.5× faster running speed.

READ FULL TEXT
research
06/07/2022

Structured Context Transformer for Generic Event Boundary Detection

Generic Event Boundary Detection (GEBD) aims to detect moments where hum...
research
01/11/2023

Generic Event Boundary Detection in Video with Pyramid Features

Generic event boundary detection (GEBD) aims to split video into chunks ...
research
03/01/2022

Temporal Perceiver: A General Architecture for Arbitrary Boundary Detection

Generic Boundary Detection (GBD) aims at locating general boundaries tha...
research
11/29/2021

UBoCo : Unsupervised Boundary Contrastive Learning for Generic Event Boundary Detection

Generic Event Boundary Detection (GEBD) is a newly suggested video under...
research
11/27/2016

Long-Term Image Boundary Prediction

Boundary estimation in images and videos has been a very active topic of...
research
10/11/2022

Motion Aware Self-Supervision for Generic Event Boundary Detection

The task of Generic Event Boundary Detection (GEBD) aims to detect momen...
research
09/30/2021

CoSeg: Cognitively Inspired Unsupervised Generic Event Segmentation

Some cognitive research has discovered that humans accomplish event segm...

Please sign up or login with your details

Forgot password? Click here to reset