DeepAI AI Chat
Log In Sign Up

REVECA – Rich Encoder-decoder framework for Video Event CAptioner

by   Jaehyuk Heo, et al.
Korea University

We describe an approach used in the Generic Boundary Event Captioning challenge at the Long-Form Video Understanding Workshop held at CVPR 2022. We designed a Rich Encoder-decoder framework for Video Event CAptioner (REVECA) that utilizes spatial and temporal information from the video to generate a caption for the corresponding the event boundary. REVECA uses frame position embedding to incorporate information before and after the event boundary. Furthermore, it employs features extracted using the temporal segment network and temporal-based pairwise difference method to learn temporal information. A semantic segmentation mask for the attentional pooling process is adopted to learn the subject of an event. Finally, LoRA is applied to fine-tune the image encoder to enhance the learning efficiency. REVECA yielded an average score of 50.97 on the Kinetics-GEBC test data, which is an improvement of 10.17 over the baseline method. Our code is available in


Exploiting Context Information for Generic Event Boundary Captioning

Generic Event Boundary Captioning (GEBC) aims to generate three sentence...

Discriminative Latent Semantic Graph for Video Captioning

Video captioning aims to automatically generate natural language sentenc...

Discerning Generic Event Boundaries in Long-Form Wild Videos

Detecting generic, taxonomy-free event boundaries invideos represents a ...

Video + CLIP Baseline for Ego4D Long-term Action Anticipation

In this report, we introduce our adaptation of image-text models for lon...

Small Lesion Segmentation in Brain MRIs with Subpixel Embedding

We present a method to segment MRI scans of the human brain into ischemi...

Masked Autoencoders for Generic Event Boundary Detection CVPR'2022 Kinetics-GEBD Challenge

Generic Event Boundary Detection (GEBD) tasks aim at detecting generic, ...

Winning the CVPR'2021 Kinetics-GEBD Challenge: Contrastive Learning Approach

Generic Event Boundary Detection (GEBD) is a newly introduced task that ...

Code Repositories


Generic Event Boundary Captioning (GEBC) Challenge at LOVEU@CVPR 2022 - 3rd place (REVECA)

view repo