Log In Sign Up

Annotation Cleaning for the MSR-Video to Text Dataset

by   Haoran Chen, et al.

The video captioning task is to describe the video contents with natural language by the machine. Many methods have been proposed for solving this task. A large dataset called MSR Video to Text (MSR-VTT) is often used as the benckmark dataset for testing the performance of the methods. However, we found that the human annotations, i.e., the descriptions of video contents in the dataset are quite noisy, e.g., there are many duplicate captions and many captions contain grammatical problems. These problems may pose difficulties to video captioning models for learning. We cleaned the MSR-VTT annotations by removing these problems, then tested several typical video captioning models on the cleaned dataset. Experimental results showed that data cleaning boosted the performances of the models measured by popular quantitative metrics. We recruited subjects to evaluate the results of a model trained on the original and cleaned datasets. The human behavior experiment demonstrated that trained on the cleaned dataset, the model generated captions that were more coherent and more relevant to contents of the video clips. The cleaned dataset is publicly available.


page 1

page 2

page 4

page 6


BERTHA: Video Captioning Evaluation Via Transfer-Learned Human Assessment

Evaluating video captioning systems is a challenging task as there are m...

Semantically Sensible Video Captioning (SSVC)

Video captioning, i.e. the task of generating captions from video sequen...

Enriching Video Captions With Contextual Text

Understanding video content and generating caption with context is an im...

A Semantics-Assisted Video Captioning Model Trained with Scheduled Sampling

Given the features of a video, recurrent neural network can be used to a...

M-VAD Names: a Dataset for Video Captioning with Naming

Current movie captioning architectures are not capable of mentioning cha...

Automatic Generation of Descriptive Titles for Video Clips Using Deep Learning

Over the last decade, the use of Deep Learning in many applications prod...

Taking an Emotional Look at Video Paragraph Captioning

Translating visual data into natural language is essential for machines ...