Mounting Video Metadata on Transformer-based Language Model for Open-ended Video Question Answering

by   Donggeon Lee, et al.
Seoul National University

Video question answering has recently received a lot of attention from multimodal video researchers. Most video question answering datasets are usually in the form of multiple-choice. But, the model for the multiple-choice task does not infer the answer. Rather it compares the answer candidates for picking the correct answer. Furthermore, it makes it difficult to extend to other tasks. In this paper, we challenge the existing multiple-choice video question answering by changing it to open-ended video question answering. To tackle open-ended question answering, we use the pretrained GPT2 model. The model is fine-tuned with video inputs and subtitles. An ablation study is performed by changing the existing DramaQA dataset to an open-ended question answering, and it shows that performance can be improved using video metadata.


Video Question Answering via Attribute-Augmented Attention Network Learning

Video Question Answering is a challenging problem in visual information ...

Extracting Psychological Indicators Using Question Answering

In this work, we propose a method for extracting text spans that may ind...

MCQA: Multimodal Co-attention Based Network for Question Answering

We present MCQA, a learning-based algorithm for multimodal question answ...

Context-guided Triple Matching for Multiple Choice Question Answering

The task of multiple choice question answering (MCQA) refers to identify...

A Few More Examples May Be Worth Billions of Parameters

We investigate the dynamics of increasing the number of model parameters...

Does Circuit Analysis Interpretability Scale? Evidence from Multiple Choice Capabilities in Chinchilla

Circuit analysis is a promising technique for understanding the internal...

PiggyBack: Pretrained Visual Question Answering Environment for Backing up Non-deep Learning Professionals

We propose a PiggyBack, a Visual Question Answering platform that allows...

Please sign up or login with your details

Forgot password? Click here to reset