research
∙
06/07/2022
Revealing Single Frame Bias for Video-and-Language Learning
Training an effective video-and-language model intuitively requires mult...
share
research
∙
03/10/2022
LoopITR: Combining Dual and Cross Encoder Architectures for Image-Text Retrieval
Dual encoders and cross encoders have been widely used for image-text re...
research
∙
02/15/2022
CommerceMM: Large-Scale Commerce MultiModal Representation Learning with Omni Retrieval
We introduce CommerceMM - a multimodal model capable of providing a dive...
research
∙
07/30/2021
MTVR: Multilingual Moment Retrieval in Videos
We introduce mTVR, a large-scale multilingual video moment retrieval dat...
research
∙
07/20/2021
QVHighlights: Detecting Moments and Highlights in Videos via Natural Language Queries
Detecting customized moments and highlights from videos given natural la...
research
∙
02/11/2021
Less is More: ClipBERT for Video-and-Language Learning via Sparse Sampling
The canonical approach to video-and-language learning (e.g., video quest...
research
∙
10/15/2020
What is More Likely to Happen Next? Video-and-Language Future Event Prediction
Given a video with aligned dialogue, people can often infer what is more...
research
∙
05/11/2020
MART: Memory-Augmented Recurrent Transformer for Coherent Video Paragraph Captioning
Generating multi-sentence descriptions for videos is one of the most cha...
research
∙
01/24/2020
TVR: A Large-Scale Dataset for Video-Subtitle Moment Retrieval
We introduce a new multimodal retrieval task - TV show Retrieval (TVR), ...
research
∙
06/15/2019
IMP: Instance Mask Projection for High Accuracy Semantic Segmentation of Things
In this work, we present a new operator, called Instance Mask Projection...
research
∙
04/25/2019
TVQA+: Spatio-Temporal Grounding for Video Question Answering
We present the task of Spatio-Temporal Video Question Answering, which r...
research
∙
04/09/2019
Multi-Target Embodied Question Answering
Embodied Question Answering (EQA) is a relatively new task where an agen...
research
∙
03/30/2019
Dance Dance Generation: Motion Transfer for Internet Videos
This work presents computational methods for transferring body movements...
research
∙
09/05/2018
TVQA: Localized, Compositional Video Question Answering
Recent years have witnessed an increasing interest in image-based questi...
research
∙
01/27/2018
Image2GIF: Generating Cinemagraphs using Recurrent Deep Q-Networks
Given a still photograph, one can imagine how dynamic objects might move...
research
∙
01/24/2018
MAttNet: Modular Attention Network for Referring Expression Comprehension
In this paper, we address referring expression comprehension: localizing...
research
∙
12/04/2017
Visual to Sound: Generating Natural Sound for Videos in the Wild
As two of the five traditional human senses (sight, hearing, taste, smel...
research
∙
08/09/2017
Hierarchically-Attentive RNN for Album Summarization and Storytelling
We address the problem of end-to-end visual storytelling. Given a photo ...
research
∙
12/30/2016
A Joint Speaker-Listener-Reinforcer Model for Referring Expressions
Referring expressions are natural language constructions used to identif...
research
∙
11/01/2016
Solving Visual Madlibs with Multiple Cues
This paper presents an approach for answering fill-in-the-blank multiple...
research
∙
08/27/2016
Learning Temporal Transformations From Time-Lapse Videos
Based on life-long observations of physical, chemical, and biologic phen...
research
∙
08/12/2016
When was that made?
In this paper, we explore deep learning methods for estimating when obje...
research
∙
07/31/2016
Modeling Context in Referring Expressions
Humans refer to objects in their environments all the time, especially i...
research
∙
05/31/2015