research
∙
05/23/2023
Perception Test: A Diagnostic Benchmark for Multimodal Video Models
We propose a novel multimodal video benchmark - the Perception Test - to...
share
research
∙
05/03/2023
Making the Most of What You Have: Adapting Pre-trained Visual Language Models in the Low-data Regime
Large-scale visual language models are widely used as pre-trained models...
research
∙
02/27/2023
Vid2Seq: Large-Scale Pretraining of a Visual Language Model for Dense Video Captioning
In this work, we introduce Vid2Seq, a multi-modal single-stage dense eve...
research
∙
01/23/2023
Zorro: the masked multimodal transformer
Attention-based models are appealing for multimodal processing because i...
research
∙
11/24/2022
Multi-Task Learning of Object State Changes from Uncurated Videos
We aim to learn to temporally localize object state changes and the corr...
research
∙
06/16/2022
Zero-Shot Video Question Answering via Frozen Bidirectional Language Models
Video question answering (VideoQA) is a complex task that requires diver...
research
∙
05/10/2022
Learning to Answer Visual Questions from Web Videos
Recent methods for visual question answering rely on large-scale annotat...
research
∙
04/29/2022
Flamingo: a Visual Language Model for Few-Shot Learning
Building models that can be rapidly adapted to numerous tasks using only...
research
∙
03/30/2022
TubeDETR: Spatio-Temporal Video Grounding with Transformers
We consider the problem of localizing a spatio-temporal tube in a video ...
research
∙
03/22/2022
Look for the Change: Learning Object States and State-Modifying Actions from Untrimmed Web Videos
Human actions often induce changes of object states such as "cutting an ...
research
∙
03/30/2021
Thinking Fast and Slow: Efficient Text-to-Visual Retrieval with Transformers
Our objective is language-based search of large-scale image and video da...
research
∙
12/01/2020
Just Ask: Learning to Answer Questions from Millions of Narrated Videos
Modern approaches to visual question answering require large annotated d...
research
∙
08/03/2020
RareAct: A video dataset of unusual interactions
This paper introduces a manually annotated video dataset of unusual acti...
research
∙
12/13/2019
End-to-End Learning of Visual Representations from Uncurated Instructional Videos
Annotating videos is cumbersome, expensive and not scalable. Yet, many s...
research
∙
06/07/2019
HowTo100M: Learning a Text-Video Embedding by Watching Hundred Million Narrated Video Clips
Learning text-video embeddings usually requires a dataset of video clips...
research
∙
04/07/2018
Learning a Text-Video Embedding from Incomplete and Heterogeneous Data
Joint understanding of video and language is an active research area wit...
research
∙
07/27/2017
Learning from Video and Text via Large-Scale Discriminative Clustering
Discriminative clustering has been successfully applied to a number of w...
research
∙
06/21/2017