b'Linjie Li'

research

∙ 06/15/2022

Coarse-to-Fine Vision-Language Pre-training with Fusion in the Backbone

Vision-language (VL) pre-training has recently received considerable att...

Zi-Yi Dou, et al. ∙

research

∙ 06/14/2022

LAVENDER: Unifying Video-Language Understanding as Masked Language Modeling

Unified vision-language frameworks have greatly advanced in recent years...

Linjie Li, et al. ∙

research

∙ 05/27/2022

GIT: A Generative Image-to-text Transformer for Vision and Language

In this paper, we design and train a Generative Image-to-text Transforme...

Jianfeng Wang, et al. ∙

research

∙ 05/03/2022

Cross-modal Representation Learning for Zero-shot Action Recognition

We present a cross-modal Transformer-based framework, which jointly enco...

Chung-Ching Lin, et al. ∙

research

∙ 12/08/2021

MLP Architectures for Vision-and-Language Modeling: An Empirical Study

We initiate the first empirical study on the use of MLP architectures fo...

Yixin Nie, et al. ∙

research

∙ 11/25/2021

SwinBERT: End-to-End Transformers with Sparse Attention for Video Captioning

The canonical approach to video captioning dictates a caption generation...

Kevin Lin, et al. ∙

research

∙ 11/24/2021

VIOLET : End-to-End Video-Language Transformers with Masked Visual-token Modeling

A great challenge in video-language (VidL) modeling lies in the disconne...

Tsu-Jui Fu, et al. ∙

research

∙ 06/08/2021

VALUE: A Multi-Task Benchmark for Video-and-Language Understanding Evaluation

Most existing video-and-language (VidL) research focuses on a single dat...

Linjie Li, et al. ∙

research

∙ 06/01/2021

Adversarial VQA: A New Benchmark for Evaluating the Robustness of VQA Models

With large-scale pre-training, the past two years have witnessed signifi...

Linjie Li, et al. ∙

research

∙ 04/23/2021

Playing Lottery Tickets with Vision and Language

Large-scale transformer-based pre-training has recently revolutionized v...

Zhe Gan, et al. ∙

research

∙ 04/01/2021

UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-training

Vision-and-language pre-training has achieved impressive success in lear...

Mingyang Zhou, et al. ∙

research

∙ 03/16/2021

LightningDOT: Pre-training Visual-Semantic Embeddings for Real-Time Image-Text Retrieval

Multimodal pre-training has propelled great advancement in vision-and-la...

Siqi Sun, et al. ∙

research

∙ 02/11/2021

Less is More: ClipBERT for Video-and-Language Learning via Sparse Sampling

The canonical approach to video-and-language learning (e.g., video quest...

Jie Lei, et al. ∙

research

∙ 12/15/2020

A Closer Look at the Robustness of Vision-and-Language Pre-trained Models

Large-scale pre-trained multimodal transformers, such as ViLBERT and UNI...

Linjie Li, et al. ∙

research

∙ 06/26/2020

Graph Optimal Transport for Cross-Domain Alignment

Cross-domain alignment between two sets of entities (e.g., objects in an...

Liqun Chen, et al. ∙

research

∙ 06/11/2020

Large-Scale Adversarial Training for Vision-and-Language Representation Learning

We present VILLA, the first known effort on large-scale adversarial trai...

Zhe Gan, et al. ∙

research

∙ 05/01/2020

HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training

We present HERO, a Hierarchical EncodeR for Omni-representation learning...

Linjie Li, et al. ∙

research

∙ 10/08/2019

Meta Module Network for Compositional Visual Reasoning

There are two main lines of research on visual reasoning: neural module ...

Wenhu Chen, et al. ∙

research

∙ 09/25/2019

UNITER: Learning UNiversal Image-TExt Representations

Joint image-text embedding is the bedrock for most Vision-and-Language (...

Yen-Chun Chen, et al. ∙

research

∙ 03/29/2019

Relation-aware Graph Attention Network for Visual Question Answering

In order to answer semantically-complicated questions about an image, a ...

Linjie Li, et al. ∙

research

∙ 02/01/2019

Multi-step Reasoning via Recurrent Dual Attention for Visual Dialog

This paper presents Recurrent Dual Attention Network (ReDAN) for visual ...

Zhe Gan, et al. ∙

research

∙ 05/05/2017

Learning to see people like people

Humans make complex inferences on faces, ranging from objective properti...

Amanda Song, et al. ∙

Linjie Li

Featured Co-authors

Jianfeng Gao

Lawrence Carin

Mohit Bansal

William Yang Wang

Yu Cheng

Zhou Yu

Yann LeCun

Zhe Gan

Nanyun Peng

Jingjing Liu

Tianlong Chen

Coarse-to-Fine Vision-Language Pre-training with Fusion in the Backbone

LAVENDER: Unifying Video-Language Understanding as Masked Language Modeling

GIT: A Generative Image-to-text Transformer for Vision and Language

Cross-modal Representation Learning for Zero-shot Action Recognition

MLP Architectures for Vision-and-Language Modeling: An Empirical Study

SwinBERT: End-to-End Transformers with Sparse Attention for Video Captioning

VIOLET : End-to-End Video-Language Transformers with Masked Visual-token Modeling

VALUE: A Multi-Task Benchmark for Video-and-Language Understanding Evaluation

Adversarial VQA: A New Benchmark for Evaluating the Robustness of VQA Models

Playing Lottery Tickets with Vision and Language

UC2: Universal Cross-lingual Cross-modal Vision-and-Language Pre-training

LightningDOT: Pre-training Visual-Semantic Embeddings for Real-Time Image-Text Retrieval

Less is More: ClipBERT for Video-and-Language Learning via Sparse Sampling

A Closer Look at the Robustness of Vision-and-Language Pre-trained Models

Graph Optimal Transport for Cross-Domain Alignment

Large-Scale Adversarial Training for Vision-and-Language Representation Learning

HERO: Hierarchical Encoder for Video+Language Omni-representation Pre-training

Meta Module Network for Compositional Visual Reasoning

UNITER: Learning UNiversal Image-TExt Representations

Relation-aware Graph Attention Network for Visual Question Answering

Multi-step Reasoning via Recurrent Dual Attention for Visual Dialog

Learning to see people like people