b'Shih-Fu Chang'

research

∙ 06/14/2022

Multimodal Event Graphs: Towards Event Centric Understanding of Multimodal World

Understanding how events described or shown in multimedia content relate...

Hammad A. Ayyubi, et al. ∙

research

∙ 06/05/2022

Towards Fast Adaptation of Pretrained Contrastive Models for Multi-channel Video-Language Retrieval

Multi-channel video-language retrieval require models to understand info...

Xudong Lin, et al. ∙

research

∙ 05/22/2022

Language Models with Image Descriptors are Strong Few-Shot Video-Language Learners

The goal of this work is to build flexible video-language models that ca...

Zhenhailong Wang, et al. ∙

research

∙ 04/22/2022

Multimodal Adaptive Distillation for Leveraging Unimodal Encoders for Vision-Language Tasks

Cross-modal encoders for vision-language (VL) tasks are often pretrained...

Zhecan Wang, et al. ∙

research

∙ 04/16/2022

Multimodal Few-Shot Object Detection with Meta-Learning Based Cross-Modal Prompting

We study multimodal few-shot object detection (FSOD) in this paper, usin...

Guangxing Han, et al. ∙

research

∙ 03/29/2022

Fine-Grained Visual Entailment

Visual entailment is a recently proposed multimodal reasoning task where...

Christopher Thomas, et al. ∙

research

∙ 03/28/2022

Few-Shot Object Detection with Fully Cross-Transformer

Few-shot object detection (FSOD), with the aim to detect novel objects u...

Guangxing Han, et al. ∙

research

∙ 01/26/2022

Learning To Recognize Procedural Activities with Distant Supervision

In this paper we consider the problem of classifying fine-grained, multi...

Xudong Lin, et al. ∙

research

∙ 01/15/2022

CLIP-TD: CLIP Targeted Distillation for Vision-Language Tasks

Contrastive language-image pretraining (CLIP) links vision and language ...

Zhecan Wang, et al. ∙

research

∙ 01/13/2022

CLIP-Event: Connecting Text and Images with Event Structures

Vision-language (V+L) pretraining models have achieved great success in ...

Manling Li, et al. ∙

research

∙ 12/20/2021

MuMuQA: Multimedia Multi-Hop News Question Answering via Cross-Media Knowledge Extraction and Grounding

Recently, there has been an increasing interest in building question ans...

Revanth Gangi Reddy, et al. ∙

research

∙ 12/17/2021

Query Adaptive Few-Shot Object Detection with Heterogeneous Graph Convolutional Networks

Few-shot object detection (FSOD) aims to detect never-seen objects using...

Guangxing Han, et al. ∙

research

∙ 12/16/2021

SGEITL: Scene Graph Enhanced Image-Text Learning for Visual Commonsense Reasoning

Answering complex questions about images is an ambitious goal for machin...

Zhecan Wang, et al. ∙

research

∙ 12/01/2021

PreViTS: Contrastive Pretraining with Video Tracking Supervision

Videos are a rich source for self-supervised learning (SSL) of visual re...

Brian Chen, et al. ∙

research

∙ 09/27/2021

Joint Multimedia Event Extraction from Video and Article

Visual and textual modalities contribute complementary information about...

Brian Chen, et al. ∙

research

∙ 04/26/2021

Multimodal Clustering Networks for Self-supervised Learning from Unlabeled Videos

Multimodal self-supervised learning is getting more and more attention a...

Brian Chen, et al. ∙

research

∙ 04/22/2021

VATT: Transformers for Multimodal Self-Supervised Learning from Raw Video, Audio and Text

We present a framework for learning multimodal representations from unla...

Hassan Akbari, et al. ∙

research

∙ 04/15/2021

Meta Faster R-CNN: Towards Accurate Few-Shot Object Detection with Attentive Feature Alignment

Few-shot object detection (FSOD) aims to detect objects using only few e...

Guangxing Han, et al. ∙

research

∙ 03/23/2021

Co-Grounding Networks with Semantic Attention for Referring Expression Comprehension in Videos

In this paper, we address the problem of referring expression comprehens...

Sijie Song, et al. ∙

research

∙ 01/28/2021

VX2TEXT: End-to-End Learning of Video-Based Text Generation From Multimodal Inputs

We present Vx2Text, a framework for text generation from multimodal inpu...

Xudong Lin, et al. ∙

research

∙ 12/24/2020

Task-Adaptive Negative Class Envision for Few-Shot Open-Set Recognition

Recent works seek to endow recognition systems with the ability to handl...

Shiyuan Huang, et al. ∙

research

∙ 11/20/2020

Open-Vocabulary Object Detection Using Captions

Despite the remarkable accuracy of deep neural networks in object detect...

Alireza Zareian, et al. ∙

research

∙ 11/18/2020

Neuro-Symbolic Representations for Video Captioning: A Case for Leveraging Inductive Biases for Vision and Language

Neuro-symbolic representations have proved effective in learning structu...

Hassan Akbari, et al. ∙

research

∙ 10/24/2020

Weakly-supervised VisualBERT: Pre-training without Parallel Images and Captions

Pre-trained contextual vision-and-language (V L) models have brought i...

Liunian Harold Li, et al. ∙

research

∙ 10/09/2020

Uncertainty-Aware Few-Shot Image Classification

Few-shot image classification aims to learn to recognize new categories ...

Zhizheng Zhang, et al. ∙

research

∙ 09/03/2020

Ref-NMS: Breaking Proposal Bottlenecks in Two-Stage Referring Expression Grounding

The prevailing framework for solving referring expression grounding is b...

Long Chen, et al. ∙

research

∙ 07/01/2020

COVID-19 Literature Knowledge Graph Construction and Drug Repurposing Report Generation

To combat COVID-19, both clinicians and scientists need to digest the va...

Qingyun Wang, et al. ∙

research

∙ 06/17/2020

Learning Visual Commonsense for Robust Scene Graph Generation

Scene graph generation models understand the scene through object and pr...

Alireza Zareian, et al. ∙

research

∙ 05/19/2020

Deep Learning Guided Building Reconstruction from Satellite Imagery-derived Point Clouds

3D urban reconstruction of buildings from remotely sensed imagery has dr...

Bo Xu, et al. ∙

research

∙ 05/05/2020

Cross-media Structured Common Space for Multimedia Event Extraction

We introduce a new task, MultiMedia Event Extraction (M2E2), which aims ...

Manling Li, et al. ∙

research

∙ 03/08/2020

Unifying Specialist Image Embedding into Universal Image Embedding

Deep image embedding provides a way to measure the semantic similarity o...

Yang Feng, et al. ∙

research

∙ 02/11/2020

Training with Streaming Annotation

In this paper, we address a practical scenario where training data is re...

Tongtao Zhang, et al. ∙

research

∙ 01/08/2020

Weakly Supervised Visual Semantic Parsing

Scene Graph Generation (SGG) aims to extract entities, predicates and th...

Alireza Zareian, et al. ∙

research

∙ 01/07/2020

Bridging Knowledge Graphs to Generate Scene Graphs

Scene graphs are powerful representations that encode images into their ...

Alireza Zareian, et al. ∙

research

∙ 01/05/2020

General Partial Label Learning via Dual Bipartite Graph Autoencoder

We formulate a practical yet challenging problem: General Partial Label ...

Brian Chen, et al. ∙

research

∙ 12/10/2019

Flow-Distilled IP Two-Stream Networks for Compressed Video Action Recognition

Two-stream networks have achieved great success in video recognition. A ...

Shiyuan Huang, et al. ∙

research

∙ 12/10/2019

Flow-Distilled IP Two-Stream Networks for Compressed Video ActionRecognition

Two-stream networks have achieved great success in video recognition. A ...

Shiyuan Huang, et al. ∙

research

∙ 11/25/2019

Learning to Learn Words from Narrated Video

When we travel, we often encounter new scenarios we have never experienc...

Dídac Surís, et al. ∙

research

∙ 10/24/2019

LPAT: Learning to Predict Adaptive Threshold for Weakly-supervised Temporal Action Localization

Recently, Weakly-supervised Temporal Action Localization (WTAL) has been...

Xudong Lin, et al. ∙

research

∙ 10/12/2019

Context-Gated Convolution

As the basic building block of Convolutional Neural Networks (CNNs), the...

Xudong Lin, et al. ∙

research

∙ 08/06/2019

Report of 2017 NSF Workshop on Multimedia Challenges, Opportunities and Research Roadmaps

With the transformative technologies and the rapidly changing global R ...

Shih-Fu Chang, et al. ∙

research

∙ 07/15/2019

Detecting and Simulating Artifacts in GAN Fake Images

To detect GAN generated images, conventional supervised machine learning...

Xu Zhang, et al. ∙

research

∙ 07/08/2019

Variational Context: Exploiting Visual and Textual Context for Grounding Referring Expressions

We focus on grounding (i.e., localizing or linking) referring expression...

Yulei Niu, et al. ∙

research

∙ 05/23/2019

CDSA: Cross-Dimensional Self-Attention for Multivariate, Geo-tagged Time Series Imputation

Many real-world applications involve multivariate, geo-tagged time serie...

Jiawei Ma, et al. ∙

research

∙ 04/06/2019

Unsupervised Embedding Learning via Invariant and Spreading Instance Feature

This paper studies the unsupervised embedding learning problem, which re...

Mang Ye, et al. ∙

research

∙ 03/04/2019

Unsupervised Rank-Preserving Hashing for Large-Scale Image Retrieval

We propose an unsupervised hashing method which aims to produce binary c...

Svebor Karaman, et al. ∙

research

∙ 01/11/2019

DMC-Net: Generating Discriminative Motion Cues for Fast Compressed Video Action Recognition

Motion has shown to be useful for video understanding, where motion is t...

Zheng Shou, et al. ∙

research

∙ 12/06/2018

Scene Dynamics: Counterfactual Critic Multi-Agent Training for Scene Graph Generation

Scene graphs -- objects as nodes and visual relationships as edges -- de...

Long Chen, et al. ∙

research

∙ 11/28/2018

Multi-level Multimodal Common Semantic Space for Image-Phrase Grounding

We address the problem of phrase grounding by learning a multi-level com...

Hassan Akbari, et al. ∙

research

∙ 11/28/2018

Multi-granularity Generator for Temporal Action Proposal

Temporal action proposal generation is an important task, aiming to loca...

Yuan Liu, et al. ∙

Shih-Fu Chang

Featured Co-authors