Louis-Philippe Morency

research

∙ 06/28/2023

MultiZoo MultiBench: A Standardized Toolkit for Multimodal Deep Learning

Learning multimodal representations involves integrating information fro...

0 Paul Pu Liang, et al. ∙

research

∙ 06/13/2023

Neural Mixed Effects for Nonlinear Personalized Predictions

Personalized prediction is a machine learning approach that predicts a p...

0 Torsten Wörtwein, et al. ∙

research

∙ 06/08/2023

Factorized Contrastive Learning: Going Beyond Multi-view Redundancy

In a wide range of multimodal tasks, contrastive learning has become a p...

0 Paul Pu Liang, et al. ∙

research

∙ 06/07/2023

Language Models Get a Gender Makeover: Mitigating Gender Bias with Few-Shot Data Interventions

Societal biases present in pre-trained large language models are a criti...

0 Himanshu Thakur, et al. ∙

research

∙ 06/07/2023

Multimodal Learning Without Labeled Multimodal Data: Guarantees and Applications

In many machine learning systems that jointly learn from multiple modali...

6 Paul Pu Liang, et al. ∙

research

∙ 06/07/2023

Multimodal Fusion Interactions: A Study of Human and Automatic Quantification

Multimodal fusion of multiple heterogeneous and interconnected signals i...

0 Paul Pu Liang, et al. ∙

research

∙ 05/24/2023

SenteCon: Leveraging Lexicons to Learn Human-Interpretable Language Representations

Although deep language representations have become the dominant form of ...

0 Victoria Lin, et al. ∙

research

∙ 05/23/2023

Difference-Masking: Choosing What to Mask in Continued Pretraining

Self-supervised learning (SSL) and the objective of masking-and-predicti...

0 Alex Wilf, et al. ∙

research

∙ 05/23/2023

Counterfactual Augmentation for Multimodal Learning Under Presentation Bias

In real-world machine learning systems, labels are often derived from us...

0 Victoria Lin, et al. ∙

research

∙ 05/18/2023

Expanding the Role of Affective Phenomena in Multimodal Interaction Research

In recent decades, the field of affective computing has made substantial...

0 Leena Mathur, et al. ∙

research

∙ 02/23/2023

Quantifying Modeling Feature Interactions: An Information Decomposition Framework

The recent explosion of interest in multimodal applications has resulted...

0 Paul Pu Liang, et al. ∙

research

∙ 12/20/2022

Cross-modal Attention Congruence Regularization for Vision-Language Relation Alignment

Despite recent progress towards scaling up multimodal vision-language mo...

9 rohan-pandey, et al. ∙

research

∙ 11/23/2022

SeedBERT: Recovering Annotator Rating Distributions from an Aggregated Label

Many machine learning tasks – particularly those in affective computing ...

0 Aneesha Sampath, et al. ∙

research

∙ 11/10/2022

Nano: Nested Human-in-the-Loop Reward Learning for Few-shot Language Model Control

Pretrained language models have demonstrated extraordinary capabilities ...

0 Xiang Fan, et al. ∙

research

∙ 09/25/2022

Paraphrasing Is All You Need for Novel Object Captioning

Novel object captioning (NOC) aims to describe images containing objects...

3 Cheng-Fu Yang, et al. ∙

research

∙ 09/07/2022

Foundations and Recent Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questions

Multimodal machine learning is a vibrant multi-disciplinary research fie...

7 Paul Pu Liang, et al. ∙

research

∙ 08/17/2022

Multimodal Lecture Presentations Dataset: Understanding Multimodality in Educational Slides

Lecture slide presentations, a sequence of pages that contain text and f...

26 Dong Won Lee, et al. ∙

research

∙ 07/29/2022

Face-to-Face Contrastive Learning for Social Intelligence Question-Answering

Creating artificial social intelligence - algorithms that can understand...

7 Alex Wilf, et al. ∙

research

∙ 06/30/2022

MultiViz: An Analysis Benchmark for Visualizing and Understanding Multimodal Models

The promise of multimodal models for real-world applications has inspire...

2 Paul Pu Liang, et al. ∙

research

∙ 03/21/2022

PACS: A Dataset for Physical Audiovisual CommonSense Reasoning

In order for AI to be safely deployed in real-world scenarios such as ho...

4 Samuel Yu, et al. ∙

research

∙ 03/03/2022

DIME: Fine-grained Interpretations of Multimodal Models via Disentangled Local Explanations

The ability for a human to understand an Artificial Intelligence (AI) mo...

8 Yiwei Lyu, et al. ∙

research

∙ 03/02/2022

HighMMT: Towards Modality and Task Generalization for High-Modality Representation Learning

Learning multimodal representations involves discovering correspondences...

10 Paul Pu Liang, et al. ∙

research

∙ 02/14/2022

Learning Weakly-Supervised Contrastive Representations

We argue that a form of the valuable information provided by the auxilia...

16 Yao-Hung Hubert Tsai, et al. ∙

research

∙ 02/11/2022

Conditional Contrastive Learning with Kernel

Conditional contrastive learning frameworks consider the conditional sam...

10 Yao-Hung Hubert Tsai, et al. ∙

research

∙ 10/26/2021

Relay Variational Inference: A Method for Accelerated Encoderless VI

Variational Inference (VI) offers a method for approximating intractable...

33 Amir Zadeh, et al. ∙

research

∙ 07/28/2021

Bi-Bimodal Modality Fusion for Correlation-Controlled Multimodal Sentiment Analysis

Multimodal sentiment analysis aims to extract and integrate semantic inf...

13 Wei Han, et al. ∙

research

∙ 07/15/2021

MultiBench: Multiscale Benchmarks for Multimodal Representation Learning

Learning multimodal representations involves integrating information fro...

0 Paul Pu Liang, et al. ∙

research

∙ 06/24/2021

Towards Understanding and Mitigating Social Biases in Language Models

As machine learning methods are deployed in real-world settings such as ...

16 Paul Pu Liang, et al. ∙

research

∙ 06/24/2021

Learning Language and Multimodal Privacy-Preserving Markers of Mood from Mobile Data

Mental health conditions remain underdiagnosed even in countries with co...

12 Paul Pu Liang, et al. ∙

research

∙ 06/05/2021

Integrating Auxiliary Information in Self-supervised Learning

This paper presents to integrate the auxiliary information (e.g., additi...

2 Yao-Hung Hubert Tsai, et al. ∙

research

∙ 06/05/2021

Conditional Contrastive Learning: Removing Undesirable Information in Self-Supervised Representations

Self-supervised learning is a form of unsupervised learning that leverag...

44 Yao-Hung Hubert Tsai, et al. ∙

research

∙ 04/28/2021

A Note on Connecting Barlow Twins with Negative-Sample-Free Contrastive Learning

In this report, we relate the algorithmic design of Barlow Twins' method...

8 Yao-Hung Hubert Tsai, et al. ∙

research

∙ 03/21/2021

Self-supervised Representation Learning with Relative Predictive Coding

This paper introduces Relative Predictive Coding (RPC), a new contrastiv...

32 Yao-Hung Hubert Tsai, et al. ∙

research

∙ 01/22/2021

Understanding the Tradeoffs in Client-Side Privacy for Speech Recognition

Existing approaches to ensuring privacy of user speech data primarily fo...

0 Peter Wu, et al. ∙

research

∙ 01/03/2021

StarNet: Gradient-free Training of Deep Generative Models using Determined System of Linear Equations

In this paper we present an approach for training deep generative models...

11 Amir Zadeh, et al. ∙

research

∙ 12/04/2020

Cross-Modal Generalization: Learning in Low Resource Modalities via Meta-Alignment

The natural world is abundant with concepts expressed via visual, acoust...

6 Paul Pu Liang, et al. ∙

research

∙ 12/04/2020

Multimodal Privacy-preserving Mood Prediction from Mobile Data: A Preliminary Study

Mental health conditions remain under-diagnosed even in countries with c...

3 Terrance Liu, et al. ∙

research

∙ 10/27/2020

Unsupervised Domain Adaptation for Visual Navigation

Advances in visual navigation methods have led to intelligent embodied n...

5 Shangda Li, et al. ∙

research

∙ 10/22/2020

MTGAT: Multimodal Temporal Graph Attention Networks for Unaligned Human Multimodal Language Sequences

Human communication is multimodal in nature; it is through multiple moda...

0 Jianing Yang, et al. ∙

research

∙ 08/31/2020

Toward Multimodal Modeling of Emotional Expressiveness

Emotional expressiveness captures the extent to which a person tends to ...

1 Victoria Lin, et al. ∙

research

∙ 07/24/2020

Style Transfer for Co-Speech Gesture Animation: A Multi-Speaker Conditional-Mixture Approach

How can we teach robots or virtual assistants to gesture naturally? Can ...

0 Chaitanya Ahuja, et al. ∙

research

∙ 07/16/2020

Towards Debiasing Sentence Representations

As natural language processing methods are increasingly deployed in real...

0 Paul Pu Liang, et al. ∙

research

∙ 07/07/2020

What Gives the Answer Away? Question Answering Bias Analysis on Video QA Datasets

Question answering biases in video QA datasets can mislead multimodal mo...

0 Jianing Yang, et al. ∙

research

∙ 06/10/2020

Demystifying Self-Supervised Learning: An Information-Theoretical Framework

Self-supervised representation learning adopts self-defined signals as s...

22 Yao-Hung Hubert Tsai, et al. ∙

research

∙ 06/09/2020

Neural Methods for Point-wise Dependency Estimation

Since its inception, the neural estimation of mutual information (MI) ha...

13 Yao-Hung Hubert Tsai, et al. ∙

research

∙ 05/03/2020

Improving Aspect-Level Sentiment Analysis with Aspect Extraction

Aspect-based sentiment analysis (ABSA), a popular research area in NLP h...

1 Navonil Majumder, et al. ∙

research

∙ 04/29/2020

Interpretable Multimodal Routing for Human Multimodal Language

The human language has heterogeneous sources of information, including t...

0 Yao-Hung Hubert Tsai, et al. ∙

research

∙ 03/06/2020

Diverse and Admissible Trajectory Forecasting through Multimodal Context Understanding

Multi-agent trajectory forecasting in autonomous driving requires an age...

9 Seong Hyeon Park, et al. ∙

research

∙ 03/04/2020

On Emergent Communication in Competitive Multi-Agent Teams

Several recent works have found the emergence of grounded compositional ...

10 Paul Pu Liang, et al. ∙

research

∙ 02/16/2020

Learning Not to Learn in the Presence of Noisy Labels

Learning in the presence of label noise is a challenging yet important t...

11 Liu Ziyin, et al. ∙

Louis-Philippe Morency

Featured Co-authors