research
∙
08/27/2023
Towards Vision-Language Mechanistic Interpretability: A Causal Tracing Tool for BLIP
Mechanistic interpretability seeks to understand the neural mechanisms t...
share
research
∙
06/28/2023
MultiZoo MultiBench: A Standardized Toolkit for Multimodal Deep Learning
Learning multimodal representations involves integrating information fro...
research
∙
06/08/2023
Factorized Contrastive Learning: Going Beyond Multi-view Redundancy
In a wide range of multimodal tasks, contrastive learning has become a p...
research
∙
06/07/2023
Language Models Get a Gender Makeover: Mitigating Gender Bias with Few-Shot Data Interventions
Societal biases present in pre-trained large language models are a criti...
research
∙
06/07/2023
Multimodal Learning Without Labeled Multimodal Data: Guarantees and Applications
In many machine learning systems that jointly learn from multiple modali...
research
∙
06/07/2023
Multimodal Fusion Interactions: A Study of Human and Automatic Quantification
Multimodal fusion of multiple heterogeneous and interconnected signals i...
research
∙
05/23/2023
Difference-Masking: Choosing What to Mask in Continued Pretraining
Self-supervised learning (SSL) and the objective of masking-and-predicti...
research
∙
05/21/2023
HIINT: Historical, Intra- and Inter- personal Dynamics Modeling with Cross-person Memory Transformer
Accurately modeling affect dynamics, which refers to the changes and flu...
research
∙
02/23/2023
Quantifying Modeling Feature Interactions: An Information Decomposition Framework
The recent explosion of interest in multimodal applications has resulted...
research
∙
02/09/2023
Read and Reap the Rewards: Learning to Play Atari with the Help of Instruction Manuals
High sample complexity has long been a challenge for RL. On the other ha...
research
∙
12/20/2022
Cross-modal Attention Congruence Regularization for Vision-Language Relation Alignment
Despite recent progress towards scaling up multimodal vision-language mo...
research
∙
11/10/2022
Nano: Nested Human-in-the-Loop Reward Learning for Few-shot Language Model Control
Pretrained language models have demonstrated extraordinary capabilities ...
research
∙
09/07/2022
Foundations and Recent Trends in Multimodal Machine Learning: Principles, Challenges, and Open Questions
Multimodal machine learning is a vibrant multi-disciplinary research fie...
research
∙
08/17/2022
Multimodal Lecture Presentations Dataset: Understanding Multimodality in Educational Slides
Lecture slide presentations, a sequence of pages that contain text and f...
research
∙
07/29/2022
Face-to-Face Contrastive Learning for Social Intelligence Question-Answering
Creating artificial social intelligence - algorithms that can understand...
research
∙
06/30/2022
MultiViz: An Analysis Benchmark for Visualizing and Understanding Multimodal Models
The promise of multimodal models for real-world applications has inspire...
research
∙
03/21/2022
PACS: A Dataset for Physical Audiovisual CommonSense Reasoning
In order for AI to be safely deployed in real-world scenarios such as ho...
research
∙
03/03/2022
DIME: Fine-grained Interpretations of Multimodal Models via Disentangled Local Explanations
The ability for a human to understand an Artificial Intelligence (AI) mo...
research
∙
03/02/2022
HighMMT: Towards Modality and Task Generalization for High-Modality Representation Learning
Learning multimodal representations involves discovering correspondences...
research
∙
07/15/2021
MultiBench: Multiscale Benchmarks for Multimodal Representation Learning
Learning multimodal representations involves integrating information fro...
research
∙
06/24/2021
Towards Understanding and Mitigating Social Biases in Language Models
As machine learning methods are deployed in real-world settings such as ...
research
∙
06/24/2021
Learning Language and Multimodal Privacy-Preserving Markers of Mood from Mobile Data
Mental health conditions remain underdiagnosed even in countries with co...
research
∙
04/24/2021
Ask Explore: Grounded Question Answering for Curiosity-Driven Exploration
In many real-world scenarios where extrinsic rewards to the agent are ex...
research
∙
01/22/2021
Understanding the Tradeoffs in Client-Side Privacy for Speech Recognition
Existing approaches to ensuring privacy of user speech data primarily fo...
research
∙
12/04/2020
Cross-Modal Generalization: Learning in Low Resource Modalities via Meta-Alignment
The natural world is abundant with concepts expressed via visual, acoust...
research
∙
12/04/2020
Multimodal Privacy-preserving Mood Prediction from Mobile Data: A Preliminary Study
Mental health conditions remain under-diagnosed even in countries with c...
research
∙
10/23/2020
An Investigation of how Label Smoothing Affects Generalization
It has been hypothesized that label smoothing can reduce overfitting and...
research
∙
07/16/2020
Towards Debiasing Sentence Representations
As natural language processing methods are increasingly deployed in real...
research
∙
03/18/2020
Anchor Transform: Learning Sparse Representations of Discrete Objects
Learning continuous representations of discrete objects such as text, us...
research
∙
03/06/2020
Diverse and Admissible Trajectory Forecasting through Multimodal Context Understanding
Multi-agent trajectory forecasting in autonomous driving requires an age...
research
∙
03/04/2020
On Emergent Communication in Competitive Multi-Agent Teams
Several recent works have found the emergence of grounded compositional ...
research
∙
02/16/2020
Learning Not to Learn in the Presence of Noisy Labels
Learning in the presence of label noise is a challenging yet important t...
research
∙
01/06/2020
Think Locally, Act Globally: Federated Learning with Local and Global Representations
Federated learning is an emerging research paradigm to train models on p...
research
∙
11/22/2019
Factorized Multimodal Transformer for Multimodal Sequential Learning
The complex world around us is inherently multimodal and sequential (con...
research
∙
07/01/2019
Learning Representations from Imperfect Time Series Data via Tensor Rank Regularization
There has been an increased interest in multimodal language processing i...
research
∙
06/29/2019
Deep Gamblers: Learning to Abstain with Portfolio Theory
We deal with the selective classification problem (supervised-learning p...
research
∙
06/01/2019
Multimodal Transformer for Unaligned Multimodal Language Sequences
Human language is often multimodal, which comprehends a mixture of natur...
research
∙
05/14/2019
Strong and Simple Baselines for Multimodal Utterance Embeddings
Human language is a rich multimodal signal consisting of spoken words, f...
research
∙
03/03/2019
Variational Auto-Decoder: Neural Generative Modeling from Partial Data
Learning a generative model from partial data (data with missingness) is...
research
∙
03/03/2019
Variational Auto-Decoder
Learning a generative model from partial data (data with missingness) is...
research
∙
12/19/2018
An Empirical Evaluation of Sketched SVD and its Application to Leverage Score Ordering
The power of randomized algorithms in numerical methods have led to fast...
research
∙
12/19/2018
Found in Translation: Learning Robust Joint Representations by Cyclic Translations Between Modalities
Multimodal sentiment analysis is a core research area that studies speak...
research
∙
11/23/2018
Words Can Shift: Dynamically Adjusting Word Representations Using Nonverbal Behaviors
Humans convey their intentions through the usage of both verbal and nonv...
research
∙
08/12/2018
Multimodal Language Analysis with Recurrent Multistage Fusion
Computational modeling of human multimodal language is an emerging resea...
research
∙
08/12/2018
Multimodal Local-Global Ranking Fusion for Emotion Recognition
Emotion recognition is a core research area at the intersection of artif...
research
∙
07/11/2018
Seq2Seq2Sentiment: Multimodal Sequence to Sequence Models for Sentiment Analysis
Multimodal machine learning is a core research area spanning the languag...
research
∙
06/16/2018
Learning Factorized Multimodal Representations
Learning representations of multimodal data is a fundamentally complex r...
research
∙
05/31/2018
Efficient Low-rank Multimodal Fusion with Modality-Specific Factors
Multimodal research is an emerging field of artificial intelligence, and...
research
∙
02/03/2018
Memory Fusion Network for Multi-view Sequential Learning
Multi-view sequential learning is a fundamental problem in machine learn...
research
∙
02/03/2018