DeepAI AI Chat
Log In Sign Up

Dyadformer: A Multi-modal Transformer for Long-Range Modeling of Dyadic Interactions

by   David Curto, et al.

Personality computing has become an emerging topic in computer vision, due to the wide range of applications it can be used for. However, most works on the topic have focused on analyzing the individual, even when applied to interaction scenarios, and for short periods of time. To address these limitations, we present the Dyadformer, a novel multi-modal multi-subject Transformer architecture to model individual and interpersonal features in dyadic interactions using variable time windows, thus allowing the capture of long-term interdependencies. Our proposed cross-subject layer allows the network to explicitly model interactions among subjects through attentional operations. This proof-of-concept approach shows how multi-modality and joint modeling of both interactants for longer periods of time helps to predict individual attributes. With Dyadformer, we improve state-of-the-art self-reported personality inference results on individual subjects on the UDIVA v0.5 dataset.


page 1

page 2

page 3

page 4


Husformer: A Multi-Modal Transformer for Multi-Modal Human State Recognition

Human state recognition is a critical topic with pervasive and important...

DocFormer: End-to-End Transformer for Document Understanding

We present DocFormer – a multi-modal transformer based architecture for ...

Multi-modal analysis of genetically-related subjects using SIFT descriptors in brain MRI

So far, fingerprinting studies have focused on identifying features from...

DXM-TransFuse U-net: Dual Cross-Modal Transformer Fusion U-net for Automated Nerve Identification

Accurate nerve identification is critical during surgical procedures for...

InterBERT: Vision-and-Language Interaction for Multi-modal Pretraining

Multi-modal pretraining for learning high-level multi-modal representati...

Multi-Modal Simultaneous Forecasting of Vehicle Position Sequences using Social Attention

Vehicle trajectory forecasting models use a wide variety of frameworks f...

Under the Hood of Transformer Networks for Trajectory Forecasting

Transformer Networks have established themselves as the de-facto state-o...