DeepAI AI Chat
Log In Sign Up

Action-Affect Classification and Morphing using Multi-Task Representation Learning

by   Timothy J. Shields, et al.

Most recent work focused on affect from facial expressions, and not as much on body. This work focuses on body affect analysis. Affect does not occur in isolation. Humans usually couple affect with an action in natural interactions; for example, a person could be talking and smiling. Recognizing body affect in sequences requires efficient algorithms to capture both the micro movements that differentiate between happy and sad and the macro variations between different actions. We depart from traditional approaches for time-series data analytics by proposing a multi-task learning model that learns a shared representation that is well-suited for action-affect classification as well as generation. For this paper we choose Conditional Restricted Boltzmann Machines to be our building block. We propose a new model that enhances the CRBM model with a factored multi-task component to become Multi-Task Conditional Restricted Boltzmann Machines (MTCRBMs). We evaluate our approach on two publicly available datasets, the Body Affect dataset and the Tower Game dataset, and show superior classification performance improvement over the state-of-the-art, as well as the generative abilities of our model.


Facial Affect Recognition in the Wild Using Multi-Task Learning Convolutional Network

This paper presents a neural network based method Multi-Task Affect Net(...

Visual Person Understanding through Multi-Task and Multi-Dataset Learning

We address the problem of learning a single model for person re-identifi...

Affect Analysis in-the-wild: Valence-Arousal, Expressions, Action Units and a Unified Framework

Affect recognition based on subjects' facial expressions has been a topi...

Multi-Task Dynamical Systems

Time series datasets are often composed of a variety of sequences from t...

Multi-Task Variational Information Bottleneck

In this paper we propose a multi-task deep learning model called multi-t...

Evaluating Temporal Patterns in Applied Infant Affect Recognition

Agents must monitor their partners' affective states continuously in ord...