A Systematic Evaluation of Domain Adaptation in Facial Expression Recognition

by   Yan San Kong, et al.

Facial Expression Recognition is a commercially important application, but one common limitation is that applications often require making predictions on out-of-sample distributions, where target images may have very different properties from the images that the model was trained on. How well, or badly, do these models do on unseen target domains? In this paper, we provide a systematic evaluation of domain adaptation in facial expression recognition. Using state-of-the-art transfer learning techniques and six commonly-used facial expression datasets (three collected in the lab and three "in-the-wild"), we conduct extensive round-robin experiments to examine the classification accuracies for a state-of-the-art CNN model. We also perform multi-source experiments where we examine a model's ability to transfer from multiple source datasets, including (i) within-setting (e.g., lab to lab), (ii) cross-setting (e.g., in-the-wild to lab), (iii) mixed-setting (e.g., lab and wild to lab) transfer learning experiments. We find sobering results that the accuracy of transfer learning is not high, and varies idiosyncratically with the target dataset, and to a lesser extent the source dataset. Generally, the best settings for transfer include fine-tuning the weights of a pre-trained model, and we find that training with more datasets, regardless of setting, improves transfer performance. We end with a discussion of the need for more – and regular – systematic investigations into the generalizability of FER models, especially for deployed applications.



There are no comments yet.


page 1

page 3


Cross-database non-frontal facial expression recognition based on transductive deep transfer learning

Cross-database non-frontal expression recognition is a very meaningful b...

Gaussian Process Domain Experts for Model Adaptation in Facial Behavior Analysis

We present a novel approach for supervised domain adaptation that is bas...

Facial Expression Recognition in the Wild using Rich Deep Features

Facial Expression Recognition is an active area of research in computer ...

Memory Integrity of CNNs for Cross-Dataset Facial Expression Recognition

Facial expression recognition is a major problem in the domain of artifi...

A Deeper Look at Facial Expression Dataset Bias

Datasets play an important role in the progress of facial expression rec...

Aff-Wild Database and AffWildNet

In the context of HCI, building an automatic system to recognize affect ...

Deep Learning for Domain Adaption: Engagement Recognition

Engagement is a key indicator of the quality of learning experience, and...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.