DeepAI AI Chat
Log In Sign Up

Landmark-Aware and Part-based Ensemble Transfer Learning Network for Facial Expression Recognition from Static images

by   Rohan Wadhawan, et al.

Facial Expression Recognition from static images is a challenging problem in computer vision applications. Convolutional Neural Network (CNN), the state-of-the-art method for various computer vision tasks, has had limited success in predicting expressions from faces having extreme poses, illumination, and occlusion conditions. To mitigate this issue, CNNs are often accompanied by techniques like transfer, multi-task, or ensemble learning that often provide high accuracy at the cost of high computational complexity. In this work, we propose a Part-based Ensemble Transfer Learning network, which models how humans recognize facial expressions by correlating the spatial orientation pattern of the facial features with a specific expression. It consists of 5 sub-networks, in which each sub-network performs transfer learning from one of the five subsets of facial landmarks: eyebrows, eyes, nose, mouth, or jaw to expression classification. We test the proposed network on the CK+, JAFFE, and SFEW datasets, and it outperforms the benchmark for CK+ and JAFFE datasets by 0.51% and 5.34%, respectively. Additionally, it consists of a total of 1.65M model parameters and requires only 3.28 × 10^6 FLOPS, which ensures computational efficiency for real-time deployment. Grad-CAM visualizations of our proposed ensemble highlight the complementary nature of its sub-networks, a key design parameter of an effective ensemble network. Lastly, cross-dataset evaluation results reveal that our proposed ensemble has a high generalization capacity. Our model trained on the SFEW Train dataset achieves an accuracy of 47.53% on the CK+ dataset, which is higher than what it achieves on the SFEW Valid dataset.


page 2

page 3

page 4

page 8

page 11

page 12


Multi-Region Ensemble Convolutional Neural Network for Facial Expression Recognition

Facial expressions play an important role in conveying the emotional sta...

Multi-Domain Norm-referenced Encoding Enables Data Efficient Transfer Learning of Facial Expression Recognition

People can innately recognize human facial expressions in unnatural form...

LBVCNN: Local Binary Volume Convolutional Neural Network for Facial Expression Recognition from Image Sequences

Recognizing facial expressions is one of the central problems in compute...

Deep Multi-Facial Patches Aggregation Network For Facial Expression Recognition

In this paper, we propose an approach for Facial Expressions Recognition...

Facial Expression Translation using Landmark Guided GANs

We propose a simple yet powerful Landmark guided Generative Adversarial ...

Efficient Facial Feature Learning with Wide Ensemble-based Convolutional Neural Networks

Ensemble methods, traditionally built with independently trained de-corr...