Deep Convolutional Neural Networks (DCNN) have demonstrated impressive performance improvements for many problems in computer vision. One of the most important reasons behind its success is the availability of large-scale training databases, for example, ImageNet for image classification, Places 
for scene recognition, CompCars for fine-grained recognition and MegaFace  for face recognition.
However, it is not uncommon to have small datasets in many application areas, facial expression recognition being one of them. With a relatively small set of training images, even when regularization techniques such as Dropout 6] are used, the results are not satisfactory. The mostly used method is to fine-tune a network that has been pre-trained on a large dataset. Because of the generality of the pre-learned features, this approach has achieved great success .
Motivated by this observation, several previous works [8, 9] on expression recognition utilize face recognition datasets to pre-train the network, which is then fine-tuned on the expression dataset. The large amount of labeled face data [4, 10]
, makes it possible to train a fairly complicated and deep network. Moreover, the close relationship between the two domains facilites the transfer learning of features.
Although this strategy performs well, it has two notable problems: (i) the fine-tuned face net may still contain information useful for subject identification. This is because of the large size gap (several orders of magnititudes) between face and expression datasets. As we can see from Fig. 1, the images (black-boxed) generated by the face net are dominated by faces as they should, which weakens the network’s ability to represent the different expressions. (ii) the network designed for the face recognition domain is often too big for the expression task, thus the overfitting issue is still severe.
In this paper, we present FaceNet2ExpNet, a novel learning algorithm that incorporates face domain knowledge to regularize the training of an expression recognition network. Specially we first propose a new distribution function to model the high-level neurons of the expression net using the information derived from the fine-tuned face net. Such modeling naturally leads to a regression loss which serves as feature-level regularization that pushes the intermediate features of the expression net to be close to those of the fine-tuned face net. Next, to further improve the discriminativeness of the learned features, we refine the network with strong supervision from the label information. We adopt a conventional network architecture, consisting of convolutional blocks followed by fully-connected layers, to design our expression net. The training is carried out in two stages: in the first stage, only the convolutional layers are trained. We utilize the deep features from the face net as the supervision signal to make the learning easier. It also contains meaningful knowledge about human faces, which is important for expression recognition, too. After the first stage of learning is completed, we add randomly initialized fully-connected (FC) layers and jointly train the whole network using the label information in the second stage. As observed by previous works , FC layers generally capture domain-specific semantics. So we only utilize the face net to guide the learning of the convolutional layers and the FC layers are trained from scratch. Moreover, we empirically find that late middle layer (e.g. pool5 for VGG-16 ) is more suitable for training supervision due to the richness of low entropy neurons. In both training stages, only expression images are used.
From Fig. 1, we can see that the models trained with our method capture the key properties of different expressions. For example, the angry expression is displayed by frowned eye brows and a closed mouth; the surprise expression is represented by a large opened mouth and eyes. This method is different from knowledge distillation . Here we do not have a large accurate network trained on the same domain to produce reliable outputs from softmax. It is also different from FitNets , which is mainly used to train a thinner and deeper network.
To validate the effectiveness of our method, we perform experiments on both constrained (CK+, Oulu-CASIA, TFD) and unconstrained datasets (SFEW). For all the four datasets, we achieve better results than the current state-of-the-art.
The remainder of this paper is organized as follows. Section 2 briefly introduces related works. The FaceNet2ExpNet algorithm is presented in Section 3. Experiments and computational analysis are discussed in Section 4 and Section 5. We conclude this work in Section 6.
Ii Related Works
In , Zhong et al. observed that only a few active facial patches are useful for expression recognition. These active patches include: common patches for the recognition of all expressions and specific patches that are only important for single expression. To locate these patches, a two-stage multi-task sparse learning framework is proposed. In the first stage, multi-task learning with group sparsity is performed to search for the common patches. In the second stage, face recognition is utilized to find the specific patches. However, the sequential search process is likely to find overlapped patches. To solve this problem, Liu et al. 
integrated the sparse vector machine and multi-task learning into a unified framework. Instead of performing the patch selection in two separate phrases, an expression specific feature selection vector and a common feature selection vector are employed together. To get more discriminative features instead of hand-crafted features, Liu et al.
used patch-based learning method. Subsequently, a group feature selection scheme based on the maximal mutual information and minimal redundancy criterion is presented. Lastly, three layers of restricted Boltzman machines (RBM) are stacked to learn hierarchical features. To further boost the performance, a loopy boosted deep belief network (DBN) framework was explored in
. Feature learning, feature selection and classifier design are learned jointly. In the forward phase, several DBNs extract features from the overlapped facial patches. Then, AdaBoosting is adopted to combine these patch-based DBNs. In the fine-tuning phase, the loss from both weak and strong classifiers are backproped. In, to utilize the temporal information for video-based expression recognition, 3D CNN was applied to learn low-level features. Then, a GMM model is trained on the features, and the covariance matrix for each component composes the expressionlet. Motivated by the domain knowledge that facial expression can be decomposed into a combination of facial action units (AU), a deformable facial part model was explored in . Multiple part filters are learned to detect the location of discriminative facial parts. To further cope with the pose and identity variations, a quadratic deformation cost is used.
More recently, Jung et al.  trained a deep temporal geometry network and a deep temporal appearance network with facial landmarks and images. To effectively fuse these two networks, a joint fine-tuning method is proposed. Specifically, the weight values are frozen and only the top layers are trained. In , Mollahosseini et al. discovered that the inception network architecture works very well for expression recognition task. Multiple cross dataset experiments are performed to show the generality of the learned model. In [24, 25], a two-step training procedure is suggested, where in the first step, the network was trained using a relatively large expression dataset followed by training on the target dataset. Even though the image is of low resolution and the label of the relatively large dataset is noisy, this approach is effective. The work closely related to ours is , which proposed to employ a peak expression image (easy sample) to help the training of a network with input from a weak expression image (hard sample). This is also achieved by a regression loss between the intermediate feature maps. However, a pair of the same subject and the same expression image is required as input for training. This is not always possible, especially in unconstrained expression recognition scenario, where the subject identities are usually unknown.
We write our expression net as:
where represents the fully connected layers, and corresponds to the convolutional layers. and are the parameters to be learned. I is the input image, and O is the output before softmax.
First, the parameters of the convolutional layers are learned. In , Xie et al. observed that the high-level neurons are exponentially decayed. To be more specific, by denoting the outputs of the layer as , and the average response value over the spatial dimension as
where is the number of output channels in the layer, and , is the width and height of the response maps, respectively. Then the distribution function can be formulated as follows:
where , and is a normalization constant. is the norm.
To incorporate the knowledge of a face net, we propose to extend (2) to have the following form, i.e., :
The mean is modeled by the face net, . And represents the face net’s convolutional layers. This is motivated by the observation that the fine-tuned face net already achieves competitive performance on the expression dataset, so it should provide a good initialization point for the expression net. Thus, we do not want the latter to deviate much from the former.
Iii-B Training Algorithm
The training algorithm has the following two steps:
In the first stage, we train the convolutional layers with the loss function in (4). The face net is frozen, and the outputs from the last pooling layer are used to provide supervision for the expression net. We provide more explanations on this choice in the next section.
In the second stage, we append the fully connected layers to the trained convolutional layers. The whole network is jointly learned using the cross-entropy loss, defined as follows:
Where is the ground truth for the image, and is the predicated label. The complete training algorithm is illustrated in Fig. 2.
with stride 2. The numbers of the output channels are 64, 128, 256, 512, 512. After the last pooling layer, we add another
convolutional layer, which serves to bridge the gap between face and expression domains. Moreover, it also helps to adapt the dimension if the last pooling layer of the expression net does not match the face net. To reduce overfitting, we have only one fully-connected layer with dimension 256. Note, if the spatial size of the last pooling layer between the face net and expression net does not match exactly, then deconvolution (fractionally strided convolution) can be used for upsampling.
Iii-C Which Layer to Transfer?
In this section, we explore the layer selection problem for the first stage supervision transfer. Since the fine-tuned face network outperforms the pre-trained network on expression recognition, we hypothesize that there may be interesting differences in the network before and after fine-tuning. These differences might help us understand better which layer is more suitable to guide the training of the expression network.
To this end, we first investigate the expression sensitivity of the neurons in the network, using VGG-16 as a working example. For each neuron, the images are ranked by the maximum response values. Then the top ( = 100 in our experiments) images are binned according to the expression labels. We compute the entropy for the neuron as , where denotes the histogram count for bin and denotes the number of quantized label bins (we normalize the histogram to sum to 1). If the neuron has a low entropy, then it should be more expression sensitive since its label distribution is peaky. To validate our assumption, we histogram the entropy for pool4, pool5, FC6 and FC7 layers. In Fig. 3, it is interesting to notice that as the layer goes deeper, more low entropy neurons start to emerge in the fine-tuned network compared with the pre-trained network. This phenomenon is particularly obvious in the fully-connected layers, which are often treated as discriminative features. While for pool4, the distribution does not change too much.
Since these low entropy neurons indicate layer discriminativeness, we next compute the number of low expressive score (LES) neurons for each layer (here low expressive score is the entropy lower than the minimum average entropy score among the four selected layers). In Table I., we find that in comparison with the pre-trained network, the LES neurons increase dramatically in the fine-tuned network, especially starting from pool5 layer. Moreover, convolutional layers have a larger number of these neurons than FC layers. These results suggest that maybe late middle layer, such as pool5, is a good tradeoff between supervision richness and representation discriminativeness.
We validate the effectiveness of our method on four widely used databases: CK+ , Oulu-CASIA , Toronto Face Database (TFD)  and Static Facial Expression in the Wild (SFEW) . The numbers of images for different expressions are shown in Table. II. In the following, we reference our method FaceNet2ExpNet as FN2EN.
We apply the Viola Jones  face detector and IntraFace  for face detection and landmark detection. The faces are normalized, cropped, and resized to . We utilize conventional data augmentation in the form of random sampling and horizontal flipping. The min-batch size is 64, the momentum is fixed to be 0.9 and the dropout is set at 0.5.
For network training, in the first stage, the regression loss is very large. So we start with a very small learning rate 1e-7, and decrease it after 100 epochs. The total training epochs for this stage is 300. We also try gradient clipping, and find that though it enables us to use a bigger learning rate, the results are not better compared to when a small learning rate was used. In the second stage, the fully connected layer is randomly initialized from a Gaussian distribution, and the convolutional layers are initialized from the first stage. The learning rate is 1e-4, and decreased by 0.1 after 20 epochs. We train it for 50 epochs in total. Stochastic Gradient Descent (SGD) is adopted as the optimization algorithm. For testing, asingle center crop with size is used. The settings are same for all the experiments. We use the face net from 34]. Upon publication, the trained expression models will be made publicly available.
Iv-B Neuron Visualization
We first show that the model trained with our algorithm captures the semantic concepts related to facial expression very well. Given a hidden neuron, the face images that obtain high response are averaged. We visualize these mean images for several neurons in Fig. 4 and Fig. 5 on CK+ and Oulu-CASIA, respectively. Human can easily assign each neuron with a semantic concept it measures (i.e. the text in black). For example, the neuron 11 in the first column in Fig. 4 corresponds to Anger, and the neuron 53 in Fig. 5 represents Happy. Interestingly, the high-level concepts learned by the neurons across the two datasets are very consistent.
CK+ consists of 529 videos from 123 subjects, 327 of them annotated with eight expression labels. Each video starts with a neutral expression, and reaches the peak in the last frame. As in other works , we extract the last three frames and the first frame of each video to compose our image-based CK+ database. The total number of the images is 1308, which is split into 10 folds. The subjects are divided into ten groups by ID in ascending order.
|Method||Average Accuracy||#Exp. Classes|
|CSPL ||89.9%||Six Classes|
|AUDN ||92.1%||Eight Classes|
|Train From Scratch (BN)||88.7%|
|VGG Fine-Tune (baseline)||89.9%|
In Table III, we compare our approach with both traditional and deep learning-based methods in terms of average accuracy. We consider the fine-tuned VGG-16 face net as our baseline. To further show the superiority of our method, we also include the results on training from scratch with batch normalization. The network architecture is same as FNEN. The first block shows the results for six classes, while the second block shows the results for eight classes, including both contempt and neutral expressions. Among them, 3DCNN-DAP , STM-ExpLet  and DTAGN  are image-sequence based methods, while others are image-based. For both cases, our method significantly outperforms all others, achieving 98.6% vs the pervious best of 97.3% for six classes, and 96.8% vs 92.1% for eight classes.
Because of the high accuracy on the six class problem, here we only show the confusion matrix for eight class problem. From Fig. 6 we can see that both disgust and fear expressions are perfectly classified, while contempt is the most difficult to classify. It is because this expression has the least number of training images, and the way people show it is very subtle. Surprisingly, from the visualization in Fig. 1, the network is still able to capture the speciality of contempt: the conner of the mouth is pulled up. This demonstrates the effectiveness of our training method.
Iv-D Oulu-CAS VIS
Oulu-CASIA has 480 image sequences taken under Dark, Strong, Weak illumination conditions. In this experiment, only videos with Strong condition captured by a VIS camera are used. There are 80 subjects and six expressions in total. Similar to CK+, the first frame is always neutral while the last frame has the peak expression. Only the last three frames are used, and the total number of images is 1440. A ten-fold cross validation is performed, and the split is subject independent.
Table IV reports the results of average accuracy for the different approaches. As can be seen, our method achieves substantial improvements over the previous best performance achieved by PPDN , with a gain of 3.1%. The confusion matrix is shown in Fig. 7. The proposed method performs well in recognizing fear and happy, while angry is the hardest expression, which is mostly confused with disgust.
The TFD is the largest expression dataset so far, which is comprised of images from many different sources. It contains 4178 images, each of which is assigned one of seven expression labels. The images are divided into 5 separate folds, each containing train, valid and test partitions. We train our networks using the training set and report the average results over five folds on the test sets.
Table V summarizes our TFD results. As we can see, the fine-tuned VGG face is a fairly strong baseline, which is almost on par with the current state-of-the-art, vs . Our method performs the best, significantly outperforming bootstrap-recon  by . From the confusion matrix, we find that fear has the lowest recognition rate and is easy to be confused with surprise. When inspecting the dataset, we find the images from the two expressions indeed have very similar facial appearances: mouth and eyes are wide open.
Different from the previous three datasets, SFEW is targeted for unconstrained expression recognition. So the images are all extracted from films clips, and labeled with seven expressions. The poses are large, and the expression is much more difficult to recognize. Furthermore, it has only 891 training images. Because we do not have access to the test data, here we report the results on the validation data.
|Method||Average Accuracy||Extra Train Data|
|Mapped LBP ||41.92%|
|Train From Scratch (BN)||39.55%|
|VGG Fine-Tune (baseline)||41.23%|
|Transfer Learning ||48.50%||FER2013|
|Multiple Deep Network ||52.29%|
In Table VI, we divide the methods into two blocks, where the first block only uses the training images from SFEW, while the second block utilizes FER2013  as additional training data. For both settings, our method achieves best recognition rates. Especially with more training data, we surpass Multiple Deep Network Learning  by almost 3%, which is the runner-up in EmotiW 2015. We do not compare the result with the winner  since they use 216 deep CNNs to get 56.40%, while we only use a single CNN (1.25% higher than our method). From the confusion matrix Fig. 9, we can see the accuracy for fear is much lower than other expressions. This is also observed in other works .
V Computational speed analysis
Vi Conclusions and Future Works
In this paper, we present FaceNet2ExpNet, a novel two-stage training algorithm for expression recognition. In the first stage, we propose a probabilistic distribution function to model the high level neuron response based on already fine-tuned face net, thereby leading to feature level regularization that exploits the rich face information in the face net. In the second stage, we perform label supervision to boost the final discriminative capability. As a result, FaceNet2ExpNet improves visual feature representation and outperforms various state-of-the-art methods on four public datasets. In future, we plan to apply this training method to other domains with small datasets.
This research is based upon work supported by the Office of the Director of National Intelligence (ODNI), Intelligence Advanced Research Projects Activity (IARPA), via IARPA R&D Contract No. 2014-14071600012. The views and conclusions contained herein are those of the authors and should not be interpreted as necessarily representing the official policies or endorsements, either expressed or implied, of the ODNI, IARPA, or the U.S. Government. The U.S. Government is authorized to reproduce and distribute reprints for Governmental purposes notwithstanding any copyright annotation thereon.
-  J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei, “ImageNet: A Large-Scale Hierarchical Image Database,” in CVPR09, 2009.
-  B. Zhou, A. Lapedriza, J. Xiao, A. Torralba, and A. Oliva, “Learning deep features for scene recognition using places database,” in Advances in neural information processing systems, pp. 487–495, 2014.
L. Yang, P. Luo, C. Change Loy, and X. Tang, “A large-scale car dataset for
fine-grained categorization and verification,” in
Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3973–3981, 2015.
-  I. Kemelmacher-Shlizerman, S. Seitz, D. Miller, and E. Brossard, “The megaface benchmark: 1 million faces for recognition at scale,” arXiv preprint arXiv:1512.00596, 2015.
N. Srivastava, G. E. Hinton, A. Krizhevsky, I. Sutskever, and R. Salakhutdinov,
“Dropout: a simple way to prevent neural networks from overfitting.,”
Journal of Machine Learning Research, vol. 15, no. 1, pp. 1929–1958, 2014.
-  S. Ioffe and C. Szegedy, “Batch normalization: Accelerating deep network training by reducing internal covariate shift,” arXiv preprint arXiv:1502.03167, 2015.
-  R. Girshick, J. Donahue, T. Darrell, and J. Malik, “Rich feature hierarchies for accurate object detection and semantic segmentation,” in Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 580–587, 2014.
-  G. Levi and T. Hassner, “Emotion recognition in the wild via convolutional neural networks and mapped binary patterns,” in Proceedings of the 2015 ACM on International Conference on Multimodal Interaction, pp. 503–510, ACM, 2015.
-  X. Zhao, X. Liang, L. Liu, T. Li, N. Vasconcelos, and S. Yan, “Peak-piloted deep network for facial expression recognition,” arXiv preprint arXiv:1607.06997, 2016.
-  D. Yi, Z. Lei, S. Liao, and S. Z. Li, “Learning face representation from scratch,” arXiv preprint arXiv:1411.7923, 2014.
-  Deepdraw., “Deepdraw on github.com/auduno/ deepdraw,”
-  S. Vittayakorn, T. Umeda, K. Murasaki, K. Sudo, T. Okatani, and K. Yamaguchi, “Automatic attribute discovery with neural activations,” arXiv preprint arXiv:1607.07262, 2016.
-  K. Simonyan and A. Zisserman, “Very deep convolutional networks for large-scale image recognition,” CoRR, vol. abs/1409.1556, 2014.
-  G. Hinton, O. Vinyals, and J. Dean, “Distilling the knowledge in a neural network,” arXiv preprint arXiv:1503.02531, 2015.
-  A. Romero, N. Ballas, S. E. Kahou, A. Chassang, C. Gatta, and Y. Bengio, “Fitnets: Hints for thin deep nets,” arXiv preprint arXiv:1412.6550, 2014.
-  L. Zhong, Q. Liu, P. Yang, B. Liu, J. Huang, and D. N. Metaxas, “Learning active facial patches for expression analysis,” in Computer Vision and Pattern Recognition (CVPR), 2012 IEEE Conference on, pp. 2562–2569, IEEE, 2012.
-  P. Liu, J. T. Zhou, I. W.-H. Tsang, Z. Meng, S. Han, and Y. Tong, “Feature disentangling machine-a novel approach of feature selection and disentangling in facial expression analysis,” in European Conference on Computer Vision, pp. 151–166, Springer, 2014.
-  M. Liu, S. Li, S. Shan, and X. Chen, “Au-aware deep networks for facial expression recognition,” in Automatic Face and Gesture Recognition (FG), 2013 10th IEEE International Conference and Workshops on, pp. 1–6, IEEE, 2013.
-  P. Liu, S. Han, Z. Meng, and Y. Tong, “Facial expression recognition via a boosted deep belief network,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1805–1812, 2014.
-  M. Liu, S. Shan, R. Wang, and X. Chen, “Learning expressionlets on spatio-temporal manifold for dynamic facial expression recognition,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1749–1756, 2014.
-  M. Liu, S. Li, S. Shan, R. Wang, and X. Chen, “Deeply learning deformable facial action parts model for dynamic expression analysis,” in ACCV 2014, pp. 143–157, 2014.
-  H. Jung, S. Lee, S. Park, I. Lee, C. Ahn, and J. Kim, “Deep temporal appearance-geometry network for facial expression recognition,” in ICCV, 2015.
-  A. Mollahosseini, D. Chan, and M. H. Mahoor, “Going deeper in facial expression recognition using deep neural networks,” in 2016 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1–10, IEEE, 2016.
-  Z. Yu and C. Zhang, “Image based static facial expression recognition with multiple deep network learning,” in Proceedings of the 2015 ACM on International Conference on Multimodal Interaction, pp. 435–442, ACM, 2015.
-  H.-W. Ng, V. D. Nguyen, V. Vonikakis, and S. Winkler, “Deep learning for emotion recognition on small datasets using transfer learning,” in Proceedings of the 2015 ACM on International Conference on Multimodal Interaction, pp. 443–449, ACM, 2015.
-  L. Xie, L. Zheng, J. Wang, A. Yuille, and Q. Tian, “Interactive: Inter-layer activeness propagation,” arXiv preprint arXiv:1605.00052, 2016.
-  P. Lucey, J. F. Cohn, T. Kanade, J. Saragih, Z. Ambadar, and I. Matthews, “The extended cohn-kanade dataset (ck+): A complete dataset for action unit and emotion-specified expression,” in CVPR Workshop, pp. 94–101, 2010.
-  G. Zhao, X. Huang, M. Taini, S. Z. Li, and M. Pietikäinen, “Facial expression recognition from near-infrared videos,” Image and Vision Computing, vol. 29, no. 9, pp. 607–619, 2011.
-  J. M. Susskind, A. K. Anderson, and G. E. Hinton, “The toronto face database,” Department of Computer Science, University of Toronto, Toronto, ON, Canada, Tech. Rep, vol. 3, 2010.
-  A. Dhall, O. Ramana Murthy, R. Goecke, J. Joshi, and T. Gedeon, “Video and image based emotion recognition challenges in the wild: Emotiw 2015,” in Proceedings of the 2015 ACM on International Conference on Multimodal Interaction, pp. 423–426, ACM, 2015.
-  P. Viola and M. J. Jones, “Robust real-time face detection,” International journal of computer vision, vol. 57, no. 2, pp. 137–154, 2004.
-  F. De la Torre, W.-S. Chu, X. Xiong, F. Vicente, X. Ding, and J. Cohn, “Intraface,” in Automatic Face and Gesture Recognition (FG), 2015 11th IEEE International Conference and Workshops on, vol. 1, pp. 1–8, IEEE, 2015.
-  O. M. Parkhi, A. Vedaldi, and A. Zisserman, “Deep face recognition,” in BMVC, 2015.
-  Y. Jia, E. Shelhamer, J. Donahue, S. Karayev, J. Long, R. Girshick, S. Guadarrama, and T. Darrell, “Caffe: Convolutional architecture for fast feature embedding,” in Proceedings of the 22nd ACM international conference on Multimedia, pp. 675–678, ACM, 2014.
-  M. S. Bartlett, G. Littlewort, M. Frank, C. Lainscsek, I. Fasel, and J. Movellan, “Recognizing facial expression: machine learning and application to spontaneous behavior,” in 2005 IEEE Computer Society Conference on Computer Vision and Pattern Recognition (CVPR’05), vol. 2, pp. 568–573, IEEE, 2005.
-  X. Feng, M. Pietikäinen, and A. Hadid, “Facial expression recognition based on local binary patterns,” Pattern Recognition and Image Analysis, vol. 17, no. 4, pp. 592–598, 2007.
-  K. Sikka, G. Sharma, and M. Bartlett, “Lomo: Latent ordinal model for facial analysis in videos,” arXiv preprint arXiv:1604.01500, 2016.
-  A. Klaser, M. Marszałek, and C. Schmid, “A spatio-temporal descriptor based on 3d-gradients,” in BMVC 2008-19th British Machine Vision Conference, pp. 275–1, British Machine Vision Association, 2008.
-  Y. Guo, G. Zhao, and M. Pietikäinen, “Dynamic facial expression recognition using longitudinal facial expression atlases,” in ECCV, pp. 631–644, 2012.
-  M. N. Dailey, G. W. Cottrell, C. Padgett, and R. Adolphs, “Empath: A neural network that categorizes facial expressions,” Journal of cognitive neuroscience, vol. 14, no. 8, pp. 1158–1173, 2002.
-  J. Susskind, V. Mnih, G. Hinton, et al., “On deep generative models with applications to recognition,” in Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on, pp. 2857–2864, IEEE, 2011.
-  S. Rifai, Y. Bengio, A. Courville, P. Vincent, and M. Mirza, “Disentangling factors of variation for facial expression recognition,” in European Conference on Computer Vision, pp. 808–822, Springer, 2012.
-  S. Reed, K. Sohn, Y. Zhang, and H. Lee, “Learning to disentangle factors of variation with manifold interaction,” in Proceedings of the 31st International Conference on Machine Learning (ICML-14), pp. 1431–1439, 2014.
-  S. Reed, H. Lee, D. Anguelov, C. Szegedy, D. Erhan, and A. Rabinovich, “Training deep neural networks on noisy labels with bootstrapping,” arXiv preprint arXiv:1412.6596, 2014.
-  I. J. Goodfellow, D. Erhan, P. L. Carrier, A. Courville, M. Mirza, B. Hamner, W. Cukierski, Y. Tang, D. Thaler, D.-H. Lee, et al., “Challenges in representation learning: A report on three machine learning contests,” in International Conference on Neural Information Processing, pp. 117–124, Springer, 2013.
-  B.-K. Kim, J. Roh, S.-Y. Dong, and S.-Y. Lee, “Hierarchical committee of deep convolutional neural networks for robust facial expression recognition,” Journal on Multimodal User Interfaces, pp. 1–17, 2016.
-  A. Krizhevsky, I. Sutskever, and G. E. Hinton, “Imagenet classification with deep convolutional neural networks,” in Advances in neural information processing systems, pp. 1097–1105, 2012.
-  K. Chatfield, K. Simonyan, A. Vedaldi, and A. Zisserman, “Return of the devil in the details: Delving deep into convolutional nets,” arXiv preprint arXiv:1405.3531, 2014.