Introduction
Pose estimation is an important problem in computer vision, and has enabled many practical application from face expression
[1] to activity tracking [2]. Researchers design a new algorithm called explicit shape regression (ESR) to find out face alignment from a picture [3]. Figure 1 shows how the system uses ESR to learn a shape of a human face image. A simple way to identify a face is to find out facial landmarks like eyes, nose, mouth and chin.The researchers define a face shape and is composed of facial landmarks. Therefore, they get . The objective of the researchers is to estimate a shape of a face image. The way to know the accuracy of the estimation is to minimize the alignment error. Equation 1 show how to find alignment error and used for the cascaded learning framework [4].
(1) 
where is the ground true shape of the image. Nevertheless, the true shape is unknown in testing procedure, so optimizationbased method and regressionbased method are the most popular approaches for this problem. The researchers use the regressionbased method, and, then, they create the algorithm, ESR. Compared to most previous works, ESR does not use any parametric shape models. Considering all facial landmarks are regressed jointly, the researchers train the regressor by reducing the alignment error over training data. This regressor solve large shape variantions and guarantee robustness. Moreover, the researchers design the second regressor to solve small variantion and ensure accuracy. Besides, ESR used the improved version of the cascaded pose regression framework. The layer based regressor has also be adopted for many other applications, such as depth completion [6]. The next section goes over some related works.
Related Works
In previous works, active appearance models[5] (AAM) is one of influential models in alignment approaches. In 1998, Cootes et al. introduce AAM to interpret an image. In the same year, they publish how this model interpret face image[7] and recognize face[8]. In 2007, Saragih and Goechke[10] propose a nonlinear discriminative approach to AAM. In 2010, Laurens and Emile[11] make an extension of AAM to solve the large variation in face appearance. In 2011, Sauer and Cootes[13]
use random forest and boosting regression with AAM. In face alignment, regressionbased method is a wellknown one. In 2007, Cristinancce and Cootes
[14] construct constrained local models for face interpretion. In 2010, Valstar et al.[15] apply boosted regression find facial features. Cascaded pose regressor (CPR) was originally derived from the boosted regression category [4]. CPR has also been successfully used for face detection in videos
[12].Method
This section illustrates the ESR. A basic and essential terms are required to declared here, the normalized shape . is produced from Equation 2 that looks for a that can make as close to as the mean shape as possible.
(2) 
where is the mean shape and is an input shape.
There are training samples , and the stage regressors . In every Stage , the stage regressor is learnt like Equation 3.
(3) 
where is the estimated shape in previous Stage , and is the normalized regression target.
In testing, in each Stage , the normalized shape is computed as follow,
(4) 
where the normalized shape is updated by the regressor from . The normalization reduces the complication of the regression. Suppose there are two facial images, and with the estimated shape and is transformed from . The results of both regressions are different due to transformation. However, normalization, simplifying the problem, would give the same result in both regressions. Algorithem 1 below shows how ESR processing in both training and testing.
In ESR training, the researchers initialize the training data, compute normalized targets and get the stage regressors. In the ESR testing, ther researchers initialize the testing data, normalize targets and output the shape using the regressor.
In the initialization, the researchers has an composed of exemplar shapes which are viewed as representative shapes or groundtruth shapes from the training data. Each exemplar shape generates a number of initial shapes. The researchers’ implementation here is 20. The initialization returns a triple , the facial image, the groundtruth shape and the initial shape. Therefore, the number of a triple that has identical facial image and identical groundtruth shape is 20.
To improve the performance of the experiments, the researchers introduce twolevel boosted regression, externallevel and internallevel. The stage regressor is internallevel and is called primitive regressor. The number of iterations is , where is the number of iterations in the externallevel and is that in the others. Figure 2 shows the tradeoffs between two level boosted regression. When the total iterations is 5000, the researchers find the lowest mean error as and .
A fern[16] is used as the internallevel regressor. Fern can be considered as pixel domain comparison in contrast to features, and can be used for image matching [9]. In the researchers’ implementation, a fern contains features. Thus, the feature space and all training samples are divided into bins and every bin means a regression output . The prediction of a bin is calculated like Equation 5,
(5) 
where the set implies the samples in the th bin. The best solution is the average,
(6) 
To solve the overfitting problem, Equation 6 is revised into Equation 7.
(7) 
where is a free shrinkage parameter. Given that the final regressed shape starts from the initial shape and updates by catching the information from the groundtruth shapes, is computed like Equation 8,
(8) 
Before demonstrating the procedure of the internalboosted regression, shape indexed features is introduced as an important ingredient in the regression. The researchers use pixeldifference features, which means the intensity difference of two pixels in the image. The extracted pixels are unchangeable to similarity transform and normalization. Each pixel is indexed by the local coorinate , where is a landmark associated with the pixel. Considering that pixels indexed by the same global coordinates may be variant because of different face shape while those in the same local coordinates are invariant though in different face shape (detail in Figure 3),
the researchers find the local coordinates first and, then, turn them back to the global coordinates for the intensity difference of two pixels. The transform is showed in Euqation 9.
(9) 
where is the operator to get the and coordinates of a landmark from the shape. In Figure 3 and the explanation above, different samples has the identical . Alogorithm 2 shows how the researchers get shape indexed features.1.0
In Algorithm 2, numbers of pixels are generated, so there are number of pixeldifference features. This makes a huge number of calculations. To optimize the performance and improve the efficiency, the researchers create a correlationbased feature selection to reduce unnecessary computations.
The researchers selects out of features to form a fern regressor. Two properities are required for a good fern: high correlation between each feature in the fern and the regression target and low correlation between features enough to composed complementarily. They propse Equation 10 to maximizing feature’s correlation:
(10) 
where is the regression target with rows and columns, and is pixeldifference features matrix with rows and columns. is the number of samples. Each column of feature matrix represents a pixeldifference feature. means a projection
into a column vector from unit Gaussian. The researchers write a pixeldifference feature as
, and they get Equation 10 about the correlation below:(11) 
The pixelpixel covariances can be precomputed and reused with in each internallevel boosted regression due to fixity of the shape indexed pixels, which reduces the complexity from to . Algorithm 3 is the method to selecting correlatioinbased feature. 1.0
After introduction of shape indexed features and correlationbased feature selection, the internalboosted regression is explained in Algorithm 4. The regression consists of primitive regressors , which are ferns.
thresholds are sampled randomly from an uniform distribution provided the range of pixel difference feature is
, the range of the uniform distribution is . In each iteration a new primitive regressor is learn from the residues left by previous regressors. 1.0Experiments and results
The researchers compare their approach to previous approaches. Figure 46 show samples of images from different datasets.
0.1 Comparison with CE on LFPW
0.2 Comparison with CDS on LFPW87
0.3 Comparison with STASM and CompASM on Helen
Reference
References
 [1] S.B. Gokturk, J.Y. Bouguet, “Modelbased face tracking for viewindependent facial expression recognition, ” Automatic Face and Gesture Recognition, pp. 55 37, 2012.
 [2] J. Shen and J. Yang, “Automatic human animation for nonhumanoid 3d characters,” International Conference on ComputerAided Design and Computer Graphics (CAD/Graphics), pp. 220221, 2015.
 [3] Cao, Xudong, et al. ”Face alignment by explicit shape regression.” International Journal of Computer Vision 107.2 (2014): 177190.

[4]
Dollár, Piotr, Peter Welinder, and Pietro Perona. ”Cascaded pose regression.” Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on. IEEE, 2010.
 [5] Cootes, Timothy F., Gareth J. Edwards, and Christopher J. Taylor. ”Active appearance models.” European conference on computer vision. Springer Berlin Heidelberg, 1998.
 [6] J. Shen and S. C. S. Cheung, “Layer Depth Denoising and Completion for StructuredLight RGBD Cameras,” IEEE Conference on Computer Vision and Pattern Recognition, pp. 11871194, 2013.
 [7] Edwards, Gareth J., Christopher J. Taylor, and Timothy F. Cootes. ”Interpreting face images using active appearance models.” Automatic Face and Gesture Recognition, 1998. Proceedings. Third IEEE International Conference on. IEEE, 1998.
 [8] Edwards, Gareth, Timothy Cootes, and Christopher Taylor. ”Face recognition using active appearance models.” Computer Vision—ECCV’98 (1998): 581595.
 [9] J. Shen and W. Tan, “Imagebased indoor placefinder using image to plane matching,” IEEE International Conference on Multimedia and Expo, pp. 16, 2013.
 [10] Saragih, Jason, and Roland Goecke. ”A nonlinear discriminative approach to AAM fitting.” Computer Vision, 2007. ICCV 2007. IEEE 11th International Conference on. IEEE, 2007.
 [11] Van Der Maaten, Laurens, and Emile Hendriks. ”Capturing appearance variation in active appearance models.” Computer Vision and Pattern Recognition Workshops (CVPRW), 2010 IEEE Computer Society Conference on. IEEE, 2010.
 [12] R. Liu, J. Shen, Q. Sun, J. Yang, S. C. S. Cheung, “Cascaded Pose Regression Revisited: Face Alignment in Videos,” IEEE Third International Conference on Multimedia Big Data (BigMM), pp. 291298, 2017.
 [13] Patrick Sauer, Tim Cootes and Chris Taylor. Accurate Regression Procedures for Active Appearance Models. In Jesse Hoey, Stephen McKenna and Emanuele Trucco, Proceedings of the British Machine Vision Conference, pages 30.130.11. BMVA Press, September 2011. http://dx.doi.org/10.5244/C.25.30
 [14] Cristinacce, David, and Timothy F. Cootes. ”Boosted regression active shape models.” BMVC. Vol. 2. 2007.
 [15] Valstar, Michel, et al. ”Facial point detection using boosted regression and graph models.” Computer Vision and Pattern Recognition (CVPR), 2010 IEEE Conference on. IEEE, 2010.
 [16] Ozuysal, Mustafa, et al. ”Fast keypoint recognition using random ferns.” IEEE transactions on pattern analysis and machine intelligence 32.3 (2010): 448461.
 [17] Belhumeur, Peter N., et al. ”Localizing parts of faces using a consensus of exemplars.” IEEE transactions on pattern analysis and machine intelligence 35.12 (2013): 29302940.
 [18] Liang, Lin, et al. ”Face alignment via componentbased discriminative search.” Computer Vision–ECCV 2008 (2008): 7285.
 [19] Le, Vuong, et al. ”Interactive facial feature localization.” Computer Vision–ECCV 2012 (2012): 679692.
 [20] Milborrow, Stephen, and Fred Nicolls. ”Locating facial features with an extended active shape model.” European conference on computer vision. Springer Berlin Heidelberg, 2008.
Comments
There are no comments yet.