Automated Assessment of Facial Wrinkling: a case study on the effect of smoking

Facial wrinkle is one of the most prominent biological changes that accompanying the natural aging process. However, there are some external factors contributing to premature wrinkles development, such as sun exposure and smoking. Clinical studies have shown that heavy smoking causes premature wrinkles development. However, there is no computerised system that can automatically assess the facial wrinkles on the whole face. This study investigates the effect of smoking on facial wrinkling using a social habit face dataset and an automated computerised computer vision algorithm. The wrinkles pattern represented in the intensity of 0-255 was first extracted using a modified Hybrid Hessian Filter. The face was divided into ten predefined regions, where the wrinkles in each region was extracted. Then the statistical analysis was performed to analyse which region is effected mainly by smoking. The result showed that the density of wrinkles for smokers in two regions around the mouth was significantly higher than the non-smokers, at p-value of 0.05. Other regions are inconclusive due to lack of large scale dataset. Finally, the wrinkle was visually compared between smoker and non-smoker faces by generating a generic 3D face model.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 2

page 3

page 4

12/05/2017

Recognizing Gender from Human Facial Regions using Genetic Algorithm

Recently, recognition of gender from facial images has gained a lot of i...
05/07/2006

Recognition of expression variant faces using masked log-Gabor features and Principal Component Analysis

In this article we propose a method for the recognition of faces with di...
10/25/2013

Gender Classification Using Gradient Direction Pattern

A novel methodology for gender classification is presented in this paper...
03/20/2021

Automatic Quantification of Facial Asymmetry using Facial Landmarks

One-sided facial paralysis causes uneven movements of facial muscles on ...
02/17/2021

Automated Detection of Equine Facial Action Units

The recently developed Equine Facial Action Coding System (EquiFACS) pro...
02/02/2021

Facial Manipulation Detection Based on the Color Distribution Analysis in Edge Region

In this work, we present a generalized and robust facial manipulation de...
07/22/2020

Risk Assessment in the Face-based Watchlist Screening in e-Border

This paper concerns with facial-based watchlist technology as a componen...
This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

I Introduction

Smoking is a major health problem and a leading cause of preventable death and responsible for more than 30% of cancer-related deaths [1]. In United States smoking is responsible for approximately 443,000 premature deaths yearly [2]. On average, chronic smoking shortens life expectancy by at least 10 years [3]. The skin, like every other organ system in a human body, is affected negatively by smoking [4], leaving it dry, wrinkled and discolored [5].

Wrinkling is a normal phenomenon associated with age progression, it is a popular feature that troubled individuals [6]. In the earliest work since 1965, Ippen and Ippen [7] pointed out the relationship between smoking and face wrinkling. In their study, they conducted the experiment on German women. They stated that the wrinkles and folds covered the smoker’s face, especially on the cheeks. Daniell [8] later reported a similar relation in both men and women. In 1985, Model [9] reported that individuals who had smoked a cigarette for ten years or more can be identified by their facial features alone, which called “smoker face”. “Smoker face” contains more lines or wrinkles on the face.

The decreased moisture in the stratum corneum of the face contributes to facial wrinkling due to direct toxicity of the smoke. Pursing the lips during smoking with contraction of the facial muscles and squinting due to eye irritation from the smoke might cause the formation of wrinkles around the mouth and eyes (crow’s feet). Smoking was determined to be a strong predictor of skin aging [10]. Yin et al. [11] found that the wrinkle depth is significantly more prominent in smokers with a smoking history of at least 35 packs per year than non-smokers. Figure 1 illustrates the effect of smoking on skin, where a smoker’s skin patch (Figure 1(b)) has deeper wrinkles compared to the skin patch of a non-smoker (Figure 1(a)).

This paper presents a case study on the use of computerised algorithm on the facial wrinkles assessment on smokers and non-smokers. First, we present a dataset that consists of smokers and non-smokers, then we extract the facial wrinkles using our proposed modified Hybrid Hessian Filter and perform statistical analysis on the quantification of the wrinkles. Finally, we generate the wrinkles on a generic 3D model to visually compare the facial wrinkles on a smoker and a non-smoker.

The paper is organized into the following sections: Section II represents the basics of the wrinkles detection method used in this work. Section III shows the proposed method and details of the experiment. Section IV elaborates the results and discussion. Finally, Section V concludes the paper and outlines future works.

Fig. 1: The effect of smoking on face skin: (a) illustrates the skin from a non-smoker at age 80 and (b) illustrates the skin from a smoker at age 78.
Fig. 2: Sample face skin from the social habit face dataset [12].

Ii Face Wrinkles

Facial wrinkles are important features of aging on human skin. Detection of face wrinkles from 2D images play an important role in several image-based applications related to aging, like age estimation

[13, 14] and synthesis [15], facial expression recognition [16], face modelling [17], and can be used as a soft biometric [18]. The effect of smoking on wrinkling might have potential impact on the overall performance of these applications.

Ii-a Wrinkle Detection Method

The majority of studies that investigated the association between smoking and wrinkles assessed the wrinkles using subjective method (such as clinical scores for facial wrinkles). However, the clinical scores on wrinkles might be partially influenced by the judgement of individual investigator. Thus, it is necessary to use an objective method to evaluate the facial wrinkles. Cula et al. [19] developed an algorithm for automatic detection of facial wrinkles, this algorithm considered as the first algorithm that records a good results for automatic wrinkles detection. Recently, there are many researchers [20][21] attempted to improve the quality of automatic wrinkles detection methods.

Ng et al. [22]

developed a novel method for automatic wrinkles detection, which called Hybrid Hessian Filter (HHF). HHF is an algorithm for automatic wrinkles detection in 2D facial images. The algorithm is based on the directional gradient and Hessian matrix. HHF detect the wrinkles by computing the Hessian matrix for all pixels of the image. The maximum eigenvalues of the Hessian matrix will indicate if a point belongs to a ridge regardless of the ridge orientation. The eigenvalues are independent vector measures by the components of the second derivatives of the field at each point (x, y). A small eigenvalue indicates a low change rate of the field in the corresponding eigen-direction, and vice versa.

Iii Proposed Method and Experiment

This section discussed the dataset, the proposed automated wrinkles detection algorithm, the wrinkles quantification and visualisation methods.

Iii-a Dataset

There are limited datasets for smokers and non-smokers. The majority of the face datasets does not provide the subject’s smoking status. Recently, Alarifi et. al. [12] introduce a social habit dataset. The dataset is an ongoing collection of high quality images of faces with the social habit of the participants recorded. The dataset consists of 164 images of participants with a mean age of 48.43 (21.44), with age range between 18 to 92. However, due to poor illumination and misalignment, some of these images were excluded from this experiment. In addition, since we are analysing wrinkles on the whole face, we do not include the participants who have occluded face regions, i.e. moustache or beard. Therefore, the experiment is conducted by using 83 images, with the average age of 49 (21) years. The majority of the participants are white British and others are: Asian, African, Malaysian, Arabic, Swedish, and American. There are 41 non-smokers (no history of smoking) and 42 smokers (where some have stopped smoking), Figure 2 shows some sample face images from the social habit face dataset.

Iii-B Face Alignment

As in many face research [23][24], the face was first detected and aligned. There are freely available tools for facial landmarks detection [25], face annotation and alignment [26]. In this work, we use Face++ detector [25]

that utilised a deep learning approach to detect the face. A total of 88 landmarks was obtained (as shown in Figure

3(a)), where 64 points were used to determine inner regions (eyes, eyebrows, mouth and nose) and 24 for contour.

Fig. 3: Illustration of the process in wrinkles extractions. (a) Facial landmarks are detected by using Face++ detector, (b) Aligned face with the face mask overlay for wrinkles region, and (c) Wrinkles extraction using modified HHF.

Then the face shape was aligned to mean shape using procrustes analysis [27]. Each aligned shape is the sum of distances of each shape to the mean as:

(1)

where is the procrustes distance and x is a set of landmark points for one sample. Figure 3(b) illustrate the result from alignment process, where piecewise affine warping [28] was used to warp the texture samples.

Iii-C The Modified Hybrid Hessian Filter Algorithm

The original HHF [22] algorithm detected only horizontal lines, but the facial wrinkles can appear as vertical lines in some face regions, and some of these regions can mainly be effected by smoking [9]. So the HHF algorithm was modified to detect the vertical lines in addition to horizontal lines.

Fig. 4: Visual comparison of the wrinkles extracted from two participants. Left image: Wrinkles extracted from non-smoker at age 80. Right image: Wrinkles extracted from smoker at age 78 shows more prominent wrinkles in terms of depth and density.

The 2D face colour image is converted to greyscale image, and we denoted it as I(x,y). The directional gradient (Gx, Gy) is computed from the greyscale image as:

(2)

where and are the directional gradients. Gx is and Gy is . The directional gradient has greatly smoothed the image and preserved the data of interest.

The Gy has been used as the input images for HHF as the input to calculate Hessian matrix H to extract the horizontal lines. To modify the algorithm for vertical lines detection, Gx is used as the input for our modified HHF. The Hessian matrix H is defined as

(3)

where , and are the outputs of second derivative. The remaining steps to detect the wrinkles are as described in [22].

Iii-D Wrinkles Detection and Quantification

Fig. 5: Face mask with ten predefined wrinkle regions by [21].
Fig. 6: Visual comparison of wrinkles on generic face model. Left image: Wrinkles extracted from a non-smoker at age 80 and visualised on a face model. Right image: Wrinkles extracted from a smoker at age 78 and visualised on a face model.

In this step, face template or mask (as shown in Figure 5) has been used with ten predefined wrinkles regions and fixed coordinates for mouth and eyes [21]. The mask was divided to 10 regions which represents as: forehead, glabella, upper eyelids, crow’s feet (or eye corners), lower eyelids (or eyebag), cheeks, nasolabial grooves (or nasolabial folds), upper lips, marionette and lower lips, all these regions have been used to construct aging patterns. Based on the three points (eye, mouth, and nose), each face image was normalized to the mask by using piecewise affine warping [28]. Finally, the wrinkles pattern for ten regions was constructed using our proposed modified HHF (as shown in Figure 3(c)) [22].

Iii-E Wrinkles Visualisation

By converting the wrinkle intensity in the extracted wrinkle regions (as in Figure 4) into a normal map [29], wrinkles can be visualised on a face model. Many applications support the production of normal maps from black and white images, such as Nvidia’s Photoshop plug-in. To visualise different wrinkles appearances on the smoker and non-smoker, we use an online generator (freely available from http://cpetry.github.io/NormalMap-Online/). First an average face image was fitted to a generic 3D model, and the normal map was identically mapped to the model. As HHF creates intensity base wrinkle maps, these appear as a large hill structures on the face. Because of this, the weight of the normal map is set to negative to ensure the wrinkles went inwards to face and has its overall intensity reduced to 30% of its original. The generated wrinkles model is illustrated in Figure 6 (Left) for a non-smoker and Figure 6 (Right) for a smoker.

Iv Results and Discussion

This study investigates the effect of smoking on facial wrinkling using computerized algorithm. Most of the previous study were conducted using clinical assessment for wrinkles and other face features as in [6, 7, 9, 30, 31]. The computerized studies either used wrinkles in addition to other face features [32] or on skin replica rather than face images [11].

Age group Overall Average Density Smoker Average Density Non-Smoker Average Density
18 - 27 1121.61 1232.75 1032.70
28 - 37 1249.83 1227.40 1265.86
38 - 47 1307.67 1641.40 1069.29
48 - 57 2676.13 2588.39 2938.00
58 - 67 2347.71 2886.67 1943.50
68 - 77 2340.44 3172.22 3308.67
78 5311.00 6006.00 3226.00
TABLE I: The average wrinkles density for smokers and non-smokers.
Fig. 7: A graph illustrates the average wrinkles density versus the age groups.

For this experiment, the wrinkles density was assessed in 83 images, with 42 smokers and 41 non-smokers between 18 to 88 age range. After constructing the wrinkles pattern using the modified HHF, the wrinkles on a defined face region was extracted from the wrinkles pattern using threshold segmentation. The wrinkles density was calculated for each image, which is the area of wrinkle region (depth) divided by the area of the face mask. To analyze the correlation of age and the wrinkles density for smokers, non-smokers and both, the data was divided into 7 groups with 10 intervals (where this ensured that each group consisted of a balanced number of smokers and non-smokers). The average of wrinkles density for the grouped data was calculated for smokers, non-smokers and both as shown in Table 1.

The result showed that the correlation is 0.8354, 0.8996 and 0.8604 between age and average density, age and smoker average density, age and non-smoker average density, respectively. This result showed that the wrinkles density for smoker is higher than non-smoker through age progression, this result is illustrated in Figure 7. The average density of wrinkles at each region was calculated for smoker and non-smoker as shown in Table 2, and the graph in Figure 8 illustrates the comparison between the average density of wrinkles for smokers and non-smokers at each region.

Regions Average density of wrinkles for non-smoker Average density of wrinkles for smoker p-value
Region 1 56.9268 27.5714 0.051
Region 2 26.2927 45.8810 0.186
Region 3 76.5366 89.8333 0.946
Region 4 49.8293 59.6905 0.432
Region 5 161.0488 206.7857 0.936
Region 6 1295.7561 1875.9524 0.297
Region 7 70.9512 167.2381 0.000
Region 8 26.6585 54.4524 0.015
Region 9 63.0488 107.5952 0.064
Region 10 40.0976 55.11905 0.426
TABLE II: The average wrinkles density for smokers and non-smokers at each face region.
Fig. 8: A graph illustrates the average wrinkles density at each region.

The data was analysed statistically by using SPSS, for each region, the wrinkles density was compared between smoker and non-smoker. The result showed that the density of wrinkles for smokers in region 7 and 8 (around the mouth) was significantly higher than the density of wrinkles for non-smokers in the same regions, with the p-value of 0.000 and 0.015 for region 7 and region 8, respectively. This result can be shown clearly in the visual comparison of the wrinkles in Figure 6, where the wrinkles density is higher in the regions around the mouth and eyes for smoker participant. However, due to the lack of large-scale images in this experiment, some of the results are inconclusive. In addition, with only frontal face images, it is difficult to analyse the region around eyes - particularly the crow’s feet, which will work best with profile face images.

V Conclusion

Unlike a previous studies that investigates the association between smoking and facial wrinkles, this paper provides a fully automated work to investigate the effect of smoking on facial wrinkling using a social habit face dataset.

Despite the limitations of this experiment, it gave new insights to the potential use of computerized algorithms, which has a high correlation between the age and the wrinkles density, and aligned with the state-of-the-art research. The result showed that there was a significant association between smoking and increasing the density of wrinkles as we age (as in Figure 7). The visual comparison of wrinkles also showed the differences between the faces of smoker and non-smoker, where the number and depth of wrinkles are the clearest on a smoker’s face. In the future, this work will be extended by using additional number of images, including more frontal face images and the profile face images to analyse the regions around eyes, particularly the crow’s feet.

References

  • [1] G. Campanile, G. Hautmann, and T. Lotti, “Cigarette smoking, wound healing, and face-lift,” Clinics in dermatology, vol. 16, no. 5, pp. 575–578, 1998.
  • [2] C. for Disease Control and P. (US), “How tobacco smoke causes disease: The biology and behavioral basis for smoking-attributable disease: A report of the surgeon general,” https://www.ncbi.nlm.nih.gov/books/NBK53017/, 2010, a Report of the Surgeon General. [Online]. Available: https://www.ncbi.nlm.nih.gov/books/NBK53017/
  • [3] R. Doll, R. Peto, J. Boreham, and I. Sutherland, “Mortality in relation to smoking: 50 years’ observations on male british doctors,” Bmj, vol. 328, no. 7455, p. 1519, 2004.
  • [4] J. F. Gill, S. Y. Siegrid, and I. M. Neuhaus, “Tobacco smoking and dermatologic surgery,” Journal of the American Academy of Dermatology, vol. 68, no. 1, pp. 167–172, 2013.
  • [5] G. Cope, “Smoking and skin ageing: how aesthetic nurses can identify and prevent damage,” Journal of Aesthetic Nursing, vol. 2, no. 7, pp. 328–332, 2013.
  • [6] E. Aizen and A. Gilhar, “Smoking effect on skin wrinkling in the aged population,” International journal of dermatology, vol. 40, no. 7, pp. 431–433, 2001.
  • [7] M. Ippen and H. Ippen, “Approaches to a prophylaxis of skin aging,” J Soc Cosmetic Chemists, vol. 16, pp. 305–8, 1965.
  • [8] H. W. Daniell, “Smoker’s wrinklesa study in the epidemiology of crow’s feet,” Annals of internal medicine, vol. 75, no. 6, pp. 873–880, 1971.
  • [9] D. Model, “Smoker9s face: an underrated clinical sign?” Br Med J (Clin Res Ed), vol. 291, no. 6511, pp. 1760–1762, 1985.
  • [10] A. Morita, “Tobacco smoke causes premature skin aging,” Journal of dermatological science, vol. 48, no. 3, pp. 169–175, 2007.
  • [11] L. Yin, A. Morita, and T. Tsuji, “Skin premature aging induced by tobacco smoking: the objective evidence of skin replica analysis,” Journal of Dermatological Science, vol. 27, pp. 26–31, 2001.
  • [12]

    J. Alarifi, M. Goyal, A. Davison, D. Dancey, R. Khan, and M. H. Yap, “Facial skin classification using convolutional neural networks,” in

    Image Analysis and Recognition: 14th International Conference, ICIAR 2017, Montreal, QC, Canada, July 5–7, 2017, Proceedings, vol. 10317.   Springer, 2017, p. 479.
  • [13] Y. H. Kwon et al., “Age classification from facial images,” in

    Computer Vision and Pattern Recognition, 1994. Proceedings CVPR’94., 1994 IEEE Computer Society Conference on

    .   IEEE, 1994, pp. 762–767.
  • [14] Y.-H. Choi, Y.-S. Tak, S. Rho, and E. Hwang, “Accurate wrinkle representation scheme for skin age estimation,” in Multimedia and Ubiquitous Engineering (MUE), 2011 5th FTRA International Conference on.   IEEE, 2011, pp. 226–231.
  • [15] N. Ramanathan, R. Chellappa, and S. Biswas, “Computational methods for modeling facial aging: A survey,” Journal of Visual Languages & Computing, vol. 20, no. 3, pp. 131–144, 2009.
  • [16] Y. Huang, Y. Li, and N. Fan, “Robust symbolic dual-view facial expression recognition with skin wrinkles: local versus global approach,” IEEE Transactions on Multimedia, vol. 12, no. 6, pp. 536–543, 2010.
  • [17] Y. Bando, T. Kuratate, and T. Nishita, “A simple method for modeling wrinkles on human skin,” in Computer Graphics and Applications, 2002. Proceedings. 10th Pacific Conference on.   IEEE, 2002, pp. 166–175.
  • [18] N. Batool and R. Chellappa, “Modeling and detection of wrinkles in aging human faces using marked point processes,” in European Conference on Computer Vision.   Springer, 2012, pp. 178–188.
  • [19] G. O. Cula, P. R. Bargo, A. Nkengne, and N. Kollias, “Assessing facial wrinkles: automatic detection and quantification,” Skin Research and Technology, vol. 19, no. 1, 2013.
  • [20] C.-C. Ng, M. H. Yap, N. Costen, and B. Li, “Wrinkle detection using hessian line tracking,” IEEE Access, vol. 3, pp. 1079–1088, 2015.
  • [21] C.-C. Ng, M. H. Yap, N. Costen, and B. Li, “Will wrinkle estimate the face age?” in Systems, Man, and Cybernetics (SMC), 2015 IEEE International Conference on.   IEEE, 2015, pp. 2418–2423.
  • [22] C.-C. Ng, M. H. Yap, N. Costen, and B. Li, “Automatic wrinkle detection using hybrid hessian filter,” in Asian Conference on Computer Vision.   Springer, 2014, pp. 609–622.
  • [23]

    M. H. Yap, H. Ugail, R. Zwiggelaar, B. Rajoub, V. Doherty, S. Appleyard, and G. Hurdy, “A short review of methods for face detection and multifractal analysis,” in

    CyberWorlds, 2009. CW’09. International Conference on.   IEEE, 2009, pp. 231–236.
  • [24] A. K. Davison, M. H. Yap, and C. Lansley, “Micro-facial movement detection using individualised baselines and histogram-based descriptors,” in Systems, Man, and Cybernetics (SMC), 2015 IEEE International Conference on.   IEEE, 2015, pp. 1864–1869.
  • [25] E. Zhou, H. Fan, Z. Cao, Y. Jiang, and Q. Yin, “Extensive facial landmark localization with coarse-to-fine convolutional network cascade,” in Proceedings of the IEEE International Conference on Computer Vision Workshops, 2013, pp. 386–391.
  • [26] C. Kendrick, K. Tan, T. Williams, and M. H. Yap, “An online tool for the annotation of 3d models,” in Automatic Face & Gesture Recognition (FG 2017), 2017 12th IEEE International Conference on.   IEEE, 2017, pp. 362–369.
  • [27] C. Goodall, “Procrustes methods in the statistical analysis of shape,” Journal of the Royal Statistical Society. Series B (Methodological), pp. 285–339, 1991.
  • [28] T. F. Cootes, G. J. Edwards, and C. J. Taylor, “Active appearance models,” IEEE Transactions on pattern analysis and machine intelligence, vol. 23, no. 6, pp. 681–685, 2001.
  • [29] X. Gu, S. J. Gortler, and H. Hoppe, “Geometry images,” ACM Transactions on Graphics (TOG), vol. 21, no. 3, pp. 355–361, 2002.
  • [30] C. Bulpitt, H. Markowe, and M. Shipley, “Why do some people look older than they should?” Postgraduate medical journal, vol. 77, no. 911, pp. 578–581, 2001.
  • [31] W.-C. Leung and I. Harvey, “Is skin ageing in the elderly caused by sun exposure or smoking?” British Journal of Dermatology, vol. 147, no. 6, pp. 1187–1191, 2002.
  • [32] A. Raitio, J. Kontinen, M. Rasi, R. Bloigu, J. Röning, and A. Oikarinen, “Comparison of clinical and computerized image analyses in the assessment of skin ageing in smokers and non-smokers.” Acta dermato-venereologica, vol. 84, no. 6, 2004.