Finding Missing Children: Aging Deep Face Features

11/18/2019 ∙ by Debayan Deb, et al. ∙ Michigan State University 36

Given a gallery of face images of missing children, state-of-the-art face recognition systems fall short in identifying a child (probe) recovered at a later age. We propose an age-progression module that can age-progress deep face features output by any commodity face matcher. For time lapses larger than 10 years (the missing child is found after 10 or more years), the proposed age-progression module improves the closed-set identification accuracy of FaceNet from 40 celebrity dataset, namely ITWCC. The proposed method also outperforms state-of-the-art approaches with a rank-1 identification rate from 94.91 95.91 These results suggest that aging face features enhances the ability to identify young children who are possible victims of child trafficking or abduction.

READ FULL TEXT VIEW PDF
POST COMMENT

Comments

There are no comments yet.

Authors

page 1

page 2

page 4

page 6

page 8

This week in AI

Get the week's most popular data science and artificial intelligence research sent straight to your inbox every Saturday.

1 Introduction

Human trafficking is one of the most adverse social issues currently faced by countries worldwide. According to the United Nations Children’s Fund (UNICEF) and the Inter-Agency Coordination Group against Trafficking (ICAT), 28% of the identified victims of human trafficking globally are children111The United Nations Convention on the Rights of the Child defines a child as “a human being below the age of 18 years unless under the law applicable to the child, majority is attained earlier” [6] [7]. The Wall Street Journal reported in

that it is estimated that around

million children go missing around the world every year [8]. Children separated from their parents, such as refugees and migrants, are most vulnerable to trafficking. According to the FBI, in there were NCIC (National Crime Information Center) entries for missing children in the United States [9]. As of 2018, juveniles under the age of 18 account for % of the total active missing records in NCIC [9]. The actual number of missing children is much more than these official statistics as only a limited number of cases are reported because of the fear of traffickers, lack of information, and mistrust of authorities.

Face recognition is perhaps the most promising biometric technology for recovering missing children, since parents and relatives are more likely to have a lost child’s face photograph than other biometric modalities such as fingerprint or iris222Indeed, face is certainly not the only biometric modality for identification of lost children. Sharbat Gula, first photographed in 1984 (age 12) in a refugee camp in Pakistan, was later recovered via iris recognition at the age of 30 from a remote part of Afghanistan in 2002 [10].. While Automated Face Recognition (AFR) systems have been able to achieve high identification rates [3, 2, 11], their ability to recognize children as they age is still limited.

A human face undergoes various temporal changes, including skin texture, weight, facial hair, etc. (see Figure Finding Missing Children: Aging Deep Face Features[12, 13]. Several studies have analyzed the extent to which facial aging affects the performance of AFR (see Table 2). Two major conclusions can be drawn based on these studies: (i) Performance decreases with an increase in time lapse between subsequent image acquisitions [14, 15, 16], and (ii) performance degrades more rapidly in the case of younger individuals than older individuals [16, 17]. Figure 7 illustrates that state-of-the-art face matchers fail considerably when it comes to matching an enrolled child in the gallery with the corresponding probe over large time lapses. Thus, it is essential to enhance the longitudinal performance of AFR systems, especially when the child is enrolled at a young age.

Locating missing children is analogous to the identification scenario (either open-set or closed-set) in face recognition where we search a gallery of missing children to determine the identity of a child retrieved at a later age (probe). As the time gap between a probe image and the true mate in the gallery, gets larger, the search problem gets harder.

(a) Saroo Brierley [18]444The award-winning 2016 movie, Lion, is based on the true story of Saroo Brierley [21].
(b) Jaycee Dugard [19]
(c) Carlina White [20]
Figure 4: Face images of missing children in three high profile cases who were successfully recovered after a large time lapse.
(a)
(b)
Figure 7: Heat map of rank-1 identification accuracy (%) (a) without modifying FaceNet features by the proposed aging module and (b) with modifying FaceNet features by the proposed aging module (darker colors indicate higher accuracy). The age of the child in the gallery along with time lapse to the probe are shown along the two axes.

Prior studies on face recognition under aging, both for adults and children, explored both generative and discriminative models. Given a probe face image, generative models can generate face images that can either predict how the person will look over time (age progression) or estimate how he looked in previous years (age regression) by utilizing Generative Adversarial Networks (GANs) [22, 23, 24, 25, 26, 27]. The primary motivation is to enhance the visual quality of the age progressed or regressed face images, rather than enhancing the face recognition performance. On the other hand, discriminative approaches focus on age-invariant face recognition under the assumption that age and identity related information can be separated [28, 29, 30, 31, 4, 5]. By separating age-related components, only the identity-related information is used for face matching. Since age and identity are highly correlated in the feature space, the task of disentangling them from face embeddings is not only difficult but can also be detrimental to AFR performance [32, 33].

A majority of the prior studies on cross-age face recognition555Face matching or retrieval under aging changes [28, 30, 31, 26, 5, 4] evaluate the performance of their models on longitudinal face datasets, such as MORPH ( subjects in the age range of - years) and CACD ( subjects in the age range of - years), which mainly comprise adult face images. Indeed, some benchmark face datasets such as FG-NET ( subjects in the age range of - years) do include a small number of children, however, the associated protocol is based on matching all possible comparisons for all ages, which does not explicitly provide child-to-adult matching performance. Moreover, earlier studies employ cross-sectional techniques where the temporal performance is analyzed according to differences between age groups [5, 23, 34]. In cross-sectional or cohort-based approaches, which age groups or time lapses are evaluated is often arbitrary and varies from one study to another, thereby, making comparisons between studies difficult [35, 36]. Furthermore, cross-sectional analysis with summary statistics does not investigate whether age-related face recognition performance trends are due to other noise factors such as variations in illumination, expression, and pose. For these reasons, since facial aging is longitudinal by nature, cross-sectional analysis is not the correct model for exploring aggregated effects [35, 37, 38]. The correct model is the longitudinal model that has been utilized for temporal data for fingerprints [35], face [37, 15] and iris [38].

We propose an age-progression module666Though our module is not strictly restricted to age-progression, we use the word progression largely because in the missing children scenario the gallery would generally be younger than the probe. Our module does both age-progression and age-regression when we benchmark our performance on public datasets. that learns a projection in the feature space and can be used as a wrapper around any commodity face matcher. Our module can also synthesize the face image corresponding to aged features for a given individual and specified target age. Our empirical results show that the proposed module, based on an encoder-decoder architecture, can enhance the longitudinal face recognition performance of three face matchers (FaceNet [2], CosFace [3], and a commercial-off-the-shelf (COTS) matcher) for matching children as they age.

The contributions of the paper can be summarized as follows:

  • [topsep=0.5em, itemsep=-0.1em]

  • A feature aging strategy for traversing the face manifold in the deep feature space, such that the identity of the subject is preserved while only the age component is progressed or regressed in the face embedding.

  • Visualizing the aged face from the age-progressed face features, via a decoder, which illustrates that our proposed method can indeed model the age manifold while preserving the identity information.

  • With the proposed age-progression module, rank-1 identification rates of a state-of-the-art matcher, CosFace [3], increase from to on CFA (a child face aging dataset), and to on ITWCC [39] (a child celebrity dataset). In addition, the proposed module boosts accuracies from and to and on FG-NET and CACD-VS respectively, which are the two public face aging benchmark datasets [40, 41]777

    We follow the exact protocols provided with these datasets. We open-source our code for reproducibility: [

    url omitted for blind review]..

2 Related Work

 

Dataset No. of Subjects No. of Images No. Images / Subject Age Range (years) Avg. Age (years) Public888MORPH: https://bit.ly/31P6QMw, CACD: https://bit.ly/343CdVd, FG-NET: https://bit.ly/2MQPL0O, UTKFace: https://bit.ly/2JpvX2b

 

MORPH-II [42] 13,000 55,134 2-53 (avg. 4.2) 16-77 42 Yes
CACD [41] 2,000 163,446 22-139 (avg. 81.7) 16-62 31 Yes

 

FG-NET [40] 82 1,002 6-18 (avg. 12.2) 0-69 16 Yes
UTKFace [23] N/A 23,708 N/A 0-116 33 Yes
ITWCC [1] 745 7,990 3-37 (avg. 10.7) 0-32 13 No
CLF [17] 919 3,682 2-6 (avg. 4.0) 2-18 8 No
CFA 9,196 25,180 2-6 (avg. 2.7) 2-20 10 No

 

  • Dataset does not include subject labels; Only a collection of face images along with the corresponding ages.

  • Concerns about privacy issues are making it extremely difficult for researchers to place the child face images in public domain.

Table 1: Face aging datasets. Datasets below solid line includes longitudinal face images of children.

 

Study Objective Dataset Age groups or range (years)

 

Yang et al. [26]* Age progression of face images MORPH-II, CACD 31-40, 41-50, 50+
Wang et al. [5]* Decomposing age and identity MORPH-II, FG-NET, CACD 0-12, 13-18, 19-25, 26-35, 36-45, 46-55, 56-65, 65+
Best-Rowden et al. [36] Model for change in genuine scores over time PCSO, MSP 18-83

 

Ricanek et al. [1] Face comparison of infants to adults ITWCC 0-33
Deb et al. [17] Feasibility study of AFR for children CLF 2-18
This study Aging face features for enhanced AFR for children CFA, ITWCC, FG-NET, CACD 0-18

 

  • Study uses cross-sectional model (ages are partitioned into age groups) and not the more appropriate longitudinal model [15][35].

Table 2: Related work on cross-age face recognition. Studies below bold line deal with children.

2.1 Discriminative Approaches

Approaches prior to deep learning leveraged robust local descriptors 

[43, 44, 45, 29, 46] to tackle recognition performance degradation due to face aging. Recent approaches focus on age-invariant face recognition by attempting to discard age-related information from deep face features [28, 30, 47, 4, 5]. All these methods operate under two critical assumptions: (1) age and identity related features can be disentangled, and (2) the identity-specific feature is adequate for face recognition performance. Several studies, on the other hand, show that age is indeed a major contributor to face recognition performance [32, 33]. Therefore, instead of completely discarding age factors, we exploit the age-related information to progress or regress the deep feature directly to the desired age.

2.2 Generative Approaches

Ongoing studies leverage Conditional Auto-encoders and Generative Adversarial Networks (GANs) to synthesize faces by automatically learning aging patterns from face aging datasets [48, 23, 4, 26, 25, 24]. The primary objective of these methods is to synthesize visually realistic face images that appear to be age progressed, and therefore, a majority of these studies do not report the recognition rates.

2.3 Face Aging for Children

Best-Rowden et al. studied face recognition performance of newborns, infants, and toddlers (ages 0 to 4 years) on 314 subjects acquired over a maximum time lapse of only one year [37]. Their results showed a True Accept Rate (TAR) of 47.93% at 0.1% False Accept Rate (FAR) for an age group of [0, 4] years for a commodity face matcher. Deb et al. fine-tuned FaceNet [2] to achieve a rank-1 identification accuracy of only for a time lapse between the gallery and probe image of 1 year. Srinivas et al. showed that the rank-1 performance of state of the art commercial face matchers on longitudinal face images from the In-the-Wild Child Celebrity (ITWCC) [39] dataset ranges from to . These studies (see Table 2

) primarily focused on evaluating the longitudinal face recognition performance of state-of-the-art face matchers rather than proposing a solution to improve face recognition performance on children as they age. To the best of our knowledge, our study is the first to propose a model for aging deep face features extracted from any commodity face matcher to enhance longitudinal face recognition accuracy on children.

Table 1 summarizes longitudinal face datasets that include children and Table 2 shows related work in this area.

Probe Age: 10 Age: 30 Age: 70 Age: 90
Figure 8:

Decoded images from age-progressed features via linear interpolation. The figure indicates that traversing in a linear fashion in the feature space can indeed generate age-progressed features. Here, the two probe images correspond to ages of 5 and 12 years, respectively.

3 Aging Deep Face Features

Directly manipulating pixels in a face image for age-progression may not preserve the identity of the child in the feature space. Therefore, we propose an age-progression module that learns a projection of the deep features in a lower-dimensional feature space which can directly improve the accuracy of face recognition systems in identifying children over large time lapses (see Figure 10).

3.1 Motivation

In order to analyze the effect of aging on face matching performance of children, let , where is the set of all possible ages in the dataset. Here, , where is the set of all images of missing children in the dataset, , acquired at age . Using a commodity face matcher such as FaceNet999The open-source face matcher, FaceNet, is available at https://github.com/davidsandberg/facenet., we can extract a deep feature representation of the image which we denote as .

We first compute the mean face representation for all ages in . For age , we obtain the mean face feature as

We extract the mean face features , from the UTKFace dataset [23]. In order to isolate the differences induced in the face embeddings due to variations in age, let us define an 

attribute vector

as the difference between any two mean face features at the ages and ,

where . Similar to deep feature interpolation [49], we map a child’s face image at age , , to a point in the feature space and move it linearly along the attribute vector , via

(1)

In figure 8, we show a few decoded image examples when face features for two children, at ages 5 and 12, are moved in this linear fashion along the manifold ().

This experiment indicates that

  • [topsep=0.2em, itemsep=-0.2em]

  • Face embeddings capture sufficient information about age required for age-progression.

  • Age-progression can be manifested by linearly interpolating in the feature space.

3.2 Learning Feature Age-Progression

An ideal face feature space should only encode the identity-salient features and age-related components should be disentangled from identity-relevant features. However, in reality, face matchers naturally encode age-related information in the latent space which has been shown to enhance the discriminative power [32, 33]. We aim to develop an age-progression method that can learn a projection within any face matcher’s feature space (see Figure 9).

A pre-trained face matcher embeds a face image, to a -dimensional Euclidean space101010Assume these feature vectors are constrained to lie in a -dimensional hypersphere, i.e., ., . Assume we have a training set of image pairs, , where, and are two images of the same person acquired at ages and , respectively. Here, , where is the face image domain and is the set of all possible ages. Our goal is to learn a model that takes a face feature vector, , and synthesizes a face embedding for the desired age, , such that the identity of the person is preserved while the age-related components are similar to that of .

We propose an encoder-decoder architecture that can automatically learn age-progression in the feature space. The encoder is a stack of fully connected linear layers that maps a feature vector to a -dimensional latent representation . The encoder is conditioned on the input feature, , the age at image acquisition, , and the desired age after progression, . The decoder is also a stack of fully connected linear layers that synthesize an age-progressed version of the original face feature , given its latent representation . In order to ensure that the identity-salient features are preserved and the synthesized features are age-progressed to the desired age, we use train the age-progression module via a mean squared error (MSE) loss which measures the quality of the predicted features:

(2)

where is the set of all genuine pairs. After the model is trained, the age-progression module can progress a face feature to the desired age.

Figure 9: Training the age-progression module. For each face feature acquired at age , the predicted features for the desired age, , are regressed to the genuine face feature at age .
Figure 10: Overview of the proposed deep feature age-progression method. The age-progression module can progress a face feature vector to any desired age.

4 Implementation Details

Age-Progression Module

The proposed feature aging scheme consists of two components: (a) an encoder and (b) a decoder. For all the experiments, we employ a fully connected linear layer for the encoder and decoder and empirically set the output of each layer to be of the same size as the input feature vector, i.e.,

. We do not apply any non linear activation function to the fully connected layers. We train the module for 2,000 iterations with a learning rate of

using Adam optimizer ().

Face Matcher

For all our experiments, we employ 3 face matchers111111Both the open-source matchers and the COTS matcher achieve 99% accuracy on LFW under LFW protocol.. Two of them, FaceNet [2] and CosFace [3], are publicly available. FaceNet is trained on VGGFace2 dataset [50] using the Softmax+Center Loss [2]. CosFace is a 64-layer residual network [51] and is trained on MS-ArcFace dataset [52]

using AM-Softmax loss function 

[52]. Both matchers extract a -dimensional feature vector. We also evaluate results on a commercial-off-the-shelf (COTS) face matcher, COTS121212This particular COTS utilizes CNNs for face recognition and has been used for identifying children in prior studies [39, 17]. COTS is one of the top performers in the NIST Ongoing Face Recognition Vendor Test (FRVT) [16].. This is a closed system so we do not have access to its feature vector.

 

CFA (Constrained) ITWCC (Semi-Constrained) [39]
Method Verification Closed-set Open-set Verification Closed-set Open-set

 

0.1% FAR Rank-1 Rank-1 @ 0.1% FAR 0.1% FAR Rank-1 Rank-1 @ 0.1% FAR

 

COTS [39] 89.96 4.93 95.26 0.77 90.65 0.59 52.23 7.13 86.47 2.25 30.87 5.35

 

FaceNet [2] (w/o feature aging) 37.74 5.16 79.42 0.80 49.37 2.90 21.97 3.60 60.53 2.30 6.30 2.30
FaceNet (with feature aging) 48.92 3.93 84.07 0.56 56.72 0.46 25.72 4.41 68.19 0.91 5.63 1.83

 

CosFace [3] (w/o feature aging) 81.26 3.38 94.46 0.25 90.23 0.62 49.63 3.41 84.69 2.09 31.27 5.76
CosFace (with feature aging) 89.29 3.82 95.73 0.30 91.89 0.98 51.55 3.74 88.45 1.15 32.88 5.67

 

  • A probe first claims to be present in the gallery. We accept or reject this claim based on a pre-determined threshold @ FAR (verification). If the probe is accepted, the ranked list of gallery images which match the probe with similarity scores above the threshold are returned as the candidate list (identification).

Table 3:

The mean and standard deviation of face recognition performances across 5-folds on CFA and ITWCC 

[39] datasets with and without proposed deep feature aging. The proposed age-progression method improves the performance of FaceNet and CosFace on cross age face matching.

5 Experimental Results

5 years 6 years 8 years 11 years

CFA
3 years 5 years 12 years 13 years
ITWCC [39]

Figure 11: Examples of longitudinal face images from CFA and ITWCC [39] datasets. Each row consists of images of one subject; age at image acquisition is given below each image
(a) CFA
(b) ITWCC
Figure 14: Rank-1 search accuracy for FaceNet [2] and CosFace [3] on (a) CFA and (b) ITWCC datasets along with our proposed age-progression method.

For evaluating child face performance, we utilized two datasets (see Table 1):

  • [topsep=0.2em, itemsep=-0.2em]

  • Children’s Face Aging (CFA) dataset comprises annual school portraits of children varying in the age range of years.

  • In The Wild Child Celebrity (ITWCC) [39] dataset contains images of of child celebrities.

Evaluation Protocol

For evaluating recognition performance on CFA and ITWCC datasets, we use 5-fold cross validation where the subjects are disjoint in each fold. folds are used for training and the th for testing. To compensate for the small size of ITWCC dataset, we augment the training data with CFA when evaluating on ITWCC. As locating missing children is akin to the identification scenario, we compute both the closed-set identification accuracy (recovered child is in the gallery) at rank-1 and the rank-1 open-set identification accuracy (recovered child may or may not be in the gallery) at False Accept Rate. Following the protocol outlined in ITWCC [39], we evaluate our results under the Youngest vs. Oldest scenario, where the youngest image of a child is enrolled in the gallery and the oldest image of the same child is the probe image. The gallery and probe sets contain and images each for CFA and ITWCC datasets, respectively. For the open-set identification scenario, we extend the probe set by adding face images of subjects in the age range of to years from UTKFace [23] dataset. In addition to closed-set and open-set identification results, we also report the verification rate (TAR @ FAR).

Results

We report the mean and the standard deviation across all five folds in Table 3. We find that our age-progression method improves the search accuracy of both FaceNet [2] and CosFace [3]. In addition, with the proposed feature aging module, an open-source face matcher CosFace [3] can outperform the COTS matcher131313CosFace [3] matcher takes about ms to search for a probe in a gallery of images of missing children, while our model takes approximately ms (on a GTX 1080 Ti) to search for a probe through the same gallery size..

Figures (a)a and (b)b show the performance under the Youngest vs Oldest protocol [39]. While our aging model improves matching over all time lapses, its contribution gets larger as the time lapse increases.

In Figure 19, we show some example cases where CosFace [3], without the proposed deep feature aging module, retrieves a wrong child from the gallery at rank-1. With the proposed method, we can correctly identify the true mates and retrieve them at rank-1.

5.1 Comparison with State-of-the-Art

In order to evaluate the generalizability of our module, we train it on CFA and ITWCC [39] datasets and benchmark our performance on a publicly available aging dataset, FG-NET [40], which also contains children. We follow the standard leave-one-out protocol [29, 43] in Table 4. We find that our proposed feature aging module can enhance the performance of CosFace [3]. We also fine-tuned the last layer of CosFace on the same training set, however, the decrease in accuracy (Tab. 4) clearly suggests that moving to a new latent space can inhibit the original features. Our module can boost the performance while still operating in the same feature space as the original matcher.

In addition, we also benchmark our performance on an adult aging dataset, CACD-VS141414Since CACD-VS does not have age labels, we use DEX [53] (a publicly available age estimator) to estimate the ages.. However, note that unlike prior studies [28, 31, 4], we do not fine-tune our model on the CACD-VS dataset. In Table 5, the proposed feature aging module enhances the performance of CosFace [3] on CACD-VS showing that our model also aids in face recognition under aging for adults.

 

Method Rank-1 (%)

 

HFA [43] 69.00%
MEFA [44] 76.20%
CAN [47] 86.50%
LF-CNN [28] 88.10%
AIM [4] 93.20%
Wang et al. [5] 94.50%

 

COTS 93.61%
CosFace [3] (w/o feature aging) 94.91%
CosFace (finetuned on children) 93.71%
CosFace (with feature aging) 95.91%

 

Table 4: Face recognition performance on FG-NET [40].

 

Method Accuracy (%)

 

HFA [43] 84.40%
CARC [41] 87.60%
LF-CNN [28] 98.50%
OE-CNN [31] 99.20%
AIM [4] 99.38%
Wang et al. [5] 99.40%

 

COTS 99.32%
CosFace [3] (w/o feature aging) 99.50%
CosFace (with feature aging) 99.58%

 

Table 5: Face recognition performance on CACD-VS [41].

5.2 Discussion

Visualization

Using a decoder, we visualize the aged images from the age-progressed features via the proposed module in Figure 15 for CosFace [3]151515Note that the decoder does not have any explicit age information and it is trained on a non-cross-age face dataset. Implementation details can be found in supplementary material.. We find that the age-progression module can retain the identity information in the probe’s deep face feature while age-related information is progressed as expected.

Probe: 7 years Age: 15 Age: 17 Age: 19
Figure 15: Face images decoded from age-progressed features (Row 2) along with ground truth face images (ITWCC [39]) (Row 1) at target ages.
Figure 16: The decoded image for a probe feature at age 11 is shown in column 2. The decoded features at the desired ages via our proposed age-progression module are shown in row 1. Row 2 shows the changes in face features when the probe feature is regressed (column 1) and progressed (column 3). Brighter colors indicate major changes in the face features.

Age-Sensitive Features

In Figure 16, we visualize the difference between (a) the decoded image from age-progressed or age-regressed features predicted by our proposed method and (b) the decoded image from the probe feature using CosFace [3]. We find that when a probe feature is regressed to a younger age, our method attempts to reduce the size of the head and the eyes, whereas, age-progression enlarges the head, adds makeup, and adds aging effects such as wrinkles around the cheeks.

6 Two Case Studies of Missing Children

Carlina White was abducted from the Harlem hospital center in New York City when she was just days old. She was reunited with her parents years later when she saw a photo resembling her as a baby on the National Center for Missing and Exploited Children website161616http://www.missingkids.org (see Figure (c)c). We contructed a gallery of missing children consisting of face images in the age range - years from the UTKFace [23] dataset and Carlina’s image as an infant when she went missing (19 days old). Her face image when she was later found (23 years old) was used as the probe. State-of-the-art face matchers, CosFace [3] and COTS, were able to retrieve probe’s true mate at ranks and respectively. It is infeasible for a human operator to look through such a large number of retrieved images to ascertain the true mate. With the proposed feature aging module, CosFace is able to retrieve the true mate at rank 268, which is a significant improvement in narrowing down the search.

In another missing child case, Richard Wayne Landers was abducted by his grandparents at age in July 1994 in Indiana. In 2013, investigators identified Richard (then, years old) through a Social Security database search (see Figure (b)b). Akin to Carlina’s case, adding Richard’s year old face image in the gallery and keeping his face image at age as the probe, CosFace [3] was able to retrieve his younger image at rank 23. With the proposed feature aging, CosFace was able to retrieve his younger image at rank 1.

(a) ITWCC
(b) Richard Landers: A case study [54] (see Section 6)
Figure 19: Identities incorrectly retrieved at Rank-1 by CosFace [3] without our proposed age-progression module (highlighted in red). CosFace with the proposed method can correctly retrieve the true mates in the gallery at rank-1 (highlighted in green)

These examples show the applicability of our feature aging module to real world missing children cases. By improving the search accuracy of any face matcher in children-to-adult matching, our model makes a significant contribution to the social good by reuniting missing children with their loved ones.

7 Summary

We propose a new method for aging deep face features that can be used as a wrapper around any commodity face matcher to enhance the longitudinal face recognition performance in identifying missing children. The proposed method boosts the rank-1 identification accuracies of FaceNet from 40.00% to 49.56% and CosFace from 56.88% to 61.25% on a child celebrity dataset, namely, ITWCC. Moreover, with the proposed method, rank-1 accuracy of CosFace on a public aging face dataset, FG-NET, increases from 94.91% to 95.91%, outperforming state-of-the-art. These results suggest that aging face features can enhance the ability of commodity face matchers to locate and identify young children who are lost at a young age in order to reuinte them back with their families. We plan to extend our work to unconstrained child face images which is typical in child trafficking cases.

References

  • [1] Karl Ricanek, Shivani Bhardwaj, and Michael Sodomsky. A review of face recognition against longitudinal child faces. In IEEE BIOSIG, 2015.
  • [2] Florian Schroff, Dmitry Kalenichenko, and James Philbin. Facenet: A unified embedding for face recognition and clustering. In CVPR, pages 815–823, 2015.
  • [3] Hao Wang, Yitong Wang, Zheng Zhou, Xing Ji, Dihong Gong, Jingchao Zhou, Zhifeng Li, and Wei Liu. Cosface: Large margin cosine loss for deep face recognition. In CVPR, 2018.
  • [4] Jian Zhao, Yu Cheng, Yi Cheng, Yang Yang, Fang Zhao, Jianshu Li, Hengzhu Liu, Shuicheng Yan, and Jiashi Feng.

    Look across elapse: Disentangled representation learning and photorealistic cross-age face synthesis for age-invariant face recognition.

    In AAAI, 2019.
  • [5] Hao Wang, Dihong Gong, Zhifeng Li, and Wei Liu. Decorrelated adversarial learning for age-invariant face recognition. In CVPR, 2019.
  • [6] Convention on the Rights of the Child. https://web.archive.org/web/20101031104336/http://www.hakani.org/en/convention/Convention_Rights_Child.pdf, 1989.
  • [7] Children account for nearly one-third of identified trafficking victims globally. https://uni.cf/2OqsMIt, 2018.
  • [8] Pooling Resources to Fight Child Abuse and Abduction. https://on.wsj.com/32R9Vgz, 2018.
  • [9] NCIC Missing Person and Unidentified Person Statistics. https://www.fbi.gov/file-repository/2018-ncic-missing-person
    -and-unidentified-person-statistics.pdf/view
    , 2018.
  • [10] A life revealed. http://www.nationalgeographic.com/magazine/2002/04/afghan-girl-revealed/, 2002.
  • [11] Patrick J Grother, George W Quinn, and P Jonathon Phillips. Report on the evaluation of 2d still-image face recognition algorithms. NIST Interagency Report, 7709:106, 2010.
  • [12] Sydney R. Coleman and Rajiv Grover. The Anatomy of the Aging Face: Volume Loss and Changes in 3-Dimensional Topography. Aesthetic Surgery Journal, 26:S4–S9, 2006.
  • [13] N. Ramanathan and Rama Chellappa. Modeling age progression in young faces. In CVPR, 2006.
  • [14] Brendan Klare and Anil K Jain. Face recognition across time lapse: On learning feature subspaces. In IEEE IJCB, 2011.
  • [15] Debayan Deb, Lacey Best-Rowden, and Anil K Jain. Face recognition performance under aging. In CVPRW, 2017.
  • [16] Patrick J. Grother, Mei Ngan, and Kayee Hanaoka. Ongoing Face Recognition Vendor Test (FRVT), Part 2: Identification. NIST Interagency Report, 2018.
  • [17] Debayan Deb, Neeta Nain, and Anil K Jain. Longitudinal study of child face recognition. In IEEE ICB, 2018.
  • [18] S. Brierley. A Long Way Home: A Memoir. Berkley, 2015.
  • [19] Jaycee Lee Dugard. A Stolen Life: A Memoir. Simon & Schuster, 2011.
  • [20] Vondie Curtis-Hall. Abducted: The Carlina White Story. https://www.imdb.com/title/tt2212658/, 2012.
  • [21] G. Davis. http://lionmovie.com/, 2016.
  • [22] Xin Geng, Zhi-Hua Zhou, and Kate Smith-Miles. Automatic age estimation based on facial aging patterns. IEEE T-PAMI, 29(12):2234–2240, 2007.
  • [23] Zhifei Zhang, Yang Song, and Hairong Qi.

    Age progression/regression by conditional adversarial autoencoder.

    In CVPR, 2017.
  • [24] Zhonghua Zhai and Jian Zhai. Identity-preserving conditional generative adversarial network. In IEEE IJCNN, 2018.
  • [25] Grigory Antipov, Moez Baccouche, and Jean-Luc Dugelay. Face aging with conditional generative adversarial networks. In IEEE ICIP, 2017.
  • [26] Hongyu Yang, Di Huang, Yunhong Wang, and Anil K Jain. Learning face age progression: A pyramid architecture of gans. In CVPR, 2018.
  • [27] Andreas Lanitis, Christopher J. Taylor, and Timothy F Cootes. Toward automatic simulation of aging effects on face images. IEEE T-PAMI, 24(4):442–455, 2002.
  • [28] Chi Nhan Duong, Kha Gia Quach, Khoa Luu, Ngan Le, and Marios Savvides. Temporal non-volume preserving approach to facial age-progression and age-invariant face recognition. In ICCV, 2017.
  • [29] Zhifeng Li, Unsang Park, and Anil K Jain. A discriminative model for age invariant face recognition. IEEE TIFS, 6(3):1028–1037, 2011.
  • [30] Tianyue Zheng, Weihong Deng, and Jiani Hu.

    Age estimation guided convolutional neural network for age-invariant face recognition.

    In CVPRW, 2017.
  • [31] Yitong Wang, Dihong Gong, Zheng Zhou, Xing Ji, Hao Wang, Zhifeng Li, Wei Liu, and Tong Zhang. Orthogonal deep features decomposition for age-invariant face recognition. In ECCV, 2018.
  • [32] Matthew Q Hill, Connor J Parde, Carlos D Castillo, Y Ivette Colon, Rajeev Ranjan, Jun-Cheng Chen, Volker Blanz, and Alice J O’Toole. Deep convolutional neural networks in the face of caricature: Identity and image revealed. arXiv preprint arXiv:1812.10902, 2018.
  • [33] Prithviraj Dhar, Ankan Bansal, Carlos D Castillo, Joshua Gleason, P Jonathon Phillips, and Rama Chellappa. How are attributes expressed in face dcnns? arXiv preprint arXiv:1910.05657, 2019.
  • [34] Michał Bereta, Paweł Karczmarek, Witold Pedrycz, and Marek Reformat. Local descriptors in application to the aging problem in face recognition. Pattern Recognition, 46(10):2634–2646, 2013.
  • [35] Soweon Yoon and Anil K Jain. Longitudinal study of fingerprint recognition. PNAS, 112(28):8555–8560, 2015.
  • [36] Lacey Best-Rowden and Anil K Jain. Longitudinal study of automatic face recognition. IEEE T-PAMI, 40(1):148–162, 2017.
  • [37] Lacey Best-Rowden, Yovahn Hoole, and Anil Jain. Automatic face recognition of newborns, infants, and toddlers: A longitudinal evaluation. In IEEE BIOSIG, pages 1–8, 2016.
  • [38] Patrick J Grother, James R Matey, Elham Tabassi, George W Quinn, and Michael Chumakov. Irex vi-temporal stability of iris recognition accuracy. NIST Interagency Report, 7948, 2013.
  • [39] Nisha Srinivas, Karl Ricanek, Dana Michalski, David S. Bolme, and Michael A. King. Face Recognition Algorithm Bias: Performance Differences on Images of Children and Adults. In CVPR Workshops, 2019.
  • [40] FG-NET dataset. https://yanweifu.github.io/FG_NET_data/index.html, 2014.
  • [41] Bor-Chun Chen, Chu-Song Chen, and Winston H Hsu. Cross-age reference coding for age-invariant face recognition and retrieval. In ECCV, 2014.
  • [42] Karl Ricanek and Tamirat Tesafaye. Morph: A longitudinal image database of normal adult age-progression. In IEEE FG, 2006.
  • [43] Dihong Gong, Zhifeng Li, Dahua Lin, Jianzhuang Liu, and Xiaoou Tang. Hidden factor analysis for age invariant face recognition. In CVPR, 2013.
  • [44] Dihong Gong, Zhifeng Li, Dacheng Tao, Jianzhuang Liu, and Xuelong Li. A maximum entropy feature descriptor for age invariant face recognition. In CVPR, 2015.
  • [45] Zhifeng Li, Dihong Gong, Xuelong Li, and Dacheng Tao. Aging face recognition: A hierarchical learning model based on local patterns selection. IEEE TIP, 25(5):2146–2154, 2016.
  • [46] Haibin Ling, Stefano Soatto, Narayanan Ramanathan, and David W Jacobs. Face verification across age progression using discriminative methods. IEEE TIFS, 5(1):82–91, 2009.
  • [47] Chenfei Xu, Qihe Liu, and Mao Ye. Age invariant face recognition and retrieval by coupled auto-encoder networks. Neurocomputing, 222:62–71, 2017.
  • [48] Wei Wang, Zhen Cui, Yan Yan, Jiashi Feng, Shuicheng Yan, Xiangbo Shu, and Nicu Sebe. Recurrent Face Aging. In CVPR, 2016.
  • [49] Paul Upchurch, Jacob Gardner, Geoff Pleiss, Robert Pless, Noah Snavely, Kavita Bala, and Kilian Weinberger. Deep feature interpolation for image content changes. In CVPR, 2017.
  • [50] Qiong Cao, Li Shen, Weidi Xie, Omkar M Parkhi, and Andrew Zisserman. Vggface2: A dataset for recognising faces across pose and age. In IEEE FG, 2018.
  • [51] Weiyang Liu, Yandong Wen, Zhiding Yu, Ming Li, Bhiksha Raj, and Le Song. Sphereface: Deep hypersphere embedding for face recognition. In CVPR, 2017.
  • [52] Jiankang Deng, Jia Guo, Niannan Xue, and Stefanos Zafeiriou. Arcface: Additive angular margin loss for deep face recognition. In CVPR, 2019.
  • [53] Rasmus Rothe, Radu Timofte, and Luc Van Gool. Dex: Deep expectation of apparent age from a single image. In ICCV Workshop.
  • [54] Lawyer: Mom of abducted Indiana boy wasn’t homeless. https://bit.ly/3742yEU, 2013.