1 Introduction
Age progression [1], also called age synthesis [2] or face aging [3], is defined as aesthetically rendering a face image with natural aging and rejuvenating effects for an individual face. It has been widely applied to various application domains, e.g., crossage face analysis [4], authentication systems, finding lost children, entertainment, etc. There are two main categories of solutions to the age progression task: prototypingbased age progression [5, 6, 7] and modelingbased age progression [3, 8, 9]. Prototypingbased age progression transfers the difference between two prototypes (e.g., average faces) of the predivided source age group and the target age group into the input individual face, of which its age belongs to the source age group. Modelingbased age progression models the facial parameters of different ages (age ranges) for the shape and texture synthesis.
Intuitively, the natural aging process of a specific person usually follows the general rules of the human aging process. Meanwhile, the natural aging face of a specific person also contains some personalized facial characteristics, e.g., mole, birthmark, etc., which are almost invariant with time. Generally, prototypingbased age progression methods cannot well preserve this personality of an individual face, since they are based on the general rules of the human aging process for a relatively large population. And modelingbased age progression methods do not specially consider the personalized details for a specific person. Moreover, they require a large number of dense longterm (e.g., age span exceeds 20 years) face aging sequences for building the complex functions. However, collecting these required sequences in the real world is very difficult or even impossible. Fortunately, we have observed that the shortterm (e.g., age span smaller than 10 years) face aging sequences are available in the Internet, such as photos of celebrities at different ages on Facebook/Twitter. Some available face aging datasets [10, 11, 12] also contain the dense shortterm sequences. Therefore, generating personalized age progression for an individual face by leveraging shortterm face aging sequences is more feasible.
In this work, we propose a Bilevel Dictionary Learning based Personalized Age Progression (BDLPAP) method, which leverages shortterm face aging pairs to automatically render aging faces in a personalized way, as shown in Figure 1. Primarily, based on the aging(in)variant patterns in the face aging process, an individual face can be decomposed into an aging layer and a personalized layer. The former shows the general aging characteristics (e.g., wrinkles), while the latter shows the personalized facial characteristics (e.g., mole). For different human age groups (e.g., 1115, 1620, etc), we design the corresponding aging dictionaries to reveal the general aging characteristics, where the dictionary bases with the same index yet from two neighboring aging dictionaries form a particular aging patterns (e.g., they are linked by a dotted line in Figure 1). Therefore, the aging layer of the aging face can be reconstructed by a linear combination of several aging dictionary bases with sparse representation. The motivation of the sparsity assumption is to use fewer dictionary bases for reconstruction such that the reconstructed aging layer of face can be shaper and less blurred [13, 14]. The residual between the reconstructed aging layer and the input face can be defined as the personalized layer, which is invariant in the aging process. Finally, we render the aging face in a future age group for an input face by fusing the reconstructed aging layer of this age group and the personalized layer.
To learn the aging dictionaries, we propose a bilevel dictionary learning model, and utilize the more practical shortterm face aging pairs as the training set instead of the possibly unavailable longterm face aging sequences. In the bilevel dictionary learning, we assume that the sparse representation of a youngeraging layer of one face w.r.t. the youngeraging dictionary can directly reconstruct its olderaging layer using the olderaging dictionary. The distribution of the collected face aging pairs in this work is shown in the upper part of Figure 2. We can see that: 1) each age group has its own aging dictionary; 2) every two neighboring age groups are linked by several dense shortterm face aging pairs, which makes all the age groups linked together; 3) the personalized layer contains the personalized facial characteristics. These three properties are able to guarantee that all aging dictionaries can be trained well by the bilevel dictionary learning on the shortterm face aging pairs.
The main contributions of this work are summarized as follows:

We propose a Personalized Age Progression method based on the Bilevel Dictionary Learning (BDLPAP) to render aging faces, which can preserve the personalized facial characteristics.

Since it is challenging or even impossible to collect intraperson face sequences of all age groups, the proposed method only requires the easyacquired shortterm face aging pairs to learn all aging dictionary bases of human aging, which is more feasible.

Extensive experiments well validate the advantage of the proposed solution over other stateofthearts in term of personalized aging progression, as well as the performance gain for crossage face verification by synthesizing aging faces.
Compared to our preliminary work in [15], we have the following extensions in this paper: 1) we extend the preliminary coupled dictionary learning model to a novel bilevel dictionary learning model, which leads to a more effective and efficient personalized age progression method; 2) we compare the proposed age progression method with more stateoftheart methods by conducting age progression experiments and crossage face verification experiments; 3) we illustrate the effectiveness of the proposed personalized age progression method by comparing to its unpersonalized version; 4) we show the efficiency of the proposed method by comparing the executing time of different age progression methods.
The rest of this paper is organized as follows. Section 2 reviews related works. Section 3 details the proposed method, including the whole framework, dictionary learning model, objective function, optimization procedure, and age progression synthesis. Experiments are conducted in Section 4. Finally, Section 5 concludes this work and discusses the future works.
2 Related Work
2.1 Age Progression Methods
Age progression has been comprehensively reviewed in literature [2, 1, 16, 17]. As one of the early studies, Burt et al. [18] focused on creating average faces for different ages and transferring the facial difference between the average faces into the input face. This method gives an insight into the age progression task. Thereafter, some prototypingbased aging methods [5, 6, 19] are proposed to improve the age progression method in [18]. Kemelmacher et al. [5] proposed an IlluminationAware Age Progression (IAAP) method by leveraging the difference between the warped average faces instead of the original average faces. Here the warped average face is computed based on the flow from the average face to the input face. Generally, the aging progression of an individual is stochastic and nondeterministic in the time dimension. Therefore, Shu et al. [19] presented a kinshipguided age progression approach to automatically generate aging faces by leveraging kinship information. The aging faces generated by these methods almost have no personalized characteristic and the aging speed of different people is synchronous. Although some researchers tried to consider the individualspecific face age progression [20, 21, 22], lack of personality is still a challenging problem.
Modelingbased age progression is the other type of the age progression, which considers shape and texture synthesis simultaneously [23]. Some modelingbased age progression methods have been proposed, including active appearance model [24], craniofacial growth model [25], andor graph model [3], statistical model [26], implicit function [27, 28], etc. Generally, to model large appearance changes over a longterm face aging sequence, modelingbased age progression requires sufficient training data. Suo et al. [29]
attempted to learn longterm aging patterns from available shortterm aging databases by a proposed concatenational graph evolution aging model. Recent years, inspired by the success of Recurrent Neural Network (RNN), Wang et al.
[30]exploited RNNs to model the whole aging sequence. The bottom layer in a RNN works as an encoder, which projects the image to a highdimension space. The top layer in a RNN works as a decoder, which decodes the hidden representation to an aging face.
2.2 Age Progression based Face Analysis
Face analysis (e.g., face verification and face recognition)
[31, 32, 33] has achieved great progress in the last decade. Usually, as a natural biological process, face aging can change the appearance and texture of the facial landmarks, such as wrinkles, senile plaque, mustache, etc. Therefore, the performance of existing face analysis methods for the crossage face analysis will degree since the age gap exists between the two faces of one person at different ages. By conducting experiments on a passport photo database, Ling et al. [34] validated that the face aging would increase the challenge of face recognition. Towards this end, some methods are proposed to address the crossage face analysis problem in recent years [10, 35, 36, 4, 37, 34, 38, 39, 40, 41]. These methods can be divided into two categories. Methods of one category aim to learn the ageinvariant features for crossage face analysis [38, 34, 10, 42]. For example, Ling et al. [34] found that the gradient direction is insensitive to the face aging, and then proposed to use the gradient directions of pyramid structure as the feature descriptors of face images. Gong et al. [42] proposed to separate the face pattens into the ageinvariant and agevariant patterns, and then use the ageinvariant patterns as the inputs of learning model. We can conclude that how to learn the ageinvariant features or extract the ageinvariant patterns of face images is the key point. Methods of the other category are proposed to use the age synthesis to eliminate the age gap [39, 36, 37, 4, 43, 44, 3, 29, 45]. Taking the face verification as an example, for two faces of one person at different ages, the age progression method renders a synthesized aging face from the younger face. And then, the synthesized face instead of the original face is used to implement the face verification.2.3 Face Aging Datasets
FGNET [11]
is one popular face aging dataset. It has been used to evaluate the age estimation
[46, 47], crossage face verification, crossage face recognition [48, 37], and age progression [5, 29, 3, 49]. This dataset contains 1,002 face photos from 82 persons within 069 age range: about 64% of the images are from children (with ages 18), and around 36% are from adults (with ages 18). All photos are taken by digital camera and filmbased camera. Morph [12] is one of the largest face aging dataset. It has been also used to evaluate the age estimation, age progression, and face recognition. It is composed of Album1 subset and Album2 subset. The Album1 subset contains 1,690 face images from 515 persons. And the Album2 subset contains 94,000 face images from 24,000 people, which are collected from different places. In Morph, the size of all images is resized to or , and each image has the corresponding age, gender and race labels. Recently, about 55,000 face images from 13,000 people in Morph are available on the website. In recent years, Chen et al. [10] released a new CrossAge Celebrity Dataset (CACD), which is collected from image search websites. CACD contains 163,446 face images from 2,000 celebrities within 1662 age range. The age label of each face photo is estimated by referring to its shooting time, namely the shooting time is the age label. Although these age labels are inaccurate, the relative age for the same person is accurate. Thus, this dataset can be used for the crossage face search and recognition.3 The Proposed Method
3.1 Overview of the Framework
The framework of the proposed age progression method is shown in Figure 2. The offline phase is described as follows. First, we collect the dense shortterm aging pairs of the same persons from Internet and also from available datasets. Second, for each predefined age group, we learn a corresponding aging dictionary to represent its aging characteristics by the proposed bilevel dictionary learning model. In the online phase, for an input face, we generate its aging faces step by step, from the current age to the target age. Specifically, we first generate the aging face in the next age group by the corresponding aging dictionary with an implicitly common sparse representation, as well as a personalized layer. After that, taking this new aging face as the input, and repeat the above process until all aging faces have been rendered.
3.2 Coupled Dictionary Learning
We divide the human age range into age groups (each group spans less than 10 years) in this work. Let denote a selected face aging sequence of the th person covering all age groups, where ( is the number for the persons). Here, the face photo falls into the th age group, where is the number of pixels in a face photo. For the th age group (), we define its aging dictionary to capture the aging characteristics, which will be learned in the following.
Personalityaware formulation. The aging dictionary learning in this work considers the personalized details of an individual when representing the face aging sequences on their own aging dictionaries. Since the personalized characteristics are aginginvariant, such as mole, birthmark, permanent scar, etc., we plan to add a personalized layer for a face aging sequence to indicate the personalized details of the th person in the aging process. Moreover, considering the computational efficiency, we utilize PCA to reduce the dimension of the dictionary. Let denote a PCA projected matrix of all data in the th age group, and we have . Thus the original aging dictionary is redefined as , where denotes the number of dictionary bases. All aging dictionaries compose an overall aging dictionary , where . So far, the aging face of equals the linearly weighted combination of the aging dictionary bases in the ()th age group and the personalized layer , i.e., for , where and are the common sparse representation and the personalized layer, respectively. For face aging sequences covering all age groups, a personalityaware dictionary learning model is formulated as follows,
(1)  
where denotes the th column (base) of , and control the sparsity penalty and regularization, respectively. is used to represent the specific aging characteristics in the th age group.
Shortterm coupled learning. We observe that one person always has the dense shortterm face aging photos, but no longterm face aging photos covering all age groups. Collecting these longterm face aging sequences in the real world is extremely difficult or even unlikely. Therefore, we have to use the shotterm face aging pairs instead of the longterm face sequences. Here, each face aging pair includes a certain person’s two face images spanning two neighboring age groups. Let () denote a set of face images in the th age group, we assume that and share face pairs from different persons. Formally, we use to denote the face pair of the th person, where is a local index related to the th age group. For the face aging pairs spanning the th and ()th age groups, we reformulate a coupled dictionary learning model to simultaneously learn all aging dictionaries, i.e.,
(2)  
In Eq. (2), every two neighboring aging dictionaries and corresponding to two age groups are implicitly coupled via the common sparse representation , and the personalized layer is to capture the personalized details of the th person in the th age group, who has the face pair spanning the th and ()th age groups. Let , , , , and , and Eq. (2) can be rewritten in the matrix form
(3)  
where .
In Eq. (3), the optimization w.r.t. has the closedform solution by fixing and , where . By fixing , the optimization w.r.t. and becomes a joint sparse coding problem [50]. Such joint sparse coding problem can be solved in the concatenated feature space of aging layers and , but not in each feature space separately, namely we have
(4) 
which can be effectively solved by the SPAMS toolbox^{1}^{1}1http://spamsdevel.gforge.inria.fr/.
3.3 Bilevel Dictionary Learning
If the aging dictionaries are learned by the coupled dictionary learning model, for an input face belonging to the th age group, its sparse representation and personalized layer should be computed to generate its aging face aging face in the ()th age group. According to Eq. (3), and should be calculated by solving the following problem,
(5) 
Unfortunately, is unavailable for a new coming face in practice. Thus, there is no way to enforce the equivalence constraint on the representations of and , as has been done in the training phase of Eq. (3). To handle this problem, an intuitive way it to use the average face in the target age group to replace , as done in the preliminary work [15]. However, this solution has two drawbacks. First, since the average face and the target aging face have the different facial characteristics, the aging result is not satisfied. Second, to obtain the desired aging faces, we need to repeat the process of age progression synthesis several times to improve the aging performance, which is timeconsuming.
To better address the above problem, let us revisit our primary goal again. We want to learn the aging dictionaries to ensure that the sparse representations of two faces in each pair, as well as their personalized layers, are the same. If so, we can generate the aging face for an input face by using the dictionary , the sparse representation and the personalized layer , i.e., . Formally, an ideal pair of every two neighboring aging dictionaries and should satisfy the following two equations for all face pairs spanning the th and ()th age groups,
(6)  
(7) 
Based on the above analysis, we should first learn and by Eq. (6). And then and should be learned to satisfy Eq. (6) and Eq. (7). To this end, we can reformulate our objective function as
(8) 
which is a bilevel optimization problem [51].
Rewrite Eq. (8) in the matrix form after some algebratic steps, we can obtain a novel bilevel dictionary learning model for all face pairs spanning the th and ()th age groups ():
(9) 
where , , , and controls the regularization.
According to the new formulation in Eq. (9), for a new coming face , the calculations of its sparse representation and personalized layer are independent with the unavailable , while they only depend on the available face .
3.4 Optimization Procedure
The objective function in Eq. (9) is convex w.r.t. and separately, which can be iteratively solved through two alternating subprocedures of optimization. Specifically, we fix the other variables when updating one variable.
3.4.1 Updating
We update by fixing , and then the objective function in Eq. (9) becomes
(10) 
First, we iteratively update and , wherein the problem can be solved by the SPAMS toolbox. And then, the problem in Eq. (10) becomes a Quadratically Constrained Quadratic Program (QCQP) that can be solved by the CVX toolbox^{2}^{2}2http://cvxr.com/cvx/.
3.4.2 Updating
When updating , we fix , and then Eq. (9) becomes a bilevel optimization problem. Similar to [51, 50]
, we solve this bilevel optimization problem based on the firstorder projected stochastic gradient descent. For brevity, we simplify the subscripts of
, , and , i.e., , , and . For , we compute the gradient of with respect to(11) 
where , and . denotes the index set for , and is defined as follows,
Definition 1.
Let , we solve the following problem to obtain the optimizing :
(12) 
If we obtain by Eq. (12), we can define is the index set of nonzero elements of .
In Eq. (11), it is easy to find that
(13) 
where . The detailed deduction of and in Eq. (11) can be found in Appendix A.1 and A.2 of the supplemental material, respectively. Finally, we can obtain the updating way of , as follows,
(14) 
where is the step size.
The proposed bilevel dictionary learning algorithm is summarized in Algorithm 1. The convergence criterion is that the iteration steps shall end when the relative cost of objective function is smaller than a predefined threshold. The convergence curves for solving and on male and female sub datasets respectively are shown in Figure 3. We found that the algorithm achieves convergence after about iterations.
3.5 Age Progression Synthesis
After learning all aging dictionaries , for a given face belonging to the th age group^{3}^{3}3Here, its age range and gender are estimated by an age estimator and a gender recognition system in [52], respectively., we can generate its aging face sequence step by step, from the current age to the target age. Specifically, we first generate the aging face in the next age group (i.e., the ()th age group) by the corresponding aging dictionary with a sparse representation , as well as a personalized layer . Here, and are calculated by Eq. (6) in an alternately iterative way. The iteration steps of optimization in Eq. (6) shall end when the relative cost of objective function is smaller than a predefined threshold.
When we have obtained and , the aging face in the next age group (i.e., the ()th age group) can be generated by the following equation:
(15) 
After that, taking this new aging face in the ()th age group as the input of age synthesis for the ()th age group. We repeat this process until all aging faces are generated.
4 Experiments
4.1 Experimental Settings
Data collection. To train the highquality aging dictionary, it is crucial to collect sufficient and dense shorttime face aging pairs. We download a large number of face photos covering different ages of the same persons from Google and Bing image searches, social media websites(by some content and context links [53]), and other two available face aging datasets, CACD [10] and MORPH [12]. Both CACD and MORPH contain quite a number of shortterm intraperson photos. Since face images from the Internet and CACD dataset are mostly “in the wild”, we select the photos with approximately frontal faces ( to ) and relatively natural illumination and expressions. For all face images, face alignment [54] is implemented to obtain aligned faces, which are cropped into the size . To boost the aging relationship between the neighboring aging dictionaries, we employ Collection Flow [55] to correct all the faces into the common neutral expression. We group all images (the age from 0 to 80) into age groups (i.e., ): 05, 610, 1115, 1620, 2130, 3140, 4150, 5160, and 6180 of two genders, and find that no person has aging faces covering all aging groups. Actually, the aging faces of most persons fall into only one or two age groups (i.e., most persons have face photos spanning no more than 20 years). Therefore, we further select those intraperson face photos which densely fall into two neighboring age groups. Finally, there are 3,200 intraperson face pairs for training (1,600 pairs for males, and 1,600 pairs for females)^{4}^{4}4Dataset is released at http://imag.njust.edu.cn/FAD.html. Every two neighboring age groups for one gender share 400 face aging pairs of the same persons. Since male and female have different aging characteristics, we train aging dictionaries for male and female, respectively.
PCA projection. We stack images in the th age group as columns of a data matrix , where and in experiments. The SVD of is . We define the projected matrix , where is truncated to the rank = , and in experiments.
Parameter setting. The parameters , and in Eq. (3) are empirically set as , and . The number of bases of each aging dictionary is set as .
Aging evaluation. We adopt three strategies to comprehensively evaluate the proposed age progression method. First, we qualitatively evaluate the proposed method on FGNET [11]. We show the age progression for every photo in FGNET, and do the qualitative comparison with the corresponding ground truth (available older photos) for each person. For reference, we also reproduce some aging results of other representative methods. Second, we conduct user study to test the aging faces of the proposed method compared with the prior works which reported their best aging results. The proposed method uses the same inputs as in these prior works. Third, we compare the proposed method by the crossage face verification [42, 39]. Crossage face recognition [10, 56]
and crossage face verification are challenging in extreme facial analysis scenarios due to the age gap, which is similar to the semantic gap between the image and tag in the field of computer vision
[57, 58]. A straightforward way for crossage facial analysis is to use the aging synthesis to reduce the age gap. Specifically, we can synthesize all the faces to their aging faces within the same age range, and then implement the face verification algorithm. In turn, we can also use the face verification to validate whether the intraperson pair of aging face and ground truth face (without age gap) is more similar than the original intraperson face pair with the age gap.4.2 Qualitative Comparison with Ground Truth
Since FGNET provides the groundtruth aging faces, we compare the proposed BDLPAP method with an online Face Transformer demo (i.e., FT Demo^{5}^{5}5http://cherry.dcs.aber.ac.uk/Transformer/), and the representative IlluminationAware Age Progression (IAAP) method [5] on this dataset. FT Demo requires manual location of facial features, while IAAP uses common aging characteristics of average faces for the age progression of all input faces.
Some aging results generated by these three compared methods are given in Figure 4, covering from baby/childhood/teenager (input) to adult/agedness (output), as well as from adult (input) to agedness (output). By comparing with ground truth, we can see that the aging results generated by the proposed BDLPAP look more like the ground truth faces than the aging results of other two methods. In particular, the proposed BDLPAP can generate personalized aging faces for different individual inputs. In term of texture change, the aging face of BDLPAP in Figure 4(a) has no mustache that is closer to ground truth, while the aging face of BDLPAP in Figure 4(b) has white mustache that is closer to ground truth; In term of shape change, the aging faces of BDLPAP in Figure 4(e)(f) have more approximate facial outline to the ground truth; In term of aging speed, the faces of FT Demo and IAAP in Figure 4(d) are aging more slowly, while one of FT Demo in Figure 4(c) is faster. Overall, the age speed of IAAP is slower than ground truth since IAAP is based on smoothed average faces, which maybe loses some facial texture details, such as freckle, nevus, aging spots, etc. FT Demo performs the worst, especially in shape change. Our aging results in Figure 4 are more similar to the ground truth, which means BDLPAP can preserve much more personalized results. Moreover, as shown in Figure 4(g)(h), the aging results of BDLPAP are more robust than other methods for the input faces with noise or low resolution.
4.3 Quantitative Comparison with Prior Works
Some prior works related to the age progression have posted their best face aging results with input faces at different ages, including [3], [59], [1], [37], [60], [61], [9], [62], [63], [36], [30] and [5]. There are 261 aging results with 87 input faces in total. The proposed BDLPAP for each input face is implemented to generate the corresponding aging faces at the same ages (ranges) of the posted results.
We conduct user study to compare the aging results of the proposed BDLPAP with the posted aging results in the prior works, as well as the aging results generated by Coupled Dictionary Learning based Personalized Age Progression (CDLPAP) in the preliminary work [15]. To avoid bias as much as possible, we invite 50 users covering a wide age range and from all walks of life. For each comp arison group including an input face, and three aging results in a random order, all the 50 users are asked to answer the question: which aging face is the best in terms of Personality and Reliability. Reliability means the aging face should be natural and authentic at the synthetic age, while Personality means the aging faces for different inputs should be identitypreserved and diverse. User can choose one from the three aging results as the best, and choose “None” if she/he thinks all the three aging results are unsatisfied. There are 50 ratings for each comparison, 261 comparison groups, and then 13,050 ratings in total. The voting results are as follows: 36.5% for BDLPAP best; 34.8% for CDLPAP best; 26.7% for prior works best; and 2.0% for “none is satisfied”. We show some comparison groups for voting in Figure 5. Overall, for the input face of a person in any age range, BDLPAP, CDLPAP and these prior works can generate an authentic and reliable aging face of any olderage range. In particular, for different inputs, aging faces rendered by BDLPAP and CDLPAP have more personalized aging characteristics, which further improve the appealing visual sense. For example in Figure 5, the aging faces of BDLPAP in the same age range in the first and the second group of the first column have different aging speeds: the former is faster than the latter; the aging faces of prior works with different inputs in the first and second groups of the third column are similar, while the aging results of BDLPAP and CDLPAP are more diverse for different individual inputs. Moreover, we can see that the aging results of BDLPAP are comparable with the aging results of CDLPAP. Specifically, BDLPAP can synthesize the more sharp and highdefinition aging faces than CDLPAP.
4.4 Evaluation on CrossAge Face Verification
To validate the improved performance of crossage face verification with the help of the proposed BDLPAP, we prepare for the intraperson pairs and interperson pairs with cross ages on the FGNET dataset. By removing undetected face photos and face pairs with age span no more than 20 years, we select 1,832 pairs (916 intraperson pairs and 916 interperson pairs), called “Original Pairs”. Among the 1,832 pairs, we render the younger face in each pair to the aging face with the same age of the older face by the proposed BDLPAP. Replacing each younger face with the corresponding aging face, we newly construct 1,832 pairs of the aging face and older face, called “BDLPAP Synthetic Pairs”. Figure 6 shows the comparisons of and the original face pairs and the synthetic pairs by the proposed method, and Figure 7(a) shows the pair setting. To evaluate performance of the proposed BDLPAP, we also prepare the “IAAP Synthetic Pairs”, “RFA Synthetic Pairs”, and “CDLPAP Synthetic Pairs” by IAAP [5], Recurrent Face Aging (RFA) [30], and CDLPAP [15], respectively.
Pair settings  Original Pairs  IAAP  RFA  CDLPAP  BDLPAP 
Synthetic Pairs  Synthetic Pairs  Synthetic Pairs  Synthetic Pairs  
EER (%)  14.89  10.36  8.69  8.53  8.06 
The detailed implementation of face verification is given as follows. First, we formulate a face verification model with deep Convolutional Neural Networks (deep ConvNets), which is based on DeepID2
[64]. Since we focus on the age progression in this work, please refer to [64, 65] for more details of face verification with deep ConvNets. Second, we train this face verification model on the LFW dataset [66], which is constructed for face verification. Third, we test the face verification on different face pairs.The False Acceptance Rate  False Rejection Rate (FARFRR) curves and the Equal Error Rates (EER) on original pairs and synthetic pairs are shown in Figure 7(b) and Table I, respectively. We can see that the face verification on BDL Synthetic Pairs achieves the lower ERR than on Original Pairs, IAAP Synthetic Pairs, RFA Synthetic Pairs, and CDLPAP Synthetic Pairs. This illustrates that the aging faces by the proposed age progression method can effectively mitigate the effect of the age gap in crossage face verification. The results also validate that, for a given input face, BDLPAP can render a personalized and authentic aging face closer to the ground truth than the other compared methods.
4.5 Effect of Personalized Layer
To demonstrate the superiority of the personalization introduced by the proposed BDLPAP, we conduct both qualitative and quantitative comparisons of BDLPAP and its unpersonalized version BDLAP on FGNET. The proposed BDLPAP can be degenerated into BDLAP by the following two steps: 1) in the optimization procedure of Eq. (9), we set
to be a fullzero matrix when initializing and updating it in each iteration; 2) in the synthesis of age progression, we directly set
to be a fullzero vector in Eq. (
6).We first qualitatively compare the experimental results of BDLPAP and BDLAP, which are illustrated in Figure 8. We can see that the aging results generated by BDLPAP are more similar to the ground truth than the counterparts of BDLAP. In particular, compared with BDLAP, BDLPAP can preserve much more personalized facial characteristics, such as the special eyebrow shape in the topleft face and the mole in the fifth face of the first row, which are preserved by BDLPAP but discarded by BDLAP.
We also use the twoway ANalysis Of VAriance (ANOVA)
[67] test to quantitatively compare BDLPAP and BDLAP. To avoid bias as much as possible, we invite 30 users covering a wide range of age and from all walks of life to browse 50 comparison groups, each of which includes the aging results of BDLPAP and BDLAP, the input face and the ground truth. Two aging faces in each comparison group are shown to users with a random order. For each comparison, if one user thinks the two results are comparable with each other, BDLPAP and BDLAP are assigned with score 1 respectively; if the user thinks one aging result is better than the other one, the method corresponding to the better aging result is assigned with score 2, and the other method is assigned with score 0. The comparison results are given in Table II. We can see that BDLPAP gets a score , which is much higher than the score obtained by BDLAP. This result validates the effectiveness of bringing in the personalization. Moreover, the values of ANOVA test show that this superiority is statistically significant and the difference of the users is insignificant.BDLPAP vs. BDLAP  Factor of approaches  Factor of users  

BDLPAP  BDLAP  statistic  value  statistic  value 
The left part illustrates the average rating scores and standard deviation values from the user study on the comparisons of BDLPAP and BDLAP . The right part shows the ANOVA test results.
4.6 Comparison of running time
Another advantage of BDLPAP lies in the significant reduction of the running time of age progression synthesis compared with CDLPAP [15]. To illustrate this advantage of BDLPAP, we test the running time of BDLPAP and CDLPAP on a PC with CPU Intel Core i7 3.6 GHz and 6 GB memory. Table III shows the comparison of the running time. When we generate an aging face in the “610” age group from an input face in the “05” age group, BDLPAP and CDLPAP take 2.2s and 13.5s, respectively. And when we generate an aging face in the “6180” age group from the input face in the “05” age group, BDLPAP and CDLPAP take 17.6s and 108.0s, respectively. We can see that BDLPAP is much more efficient than CDLPAP, and the running time of BDLPAP and CDLPAP increases accompanied with the increment of the age difference between the input face and the aging face. Moreover, we compare the running time of IAAP [5] and BDLPAP. Since FT Demo requires a series of manual operations, we do not compare its running time. Given an input face in the “05” age group, generating its aging face sequence covering all the future age groups needs about 72.0s and 17.6s by IAAP and BDLPAP, respectively. Thus, BDLPAP is also much faster than IAAP.
5 Conclusions and Future Work
In this work, we proposed a personalized age progression method. Basically, we design multiple aging dictionaries for different age groups, in which the dictionary bases from two neighboring dictionaries respectively form a particular aging process pattern across different age groups, and a linear combination of these patterns expresses a particular aging process. Moreover, we define the aging layer and the personalized layer for an individual to capture the aging characteristics and the personalized characteristics, respectively. We train all aging dictionaries on the collected shortterm aging database. Specifically, the younger and older age face pairs of the same persons are used to train two aging dictionaries in the neighboring age groups with the common sparse representation, excluding the specific personalized layer. Given a face, we render its aging face sequence from the current age to the future age step by step on the learned aging dictionaries. In future, we consider the face antiaging synthesis, namely restoring the younger face for a given older face.
6 Acknowledgments
This work was partially supported by the 973 Program (Project No. 2014CB347600), the National Natural Science Foundation of China (Grant No. 61522203, 61572252 and 61672285), the Natural Science Foundation of Jiangsu Province (Grant No. BK20140058 and BK20150755), and the National Ten Thousand Talent Program of China (Young TopNotch Talent). Jinhui Tang is the corresponding author.
References
 [1] C. R. B. S. Ramanathan, Narayanan, “Age progression in human faces: A survey,” J. of Visual Languages and Computing, vol. 15, pp. 3349–3361, 2009.
 [2] Y. Fu, G. Guo, and T. S. Huang, “Age synthesis and estimation via faces: A survey,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 32, no. 11, pp. 1955–1976, 2010.
 [3] J. Suo, S.C. Zhu, S. Shan, and X. Chen, “A compositional and dynamic model for face aging,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 32, no. 3, pp. 385–401, 2010.
 [4] U. Park, Y. Tong, and A. K. Jain, “Ageinvariant face recognition,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 32, no. 5, pp. 947–954, 2010.
 [5] I. KemelmacherShlizerman, S. Suwajanakorn, and S. M. Seitz, “Illuminationaware age progression,” in CVPR, 2014.
 [6] B. Tiddeman, M. Burt, and D. Perrett, “Prototyping and transforming facial textures for perception research,” Computer Graphics and Applications, vol. 21, no. 5, pp. 42–50, 2001.
 [7] M. R. Gandhi, “A method for automatic synthesis of aged human facial images,” Ph.D. dissertation, McGill University, Canada, 2004.
 [8] Y. Tazoe, H. Gohara, A. Maejima, and S. Morishima, “Facial aging simulator considering geometry and patchtiled texture,” in ACM SIGGRAPH, 2012.
 [9] Y. Liang, Y. Xu, L. Liu, S. Liao, and B. Zou, “A multilayer model for face aging simulation,” in Trans. edutainment VI, 2011, pp. 182–192.
 [10] B.C. Chen, C.S. Chen, and W. H. Hsu, “Crossage reference coding for ageinvariant face recognition and retrieval,” in ECCV, 2014.
 [11] “Face and gesture recognition,” in Network: FGNET Aging Database,http://sting.cycollege.ac.cy/alanitis/fgnetaging/, 2000.
 [12] K. Ricanek and T. Tesafaye, “Morph: A longitudinal image database of normal adult ageprogression,” in FG, 2006.
 [13] J. Wright, A. Y. Yang, A. Ganesh, S. S. Sastry, and Y. Ma, “Robust face recognition via sparse representation,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 31, no. 2, pp. 210–227, 2009.
 [14] X. Shu, J. Tang, G.J. Qi, Z. Li, Y.G. Jiang, and S. Yan, “Image classification with tailored finegrained dictionaries,” IEEE Trans. Circuits and Systems for Video Technology, 2016.
 [15] X. Shu, J. Tang, H. Lai, L. Liu, and S. Yan, “Personalized age progression with aging dictionary,” in ICCV, 2015.
 [16] N. Ramanathan, R. Chellappa, and S. Biswas, “Computational methods for modeling facial aging: A survey,” J. of Visual Languages and Computing, vol. 20, no. 3, pp. 131–144, 2009.
 [17] X. Shu, G. Xie, Z. Li, and J. Tang, “Age progression: Current technologies and applications,” Neurocomputing, vol. 208, pp. 249–261, 2016.
 [18] D. M. Burt and D. I. Perrett, “Perception of age in adult caucasian male faces: Computer graphic manipulation of shape and colour information,” Proceedings of the Royal Society of London, 1995.
 [19] X. Shu, J. Tang, H. Lai, Z. Niu, and S. Yan, “Kinshipguided age progression,” Pattern Recognition, vol. 59, pp. 156–167, 2016.
 [20] C. M. Scandrett, C. J. Solomon, and S. J. Gibson, “A personspecific, rigorous aging model of the human face,” Pattern Recognition Letters, vol. 27, no. 15, pp. 1776–1787, 2006.

[21]
D. Hubball, M. Chen, and P. W. Grant, “Imagebased aging using evolutionary computing,” in
Computer Graphics Forum, vol. 27, no. 2, 2008, pp. 607–616.  [22] D. W. Hunter, “Synthesis of facial ageing transforms using threedimensional morphable models,” Ph.D. dissertation, University of St Andrews, 2009.
 [23] A. M. Albert, K. Ricanek Jr, and E. Patterson, “A review of the literature on the aging adult skull and face: Implications for forensic science research and applications,” Forensic Science International, vol. 172, no. 1, pp. 1–9, 2007.
 [24] A. Lanitis, C. J. Taylor, and T. F. Cootes, “Toward automatic simulation of aging effects on face images,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 24, no. 4, pp. 442–455, 2002.
 [25] N. Ramanathan and R. Chellappa, “Face verification across age progression,” IEEE Trans. Image Processing, vol. 15, no. 11, pp. 3349–3361, 2006.
 [26] P. Paysan, “Statistical modeling of facial aging based on 3d scans,” Ph.D. dissertation, University of Basel, 2010.
 [27] A. C. Berg, F. Perales, and L. Gonzalez, “A facial aging simulation method using flaccidity deformation criteria,” in ICIV, 2006.
 [28] G. Schroeder, L. Magalhes, and R. Rodrigues, “Facial aging using image warping,” in Western NY Image Processing Workshop, 2007.
 [29] J. Suo, X. Chen, S. Shan, W. Gao, and Q. Dai, “A concatenational graph evolution aging model,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 34, no. 11, pp. 2083–2096, 2012.
 [30] W. Wang, Z. Cui, Y. Yan, J. Feng, S. Yan, X. Shu, and N. Sebe, “Recurrent face aging,” in ICCV, 2015.

[31]
Y. Sun, X. Wang, and X. Tang, “Deep learning face representation from predicting 10,000 classes,” in
CVPR, 2014.  [32] F. Schroff, D. Kalenichenko, and J. Philbin, “Facenet: A unified embedding for face recognition and clustering,” CVPR, 2015.
 [33] C. Ding, J. Choi, D. Tao, and L. S. Davis, “Multidirectional multilevel dualcross patterns for robust face recognition,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 38, no. 3, pp. 518–531, 2016.
 [34] H. Ling, S. Soatto, N. Ramanathan, and D. W. Jacobs, “A study of face recognition as people age,” in ICCV, 2007.
 [35] G. Guo, G. Mu, and K. Ricanek, “Crossage face recognition on a very large database: the performance versus age intervals and improvement using soft biometric traits,” in ICPR, 2010.
 [36] J. Wang, Y. Shang, G. Su, and X. Lin, “Age simulation for face recognition,” in ICPR, 2006.
 [37] U. Park, Y. Tong, and A. K. Jain, “Face recognition with temporal invariance: A 3d aging model,” in FG, 2008.
 [38] D. Gong, Z. Li, D. Tao, J. Liu, and X. Li, “A maximum entropy feature descriptor for age invariant face recognition,” in CVPR, 2015.
 [39] T. Wu, P. Turaga, and R. Chellappa, “Age estimation and face verification across aging using landmarks,” IEEE Trans. Information Forensics and Security, vol. 7, no. 6, pp. 1780–1788, 2012.
 [40] A. Lanitis, “A survey of the effects of aging on biometric identity verification,” International Journal of Biometrics, vol. 2, no. 1, pp. 34–52, 2009.
 [41] H. Ling, S. Soatto, N. Ramanathan, and D. W. Jacobs, “Face verification across age progression using discriminative methods,” IEEE Trans. Information Forensics and Security, vol. 5, no. 1, pp. 82–91, 2010.
 [42] D. Gong, Z. Li, D. Lin, J. Liu, and X. Tang, “Hidden factor analysis for age invariant face recognition,” in ICCV, 2013.
 [43] E. Patterson, K. Ricanek, M. Albert, and E. Boone, “Automatic representation of adult aging in facial images,” in Int’l Conf. on Visualization, Imaging and Image Processing, 2006.
 [44] J. Suo, X. Chen, S. Shan, and W. Gao, “Learning long term face aging patterns from partially dense aging databases,” in ICCV, 2009.
 [45] L. Liu, C. Xiong, H. Zhang, Z. Niu, M. Wang, and S. Yan, “Deep aging face verification with large gaps,” IEEE Trans. Multimedia, vol. 18, no. 1, pp. 64–75, 2016.
 [46] X. Geng, Z.H. Zhou, and K. SmithMiles, “Automatic age estimation based on facial aging patterns,” IEEE Trans. Pattern Analysis and Machine Intelligence, vol. 29, no. 12, pp. 2234–2240, 2007.
 [47] G. Guo, G. Mu, Y. Fu, and T. S. Huang, “Human age estimation using bioinspired features,” in CVPR, 2009.

[48]
K.Y. Chang, C.S. Chen, and Y.P. Hung, “Ordinal hyperplanes ranker with cost sensitivities for age estimation,” in
CVPR, 2011.  [49] J. Suo, F. Min, S. Zhu, S. Shan, and X. Chen, “A multiresolution dynamic model for face aging simulation,” in CVPR, 2007.

[50]
J. Yang, Z. Wang, Z. Lin, S. Cohen, and T. Huang, “Coupled dictionary training for image superresolution,”
IEEE Trans. Image Processing, vol. 21, no. 8, pp. 3467–3478, 2012.  [51] J. Yang, Z. Wang, Z. Lin, X. Shu, and T. S. Huang, “Bilevel sparse coding for coupled feature spaces,” in CVPR, 2012.
 [52] S. Li, J. Xing, Z. Niu, S. Shan, and S. Yan, “Shape driven kernel adaptation in convolutional neural network for robust facial trait recognition,” in CVPR, 2015.
 [53] G. Qi, C. C. Aggarwal, Q. Tian, H. Ji, and T. S. Huang, “Exploring context and content links in social media: A latent space method,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 5, pp. 850–862, 2012.
 [54] P. Viola and M. Jones, “Rapid object detection using a boosted cascade of simple features,” in CVPR, 2001.
 [55] I. KemelmacherShlizerman and S. M. Seitz, “Collection flow,” in CVPR, 2012.
 [56] D. Yadav, M. Vatsa, R. Singh, and M. Tistarelli, “Bacteria foraging fusion for face recognition across age progression,” in CVPR, 2013.
 [57] G. Qi, W. Liu, C. C. Aggarwal, and T. S. Huang, “Joint intermodal and intramodal label transfers for extremely rare or unseen classes.” IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 1–1, 2016.

[58]
G. Qi, X. Hua, Y. Rui, J. Tang, and H. Zhang, “Twodimensional multilabel active learning with an efficient online adaptation model for image classification,”
IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 31, no. 10, pp. 1880–1897, 2009.  [59] K. Scherbaum, M. Sunkel, H.P. Seidel, and V. Blanz, “Prediction of individual nonlinear aging trajectories of faces,” in Computer Graphics Forum, vol. 26, no. 3, 2007, pp. 285–294.
 [60] E. Patterson, A. Sethuram, M. Albert, and K. Ricanek, “Comparison of synthetic face aging to age progression by forensic sketch artist,” in Int’l Conf. on Visualization, Imaging and Image Processing, 2007.
 [61] Y. Liang, C. Li, H. Yue, and Y. Luo, “Age simulation in young face images,” in Bioinformatics and Biomedical Engineering, 2007, pp. 494–497.
 [62] C.T. Shen, W.H. Lu, S.W. Shih, and H.Y. M. Liao, “Exemplarbased age progression prediction in children faces,” in ISM, 2011.
 [63] A. Sethuram, K. Ricanek, and E. Patterson, “A hierarchical approach to facial aging,” in CVPR, 2010.
 [64] Y. Sun, Y. Chen, X. Wang, and X. Tang, “Deep learning face representation by joint identificationverification,” in NIPS, 2014.
 [65] Y. Taigman, M. Yang, M. Ranzato, and L. Wolf, “Deepface: Closing the gap to humanlevel performance in face verification,” in CVPR, 2014.
 [66] G. B. Huang, M. Ramesh, T. Berg, and E. LearnedMiller, “Labeled faces in the wild: A database for studying face recognition in unconstrained environments,” University of Massachusetts, Tech. Rep. 0749, 2007.
 [67] B. M. King and E. W. Minium, Statistical Reasoning In Psychology And Education. New York: Wiley, 2003.
Comments
There are no comments yet.