UVA: A Universal Variational Framework for Continuous Age Analysis

1 Introduction

Facial age analysis, including age translation, age generation and age estimation, is one of crucial components in modern face analysis for entertainment and forensics. Age translation (also known as face aging) aims to aesthetically render the facial appearance to a given age. In recent years, with the developments of generative adversarial network (GAN) [7]

and image-to-image translation

[12, 39], impressive progresses [38, 32, 29, 18, 17]

have been achieved on age translation. These methods often utilize a target age vector as a condition to dominate the facial appearance translation. Fig.

2 summarizes the commonly used frameworks for age translation. As shown in Fig. 2 (a), IPC-GAN [29] directly incorporates the target age with inputs to synthesize facial aging images, while CAAE [38] (shown in Fig. 2 (b)) guides age translation by concatenating the age label with the facial image representations in the latent space. It is obvious that the performance of face aging depends on the accurate age labels. Although previous methods [32, 18, 17] have achieved remarkable visual results, in practice, it is difficult to collect labeled images of continuous ages for the intensive aging progression. Since all the existing datasets perform a long-tailed age distribution, researchers often employ the time span of 10 years as the age clusters for age translation. This age cluster formulation potentially limits the diversity of aging patterns, especially for the younger.

Figure 1: Continuous age translation results on UTKFace. The first column are the inputs and the rest columns are the synthesized faces from 0 to 100 years old. Note that there are only 8 images at age 100 in UTKFace.

Recently, the variational autoencoder (VAE)

[15] shows the promising ability in discovering interpretability of the latent factors [3, 11, 31]. By augmenting VAE with a hyper-parameter

β

β

-VAE [19] controls the degree of disentanglement in the latent space. Benefiting from the disentangling abilities in VAE, this paper proposes a novel Universal Variational Aging (UVA) framework to formulate facial age priors in a disentangling manner. Compared with the existing methods, UVA is more capable of disentangling the facial images into an age-related distribution and an age-irrelevant distribution in the latent space, rather than directly utilizing age labels as conditions for age translation. To be specific, the proposed method introduces two latent variables to model the age-related and age-irrelevant information, and employs the variational evidence lower bound (ELBO) to encourage these two parts to be disentangled in the latent space. As shown in Fig. 2

(c), the age-related distribution is assumed as a Gaussian distribution, where the mean value

μ_{R}

is the real age of the input image, while the age-irrelevant prior is set to a normal distribution

N (0, I)

. The disentangling manner makes UVA perform more flexible and controllable for facial age analysis. Additionally, to synthesize photorealistic facial images, an extended conditional version of introspective adversarial learning mechanism [11] is introduced to UVA, which self-estimates the differences between the real and generated samples.

Figure 2: Compared with the previous conditional generative models [29, 38], UVA learns disentangled age-related and age-irrelevant distributions, which is more suitable for the long-tailed data.

In this way, by manipulating the mean value $μ_{R}$ of age-related distribution, UVA can easily realize facial age translation with arbitrary age (shown in Fig. 1), whether the age label exists or not in the training dataset. We furter observe an interesting phenomenon that when sampling noise from the age-irrelevant distribution, UVA can generate photorealistic face images with a specific age. Moreover, given a face image as input, we can easily obtain its age label from the mean value of the age-related distribution, which indicates the ability of UVA to achieve age estimation. As stated above, we can implement three different tasks for facial age analysis in a unified framework. To the best of our knowledge, UVA is the first attempt to achieve facial age analysis, including age translation, age generation and age estimation, in a universal framework. The main contributions of UVA are as follows:

We propose a novel Universal Variational Aging (UVA) framework to formulate the continuous facial aging mechanism in a disentangling manner. It leads to a universal framework for facial age analysis, including age translation, age generation and age estimation.
Benefiting from the variational evidence lower bound, in UVA, the facial images are encoded and disentangled into an age-related distribution and an age-irrelevant distribution in the latent space. An extended conditional introspective adversarial learning mechanism is introduced to obtain photorealistic facial image synthesis.
Different from the existing conditional age translation methods, which utilize the age labels/clusters as a condition, UVA tries to estimate an age distribution from the long-tailed facial age dataset. This age distribution estimation provides a new condition means to model continuous ages that contributes to the interpretability of the image synthesis for facial age analysis.
The qualitative and quantitative experiments demonstrate that UVA successfully formulates the facial age prior in a disentangling manner, obtaining state-of-the-art resuts on the CACD2000 [4], Morph [23], UTKFace [36], MegaAge-Asian [37] and FG-NET [16] datasets.

2 Related Work

2.1 Variational Autoencoder

Variational Autoencoder (VAE) [15, 22] consists of two networks: an inference network $q_{ϕ} (z | x)$ maps the data $x$ to the latent variable $z$ , which is assumed as a gaussian distribution, and a generative network $p_{θ} (x | z)$ reversely maps the latent variable $z$ to the visible data $x$ . The object of VAE is to maximize the variational lower bound (or evidence lower bound, ELBO) of $log p_{θ} (x)$ :

\begin{matrix} log p_{θ} (x) \geq E_{q_{ϕ} (z | x)} log p_{θ} (x | z) - D_{K L} (q_{ϕ} (z | x) | | p (z)) \end{matrix}

(1)

Figure 3: Overview of the architecture and training flow of our approach. Our model contains two components, the inference network $E$ and the generative network $G$ . $X$ , $X_{r}$ and $X_{s}$ denote the real sample, the reconstruction sample and the new sample, respectively. Please refer to Section 3 for more details.

VAE has shown promising ability to generate complicated data, including faces [11], natural images [8], text [25] and segmentation [10, 27]. Following IntroVAE [11], we adopt introspective adversarial learning in our method to produce high-quality and realistic face images.

2.2 Age Translation and Generation

Recently, deep conditional generative models have shown considerable ability in age translation [38, 29, 18, 17]. Zhang et al. [38] propose a Conditional Adversarial Autoencoder(CAAE) to transform an input facial image to the target age. To capture the rich textures in the local facial parts, Li et al. [18] propose a Global and Local Consistent Age Generative Adversarial Network (GLCA-GAN) method. Meanwhile, Identity-Preserved Conditional Generative Adversarial Networks (IPCGAN) [29] introduces an identity-preserved term and an age classification term into age translation. Although these methods have achieved promising visual results, they have limitations in modeling continuous aging mechanism.

With the development of deep generative models, i.e., Generative Adversarial Network(GAN)[7], Variational Autoencoder(VAE), face generation has achieved dramatically success in recent years. Karras et al. [13] provide a new progressive training way for GAN, which grows both the generator and discriminator progressively. Huang et al. [11] propose the Introspective Variational Autoencoders(IntroVAE), in which the inference model and the generator are jointly trained in an introspective way.

3 Approach

In this section, we propose a universal variational aging framework for age translation, age generation and age estimation. The key idea is to employ an inference network $E$ to embed the facial image into two disentangled variational representations, where one is age-related and another is age-irrelevant, and a generator network $G$ to produce photo-realistic images from the re-sampled latent representations. As depicted in Fig. 3, we assign two different priors to regularize the inferred representations, and train $E$ and $G$ with age preserving regularization in an introspective adversarial manner. The details are given in the following.

3.1 Disentangled Variational Representations

In the original VAE [15], a probabilistic latent variable model is learnt by maximizing the variational lower bound to the marginal likelihood of the observable variables. However, the latent variable $z$ is difficult to interpret and control, for each element of $z$ is treated equally in the training. To alleviate this, we manually split $z$ into two parts, i.e., one part $z_{R}$ representing the age-related information and another part $z_{I}$ representing the age-irrelevant information.

Assume $z_{R}$ and $z_{I}$ are independent on each other, then the posterior distribution $q_{ϕ} (z | x) = q_{ϕ} (z_{R} | x) q_{ϕ} (z_{I} | x)$ . The prior distribution $p (z) = p_{R} (z_{R}) p_{I} (z_{I})$ , where $p_{R} (z_{R})$ and $p_{I} (z_{I})$ are the prior distributions for $z_{R}$ and $z_{I}$ , respectively. According to Eq. (1), the optimization objective for the modified VAE is to maximize the lower bound(or evidence lower bound, ELBO) of $log p_{θ} (x)$ :

(2)

To make $z_{R}$ and $z_{I}$ correspond with different types of facial information, $p_{R} (z_{R})$ and $p_{I} (z_{I})$ are set to be different distributions. They are both centered isotropic multivariate Gaussian but $p_{R} (z_{R}) = N (y, I)$ and $p_{I} (z_{I}) = N (0, I)$ , where y is a vector filled by the age label $y$ of $x$ . Through using the two age-related and age-irrelevant priors, maximizing the above variational lower bound encourages the posterior distribution $q_{ϕ} (z_{R} | x)$ and $q_{ϕ} (z_{I} | x)$ to be disentangled and to model the age-related and age-irrelevant information, respectively.

Following the original VAE [15], we assume the posterior $q_{ϕ} (z_{R} | x)$ and $q_{ϕ} (z_{I} | x)$ follows two centered isotropic multivariate Gaussian, respectively, i.e., $q_{ϕ} (z_{R} | x) = N (z_{R}; μ_{R}, σ_{R}^{2})$ , and $q_{ϕ} (z_{I} | x) = N (z_{I}; μ_{I}, σ_{I}^{2})$ . As depicted in Fig. 3, $μ_{R}$ , $σ_{R}$ , $μ_{I}$ and $σ_{I}$ are the output vectors of the inference network $E$ . The input $z$ of the generator network $G$ is the concatenation of $z_{R}$ and $z_{I}$ , where $z_{R}$ and $z_{I}$ are sampled from $N (z_{R}; μ_{R}, σ_{R}^{2})$ and $N (z_{I}; μ_{I}, σ_{I}^{2})$ using a reparameterization trick, respectively, i.e., $z_{R} = μ_{R} + ϵ_{R} ⊙ σ_{R}$ , $z_{I} = μ_{I} + ϵ_{I} ⊙ σ_{I}$ , where $ϵ_{R} \sim N (0, I)$ , $ϵ_{I} \sim N (0, I)$ .

For description convenience, the optimization object in Eq. (2) is rewritten in the negative version:

L_{E L B O} = L_{r e c} + L_{a g e_k l} + L_{k l} (μ_{I}, σ_{I})

(3)

where $L_{r e c}$ , $L_{a g e_k l}$ and $L_{k l} (μ_{I}, σ_{I})$ denote the three terms in Eq. (2), respectively. They can be computed as below:

L_{r e c} = \frac{1}{2} ∥ x - x_{r} ∥_{F}^{2}

(4)

L_{a g e_k l} = \frac{1}{2} C \sum i = 1 ((μ_{R}^{i} - y)^{2} + (σ_{R}^{i})^{2} - log ((σ_{R}^{i})^{2}) - 1)

(5)

L_{k l} (μ_{I}, σ_{I}) = \frac{1}{2} C \sum i = 1 ((μ_{I}^{i})^{2} + (σ_{I}^{i})^{2} - log ((σ_{I}^{i})^{2}) - 1)

(6)

where $x_{r}$ is the reconstruction image of the input $x$ , $y$ is the age label of $x$ , $C$ denotes the dimension of $z_{R}$ and $z_{I}$ .

3.2 Introspective Adversarial Learning

To alleviate the problem of generating blurry samples in VAEs, the introspective adversarial learning mechanism [11] is introduced to the proposed UVA. This makes the model able to self-estimate the differences between the real and generated samples without extra adversarial discriminators. The inference network $E$ is encouraged to distinguish between the real and generated samples while the generator network $G$ tries to fool it as GANs.

Different from IntroVAE [11], the proposed method employs a part of the posterior distribution, rather than the whole distribution, to serve as the estimator of the image reality. Specifically, the age-irrelevant posterior $q_{ϕ} (z_{I} | x)$ is selected to help the adversarial learning. When training $E$ , the model minimizes the KL-distance of the posterior $q_{ϕ} (z_{I} | x)$ from its prior $p_{I} (z_{I})$ for the real data and maximize it for the generated samples. When training $G$ , the model minimizes this KL-distance for the generated samples.

Similar to IntroVAE [11], the proposed UVA is trained to discriminate the real data from both the model reconstructions and samples. As shown in Fig. 3, these two types of samples are the reconstruction sample $x_{r} = G (z_{R}, z_{I})$ and the new samples $x_{s} = G (z_{R}, {^z}_{I})$ , where $z_{R}$ , $z_{I}$ and ${^z}_{I}$ are sampled from $q_{ϕ} (z_{R} | x)$ , $q_{ϕ} (z_{I} | x)$ and $p_{I} (z_{I})$ , respectively.

The adversarial training objects for $E$ and $G$ are defined as below:

\begin{matrix} L_{a d v}^{E} = & L_{k l} (μ_{I}, σ_{I}) + α {{[m - L_{k l} (μ_{I}^{'}, σ_{I}^{'})]}_{k l}^{+} + {[m - L_{k l} (μ_{I}^{''}, σ_{I}^{''})]}_{k l}^{+}}, \end{matrix}

(7)

L_{a d v}^{G} = L_{k l} (μ_{I}^{'}, σ_{I}^{'}) + L_{k l} (μ_{I}^{''}, σ_{I}^{''}),

(8)

where $m$ is a positive margin, $α$ is a weighting coefficient, $(μ_{I}, σ_{I})$ , $(μ_{I}^{'}, σ_{I}^{'})$ and $(μ_{I}^{''}, σ_{I}^{''})$ are computed from the real data $x$ , the reconstruction sample $x_{r}$ and the new samples $x_{s}$ , respectively.

The proposed UVA can be viewed as a conditional version of IntroVAE [11]. The disentangled variational representation and the modified introspective adversarial learning makes it superior to IntroVAE in the interpretability and controllability of image generation.

3.3 Age Preserving Regularization

Age accuracy is important to facial age analysis. The most current face aging methods[38, 29, 18, 17, 32] usually utilize an additional pre-trained age estimation network [30] to supervise the generated face images. While in our method, facial age can be estimated using the inference network $E$ by computing the mean value of the inferred vector $μ_{R}$ . To better capture and disentangle age-related information, an age regularization term is utilized on the learned representation $z_{R}$ :

\begin{matrix} L_{r e g} = ∣ ∣ ∣ \frac{1}{C} C \sum c = 1 μ_{R}^{i} - y ∣ ∣ ∣ \end{matrix}

(9)

where $C$ denotes the dimension of $z_{R}$ , $μ_{R}$ is the output vector of the input $x$ , $y$ is the age label of $x$ .

For further supervising the generator $G$ to reconstruct the age-related information, a similar age regularization term is also employed on the reconstructed and generated samples in Fig. 3, i.e., $x_{r} = G (z_{R}, z_{I})$ and $x_{s} = G (z_{R}, {^z}_{I})$ . The age preserving loss is reformulated as:

\begin{matrix} L_{a g e_k e e p} = ∣ ∣ ∣ \frac{1}{C} C \sum i = 1 {μ^{'}}_{R}^{i} - y ∣ ∣ ∣ + ∣ ∣ ∣ \frac{1}{C} C \sum i = 1 {μ^{''}}_{R}^{i} - y ∣ ∣ ∣ \end{matrix}

(10)

where $μ^{'}$ and $μ^{''}$ are computed by $E$ from $x_{r}$ and $x_{s}$ , respectively.

3.4 Training UVA networks

As shown in Fig. 3, the inference network $E$ embeds the facial image $x$ into two disentangled distributions, i.e., $N (z_{R}; μ_{R}, σ_{R}^{2})$ and $N (z_{I}; μ_{I}, σ_{I}^{2})$ , where $(μ_{R}, σ_{R})$ and $(μ_{I}, σ_{I})$ are the output vectors of $E$ . The generator network $G$ produces the reconstruction $x_{r} = G (z_{R}, z_{I})$ and the sample $x_{s} = G (z_{R}, {^z}_{I})$ , where $z_{R}$ , $z_{I}$ and ${^z}_{I}$ are sampled from $q_{ϕ} (z_{R} | x)$ , $q_{ϕ} (z_{I} | x)$ and $p_{I} (z_{I})$ , respectively. To learn the disentangled variational representations and produce high-fidelity facial images, the network is trained using a weighted sum of the above losses, defined as:

\begin{matrix} L^{E} = L_{r e c} + λ_{1} L_{a g e_k l} + λ_{2} L_{a d v}^{E} + λ_{3} L_{r e g} \end{matrix}

(11)

\begin{matrix} L^{G} = L_{r e c} + λ_{4} L_{a d v}^{G} + λ_{5} L_{a g e_k e e p} \end{matrix}

(12)

where $λ_{1}$ , $λ_{2}$ , $λ_{3}$ , $λ_{4}$ , $λ_{5}$ are the trade-off parameters to balance the important of losses. Noted that the third term $L_{k l} (μ_{I}, σ_{I})$ in the ELBO loss in Eq. (3) has been contained in the adversarial loss for $E$ , i.e., the first term of $L_{a d v}^{E}$ in Eq. (7).

3.5 Inference and Sampling

By regularizing the disentangled representations with the age-related prior $p_{R} (z_{R}) = N (y, I)$ and age-irrelevant prior $p_{I} (z_{I}) = N (0, I)$ , our UVA is thus a universal framework for age translation, generation and estimation tasks, as illustrated in Fig. 2 (c).

Age Translation To achieve the age translation task, we concatenate the age-irrelevant variable $z_{I}$ and a target age variable ${^z}_{R}$ as the input of the generator $G$ ,where $z_{I}$ is sampled from the posterior distribution $q_{ϕ} (z_{I} | x)$ over the input face $x$ and ${^z}_{R}$ is sampled from $p_{R} (z_{R})$ . The age translation result $^x$ is written as:

(13)

Age Generation For the age generation task, there are two settings. One is to generate image from noise ${^z}_{I}$ with any ages ${^z}_{R}$ . To be specific, we concatenate a target age variable ${^z}_{R}$ and an age-irrelevant variable ${^z}_{I}$ as the input of $G$ , where ${^z}_{R}$ and ${^z}_{I}$ are sampled from $p_{R} (z_{R})$ and $p_{I} (z_{I})$ , respectively. Then the age generation result is:

\begin{matrix} ^x = G ({^z}_{R}, {^z}_{I}) \end{matrix}

(14)

The other is to generate image from noise ${^z}_{I}$ , which shares the same age-related variable $z_{R}$ with the input. Specifically, we concatenate the age-related variable $z_{R}$ and an age-irrelevant variable ${^z}_{I}$ as the input of the $G$ , where $z_{R}$ is sampled from the posterior distribution $q_{ϕ} (z_{R} | x)$ over the input face $x$ and ${^z}_{I}$ is sampled from $p (z_{I})$ . The age generation result is formulated as:

\begin{matrix} ^x = G (z_{R}, {^z}_{I}) \end{matrix}

(15)

Age Estimation In this paper, age estimation is also conducted by the proposed UVA to verify the age-related variable extracting and disentangling performance. We calculate the mean value of $C$ -dimension vector $μ_{R}$ as the age estimation result, defined as:

\begin{matrix} ^y = \frac{1}{C} C \sum i = 1 μ_{R}^{i} \end{matrix}

(16)

where $μ_{R}$ is one of the output vectors of the inference network $E$ .

4 Experiments

4.1 Datasets and Settings

4.1.1 Datasets

Cross-Age Celebrity Dataset (CACD2000) [4] consists of 163,446 color facial images of 2,000 celebrities, where the ages range from 14 to 62 years old. However, there are many dirty data in it, which leads to a challenging model training. We choose the classical 80-20 split on CACD2000.

Morph [23] is the largest public available dataset collected in the constrained environment. It contains 55,349 color facial images of 13,672 subjects with ages ranging from 16 to 77 years old. Conventional 80-20 split is conducted on Morph.

UTKFace [36] is a large-scale facial age dataset with a long age span, which ranges from 0 to 116 years old. It contains over 20,000 facial images in the wild with large variations in expression, occlusion, resolution and pose. We employ classical 80-20 split on UTKFace.

MegaAge-Asian [37] is a newly released facial age dataset, consisting of 40,000 Asian faces with ages ranging from 0 to 70 years old. There are extremely variations, such as distortion, large-area occlusion and heavy makeup in MegaAge-Asian. Following [37], we reserve 3,945 images for testing and the remains are treated as training set.

FG-NET [16] contains 1,002 facial images of 82 subjects. We employ it as the testing set to evaluate the generalization of UVA.

4.1.2 Experimental Settings

Following [17], we employ the multi-task cascaded CNN [35] to detect the faces. All the facial images are cropped and aligned into 224 $\times$ 224, while the most existing methods [29, 38, 18] only achieve age translation with 128 $\times$

128. Our model is implemented with Pytorch. During training, we choose Adam optimizer

[14] with

β_{1}

of 0.9,

β_{2}

of 0.99, a fixed learning rate of

2 \times 10^{- 4}

and batch size of 28. The trade-off parameters

λ_{1}

λ_{2}

λ_{3}

λ_{4}

λ_{5}

are all set to 1. Besides,

m

is set to 1000 and

α

is set to 0.5. More details of the network architectures and training processes are provided in the supplementary materials.

4.2 Qualitative Evaluation of UVA

Figure 4: Age translation results on CACD2000. The first column is the input and the rest five columns are the synthesized results.

By manipulating the mean value $μ_{R}$ and sampling from age-related distribution, the proposed UVA can translate arbitrary age based on the input. Fig. 4 and 5 present the age translation results on CACD2000 and Morph, respectively. We observe that the synthesized faces are getting older and older with ages growing. Specifically, the face contours become longer, the forehead hairline are higher and the nasolabial folds are deepened.

Figure 5: Age translation results on Morph. The first column is the input and the rest five columns are the synthesized results.

4.2.1 Age Translation

Figure 6: Age translation results on MegaAge-Asian. The first column is the input and the rest are the synthesized results.

Since both CACD2000 and Morph lack of images of children, we conduct age translation on MegaAge-Asian and UTKFace. Fig. 6 shows the translation results on MegaAge-Asian from 10 to 70 years old. Fig. 7 describes the results on UTKFace from 0 to 115 years old with an interval of 5 years old. Obviously, from birth to adulthood, the aging effect is mainly shown on craniofacial growth, while the aging effect from adulthood to elder is reflected on the skin aging, which is consistent with human physiology.

Figure 7: Age translation results on UTKFace from 0 to 115 years old.

Figure 8: Cross-dataset age translation results on FG-NET.

Figure 9: Comparison with the previous works on FG-NET.

Furthermore, superior to the most existing methods [38, 29, 18, 17] that can only generate images at specific age groups, our UVA is able to realize continuous age translation with 1-year-old interval. The translation results on UTKFace from 0 to 119 years old with 1-year-old interval is presented in the supplementary materials, due to the page limitation.

To evaluate the model generalization, we test UVA on FG-NET and show the translation results in Fig. 8. The left image of each subject is the input and the right one is the translated image with 60 years old. Note that our UVA is trained only on CACD2000 and tested on FG-NET.

Different from the previous age translation works that divide the data into four or nine age groups, our proposed UVA models continuous aging mechanism. The comparison with prior works, including AgingBooth App[1], CDL[26], RFA[28] and Yang et al.[33], is depicted in Fig. 9.

4.2.2 Age Generation

Figure 10: Age generation results on Morph and CACD2000. For each image group, the first column is the input and the rest three columns are the generated faces with the same age-related representation as the input face.

Benefiting from the disentangled age-related and age-irrelevant distributions in the latent space, the proposed UVA is able to realize age generation. On the one hand, conditioned on a given facial image, UVA can generate new faces with the same age-related representation as the given image by fixing $z_{R}$ and sampling ${^z}_{I}$ . Fig. 10 presents some results under this situation on the Morph and CACD2000 datasets. Our UVA generates diverse faces (such as different genders, appearances and haircuts) with the same age as the input facial image.

Figure 11: Age generation results on CACD2000. In the left, each row has the same age-irrelevant representation and each column has the same age-related representation. The right shows the most similar image from google for the generated image in the left. (All of the four subjects are generated from noise.)

Figure 12: Age generation results on Morph. In the left, each row has the same age-irrelevant representation and each column has the same age-related representation. The right shows the most similar image from google for the generated image in the left.(All of the four subjects are generated from noise.)

On the other hand, by manipulating an arbitrary $μ_{R}$ of age-related distribution and sampling both ${^z}_{R}$ and ${^z}_{I}$ , UVA has the ability to generate faces with various ages. Fig. 11 and 12 display the age generation results from 20 to 60 on the CACD2000 and Morph datasets, respectively. Each row has the same age-irrelevant representation ${^z}_{I}$ and each column has the same age-related representation ${^z}_{R}$ . We can observe that the age-irrelevant information, such as gender and identity, is preserved across rows, and the age-related information, such as white hairs and wrinkles, is preserved across columns. These demonstrate that our UVA effectively disentangles age-related and age-irrelevant representations in the latent space. More results on the UTKFace and MegaAge-Asian datasets are presented in the supplementary materials, due to the page limitation.

4.2.3 Generalized Abilities of UVA

Benefiting from the disentangling manner, UVA has the abilities to estimate the age-related distribution from the facial images, even if the training dataset is long-tailed. Since the facial images in Morph ages from 16 to 77, when performing age translation with 10 years old, as shown in Fig. 13, it is amazing that UVA can synthesize photorealistic aging images. This observation demonstrates the generalized abilities of our framework, and also indicates that UVA can efficiently and accurately estimate the age-related distribution from the facial images, even if the training dataset performs a long-tailed age distribution.

Figure 13: Age translation results on Morph. Note that the target age (10 years old) is not existing in the Morph dataset.

4.3 Quantitative Evaluation of UVA

4.3.1 Age Estimation

To further demonstrate the disentangling ability of UVA, we conduct age estimation experiments on Morph and MegaAge-Asian, both of which are widely used datasets in age estimation task. We detail the evaluation metrics in the supplementary materials.

For experiment on Morph, following [21], we choose the 80-20 random split protocol and report the mean absolute error(MAE). The results are listed in Table 1, where our model has not been pre-trained on the external data. We can see from Table 1 that the proposed UVA achieves comparative performance with the stat-of-the-art methods.

Methods	Pre-trained	Morph
Methods	Pre-trained	MAE
OR-CNN[20]	-	3.34
DEX[24]	IMDB-WIKI $^{*}$	2.68
Ranking [5]	Audience	2.96
Posterior[37]	-	2.87
SSR-Net[34]	IMDB-WIKI	2.52
M-V Loss[21]	-	2.514
ThinAgeNet [6]	MS-Celeb-1M $^{*}$	2.346
UVA	-	2.52

Used partial data of the dataset;

Table 1: Comparisons with state-of-the-art methods on Morph. Lower MAE is better.

For age estimation on MageAge-Asian, following [37], we reserve 3,945 images for testing and employ the Cumulative Accuracy (CA) as the evaluation metric. We report CA(3), CA(5), CA(7) as [37, 34] in Table 2. Note that our model has not pre-trained on the external data and achieves comparative performance.

Methods	Pre-trained	MegaAge-Asian
Methods	Pre-trained	CA(3)	CA(5)	CA(7)
Posterior[37]	IMDB-WIKI	62.08	80.43	90.42
MobileNet[34]	IMDB-WIKI	44.0	60.6	-
DenseNet[34]	IMDB-WIKI	51.7	69.4	-
SSR-Net[34]	IMDB-WIKI	54.9	74.1	-
UVA	-	60.47	79.95	90.44

Table 2: Comparisons with state-of-the-art methods on MegaAge-Asian. The unit of CA(n) is

%

. Higher CA(n) is better.

The age estimation results on Morph and MegaAge-Asian are nearly as good as the state-of-the-arts, which also demonstrates that the age-related representation is well disentangled from the age-irrelevant representation.

4.3.2 Aging Accuracy

We conduct aging accuracy experiments of UVA on Morph and CACD2000, which is an essential quantitative metric to evaluate age translation. For fair comparison, following [33, 18], we utilize the online face analysis tool of Face++ [2] to evaluate the ages of the synthetic results, and divide the testing data of Morph and CACD2000 into four age groups: 30-(AG0), 31-40(AG1), 41-50(AG2), 51+(AG3). We choose AG0 as the input and synthesize images in AG1, AG2 and AG3. Then we estimate the ages of the synthesized images and calculate the average ages for each group.

(a) on Morph
Method	Input	AG1	AG2	AG3
CAAE[38]	-	28.13	32.50	36.83
Yang et al.[33]	-	42.84	50.78	59.91
UVA	-	36.59	50.14	60.78
Real Data	28.19	38.89	48.10	58.22
(b) on CACD2000
Method	Input	AG1	AG2	AG3
CAAE[38]	-	31.32	34.94	36.91
Yang et al.[33]	-	44.29	48.34	52.02
UVA	-	39.19	45.73	51.32
Real Data	30.73	39.08	47.06	53.68

Table 3: Comparisons of the aging accuracy.

As shown in Table 3, we compare the UVA with CAAE [38] and Yang et al. [33] on Morph and CACD2000. We observe that the transformed ages by UVA are closer to the natural data than by CAAE [38] and is comparable to Yang et al. [33]. Note that, for age translation among multiple ages, the proposed UVA only needs to train one model, but Yang et al. [33] need to train multiple models for each transformation, such as Input $\to$ AG1, Input $\to$ AG2 and Input $\to$ AG3.

4.3.3 Diversity

In this subsection, we utilize the $F r ´ e c h e t$ Inception Distance (FID) [9] to evaluate the quality and diversity of the generated data. FID measures the $F r ´ e c h e t$ distance between the generated and the real distributions in the feature space. Here, the testing images are generated from noise ${^z}_{I}$ with target ages ${^z}_{R}$ , where ${^z}_{I}$ and ${^z}_{R}$ are sampled from $p (z_{I})$ and $p (z_{R})$ , respectively. The FID results are detailed in Table 4.

	Morph	CACD2000	UTKFace	MegaAge-Asian
FID	17.154	37.08	31.18	32.79

Table 4: The FID results on the four datasets.

4.4 Ablation Study

We conduct ablation study on Morph to evaluate the effectiveness of introspective adversarial learning and age preserving regularization. We introduce the details in the supplemental materials, due to the page limitation.

5 Conclusion

This paper has proposed a Universal Variational Aging framework for continuous age analysis. Benefiting from variational evidence lower bound, the facial images are disentangled into an age-related distribution and an age-irrelevant distribution in the latent space. A conditional introspective adversarial learning mechanism is introduced to improve the quality of facial aging synthesis. In this way, we can deal with long-tailed data and implement three different tasks, including age translation, age generation and age estimation. To the best of our knowledge, UVA is the first attempt to achieve facial age analysis in a universal framework. The qualitative and quantitative experiments demonstrate the superiority of the proposed UVA on five popular datasets, including CACD2000, Morph, UTKFace, MegaAge-Asian and FG-NET. This indicates that UVA can efficiently formulate the facial age prior, which contributes to both photorealistic and interpretable image synthesis for facial age analysis.

References

[1] Agingbooth. pivi and co. https://itunes.apple.com/us/app/ agingbooth/id357467791?mt=8/, [Online].
[2] Face++ research toolkit. megvii inc. http://www. faceplusplus.com, [Online].
[3] C. P. Burgess, I. Higgins, A. Pal, L. Matthey, N. Watters, G. Desjardins, and A. Lerchner. Understanding disentangling in beta-vae. arXiv preprint arXiv:1804.03599, 2018.
[4] B.-C. Chen, C.-S. Chen, and W. H. Hsu. Face recognition and retrieval using cross-age reference coding with cross-age celebrity dataset. IEEE Transactions on Multimedia, 17(6):804–815, 2015.
[5] S. Chen, C. Zhang, M. Dong, J. Le, and M. Rao. Using ranking-cnn for age estimation. In CVPR, 2017.
[6] B.-B. Gao, H.-Y. Zhou, J. Wu, and X. Geng. Age estimation using expectation of label distribution learning. In IJCAI, 2018.
[7] I. Goodfellow, J. Pouget-Abadie, M. Mirza, B. Xu, D. Warde-Farley, S. Ozair, A. Courville, and Y. Bengio. Generative adversarial nets. In NeurIPS, 2014.
[8] I. Gulrajani, K. Kumar, F. Ahmed, A. A. Taiga, F. Visin, D. Vazquez, and A. Courville. Pixelvae: A latent variable model for natural images. arXiv preprint arXiv:1611.05013, 2016.
[9] M. Heusel, H. Ramsauer, T. Unterthiner, B. Nessler, and S. Hochreiter. Gans trained by a two time-scale update rule converge to a local nash equilibrium. In NeurIPS, 2017.
[10] X. Hou, L. Shen, K. Sun, and G. Qiu. Deep feature consistent variational autoencoder. In WACV. IEEE, 2017.
[11] H. Huang, R. He, Z. Sun, T. Tan, et al. Introvae: Introspective variational autoencoders for photographic image synthesis. In NeurIPS, 2018.
[12] P. Isola, J.-Y. Zhu, T. Zhou, and A. A. Efros.
Image-to-image translation with conditional adversarial networks.
CVPR, 2017.
[13] T. Karras, T. Aila, S. Laine, and J. Lehtinen. Progressive growing of gans for improved quality, stability, and variation. arXiv preprint arXiv:1710.10196, 2017.
[14] D. P. Kingma and J. Ba. Adam: A method for stochastic optimization. arXiv preprint arXiv:1412.6980, 2014.
[15] D. P. Kingma and M. Welling. Auto-encoding variational bayes. In ICLR, 2014.
[16] A. Lanitis, C. J. Taylor, and T. F. Cootes. Toward automatic simulation of aging effects on face images. IEEE Transactions on Pattern Analysis and Machine Intelligence, 24(4):442–455, 2002.
[17] P. Li, Y. Hu, R. He, and Z. Sun. Global and local consistent wavelet-domain age synthesis. IEEE Transactions on Information Forensics and Security, 2018.
[18] P. Li, Y. Hu, Q. Li, R. He, and Z. Sun. Global and local consistent age generative adversarial networks. ICPR, 2018.
[19] L. Matthey, A. Pal, C. Burgess, X. Glorot, M. Botvinick, S. Mohamed, and A. Lerchner. $β$ -VAE: Learning basic visual concepts with a constrained variational framework. In ICLR, 2017.
[20] Z. Niu, M. Zhou, L. Wang, X. Gao, and G. Hua. Ordinal regression with multiple output cnn for age estimation. In CVPR, 2016.
[21] H. Pan, H. Han, S. Shan, and X. Chen.
Mean-variance loss for deep age estimation from a face.
In CVPR, 2018.
[22] D. J. Rezende, S. Mohamed, and D. Wierstra.
Stochastic backpropagation and approximate inference in deep generative models.
ICML, 2014.
[23] K. Ricanek and T. Tesafaye. Morph: A longitudinal image database of normal adult age-progression. In FGR. IEEE, 2006.
[24] R. Rothe, R. Timofte, and L. Van Gool. Deep expectation of real and apparent age from a single image without facial landmarks.
International Journal of Computer Vision
, 126(2-4):144–157, 2018.
[25] S. Semeniuta, A. Severyn, and E. Barth. A hybrid convolutional variational autoencoder for text generation. arXiv preprint arXiv:1702.02390, 2017.
[26] X. Shu, J. Tang, H. Lai, L. Liu, and S. Yan. Personalized age progression with aging dictionary. In ICCV, Dec. 2015.
[27] K. Sohn, H. Lee, and X. Yan. Learning structured output representation using deep conditional generative models. In NeurIPS, 2015.
[28] W. Wang, Z. Cui, Y. Yan, J. Feng, S. Yan, X. Shu, and N. Sebe. Recurrent face aging. In CVPR, Jun. 2016.
[29] Z. Wang, X. Tang, W. Luo, and S. Gao. Face aging with identity-preserved conditional generative adversarial networks. In CVPR, 2018.
[30] X. Wu, R. He, Z. Sun, and T. Tan. A light cnn for deep face representation with noisy labels. IEEE Transactions on Information Forensics and Security, 13(11):2884–2896, 2018.
[31] X. Wu, H. Huang, V. M. Patel, R. He, and Z. Sun. Disentangled variational representation for heterogeneous face recognition. In AAAI, 2019.
[32] H. Yang, D. Huang, Y. Wang, and A. K. Jain. Learning face age progression: A pyramid architecture of gans. In CVPR, 2018.
[33] H. Yang, D. Huang, Y. Wang, and A. K. Jain. Learning face age progression: A pyramid architecture of gans. CVPR, 2018.
[34] T.-Y. Yang, Y.-H. Huang, Y.-Y. Lin, P.-C. Hsiu, and Y.-Y. Chuang. Ssr-net: A compact soft stagewise regression network for age estimation. In IJCAI, 2018.
[35] K. Zhang, Z. Zhang, Z. Li, and Y. Qiao. Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Processing Letters, 23(10):1499–1503, 2016.
[36] S. Y. Zhang, Zhifei and H. Qi. Age progression/regression by conditional adversarial autoencoder. In CVPR. IEEE, 2017.
[37] Y. Zhang, L. Liu, C. Li, et al. Quantifying facial age by posterior of age comparisons. arXiv preprint arXiv:1708.09687, 2017.
[38] Z. Zhang, Y. Song, and H. Qi. Age progression/regression by conditional adversarial autoencoder. In CVPR, 2017.
[39] J.-Y. Zhu, T. Park, P. Isola, and A. A. Efros. Unpaired image-to-image translation using cycle-consistent adversarial networks. ICCV, 2017.

6 Supplementary Materials

In this supplementary material, we first introduce the network architecture and the training algorithm of our proposed UVA. Then we depict the metrics used for age estimation experiments. Additional comparisons are conducted to demonstrate the continuous manner of UVA in Section 4. Besides, Section 5 presents the ablation study, followed by additional results of age generation and translation.

6.1 Network Architecture

The network architectures of the proposed UVA are shown in Table 5. All images are aligned to 256 $\times$ 256 as the inputs.

Layer	Input	Filter/Stride	Output Size
Encoder
E $_$ conv1	$x$	$5 \times 5 / 1$	$256 \times 256 \times 16$
E $_$ avgpool1	E $_$ conv1	$2 \times 2 / 2$	$128 \times 128 \times 16$
E $_$ res1	E $_$ avgpool1	$3 \times 3 / 1$	$128 \times 128 \times 32$
E $_$ avgpool2	E $_$ res1	$2 \times 2 / 2$	$64 \times 64 \times 32$
E $_$ res2	E $_$ avgpool2	$3 \times 3 / 1$	$64 \times 64 \times 64$
E $_$ avgpool3	E $_$ res2	$2 \times 2 / 2$	$32 \times 32 \times 64$
E $_$ res3	E $_$ avgpool3	$3 \times 3 / 1$	$32 \times 32 \times 128$
E $_$ avgpool4	E $_$ res3	$2 \times 2 / 2$	$16 \times 16 \times 128$
E $_$ res4	E $_$ avgpool4	$3 \times 3 / 1$	$16 \times 16 \times 256$
E $_$ avgpool5	E $_$ res4	$2 \times 2 / 2$	$8 \times 8 \times 256$
E $_$ res5	E $_$ avgpool5	$3 \times 3 / 1$	$8 \times 8 \times 512$
E $_$ avgpool6	E $_$ res5	$2 \times 2 / 2$	$4 \times 4 \times 512$
E $_$ res6	E $_$ avgpool6	$3 \times 3 / 1$	$4 \times 4 \times 512$
E $_$ flatten, fc	E $_$ res6	-	1024
E $_$ split	E $_$ fc	-	256,256,256,256
E $_$ repara1	E $_$ split[0,1]	-	256
E $_$ repara2	E $_$ split[2,3]	-	256
E $_$ cat	E $_$ repara1, E $_$ repara2,	-	512
Generator
G $_$ fc,reshape	E $_$ cat	-	$4 \times 4 \times 512$
G $_$ res1	G $_$ fc	$3 \times 3 / 1$	$4 \times 4 \times 512$
G $_$ upsample1	G $_$ res1	$\times 2$ , nearest	$8 \times 8 \times 512$
G $_$ res2	G $_$ upsample1	$3 \times 3 / 1$	$8 \times 8 \times 512$
G $_$ upsample2	G $_$ res2	$\times 2$ , nearest	$16 \times 16 \times 512$
G $_$ res3	G $_$ upsample2	$3 \times 3 / 1$	$16 \times 16 \times 256$
G $_$ upsample3	L $_$ res3	$\times 2$ , nearest	$32 \times 32 \times 256$
G $_$ res4	G $_$ upsample3	$3 \times 3 / 1$	$32 \times 32 \times 128$
G $_$ upsample4	G $_$ res4	$\times 2$ , nearest	$64 \times 64 \times 128$
G $_$ res5	G $_$ upsample4	$3 \times 3 / 1$	$64 \times 64 \times 64$
G $_$ upsample5	L $_$ res5	$\times 2$ , nearest	$128 \times 128 \times 64$
G $_$ res6	G $_$ upsample5	$3 \times 3 / 1$	$128 \times 128 \times 32$
G $_$ upsample6	G $_$ res7	$\times 2$ , nearest	$256 \times 256 \times 32$
G $_$ res7	G $_$ upsample6	$3 \times 3 / 1$	$256 \times 256 \times 32$
G $_$ conv1	G $_$ res7	$5 \times 5 / 1$	$256 \times 256 \times 3$

Table 5: Network architectures of UVA.

6.2 Training Algorithm

The training process of UVA is described in Algorithm 1.

1.3

Algorithm 1 Training UVA Model

θ_{G}, ϕ_{E} \leftarrow

Initialize network parameters

2:while not converged do

X \leftarrow

Random mini-batch from training set

μ_{R}, σ_{R}, μ_{I}, σ_{I} \leftarrow

E n c (X)

Z_{R} \leftarrow

μ_{R} + ϵ ⊙ σ_{R}

Z_{I} \leftarrow

μ_{I} + ϵ ⊙ σ_{I}

{⌢ Z}_{I}

Samples from prior

N (0, I)

X_{r} \leftarrow

D e c (Z_{R}, Z_{I})

X_{s} \leftarrow

D e c (Z_{R}, {⌢ Z}_{I})

Z_{R}^{'} \leftarrow

μ_{R}^{'} + ϵ ⊙ σ_{R}^{'}

Z_{I}^{'} \leftarrow

μ_{I}^{'} + ϵ ⊙ σ_{I}^{'}

Z_{R}^{''} \leftarrow

μ_{R}^{''} + ϵ ⊙ σ_{R}^{''}

Z_{I}^{''} \leftarrow

μ_{I}^{''} + ϵ ⊙ σ_{I}^{''}

10:

L_{r e c} \leftarrow L_{r e c} (X_{r}, X)

11:

L_{a g e_k l} \leftarrow L_{a g e_{k l}} (μ_{R}, σ_{R}, Y)

12:

L_{r e g} \leftarrow | μ_{R} - Y |

13:

μ_{R}^{'}, σ_{R}^{'}, μ_{I}^{'}, σ_{I}^{'} \leftarrow

E n c (X_{r}), μ_{R}^{''}, σ_{R}^{''}, μ_{I}^{''}, σ_{I}^{''} \leftarrow

E n c (X_{s})

14:

L_{a g e_k e e p} \leftarrow | μ_{R}^{'} - Y | + | μ_{R}^{''} - Y |

15:

L_{a d v}^{E} \leftarrow L_{k l} (μ_{I}, σ_{I}) + α {[m - L_{k l} (μ_{I}^{'}, σ_{I}^{'})]^{+} + [m - L_{k l} (μ_{I}^{''}, σ_{I}^{''})]^{+}}

16:

L_{a d v}^{G} \leftarrow α {L_{k l} (μ_{I}^{'}, σ_{I}^{'}) + L_{k l} (μ_{I}^{''}, σ_{I}^{''})}

17:

ϕ_{E} \leftarrow ϕ_{E} - η \nabla_{ϕ_{E}} (L_{r e c} + λ_{1} L_{a g e_k l} + λ_{2} L_{a d v}^{E} + λ_{3} L_{r e g})

▹

Perform Adam updates for

ϕ_{E}

18:

θ_{G} \leftarrow θ_{G} - η \nabla_{θ_{G}} (L_{r e c} + λ_{4} L_{a d v}^{G} + λ_{5} L_{a g e_k e e p})

▹

Perform Adam updates for

θ_{G}

19:end while

6.3 Evaluation Metrics of Age Estimation

We evaluate the performance of UVA on age estimation task with the Mean Absolute Error(MAE) and Cumulative Accuracy (CA).

MAE is widely used to evaluate the performance of age estimation models,which is defined as the average distance between the inferred age and the ground-truth:

\begin{matrix} M A E = \frac{1}{N} N \sum i = 1 | {^y}_{i} - y_{i} | \end{matrix}

(17)

The Cumulative Accuracy (CA) is defined as:

\begin{matrix} C A (n) = \frac{K_{n}}{K} \times 100 % \end{matrix}

(18)

where $K_{n}$ is the number of the test images whose absolute estimated error is smaller than $n$ .

6.4 Additional Comparison Results on FG-NET

Figure 14: Comparison with CAAE on FG-NET.

Additional comparison results with the conditional adversarial autoencoder (CAAE) [38] on FG-NET are shown in Figure 14. Note that our model is trained on UTKFace. We observe that CAAE can only translate the input into specific age groups, including 0-5, 6-10, 11-15, 16-20, 21-30, 31-40, 41-50, 51-60, 61-70, and 71-80, while UVA can perform age translation with the arbitrary age. In addition, CAAE only focuses on the face appearances without hair, while UVA achieves age translation on the whole face. As shown in Fig. 14, the hairline gradually becomes higher as age increases.

6.5 Ablation Study

Figure 15: Model comparison: age translation results of UVA and its variants. For each subject, the first row is the synthesis results of UVA, while the second, third and fourth rows are the synthesis results without introspective adversarial loss, age preserving loss and age regularization term, respectively.

In this section, we conduct ablation study on Morph to evaluate the effectiveness of the introspective adversarial loss, the age preserving loss and the age regularization term, respectively. We report the qualitative visualization results for a comprehensive comparison. Fig. 15 shows visual comparisons between UVA and its three incomplete variants. We observe that the proposed UVA is visually better than its variants across all ages. Without the $L_{a g e_k e e p}$ or $L_{r e g}$ loss, the aging images lack the detailed texture information. Besides, without the $L_{a d v}$ loss, the aging faces are obviously blur. The visualization results demonstrate that all three components are essential for UVA.

6.6 Age Generation

Benefiting from the disentangled age-related and age-irrelevant distributions in the latent space, the proposed UVA is able to realize age generation. On the one hand, given a facial image, when fixing age-related distribution $z_{R}$ and sampling ${^z}_{I}$ from noise, UVA can generate various faces with the specific age. Fig. 18 shows the age generation results on Morph, CACD2000, MegaAge-Asian and UTKFace. On the other hand, by manipulating $μ_{R}$ of the age-related distribution and sampling ${^z}_{I}$ from noise, UVA can generate faces with various ages. Fig. 16 shows the age generation results from 10 to 70 on MegaAge-Asian.

Figure 16: Age generation results on MegaAge-Asian. Each row has the same age-irrelevant representation and each column has the same age-related representation. The images in MegaAge-Asian are low-resolution and limited-quality, leading to blurred synthesized results.

6.7 Age Translation

Figure 18:
Age generation results with age feature extracted from the given image on Morph, CACD2000, MegaAge-Asian and UTKFace. For each subject, the first column is the input face, while the rest three columns are the generated faces.

As shown in Fig. 17, we observe that UTKFace exhibits a long-tailed age distribution. The age translation results on UTKFace are presented in Fig. 19, 20 and 21. The ages range from 0 to 119 years old with 1-year-old aging interval. Since the images in UTKFace range from 0 to 116 years old, it is amazing that UVA can synthesize photo-realistic aging images even with 117,118 and 119 years old. These results suggest that UVA can effeciently and accurately estimate the age-related distribution from the facial images, even if the training dataset performs a long-tailed age distribution. Considering that generating high-resolution images is significant to enlarge the application field of face aging, in the future, we will build a new facial age dataset to support the age-related study.