1 Introduction
The metamorphosis between two objects is commonly called ”morphing”. It is the process of changing one figure into another. In recent years, image morphing techniques have been widely used in the entertainment industry. Many techniques have been developed to achieve a desired morphing effect. [17]
In 2D image morphing, S.Y. Lee et al. [12] proposed a technique that generates a continuous and onetoone deformation from the twodimensional positional constraints by using a deformable surface model. The transition behavior can be controlled by assigning the transition curves for selected points on an image. Similarly, A. Gregory et al. [6] proposed a method of geometry morphing, which creates a morph by defining morphing trajectories between the feature pairs and by interpolating them across the merged polyhedron. Attributed to these technologies, the user can achieve a satisfactory visual effect by simply setting the correspondence between graphical features. In particular, for images of human faces, it is possible to pick the feature points of the face automatically. V. Zanella et al. [19] use the socalled ”Active Shape Models” [3] to find the facial features in the 2D images and perform the morphing of face images in frontal view automatically. To make the metamorphosis look realer, the morphing path must be constrained by some physical laws. Researchers, such as Verbeek et al. [16], Y. Bao et al. [1] and L. Younes et al. [2], proposed methods of determining the morphing paths by optimizing an energy functional, which characterizes the intrinsic deformation of the surface away from its rest shape. Althrough 2D image morphing technique has pretty mature, 3D image morphing remains challenges, especially when the virtual real morphing effects are desired. In addition, in order to achieve a satisfactory visual effect, the texture images also need to be computed in the process of visualization.
On the other hand, with the advance of the threedimensional imaging technology, surface morphing in 3D has become very important. Comparing to the 2D image matching problem, surface matching problem is much more difficult, since the surface matching involves the correspondence in coordinates and the geometric information of images in is far richer than images in 2D. D. DeCarlo et al. [4] proposed a method of generating smoothlooking transformations between pairs of surfaces that may differ in topology by specifying a sparse control mesh on each surface and by associating each face in one control mesh with a corresponding face in the other. Y.S. Liu et al. [14] proposed a morphing method that works well even when two input 3D triangle meshes have very different shapes by creating consistent meshes for the original source and target models. Schröder et al. [13] proposed a variational approach based on minimizing bending and stretching in which corresponding feature points and line segments are matched. E. Jeong et al. [11] proposed a featurebased morphing technique for two objects equipped with surface light fields by using spherical embedding of meshes. M. Yang et al. [18] proposes a realistic 3D morphing method based on GPU for realtime animation of facial expressions by rendering the real texture on the digital character.
In this study, we propose an efficient way to obtain the desired morphing effects. Our goal is to generate the morphing sequence between human faces in which features of the given surfaces are maintained during the morphing process. Moreover, in order to control digital models and perform digital acting via real agents, we also construct the single mesh of human faces through the correspondence matching. In our approach, we utilize the conformal mapping technique. A Möbius transformation and a deformation from a modified thinplate model are computed to achieve high accurate feature correspondence. A single mesh based on geodesic paths among feature points is computed very efficiently. With the help provided by this single mesh, the texture information and the geometric information, such as the conformal factor and the mean curvature , of each frame in the morphing sequence can be easily interpolated from the given surfaces through various homotopy methods. Then the 3D images of each intermediate frame can be computed from the reconstruction algorithm proposed by Gu and Yau. Our method guarantees the piecewise smoothness of homotopy path. So, we can avoid the vibration of each frames caused by the unstableness of the 3D camera. In addition, we merely require 10% the number of original frames to simulate the real motion of human facial expressions, so that the size of the information data is significantly reduced but the resolution of the 3D images does not decrease.
In the following, we review the computation of the Riemann conformal mapping in section 2. We present our surface matching technique in section 3 and introduce the way of homotopy in section 4. After that, we show how we reconstruct the morphing sequence in section 6 and demonstrate some morphing results in section 7.
2 Idea for Computing Conformal Mapping
The Riemann conformal mapping plays an important role in the surface matching. In this section, we briefly review the method of computing Riemann conformal mapping.
2.1 Spherical Conformal Mapping
The spherical conformal mapping is first computed by Gu and Yau [8] in 2003. The idea is based on minimizing the harmonic energy through a nonlinear heat diffusion process as following: Suppose the desired map is Let and denote the image of the vertex and the normal at respectively. The normal and tangent components of are defined as
and
respectively. The harmonic map is then computed by minimizing the harmonic energy associated with through the nonlinear heat diffusion process
with the constrain . The equation can be written in the following form,
and can be solved numerically by using the quasiimplicit Euler method (QIEM) [7],
where, is the discrete Laplacian, is a diagonal matrix with
The initial map for the above iterative formula can be obtained from the Gauss map. It is well known that the efficiency of the explicit scheme is usually not satisfactory since the time step is generally very small due to the diffusive nature. The quasiimplicit Euler method [7] proposed by W.W. Lin et al. is shown to be very robust on solving the nonlinear diffusion equation.
2.2 Riemann Conformal Mapping
The idea for computing Riemann conformal mapping, shown in Figure 2, was first proposed by Gu and Yau [10]. For a given surface with single boundary, we make it into a closed surface by using the double covering technique. Then we compute the spherical conformal mapping of . Finally, we cut out the semisphere along the equator and map the unit semisphere onto the unit disk conformally by using the stereographic projection.
2.3 Numerical Results
We demonstrate the robustness of the quasiimplicit Euler method by computing the Riemann conformal mappings of human facial expressions. Two different facial expressions and the associated conformal mappings are shown in Figure 3. In order to check the conformality of the QIEM, we paste the checkerboard grid on the image of the Riemann conformal mapping , and put it back to the surface by using the inverse of the Riemann conformal mapping . If the mapping is anglepreserving, every angle should be nearly 90 degrees. The histograms of the angle distribution, shown in Figure 3, indicate that the QIEM is very accurate on anglepreserving.
3 Surface Matching with Minimal Global Error
Surface matching plays a critical role in surface morphing. L. Younes et al. [2] proposed a simple method to interpolate 2D landmark matching by constructing a diffeomorphism between two domains. When it comes to surface matching, it would be much more difficult. However, with the Riemann conformal mappings, we reduce the surface matching problem into the unit disk matching problem. Hence we can apply the similar idea of the 2D landmark matching to the surface matching.
For the surface matching, X. Gu et al. [8] use landmarks to obtain the optimal Möbius transformation as the matching function between the spherical conformal mappings of two brains. When it comes to the matching between two different facial expressions, a slight twist is allowed since the transformation of facial expressions is actually not conformal. In the following, we propose to match the landmarks of each facial expressions by composition of the Möbius transformation and deformation from the plate matching.
We assume that the correspondence of the boundary points of two different human faces and are known. The assumption can be realized by putting markers on the boundary of the human faces during scanning. Suppose landmarks are selected for each facial expression and denoted by
and
Our purpose is to construct a matching function where First, each surface is mapped to the unit disk by the Riemann conformal mappings
We denote by and by , respectively,
Next, a uniform grid points of an checkerboard ranged on , , is lay on . In the thinplate model, the deformation field is approximated by the span of the Green functions of the bending operator at each grid point where is the distance between and . Therefore the matching function between the unit disks is defined by with and
for where are unknown coefficients, . To determine these coefficients for matching landmarks on conformal parametric domain, we solve the least square problems
where
and are the conformal factors, resulted from the Riemann conformal mappings and , at , respectively, and
The least square problem can be easily solved by QR method when or solved by Tikhonov regularization [5]
when Then the matching function between and can be obtained by taking Figure 5 shows the result of global matching and Table I shows the comparison between the optimal Möbius transformation (OMT) and the optimal Möbius transformation with the global matching function (OMGMF).
There is a variety ways to measure the distortion of the matching function, such as Euclidean norm on the space, the infinite norm on the unit disk, etc. In order to verify that the matching pairs are mapped to by we define the conformal matching energy
which measures the square of Euclidean distance at each matching pair on the conformal disk. On another point of view, we define the local matching energy
which directly measures the square of Euclidean distance at each matching pair in . In order to ensure that the global error is relatively small, we also compare the global matching energy, which is defined by
According to Table I, the global matching function significantly reduces the matching energies, which indicates that the global matching function works well on the surface matching.
OMT  OMGMF  

4 Surface Morphing
The effect of the traditional image morphing by using the direct interpolation is not satisfactory since there are shadows in the area of wrong correspondence. In 3D morphing, it could be even worse. To improve this phenomenon, D. Smythe proposed the mesh warping technique [17], which partitions the 2D image into several pieces and interpolate them piece by piece. Base on this idea, we compute a geodesic frame on the surfaces in 3D. The frame consists of the discrete surface geodesics that connect preselected feature points. The frame can be refined to better capture characters of various facial expressions. The geodesic frames for the eight facial expressions are shown in Figure 6.
The method of computing discrete surface geodesics on triangular meshes was proposed by D. Martínez et al. [15] which computes the initial path by using Sethian’s fast marching method and correct the path with the path correcting iterations. For a mesh with a large number of vertices, the fast marching method could be very time consuming. To improve the efficiency, we calculate the initial path by

construct the frame on the unit disk, where feature points are connected by straight line segments,

the initial paths are obtained by taking the inverse conformal map of these line segments.

Apply Martínez’s algorithm to obtain the geodesic frame.
The resulted geodesic frame is called the single mesh. In our calculation, the initial paths in the frame mostly converge to the geodesics within 5 steps in the path correcting iterations. The geodesic frame is remapped to partition the conformal parametric domain as shown in Figure 7.
neutral  smile  sad  pout 
bitter smile  pain  wry  ferocious 
neutral  smile  sad  pout 
bitter smile  pain  wry  ferocious 
5 Surface Registration via the Geodesic Frame of the Surface
To build an onetoone surface registration, we rely on the aforementioned partition mesh . Let be the partition mesh on . The partition mesh on is determined by the where is the matching function discussed in section 3. To introduce our registration map, first let us introduce some notations. The partition mesh of the unit disk is denoted by , here
and
are the set of vertices and the set of triangles in where is the coordinate of the th vertex and is the indices of the vertices of the th triangle, and denote the number of vertices in and the number of triangles in . Obviously, the closed region determined by can be easily represented by
where is the barycentric coordinate of with respect to the vertices , and . A simple piecewise linear registration map can be easily constructed by the affine mapping
if , between each pair of triangles and , where
Recall is a unique representation of So, the surface registration can be realized through
Similarly, the texture images and of the surface and , respectively, can be registrated by
In the following, we introduce how we utilize the above surface registration method to generate the morphing sequence through the cubic spline homotopy of the mean curvatures and the conformal factors. Suppose 3D images of facial expressions are captured a time . Using the above surface registration method, the registration maps can be easily computed. Using these registration maps, a morphing path , and , can be created, here denotes the location where a point is morphed at time . Since is a unique representation of a surface, the morphing path can also be uniquely determined by the evolution of the conformal factor and the mean curvature. Here, we employee a piecewise cubic spline homotopy to interpolate
A sketch in Figure 8 illustrates this idea. Let denote the piecewise cubic spline function with given data . The conformal factor and the mean curvature at can now be evaluated by
Similarly, suppose the texture images of the surface , are given. The texture image associated with the surface along the morphing path can be computed by the cubic spline homotopy
Finally, by reconstructing the 3D surfaces from their representation and applying the computed texture image to , the morphing sequence between and can be obtained for any period from to . In the following section, we shall introduce the surface reconstrution algorithm in detail.
6 LaplaceBeltrami Surface Reconstruction
In the previous section, we have obtained the mean curvatures and the conformal factors at each time period , . Gu and Yau proposed that any surface in 3D Euclidean space can be determined by its conformal factor and mean curvature uniquely up to rigid motions [9]. We could generate the morphing sequence of surfaces by reconstructing the unique surface via , .
In the following, we introduce how we utilize the mean curvatures and the conformal factors to reconstruct surfaces by solving the LaplaceBeltrami equations. Applying Gu and Yau’s method, for the given mean curvature and the conformal factor , we reconstruct the unique 3D surface by solving the LaplaceBeltrami equations
where
and is the normal of the surface . The detail algorithm for solving the LaplaceBeltrami equations can be seen in Algorithm 1.
We reconstruct 7 frames from the video at time respectively. The surface captured from the video at time is denoted by and the reconstructed surface is denoted by . In order to verify that the reconstructed surface is approximately the real surface, we compute the difference between the real surface and the reconstructed surface in sense
and in sense
respectively. Table II shows the reconstruction error in both norm and norm.
t  

7 Numerical Results of Surface Morphing in 3D
In the following, we show some surface morphing results via merely two images by using the surface morphing technique which we have mentioned above. In each figure, the left most image is the role of initial surface while the right most image is the role of terminal surface and the images in the middle are the morphing sequence between and .
Figure 9 and Figure 10 show the morphing sequence between two different facial expression, respectively. Figure 11 and Figure 12 show the morphing sequence between faces of two different people, respectively. Figure 13 shows the morphing sequence between a human face and a loin’s head.
It is interesting that how much the global matching function improves the morphing sequence. The affine mapping constructed by using the optimal Möbius transformation is denoted by and the affine mapping constructed by using the optimal Möbius transformation with the global matching function is denoted by , . In order to measure the rate of improvement, we reconstruct the surfaces
and
where
and
Then, we compute the surface difference
and
The rate of improvement is defined by
Table III indicates that the global matching function improves approximately of the surface difference.
Improvement Rate  

0.3  42.08%  
0.4  45.37%  
0.5  54.22%  
1.3  55.01%  
1.4  51.36% 
Moreover, extrapolation can also be achieved by using this homotopy technique. The real surface at time is denoted by and the reconstructed surface at time is denoted by where
Table IV shows the surface difference between the real surface and the reconstructed surface in sense
and in sense
respectively, at time and
1.4  

1.5  
1.6  
1.7 
8 Conclusion
In this paper, we proposed a 3D surface morphing method between different simply connected surfaces with single boundary in which smooth transient on both geometric characteristics and texture of the surfaces is considered. Similar to the traditional morphing approaches based on boundary representation, a wrap has to be created via feature correspondence and interpolation between shapes based on the wrap is employed to generate the morphing sequence. By taking advantage of the conformal parameterization and the unique surface representation of conformal factor and mean curvature, the wrap can be easily obtained by the composition of deformations from the Möbius transformation and the thinplate matching function. To mimic the nonisomorphic risk that usually occurs in matching largely deformed surfaces, a single mesh based on geodesic frame is employed. As a result, the correspondence, including geometric information and texture information, of the whole surface can be defined and interpolation among original surface and target surface can be computed by the usual cubic spline homotopy in a disk parametric domain. Finally, the morphing sequence can be generated from the surface reconstruction algorithm in section 6. To make the proposed morphing approach more attractive in real applications, we improve the efficiency in computing the conformal parameterization and geodesic frames. We propose an nonlinear iterative surface reconstruction algorithm (Algorithm 1). The surface reconstruction algorithm can be accelerated by using the multigrid method on a uniform mesh by which multiresolution surfaces can also be obtained. Several morphing effects among different 3D facial expressions are presented to demonstrate the feasibility of the proposed morphing method.
References
 [1] Y. Bao, X. Guo, and H. Qin. Physically based morphing of pointsampled surfaces. Computer Animation and Virtual Worlds, 16:509–518, 2005.

[2]
V. Camion and L. Younes.
Geodesic interpolating splines.
Energy Minimization Methods in Computer Vision and Pattern Recognition, Lecture notes in Computer Science
, 2134:513–527, 2001.  [3] T. F. Cootes, C. J. Taylor, D. H. Cooper, and J. Graham. Actives shape models  their training and application. Computer Vision and Image Understanding, 61(1), 1995.
 [4] D. DeCarlo and J. Gallier. Topological evolution of surfaces. Graphics Interface, pages 194–203, 1996.
 [5] H. W. Engl, M. Hanke, and A. Neubauer. Regularization of Inverse Problems. Kluwer Academic Publishers, 1 edition, 1996.
 [6] A. Gregory, A. State, M. C. Lin, D. Manocha, and M. A. Livingston. Interactive surface decomposition for polyhedral morphing. The Visual Computer, 15:453–470, 1999.
 [7] X. Gu, T.M. Huang, W.Q. Huang, S.S. Lin, W.W. Lin, and S.T. Yau. High performance computing for spherical conformal and Riemann mappings. Geometry, Imaging and Computing, 1(2):223–258, 2014.
 [8] X. Gu, Y. Wang, T. F. Chan, P. M. Thompson, and S.T. Yau. Genus zero surface conformal mapping and its application to brain surface mapping. IEEE Transactions on Medical Imaging, 8, 2004.
 [9] X. Gu, Y. Wang, and S.T. Yau. Geometric compression using Riemann surface structure. Commumications in Information and Systems, 3:171–182, 2004.
 [10] X. Gu and S.T. Yau. Computational Conformal Geometry. Higher Education Press, 1 edition, 2008.
 [11] E. Jeong, M. Yoon, Y. Lee, M. Ahn, S. Lee, and B. Guo. Featurebased surface light field morphing. Proceedings of the 11th Pacific Conference on Computer Graphics and Applications, 2003.
 [12] S.Y. Lee, K.Y. Chwa, J. Hahn, and S. Y. Shin. Image morphing using deformable surfaces. Proceedings of Computer Animation, pages 31–39, 1994.
 [13] N. Litke, M. Droske, M. Rumpf, and P. Schröder. An image processing approach to surface matching. The Eurographics Association, 2005.
 [14] Y.S. Liu, H.B. Yan, and R. R. Martin. Asrigidaspossible surface morphing. Journal of Computer Science and Technology, 26:548–557, 2011.
 [15] D. Martínez, L. Velho, and P. C. Carvalho. Computing geodesics on triangular meshes. Computers and Graphics, 29:667–675, 2005.
 [16] L. J. van Vliet and P. W. Verbeek. Curvature and bending energy in 2d and 3d images. Scandinavian Conference on Image Analysis, pages 1403–1410, 1993.
 [17] G. Wolberg. Image morphing: a survey. The Visual Computer, 14:360–372, 1998.
 [18] M. Yang, K. Wang, and L. Zhang. Realistic realtime facial expressions animation via 3d morphing target. Journal of Software, 8(2), 2013.
 [19] V. Zanella, G. Ramirez, H. Vargas, and L. V. Rosas. Automatic morphing of face images. 9th International Conference, ICANNGA 2009, 2009.
Comments
There are no comments yet.