A Facial Aging Simulation Method Using flaccidity deformation criteriaAlexandre Cruz Berg Lutheran University of Brazil.Dept Computer ScienceRua Miguel Tostes, 101. 92420-280 Canoas, RS, Brazil berg@ulbra.tche.br Francisco José Perales LopezUniversitat les Illes Balears.Dept Mathmatics InformaticsCtra Valldemossa, km 7,5E-07071 Palma MallorcaSpainpaco.perales@uib.esManuel GonzálezUniversitat les Illes Balears.Dept Mathmatics InformaticsCtra Valldemossa, km 7,5E-07071 Palma MallorcaSpainmanuel.gonzales@uib.esAbstractDue to the fact that the aging human face encompasses skull bones, facial muscles, and tissues, we render it using the effects of flaccidity through the observation of family groups categorized by sex, race and age. Considering that patterns of aging are consistent, facial ptosis becomes manifest toward the end of the fourth decade. In order to simulate facial aging according to these patterns, we used surfaces with control points so that it was possible to represent the effect of aging through flaccidity. The main use of these surfaces is to simulate flaccidity and aging consequently.1.IntroductionThe synthesis of realistic virtual views remains one of the central research topics in computer graphics. The range of applications encompasses many fields, including: visual interfaces for communications, integrated environments of virtual reality, as well as visual effects commonly used in film production.The ultimate goal of the research on realistic rendering is to display a scene on a screen so that it appears as if the object exists behind the screen. This description, however, is somewhat ambiguous and doesn't provide a quality measure for synthesized images. Certain areas, such as plastic surgery, need this quality evaluation on synthesized faces to make sure how the patient look like and more often how the patient will look like in the future. Instead, in computer graphics and computer vision communities, considerable effort has been put forthto synthesize the virtual view of real or imaginary scenes so that they look like the real scenes.Much work that plastic surgeons put in this fieldis to retard aging process but aging is an inevitable process. Age changes cause major variations in the appearance of human faces [1]. Some aspects of aging are uncontrollable and are based on hereditary factors; others are somewhat controllable, resulting from many social factors including lifestyle, among others [2].1.1.Related WorkMany works about aging human faces have been done. We can list some related work in the simulation of facial skin deformation [3].One approach is based on geometric models, physically based models and biomechanical models using either a particle system or a continuous system.Many geometrical models have been developed, such as parametric model [4] and geometric operators [5]. The finite element method is also employed for more accurate calculation of skin deformation, especially for potential medical applications such as plastic surgery [6]. Overall, those works simulate wrinkles but none of them have used flaccidity as causing creases and aging consequently.In this work is presented this effort in aging virtual human faces, by addressing the synthesis of new facial images of subjects for a given target age.We present a scheme that uses aging function to perform this synthesis thru flaccidity. This scheme enforces perceptually realistic images by preserving the identity of the subject. The main difference between our model and the previous ones is that we simulate increase of fat and muscular mass diminish causing flaccidity as one responsible element for the sprouting of lines and aging human face.In the next section will plan to present the methodology. Also in section 3, we introduce the measurements procedure, defining structural alterations of the face. In section 4, we present a visual facial model. We describe age simulation thrua deformation approach in section 5. In the last section we conclude the main results and future work.2.MethodologyA methodology to model the aging of human face allows us to recover the face aging process. This methodology consists of: 1) defining the variations of certain face regions, where the aging process is perceptible; 2) measuring the variations of those regions for a period of time in a group of people and finally 3) making up a model through the measurements based on personal features.That could be used as a standard to a whole group in order to design aging curves to the facial regions defined.¦njjjpVM2.1Mathematical Background and AnalysisHuman society values beauty and youth. It is well known that the aging process is influenced by several parameters such: feeding, weight, stress level, race, religious factors, genetics, etc. Finding a standard set of characteristics that could possibly emulate and represent the aging process is a difficult proposition.This standard set was obtained through a mathematical analysis of some face measurements in a specific group of people, whose photographs in different ages were available [7]. To each person in the group, there were, at least, four digitized photographs. The oldest of them was taken as a standard to the most recent one. Hence, some face alterations were attained through the passing of time for the same person.The diversity of the generated data has led to the designing of a mathematical model, which enabled the acquiring of a behavior pattern to all persons of the same group, as the form of a curve defined over the domain [0,1] in general, in order to define over any interval [0,Į] for an individual face. The unknown points Įi are found using the blossoming principle [8] to form the control polygon of that face.The first step consisted in the selection of the group to be studied. Proposing the assessment of the face aging characteristics it will be necessary to have a photographic follow-up along time for a group of people, in which their face alterations were measurable.The database used in this work consisted of files of patients who were submitted to plastic surgery at Medical Center Praia do Guaíba, located in Porto Alegre, Brazil.3.MeasurementsAccording to anatomic principles [9] the vectors of aging can be described aswhich alter the position and appearance of key anatomic structures of the face as can be shown in figure 1 which compares a Caucasian mother age 66 (left side) with her Caucasian daughters, ages 37 (right above) and 33 (right below) respectively.Figure 1 - Observation of family groupsTherefore, basic anatomic and surgical principles must be applied when planning rejuvenative facial surgery and treating specific problems concomitantwith the aging process.4.Visual Facial ModelThe fact that human face has an especially irregular format and interior components (bones, muscles and fabrics) to possess a complex structure and deformations of different face characteristics of person to person, becomes the modeling of the face a difficult task. The modeling carried through in the present work was based on the model, where the mesh of polygons corresponds to an elastic mesh, simulating the dermis of the face. The deformations in this mesh, necessary to simulate the aging curves, are obtained through the displacement of the vertexes, considering x(t) as a planar curve, which is located within the (u,v ) unit square. So, we can cover the square with a regular grid of points b i,j =[i/m,j/n]T ; i=0,...,m; j=0,...,n. leading to every point (u,v ) asfrom the linear precision property of Bernstein polynomials. Using comparisons with parents we can distort the grid of b i,j into a grid b'i,j , the point (u,v )will be mapped to a point (u',v') asIn order to construct our 3D mesh we introduce the patch byAs the displacements of the vertexes conform to the certain measures gotten through curves of aging and no type of movement in the face is carried through, the parameters of this modeling had been based on the conformation parameter.4.1Textures mappingIn most cases the result gotten in the modeling of the face becomes a little artificial. Using textures mapping can solve this problem. This technique allows an extraordinary increase in the realism of the shaped images and consists of applying on the shaped object, existing textures of the real images of the object.In this case, to do the mapping of an extracted texture of a real image, it is necessary that the textureaccurately correspond to the model 3D of that is made use [9].The detected feature points are used for automatic texture mapping. The main idea of texture mapping is that we get an image by combining two orthogonal pictures in a proper way and then give correct texture coordinates of every point on a head.To give a proper coordinate on a combined image for every point on a head, we first project an individualized 3D head onto three planes, the front (x, y), the left (y, z) and the right (y, z) planes. With the information of feature lines, which are used for image merging, we decide on which plane a 3D-head point on is projected.The projected points on one of three planes arethen transferred to one of feature points spaces suchas the front and the side in 2D. Then they are transferred to the image space and finally to the combined image space.The result of the texture mapping (figure 2) is excellent when it is desired to simulate some alteration of the face that does not involve a type of expression, as neutral. The picture pose must be the same that the 3D scanned data.¦¦¦ mi nj lk n j m i lk k j i w B v B u B b w v u 000,,)()()(')',','(¦¦ m i nj n jmij i v B u B b v u 00,)()(),(¦¦ m i nj n j m i j i v B u B b v u 00,)()(')','(¦¦¦ mi nj lk n j m i lk k j i w B v B u B b w v u 000,,)()()(')',','(Figure 2 - Image shaped with texturemapping5.Age SimulationThis method involves the deformation of a face starting with control segments that define the edges of the faces, as¦¦¦ mi nj lk n j m i lk k j i w B v B u B b w v u 000,,)()()(')',','(Those segments are defined in the original face and their positions are changed to a target face. From those new positions the new position of each vertex in the face is determined.The definition of edges in the face is a fundamental step, since in that phase the applied aging curves are selected. Hence, the face is divided in influencing regions according to their principal edges and characteristics.Considering the face morphology and the modeling of the face aging developed [10], the face was divided in six basic regions (figure 3).The frontal region (1) is limited by the eyelids and the forehead control lines. The distance between these limits enlarges with forward aging.The orbitary region (2) is one of the most important aging parameters because a great number of wrinkles appears and the palpebral pouch increases [11]. In nasal region (3) is observed an enlargement of its contour.The orolabial region (4) is defined by 2 horizontal control segments bounding the upper and lower lips and other 2 segments that define the nasogenian fold. Figure 3 - Regions considering the agingparametersThe lips become thinner and the nasogenian fold deeper and larger. The mental region (5) have 8 control segments that define the low limit of the face and descend with aging. In ear curve (6) is observed an enlargement of its size. The choice of feature lines was based in the characteristic age points in figure 6.The target face is obtained from the aging curves applied to the source face, i.e., with the new control segment position, each vertex of the new image has its position defined by the corresponding vertex in the target face. This final face corresponds to the face in the new age, which was obtained through the application of the numerical modeling of the frontal face aging.The definition of the straight-line segment will control the aging process, leading to a series of tests until the visual result was adequate to the results obtained from the aging curves. The extremes of the segments are interpolated according to the previously defined curves, obtained by piecewise bilinear interpolation [12].Horizontal and vertical orienting auxiliary lines were defined to characterize the extreme points of the control segments (figure 4). Some points, that delimit the control segments, are marked from the intersection of the auxiliary lines with the contour of the face, eyebrow, superior part of the head and the eyes. Others are directly defined without the use of auxiliary lines, such as: eyelid hollow, eyebrow edges, subnasion, mouth, nasolabial wrinkle andnose sides.Figure 4 - Points of the control segmentsOnce the control segments characterize the target image, the following step of the aging process can be undertaken, corresponding to the transformations of the original points to the new positions in the target image. The transformations applied to the segments are given by the aging curves, presented in section 4.In the present work the target segments are calculated by polynomial interpolations, based on parametric curves [12].5.1Deformation approachThe common goal of deformation models is to regulate deformations of a geometric model by providing smoothness constraints. In our age simulation approach, a mesh-independent deformation model is proposed. First, connected piece-wise 3D parametric volumes are generated automatically from a given face mesh according to facial feature points.These volumes cover most regions of a face that can be deformed. Then, by moving the control pointsof each volume, face mesh is deformed. By using non-parallel volumes [13], irregular 3D manifolds are formed. As a result, smaller number of deformvolumes are necessary and the number of freedom incontrol points are reduced. Moreover, based on facialfeature points, this model is mesh independent,which means that it can be easily adopted to deformany face model.After this mesh is constructed, for each vertex on the mesh, it needs to be determined which particularparametric volume it belongs to and what valueparameters are. Then, moving control points ofparametric volumes in 3D will cause smooth facialdeformations, generating facial aging throughflaccidity, automatically through the use of the agingparameters. This deformation is written in matricesas , where V is the nodal displacements offace mesh, B is the mapping matrix composed ofBernstein polynomials, and E is the displacementvector of parametric volume control nodes.BE V Given a quadrilateral mesh of points m i,j ,, we define acontinuous aged surface via a parametricinterpolation of the discretely sampled similaritiespoints. The aged position is defined via abicubic polynomial interpolation of the form with d m,n chosen to satisfy the known normal and continuity conditions at the sample points x i,j .>@>M N j i ,...,1,...,1),(u @@>@>1,,1,),,( j j v i i u v u x ¦3,,),(n m n m n m v u d v u x An interactive tool is programmed to manipulate control points E to achieve aged expressions making possible to simulate aging through age ranges. Basic aged expression units are orbicularis oculi, cheek, eyebrow, eyelid, region of chin, and neck [14]. In general, for each segment, there is an associated transformation, whose behavior can be observed by curves. The only segments that do not suffer any transformation are the contour of the eyes and the superior side of the head.5.2Deformation approachThe developed program also performs shape transformations according to the created aging curves, not including any quantification over the alterations made in texture and skin and hair color. Firstly, in the input model the subjects are required to perform different ages, as previouslymentioned, the first frame needs to be approximately frontal view and with no expression.Secondly, in the facial model initialization, from the first frame, facial features points are extracted manually. The 3D fitting algorithm [15] is then applied to warp the generic model for the person whose face is used. The warping process and from facial feature points and their norms, parametric volumes are automatically generated.Finally, aging field works to relieve the drifting problem in template matching algorithm, templates from the previous frame and templates from the initial frame are applied in order to combine the aging sequence. Our experiments show that this approach is very effective. Despite interest has been put in presenting a friendly user interface, we have to keep in mind that the software system is research oriented. In this kind of applications an important point is the flexibility to add and remove test facilities. 6.Results The presented results in the following figuresrefer to the emulations made on the frontalphotographs, principal focus of this paper, with theobjective to apply the developed program to otherpersons outside the analyzed group. The comparisonswith other photographs of the tested persons dependon their quality and on the position in which theywere taken. An assessment was made of the new positions, of the control segments. It consisted in: after aging a face, from the first age to the second one, through the use of polynomial interpolation of the control segments in the models in the young age, the new positions are then compared with the ones in the model of a relative of older age (figure 5). The processed faces were qualitatively compared with theperson’s photograph at the same age. Figure 5 - Synthetic young age model,region-marked model and aged modelAlso the eyelid hollow, very subtle falling of the eyebrow, thinning of the lips with the enlarging of the nasion and the superior part of the lip, enlargingof the front and changing in the nasolabial wrinkle.7.ConclusionsModelling biological phenomena is a great deal of work, especially when the biggest part of the information about the subject involves only qualitative data. Thus, this research developed had has a challenge in the designing of a model to represent the face aging from qualitative data.Due to its multi-disciplinary character, the developed methodology to model and emulate the face aging involved the study of several other related fields, such as medicine, computing, statistics and mathematics.The possibilities opened by the presented method and some further research on this field can lead to new proposals of enhancing the current techniques of plastic face surgery. It is possible to suggest the ideal age to perform face lifting. Once the most affected aging regions are known and how this process occurs over time. Also missing persons can be recognized based on old photographs using this technique. AcknowledgementsThe project TIN2004-07926 of Spanish Government have subsidized this work.8. References[1] Burt, D. M. et al., Perc. age in adult Caucasianmale faces, in Proc. R. Soc., 259, pp 137-143,1995.[2] Berg, A C. Aging of Orbicularis Muscle inVirtual Human Faces. IEEE 7th InternationalConference on Information Visualization, London, UK, 2003a.[3] Beier , T., S. Neely, Feature-based imagemetamorphosis, In Computer Graphics (Proc.SIGGRAPH), pp. 35-42, 1992.[4] Parke, F. I. P arametrized Models for FacialAnimation, IEEE Computer & Graphics Applications, Nov. 1982.[5] Waters, K.; A Muscle Model for Animating ThreeDimensional Facial Expression. Proc SIGGRAPH'87,Computer Graphics, Vol. 21, Nº4, United States, 1987. [6] Koch, R.M. et alia.. Simulation Facial SurgeryUsing Finite Element Models, Proceedings of SIGGRAPH'96, Computer Graphics, 1996.[7] Kurihara, Tsuneya; Kiyoshi Arai, ATransformation Method for Modeling and Animation of the Human Face from Photographs, Computer Animatio n, Springer-Verlag Tokyo, pp.45-58, 1991.[8] Kent, J., W. Carlson , R. Parent, ShapeTransformation for Polygon Objects, In Computer Graphics (Proc. SIGGRAPH), pp. 47-54, 1992. [9] Sorensen, P., Morphing Magic, in ComputerGraphics World, January 1992.[10]Pitanguy, I., Quintaes, G. de A., Cavalcanti, M.A., Leite, L. A. de S., Anatomia doEnvelhecimento da Face, in Revista Brasileira deCirurgia, Vol 67, 1977.[11]Pitanguy, I., F. R. Leta, D. Pamplona, H. I.Weber, Defining and measuring ageing parameters, in Applied Mathematics and Computation , 1996.[12]Fisher, J.; Lowther, J.; Ching-Kuang S. Curveand Surface Interpolation and Approximation: Knowledge Unit and Software Tool. ITiCSE’04,Leeds, UK June 28–30, 2004.[13]Lerios, A. et al., Feature-Based VolumeMetamorphosis, in SIGGRAPH 95 - Proceedings,pp 449-456, ACM Press, N.Y, 1995.[14]Berg, A C. Facial Aging in a VirtualEnvironment. Memória de Investigación, UIB, Spain, 2003b.[15]Hall, V., Morphing in 2-D and 3-D, in Dr.Dobb's Journal, July 1993.。
