Abstract
Automated 3D facial similarity measure is a challenging and valuable research topic in anthropology and computer graphics. It is widely used in various fields, such as criminal investigation, kinship confirmation, and face recognition. This paper proposes a 3D facial similarity measure method based on a combination of geodesic and curvature features. Firstly, a geodesic network is generated for each face with geodesics and iso-geodesics determined and these network points are adopted as the correspondence across face models. Then, four metrics associated with curvatures, that is, the mean curvature, Gaussian curvature, shape index, and curvedness, are computed for each network point by using a weighted average of its neighborhood points. Finally, correlation coefficients according to these metrics are computed, respectively, as the similarity measures between two 3D face models. Experiments of different persons’ 3D facial models and different 3D facial models of the same person are implemented and compared with a subjective face similarity study. The results show that the geodesic network plays an important role in 3D facial similarity measure. The similarity measure defined by shape index is consistent with human’s subjective evaluation basically, and it can measure the 3D face similarity more objectively than the other indices.
1. Introduction
Humans understand their living environment through instinctive perception of similarity or dissimilarity among objects around them. Research [1] has shown that people are particularly sensitive to facial similarity. Facial similarity is thus an important subject where its measures have widespread applications in many fields such as criminal investigation, kinship confirmation, and face recognition. Measurement of facial similarity is also an important research issue in anthropology, that is, research on the facial ethnical diversity. Although the perception of facial similarity is part of human instinct, the automation of face similarity measure is a challenging task since the structures of human faces are similar by and large.
Face recognition [2, 3] is thus a challenging task since the underlying issue is related to facial similarity measure. The aim of face recognition is to identify an unknown face through a certain similarity measure of face features. Therefore, in face recognition, discriminative features that can reflect the difference among different identities are extracted based on the adopted similarity measure in accordance with human perception. Previous researches are mainly focused on 2D face recognition [4]. With the development of 3D digitization technologies, the acquirement of 3D face data is becoming easier and easier. Recently, 3D face recognition has become an active research topic since it is less sensitive to variations in environmental conditions such as poses and illuminations in 2D face recognition. Therefore, 3D face similarity measure is an important research issue in recent context.
In 3D face similarity measure, geodesics refer to kernel geometric elements. A geodesic on a surface is the curve with geodesic curvature identically being zero. It is the shortest path between two points, and is also an intrinsic invariant on surfaces. Surface curvatures such as Gaussian, mean and principal curvatures are also important surface attributes. Both geodesics and curvatures have important roles in curved surface analysis. This paper proposes a 3D facial similarity measure method based on a combination of geodesics and curvatures. Geodesics are used to generate the correspondence points among 3D face models. Four curvatures, that is, the mean curvature, Gaussian curvature, shape index, and curvedness, are used for the similarity measure evaluation.
The rest of this paper is organized as follows. Related works in the field are presented in Section 2. Section 3 introduces the background about the geodesics. Section 4 describes the construction of the adopted geodesic network. Section 5 describes four metric computation associated with curvature. Section 6 presents the proposed similarity measures. Experimental results are provided in Section 7. Finally, conclusions are provided in Section 8.
2. Related Works
Similarity measure is commonly used to analyze and interpret data. In forensic medicine, some researchers [5, 6] evaluate their facial reconstruction results by subjective methods. A survey strategy is usually designed and then respondents are invited to compare the facial reconstruction results of some persons with their pictures. Although the subjective evaluation matches the human perception to some extent, it requires too many respondents and is time-consuming. More importantly, the subjective evaluation is usually limited by factors such as subjective cognition or life experiences. Hence, subjective methods cannot be easily generalized, and objective methods for evaluating the facial similarity are preferred.
Kare et al. [7] used the bidimensional regression (BDR) for 2D face similarity measure since BDR can assess the geometry resemblance between two planar point sets. Li et al. [8] proposed a method for 3D face similarity measure based on iso-geodesic stripes. They extracted a series of iso-geodesic stripes and the similarity of two faces was evaluated by comparing the 3D space distribution among the vertex sets in these stripes. These methods only discussed the influence of similarity measure in face recognition and did not consider the human perception. In order to make the similarity measure consistent with human perception, some researchers combine the human evaluation and automated methods. Holub et al. [9] built training samples by rating the facial similarity from a group of persons of 2D images and then learned the mapping from the face features extracted around some manually annotated facial key-points to the similarity ratings. The annotation of the facial key-points was apparently very tedious. Moorthy et al. [10] extracted Gabor features from feature points automatically detected on the range and texture images and demonstrated that the Gabor features correlate with human perception by comparing the evaluation results of the Gabor based measure with the subjective evaluation they designed. This method was mainly for range and texture images face similarity measure.
Similarity measure is the basis of face recognition. Early research in 3D face recognition mainly focused on curvature analysis. Cartoux et al. [11] proposed an approach based on principal curvature to segment a range image and to find the face symmetry plane for face recognition. Lee and Milios [12] created an Extended Gaussian Image (EGI) for each convex region which was segmented based on the sign of the mean and Gaussian curvatures, and performed matching of faces by correlating EGIs. Gordon [13] adopted Gaussian and mean curvatures to characterize delicate features in 3D faces. Tanaka et al. [14, 15] created EGI for each face based on the analysis of maximum and minimum principal curvatures and their directions. Recently, Smeets et al. [16] used meshSIFT features for 3D face recognition. They detected salient points in scale space by mean curvature extreme and then extracted features in the neighborhood of each salient point for face matching. This method was shown to be robust to expression variations, missing data, and outliers. Recently, some researchers used geodesics or iso-geodesics for face recognition due to the good nature of geodesics. Ter Haar and Veltkamp [17, 18] extracted the facial contour curve according to the geodesic distance and compared the similarity of facial contour curves for face recognition. Jahanbin et al. [19] used iso-depth and iso-geodesic facial curves for face matching. Berretti et al. [20, 21] evaluated the similarity by the spatial distribution features of the iso-geodesic facial stripes of an equal width. These methods based on geodesics are robust to face expression. These results show that designing similarity measure based on the face features via geodesic and curvatures have good recognition ability.
In our paper, a new approach is proposed for 3D facial similarity measure based on geodesic network and curvatures. Firstly, a geodesic network is generated for each face with geodesics and iso-geodesics and these network points are adopted as the correspondence across face models. Then, four metrics associated with curvatures, that is, the mean curvature, Gaussian curvature, shape index, and curvedness, are computed for each network point by using a weighted average of its neighborhood points. Finally, correlation coefficients according to these metrics are computed, respectively, as the similarity measures between two 3D face models.
Our method is different from other similar method above mentioned in the following aspects. Firstly, our method makes full use of geodesic properties and uses both geodesic and geodesic distance to build geodesic network by extracting geodesics and iso-geodesics. The above mentioned 3D recognition methods [17–21] or similarity methods [8] only extract the iso-geodesic or iso-geodesic stripes according to geodesic distance and ignore the geodesic which is intrinsic and is invariant under isometric transformation. So our method is more robust to face expression. Secondly, our method combines geodesic and curvatures for similarity measure instead of only using curvature for face recognition in [11–15]. Our method is more effective than the methods only based on curvatures because using geodesic network can better locate the salience points. Furthermore, we also compare the four metrics associated with curvatures, that is, the mean curvature, Gaussian curvature, shape index (SI), and curvedness, and experimental results illustrate that shape index can capture shape of a surface and the similarity value defined by SI is more reasonable than others. Thirdly, our method can not only recognize the face which is realized in face recognition method but also can give the quantity similarity value between two facial models; namely, we can measure how much one face looks like another face. So our method has more discriminating than other face recognition methods. The similarity of 3D face was paid little attention before as most researches of 3D face focus on face recognition. Our method provides an effective, robust, and discriminating 3D facial similarity measure method.
3. The Fundamentals of Geodesic Algorithms
In order to construct a geodesic network, geodesic distances are computed in advance. In this section, we shall provide an overview of the geodesic algorithm adopted in this paper.
The most classical geodesic algorithm is MMP algorithm proposed by Mitchell et al. [22]. The basic idea of MMP algorithm is based on the principle of light propagation along straight line. In MMP algorithm, firstly, a window function which records the shortest path information with a common edge sequence is defined, and then windows are generated. The positions of the pseudosources are calculated based on the windows, then the geodesic distance from any a point to source point on the model can be calculated through the windows. Geodesic can be obtained by backtracking. The first point of geodesic can be found by finding the point with the shortest geodesic distance on all windows of the face which the point is in. The other points of geodesics can be found in turn by traveling the adjacent windows. Thus, we can find the geodesic path from source to any one point on the surface.
We use the MMP algorithm which is implemented by Surazhsky et al. [23] in 2005. So we describe the specific process of the algorithm briefly.
(1) Definition of the Window. Firstly, a window is defined as a 6-tuple (Figure 1), where , respectively, measure distance along the edge, and , respectively, represent the distance from pseudosource to the two endpoints of the window, represents geodesic distance from pseudosource point to the source, and specifies the side of the edge on which the source lies.
(2) Generation, Propagation, and Cropping of the Window
Generating Window. An edge of a triangle adjacent to source point is taken as the initial window, and then a new window is generated by expanding it to the other two sides of the triangle. All generated windows are stored in a priority queue according to their order of distance from the source point.
Propagate Window. The propagation of the window includes three cases: an interval window may generate a new window (Figure 2(a)) or two new windows (Figure 2(b)); a pseudosource window, if it is located within the spherical point (the angle of the vertex is less than ), it does not generate new subwindow because the shortest path does not pass spherical point; if it is located in a saddle point (the angle of the vertex is greater than ), it will generate both the interval window and pseudosource window (Figure 2(c)).
(a) Generate one window
(b) Generate two windows
(c) Generate the interval and pseudosource window
Cropping Window. When the new created window overlaps with the original window , the windows need to be cropped by finding a point which satisfies the equation (Figure 3).
(3) Calculation of Geodesic Distance. Firstly the position of the pseudosource point is calculated. The two endpoints of the window are taken as the centers of circles; circles with radii and are drawn. When there is only one cross point, it is a pseudosource. When there are two intersections, a parameter is used to determine which one is the pseudosource. For any point of the model, if it is located inside the triangle, as shown in Figure 4(a), the shortest path length to source point through the window is where is the th point of the th window and is the th pseudosource. The geodesic distance is the minimum distance of all windows in three edges of the triangle; that is, . When the point is located on the edge or vertex, its geodesic distance to the source is .
(a) Calculate the geodesic distance
(b) Find the geodesic path
(4) Construction of Geodesic Path. After all edges are covered by windows representing geodesic distance, geodesic path can be constructed by tracking the shortest path from an arbitrary point of the surface back to the source point. First, find the point and its window which the shortest geodesic distance to point in its triangle, this is the first point in geodesic path (Figure 4(b)). Then, we can trace back to the source point to find the points of the shortest geodesic path. Geodesic path can be constructed by connecting these points in sequence.
4. Geodesic Network Construction
In order to compare the similarity between two 3D facial models, we need to establish the correspondence between the two facial models. We construct geodesic network through a number of evenly distributed geodesics and iso-geodesics. The network vertices are taken as corresponding points of two 3D facial models.
We assume that the compared three-dimensional facial model is a full triangular mesh model, which can be seen as a connected manifold surface in space. The surface can be represented as a triangular mesh with vertices , NE edges , s.t. iff and NT triangular faces , s.t. , and meets the following requirements: (1) there are no isolated vertices; (2) each edge only belongs to one triangle or is shared by two triangular faces; (3) any two triangular faces either do not intersect or only share one same vertex or a same edge; (4) there exists a geodesic between any two points of surface [24]. We can use the classic MMP algorithm [22] to compute geodesic on such models.
Before comparing the similarity of two facial surfaces, a unified coordinate system needs to be established, and 3D face models should be aligned to eliminate the effects of the translation, rotation of three-dimensional model [25].
4.1. Find the Nose Tip as a Source Point of Geodesics
In our method of geodesic extraction, the nose tip point is taken as the center and the geodesics are computed from the nose tip. So the first step is to find the tip point. In standard posture, the nose tip is the highest point of the whole face, so we can find the point of the biggest value (or value) of the entire 3D face model as the tip point of the nose (), see Figure 5(a). It is also the source point of the geodesics.
(a) The tip of nose
(b) Iso-geodesic
(c) The first geodesic
(d) Tangent plane
(e) Geodesic and iso-geodesic
(f) Geodesic network points
4.2. Extract Iso-Geodesics
An iso-geodesic is a line composed of points with equal geodesic distance leading to the tip point of nose. All iso-geodesics extracted from a face model form an iso-geodesic set . We use the following method to extract the iso-geodesics with equal interval between each two iso-geodesics. Firstly, the geodesic distances from the nose tip to all points of the facial model are computed. Then, the shortest geodesic distance from the nose tip to the boundary point of the facial model is found and the outermost iso-geodesic is extracted according to the shortest geodesic distance. Next the geodesic distance from the outermost iso-geodesic to the nose tip is equally divided into equal parts. Finally, the other iso-geodesics are extracted according to the same geodesic distance with equal intervals. Let width be the interval between two iso-geodesics, indicate the boundary point set, be the number of the division, and denote the geodesic distance function. Extracted the th iso-geodesic can be expressed as the following formula:
Through experiments we found that extracting eight iso-geodesics can obtain an almost ideal comparison. Hence, in our experimental setting, we extract eight iso-geodesics for facial similarity comparison, see Figure 5(b).
4.3. Find Equal Division Points on the Outermost Iso-Geodesic as Target Points of Geodesics
Firstly, on the outermost iso-geodesic, the point which connects the nose tip point and the middle point of “eyebrows” is found in order to find the first geodesic. In the standard pose, on the outermost iso-geodesic we find the point which has the same y coordinate component with the nose tip as the middle point of “eyebrows.” Through the two points we can find a plane vertical to the face. The cross curve of the plane and the face which connects the nose tip and the middle point of “eyebrows” can evenly divide a face into two parts. Thus, this curve is taken as the initial geodesic, for example, the first geodesic (Figure 5(c)).
Then from the nose tip point, geodesics can be obtained at a certain angle interval. In order to get the angle division, the tangent plane of the nose tip point is taken as the projection plane of the outermost iso-geodesic. In the standard pose, 3D facial models preprocessed, have been given a united coordinate system and have been aligned. The tangent plane is the plane where the point has the equal -coordinate. In order to simplify the calculation, firstly we build a coordinate system taking the nose tip as the origin point. Then all points’ coordinates are transformed into the new coordinates. Thus, the tangent plane can be found by the equation . On the tangent plane, the lines from the nose tip point to the projected outermost geodesic are computed by a certain angular interval, and their cross points are obtained (Figure 5(d)).
Finally, the cross points are projected into the original outermost iso-geodesic, which are the points that have equal angle between them. These points are taken as the target points of the geodesics. The equal division points are described by the following formula: where is the equal points, is the projection of on the tangent plane, and is the projection of on the tangent plane, is the angle of line between and . is the number of the equal division points and is equal to the number of geodesics. And these equal division points are taken as target points of geodesics.
4.4. Computing the Geodesics from the Tip of Nose to Each Equal Division Point
The geodesics from the source point (the nose tip) to the target points (equal division points ) are computed. All geodesics from the nose tip to equal division point can be represented as the set : , wherein where represents the geodesic from the point to an point , is the number of geodesics. This is a single-source-all-destination geodesic problem and can be solved by the existing geodesic algorithms, such as MMP [22], ICH [26], and PCH [27]. The classical MMP algorithm is used in our method and it is introduced in detail in Section 3. The geodesics are shown in Figure 5(e).
4.5. Calculate the Intersections of the Geodesics and Iso-Geodesics
After obtaining the geodesics and iso-geodesics, we can find the intersections of the geodesics and iso-geodesics; see Figure 5(f). They are the points which belong to both geodesic and iso-geodesic as follows: where is the cross point of the th geodesic and th iso-geodesic. is the total number of geodesics and is the total number of iso-geodesics.
Since we select the same initial directions for geodesics between two 3D facial models and the same center (the tip of nose), the cross-points of the geodesics and iso-geodesics are the corresponding points between two faces.
5. Computing Four Metrics Associated with Curvature
After constructing a network with a number of evenly distributed geodesics and iso-geodesics, we can compare the features of these corresponding network points between two 3D face models. Surface curvatures such as Gaussian, mean, and principal curvatures are intrinsic surface properties and have played important roles in curved surface analysis. We compute four kinds of values associated with curvature: the mean curvature, Gaussian curvature, shape index, and curvedness of the neighborhood around network points and compare the corresponding correlation coefficients respectively as the similarity measurement between two 3D face models.
We firstly calculate the principal curvatures and (reference to [28]), and then calculate the mean curvature, Gaussian curvature, shape index, and curvedness by and .
5.1. Principal Curvatures Estimation
Principal curvatures are two extreme of normal curvatures at a point on a surface, namely, the maximum/minimum normal curvatures , . A finite-differences approach [28] is used for estimating curvatures on triangle meshes in our method. We provide here a brief overview of the curvature algorithm.
The finite-differences approach is an extension of a common algorithm for finding per-vertex normal by averaging adjacent per-face normal. Its main principle is to solve the eigenvalues of the Weingarten matrix through per-vertex normal.
As we know from differential geometry, for a smooth surface, normal curvature satisfies the following equation: where is a unit-length vector in local tangent plane. The symmetric matrix is called Weingarten matrix or the second fundamental tensor. It can be diagonalized as follows: where and are the principal curvatures and is the principal directions. The Weingarten matrix is defined in terms of the directional derivatives of the surface normal: where are the directions of an orthonormal coordinate system in the tangent frame. Any vector in the tangent plane multiplying this matrix gives the derivative of the normal in that direction:
For discrete triangular mesh, Rusinkiewicz [28] uses three well-defined directions (the edges) together with the differences in normal in those directions (Figure 6). It can be expressed as follows:
Thus, can be solved by the least squares method. In order to average the contributions from adjacent triangles, each vertex is assumed to have its own orthonormal coordinate system , which is defined in the plane perpendicular to its normal, and derive a change-of-coordinates formula for transforming a curvature tensor into the vertex coordinate frame. The “Voronoi area” is taken as the weight of each adjacent triangle of vertex .
So the principal curvatures can be obtained by the following steps.
Step 1. Compute per-vertex normal.
Step 2. For each face, edge vectors and normal differences are computed, is solved by using least squares, the coordinates are changed into the vertex coordinate frame and each adjacent vertex curvature is averaged by “Voronoi area” weight for each vertex.
Step 3. For each vertex, the accumulated is divided by the sum of the weights. If desired, the eigenvalues and eigenvectors of are principal curvatures and directions, respectively.
5.2. Compute Four Metrics Associated with Curvature
After the principal curvatures and have been calculated, the mean curvature, Gaussian curvature, shape index, and curvedness can be obtained by and through the following formula. At a point on a surface, the Gaussian curvature is defined as the product of the two principal curvatures at the point for regular surface:
The mean curvature is defined as the average of the two principal curvatures at the point :
Shape index SI quantitatively measures the shape of a surface at a point . It is defined as where and are the principal curvatures with . This index is included in the Curvedness-Orientation-Shape Map On Sphere (COSMOS) representation [29]. The shape index captures the intuitive notion of “local” shape of a surface. Every distinct surface shape corresponds to a unique value of SI except the planar shape.
Another descriptor included in the COSMOS representation is the curvedness which represents the amount of curvature in a region. The curvedness of a surface at a point is defined as Curvedness measures how highly or gently bended a surface is, and its dimension is that of the reciprocal. It can capture the scale differences between objects (e.g., a soccer ball and a cricket ball) [29]. For each network vertex , four metric value can be computed, respectively, for example, mean curvature , Gaussian curvature , shape index Si, and curvedness .
5.3. Weighted Average of the Metrics in Network Vertex Neighborhood
In order to get robust results, we take the weighted average of the metrics of the network vertex neighborhood as metric value of the point. The detail steps are as follows.
Let be a network vertex and its mean curvature is , is the mean curvature of the th point in the 1-ring neighborhood of the network point , is the mean curvature of the th point in the 2-ring neighborhood of the network point , the mean curvature’s final metric value of the network vertex is computed by weighted average of the network vertex, 1-ring neighborhood vertexes and 2-ring neighborhood vertexes according to the following formula: where is the weight of the network vertex, is the weight of the vertex in 1-ring neighborhood, is the weight of the vertex in 2-ring neighborhood, is the 1-ring neighborhood of the network point , is the 2-ring neighborhood of the network point . denotes the weighted sum of mean curvature value of all points in 1-ring neighborhood and denotes the one in 2-ring neighborhood. The numerator is the sum of the weight. So the whole formula represents the weighted average value of mean curvature of network vertex, 1-ring neighborhood, and 2-ring neighborhood vertexes.
The network vertices play a prominent contribution in facial similarity measurement because it is intrinsic nature. So the weight should be higher than and should be higher than . In our experiment, we select the weight as follows: where is the number of the intersections of geodesics and iso-geodesics. is the number of points in 1-ring neighborhood of the intersections and is the number of points in 2-ring neighborhood of the intersections.
According to the above method, for all network vertices of a face, a weighted average of the mean curvature values can be obtained, which form a vector . Here, is the number of the network vertex. In the same way, weighted average values of the Gaussian curvature, shape index, and curvedness of network vertices can be obtained and form the vector, respectively, , , and .
6. Comparison of Facial Similarity Measures
For comparison of three-dimensional face similarity, we use the network points constructed by geodesics and iso-geodesics in Section 4 and then compute the four metric associated with the curvature of network points: mean curvature, Gaussian curvature, shape index, and curvedness. We can, respectively, define the facial similarity as the correlation coefficient of the four metric values according to corresponding geodesic network points between two 3D face models.
Let us consider two three-dimensional face models and (see Figure 7) where and are sets of corresponding network points between two 3D face models and , respectively.
(a) Model
(b) Model
The order of the points in above vectors is determined by the iso-geodesics and geodesics. All iso-geodesics for each model are stored according to the geodesic distances to the nose tip point in ascending order. And all geodesics are extracted and stored clockwise according to the same initial direction and same angular interval. The geodesics and iso-geodesics of two models are stored by the same rules, thus they are one-to-one correspondence. The cross points on two models which are stored according to the orders of iso-geodesics and geodesics are also one-to-one correspondence.
Four weighted average metric value of each network points are calculated according to the above formula: mean curvature, Gaussian curvature, shape index, and curvedness as follows. Mean curvature: and . Gaussian curvature: and . Shape index: and . Curvedness: and .
In order to compare the similarity of two 3D facial models, we adopt the corresponding correlations of the above four kinds of average value, respectively, as the similarity measurement between two 3D face models. The correlation coefficients of four feature values are calculated with the following formula:
The correlation coefficient is adopted to measure the linear closeness between the two sets of variables, which ranges within the interval . In fact, the characteristics of correlation coefficient are similar to the angle cosine. When comparing the facial similarity, we take the absolute value of the correlation coefficient in order to make it within . A greater value shows stronger correlation between the compared features. A correlation coefficient of value 1 means that the two vectors are exactly the same. Therefore, when comparing two 3D faces, if the value is close to 1, the two 3D faces are close to similar. On the contrary, the two 3D faces are near different if the value is close to 0.
7. Experiments and Discussion
In order to verify the effectiveness of the proposed similarity measure method, the similarity measure experiments of different persons’ 3D facial models and different 3D facial models of the same person have been implemented.
7.1. The Similarity Measure of Different Persons’ 3D Facial Models
3D facial similarity measure should give the correct similarity score of 3D facial models of different persons. But the facial similarity is an ambiguous and relative concept for humans generally [30]. So we have the following two experiments to verify our method. First, we have experiments on morph data; that is, we generate several morph deformed faces which we know the similarity between each two faces in advance. Therefore, we can verify whether the similarity calculated by our method is correct or not. Second, we do experiments on different real face models and compare the results calculated by our method with the subjective evaluation results which are obtained by a subjective experiment of 40 subjects.
7.1.1. Experiments on Morph Data
As in [10], we randomly select two 3D facial models and then produce morph deformed faces using the formula , where the parameter can control the degree upon how much the two faces are mixed. If , the generated face is the same with . If , it is the same with . The value of reflects the similarity between the generated face with the original faces. Take , for example, the generated face should be closer to the face , while the face generated by should be closer to the face .
We use the original two 3D facial model and to generate three new faces “New1,” “New2,” and “New3” by the above morph deformation method, respectively, corresponding to . Then we compare their similarity between every two faces by our combination method of geodesic networks and curvatures and get the correlation coefficients are listed in Tables 1, 2, 3, and 4.
The data of Tables 1–4 show that the similarity of the same face is 1; that is, the value in the diagonal of the matric and the matric is symmetric. In order to display the laws of data more intuitively, we draw the following color diagram through which similarity 1 is represented by red and the min value is indicated by blue according to the value of similarity in Tables 1–4.
As can be seen from Figure 8 and Tables 1–4, the trend of similarity value between the face “New1,” “New2,” “New3,” and “” with the face defined by shape index (SI) is sequentially decrease, so the color gradually changes from red to blue (red color represents the maximal value and blue color represents the minimal value). The similarity of other faces has the same laws, the variation trends of similarity value defined by SI are consonant with the parameter values used by morph deformation face and also consistent with person’s subjective judgment. The similarity values defined by , , and have no notable trends, and their discrimination is weaker than that of SI. So the similarity values defined by SI is more reasonable than defined by , , and .
(a) SI
(b)
(c)
(d)
7.1.2. Experiments on Real 3D Face Data
The real 3D face dataset is from VRVT lab of Beijing Normal University [31, 32]. It includes 208 CT face scans of individuals aged from 19 to 75 years old. There are 81 females and 127 males. The mesh correspondence across the dataset has been established as described in [33], and each face mesh has 40969 vertices. All the 3D facial data are substantially complete and can be seen as a manifold after denoising, filling-up holes, and other preprocess.
The preprocessed 112 facial models are used to similarity comparison by using our method which based on geodesic network and curvatures. The similarity values of each two models have been calculated. In order to verify these similarity values’ reasonableness, we have conducted a subjective study to evaluate the similarity of the 112 facial models. The 112 facial models are divided into two parts, each part having eight sets. Meanwhile the 40 subjects were equally divided into two groups, each group having 20 subjects. The subjects in the first group evaluate one part and the subjects in the second group evaluate the other part. Each subject evaluates eight sets of facial models, each set having seven face models, one reference model, and six other facial models for comparison. The seven face models are showed on the screen in two rows, the reference faces are showed in the middle of first row, and the other six faces are listed in (Table 5).
Subjects are required to observe the faces on the screen and assess the similarity between each of six faces and the reference face, respectively, by their own judgment. A 7-point Likert scale [34] is used to evaluate the similarity. A value 1 represents that the two faces are completely dissimilar while a value of 7 implies that the two faces are exactly the same. Subjects were informed by these directions in advance and none of subjects had any trouble understanding the directions. After obtaining the evaluated similarity grade of each subject, we can compute the mean similarity grade between two faces evaluated by 20 subjects and take the mean similarity grade as their similarity grade. We can compare the similarity grades obtained by subjective evaluation with the objective similarity values calculated by our method. Take one set of data as an example, the comparisons between similarity scores calculated according to SI, , , and and the subjective evaluating similarity grade are listed in Tables 6, 7, 8, and 9.
From Tables 6–9, we can see that the most similar face with reference face according to SI, , or is the same one face (008-26040000), which is the highest similarity grade in the subjective study. The most similar result found by the similarity calculated by SI, , and is consistent with the subjective and the result calculated by is not consistent with the subjective.
In Table 6, the left three faces of the six faces compared with reference face, that is, Numbers 008-2604, 015-2517, and 006-1604 have high similarity values calculated by SI and high similarity grade in the subjective study, and the right three faces, that is, Numbers 004-5343, 180-29579, and 008-1544 have low similarity values calculated by SI and low similarity grade in the subjective study. The similarity order of the left three similar faces calculated by SI is also consistent with the subjective evaluation. The similarity order of the right three dissimilar faces is not consistent with the subjective evaluation, it is because that we hardly tell who is the most dissimilar when they are all dissimilar with the reference face. So the results in Table 6 show that the similarity calculated by SI is consistent with the persons’ subjective evaluation basically.
In Table 7, the left three faces of the six faces compared with reference face, that is, Numbers 008-2604, 015-2517, and 006-1604 have high similarity values calculated by and are consistent with the subjective study, but the right three faces that is, Numbers 004-5343, 180-29579, and 008-1544 also have high similarity values calculated by and are not consistent with the subjective study. So the similarity calculated by cannot distinguish the similarity degrees of different people.
In Table 8, the similarity calculated by is not consistent with the subjective evaluation, such as the face Number 015-2517 is similar with the reference face in subjective evaluation, but the similarity value calculated by is too low. The face Numbers 004-5343 and 008-1544 are dissimilar with the reference face while the similarity values calculated by is high. So the similarity calculated by does not represent the similarity degrees of different people.
In Table 9, the similarities calculated by are all low and are not consistent with the subjective evaluation, because is intrinsic and does not distinguish the faces of different people. So the similarity calculated by does not represent the similarity degrees of different people.
In summary, the similarity calculated by SI is consistent with the persons’ subjective evaluation basically and the similarities calculated by , , or are not consistent with persons’ subjective evaluation. Both the results of morph data and the real data of different people illustrate that the similarity calculated by SI can represent the similarity of different facial models. So we take it as the measure of facial similarity which is adopted in the following experiments.
7.2. The Similarity Measure of Different 3D Facial Models of the Same Person
In order to verify our similarity measure method whether it can effectively distinguish the different 3D face models of the same person with different people’s 3D face models, we have experiments on public 3D face dataset which have several models of the same person. The used data are the range data from Texas 3D Face Recognition Database and the Gavadb dataset. The similarity measure defined by SI is adopted in this section because the similarity value defined by SI is more reasonable which has been illustrated in the prior section.
7.2.1. Experiments on 3D Face Data from Gavadb Dataset
Gavadb is a 3D face dataset which includes 3D facial models of 61 individuals (45 male and 16 female) [35]. Each person has nine models scanned by a Minolta Vi-700 laser range in different poses or under different facial expressions. Each facial model is represented in a three-dimensional mesh.
In order to eliminate the effects of the translation, rotation of three-dimensional model, these 3D face models are standardized by a unified coordinate system and aligned by a TPS based registration algorithm [33]. The similarity measure between two 3D face models can be obtained by our geodesic and curvature method. We take the eight frontal facial models of four persons (Figure 9) and get the similarity as shown in Table 10.
From Table 10, we can see that the similarity of models from the same person is high and the similarity of models from different persons is low. The similarity of the same person’s models is close to 1 because the two models have the same expression. So the similarity defined by the correlation coefficient of shape index can reflect the similarity of the facial models and the results are consistent with the persons’ subjective evaluation results.
7.2.2. Experiments on the Range Data from Texas 3D Face Recognition Database
Texas 3D Face Recognition database [36] contains 1149 pairs of high resolution color and range images of 118 adult human subjects acquired by a stereo camera. It includes the range images of one person in different pose, different expression, and different illumination. We firstly recover the 3D face point cloud data (Figure 10(c)) from the range images (Figure 10(b)) by making the gray value as the third dimension coordinate value. Then, the point cloud data are triangulated into mesh models. Thirdly, the face mesh models are denoised, filled-up holes, and other pretreated. Lastly, these models can be compared by our combination of geodesic network and curvature method.
(a) Portrait
(b) Range
(c) 3D model
We take ten face models from five persons in different expressions (Figure 11) and get the similarity measures as shown in Table 11.
From Table 11, we can see that, for the models of one person in different expressions, the similarity is high and for different persons the similarity is low. So the similarity defined by the correlation coefficient of shape index can differentiate the facial models of different persons with facial models of the same person. Although the similarity value of the two models from the same person is not very close to 1 because of their different expressions and the effects of the noise of the model surface, it makes a good distinction between different persons’ models with the same person’s models. So the similarity defined by the correlation coefficient of shape index can reflect the similarity of facial models and the results are consistent with the persons’ subjective evaluation results.
7.3. Discussion
Above experimental results show that the correlation coefficient of the shape index between two 3D face models can better reflect the similarity of the two faces. So, at last, we define the facial similarity as the correlation coefficient of the two vectors of shape index on corresponding geodesic network points between two 3D face models.
In the proposed method, the number of geodesic and iso-geodesic is an important parameter. In experiments, we change the number of geodesic and iso-geodesic and find that it is not true that the larger the number is, the better the results are. For example, we double the number of geodesic and iso-geodesic, respectively, and some results do not improve but decline. In experiments we take eight iso-geodesics and fifteen geodesics, by which we can obtain the similarity values close to the actual situation.
8. Conclusion
In this paper, we proposed a new method based on a combination of geodesic network and curvatures for 3D facial similarity measure. Using the nose tip as the center, we constructed a geodesic network with numbers of evenly distributed geodesics and iso-geodesics. Then, we computed four kinds of metric average values associated with curvatures, that is, the mean curvature, Gaussian curvature, shape index, and curvedness of the neighborhood around network points. Then, the correlation coefficients according to these metrics were computed, respectively, as the similarity measures between two 3D face models. This method can be easily extended to triangle meshes with holes or point cloud data by replacing MMP algorithm by the corresponding geodesic algorithms.
Through the objective experiments of different persons’ 3D facial models, including 3D morph face models, real 3D face models and different 3D facial models of the same person on Texas 3D Face Recognition Database and the Gavadb dataset and subjective study on face similarity by inviting 40 subjects evaluating 112 face models, we find that the features of the network vertices of geodesics and iso-geodesics play an important role in 3D facial similarity measurement. The similarity measure defined by the correlation coefficient of shape index is consistent with human’s subjective evaluation basically and it can measure the 3D face similarity more objectively than other indices.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
The authors gratefully appreciated the anonymous reviewers for all of their helpful comments. The authors thank Professors Alan C. Bovik and Shalini Gupta for providing the data of Texas 3D Face Recognition Database and thank the providers of Gavadb dataset. They also thank Surazhsky et al. and Rusinkiewicz for their part public code of geodesic or curvature. This work is partially supported by a Grant from the National Natural Science Foundation of China (nos. 61170170 and 61272363) and Program for New Century Excellent Talents in University (NCET-13-0051).