Signal and Image Processing of Physiological Data: Methods for Diagnosis and Treatment PurposesView this Special Issue
Research Article | Open Access
A Multiscale Constraints Method Localization of 3D Facial Feature Points
It is an important task to locate facial feature points due to the widespread application of 3D human face models in medical fields. In this paper, we propose a 3D facial feature point localization method that combines the relative angle histograms with multiscale constraints. Firstly, the relative angle histogram of each vertex in a 3D point distribution model is calculated; then the cluster set of the facial feature points is determined using the cluster algorithm. Finally, the feature points are located precisely according to multiscale integral features. The experimental results show that the feature point localization accuracy of this algorithm is better than that of the localization method using the relative angle histograms.
With the development of 3D information acquisition technology, the research of 3D facial feature has gained more and more extensive attention. The automatic localization of the facial feature points is a study hotspot in the field of medical computer vision, which is a precondition for face recognition, face animation, face tracking, and 3D face reconstruction. At present, the 3D facial feature point localization algorithms have not been researched in depth though the 2D facial feature point localization algorithms have matured. Wang et al.  applied the Jet Bunch algorithm of 2D facial feature point localization to the process of 3D facial feature point localization; Xu et al.  proposed a hierarchical filtering algorithm, which could locate the feature points of nasal tip. While these methods are beneficial to detect other facial key feature points, they are insensitive to noises and have the rotation and translation invariance characters with a poor generality, and the localization results are unsatisfactory except the nasal tip feature points. Feng et al.  proposed a feature point localization algorithm based on relative angle histograms. In a 3D point distribution model, the relative angle histograms of all points are calculated firstly, and then the features are located according to the similarity of the histograms. The algorithm is of high efficiency and stability, but it locates the feature points just in a small scope with an inaccurate result. Li and Da  suggested a 3D facial feature point localization algorithm that combined a priori knowledge with differential characteristics, but the result is not satisfactory because of the unique and complex diversity of the face model.
In order to solve these problems, we put forward a 3D facial feature point localization method based on the relative angle histograms and multiscale constraints. The cluster point set of the facial features is created firstly, and then the multiscale integral characteristics are used to locate the feature points accurately. As a result, the accuracy of the feature point localization is improved.
2. Model Preprocessing
2.1. Coordinate System Transformation
CT (computed tomography) images are acquired from living samples in a hospital, whose contour lines are extracted using the method of combining the improved snake algorithm with the ray method . In order to obtain the 3D face sample that is constructed by a single layer triangular mesh, the 3D surface is reconstructed using the Ganapathy algorithm to connect the neighbor contour lines . When the CT data are collected in a hospital, the postures of every body’s head are different, and unavoidably the reconstructed single layer 3D models would lie in different coordinate systems. For the convenience of defining the feature points, all the models are unified to a uniform coordinate system.
A face model is usually constructed by a triangular mesh which is composed of vertexes () and triangular facets (), so could be expressed as a linear representation, . is the th point of the , , and the coordinate of is expressed by . The center of the is computed by , and then the point set is transformed into the coordinate system whose origin is .
The point set is taken as a 3D random variable, and a positive definite covariance matrix could be obtained from the transformed . Three biggest eigenvalues , , of are calculated, and the eigenvector responding to the eigenvalues is , where is an orthogonal matrix, and , where is a unit matrix.
According to the related knowledge of algebra, , could be calculated out. Define as a linear combination of , , and is orthogonalised through the equation , and then the variance among sets could be expressed as is taken as the th principal component of variable , and then we construct the coordinate system shown in Figure 1, whose origin is and coordinate axes are , , .
2.2. Feature Point Definition
The feature points of a 3D face must have connotations and lie in a cognizable key position. In the same sample the feature points are distinct from their neighbor points on geometrical characteristics, while in different samples the features of the same feature points are similar. To meet the experimental requirements, a total of 39 feature points covering the front half face are defined according to the knowledge of anthropology, anatomy, and the feature points defined by MPEG4 expert group [7, 8]. The positions of the feature points are shown in Figure 2.
3. Feature Point Cluster Based on the Relative Angle Histograms
3.1. Relative Angle Histograms
After the face model is unified into a uniform coordinate system, the next step is to calculate the relative angle histograms of points in the face model. According to Feng et al. , any point of the face model expresses the angle between vector and , and after the angle is transformed its value range is .
The relative angles of the point in the face model describe a spatial relationship between every point, which possesses rotation, translation, and scaling invariance and better noise robustness. Any point of the model has relative angles and then has relative angle distributions. The relative distributions have statistical characteristics, and the distribution curves describe in detail the global characteristics of every point in the model.
Calculate the th relative angle of any point in the model. The value range is divided into 40 parts. Count the number of points in every region for every , and the statistical result is the relative angle histogram of point . Figure 3 shows the feature points’ relative angle distributions of the forehead center and right mouth corner as well as those which are superimposed by the different samples. It is obvious that the different feature points have different distributions, while the same feature points in the different samples have similar distributions. The similar feature points are selected to be clustered into one group using the cluster method  and to set up the cluster set of the feature points.
3.2. Feature Point Cluster
There are 39 feature points in a standard face model and their relative angle histograms of the feature points are denoted by . The unlocated face sample is denoted by . The number of the vertex is and the relative angle histogram of each feature point is . Let denote the cluster number of the th feature point, and the steps to determine the th feature point cluster set of the model are as follows.
Step 1. According to the prior knowledge, the representative points are selected as the initial cluster center of each group.
Step 2. Calculate the similarity between and points in the model and sort them in decreasing order.
Step 3. Select the maximum similarity points as the cluster set of the th feature point in the model .
Figure 4 shows the cluster point set results of the nasal tip and the forehead center in a model whose feature points are to be located. Experiments prove that the fewer cluster points would contain the optimum feature points in the conspicuous feature point place of a face, such as the nasal tip and the mouth corner, while, in the smoother place, such as the forehead center, more cluster points would be needed. The next task is to select precise feature points from the cluster point set.
(a) The nasal tip
(b) The forehead center
4. Multiscale Integral Features Extraction
4.1. Volume Integral Features
The volume integral features are defined as multiscale feature values, which could weaken the noise influence with bigger robustness than the differential values, and measure the concave-convex level of the model surface.
The volume integral invariant of a surface point is defined as a functional integration in the local sphere region of this point [10–12], whose math expression iswhere is a sphere, which takes as the sphere center and as the radius. As is shown in Figure 5. is a surface indicator function. When the point is in the region, ; while is out of the region, . The geometrical significance of is a part of the sphere volume which locates in the lateral of the surface.
Equation (2) is expanded to a polynomial and its limitation value is computed under the condition , so the functional relationship between the volume integral invariant and mean curvature is
From (3) we know that the value of the volume invariant is related to the radium of integral sphere region, so the longer the radius is, the bigger the value of volume invariant becomes. Whether there was noise in the interior of the integral region, the impact on the result of the volume invariant is small, so the volume invariant is taken as a feature which possesses robustness. In practice, the geometric feature of the surface on the point is defined as , which is the ratio of the volume invariant to the volume of sphere region. The equation isAccording to the calculated values of , we describe the geometric shape of the surface on the point as convex, approximate plane, and concave. When the scale is and , the shape of the surface on the point is convex; when , the shape of the surface on the point is an approximate plane; when , the shape of the surface on the point is concave.
4.2. Multiscale Feature Extraction
Concave-convex is a relative concept. Concave-convex level will transform with the scale change. As is shown in Figure 6, under a large scale the curve in a box is convex, while under a small scale it will be smooth when the local part is amplified.
Multiscale geometrical feature extraction  means that the feature extraction algorithm extracts the same feature adopting multiple threshold values. As is shown in Figure 7, the surface geometrical features of the point on the surface are extracted under 3 scales: , , and . The large scale can reduce the noise impact more, while the small scale can more precisely evaluate the integral invariant. The large scale should be combined with the small scale in the feature extraction process. Therefore, adopting the multiscale feature extraction method can not only precisely describe the local features but also reduce the noise influence.
4.3. Precise Localization of Feature Points
We calculate the integral features of cluster points and point for three different scales of 1, 2, and 3 times of the average side length in the triangular mesh. The distance square root is defined on the integral features. As is shown in (5), every point in is calculated, as well as the of the standard feature points under the three scales. The point , the minimum value of , is chosen as a feature point of the model . Consider
5. Experimental Comparison and Analysis
We design a feature point of 3D face localization system (FaceLS) according to the proposed feature point localization algorithm. Forty sets of 3D point distribution face model data are selected randomly from a monolayer face sample library, which belongs to the North West University, and are taken as the face samples of the feature points which will be located. The algorithm is tested and compared with the localization method of Feng et al. . All the programs are operated by Windows 7, Core i5 processors, 2.8 GHz, 2 G RAM, and Matlab 2010.
The evaluation method of the feature point localization is as follows: given a distance threshold value , is the accurate position of each feature point in the untested models, and the Euclidean distance between two points can be computed as follows:
The steps of our method are as follows.
Step 1. According to the prior knowledge, the representative points are selected as the initial cluster center of each group.
Step 2. Calculate the similarity between and points in model and sort them in decreasing order.
Step 3. Select maximum similarity points as cluster set of the th feature point in model .
Step 4. We calculate the integral features of cluster points and point for three different scales of 1, 2, and 3 times of the average side length in the triangular mesh. The distance square root is defined on the integral features. As is shown in (5), every point in is calculated, as well as of the standard feature points under the three scales. The point , the minimum value of , is chosen as a feature point of the model .
Step 5. Given a distance threshold value , judge whether the point is correct with (6). If , the feature point localization is considered to be correct; otherwise it has a certain error.
Figure 8 shows the results of two methods: one is based on our method and the other is the relative angle histogram localization method of Feng et al.  for the same feature point pending model (only 28 feature points are showed, and part of the feature points are coincident from the front view). Using the two localization methods, Figure 9 shows the accuracy rate of 39 feature points in a range of the given distance threshold value. The red dotted line indicates the relative angle histogram localization method of Feng et al. , and the blue solid line presents our method. The localization result of our method is more precise. Table 1 shows the mean accuracy rates of our method are higher than those of Feng et al. .
(a) Our method
(b) Relative angle histogram method
The feature point cluster set is determined firstly by the cluster algorithm and the relative angle histogram algorithm, and then the feature points are located accurately by the steady multiscale integral features, so the accuracy rate of localization is improved greatly. The localization method based on relative angle histograms has limitations, because it locates feature points to a smaller range through comparing the similarity of relative angle histograms and this method is suitable for conspicuous feature points, while the accuracy will be decreased for the inconspicuous feature points.
In this work we propose a method that combines the relative angle histograms with the multiscale constraints for localization of the feature points in a 3D face. The feature point cluster set is determined through the cluster algorithm and the relative angle histograms, and then feature points are located accurately by the multiscale integral features, which could avoid many wrong matches in the result caused by the precision and similarity of the local geometrical characteristic. Experimental results show that this method performs well and the accuracy rate of localization is improved. But the accuracy rate of localization is nonideal under the condition of feature points on smooth part of a face, so this should be the next research direction.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
This research is supported by the National Natural Science Foundation of China (Grant no. U1261114), the Key Technologies R&D Program of Henan Province (no. 142102210637), and the Xi’an University of Science and Technology Cultivation Foundation Project (2014032). The authors would like to thank the anonymous reviewers for their valuable comments and suggestions.
- Y. Wang, C.-S. Chua, and Y.-K. Ho, “Facial feature detection and face recognition from 2D and 3D images,” Pattern Recognition Letters, vol. 23, no. 10, pp. 1191–1202, 2002.
- C. Xu, T. Tan, Y. Wang, and L. Quan, “Combining local features for robust nose location in 3D facial data,” Pattern Recognition Letters, vol. 27, no. 13, pp. 1487–1494, 2006.
- J. Feng, H. H. S. Ip, L. Y. Lai, and A. Linney, “Robust point correspondence matching and similarity measuring for 3D models by relative angle-context distributions,” Image and Vision Computing, vol. 26, no. 6, pp. 761–775, 2008.
- X. L. Li and F. P. Da, “3D face recognition based on profile and rigid regions,” Journal of Image and Graphics, vol. 15, no. 2, pp. 266–273, 2010.
- H.-A. Li, J. Zhang, B. Kang, and Y. He, “A 3D surface reconstruction algorithm based on medical tomographic images,” Journal of Computational Information Systems, vol. 9, no. 19, pp. 7873–7880, 2013.
- J.-M. Oliva, M. Perrin, and S. Coquillart, “3D reconstruction of complex polyhedral shapes from contours using a simplified generalized voronoï diagram,” Computer Graphics Forum, vol. 15, no. 3, pp. C397–C408, 1996.
- C. Cao, Y. Weng, S. Zhou, Y. Tong, and K. Zhou, “Face warehouse: a 3D facial expression database for visual computing,” IEEE Transactions on Visualization and Computer Graphics, vol. 20, no. 3, pp. 413–425, 2014.
- H.-A. Li, J. Zhang, L. Zhang, and B. Kang, “A new control curve method for image deformation,” TELKOMNIKA: Telecommunication Computing Electronics and Control, vol. 12, no. 1, pp. 135–142, 2014.
- J. Guo, J. Feng, H. S. Ye, and G. Zhou, “Retrieval angle clustering histogram and clustering for 3D model retrieval,” Journal of Image and Graphics, vol. 15, no. 11, pp. 1644–1649, 2010.
- H. Pottmann, J. Wallner, Y.-L. Yang, Y.-K. Lai, and S.-M. Hu, “Principal curvatures from the integral invariant viewpoint,” Computer Aided Geometric Design, vol. 24, no. 8-9, pp. 428–442, 2007.
- Y. L. Yang, Y. K. Lai, S. M. Hu, and P. Helmut, “Robust principal curvatures on multiple scales,” in Proceedings of the 4th Eurographics Symposium on Geometry Processing (SGP '06), pp. 223–226, 2006.
- M. Liu, B. C. Vemuri, S.-I. Amari, and F. Nielsen, “Shape retrieval using hierarchical total Bregman soft clustering,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 34, no. 12, pp. 2407–2419, 2012.
- K. Xu, Y.-H. Ai, and X.-Y. Wu, “Application of multi-scale feature extraction to surface defect classification of hot-rolled steels,” International Journal of Minerals, Metallurgy and Materials, vol. 20, no. 1, pp. 37–41, 2013.
Copyright © 2015 Hong-an Li et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.