Abstract
To solve the matching problem of the elements in different data collections, an improved coupled metric learning approach is proposed. First, we improved the supervised locality preserving projection algorithm and added the within-class and between-class information of the improved algorithm to coupled metric learning, so a novel coupled metric learning method is proposed. Furthermore, we extended this algorithm to nonlinear space, and the kernel coupled metric learning method based on supervised locality preserving projection is proposed. In kernel coupled metric learning approach, two elements of different collections are mapped to the unified high dimensional feature space by kernel function, and then generalized metric learning is performed in this space. Experiments based on Yale and CAS-PEAL-R1 face databases demonstrate that the proposed kernel coupled approach performs better in low-resolution and fuzzy face recognition and can reduce the computing time; it is an effective metric method.
1. Introduction
The metric is a function which gives the scalar distance between two patterns. Distance metric is an important basis for similarity measure between samples, and it is one of the core issues in pattern recognition. The aim of distance metric learning is to find a distance metric matrix; its essence is to obtain another representation method with better class separability by linear or nonlinear transformation.
In recent years, some researches about distance metric have been done by researchers [1–7]. They learn a distance metric by introducing sample similarity constraint or category information; the distance metric is used to improve the data clustering or classification. These researches can be concluded to two categories: linear distance metric learning and nonlinear distance metric learning. The linear distance metric learning is equivalent to learning a linear transformation in sample space, including a variety of common linear dimensionality reduction methods, such as principal component analysis [8], linear discriminant analysis [9], and independent component analysis method [10]. The nonlinear distance metric learning is equivalent to learning a nonlinear transformation in sample space; the locally linear embedding [11], isometric mapping [12], and Laplace mapping [13] are the traditional nonlinear methods. Recently, some new nonlinear distance metric methods have been proposed. Baghshah and Shouraki [14] proposed the nonlinear metric learning method based on pair-wise similarity and dissimilarity constraints and the geometrical structure of data. Babagholami-Mohamadabadi et al. [15] proposed the probabilistic nonlinear distance metric learning. The deep nonlinear metric learning method [16] based on neural networks is a new nonlinear metric learning method. In addition, there are some more flexible distance metric learning algorithms, which are based on kernel matrix [7, 17, 18].
These traditional distance metric learning methods are defined on the set of single attribute. If the elements belong to different sets with different attribute, these distance measurement methods are incapable for the distance metric. For example, for two images with different resolution, which can be considered to belong to different sets, obviously, the traditional distance metric method will not be able to directly calculate the distance. The general approach is normalized operation before recognition by using interpolation algorithm or sampling algorithm. But the interpolation inevitably introduced false information, and sampling may miss some useful information, so it is difficult to get high recognition rate.
Aiming at the shortage of traditional distance metric, Li et al. proposed the coupled metric learning (CML) [19–21]. The goal of coupled metric learning is to find a coupled distance function to meet the specific requirement in given task. The essential idea is that, firstly, the data in different collections are projected to the unified coupled space, and the data should be as close as possible in this new space. Then, the generalized metric learning is performed in this unified coupled space. Obliviously, this new metric method will have broader application scope and better recognition effect. However, the proposed coupled metric learning methods are based on linear transformation, which can be called linear coupled metric learning (LCML). These methods have two shortages in dealing with practical problems. First, the practical problems usually are nonlinear, and the linear transformation does not represent the features effectively. Secondly, the image needs to be converted into one-dimensional vector in the LCML algorithm; it is easy to cause the increasing of dimensions of autocorrelation matrix.
Based on the idea of coupled metric learning, we improved the supervised locality preserving projection algorithm and added supervised locality preserving information to coupled metric learning. So the improved coupled metric learning approach based on supervised locality preserving projection (SLPP-CML) was proposed. Introducing kernel technology into the coupled metric learning, we proposed the kernel coupled metric learning approach based on supervised locality preserving projection (SLPP-KCML). The SLPP-KCML realized the extension from linear coupled metric to nonlinear coupled metric. To verify the effectiveness of the proposed method, the experiments based on two face databases were performed. The experimental results show that a higher recognition rate can be achieved in the SLPP-KCML algorithm, and the operation time is reduced greatly.
2. Related Works
The traditional distance metric learning algorithm is to learn a distance function between the data points expressed as follows:
Distance metric learning aims to find a distance metric matrix ; it is required that is a real symmetric and positive semidefinite matrix; namely, , where is a transformation matrix
Obviously, the distance metric learning is realized by learning a transformation matrix , so the process of distance metric learning is equivalent to the process of obtaining other representation forms with better separability through linear or nonlinear transformation of the samples.
If , represent two different collections, respectively, the function is the distance metric between data and data . If , the traditional method does not work for distance metric. Even if , because the data and data , which belong to different attribute collections, the distance metric has no physical meaning.
The coupled distance metric is a distance function for the data elements of different kinds of collections. The elements of collections and are mapped from the original space to a common coupled space by using the mapping functions and . Then, the distance metric is performed in the coupled space. The measured distance can be represented mathematically as where matrix is a real symmetric and positive semidefinite matrix. Letting , we can get
The goal of coupled metric learning can be achieved by minimizing the distance function; the objective function is as follows: where is a correlation matrix of elements in collections and . According to different supervised information, we can obtain different matrix , so as to realize the different coupled metric learning.
3. The Coupled Metric Learning Based on Supervised Locality Preserving Projection (SLPP-CML)
The coupled distance metric learning must be used under the constraints of supervised information. In this paper, we improved the supervised locality preserving projection (SLPP) algorithm [22]. Based on the improved SLPP algorithm, we proposed the coupled metric learning method based on supervised locality preserving projection.
In order to better illustrate the coupled metric learning algorithm based on supervised locality preserving projection, we provide a theorem about the matrix norm.
Theorem 1. Letting , then the Frobenius norm has the following properties:(1), where is the eigenvalue of the matrix ;(2), represents the trace operation.
The coupled metric learning based on supervised locality preserving projection includes the following steps.
Step 1 (building the neighborhood relation in the same collection). We use the nearest neighbor method. First, building within-class adjacency graph in the same collection, if the data point () is one of the within-class nearest neighbors of data point (), we connect these two data points; and then, building between-class adjacency graph in same collection; if the data point () is one of the between-class nearest neighbors of data point (), these two data points are connected.
Step 2 (building the connected relation between two collections). If the data points and in two different collections belong to the same class, then these two points are connected, otherwise not connected.
Step 3 (constructing the relation matrix in the same collection). According to the neighborhood relations, the relation matrixes (similarity matrixes) of within-class and between-class are constructed, respectively.
Within-class similarity matrix is corresponding to within-class adjacency graph and the within-class similarity value is . The definition is as follows:
Between-class similarity matrix is corresponding to between-class adjacency graph and the between-class similarity value is . It can be defined as follows: where parameter is the average distance between all sample points.
Step 4 (constructing the relation matrix between two collections). The similarity value is as follows:
Step 5 (calculating the final similarity matrix between two collections). As shown in Figure 1, the similarity relations between element and elements of collection include the following several situations.(a)The similarity between and : these two data points in different collections belong to the same class and they are connected to each other, so the similarity of which is .(b)The similarity between and : these two data points belong to different class, but the relationship between and is the between-class neighborhood relation in same collection, and the similarity is the maximum similarity value, so similarity between and is .(c)The similarity between and : these two data points belong to different class and the does not have between-class neighborhood relation with any element of collection of class 1. But there is a between-class neighborhood relation of same collection between and , and within-class neighborhood relation between and . So the similarity between and is defined as the product of between-class similarity and within-class similarity , which is the maximum similarity between and ; that is, .(d)The similarity between and : these two data points belong to different class; there are not any between-class neighborhood relations between the elements of class 1 and class 3 in collection ; namely, .
Step 6 (constructing the optimal objective function). Cosider the following: where the functions and are considered to be linear; that is, , . The optimal objective function can be rewritten as follow:
Letting , , we can get Therefore, our method aims to learn two linear transformations and .
According to Theorem 1, (12) is an alternate matrix expression of (11): where represent computing the trace of matrix , and are diagonal matrixes, and their diagonal elements are the row or column sums of similarity matrix , respectively.
Assuming that , , , (12) can be rewritten as follow:
To make the equation have a unique solution, and as constraints are added, where is a vector with dimensions of , and are the numbers of samples in collections and . The solution to make (13) minimized is obtained by generalized eigendecomposition of and taking eigenvectors corresponding to the second to th smallest eigenvalues . Assuming that , its dimension is , and are the dimensions of samples in collections and , so the transformation matrix corresponds to the 1st to th rows of and corresponds to the th to th rows of .
Step 7. Bringing the matrix and to (11), the distance metric of the elements belonging to different collections can be realized.
4. The Kernel Coupled Metric Learning Based on Supervised Locality Preserving Projection (SLPP-KCML)
In practical dimension reduction and measurement process, the linear model is not well to represent the features, and it is difficult to map two complex collections to the same space using the linear transformation. So combining the kernel method, we extend the SLPP-CML algorithm; a nonlinear coupled metric learning methods based on the supervised locality preserving projection is proposed.
Assuming that the mapping functions and are nonlinear functions, namely, , , using the nonlinear mapping , , , the sample data can be mapped to the high dimensional Hilbert space. The criterion can be defined by
An alternate matrix expression is as follow: where represent computing the trace of matrix, and are diagonal matrixes, and their diagonal elements are the row or column sums of similarity matrix , respectively.
Letting , , we can get
The kernel function , ; the kernel matrixes and are real symmetric matrices. Equation (17) is an alternative expression of (16):
Obviously, the coupled metrics learning in kernel space is a process of calculating the transformation matrix and .
Assuming that , , , similar to the SLPP-CML algorithm, solving the optimal solution can be transformed into the generalized eigenvalue problem. The generalized characteristic equation is , and , ; is the eigenvector corresponding to eigenvaule . The eigenvectors corresponding to the minimum to the th smallest eigenvalues construct the feature matrix ; the size of matrix is , where and are the numbers of training samples of collections and . Finally, we can get the feature matrix corresponding to the data matrix and the feature matrix corresponding to the data matrix .
In addition, the samples mapped to the high dimensional space need centering processing. In the linear coupled metric learning, the centering can be realized by abandoning the eigenvector corresponding to eigenvalue of “zero.” However, the centering of nonlinear coupled metric learning in kernel space can be realized by centering the kernel matrix and where is the dimension of kernel matrix . is a matrix with size of and all elements are one.
5. Experiment and Analysis
5.1. Introduction of the Face Database
The proposed coupled metric learning approach is used for face recognition. It is tested on Yale face database [23] and CAS-PEAL-R1 face database [24]. The Yale face database contains 165 pictures of 15 people with the size of and 256 gray levels. These images were taken in different expression and illumination conditions. In experiment, we used the former 6 images per person as training samples, a total of 90, and the other images were used as test samples.
The CAS-PEAL-R1 face database contains 30863 face images, which was divided into two parts: the frontal face image subset; the nonfrontal face image subset. In the experiment, we used the accessory data set of the frontal face image subsets (CAS-PEAL-R1-FRONTAL-Accessory). The face images per person in CAS-PEAL-R1-FRONTAL-Accessory contain 6 different appendages; there are 3 images with different glasses and 3 images with different hats. We selected 300 images of 50 people with the size of and 256 gray levels in the experiment; the odd-numbered images were used as training samples and even-numbered images were used as test samples, respectively. Some training images are shown in Figure 2 and some test images are shown in Figure 3.
(a) Some images in Yale face database
(b) Some images in CAS-PEAL-R1-FRONTAL-Accessory
(a) Some images in Yale face database
(b) Some images in CAS-PEAL-R1-FRONTAL-Accessory
5.2. The Low-Resolution Face Recognition
Due to the differences between the different resolution cameras and the uncertainty of distance between camera and face, the resolution of face image that we collected is not uniform. Obviously, traditional measurement method can not be used to calculate the distance between two images with different resolution. The general handling method is interpolation operation, but the interpolation operation is easy to introduce false information. With the increase in false information, the distortion degree increases, as shown in Figure 4. Aiming at the problem of recognition rate declining because of image distortion, the researchers realized the low-resolution image compensation by increasing the image restoration preprocessing. But the image restoration algorithm is more complex, and the quality of image restoration has great impact on final recognition results.
(a)
(b)
However, the proposed coupled metric learning method can directly realize the feature extraction and measurement of different resolution images. This method not only saves computing time, but also avoids the negative impact of image restoration on recognition performance. To better illustrate the experimental processes, Figure 5 gives the flow of the degraded face recognition.
In experiment, the training samples include clear face and degraded face images. The size of original normal training face image is adjusted to pixel, and these adjusted faces are used as clear face images. However, there are not original low-resolution face images in public face database, so we obtained the low-resolution training face image through blurring and sampling original normal training face image, and the size of low-resolution face image is .
The test samples are the low-resolution face images, which were generated through blurring and sampling original normal test face image introduced in Section 5.1.
5.2.1. Experiment 1: The Low-Resolution Face Recognition Based on SLPP-CML
Through the theoretical analysis, the SLPP-CML algorithm has two influence factors: the number of neighbors of supervised locality preserving projection; the reserved dimensions of the feature. Therefore, the recognition results based on different parameters should be discussed and analyzed. Figure 6 shows the change of recognition rate with the change of feature dimensions, when the number of neighbors takes different values.
(a) The experiment result in Yale face database
(b) The experiment result in CAS-PEAL-R1 face database
These recognition rate curves are in two different face databases. These curves have a general change law, with the increase in feature dimensions; the recognition rate kept a decreasing trend after increasing, and the best recognition results can be achieved only in the optimal feature dimensions.
In Yale face database, the recognition rate kept a higher trend when feature dimensions remain 10–20. The optimal recognition rate is 86.67% when feature dimension is 10 and the number of neighbors is 5. In CAS-PEAL-R1 face database, the recognition rate can reach the maximum value 86.67%, when feature dimension is 40 and the number of neighbors is 2.
The experimental data show that the number of training samples of each class is 6 in Yale face database and the recognition effect is optimal when the number of neighbors is 5. In CAS-PEAL-R1 face database, we can obtain optimal recognition rate when the number of training samples of each class is 3 and the number of neighbors is 2. Obviously, the number of neighbors is ; is the number of training samples of each class.
In addition, in order to illustrate the effectiveness of SLPP-CML method, the comparative experiments were carried out. The experiment results are shown in Table 1.
The experimental data illustrated that the recognition results of feature extraction after restoration are not satisfactory. The coupled metric learning in [19] can not overcome the influences of within-class multiple modes, so the identification effect is not good. The coupled metric learning in [21] is conducive to resolving within-class multiple modes; the recognition effects have been greatly improved, but it does not fully consider the between-class relationships of training samples. The proposed SLPP-CML takes advantage of the supervisory of category information, while the within-class and between-class relationship information of training samples have been considered into the metric learning, so we can get better recognition results.
5.2.2. Experiment 2: The Low-Resolution Face Recognition Based on SLPP-KCML
The SLPP-KCML algorithm is a nonlinear coupled metric learning algorithm. Through the analysis, there are three factors which affect this algorithm: the number of neighbors of supervised locality preserving projection; the reserved dimensions of the feature; the kernel function. Based on the experiment result, the number of the nearest neighbors is the same as that of SLPP-CML. For the kernel function, we choose the Gauss function ; the value of adjustable factor affects the function performance. So, in this paper, the experiments were carried out according to the different adjustable factors and the change of feature dimensions on recognition rate; the experimental results are shown in Figure 7.
(a) The experiment result in Yale face database
(b) The experiment result in Yale CAS-PEAL-R1 face database
The curves indicated that, in Yale face database, the optimal recognition rate is 89.33% when the value of is 0.5 and the feature dimension is 20; compared with the SLPP-CML algorithm, the recognition rate increased by 2.66%. In CAS-PEAL-R1 face database, when and the feature dimensions , recognition rate is 91.33%; compared with the SLPP-CML algorithm, the recognition rate increased by 4.66%. Obviously, the nonlinear coupled metric learning method can effectively extract the classification information of face image and obtain a high recognition rate.
Considering the training time, the SLPP-CML algorithm requires calculating the eigenvalue and eigenvector of the image covariance matrix. The resolution of clear face image is pixel and low-resolution face image is pixel. So the dimensions of image covariance matrix are and the average training time is about 553.905 seconds.
However, the matrix of SLPP-KCML is related to the number of classes and the number of training samples of each class, so the dimension of the covariance matrix of Yale face database is , the size of covariance matrix of CAS-PEAL-R1 face database is , and the average training time is about 6.687 seconds. Obviously, the training speed of the SLPP-KCML algorithm is faster than SLPP-CML algorithm, and the recognition time of these two algorithms is about 0.0225 seconds. Based on the above analysis, the efficiency of SLPP-KCML algorithm is better than SLPP-CML algorithm.
5.3. The Fuzzy Face Recognition
Besides the low-resolution face image, the blurring image usually makes the performance of face recognition system decrease. The fuzzy face image is shown in Figure 8. Obviously, it is difficult to identify the fuzzy face image; a part of face details can be restored by deblurring algorithm, but it still cannot provide enough information in identification.
(a)
(b)
Similar to Section 5.2, we carried out the recognition experiments of SLPP-CML algorithm based on different face databases, and the comparative experiments with other methods were made. In the experiment, the clear images with size of are the same as those used in Section 5.1, and the fuzzy face images were generated by convolution of corresponding clear face image. The training samples are composed by clear training face images and generated fuzzy face images. The test samples are the fuzzy face images by convolution of the clear test face images and the size is pixels. The number of training and test samples has been introduced in Section 5.1. The experiment results are shown in Table 2.
The experimental data is the best recognition rate of each algorithm; the number of neighbors of SLPP-CML and SLPP-KCML algorithm is , where is the number of training samples of each class. The optimal value of adjustable factor of Gaussian kernel function in SLPP-KCML algorithm is 0.7. Table 3 gives the feature dimensions of training samples in Yale and CAS-PEAL-R1 face database of SLPP-CML and SLPP-KCML algorithm.
6. Conclusions
Aiming at the problem that the traditional metric methods can not calculate the distance of the elements in different data sets, we proposed the coupled metric learning method based on supervised locality preserving projection. First, the elements of different sets are mapped to the coupled space combined with the within-class and between-class information, and then the metric matrix learning is performed. Furthermore, we extended this algorithm to nonlinear space, and the kernel coupled metric learning method based on supervised locality preserving projection is proposed. In kernel coupled metric learning approach, two elements of different collections are mapped to the unified high dimensional feature space by kernel function, and then the traditional metric learning is performed in this space. In order to verify the effectiveness of the proposed algorithm, we have done a lot of experiments on two face databases. This algorithm can effectively extract the face nonlinear features, and the operation is simple. Low-resolution and fuzzy face recognition experiments show that the proposed method can obtain a higher recognition rate and has a high computational efficiency.
Appendix
Proof of Theorem 1
Assuming , the Frobenius norm of matrix is . So Obviously, .
Therefore, .
According to the properties of the trace of a matrix, if , the eigenvalue of matrix is ; then; .
Obviously, ; then, , ; then, , where is the eigenvalue of matrix .
Based on the above analysis, we can get that .
Assuming , then and .
According to the definition of the trace, we get that . And then, . The above equation shows that ; then, .
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgment
This research is supported by National Research Foundation for the Doctoral Program of Higher Education of China (20102304110004).