Abstract
Motivated by the key observation that children generally resemble their parents more than other persons with respect to facial appearance, distance metric (similarity) learning has been the dominant choice for state-of-the-art kinship verification via facial images in the wild. Most existing learning-based approaches to kinship verification, however, are focused on learning a genetic similarity measure in a batch learning manner, leading to less scalability for practical applications with ever-growing amount of data. To address this, we propose a new kinship verification approach by learning a sparse similarity measure in an online fashion. Experimental results on the kinship datasets show that our approach is highly competitive to the state-of-the-art alternatives in terms of verification accuracy, yet it is superior in terms of scalability for practical applications.
1. Introduction
Due to the fact that rich human characteristics, such as gender, identity, expression, and ethnicity, can be effectively extracted from facial images, a variety of face analysis problems, ranging from face recognition, facial expression recognition, and gender estimation to age estimation, have been extensively studied over the past decades [1]. Kinship verification using facial images, however, is a relatively new and challenging problem in biometrics [2], which is mainly motivated by the phenomenon that children generally look like their parents more than other people due to the kinship relation. Recent evidence in psychology has shown that facial appearance is a reliable cue to measure the genetic similarity between children and their parents [3–9]. In practice, there are some important potential applications of kinship verification via facial images, including missing children search and social media analysis [10, 11]. Figure 1 presents some image examples (with kinship relation) from the dataset KinFaceW [12].
Over the past few years, a few seminal works on kinship verification using facial images [2, 10, 11, 13–21] have been proposed. Roughly, existing methods for kinship verification are either feature-based [2, 13, 15–17, 19] or learning-based [10, 11, 14, 18, 20]. The former aims for extraction of the discriminative feature from facial images to characterize genetic property in human appearance. Learning-based approaches, however, are focused on learning a genetic measure via training data based on some discriminative learning technologies, such as subspace learning and distance metric learning, to improve the separability of facial images for kinship verification. Despite the promising results by existing learning-based approaches to kinship verification, they aim to learn a distance metric (or transform) in a batch learning way, leading to less efficiency or scalability even for medium-scale applications. Unlike the previous learning-based studies, we aim to learn a sparse similarity measure for kinship verification in an online manner. Our proposed approach not only is able to achieve highly competitive verification accuracy to state-of-the-art kinship verification method but also is superior in terms of scalability, making it more scalable for practical applications with ever-growing amount of data.
The rest of this paper is organized as follows. In Section 2 we elaborate our approach to kinship verification from facial images. Experiments and evaluations on performance are conducted in Section 3. Finally, we conclude and summarize the paper in Section 4.
2. Our Approach
In the past few years, many metric learning methods [23–27] have been proposed in the machine learning community, and they have been successfully applied in various vision tasks, including face recognition [10, 18, 28–34], activity recognition [35], human age estimation [36], gait recognition [37], and person reidentification [30, 38, 39]. The most existing metric learning methods aim at learning a Mahalanobis distance metric that measures the squared distance between a pair of samples and : where and is a positive semidefinite (PSD) matrix. Existing metric learning methods for kinship verification (e.g., [10]) are often proposed to minimize interclass variation and maximize intraclass variation simultaneously. In this work, we instead attempt to learn a genetic similarity measure explicitly, such that similarity score of a positive image pair is higher than those of the negative pairs. Let be the training set consisting of positive image pairs (with kinship relation), where denote the facial images of the th parent and child, respectively. The goal of this problem is to learn a similarity measure function that specifies the kinship constraints on any quadruple input , where , , . Formally, the kinship constraints for a quadruple input can be written by where , , , and 1 is a margin constant. There exist two triplet constraints for each quadruple input, on which similarity score of the positive pair is higher than that of image pair from different family. This can be schematically illustrated in Figure 2. We consider here a parametric similarity function with bilinear form, , where . Note that here is not necessarily symmetric or PSD. Given the similarity function and the constraints in (2), the problem of genetic similarity learning can be formulated as the following optimization task: where is a sparse regularization term that limits the model complexity and . Different from the widely used OASIS algorithm [22] designed to deal with the triplet constraints, we aim to directly deal with the quadruple constraints for kinship verification. The loss function for our problem is defined based on the hinge loss where . The optimization task of (3) corresponds to a batch learning formulation, and it is less efficient and scalable to practical applications even for medium-size training set. To tackle this issue, we instead aim to learn the similarity measure parameterized by in an online manner.
At each time , a quadruple instance is received, and the model is sequentially updated by solving the following optimization problem: where is a trace norm that encourages learning a low rank similarity matrix to limit the model complexity. The above online optimization can be solved by online gradient descent: where is the learning rate, is the identity matrix, and is the subgradient of with respect to , which can be computed by The proposed online learning algorithm for genetic similarity measure is outlined in Algorithm 1.
3. Experiment and Discussion
We conducted experiments on the kin datasets that are publicly available [10, 12] (KinFaceW-I and KinFace-II) to verify the effectiveness of our proposed approach. For both datasets, there are four kin relations: mother-son (M-S), mother-daughter (M-D), father-son (F-S), and father-daughter (F-D). All of the facial images have been aligned and cropped into 64 × 64 pixels.
We performed 5-fold cross validation on the two kin datasets, where each subset was equally divided into 5-fold so that each fold consists of nearly the same number of image pairs with kin relation. The parameters and are empirically set as 1.0 and 10−4, respectively. For the feature representations of the facial image, three descriptors, SIFT [40], learning-based (SPLE) [13], and local binary patterns (LBP) [41] are used. In our experiments, 256 bins are used for LBP, and we followed the parameter setting in [10] for SPLE and used 200 bins to encode a histogram. For the SIFT descriptors, 16 × 16 patches over a grid with space of 8 pixels are extracted from each face image.
We have compared our proposed approach with NRML [10] and OASIS [22]. The former is a state-of-the-art learning-based kinship verification approach, and the latter represents a popular algorithm for online similarity learning. Tables 1 and 2 list the mean verification rate of the three approaches with different features on the KinFaceW dataset. As shown in the two tables, our approach outperforms OASIS and is highly competitive to NRML in terms of verification rate. In addition, we can make the observation that SPLE is the best feature representation for our kinship verification problem. This is mainly attributed to the coding scheme of SPLE that is directly learned from the training examples, leading to better verification accuracy than two other hand-crafted feature descriptors.
It should be noticed that, compared to NRML, our approach is superior in terms of scalability for practical applications due to the online learning nature in model training. We believe this is very important for the kinship verification on the large-scale dataset with ever-growing amount of data.
4. Conclusion
We have presented a new scheme for kinship verification via facial images in wild conditions by explicitly learning a sparse genetic similarity measure in an online manner. Experimental results on kinship datasets demonstrated that our approach is not only highly competitive to the state-of-the-art algorithm in terms of the verification accuracy but also superior in terms of scalability for practical applications with ever-growing amount of data. For future work, we are interested in investigating the fusion of multiple feature representations within the online learning framework to further improve the overall kinship verification performance. Also, it is helpful for face (kin) verification to work out the similarity of facial components (e.g., eyes or mouths). In this setting, robust face landmarks estimation is often needed to parse the face into facial components. How to exploit it in our proposed approach to work on partially occluded faces or faces viewed from the side appears to be another interesting direction of future work.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
This work is supported by the Natural Science Foundation of China under Grant nos. 61373090 and 11178017 and the Beijing Natural Science Foundation of China under Grant no. 4132014.