Research Article | Open Access
Learning a Genetic Measure for Kinship Verification Using Facial Images
Motivated by the key observation that children generally resemble their parents more than other persons with respect to facial appearance, distance metric (similarity) learning has been the dominant choice for state-of-the-art kinship verification via facial images in the wild. Most existing learning-based approaches to kinship verification, however, are focused on learning a genetic similarity measure in a batch learning manner, leading to less scalability for practical applications with ever-growing amount of data. To address this, we propose a new kinship verification approach by learning a sparse similarity measure in an online fashion. Experimental results on the kinship datasets show that our approach is highly competitive to the state-of-the-art alternatives in terms of verification accuracy, yet it is superior in terms of scalability for practical applications.
Due to the fact that rich human characteristics, such as gender, identity, expression, and ethnicity, can be effectively extracted from facial images, a variety of face analysis problems, ranging from face recognition, facial expression recognition, and gender estimation to age estimation, have been extensively studied over the past decades . Kinship verification using facial images, however, is a relatively new and challenging problem in biometrics , which is mainly motivated by the phenomenon that children generally look like their parents more than other people due to the kinship relation. Recent evidence in psychology has shown that facial appearance is a reliable cue to measure the genetic similarity between children and their parents [3–9]. In practice, there are some important potential applications of kinship verification via facial images, including missing children search and social media analysis [10, 11]. Figure 1 presents some image examples (with kinship relation) from the dataset KinFaceW .
Over the past few years, a few seminal works on kinship verification using facial images [2, 10, 11, 13–21] have been proposed. Roughly, existing methods for kinship verification are either feature-based [2, 13, 15–17, 19] or learning-based [10, 11, 14, 18, 20]. The former aims for extraction of the discriminative feature from facial images to characterize genetic property in human appearance. Learning-based approaches, however, are focused on learning a genetic measure via training data based on some discriminative learning technologies, such as subspace learning and distance metric learning, to improve the separability of facial images for kinship verification. Despite the promising results by existing learning-based approaches to kinship verification, they aim to learn a distance metric (or transform) in a batch learning way, leading to less efficiency or scalability even for medium-scale applications. Unlike the previous learning-based studies, we aim to learn a sparse similarity measure for kinship verification in an online manner. Our proposed approach not only is able to achieve highly competitive verification accuracy to state-of-the-art kinship verification method but also is superior in terms of scalability, making it more scalable for practical applications with ever-growing amount of data.
The rest of this paper is organized as follows. In Section 2 we elaborate our approach to kinship verification from facial images. Experiments and evaluations on performance are conducted in Section 3. Finally, we conclude and summarize the paper in Section 4.
2. Our Approach
In the past few years, many metric learning methods [23–27] have been proposed in the machine learning community, and they have been successfully applied in various vision tasks, including face recognition [10, 18, 28–34], activity recognition , human age estimation , gait recognition , and person reidentification [30, 38, 39]. The most existing metric learning methods aim at learning a Mahalanobis distance metric that measures the squared distance between a pair of samples and : where and is a positive semidefinite (PSD) matrix. Existing metric learning methods for kinship verification (e.g., ) are often proposed to minimize interclass variation and maximize intraclass variation simultaneously. In this work, we instead attempt to learn a genetic similarity measure explicitly, such that similarity score of a positive image pair is higher than those of the negative pairs. Let be the training set consisting of positive image pairs (with kinship relation), where denote the facial images of the th parent and child, respectively. The goal of this problem is to learn a similarity measure function that specifies the kinship constraints on any quadruple input , where , , . Formally, the kinship constraints for a quadruple input can be written by where , , , and 1 is a margin constant. There exist two triplet constraints for each quadruple input, on which similarity score of the positive pair is higher than that of image pair from different family. This can be schematically illustrated in Figure 2. We consider here a parametric similarity function with bilinear form, , where . Note that here is not necessarily symmetric or PSD. Given the similarity function and the constraints in (2), the problem of genetic similarity learning can be formulated as the following optimization task: where is a sparse regularization term that limits the model complexity and . Different from the widely used OASIS algorithm  designed to deal with the triplet constraints, we aim to directly deal with the quadruple constraints for kinship verification. The loss function for our problem is defined based on the hinge loss where . The optimization task of (3) corresponds to a batch learning formulation, and it is less efficient and scalable to practical applications even for medium-size training set. To tackle this issue, we instead aim to learn the similarity measure parameterized by in an online manner.
At each time , a quadruple instance is received, and the model is sequentially updated by solving the following optimization problem: where is a trace norm that encourages learning a low rank similarity matrix to limit the model complexity. The above online optimization can be solved by online gradient descent: where is the learning rate, is the identity matrix, and is the subgradient of with respect to , which can be computed by The proposed online learning algorithm for genetic similarity measure is outlined in Algorithm 1.
3. Experiment and Discussion
We conducted experiments on the kin datasets that are publicly available [10, 12] (KinFaceW-I and KinFace-II) to verify the effectiveness of our proposed approach. For both datasets, there are four kin relations: mother-son (M-S), mother-daughter (M-D), father-son (F-S), and father-daughter (F-D). All of the facial images have been aligned and cropped into 64 × 64 pixels.
We performed 5-fold cross validation on the two kin datasets, where each subset was equally divided into 5-fold so that each fold consists of nearly the same number of image pairs with kin relation. The parameters and are empirically set as 1.0 and 10−4, respectively. For the feature representations of the facial image, three descriptors, SIFT , learning-based (SPLE) , and local binary patterns (LBP)  are used. In our experiments, 256 bins are used for LBP, and we followed the parameter setting in  for SPLE and used 200 bins to encode a histogram. For the SIFT descriptors, 16 × 16 patches over a grid with space of 8 pixels are extracted from each face image.
We have compared our proposed approach with NRML  and OASIS . The former is a state-of-the-art learning-based kinship verification approach, and the latter represents a popular algorithm for online similarity learning. Tables 1 and 2 list the mean verification rate of the three approaches with different features on the KinFaceW dataset. As shown in the two tables, our approach outperforms OASIS and is highly competitive to NRML in terms of verification rate. In addition, we can make the observation that SPLE is the best feature representation for our kinship verification problem. This is mainly attributed to the coding scheme of SPLE that is directly learned from the training examples, leading to better verification accuracy than two other hand-crafted feature descriptors.
It should be noticed that, compared to NRML, our approach is superior in terms of scalability for practical applications due to the online learning nature in model training. We believe this is very important for the kinship verification on the large-scale dataset with ever-growing amount of data.
We have presented a new scheme for kinship verification via facial images in wild conditions by explicitly learning a sparse genetic similarity measure in an online manner. Experimental results on kinship datasets demonstrated that our approach is not only highly competitive to the state-of-the-art algorithm in terms of the verification accuracy but also superior in terms of scalability for practical applications with ever-growing amount of data. For future work, we are interested in investigating the fusion of multiple feature representations within the online learning framework to further improve the overall kinship verification performance. Also, it is helpful for face (kin) verification to work out the similarity of facial components (e.g., eyes or mouths). In this setting, robust face landmarks estimation is often needed to parse the face into facial components. How to exploit it in our proposed approach to work on partially occluded faces or faces viewed from the side appears to be another interesting direction of future work.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
This work is supported by the Natural Science Foundation of China under Grant nos. 61373090 and 11178017 and the Beijing Natural Science Foundation of China under Grant no. 4132014.
- W. Zhao, R. Chellappa, P. J. Phillips, and A. Rosenfeld, “Face recognition: a literature survey,” ACM Computing Surveys, vol. 35, no. 4, pp. 399–458, 2003.
- R. Fang, K. Tang, N. Snavely, and T. Chen, “Towards computational models of kinship verification,” in Proceedings of 17th IEEE International Conference on Image Processing (ICIP '10), pp. 1577–1580, IEEE, Hong Kong, China, September 2010.
- L. M. DeBruine, F. G. Smith, B. C. Jones, S. Craig Roberts, M. Petrie, and T. D. Spector, “Kin recognition signals in adult faces,” Vision Research, vol. 49, no. 1, pp. 38–43, 2009.
- A. Alvergne, R. Oda, C. Faurie, A. Matsumoto-Oda, V. Durand, and M. Raymond, “Cross-cultural perceptions of facial resemblance between kin,” Journal of Vision, vol. 9, no. 6, article 23, 2009.
- M. F. dal Martello and L. T. Maloney, “Where are kin recognition signals in the human face?” Journal of Vision, vol. 6, no. 12, pp. 1356–1366, 2006.
- M. F. dal Martello and L. T. Maloney, “Lateralization of kin recognition signals in the human face,” Journal of Vision, vol. 10, no. 8, article 9, pp. 1–10, 2010.
- L. M. DeBruine, F. G. Smith, B. C. Jones, S. Craig Roberts, M. Petrie, and T. D. Spector, “Kin recognition signals in adult faces,” Vision Research, vol. 49, no. 1, pp. 38–43, 2009.
- G. Kaminski, S. Dridi, C. Graff, and E. Gentaz, “Human ability to detect kinship in strangers' faces: effects of the degree of relatedness,” Proceedings of the Royal Society B: Biological Sciences, vol. 276, no. 1670, pp. 3193–3200, 2009.
- G. Kaminski, F. Ravary, C. Graff, and E. Gentaz, “Firstborns' disadvantage in kinship detection,” Psychological Science, vol. 21, no. 12, pp. 1746–1750, 2010.
- J. Lu, X. Zhou, Y.-P. Tan, Y. Shang, and J. Zhou, “Neighborhood repulsed metric learning for kin-ship verification,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 36, no. 2, pp. 331–345, 2014.
- S. Xia, M. Shao, and Y. Fu, “Kinship verification through transfer learning,” in Proceedings of the 22nd International Joint Conference on Artificial Intelligence (IJCAI '11), pp. 2539–2544, July 2011.
- KinFaceW Dataset, http://www.xeehoo.com/zxz.html.
- X. Zhou, J. Hu, J. Lu, Y. Shang, and Y. Guan, “Kinship verification from facial images under uncontrolled conditions,” in Proceedings of the 19th ACM International Conference on Multimedia (MM '11), pp. 953–956, December 2011.
- M. Shao, S. Xia, and Y. Fu, “Genealogical face recognition based on UB KinFace database,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW '11), pp. 60–65, Colorado Springs, Colo, USA, June 2011.
- G. Guo and X. Wang, “Kinship measurement on salient facial features,” IEEE Transactions on Instrumentation and Measurement, vol. 61, no. 8, pp. 2322–2325, 2012.
- X. Zhou, J. Lu, J. Hu, and Y. Shang, “Gabor-based gradient orientation pyramid for kinship verification under uncontrolled environments,” in Proceedings of the ACM International Conference on Multimedia, pp. 2322–2325, November 2012.
- H. Dibeklioglu, A. A. Salah, and T. Gevers, “Like father, like son: facial expression dynamics for kinship verification,” in Proceedings of the 14th IEEE International Conference on Computer Vision (ICCV '13), pp. 1497–1504, December 2013.
- J. Lu, J. Hu, X. Zhou, Y. Shang, Y.-P. Tan, and G. Wang, “Neighborhood repulsed metric learning for kinship verification,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR '12), pp. 2594–2601, June 2012.
- N. Kohli, R. Singh, and M. Vatsa, “Self-similarity representation of Weber faces for kinship classification,” in Proceedings of the IEEE 5th International Conference on Biometrics: Theory, Applications and Systems (BTAS '12), pp. 245–250, September 2012.
- S. Xia, M. Shao, J. Luo, and Y. Fu, “Understanding kin relationships in a photo,” IEEE Transactions on Multimedia, vol. 14, no. 4, pp. 1046–1056, 2012.
- H. Yan, J. Lu, W. Deng, and X. Zhou, “Discriminative multimetriclearning for kinship verification,” IEEE Transactions on Information Forensics and Security, vol. 9, no. 7, pp. 1169–1178, 2014.
- G. Chechik, V. Sharma, U. Shalit, and S. Bengio, “Large scale online learning of image similarity through ranking,” Journal of Machine Learning Research (JMLR), vol. 11, pp. 1109–1135, 2010.
- E. P. Xing, A. Y. Ng, M. I. Jordan, and S. Russell, “Distance metric learning, with application to clustering with side-information,” in Proceedings of the Advances in Neural Information Processing Systems, pp. 505–512, December 2002.
- J. V. Davis, B. Kulis, P. Jain, S. Sra, and I. S. Dhillon, “Information-theoretic metric learning,” in Proceedings of the 24th International Conference on Machine Learning (ICML '07), pp. 209–216, June 2007.
- J. Goldberger, S. Roweis, G. Hinton, and R. Salakhutdinov, “Neighbourhood components analysis,” in Proceedings of the 18th Annual Conference on Neural Information Processing Systems (NIPS '04), pp. 2539–2544, December 2004.
- S. Parameswaran and K. Q. Weinberger, “Large margin multi-task metric learning,” in Proceedings of the 24th Annual Conference on Neural Information Processing Systems (NIPS '10), pp. 1867–1875, December 2010.
- K. Q. Weinberger and L. K. Saul, “Distance metric learning for large margin nearest neighbor classification,” Journal of Machine Learning Research, vol. 10, pp. 207–244, 2009.
- R. G. Cinbis, J. Verbeek, and C. Schmid, “Unsupervised metric learning for face identification in TV video,” in Proceedings of the IEEE International Conference on Computer Vision (ICCV '11), pp. 1559–1566, Barcelona, Spain, November 2011.
- M. Guillaumin, J. Verbeek, and C. Schmid, “Is that you? Metric learning approaches for face identification,” in Proceedings of the 12th International Conference on Computer Vision (ICCV '09), pp. 498–505, Kyoto, Japan, October 2009.
- M. Kostinger, M. Hirzer, P. Wohlhart, P. M. Roth, and H. Bischof, “Large scale metric learning from equivalence constraints,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR '12), pp. 2288–2295, IEEE, Providence, RI, USA, June 2012.
- J. Lu, Y.-P. Tan, and G. Wang, “Discriminative multimanifold analysis for face recognition from a single training sample per person,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 1, pp. 39–51, 2013.
- H. Nguyen and L. Bai, “Cosine similarity metric learning for face verification,” in Computer Vision—ACCV 2010: 10th Asian Conference on Computer Vision, Queenstown, New Zealand, November 8–12, 2010, Revised Selected Papers, Part II, vol. 6493 of Lecture Notes in Computer Science, pp. 709–720, Springer, Berlin, Germany, 2011.
- W. Deng, J. Hu, J. Guo, W. Cai, and D. Feng, “Robust, accurate and efficient face recognition from a single training image: a uniform pursuit approach,” Pattern Recognition, vol. 43, no. 5, pp. 1748–1762, 2010.
- W. Deng, J. Hu, J. Lu, and J. Guo, “Transform-invariant PCA: a unified approach to fully automatic face alignment, representation, and recognition,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 36, no. 6, pp. 1275–1284, 2014.
- D. Tran and A. Sorokin, “Human activity recognition with metric learning,” in Proceedings of the 10th European Conference on Computer Vision, vol. 5302 of Lecture Notes in Computer Science, pp. 548–561, Marseille, France, 2008.
- B. Xiao, X. Yang, Y. Xu, and H. Zha, “Learning distance metric for regression by semidefinite programming with application to human age estimation,” in Proceedings of the 17th ACM International Conference on Multimedia, pp. 451–460, October 2009.
- J. Lu, G. Wang, and P. Moulin, “Human identity and gender recognition from gait sequences with arbitrary walking directions,” IEEE Transactions on Information Forensics and Security, vol. 9, no. 1, pp. 51–61, 2014.
- A. Mignon and F. Jurie, “PCCA: a new approach for distance learning from sparse pairwise constraints,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR '12), pp. 2666–2672, June 2012.
- W. S. Zheng, S. Gong, and T. Xiang, “Person re-identification by probabilistic relative distance comparison,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR '11), pp. 649–656, June 2011.
- D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” International Journal of Computer Vision, vol. 60, no. 2, pp. 91–110, 2004.
- T. Ahonen, A. Hadid, and M. Pietikäinen, “Face description with local binary patterns: application to face recognition,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 12, pp. 2037–2041, 2006.
Copyright © 2015 Lu Kou et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.