Table of Contents Author Guidelines Submit a Manuscript
Mathematical Problems in Engineering
Volume 2015 (2015), Article ID 472473, 5 pages
http://dx.doi.org/10.1155/2015/472473
Research Article

Learning a Genetic Measure for Kinship Verification Using Facial Images

College of Information Engineering, Capital Normal University, Beijing 100048, China

Received 11 October 2014; Accepted 20 January 2015

Academic Editor: Yuri Vladimirovich Mikhlin

Copyright © 2015 Lu Kou et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Abstract

Motivated by the key observation that children generally resemble their parents more than other persons with respect to facial appearance, distance metric (similarity) learning has been the dominant choice for state-of-the-art kinship verification via facial images in the wild. Most existing learning-based approaches to kinship verification, however, are focused on learning a genetic similarity measure in a batch learning manner, leading to less scalability for practical applications with ever-growing amount of data. To address this, we propose a new kinship verification approach by learning a sparse similarity measure in an online fashion. Experimental results on the kinship datasets show that our approach is highly competitive to the state-of-the-art alternatives in terms of verification accuracy, yet it is superior in terms of scalability for practical applications.

1. Introduction

Due to the fact that rich human characteristics, such as gender, identity, expression, and ethnicity, can be effectively extracted from facial images, a variety of face analysis problems, ranging from face recognition, facial expression recognition, and gender estimation to age estimation, have been extensively studied over the past decades [1]. Kinship verification using facial images, however, is a relatively new and challenging problem in biometrics [2], which is mainly motivated by the phenomenon that children generally look like their parents more than other people due to the kinship relation. Recent evidence in psychology has shown that facial appearance is a reliable cue to measure the genetic similarity between children and their parents [39]. In practice, there are some important potential applications of kinship verification via facial images, including missing children search and social media analysis [10, 11]. Figure 1 presents some image examples (with kinship relation) from the dataset KinFaceW [12].

Figure 1: Some image examples (with kin relation) from KinFaceW dataset [12]. From top to bottom are the father-son (F-S), father-daughter (F-D), mother-son (M-S), and mother-daughter (M-D) kinship relations, respectively.

Over the past few years, a few seminal works on kinship verification using facial images [2, 10, 11, 1321] have been proposed. Roughly, existing methods for kinship verification are either feature-based [2, 13, 1517, 19] or learning-based [10, 11, 14, 18, 20]. The former aims for extraction of the discriminative feature from facial images to characterize genetic property in human appearance. Learning-based approaches, however, are focused on learning a genetic measure via training data based on some discriminative learning technologies, such as subspace learning and distance metric learning, to improve the separability of facial images for kinship verification. Despite the promising results by existing learning-based approaches to kinship verification, they aim to learn a distance metric (or transform) in a batch learning way, leading to less efficiency or scalability even for medium-scale applications. Unlike the previous learning-based studies, we aim to learn a sparse similarity measure for kinship verification in an online manner. Our proposed approach not only is able to achieve highly competitive verification accuracy to state-of-the-art kinship verification method but also is superior in terms of scalability, making it more scalable for practical applications with ever-growing amount of data.

The rest of this paper is organized as follows. In Section 2 we elaborate our approach to kinship verification from facial images. Experiments and evaluations on performance are conducted in Section 3. Finally, we conclude and summarize the paper in Section 4.

2. Our Approach

In the past few years, many metric learning methods [2327] have been proposed in the machine learning community, and they have been successfully applied in various vision tasks, including face recognition [10, 18, 2834], activity recognition [35], human age estimation [36], gait recognition [37], and person reidentification [30, 38, 39]. The most existing metric learning methods aim at learning a Mahalanobis distance metric that measures the squared distance between a pair of samples and : where and is a positive semidefinite (PSD) matrix. Existing metric learning methods for kinship verification (e.g., [10]) are often proposed to minimize interclass variation and maximize intraclass variation simultaneously. In this work, we instead attempt to learn a genetic similarity measure explicitly, such that similarity score of a positive image pair is higher than those of the negative pairs. Let be the training set consisting of positive image pairs (with kinship relation), where denote the facial images of the th parent and child, respectively. The goal of this problem is to learn a similarity measure function that specifies the kinship constraints on any quadruple input , where , , . Formally, the kinship constraints for a quadruple input can be written by where , , , and 1 is a margin constant. There exist two triplet constraints for each quadruple input, on which similarity score of the positive pair is higher than that of image pair from different family. This can be schematically illustrated in Figure 2. We consider here a parametric similarity function with bilinear form, , where . Note that here is not necessarily symmetric or PSD. Given the similarity function and the constraints in (2), the problem of genetic similarity learning can be formulated as the following optimization task: where is a sparse regularization term that limits the model complexity and . Different from the widely used OASIS algorithm [22] designed to deal with the triplet constraints, we aim to directly deal with the quadruple constraints for kinship verification. The loss function for our problem is defined based on the hinge loss where . The optimization task of (3) corresponds to a batch learning formulation, and it is less efficient and scalable to practical applications even for medium-size training set. To tackle this issue, we instead aim to learn the similarity measure parameterized by in an online manner.

Figure 2: Quadruple constraint for kinship verification. There exist two triplet constraints for each quadruple input, on which the similarity score of positive pair (denoted by solid arrow) is higher than those of image pairs from different family (denoted by dotted arrow).

At each time , a quadruple instance is received, and the model is sequentially updated by solving the following optimization problem: where is a trace norm that encourages learning a low rank similarity matrix to limit the model complexity. The above online optimization can be solved by online gradient descent: where is the learning rate, is the identity matrix, and is the subgradient of with respect to , which can be computed by The proposed online learning algorithm for genetic similarity measure is outlined in Algorithm 1.

Algorithm 1: Online similarity measure learning for kinship verification.

3. Experiment and Discussion

We conducted experiments on the kin datasets that are publicly available [10, 12] (KinFaceW-I and KinFace-II) to verify the effectiveness of our proposed approach. For both datasets, there are four kin relations: mother-son (M-S), mother-daughter (M-D), father-son (F-S), and father-daughter (F-D). All of the facial images have been aligned and cropped into 64 × 64 pixels.

We performed 5-fold cross validation on the two kin datasets, where each subset was equally divided into 5-fold so that each fold consists of nearly the same number of image pairs with kin relation. The parameters and are empirically set as 1.0 and 10−4, respectively. For the feature representations of the facial image, three descriptors, SIFT [40], learning-based (SPLE) [13], and local binary patterns (LBP) [41] are used. In our experiments, 256 bins are used for LBP, and we followed the parameter setting in [10] for SPLE and used 200 bins to encode a histogram. For the SIFT descriptors, 16 × 16 patches over a grid with space of 8 pixels are extracted from each face image.

We have compared our proposed approach with NRML [10] and OASIS [22]. The former is a state-of-the-art learning-based kinship verification approach, and the latter represents a popular algorithm for online similarity learning. Tables 1 and 2 list the mean verification rate of the three approaches with different features on the KinFaceW dataset. As shown in the two tables, our approach outperforms OASIS and is highly competitive to NRML in terms of verification rate. In addition, we can make the observation that SPLE is the best feature representation for our kinship verification problem. This is mainly attributed to the coding scheme of SPLE that is directly learned from the training examples, leading to better verification accuracy than two other hand-crafted feature descriptors.

Table 1: Mean verification rate (%) of different approaches on the KinFaceW-I dataset.
Table 2: Mean verification rate (%) of different approaches on the KinFaceW-II dataset.

It should be noticed that, compared to NRML, our approach is superior in terms of scalability for practical applications due to the online learning nature in model training. We believe this is very important for the kinship verification on the large-scale dataset with ever-growing amount of data.

4. Conclusion

We have presented a new scheme for kinship verification via facial images in wild conditions by explicitly learning a sparse genetic similarity measure in an online manner. Experimental results on kinship datasets demonstrated that our approach is not only highly competitive to the state-of-the-art algorithm in terms of the verification accuracy but also superior in terms of scalability for practical applications with ever-growing amount of data. For future work, we are interested in investigating the fusion of multiple feature representations within the online learning framework to further improve the overall kinship verification performance. Also, it is helpful for face (kin) verification to work out the similarity of facial components (e.g., eyes or mouths). In this setting, robust face landmarks estimation is often needed to parse the face into facial components. How to exploit it in our proposed approach to work on partially occluded faces or faces viewed from the side appears to be another interesting direction of future work.

Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

Acknowledgments

This work is supported by the Natural Science Foundation of China under Grant nos. 61373090 and 11178017 and the Beijing Natural Science Foundation of China under Grant no. 4132014.

References

  1. W. Zhao, R. Chellappa, P. J. Phillips, and A. Rosenfeld, “Face recognition: a literature survey,” ACM Computing Surveys, vol. 35, no. 4, pp. 399–458, 2003. View at Publisher · View at Google Scholar · View at Scopus
  2. R. Fang, K. Tang, N. Snavely, and T. Chen, “Towards computational models of kinship verification,” in Proceedings of 17th IEEE International Conference on Image Processing (ICIP '10), pp. 1577–1580, IEEE, Hong Kong, China, September 2010. View at Publisher · View at Google Scholar
  3. L. M. DeBruine, F. G. Smith, B. C. Jones, S. Craig Roberts, M. Petrie, and T. D. Spector, “Kin recognition signals in adult faces,” Vision Research, vol. 49, no. 1, pp. 38–43, 2009. View at Publisher · View at Google Scholar · View at Scopus
  4. A. Alvergne, R. Oda, C. Faurie, A. Matsumoto-Oda, V. Durand, and M. Raymond, “Cross-cultural perceptions of facial resemblance between kin,” Journal of Vision, vol. 9, no. 6, article 23, 2009. View at Publisher · View at Google Scholar · View at Scopus
  5. M. F. dal Martello and L. T. Maloney, “Where are kin recognition signals in the human face?” Journal of Vision, vol. 6, no. 12, pp. 1356–1366, 2006. View at Publisher · View at Google Scholar · View at Scopus
  6. M. F. dal Martello and L. T. Maloney, “Lateralization of kin recognition signals in the human face,” Journal of Vision, vol. 10, no. 8, article 9, pp. 1–10, 2010. View at Publisher · View at Google Scholar · View at Scopus
  7. L. M. DeBruine, F. G. Smith, B. C. Jones, S. Craig Roberts, M. Petrie, and T. D. Spector, “Kin recognition signals in adult faces,” Vision Research, vol. 49, no. 1, pp. 38–43, 2009. View at Publisher · View at Google Scholar · View at Scopus
  8. G. Kaminski, S. Dridi, C. Graff, and E. Gentaz, “Human ability to detect kinship in strangers' faces: effects of the degree of relatedness,” Proceedings of the Royal Society B: Biological Sciences, vol. 276, no. 1670, pp. 3193–3200, 2009. View at Publisher · View at Google Scholar · View at Scopus
  9. G. Kaminski, F. Ravary, C. Graff, and E. Gentaz, “Firstborns' disadvantage in kinship detection,” Psychological Science, vol. 21, no. 12, pp. 1746–1750, 2010. View at Publisher · View at Google Scholar · View at Scopus
  10. J. Lu, X. Zhou, Y.-P. Tan, Y. Shang, and J. Zhou, “Neighborhood repulsed metric learning for kin-ship verification,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 36, no. 2, pp. 331–345, 2014. View at Publisher · View at Google Scholar · View at Scopus
  11. S. Xia, M. Shao, and Y. Fu, “Kinship verification through transfer learning,” in Proceedings of the 22nd International Joint Conference on Artificial Intelligence (IJCAI '11), pp. 2539–2544, July 2011. View at Publisher · View at Google Scholar · View at Scopus
  12. KinFaceW Dataset, http://www.xeehoo.com/zxz.html.
  13. X. Zhou, J. Hu, J. Lu, Y. Shang, and Y. Guan, “Kinship verification from facial images under uncontrolled conditions,” in Proceedings of the 19th ACM International Conference on Multimedia (MM '11), pp. 953–956, December 2011. View at Publisher · View at Google Scholar · View at Scopus
  14. M. Shao, S. Xia, and Y. Fu, “Genealogical face recognition based on UB KinFace database,” in Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops (CVPRW '11), pp. 60–65, Colorado Springs, Colo, USA, June 2011. View at Publisher · View at Google Scholar · View at Scopus
  15. G. Guo and X. Wang, “Kinship measurement on salient facial features,” IEEE Transactions on Instrumentation and Measurement, vol. 61, no. 8, pp. 2322–2325, 2012. View at Publisher · View at Google Scholar · View at Scopus
  16. X. Zhou, J. Lu, J. Hu, and Y. Shang, “Gabor-based gradient orientation pyramid for kinship verification under uncontrolled environments,” in Proceedings of the ACM International Conference on Multimedia, pp. 2322–2325, November 2012.
  17. H. Dibeklioglu, A. A. Salah, and T. Gevers, “Like father, like son: facial expression dynamics for kinship verification,” in Proceedings of the 14th IEEE International Conference on Computer Vision (ICCV '13), pp. 1497–1504, December 2013. View at Publisher · View at Google Scholar · View at Scopus
  18. J. Lu, J. Hu, X. Zhou, Y. Shang, Y.-P. Tan, and G. Wang, “Neighborhood repulsed metric learning for kinship verification,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR '12), pp. 2594–2601, June 2012. View at Publisher · View at Google Scholar · View at Scopus
  19. N. Kohli, R. Singh, and M. Vatsa, “Self-similarity representation of Weber faces for kinship classification,” in Proceedings of the IEEE 5th International Conference on Biometrics: Theory, Applications and Systems (BTAS '12), pp. 245–250, September 2012. View at Publisher · View at Google Scholar · View at Scopus
  20. S. Xia, M. Shao, J. Luo, and Y. Fu, “Understanding kin relationships in a photo,” IEEE Transactions on Multimedia, vol. 14, no. 4, pp. 1046–1056, 2012. View at Publisher · View at Google Scholar · View at Scopus
  21. H. Yan, J. Lu, W. Deng, and X. Zhou, “Discriminative multimetriclearning for kinship verification,” IEEE Transactions on Information Forensics and Security, vol. 9, no. 7, pp. 1169–1178, 2014. View at Publisher · View at Google Scholar
  22. G. Chechik, V. Sharma, U. Shalit, and S. Bengio, “Large scale online learning of image similarity through ranking,” Journal of Machine Learning Research (JMLR), vol. 11, pp. 1109–1135, 2010. View at Publisher · View at Google Scholar · View at MathSciNet
  23. E. P. Xing, A. Y. Ng, M. I. Jordan, and S. Russell, “Distance metric learning, with application to clustering with side-information,” in Proceedings of the Advances in Neural Information Processing Systems, pp. 505–512, December 2002. View at Scopus
  24. J. V. Davis, B. Kulis, P. Jain, S. Sra, and I. S. Dhillon, “Information-theoretic metric learning,” in Proceedings of the 24th International Conference on Machine Learning (ICML '07), pp. 209–216, June 2007. View at Publisher · View at Google Scholar · View at Scopus
  25. J. Goldberger, S. Roweis, G. Hinton, and R. Salakhutdinov, “Neighbourhood components analysis,” in Proceedings of the 18th Annual Conference on Neural Information Processing Systems (NIPS '04), pp. 2539–2544, December 2004. View at Scopus
  26. S. Parameswaran and K. Q. Weinberger, “Large margin multi-task metric learning,” in Proceedings of the 24th Annual Conference on Neural Information Processing Systems (NIPS '10), pp. 1867–1875, December 2010. View at Scopus
  27. K. Q. Weinberger and L. K. Saul, “Distance metric learning for large margin nearest neighbor classification,” Journal of Machine Learning Research, vol. 10, pp. 207–244, 2009. View at Google Scholar · View at Scopus
  28. R. G. Cinbis, J. Verbeek, and C. Schmid, “Unsupervised metric learning for face identification in TV video,” in Proceedings of the IEEE International Conference on Computer Vision (ICCV '11), pp. 1559–1566, Barcelona, Spain, November 2011. View at Publisher · View at Google Scholar · View at Scopus
  29. M. Guillaumin, J. Verbeek, and C. Schmid, “Is that you? Metric learning approaches for face identification,” in Proceedings of the 12th International Conference on Computer Vision (ICCV '09), pp. 498–505, Kyoto, Japan, October 2009. View at Publisher · View at Google Scholar · View at Scopus
  30. M. Kostinger, M. Hirzer, P. Wohlhart, P. M. Roth, and H. Bischof, “Large scale metric learning from equivalence constraints,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR '12), pp. 2288–2295, IEEE, Providence, RI, USA, June 2012. View at Publisher · View at Google Scholar · View at Scopus
  31. J. Lu, Y.-P. Tan, and G. Wang, “Discriminative multimanifold analysis for face recognition from a single training sample per person,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 35, no. 1, pp. 39–51, 2013. View at Publisher · View at Google Scholar · View at Scopus
  32. H. Nguyen and L. Bai, “Cosine similarity metric learning for face verification,” in Computer Vision—ACCV 2010: 10th Asian Conference on Computer Vision, Queenstown, New Zealand, November 8–12, 2010, Revised Selected Papers, Part II, vol. 6493 of Lecture Notes in Computer Science, pp. 709–720, Springer, Berlin, Germany, 2011. View at Publisher · View at Google Scholar
  33. W. Deng, J. Hu, J. Guo, W. Cai, and D. Feng, “Robust, accurate and efficient face recognition from a single training image: a uniform pursuit approach,” Pattern Recognition, vol. 43, no. 5, pp. 1748–1762, 2010. View at Publisher · View at Google Scholar · View at Scopus
  34. W. Deng, J. Hu, J. Lu, and J. Guo, “Transform-invariant PCA: a unified approach to fully automatic face alignment, representation, and recognition,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 36, no. 6, pp. 1275–1284, 2014. View at Publisher · View at Google Scholar · View at Scopus
  35. D. Tran and A. Sorokin, “Human activity recognition with metric learning,” in Proceedings of the 10th European Conference on Computer Vision, vol. 5302 of Lecture Notes in Computer Science, pp. 548–561, Marseille, France, 2008. View at Publisher · View at Google Scholar
  36. B. Xiao, X. Yang, Y. Xu, and H. Zha, “Learning distance metric for regression by semidefinite programming with application to human age estimation,” in Proceedings of the 17th ACM International Conference on Multimedia, pp. 451–460, October 2009. View at Publisher · View at Google Scholar · View at Scopus
  37. J. Lu, G. Wang, and P. Moulin, “Human identity and gender recognition from gait sequences with arbitrary walking directions,” IEEE Transactions on Information Forensics and Security, vol. 9, no. 1, pp. 51–61, 2014. View at Publisher · View at Google Scholar · View at Scopus
  38. A. Mignon and F. Jurie, “PCCA: a new approach for distance learning from sparse pairwise constraints,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR '12), pp. 2666–2672, June 2012. View at Publisher · View at Google Scholar · View at Scopus
  39. W. S. Zheng, S. Gong, and T. Xiang, “Person re-identification by probabilistic relative distance comparison,” in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR '11), pp. 649–656, June 2011. View at Publisher · View at Google Scholar · View at Scopus
  40. D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” International Journal of Computer Vision, vol. 60, no. 2, pp. 91–110, 2004. View at Publisher · View at Google Scholar · View at Scopus
  41. T. Ahonen, A. Hadid, and M. Pietikäinen, “Face description with local binary patterns: application to face recognition,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 28, no. 12, pp. 2037–2041, 2006. View at Publisher · View at Google Scholar · View at Scopus