Abstract
This paper presents a critical evaluation of multialgorithmic face recognition systems for human authentication in unconstrained environment. We propose different frameworks of multialgorithmic face recognition system combining holistic and texture methods. Our aim is to combine the uncorrelated methods of the face recognition that supplement each other and to produce a comprehensive representation of the biometric cue to achieve optimum recognition performance. The multialgorithmic frameworks are designed to combine different face recognition methods such as (i) Eigenfaces and local binary pattern (LBP), (ii) Fisherfaces and LBP, (iii) Eigenfaces and augmented local binary pattern (A-LBP), and (iv) Fisherfaces and A-LBP. The matching scores of these multialgorithmic frameworks are processed using different normalization techniques whereas their performance is evaluated using different fusion strategies. The robustness of proposed multialgorithmic frameworks of face recognition system is tested on publicly available databases, for example, AT & T (ORL) and Labeled Faces in the Wild (LFW). The experimental results show a significant improvement in recognition accuracies of the proposed frameworks of face recognition system in comparison to their individual methods. In particular, the performance of the multialgorithmic frameworks combining face recognition methods with the devised face recognition method such as A-LBP improves significantly.
1. Introduction
Biometrics is a technology that examines unique physiological or behavioral characteristics of an individual for his/her authentication. It provides a mechanism of automatic recognition of individuals based on their one or more biometric cues. The recognition system which is typically based on a single biometric cue has not always yielded the desired results, because of the problems of noisy data, lack of uniqueness, and spoofing attacks. In the critical applications such as border crossing and emigrant check points, biometric systems of a single cue are not able to provide the desired level of performance. Therefore, multibiometric systems are used, but major challenges of these systems are to gather the information from multiple biometric cues of an individual and their fusion, for example, unobtrusive biometrics (face and fingerprint) with obtrusive biometrics (iris and ECG) fulfilling the objective that they supplement each other [1, 2]. These systems are computationally expensive and raise the response time too. One possible solution of these issues is to work on a single biometric cue with different representation so as to analyze it comprehensively for better recognition of individuals.
In order to make a viable biometric system that works more effectively in various applications under different conditions the multialgorithmic biometric approach could be a possible solution; in particular, unobtrusive biometrics such as face could be chosen as a suitable biometric identifier. Marcialis and Roli have proposed the fusion of two well-known face recognition algorithms, namely, principal component analysis (PCA) and linear discriminant analysis (LDA) that conformed benefits of fused algorithms in video-based surveillance applications [3]. Mian et al. have presented 2D and 3D multimodal hybrid face recognition algorithm and tested its performance on the FRGC v1.0 dataset [4]. Zakariya et al. have presented a multialgorithmic based approach using PCA and discrete cosine transformation (DCT) for personal identification and proved its usefulness [5]. Kar et al. have developed a multialgorithmic based face recognition system, harnessing the combination of gray level statistical correlation method with PCA or DCT methods, in order to intensify the performance of the systems [6]. Lone et al. have developed a face recognition system based on the consolidation of scores obtained from different techniques, such as PCA, DCT, template matching using correlation, and partitioned iterative function system [7]. Imran et al. have proposed fusion using popular subspace methods including PCA, LDA, locality preserving projection, and independent component analysis [8]. It is computed by considering different combinations of set of two, three, and four subspace methods. Their results have shown an improved performance of multialgorithmic face recognition technique.
The aim of this work is to combine the uncorrelated face recognition approaches such as holistic and texture based representations to achieve the multialgorithmic frameworks. The fusion of multiple information of a biometric cue shows an improvement in the recognition performance of the biometric system. This paper presents different frameworks of face recognition system utilizing multialgorithmic techniques for achieving the optimum recognition performance. The concern is to combine the normalized scores of the uncorrelated face recognition methods, to achieve the multialgorithmic characteristics, for the individual judgment. The proposed frameworks of multialgorithmic face recognition system are devised from the following face recognition techniques: (i) Eigenfaces [10] and local binary pattern (LBP) [11, 12], (ii) Fisherfaces [13] and LBP, (iii) Eigenfaces and augmented local binary pattern (A-LBP) [14–16], and (iv) Fisherfaces and A-LBP. The performance of these frameworks is evaluated using different fusion strategies.
The remainder of the paper is structured as follows: Section 2 presents a review of existing face recognition techniques. The frameworks of proposed multialgorithmic face recognition system are given in Section 3. Section 4 shows the experimental results obtained from the proposed frameworks that are tested on the public domain databases, such as AT & T (ORL) [17] and Labeled Faces in the Wild (LFW) [18]. Finally, the discussion and conclusion are outlined in Section 5.
2. Face Recognition Methods
Recognizing individuals from their faces is natural phenomena and this task is effortlessly performed by us in our daily life. An automatic face recognition system is typically designed to compute the similarity of the facial images. Over thousands of research papers are published every year on face recognition. Most of the published methods are performed well in controlled environments, for example, Eigenfaces and Fisherfaces [19, 20]. The other methods that are claimed to be performed better in uncontrolled environment mainly include LBP and A-LBP [11, 12, 14–16]. These approaches may be broadly categorized on the basis of their representations, that is, (i) holistic methods, for example, Eigenfaces and Fisherfaces, and (ii) texture based methods, for example, LBP and A-LBP.
In the last two decades, different face recognition methods based on holistic approach of facial representation are proposed [21]. It includes some of the well-known face recognition algorithms: principal component analysis (Eigenfaces) [10], linear discriminant analysis [22], Fisherfaces [13], independent component analysis [23], and elastic bunch graph matching [24]. A texture based technique uses a type of texture representation of facial images to be recognized. It includes the well-known texture representation methods, LBP [11, 12], SIFT [25], and SURF [26] whereas LBP is more popular among them. LBP is a feature based approach which is being used for classification purposes in computer vision. A LBP operator labels the pixels of an image with decimal numbers and also encodes the local structure around each pixel. Each pixel is compared with its eight neighbors in neighborhood by subtracting the central pixels value as a threshold. The resulting nonnegative values are encoded with 1 and others with 0. A binary number is obtained by concatenating all these binary codes in a clockwise direction starting from top-left corner and placed left to right. Its decimal value is then computed and used for the labeling perspective. The derived binary numbers are called LBP [11, 12].
We have worked on the modified approach of LBP called augmented local binary pattern (A-LBP) that makes use of nonuniform patterns in the representation process [16]. In the prior work of LBP, the nonuniform patterns are either treated as noise and discarded during the texture representations or used in combination with the uniform patterns. Our developed method considers the nonuniform patterns and extracts the discriminatory information available to them, so as to prove their usefulness. They are used in combination with the neighboring uniform patterns and extract the useful information from the local descriptors [14, 15].
3. Proposed Framework of Multialgorithmic Face Recognition System
The framework of the multialgorithmic face recognition systems is shown in Figure 1. The framework first preprocesses the input which is a set of images to make them acceptable for further processing. Next, the images are passed to different face recognition systems that employ holistic and texture based approaches that compute the scores that are to be used in matching process. The scores are normalized to make them homogeneous. The scores are now fused depending upon the choice of algorithms. Thus, we obtain total scores of fused algorithms. These scores are finally used in matching process and thus genuine and impostor scores are generated after comparing with the test images.
In this experiment, concern of the proposed frameworks is to fuse distinct face recognition algorithms with our A-LBP face recognition algorithm that is claimed to be better in uncontrolled environment and to achieve the robust face recognition systems. The following pair of strategies is used to fuse. It includes (i) Eigenfaces and LBP, (ii) Fisherfaces and LBP, (iii) Eigenfaces and A-LBP, and (iv) Fisherfaces and A-LBP.
The Eigenfaces technique is typically based on linearly projecting the image space to a lower dimension. The Fisherfaces technique which employs the principal component analysis for dimensionality reduction results in projection direction that maximizes the total scatter across all the classes. Principal component analysis retains unfavorable changes due to illumination and facial expression [10]. Fisherfaces is used to find out the basis vectors that minimizes within-class matrix differences and maximizes between-class matrix distances [27]. Thus, ratio of the between-class scatter and the within-class scatter is maximized [13].
Contrary to Eigenfaces and Fisherfaces, the local binary pattern is robust to the changes of illumination and facial expressions. It is a simple and efficient operator, consolidating statistical and structural approaches, both in the texture analysis. It is a powerful grayscale invariant metric derived from a general definition of texture in a local neighborhood. Due to its discriminating power and computational simplicity, it becomes a popular approach [11, 12]. Contrasting to LBP, our proposed A-LBP technique also considered the nonuniform patterns and transformed them into uniform patterns for texture representation and classification, thus preserving the discriminatory information of the chosen cues [14–16].
3.1. Score Normalization Techniques
Normalization is a process of transformation and mapping of heterogeneous scores (i.e., different scales) of distinct biometrics and makes them homogeneous (i.e., common scales) [28]. We obtain the multialgorithmic biometric frameworks where the matching scores of different biometrics or algorithms are transformed and mapped to a common scale before fusion.
Different score normalization techniques are found in literature, the most common among are min–max (MM) and tanh [29]. Let us assume that be the set of true scores of individuals and be the set of impostor scores of those individuals, where for the biometric method . The composite set of matching scores is denoted as (i.e., and ).
The distance scores () of user for method can be converted into similarity scores in the common range. Suppose it should be in using the following formula: where is the similarity scores of method . Otherwise, if the distance scores lie in the range then they are simply converted into similarity scores by subtracting them, from the value of , that is,. The summary of these score normalization techniques is as follows: MM: it transforms the raw scores into the common range of using Tanh: it corresponds to the raw scores of in the common range of as where and are the mean and the standard deviation of the true matching scores of method , respectively, and is the normalized scores of that method .
3.2. Fusion Techniques
Kittler et al. have evolved a theoretical framework for reconciling the evidence achieved from different schemes, such as sum rule, max rule, and min rule [30]. In order to use these schemes, the matching scores are converted into a posteriori probabilities conforming to a true user and an impostor. They consider the problem of classifying an input pattern into one of the possible classes based on the evidence presented by different classifiers. Let be the feature vector provided to the th classifier and let outputs of the respective classifiers be , that is, the a posteriori probability of the pattern belonging to class given the feature vector . Let be the class to which the input pattern is finally assigned. The following fusion rules have been generalized by Jain et al. for computing the value of class [31]: Sum rule: it assumes that the a posteriori probabilities computed by the respective classifiers do not deviate from the prior probabilities. This rule is preferred when there is a high level of noise leading to vagueness in the classification problem. It assigns the input pattern to class such that Max rule: it approximates the mean of the a posteriori probabilities by the maximum value. The input pattern is assigned to class such that Min rule: it is derived by bounding the product of a posteriori probabilities. Here, the input pattern is assigned to class such that where argmax is simply an operator and is the measurement process model of th representation.
4. Experimental Results
4.1. Face Databases
The proposed frameworks of multialgorithmic face recognition system are tested on different public domain face databases, for example, AT & T (ORL) [17] and Labeled Faces in the Wild [18]. The first database AT & T (ORL) consists of 40 subjects whereas each subject has 10 samples, and each image is downsized towards the size of . Next, Labeled Faces in the Wild database consists of 40 subjects, and there are 10 samples per subject and each image is downsized to the size of . These databases are different in the degree of variation, for instance, pose, illumination, expression, and the eye glasses that are presented in their facial images. A total of 800 face images are used to recognize 80 distinct individuals. The proposed frameworks of multialgorithmic face recognition system are trained for each face database independently, whereas the test face images are selected arbitrarily from the available training faces for each individual and their performance is computed.
4.2. Performance Metrics
The performance of the proposed frameworks of multialgorithmic face recognition system is computed using equal error rate (EER), which is an error, where the probability of acceptance is assumed to be the same as the probability of rejection of the people who should be correctly verified. The performance is also verified using the receiver operating characteristic (ROC) curves. The ROC curve is a two-dimensional measure of classification performance that plots the probability of the true acceptance rate (TAR) against the probability of the false acceptance rate (FAR), where . Accuracy of the system can be computed using FAR and false rejection rates (FRR):
The acceptance threshold is selected from the receiver operating characteristic curve so as to produce EER; that is, FAR FRR (the cross point where the false acceptance rate and the false rejection rates are the same). The values of FAR and FRR depend on the chosen threshold. As the threshold increases the FAR decreases and the FRR increases and vice versa. There is trade-off between the choice of FAR and FRR values that depend on the security and throughput requirements of the system to be used.
4.3. Recognition Performance Using MM Normalization Technique
Recognition performance of the proposed frameworks of multialgorithmic face recognition system is tested on publicly available databases using a Bray Curtis dissimilarity metric [32]. The recognition accuracy results obtained from the fusion of face recognition techniques using different normalization techniques are shown in Table 1. The improvement in recognition performance of different frameworks of face recognition system is clearly visible. For example, recognition performance is found better for tanh normalization technique using sum rule of fusion for AT & T (ORL) database. Further, a significant improvement in the recognition performance is also found for LFW database for same normalization and fusion techniques. The matching scores are distributed normally and thus the sum rule of fusion reduces the variance present in the scores; therefore performance improves. It results in the better recognition rate as compared to other fusion strategies.
The ROC curves for AT & T (ORL) database are plotted for different frameworks of multialgorithmic face recognition system using MM normalization technique that are shown in Figure 2. The recognition accuracy results of the fused uncorrelated algorithms such as Eigenfaces and LBP, Fisherfaces and LBP, Eigenfaces and A-LBP, and Fisherfaces and A-LBP using sum, max, and min rule are shown, respectively, in Figures 2(a), 2(b), and 2(c). Among the four multialgorithmic approaches, the last three approaches report better TAR value of 97.79%, 97.05%, and 97.95% (Table 1), respectively, using sum rule of fusion, whereas the first multialgorithmic approach reports a better value of 95.38% using the max rule of fusion. On these multialgorithmic frameworks, the recognition results reach 100% of TAR at 3% of FAR, 4% of FAR, and 8% of the FAR, respectively. A similar trend is also observed for the proposed frameworks on LFW face database.
(a) Sum rule
(b) Max rule
(c) Min rule
The ROC curves plotted for the reported results of different frameworks and fusion strategies are shown in Figure 3.
(a) Sum rule
(b) Max rule
(c) Min rule
4.4. Recognition Performance Using Tanh Normalization Technique
The ROC curves for AT & T (ORL) database are plotted for different frameworks of multialgorithmic face recognition system using tanh normalization technique that are shown in Figure 4. The recognition results of the fused techniques, for example, Eigenfaces and LBP, Fisherfaces and LBP, Eigenfaces and A-LBP, and Fisherfaces and A-LBP techniques on distinct fusion methods, are shown, respectively, in Figures 4(a), 4(b), and 4(c). The multialgorithmic frameworks report better TAR value of 95.64%, 97.79%, 97.31%, and 97.95% (Table 1), respectively, using sum rule. On these multialgorithmic frameworks, the recognition results reach 100% of TAR at 1.5% of FAR, 6% of FAR, and 10% of the FAR, respectively. A similar trend in the recognition results is also reported for the proposed frameworks of multialgorithmic face recognition system on LFW face database. The ROC curves of LFW database are shown in Figure 5. As compared to MM, tanh normalization technique performs better, because of its sensitiveness to the outliers.
(a) Sum rule
(b) Max rule
(c) Min rule
(a) Sum rule
(b) Max rule
(c) Min rule
5. Conclusion
This paper has presented the novel frameworks of multialgorithmic face recognition system and evaluated their performance in noncooperative environment. These frameworks are developed by combining evidences presented by the uncorrelated face recognition methods. We have evaluated the performance of different unibiometric face recognition methods such as Eigenfaces, Fisherfaces, LBP, and A-LBP on publicly available face databases such as AT & T (ORL) and LFW. The performance of the proposed frameworks of multialgorithmic face recognition system obtained from the fusion of Eigenfaces and LBP, Fisherfaces and LBP, Eigenfaces and A-LBP, and Fisherfaces and A-LBP on the stated databases is computed. The recognition results of the proposed frameworks are evaluated using different normalization and fusion techniques.
The strength of the proposed frameworks of multialgorithmic face recognition system is that they are prepared from unibiometric methods and provide a robust solution rather than creating a complex method. Thus, the proposed frameworks have reduced the complexities of multibiometric systems to a considerable extent. In most cases, the fused techniques have shown significant improvement in recognition performance in comparison to their unibiometric systems. It has also observed that the sum rule plays an important role in comparison to other chosen fusion methods, such as the max and min rules because it reduces the variance present in the matching scores of different methods and thus improves the recognition accuracy. The evaluated frameworks of multialgorithmic face recognition system may contribute an important role in identification of individual faces in the noncooperative environment.
Competing Interests
The authors declare that there are no competing interests regarding the publication of this paper.
Acknowledgments
The authors acknowledge the Institute of Engineering and Technology (IET), Lucknow, Dr. A. P. J. Abdul Kalam Technical University, Uttar Pradesh, Lucknow, for their partial financial support to carry out this research under the Technical Education Quality Improvement Programme (TEQIP-II) grant.