Abstract
Biometric plays a vital role in human authentication systems. Unimodal and multimodal biometrics have been active research areas for the past few decades. The investigation of palmprint recognition under various illuminations, rotations, and translations is a challenging task. The research work on multimodal palmprint recognition systems has widely increased to improve the recognition rate and reduce execution time. In this article, a multimodal palmprint biometric system is formed by combining the left and right palmprint images to obtain an optimal recognition rate. A modified multilobe ordinal filter (MMLOF) is used to extract the features. Feature-level fusion is used to fuse the left and right palmprint images. This results in a high-dimension feature vector that requires larger memory to store. It creates redundant and irrelevant features that affect the recognition rate. To overcome these limitations, the optimal MMOF features are extracted by optimization techniques such as particle swarm optimization (PSO) and the genetic algorithm (GA). Finally, PSO and GA optimization algorithms are wrapped with the nearest neighbor classifier (NN) to evaluate the fitness function. The experimental analyses are conducted to identify the performance of GA and PSO using the IITD palmprint dataset. The 1st order MMLOF with GA (multimodal) converges faster and outperforms the 1st order MMLOF with PSO (multimodal) and obtains an optimal recognition rate of 96.95%.
1. Introduction
Many problems arise in a unimodal biometric system such as intraclass variation, interclass similarity, spoofing, and nonuniversality [1]. These problems will decrease the recognition rate. In a multimodal biometric system, two or more unimodal systems are combined to form identification systems. The multimodal biometric system is used in civilian applications such as pension schemes, ATM security, and credit card transactions. Rattani et al. introduced a multimodal method by combining fingerprint and face traits [2]. Here, the feature-level fusion performs better than the score-level fusion. Nagesh kumar and Shanmukha proposed joined face and palm images by using score-level fusion [3]. Imran et al. joined ear and face traits [4]. The fusion is carried out at the decision level. Soltane and Doghmane described a human identification method by using speech and face traits to improve the unimodal biometric problem [5]. Jagadeesan and Duraiswamy reviewed and discussed the feature-level fusion to fuse the iris and fingerprint images [6]. Taouche and Belhadef proposed multimodal palmprint recognition by combining both the left and right palmprints of the same person. Feature selection is carried out by using a genetic algorithm (GA) and a backtracking search algorithm. Here, the problem arising out of unimodal biometrics is overcome by multimodal biometrics. Still, in the multimodal biometric, the fusion of two different modalities such as the face and palm or the finger and retina in the feature level causes irrelevant feature sets. This leads to a poor recognition rate, in order to overcome the fact that features from the same modality are being fused in the feature level [7].
The motivation of this research is to replace the feature-level fusion of different modalities with the same modality. In this paper, the left and right palmprint of the same person is used to form a multimodal palmprint recognition system. This creates a high-dimensional feature size of redundant values. To eliminate this problem, feature selection algorithms such as GA and PSO are used. These optimization algorithms select the optimal features that produce the optimal recognition rate.
The rest of the paper is structured as follows: a review of the recent works on multimodal biometrics along with optimization techniques can be found in Section 2. Section 3 presents the methodology of the proposed system and feature extraction using MMLOF. Section 4 describes the proposed multimodal system architecture of GA and PSO. In Section 5, the experimental results for the IITD palmprint dataset are discussed and the unimodal and multimodal palmprint biometric system with optimization algorithms such as GA and PSO are compared. Section 6 presents the conclusions.
2. Related Work
Cui and Yang proposed score-level fusion by combining the fingerprint and finger vein [8]. Krishneswari and Arumugam prescribed a multimodal method by combining palmprint and fingerprint traits [9]. Bokade and Sapkal reviewed and discussed the feature-level fusion of palmprint and face traits [10]. Bhagat et al. developed a multimodal biometric method by joining face and palm vein traits [11]. The feature-level fusion is introduced to fuse these traits; Abdolahi et al. prescribed a multimodal method by joining the iris and fingerprint. The fusion is performed by means of a decision level [12]. Mitul and Chaudhari joined two biometric traits such as fingerprint and palmprint traits. The features are extracted by using the Gabor filter. The feature-level fusion is used for joining both the fingerprint and palmprint traits [13]. Vaidhya and Sheetal conducted the experiments using two traits. Based on the entropy method, palmprint and palm vein are fused together [14].
Xu et al. have combined right and left palm images for precise personal classification. They proposed a new framework to perform multibiometrics by combining the right and left palm images. Right and left palm images produced three types of scores to execute the matching score-level fusion [15]. Still, a promising recognition rate is not attained. Velmurugan and Selvarajan et al. proposed a hybrid feature-level fusion as hand geometry and iris using linear scale authentication [16]. Ramadan and Elgallad combined palm and iris-extracted features using scale-invariant feature transform (SIFT) and texture-based descriptors. The nonhomogeneous features of different biometric modalities are combined to form a single feature vector. PSO is used as a feature selection technique to reduce the feature size. The feature-level fusion is recommended for all industries and firms to improve personal identification [17]. Shanmugasundaram et al. proposed a new hybrid improved bacterial swarm (HIBS) optimization algorithm for hand-based multimodal biometrics to reduce the equal error rate (ERR). The proposed algorithm is formed by incorporating particle swarm optimization (PSO) and bacterial foraging optimization (BFO). This algorithm mitigates weaknesses in premature and slow convergence [18].
Mistry et al. introduced a novel technique called mGA-embedded PSO for (feature selection) for facial emotion recognition. It integrates a new velocity updating strategy, a nonreplaceable memory, a subdimension-based in-depth local facial feature search, a small-population secondary swarm, and a cooperation of global exploration and local exploitation search mechanism to overcome the premature convergence problem of existing PSO [19]. Hu et al. proposed an improved shuffled frog leaping algorithm for the molecular diagnosis of disease. It introduces an absolute balance group strategy and chaos memory weight. It minimizes irrelevant features in high-dimensional data and increases accuracy [20].
Tran et al. introduced a technique called potential particle swarm optimization (PPSO) which minimizes the search space problem. It requires less than 5% of the actual feature size [21]. Huang et al. introduced a hybrid feature selection algorithm based on Relief-BSTA and binary state transition algorithm. It contains two phases: the wrapper and the filter phase [22].
Tran et al. proposed variable-length PSO for feature selection. It consists of smaller search space by having different ad shorter lengths. The features are arranged in descending order of their relevance to achieve optimal classification performance [23].
Wu et al. presented a new feature selection algorithm called hybrid improved quantum-behavior particle swarm optimization (HI-BQPSO). It combines improved quantum-behavior particle swarm optimization and a filtering method to reduce the dimensionality of the data [24].
Song et al. proposed variable-size cooperative coevolutionary PSO (VSCCPSO) for feature selection. Here, the scalability of PSO is improved by dividing the high-dimensional feature selection problem into a number of low-dimensional subproblems. The computational cost spent on segregating feature space is avoided by gathering relevant features in a similar subspace without repeatedly computing the correlation between features [25].
Song et al. proposed a novel three-phase hybrid FS algorithm based on PSO and correlation-guided clustering. This method is proposed to overcome the limitations such as the high computational cost and the “curse of dimensionality.” Feature clustering-based methods and filter methods are used initially to reduce the search space. Later, optimal feature subsets were found out by the evolutionary algorithm. Furthermore, a fast correlation-guided feature clustering strategy, an improved integer PSO, and a symmetric uncertainty-based feature deletion method are developed to develop the performance correspondingly [26].
From the aboverelated work, it is stated that a palmprint-based multimodal biometric system is applied in various real-world systems such as border security, immigration, financial transaction, and law enforcement. In related work, the use of stochastic-based feature selection has shown its ability to enhance the recognition rate compared to deterministic-based techniques. Still, the issues of execution time and high-dimensional features have not attained the optimal results in related works. In this article, PSO and GA algorithms are used as feature selection to select optimal features from the fused features of left and right palmprint images for attaining discriminant features and minimizing the execution time.
3. Proposed Multimodal Palmprint Recognition System
The multimodal palmprint recognition system has certain significant contributions.(i)The right and left palmprint images of the same individual are fused in feature-level fusion as shown in Figure 1.(ii)The complete proposed method is carried out on contactless palmprint datasets. The images which are present in the contactless palmprint dataset are more similar to the practical applications.(iii)The tetralobe and dilobe MOF pattern has been generated by differentiating the Gaussian filter along and directions with a different orientation value .
(a)
(b)
In the proposed work, MMLOF is used to extract the features from the palmprint images. It is obtained by taking the first-order and second-order derivatives of the Gaussian filter such as dilobe and tetralobe. The feature length of MMLOF is 4096 × 1. The features extracted from the left and right palms are fused together to form a single feature vector of size 8192 × 1. The size of a single feature vector is very high. To reduce the size of the feature vector, two different optimization methods are used. The optimization methods such as PSO and GA obtained the optimal feature size and achieved optimum accuracy than conventional unimodal methods. Figure 2 demonstrates the procedure of the proposed multimodal palmprint recognition system.
3.1. Feature Extraction Using MMLOF
The Gaussian curve is much more symmetric and decreases steadily about the mean. The weight assigned to the signal information decreases smoothly with respect to distance. Suppose, when the order is even, the curve will be symmetric around value zero. Similarly, when the order is odd, the curve will be antisymmetric around zero. MMLOF is made up of negative and positive lobes which are obtained in terms of scale and orientation values. Here, the first and second-order Gaussian derivatives are represented in equations (2) and (3) from the 0th order of the Gaussian basic equation (1). By varying the value of with the help of and , different filter banks are formed. These filter banks are mainly used to obtain the features from the left and right palmprint images.where, where
There are three significant rules to design modified multilobe ordinal filters.(i)While the MMLOF filter is not getting biased, the sum of coefficients of all positive and negative lobes present inside the filter is zero. For each filter, the excitatory lobe coefficient values are equal to the inhibitory lobe coefficient values.(ii)The Gaussian mask is used to design the filter. The coefficients of positive and negative lobes in the filter must be inversely proportional to their distance from the center of the lobe.(iii)The positive and negative lobes which are present in the multilobe ordinal filter are performed like a low-pass filter. The spatial information is obtained by means of low-pass filters that are present below the lobes.
The structure of the MMLOF filter is controlled by the value which is the standard deviation of the lobe. controls the filter orientation. It signifies the angle between the line passing between the center of the lobe and the horizontal axis. takes the value of 0° to 180°. In this work, the 0th order of the Gaussian filter is used to obtain the 1st order MMLOF filter and the 2nd order MMLOF filter. The microstructure of the palmprint images is obtained by means of the MMLOF filter method.
In this article, 25 different dilobe ordinal filters are formed to extract the features of palmprint images as shown in Figure 3. Likewise, 25 different trilobe ordinal filters are formed to extract the features of palmprint images as shown in Figure 4. For both trilobe and di-MMLOF filter, is set as 1.6, and and take the value from 0° to 180°. By using the trial and error method, the dilobe filter having the parameters of and is considered as an optimal filter for extracting the features from palmprint images as shown in Figure 3. Similarly, the trilobe filter having the parameters of and is considered as an optimal filter for extracting the features from palmprint images as shown in Figure 4.
The optimal filter is convoluted with the palmprint image. The palmprint image of size (64 × 64) is represented as , and the MMLOF filter of size (11 × 11) is represented as . The process of filtering is performed on the palmprint images covered by the neighbor pixels. The center of the MMLOF filter visits each pixel in . In common, the linear spatial filtering of a palmprint image of size with the MMLOF filter mask of size is given by the following expression [28]:
Here, and , where and are positive integers. is the convoluted output. The convoluted output image is of the same size like the input palmprint image. We consider an example, when of equation (5) and are varied from −5 to +5, the previous equation can be expanded such that all the mask pixels are multiplied with the corresponding image pixels, and finally, the sum of all product output values is stored in shown in equation (6). The center coefficient of the filter matches with the pixels at the location of the palmprint image.
The same procedure is repeated for all values of and in equation (5). The feature size of is 4096 × 1.
3.2. Feature-Level Fusion Using Left and Right Palmprint
The left palmprint and right palmprint are integrated at the feature extraction level. The feature vectors of the left palmprint and right palmprint images are combined by adding them consecutively. We let the left palmprint feature vector be and the right palmprint vector be . After the fusion, the resulting vector becomes .
4. Optimization Methods
The new feature vector has a very high dimension. It is not sure that higher dimension features are supposed to give a higher recognition rate. It consists of stronger and weaker features. To obtain the optimum recognition rate, optimal subset features are to be selected from the given set. At the same time, the high-dimensional feature set may increase the execution time. Therefore, the dimensionality reduction should be performed. In this article, we come up with GA and PSO optimization techniques, which are synergistically coupled with the feature extraction techniques such as 1st and 2nd order MMLOF. The optimal features are selected by GAs and PSO at the feature-level fusion. Both algorithms provide an optimum training feature set and provide an optimum recognition rate.
From Figure 5, a random binary string is generated with respect to the size of the feature set for both PSO and GA. The bit 1 representing the corresponding feature is selected and bit 0 representing the feature is deleted. Therefore, the subset consists of {a1, a3, …an, b1, b2, …bm − 1}, respectively. The number of features gets reduced when compared to the original feature set. Hence, the execution time is reduced for both GA and PSO algorithms. The advantages of GA and PSO methods are minimum execution time, minimizing immaterial or redundant information, increase of the fitness value, exploration, and exploitation. Exploration jumps into a new subregion in the subspace, but exploitation does not jump into a new subregion in the search space. The exploitation and explorations are contradictory to each other. In general view, exploitation is carried out by selection and exploration is carried out using evolutionary algorithms.
4.1. Genetic Algorithm (GA)
The GA method engages three steps to create new generations from the existing population, namely, selection, crossover, and mutation. Figure 6 represents the flow of the genetic algorithm (Algorithm1). Step 1: it is seen that population size is . The value of is the number of chromosomes and has to be initialized first. Each chromosome of the population is made up of binary strings. Step 2: a random binary string is generated with respect to the size of the feature set. The feature length of the left palmprint is (4096 × 1), and the feature length of the right palmprint is (4096 × 1). The total feature length after fusion is (8192 × 1). Bit 1 indicates that the feature is selected, and bit 0 indicates that the feature is deleted (Figure 5). Step 3: therefore, the subset consists of , respectively. The number of features gets reduced when compared to the original feature set. The feature size of the subset () is always less than the original feature set ie) . Step 4: the direct coding is used for converting the genotype to phenotype. In the direct coding method, both the genotype and phenotype are represented by the same value. Step 5: all chromosomes are evaluated by means of their fitness function. The output of the fitness function is the recognition rate. The NN classifier is used to calculate the recognition rate because it works quickly. Step 6: the best chromosomes are selected by the roulette wheel method. Then, crossover and mutation are followed to obtain the optimal fitness value in each iteration.
|
4.2. Particle Swarm Optimization (PSO)
Particle Swarm Optimization (PSO) is a stochastic optimization method. It implements the concept of social interaction to solve the problem. In Figure 7, the population size is represented as
The subset features are selected from these feature sets by using PSO. The subset features of the training and testing datasets are randomly selected by means of the initial position of the particles as shown in Figure 7. The subset features of MMLOF act as the particle in the search space. The particle (or) subset features adjust its movement according to its own moving experience of other particles. Every particle continues to track its coordinate in the problem space which is associated with the most excellent fitness (recognition rate) that has been attained so far by that particle. The fitness function or recognition rate is calculated by means of the NN classifier. The value of the present fitness function is compared with the personal best (, which is the previous individual optimum of each particle. If the current value is better than the existing value, then that value is replaced with the current position. The additional value tracked by the PSO is . It is the best value obtained so far for any particle in the neighborhood of that particle. The fundamental idea of PSO lies in accelerating every particle toward its and the locations, with random weighted acceleration at every step. The position and velocity of the particle have been updated in every step. It is signified in equations (8) and (9)as follows:where encloses the velocity of each particle, encloses the position of each particle, represents the best value of the fitness function for each particle, represents the global best fitness function, is the random numbers, and are the weight controlling factors. represents the inertia weight or the constriction coefficient (Algorithm 2).
|
5. Experimental Setup and Experimentations
The public IITD palm dataset is obtained by means of contactless palmprint images [19]. The IITD dataset images were captured in the indoor environment. It acquired contactless hand images with certain variations in translation, rotation, projection, and pose. There is a significant intraclass variation resulting due to the absence of contact between the palm and the sensor. It consists of 2760 hand images from 230 different subjects. Six hand images were captured from each left palm. Similarly, six hand images were captured from each right hand of every individual.
Table 1 shows the samples taken for training and testing. The IITD dataset is more robust and closer to the real application when compared to the Poly U dataset. The entire experiments were run in Processor (Intel) core (TM) i7-7700 CPU, RAM 16 GB, 3.60 GHz, 64-bit OS, and MATLAB R2017a. Table 1 represents the multimodal palmprint dataset.
From Table 2, it is seen that the number of iterations is 100 for both PSO and GA. Similarly, the size of population is set as 50 for both PSO and GA. For the reason of evaluating GA and PSO, the chromosome size used for the GA running in all test problems will be the same as the particle size of PSO [29]. The original PSO algorithm uses the values of and , respectively [30]. The other values of and are typically directed to slower convergence or sometimes nonconvergence. There is no suggestion in the literature concerning population size in PSO.
Most researchers exploit a population size of 10 to 50 [31]. The most popular standard for particle swarm optimization uses a constant population size of = 50 for different dimensions of = 2, 4, and 30. Bratton and Kennedy state that no population size between 20 and 100 particles produced results that were evidently inferior or superior to any other value for most of the tested problems. There is no guideline for selecting the population size [32].
The proportionate method (roulette wheel selection) is used to select the chromosomes in this work. Mutation and crossover operators are used to introduce diversity in the population. In the uniform crossover, every bit from the offspring’s individual is independently selected from the two parents with respect to a given distribution. The uniform crossover shares individual bits alone. It does not share the segments of the bit array. Due to this nature, it eliminates the positional bias of inheritance. The probability of crossover is set as 0.7. In early generations, larger mutation steps can be good for exploring chromosomes in the search space. Similarly, smaller mutation values might be needed in the final generations for better convergence. The mutation rate is set as the minimum value for the work.
6. Results and Discussion
From Figure 8, it is inferred that the recognition rate attained for the left palmprint alone is 93.47% using the 1st order MMLOF. Similarly, the recognition rate attained for the right palmprint alone is 90.86% using the 1st order MMLOF. When the left palmprint and the right palmprint are fused together by means of feature-level fusion to form a multimodal recognition system, this multimodal recognition system yields a recognition rate of 96.05%. The 1storder MMLOF + PSO for left palmprint provided a recognition rate of 94.21%. Similarly, the 1st order MMLOF + PSO for the right palmprint provides a recognition rate of 91.01%. After fusion, the features of the left and right palmprint of the 1st order MMLOF + PSO provide a recognition rate of 96.52%. The 1st order MMLOF + GA for left palmprint provides a recognition rate of 95.23%. Similarly, the 1st order MMLOF + GA for the right palmprint provides a recognition rate of 91.30%. After fusion, the features of the left and right palmprint of the 1st order MMLOF + GA provides an optimal recognition rate of 96.95%. The 1st order MMLOF provides better results when compared to the 2nd order MMLOF.
The significant difference between the 1st order MMLOF + GA with other algorithms is investigated by using the T-Test. In Figure 8, “+” shows that the 1st order MMLOF + GA is significantly superior to its counterpart, and “=” denotes that there is no considerable distinction between both algorithms. Furthermore, the recognition rates of the 1st order MMLOF + PSO (green) and 1st order MMLOF + GA (green) are significantly greater than all the unimodal techniques (shown in red and blue color). There is no significant difference between the 1st order MMLOF (green), 1st order MMLOF + GA (green), 1st order MMLOF + GA (blue), and 1st order MMLOF + PSO (blue). However, the 1st order MMLOF + GA (green) is relatively better than the remaining algorithms.
The reason is that the 1st order MMLOF is obtained by differentiating the Gaussian filter. The Gaussian filter is mainly used to smoothen the noise present in the image. At the same time, the 1st derivative of the Gaussian filter is used to find the stronger edges present in the image. The 2nd order MMLOF is obtained by differentiating the 1st derivative of the Gaussian filter. The second-order edge detected is used to find the zero crossing present in the palm images. The higher-order differential masks act as high-pass filters. It tends to amplify noise. Therefore, it does not perform well when compared to lower-order differential masks. The 1st order MMLOF + GA of left and right palmprints provides the optimal recognition rate with minimum feature size compared to the 1st order MMLOF + PSO of left and right palmprints.
From Figure 9, it is inferred that the multimodal recognition system attains a maximum feature size of 8192 1. After fusion, the features of the left and right palmprint of the 1st order MMLOF + PSO provide an optimal feature size of 7212 1. Therefore, the feature size gets reduced by using PSO for both unimodal and multimodal palmprint recognition systems.
After fusion, the features of left and right palmprints of the 1st order MMLOF + GA provide an optimal feature size of 7116 1. Therefore, the feature size gets reduced by using GA for both unimodal and multimodal palmprint recognition systems.
From Figure 10, it is inferred that the execution time of 1st order MMLOF multimodal palmprint recognition is high compared to unimodal palmprint recognition. It is seen that the feature size is reduced by using GA and PSO. Therefore, the execution time is also reduced for GA and PSO. The execution time of 1st order MMLOF multimodal palmprint recognition with optimization techniques such as GA and PSO is low compared to the execution time of 1st order MMLOF multimodal palmprint recognition without optimization. GA takes less execution time compared to the PSO of 1st order MMLOF multimodal palmprint recognition.
From Figure 11, it is seen that the 1st order MMLOF + GA (multimodal) converges to global maxima at the 40th iteration and attains the recognition of 96.95%. Similarly, the 1st order MMLOF + PSO (Multimodal) converges to the global maxima at the 42nd iteration and attains the recognition rate of 96.52%. Hence, the 1st order MMLOF + GA converges quickly when compared to PSO for the IITD palmprint dataset. It also yields the optimum recognition rate.
From Table 3, it is seen that the proposed methods are compared with the existing methods such as SIFT, LDA, and CRC [15]. The proposed methods outperform all existing methods. The optimal RER is obtained by the 1st order MMLOF + GA. Hence, the proposed algorithm outperforms the state-of-the-art methods.
From Figure 12, it is inferred that the 1st order MMLOF + GA outperforms all existing methods such as RLOC, SIFT + OLOF, SMCC, and CR-Comcode [15]. Similarly, it also outperforms existing methods such as PalmCode, OLOF, and SIFT + Aligned Comcode [33]. The execution time of the 1st order MMLOF + GA is reduced by minimizing the feature size.
7. Conclusions
In this article, modified multilobe patterns are used to increase the visibility of edges and other details present in the palmprint images. It also removes high-frequency details that often include noise present in palmprint images. Since palmprint images are highly affected by noise, for these reasons, the modified multilobe pattern is used as the preprocessing filter. The feature selection process is carried out by GA and PSO techniques. In this work, IITD left palm and right palm images are used. The 1st order MMLOF outperforms the 2nd order MMLOF for both the unimodal and multimodal palmprint recognition systems. The 1st order MMLOF is used to find the stronger edges present in the image. The higher-order differential masks behave as high-pass filters. It tends to amplify noise. The redundant features of both unimodal and multimodal palmprint recognition systems are eliminated by using PSO and GA. The 1st order MMLOF + GA (multimodal) converges faster and outperforms the 1st order MMLOF + PSO (multimodal). The 1st order MMLOF + GA (multimodal) of left and right palmprint images obtains an optimal recognition rate of 96.95%. Hence, the multimodal palmprint recognition system along with GA outperforms the unimodal palmprint recognition system. The limitation of this work is considering the minimum dataset. Further, this will be carried out by considering more number of subjects in the dataset to evaluate the robustness of the proposed algorithm. In the future, this work will be intended to apply a hybrid optimization approach to increase convergence speed. This investigation is helpful in motivating the researchers to look at a possible relation between the traits of the other multimodal biometric issues.
Data Availability
The data are available at https://www4.comp.polyu.edu.hk/∼csajaykr/IITD/Database_Palm.htm.
Conflicts of Interest
The authors declare that they have no conflicts of interest.