Abstract
Finger vein recognition is a promising biometric technology that has received significant research attention. However, most of the existing works often relied on a single feature, which failed to fully exploit the discriminative information in finger vein images, and therefore led to a limited recognition performance. To overcome this limitation, this paper proposes an encoding coefficient similarity-based multifeature sparse representation method for finger vein recognition. The proposed method not only uses multiple features to extract comprehensive information from finger vein images, but also obtains more discriminative information through constraints in the objective function. The sparsity constraint retains the key information of each feature, and the similarity constraint explores the shared information among the features. Furthermore, the proposed method is capable of fusing all kinds of features, not limited to specific ones. The optimization problem of the proposed method is efficiently solved using the alternating direction multiplier method algorithm. Experimental results on two public finger vein databases HKPU-FV and SDU-FV show that the proposed method achieves good recognition performance.
1. Introduction
Finger vein recognition is a promising biometric technology that captures the unique pattern of blood vessels beneath the skin surface using near-infrared light in a noninvasive way. Unlike other biometric technologies, finger vein patterns are difficult to replicate or forge, which makes finger vein recognition reliable and secure [1, 2]. Recently, finger vein recognition has drawn great attention from researchers.
Researchers have developed numerous methods to enhance the performance of finger vein recognition [3]. However, many existing methods relied on a single feature, which failed to fully capture discriminative information in finger vein images, thus limiting their recognition performance. To address these limitations, researchers have proposed multifeature methods [4–7]. These methods utilized multiple features to represent images and provided more comprehensive information for recognition. However, traditional multifeature methods paid little attention to the similarity between features. In reality, when multiple features describe one image, they inherently share consistent or similar information, which has significant discriminative capability. Furthermore, this shared similarity information can enhance the robustness of recognition. In cases where a feature is disturbed by noises, the similarity information can reduce the negative effect of noises, which brings the robustness of the recognition method.
In order to make full use of multiple features, this paper proposes an encoding coefficient similarity-based multifeature sparse representation (SMSR) method for finger vein recognition. The proposed method introduces an objective function consisting of three components: the reconstruction error, the sparsity constraint on each feature, and the similarity constraint between features. The reconstruction error measures the error between the original data and the reconstruction based on the encoding coefficient and dictionary. The sparsity constraint causes most encoding coefficients to be zeros, while the nonzero coefficients represent the key information of each feature. This helps to reduce the redundancy of feature representation. The similarity constraint encourages the similarity between the encoding coefficients of different features, which effectively captures the common information among features. In the matching stage, the reconstruction errors of multiple features are weighted and fused to further improve the recognition performance.
The SMSR method provides several advantages. First, it considers multiple features simultaneously, to enrich the feature representation. Second, the method obtains more discriminative information for recognition by employing sparsity constraint and similarity constraint. Finally, the SMSR method exhibits strong applicability since it is not limited to specific features.
The paper is organized as follows: Section 2 presents an overview of related work. Section 3 introduces the proposed method. Section 4 reports the experimental results, and Section 5 provides the conclusions.
2. Related Works
2.1. Single-Feature Finger Vein Recognition
In single-feature finger vein recognition, a single feature is extracted to represent an image. Frequently used single-feature extraction methods include the vein pattern method, local descriptor method, deep learning method, and sparse representation method.(1)Vein pattern method: the vein pattern method is widely used in finger vein recognition, which captures and utilizes vein patterns for recognition. There are several commonly used vein pattern extraction methods, that is, mean curvature (MeanC) [8], anatomy structure analysis-based vein extraction [9], and Gabor [10].(2)Local descriptor method: the local descriptor method focuses on capturing the texture of local regions of finger vein image. This kind of method utilizes various popular local descriptors, including local binary pattern (LBP) [11], multiscale uniform local binary pattern [12], local line binary pattern [13], and scale-invariant feature transform (SIFT) [14].(3)Deep learning method: in recent years, deep learning techniques have been widely used to extract finger vein features. In one research, a convolutional neural network [15] was employed to perform feature extraction. Another research introduced a pretrained Xception architecture [16] for both feature extraction and classification. In addition, a convolutional autoencoder [17] was utilized to learn features from finger vein images.(4)Sparse representation method: there are several finger vein recognition methods based on sparse representation. These methods include nearest centroid neighbor-based sparse representation classification [18], mutual sparse representation classification [19], and sparse reconstruction error constrained low-rank representation (SRLRR) [20]. In particular, SRLRR achieved excellent recognition performance.
2.2. Multifeature Finger Vein Recognition
In multifeature methods, multiple features are extracted to enhance recognition performance. Several researches have explored the use of multiple-feature methods for finger vein recognition. For instance, one method integrated the enhanced maximum curvature method with a histogram of oriented gradient (HOG) descriptor for finger vein recognition [4]. Another method focused on finger vein recognition using local phase quantization and local derivative pattern [5]. In addition, one method employed SIFT and speeded-up robust features as features [7].
Furthermore, there are many multimodal recognition methods that improve performance by fusing features from different modalities [2]. These include fusion of finger vein and fingerprint features [21], fusion of finger vein and knuckle pattern features [22], fusion of finger vein and face features [23], and fusion of finger vein and electrocardiogram features [24].
3. The Proposed Method
We propose an encoding coefficient SMSR method to obtain more discriminative information from finger vein images. Figure 1 illustrates the flowchart of the proposed method. Subsequently, we will provide the detailed descriptions of the proposed method.
3.1. Feature Extraction
In this step, various features are extracted from the finger vein image. Specifically, this study employs two features, that is, Grayvalue and LBP. The Grayvalue utilizes the gray values of an image as a feature. By collecting the gray values of all pixels, a Grayvalue feature vector can be obtained, to represent the gray information of the image. In addition, LBP is used to capture the texture information of an image. The computation of the LBP feature is based on comparing the gray value of each pixel with its neighboring pixels. The binary codes are generated based on the comparison results. These binary codes are then converted into decimal values, which are used as the second feature. By utilizing both the Grayvalue feature and the LBP feature, the proposed method obtains the gray and texture information of the finger vein image.
3.2. Encoding Coefficient Similarity-Based Multifeature Sparse Representation
The encoding coefficient SMSR defines an objective function that integrates the sparsity constraint for each feature and the similarity constraint between features. The defined objective function is given as follows:In this formula, represents the number of features. denotes the feature matrix of the testing images, where is the feature index. Each column of corresponds to a feature vector of a testing image. represents the encoding coefficient matrix, and denotes the dictionary consisting of training image features. denotes the squared Frobenius norm of a matrix, and represents the L1 norm.
The objective function consists of three components:(1)Reconstruction error: measures the difference between the original data and its reconstruction using the corresponding dictionary and encoding coefficient matrix . By minimizing the reconstruction error, the objective is to obtain a sparse encoding coefficient matrix for reconstructing the original data.(2)Sparsity constraint: promotes the sparsity of the coefficient matrix . By minimizing the sparsity constraint, the objective function encourages most of the elements in the coefficient matrix to be zero, thus selecting the most important information.(3)Similarity constraint: encourages the similarity between coefficient matrices and . By minimizing the similarity constraint, the coefficients between different features tend to be similar, and the shared information between different features of an image is explored.
The optimization problem aims to find the coefficient matrix that minimizes the reconstruction error and keeps the sparsity and similarity constraints.
3.3. Optimization Process
The alternating direction method of multipliers (ADMM) [25, 26] is a powerful optimization technique that has gained significant attention due to its ability to decompose complex optimization problems into simpler subproblems. Therefore, we use ADMM to solve the objective function Equation (1).
By introducing the auxiliary variable , the objective function can be transformed into:The augmented Lagrange function is obtained by introducing the Lagrange multiplier matrix :where represents the inner product of two matrices and is a penalty factor. The function of Equation (3) can be solved using an iterative update method. When a variable is updated, other variables are fixed. The update steps are given as follows:(1)Update : fix other variables, can be updated by solving the following problems:By taking the derivative of and setting it to zero, we obtainwhere is the identity matrix.(2)Update : fix other variables, can be updated by solving the following problems:By taking the derivative of and setting it to zero, the following formula can be obtained.where is defined as follows:Using the soft threshold method, the update formula of is obtained.(3)Update and : the Lagrange multiplier and are updated.in which the parameter is used to control the update speed of the penalty parameter . We set to 1.6, and the maximum value of is .
In each iteration, we compute the difference between and as . The norms of are then calculated and compared to the threshold value. If the maximum norm of is less than the threshold or the iteration reaches the maximum value, the algorithm terminates. Otherwise, the value of is updated, and the algorithm continues with the next iteration.
The optimization process is given in Algorithm 1.
| ||||||||||||||||||||
3.4. Classification
In this section, we present the classification of a testing image. For the feature matrix of all the testing images, the coefficient matrix is obtained by solving function Equation (1), which represents the linear combination of using the dictionary , that is, . Specifically, for a testing image , its feature vector is denoted as , and the coefficient vector is denoted as . The reconstruction error of on can be calculated:
Subsequently, the reconstruction errors of different features are fused, denoted by .where is the weight, which represents the contribution of the feature to the final reconstruction error .
Finally, the testing image is assigned to the category with the smallest reconstruction error, so is assigned to the category labeled according to the following formula:
4. Results and Discussion
4.1. Experiment Settings
In order to ensure the reliability of the experimental results, we conduct experiments on two public finger vein databases: HKPU-FV database from Hong Kong Polytechnic University [27] and SDU-FV database from Shandong University [28]. Some finger vein images from two databases are shown in Figure 2.
(a)
(b)
(1)HKPU-FV database contains finger vein images from 156 volunteers. The acquisition process consists of two stages. In each stage, the volunteers provided two fingers for image acquisition, and six images were obtained for each finger. Since only 105 volunteers participated in the second stage, the number of finger vein images per person in the database varies. Therefore, we only use the finger vein images collected in the first stage, totaling 1,872 () finger vein images from 312 fingers.(2)SDU-FV database involves 106 volunteers. Each volunteer provided finger vein images from six fingers, and six images were obtained for each finger. Therefore, the database has a total of 3,816 () finger vein images from 636 fingers.
For the experiments, the first half images of each finger are used as training data, and the remaining images are used as testing data. Therefore, HKPU-FV database and SDU-FV database have 936 () training images and 1,908 () training images.
To reduce interference from the image background, features are extracted from the region of interest (ROI) image. The ROI is obtained using the method in Yang et al. [29]. Initially, the Sobel operator is used to detect the finger boundaries, and then the midpoints of the boundaries are identified. These midpoints are utilized to correct the skew angle in the image. Next, a sliding window technique is employed to determine the height of the ROI, while the width is determined by using the internal tangents of the finger edges. Once both the height and width of the ROI are determined, the ROI of the finger vein image is obtained. For the convenience of processing, the size of the ROI is normalized into 96 × 64 pixels using bilinear interpolation.
In our experiments, we test the performance of our method in both identification and verification modes. In the identification mode, we use the recognition rate as a benchmark, which measures the rate of correctly recognized images overall testing images. In the verification mode, we utilize the equal error rate (EER) as a performance evaluation metric.
4.2. Comparison with Single-Feature Sparse Representation
This experiment aims to compare the proposed SMSR method with the single-feature sparse representation (SFSR) method. The experiment utilizes Grayvalue, Gabor, LBP, and HOG features, and the recognition results are presented in Table 1. These results clearly demonstrate that, compared to the SFSR method, the SMSR method achieves significantly higher recognition rates and lower EERs on two databases. For example, on HKPU-FV database, the recognition rates for Grayscale and Gabor based on SFSR are 91.67% and 91.88%, and EER values are 2.24% and 3.11%, respectively. When our Grayscale and Gabor-based SMSR method is used, the recognition rate increases to 98.08% and the EER reduces to 0.73%. Similar trends can be observed from other features. The SMSR method outperforms the SFSR method in recognition performance, which is attributed to a more comprehensive representation of finger vein images by multiple features. In addition, this experiment shows that SMSR methods can fuse different features, rather than specific ones.
Furthermore, on HKPU-FV database, the SMSR method, utilizing the combination of Grayvalue and LBP features, achieves the highest recognition rate of 99.89% with a lower EER of 0.12%. Similarly, on SDU-FV database, the same combination also yields the highest recognition rate of 93.87%, and the lowest EER of 2.61%. Considering the outstanding performance of the combination of Grayvalue and LBP features on two databases, subsequent experiments will utilize the Grayscale and Gabor-based SMSR method.
4.3. Effectiveness of Similarity Constraint
This experiment is conducted to evaluate the impact of the similarity constraint on the objective function of the SMSR method. To assess the contribution of this constraint, we remove it from the objective function to obtain the nonconsistent multifeature sparse representation (NC-MFSR) method. The objective function of the NC-MFSR method is given as follows:
The experimental results are presented in Table 2, which clearly demonstrates the performance difference between the SMSR and NC-MFSR methods. On HKPU-FV database, the SMSR method achieves a recognition rate of 99.89% and an EER of 0.12%, outperforming the NC-MFSR method, which has a lower recognition rate of 98.93% and a higher EER of 0.56%. Similarly, on SDU-FV database, the SMSR method achieves a recognition rate of 93.87% and an EER of 2.61%, which also surpasses the NC-MFSR method. These results show the effectiveness of the similarity constraint in the SMSR method. The similarity constraint in the SMSR method captures the shared information between different features, which enhances the discriminative ability of the method. Furthermore, when a feature is affected by noise, the similarity constraint is able to compensate for the affected feature by exploiting the stable information. This mutual compensation can also improve the overall recognition performance.
4.4. Comparison of Different Fusion Methods
The proposed SMSR method is compared with several commonly used multifeature fusion methods [30] to demonstrate its superior ability. The compared methods are also based on sparse representation. Detailed descriptions of the compared multifeature fusion methods are given as follows:(1)Parallel feature fusion: this method requires that the feature vectors of an image have the same dimensionality. Therefore, PCA is applied to reduce the dimensionality of two feature vectors, and the two vectors are weighted and fused to obtain a combined feature vector. Sparse representation is subsequently employed to obtain encoding coefficients, and the reconstruction error is used for classification.(2)Serial feature fusion: in this method, the two feature vectors of an image are concatenated to form a larger feature vector. Similarly, sparse representation is used to obtain encoding coefficients, and the reconstruction error is utilized for classification.(3)Score fusion: this method uses a SFSR method to obtain the encoding coefficients of each feature. Each feature is individually reconstructed, and the reconstruction errors of two features are weighted and fused for classification.
The results presented in Table 3 clearly demonstrate that the recognition performance of the SMSR method is superior to other multifeature fusion methods. Among these compared methods, the score fusion method achieves better recognition performance. However, it still cannot catch up with the SMSR method, as it fails to fully use the discriminative information from multiple features.
The objective function of the SMSR method simultaneously considers the sparsity constraint of each feature and the similarity constraint between features. In this way, it not only fully exploits key information from each feature but also extracts common information between features. The obtained information is jointly employed for recognition, enhancing the overall discriminative capability of the method. Therefore, for the multifeature fusion task, the SMSR method can obtain more discriminative information from multiple features and achieves good recognition performance.
4.5. Parameter Discussion
This experiment involves several parameters: the sparsity constraint parameter , the similarity constraint parameter , and the weight parameter .
The parameter controls the degree of sparsity in the representation. By setting to a larger value, the optimization process is more inclined to generate a sparser encoding coefficient matrix . The parameter determines the strength of the similarity constraint. A higher value of promotes greater similarity between coefficient matrices and . The weight parameter is used to fuse the reconstruction errors. As two features are used, and , the experiment only needs to focus on the value of .
Initially, we set to 0.001 and to 0.01 in order to adjust the parameter . According to the results in Table 4, yields the best recognition rate on HKPU-FV database, and achieves the highest recognition rate on SDU-FV database. Therefore, in the experiments of adjusting and , we set on HKPU-FV database, and on SDU-FV database.
Next, is fixed at 0.001, and the value of is adjusted. Based on the results shown in Table 5, the highest recognition rate is achieved when is set to 0.1 on both HKPU-FV and SDU-FV databases.
Finally, with fixed at 0.1, the value of is varied. The results in Table 6 indicate that the optimal recognition rate on HKPU-FV database is achieved with , and the highest recognition rate on SDU-FV database is obtained with . Consequently, on HKPU-FV database, we use and , and on SDU-FV database, we use and .
In summary, on HKPU-FV database, the highest recognition rate is achieved with , , and . And on SDU-FV database, the highest recognition rate is achieved with , , and .
4.6. Comparison with State-of-the-Art Methods
The purpose of this experiment is to compare the performance of the SMSR method with the state-of-the-art finger vein recognition methods on two databases. The results are summarized in Table 7.
The experimental results indicate that the SMSR method achieves better recognition performance on both the HKPU-FV and SDU-FV databases. On HKPU-FV database, it achieves the highest recognition rate of 99.89% and the lowest EER of 0.12%. Similarly, on SDU-FV database, it achieves a recognition rate of 93.87% with an EER of 2.61%. By contrast, the classical methods such as ASAVE, weighted vein code indexing, and SRLRR exhibit relatively good performance but do not catch up with the SMSR method. The promising performance of our SMSR is mainly attributed to its effectiveness in extraction of discriminative information.
4.7. Time Performance
This experiment aims to assess the time cost of the SMSR method on HKPU-FV database. It runs on a PC equipped with a 3.80 GHz processor and 32.00 GB of memory, using MATLAB 2018 software.
Table 8 presents the time spent by the SMSR method to classify 936 testing images. The SMSR classification involves both offline and online steps. The most time-consuming step is the reconstruction error calculation, which takes approximately 11.47 s, and the classification step only requires 0.03 s. In total, the SMSR method takes about 18.22 s for online recognition of 936 testing images, averaging about 0.02 s per image. This demonstrates the high efficiency of the proposed SMSR method.
In addition, Table 9 presents the time costs of our proposed SMSR method and two classical methods, that is, LDC [32] and MeanC [8]. In Table 9, for the SMSR method, the feature extraction step includes all online steps except classification. The SMSR method spends more time in the feature extraction step, approximately 18.19 s. This is mainly due to the cost of the sparse representation and reconstruction error calculation of all testing images. In the classification step, LDC and MeanC methods spend more time, that is, 43.10 and 80.39 s, respectively. This is because they need to match each testing image with all 936 registered images. In summary, the SMSR method is superior to the LDC and MeanC methods in this time cost test.
5. Conclusion
In this paper, we propose an encoding coefficient SMSR method for finger vein recognition. This method uses multiple features to comprehensively represent finger vein images. To extract more discriminative information, we introduce the sparsity and similarity constraints into the objective function. The sparsity constraint retains the unique information of each feature, and the similarity constraint explores the common information among the features. In the matching stage, we weight and fuse the reconstruction errors of multiple features to further enhance recognition performance. Our experimental results on two public finger vein datasets, HKPU-FV and SDU-FV, demonstrate the advantages of our proposed method in recognition performance and efficiency.
Data Availability
The data used to support the findings of this study are included within the article.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Acknowledgments
This work was supported in part by the National Natural Science Foundation of China (grant number 62076151) and the Shandong Provincial Natural Science Foundation (grant number ZR2022MF331), in part by the Taishan Scholar Project of Shandong Province (grant number tsqn202211182) and Youth Innovation Team of Shandong Province Higher Education Institutions (grant number 2022KJ205).