Abstract
A novel method for finger vein pattern extraction from infrared images is presented. This method involves four steps: preprocessing which performs local normalization of the image intensity, image enhancement, image segmentation, and finally postprocessing for image cleaning. In the image enhancement step, an image which will be both smooth and similar to the original is sought. The enhanced image is obtained by minimizing the objective function of a modified separable Mumford Shah Model. Since, this minimization procedure is computationally intensive for large images, a local application of the Mumford Shah Model in small window neighborhoods is proposed. The finger veins are located in concave nonsmooth regions and, so, in order to distinct them from the other tissue parts, all the differences between the smooth neighborhoods, obtained by the local application of the model, and the corresponding windows of the original image are added. After that, veins in the enhanced image have been sufficiently emphasized. Thus, after image enhancement, an accurate segmentation can be obtained readily by a local entropy thresholding method. Finally, the resulted binary image may suffer from some misclassifications and, so, a postprocessing step is performed in order to extract a robust finger vein pattern.
1. Introduction
The problem of finger vein extraction from infrared images arises mainly for biometrics purposes but it is also very important for the biomedical research community.
The general structure of a biometric system based on finger veins consists of five main stages: (1) acquisition of the infrared images exploiting the absorption of light in near infrared and infrared wavelengths by the different human tissues, (2) preprocessing of the acquired images which includes ROI (region of interest) extraction, image intensity normalization (in this type of images intensity is usually uneven due to the image acquisition system and may suffer from shading artefacts), and noise reduction, (3) segmentation or classification stage in which the preprocessed image divided into two (or more depending on the application) regions associated with veins and surrounding tissues, (4) postprocessing of the binary images which delivers the final segmentation result, free of outliers and misclassifications, and finally (5) matching of the extracted veins in order to perform the desired person identification/verification procedure. Matching procedure can be applied either directly in the extracted finger vein patterns or in their skeletons, depending on the matching algorithm that has to be used. This general structure described so far involves all the stages that may have such a system but it is worth mentioning that these stages are independent and some of them can be skipped in some applications depending on its specific requirements.
Related Work. Several methods which adopt this general architecture have already been presented starting from the pioneering work of Park et al. [1]. In this important research work, an application specific processor for vein pattern extraction and its application to a biometric identification system is proposed. The conventional vein pattern recognition algorithm [1–3] consists of a preprocessing part, applying sequentially an iterative Gaussian low pass, a high pass, and a modified median filter, a recognition part which includes the extraction of the binary veins via local thresholding, and finally the matching between the individual patterns.
An improved vein pattern extraction algorithm is proposed in [4], which compensates the loss of vein patterns in the edge area, gives more enhanced and stabilized vein pattern information, and shows better performance than the existing algorithms. Also, the problem arising from the iterative nature of filtering preprocess is solved by designing a filter that is processed only once, increasing significantly the recognition speed and reducing the hardware complexity. The proposed algorithm is implemented with an FPGA device and the false acceptance rate is five times better than the existing algorithm and the recognition speed is measured to be 100 (ms/person).
The problem with conventional hand vascular technology mentioned above is that the vascular pattern is extracted without taking into account its direction. So, there is a loss of vascular connectivity which leads to a degradation of the performance of the verification procedure. An attempt to improve this problem can be found in [5], where a direction based vascular pattern extraction algorithm based on the directional information of vascular patterns is presented for biometric applications. It applies two different filters: row vascular pattern extraction filter for abscissa vascular pattern extraction and column vascular pattern extraction filter for effective extraction of the ordinate vascular patterns. The combined output of both filters produces the final hand vascular patterns. Unlike the conventional hand vascular pattern extraction algorithm, the directional extraction approach prevents loss of the vascular pattern connectivity.
Although the above algorithm considers the directionality of veins, it also assumes that the veins oriented in only two principal directions. In [6, 7] a method for personal identification based on finger vein patterns is presented and evaluated using line tracking starting from various positions. This method allows vein patterns to have an arbitrary direction. Local dark lines are identified and line tracking is executed by moving along the lines pixel by pixel. When a dark line is not detectable, a new tracking operation starts at another position. This procedure executes repeatedly, so the dark lines that tracked multiple times are classified as veins.
Typically, the infrared images of finger veins are low contrast images, due to the light scattering effect. An algorithm for finger vein pattern extraction in infrared images is proposed in [8]. This algorithm embeds all the above issues and proposes novel preprocessing and postprocessing algorithms. Initially, the image is enhanced and the fingerprint lines are removed using 2D discrete wavelet filtering. Kernel filtering produces multiple images by rotating the kernel in six different directions, focus on the expected directions of the vein patterns. The maximum of all images is transformed into a binary image. Further improvement is achieved by a two-level morphological process; that is, a majority filter smoothes the contours and removes some of the misclassified isolated pixels, and a reconstruction procedure removes the remaining misclassified regions. The final image is segmented into two regions, the vein and the tissue.
In [9] new issues are considered and a certification system that compares vein images for low cost, high speed, and high precision certification is proposed. The equipment for authentication consists of a near infrared light source and a monochrome CCD to produce contrast enhanced images of the subcutaneous veins. The phase correlation and template matching methods are used for classification. Several noise reduction filters, sharpness filters, and histogram manipulations tested for the best effort. As a result, a high certification ratio in this system obtained.
In [10], the theoretical foundation and difficulties of hand vein recognition are introduced at first. Then, the optimum threshold of the segmentation process and the vein lines thinning problem of infrared hand images are deeply studied, followed by the presentation of a novel estimator for the segmentation threshold and an improved conditional thinning method. The method of hand vein image feature extraction based on end points and crossing points is studied initially, and the matching method based on a distance measure is used to match vein images. The matching experiments indicated that this method is efficient in terms of biometric verification.
An efficient automatic method for robust segmentation of finger vessel network and vein pattern extraction from infrared images acquired by a low-cost monochrome or multichannel camera is proposed in [11]. After brightness normalization, the fingerprint lines are eliminated using the 2D dimensional discrete wavelet transformation. A set of twelve directional kernels is constructed, based on a dyadic wavelet transform, for each scale, and is used to enhance the directional properties of veins. From maximum filters’ response along scale and direction, a neighbourhood thresholding derives a binary segmented image to produce reliable patterns of finger veins. A postprocessing module is used in case where low quality images are to be segmented. Preliminary evaluation experiments of the proposed method demonstrate a number of advantages, compared to recently published methods.
In the narrow bandwidth of the near infrared spectrum, the light is propagated through human tissue with low absorption rates, but strong scattering effects produce extremely low contrast images. In [12], an algorithm for finger vein segmentation and centerlines extraction in infrared images is presented. Finger veins are detected in pixels with positive divergence of the gradient vector field, while centerlines are extracted in pixels with positive divergence of the normalized gradient vector field estimated at various orientations. The segmentation algorithm has been evaluated on both artificial and real finger infrared images and high segmentation rates are achieved in terms of sensitivity, specificity, and accuracy using manual annotation data obtained by human observers.
A new algorithm for vein matching based on log-polar transform to address problems that occur with the changing of finger position and from differences between imaging devices for current vein matching algorithms is discussed in [13]. The new algorithm first extracts the feature area, which contains enough characteristics for image matching, depending on the structure of the finger vein ridge alignment. It then calculates the degree of similarity between the log-polar transform results of the model image feature areas and the sample image and finally analyzes the result by the degree of similarity and the relationship of relative positions between feature areas. Experiments show that the algorithm is robust for rotating and zooming images of the finger vein.
In [14], four principles (caliber uniformity, node replication, loop splitting, and virtual connection) are proposed, first to simplify the finger vein structure as a binary tree structure. Then a modified binary tree model is proposed based on the binary tree structure. The new model uses the binary tree to describe the relationships between different vein branches and uses a B-spline function to describe the spatial structure of vein branches. Experiments show that this model can quantitatively describe the relationships between, and the spatial structure of, vein branches with little representation error and low storage space requirements.
The method proposed in [15] is rooted in a local binary pattern based method and then inclined to use the best bits only for matching. After presenting the concept of PBBM and the generating algorithm, authors propose the finger vein recognition framework, which consists of preprocessing, feature extraction, and matching. Experimental results show that PBBM achieves not only better performance but also high robustness and reliability. In addition, PBBM can be used as a general framework for binary pattern based recognition.
In [16], a new and robust approach for finger vein ROI localization is introduced, and then a new scheme for effectively improving the visibility of finger vein imageries is proposed. Extensive experiments are conducted to validate this method.
A new method of personal identification based on finger vein recognition is presented in [17]. First, a stable region representing finger vein network is cropped from the image plane of an imaging sensor. A bank of Gabor filters is then used to exploit the finger vein characteristics at different orientations and scales. Based on the filtered image, both local and global finger vein features are extracted to construct a finger vein code (FVCode). Finally, finger vein recognition is implemented using the cosine similarity measure classifier, and a fusion scheme in decision level is adopted to improve the reliability of identification. Experimental results show that this method exhibit an exciting performance in personal identification.
Finally in [18], a finger vein system using the mean curvature that can be used for personal verification is proposed. As a robust extraction method, authors propose the mean curvature method, which views the vein image as a geometric shape and finds the valley-like structures with negative mean curvatures. When the matched pixel ratio is used in matching vein patterns, experimental results show that, while maintaining low complexity, the proposed method achieves 0.25% equal error rate, which is significantly lower than what existing methods can achieve.
However, the finger vein technology, as mentioned above, has also important applications in biomedical field. An initial work for localizing surface veins via near infrared (NIR) imaging and structured light ranging is presented in [19]. The eventual goal of the system is to serve as the guidance for a fully automatic (i.e., robotic) catheterization device. The proposed system is based upon near infrared (NIR) imaging, which has previously been shown effective in enhancing the visibility of surface veins. The vein regions in the 2D NIR images located using standard image processing techniques. An NIR line generating LED module is used for to implement structured light ranging and construct a 3D topographic map of the arm surface. The located veins are mapped to the arm surface to provide a camera registered representation of the arm and veins.
Also in [20, 21], a vein contrast enhancer (VCE) has been constructed to make vein access easier by capturing an infrared image of veins, enhancing the contrast using software, and projecting the vein image back onto the skin. The VCE also uses software to align the projected image with the original vein and with accuracy of 0.06 mm. Clinical evaluation of earlier monitor based vein enhancement test systems has demonstrated the clinical utility of the infrared imaging technology used in the VCE.
Although these methods achieve segmenting the infrared images, the finger vein pattern extraction task is still challenging mainly due to the fact that infrared images suffer from strong noise presence, uneven illumination, and shading, factors that complicate the application of automatic image segmentation techniques. Thus, another way to segment this kind of images is to assume that veins located in thin and concave regions (a reasonable assumption obtained by a careful inspection of the image intensity across the image) of infrared images based on this concept to extract them by optimizing a mathematical model. This can be done by using the Mumford Shah Model which has well-known capabilities in the image processing applications such as image segmentation, restoration, and image inpainting [22, 23]. Thus, in this paper, an analytical solution to a modified Mumford Shah Model minimization problem is derived and a local application of its results, in order to perform fast and accurate finger vein extraction, is proposed.
The remainder of this paper is organized as follows. In Section 2 the experimental device and the image acquisition procedure is presented. In Section 3, a detailed presentation of the finger vein pattern extraction method is given. The experimental results and discussion are included in Section 4. Finally, the most significant conclusions and some directions for future work are presented in the last section of this paper.
2. Image Acquisition
A typical hardware used to acquire infrared images consists of a finger probe, an array of infrared leds with adjustable illumination, and a video camera focus on frame, as shown in Figure 1. The finger ROI was placed inside the probe, between the open frame and the array of infrared leds light source which consists of a number of leds with adjustable illumination. The finger probe eliminates the influence of the external light sources.
The acquired image is produced as a result of several physical phenomena that happen during light propagation through human tissue, that is, absorption, diffusion, and scattering [24]. The great number of substances contained in the human body, the blood dynamics, and the mass transfer phenomena complicates significantly the light transformation effects. Therefore, the solution of the inverse problem, that is, the derivation of the arterial network from the image data, becomes unrealistic. A totally different and popular approach, adopted also in the proposed method, uses several image enhancements, feature extraction, and path reconstruction methods to derive the vein network, based on the fact that substance haemoglobin presents strong absorption in the infrared wavelengths, and therefore the veins appear in the image darker than the other human tissues (Figure 2).
3. Detection of Vein Network
In Figure 3, a flowchart of the proposed vein extraction method is given.
3.1. Image Preprocessing
3.1.1. ROI Extraction
From the original image (Figure 2) a region of interest (ROI) is defined based on several statistical properties of the histogram; that is, very low or very high brightness areas are excluded from the ROI. In the designed hardware each infrared led has adjustable intensity, giving excellent image quality, minimizing also the variance of the automatic exposure times of the image acquisition system.
The acquired image suffers from shading and nonuniform illumination both in left side and in right side of the image. This effect usually influences the performance of automatic image processing methods applied in order to extract the finger vein pattern. Thus, ROI extraction is used in order to localize the finger region and to isolate the shading artifacts. In this paper, the method proposed in [25] is adopted for ROI extraction. This method is based on the cutoff of regions with shading taking care about the different dimensions of the fingers among each person. Two masks, one for and one for direction, are used to isolate the boundary and localize the effective finger region. A typical ROI extracted by the method proposed in [25] is shown in Figure 4.
3.1.2. Brightness Normalization Based on Local Statistical Measures
In general, even in case where led’s intensity is adjusted to satisfy several statistical properties, in few areas of the acquired image unsatisfactory illumination or strong noise distortions are met. Therefore an image normalization procedure is applied to restore partially the desirable characteristics.
The proposed local normalization procedure unifies the local mean and variance of the ROI, especially useful technique for correcting nonuniform illumination or shading artifacts, using a linear transformation scheme applied on pixels’ brightness,where is the brightness of the original ROI image at pixel , is the brightness local mean, is the corresponding local standard deviation, and is the normalized image. The estimation of local mean and standard deviation is performed inside small window neighborhoods by averaging pixel intensities, a process also known as spatial smoothing.
3.2. Minimization of the Mumford Shah Model
The human veins in finger are significantly thinner than the darker structures observed in typical infrared images, as shown in Figure 2. Multiple scattering of the propagated photons reduces significantly the contrast, eliminates the tiny veins, and increases the transition regions between the vein and the surrounding tissue. The “fog” effect hides the vein structures in concave regions of the ROI. This assumption could be verified by observing the cross section profile of the veins which is Gaussian-like, as claimed in [26]. The aim of the proposed system is to focus on the concave regions enhancement, based on several connectivity properties in order to simply separate them from the rest tissue by a local entropy thresholding technique.
The concave regions are regions in the image domain where the second order derivative is positive. Direct estimation of the derivatives in digital images is an ill-posed problem due to noise presence and the variations in illumination. Instead of seeking regions which have positive second order derivatives, the minimization of an objective function similar to the objective function of the Mumford Shah Model [27] is proposed. The objective of the Mumford Shah Model is to estimate a smooth function such that it is similar to the original image . The equivalent mathematical expression leads to the minimization problem of the following objective function:where is the image domain and is a user defined parameter. The minimization of this function is computationally intensive and can be performed by the method proposed by Chan and Shen [22]. This method belongs to the category of segmentation methods which use partial differential equations (PDE) and it is iterative. Instead of deriving the global minimum of (2), in this paper a close form solution of a discrete objective function (3) similar to the objective function of the Mumford Shah Model is proposed by processing small rectangular areas. The proposed solution accelerates significantly the processing time and outperforms the classical approach [22]. As a result, fast and accurate extraction of the finger veins is obtained.
Assuming, without loss of generality, that the original image has rows and columns (however the image can have different dimensions along the two axes), the function in the discrete space is defined as follows:where is the gradient of the image and can be written asIf the partial derivatives in (4) are approximated using local differences,and (4) is substituted in (3) the following formula is obtained:The minimum of this objective function (6) regarding can be derived in a close form by differentiating the second order, positively defined function:As it can easily be observed, (7) is applied for pixels in order to avoid the boundary treatment problem for the finite difference scheme. Thus, the necessary image points are available for this type of approximation scheme. Obviously, this approach has a truncation error but, as proved experimentally, it has negligible impact in the performance of the proposed method.
Equation (7) can be rewritten in matrix form aswhich is in the form . is a sparse Hermitian matrix that depends only on parameter coefficients and size , is the vector of unknown image , and is the vector of the original image . If the matrix is invertible, the brightness of the unknown image is derived from the solution of the system of linear equations:The matrix is invertible if the determinant is nonzero (see Appendix—Lemmas A.1 and A.2 and Theorem A.3).
3.3. Modified Mumford Shah Model
From the above analysis, a sparse Hermitian matrix has been arisen. This matrix, as it is evident from (8), has the value in its central diagonal, the value −1 in the next up and down diagonal, and the value −1 positions before and after the central diagonal. Thus, it is a block tridiagonal matrix which can be inverted using an iterative algorithm such as one presented in [28]. Instead of using this computationally exhaustive algorithm, the fact that the above form of matrix can be obtained from two independent minimizations is exploited: one for the second order partial derivative in the -axis and one for the second directional derivative in the -axis. Thus, the following two objective functions have to be minimized with respect to :These minimization problems (10) led to the following set of equations:By differentiating above two equations the following formulas are obtained:
Equations (12), (13) can be written in matrix form asFrom (14) and (15) it can be observed that in (14) the matrix has the value in its central diagonal and the value −1 positions before and after the central diagonal while in (15) the matrix has the value in its central diagonal and the value −1 in the next up and down diagonal. Thus, the combination of the two matrices can give us the same results (see Appendix—Statement) as the matrix presented in (8). By exploiting this fact we can use only the matrix obtained by (15) instead of using the matrix obtained from (8). The advantage of this approach is that in this case the inversion of a symmetric matrix (15) is required which can be obtained analytically (see Appendix) and without the computationally exhaustive approaches adopted in [28–34]. In the sequel, (9) is applied one time for the original image and one for its rotated version by 90 degrees. This is done because two objective functions (one in the direction and one in the direction) have to be minimized. The enhanced image is derived by the addition (after normalization) of the two independent results (after the rotation of the second image by −90 degrees in order to have a meaningful result):where , are the original image (rearranged as vector) and its rotated version (also rearranged as vector) by 90 degrees, respectively, is the inverse of the matrix presented in (15), and , are the resulting smooth image and its rotation by 90 degrees version. Finally, is their sum after the rotation of by –90 degrees in order to have a meaningful result.
The determinant of the matrix must be nonzero in order to be invertible. This is proved in Lemmas A.1 and A.2 (see Appendix).
In practice, even in the case of small images, that is, pixels, the inversion of a sparse matrix of is required. This is obvious of too high computational cost. An effective reduction of the matrix dimensionality can be achieved using subimages, that is, multiple solutions of (13), using only a small number of neighbor pixels. In this case the number of linear equations depends on the size of the chosen window.
From the optimization criterion, the new image is a smooth image similar to the original . As a result of the image transformation method the veins are located in concave nonsmooth regions, so the veins network is enhanced in the nonsmooth image .
In order to obtain the image the following process is applied. Initially a window of size is selected and the corresponding matrix and its inverse invA are estimated. Then, the sliding window is moved along each pixel of the original image and the difference between the pixel values (of the image inside the window) obtained by (16) or (17) and the pixel values of the original image inside the window is computed. In the next step all these window differences are added and the image is obtained by keeping the central part of the result (where is the size of the original image).
The result of this process is the nonsmooth image . In this image the veins located in concave regions and for this reason a local entropy thresholding technique is applied in order to segment the nonsmooth image in concave (veins) and nonconcave (other tissue parts) regions.
This procedure is repeated for two times: one time for the original image and one for its rotation version by 90 degrees. The two nonsmooth images are added in order to obtain the final enhanced image (18).
To conclude, the concept of the proposed method originates from the continuous Mumford Shah Model. However, in this paper, a modified separable discrete model (11) is proposed and the transition between continuous and discrete space is not straightforward. Equation (3) is used only for indicating the conceptual similarity between the proposed discrete model and the continuous Mumford Shah Model because it is deemed that it is not fair to present the proposed model as an entirely novel model. Thus, the actual minimization is performed in models presented in (6) and (11) for the modified discrete and the modified separable discrete model, respectively.
3.4. Local Entropy Thresholding
Among the various existing methods used to automatically define the threshold for segmentation, the local entropy thresholding is selected, which has been successfully used in [35], because pixel intensities are not independent and this efficient entropy based thresholding takes into account the spatial distribution of intensities. It is based on the estimation of the cooccurrence matrix of the image which is a measure of the transition of intensities between adjacent pixels. Specifically, a local entropy thresholding technique, described in [36], is implemented which can preserve the structure details of an image. Two images with identical histograms but different spatial distribution will result in different entropy (also different threshold values).
In a correction note N. R. Pal and S. K. Pal [37] propose two modifications to improve the results of blood vessel extraction that are essential to the performance of image registration. These modifications were adopted also in our study because they experimentally proved superior to [36] (see Appendix—Local Entropy Thresholding).
3.5. Postprocessing
3.5.1. Morphological Dilation
The resulting binary image tends to suffer from some misclassifications (outliers). In order to have a robust segmentation in this postprocessing substep a morphological dilation [38] with a line structuring element oriented along the -axis and elongated pixels are performed. The output of this step is an image with less outliers but with still evident misclassifications. The final morphological filtering substep that follows gives us the desired robust finger vein pattern.
3.5.2. Morphological Filtering
In this substep the image is postprocessed by applying iteratively a morphological filter called majority [38]. This filter sets a pixel to 1 if five or more pixels in its 3-by-3 neighbourhood has the value 1; otherwise, it sets the pixel to 0. This filter is applied iteratively until the output image remains unchanged. This application clears the image from small misclassified regions which appears due to the presence of noise and smoothes the contours.
4. Experimental Results
4.1. Real Image Database
The original image was acquired under infrared light using an inexpensive CCD camera. The finger was placed between the camera and the light source which consists of a row of infrared leds (five elements) with adjustable illumination. The intensity of the leds adjusted as far as the illumination of the image was good enough. However, the problem of acquisition of infrared images is not a trivial task. The phenomena which were involved in the transmission of light inside the human tissue are very complicated. This fact does not permit us to acquire a sufficient number of images in order to construct an infrared finger image database and to release it in the research community for evaluation and comparison purposes. Our future work is mainly focused on the direction of improvement of the image acquisition system.
An excellent image illumination is not a strict requirement because the good performance of the proposed method remains also under adverse illumination conditions. Due to the fact that haemoglobin has strong absorption in the infrared light the veins are shown in the image darker than the other human tissues. So, the goal of our study is to extract these dark regions, corresponding to veins, from the background, corresponding to the other human parts (tissue). The original image which was acquired as described above is shown in Figure 2.
In this section the results of the application of our method and the methods proposed in [6, 7, 11, 12] are presented. The qualitative evaluation of methods performed in the ROI image is shown in Figure 4 and in the original image shown in Figure 2.
4.2. Proposed Method
For both images a window neighbourhood of size is used which results in an matrix which can be inverted very quickly. The selection of the parameter does not affect the performance of our method and thus it is arbitrarily selected as . Parameter is a weighting factor between the two terms of (2). It accounts for the degree of similarity between the original image and the estimated image. Extensive experiments are conducted in order to justify the selection of parameter λ. During the experiments the evaluation rates did not vary very much by changing the parameter λ from 0.1 to 1. The average divergence on the results was almost 2%. For the threshold computation the modified local entropy thresholding technique is used as described in the correction note [37]. In postprocessing step, a line structuring element with pixels in length and oriented in the -axis is employed.
Figures 5–8 present the results of the application of the proposed method in the ROI image of Figure 4. Figure 5 shows the nonsmooth image obtained after the application of the modified separable Mumford Shah Model in both 0 and 90 degrees direction and the addition of the results. Figure 6 shows the detection, with the help of the modified local entropy thresholding, of the concave regions of the image, where the veins tend to locate. In this binary image, concave regions (candidate pixels to be detected as veins) are shown in black while other tissue parts shown in white. Figure 7 shows the binary image after the application of the morphological dilation substep and, finally, the image in Figure 8(a) shows the extracted finger vein pattern obtained after the final morphological filtering substep.
(a)
(b)
(c)
Moreover, the results of the application of our method in the original image of Figure 2 (before ROI extraction) are presented. Figure 9 shows the nonsmooth image obtained after the application of the modified separable Mumford Shah Model in both 0 and 90 degrees directions and the addition of the results. Figure 10 shows the detection, with the help of the modified local entropy thresholding, of the concave regions of image. In this binary image, concave regions (candidate pixels to be detected as veins) are shown in black while other tissue parts shown in white. Figure 11 shows the binary image after the application of the morphological dilation substep and finally Figure 12(a) shows the extracted finger vein pattern obtained after the final morphological filtering substep.
(a)
(b)
(c)
4.3. Method [6, 7]
As mentioned, the proposed method is compared with our implementation of the method presented in [6, 7]. The repeated line tracking method requires some parameter tuning in order to run and the robustness of the extracted finger vein pattern is strongly affected by the number of repetitions. The parameters used in the experiments are for the number of iterations, and for the probabilities of selecting the three neighboring pixels in the horizontal or vertical direction, respectively, for the width of the profiles, and for the distance between the testing pixel and the cross section. In order to perform a fair comparison between method [6, 7] and the proposed method the same morphological postprocessing step is used in all experiments.
4.4. Other Methods
Moreover, the proposed method is also compared with more recent methods such as those presented in [11, 12], briefly described in the Related Work section, due to the fact that these methods have been applied in the same real and artificial image databases and thus the comparison is reasonable.
For purposes of comparison the results of the application of methods [6, 7, 11, 12] in original and ROI image are presented in the same figures as the results of the proposed method. Thus, Figure 8(b) shows the extracted finger vein pattern produced after the application of the line tracking method in the ROI image of Figures 4 and 8(c) shows the extracted finger vein pattern produced after the application of the method [11] in the ROI image of Figure 4 while Figure 12(b) shows the extracted finger vein pattern produced after the application of the line tracking method in the image of Figures 2 and 12(c) shows the extracted finger vein pattern produced after the application of the method [12] in the image of Figure 2.
Observing Figures 8 and 12, it is obvious that no safe conclusions regarding the performance of all methods can be conducted by visual inspection of the images. Instead, a comparison based on widely known measures should be done. Unfortunately, there is not a publicly available database existing that can be used for evaluation and comparison purposes between various methods. However, an artificial finger image database is constructed in this study in order to evaluate the proposed method and to compare it with the methods presented in [6, 7, 11, 12]. It is worth mentioning that the results presented for the method [6, 7] are produced by the application of our implementation to images since the code of the method is not publicly available by the authors.
4.5. Artificial Image Database
A quantitative evaluation of the proposed method in real infrared images is difficult due to the absence of manual segmentation data. The extremely low contrast images increase the disagreement of human annotation. Therefore, the proposed method is evaluated using a small set of images, each one created by the weighed sum of two artificial images. The first image is constructed using an artificial vein-like network. This network consists of connected lines of different widths with junctions and bifurcations and multiple low pass filtering to simulate the blurriness of the edges which is apparent to the real images due to the blood flow and scattering effects. The second artificial image is used to simulate the nonuniform image background of real infrared images created by applying an iterative spatial low pass Gaussian filter with a large window size to the original infrared image.
4.6. Evaluation Rates
In the finger vein segmentation process, each pixel is classified as tissue (nonvein) or vein. Consequently, there are four events, true positive (TP) and true negative (TN) when a pixel is correctly segmented as vein or nonvein and two misclassifications: a false negative (FN) appears when a pixel in a vein is segmented in the nonvein area, and a false positive (FP) when a nonvein pixel is segmented as a vein pixel.
Two widely known statistical measures are used for method evaluation: sensitivity and specificity, which are used to evaluate the performance of the binary segmentation outcome. The sensitivity is a normalized measure of true positives, while specificity measures the proportion of true negatives:Usually, there is a tradeoff between two measures. Finally, the accuracy of the binary classification is defined by where and represent the total number of positive (vein) and negative (nonvein) pixels in the segmentation process and are the degree of conformity between the estimated binary classification and the ground truth obtained through a manual segmentation. Thus, the accuracy is strongly related to the segmentation quality and for this reason it is used to evaluate and compare different methods.
4.7. Evaluation Results
The proposed method and the methods [6, 7, 11, 12] are evaluated quantitatively on the artificial image database. Each image of the set is constructed according to the above procedure. The evaluation is performed using the widely known statistical measures of sensitivity, specificity, and accuracy.
Table 1 shows the mean sensitivity, specificity, and accuracy of the proposed method and the methods [6, 7, 11] (without preprocessing/postprocessing) and [12] on the artificial finger image database, while Figure 13(a) shows the ROC curve of the proposed method produced by varying the local segmentation threshold and estimating the corresponding measures, Figure 13(b) shows the ROC curve of the method [6, 7] produced by varying the local segmentation threshold and estimating the corresponding measures, Figure 13(c) shows the ROC curve of the method [11] produced by varying the local segmentation threshold and estimating the corresponding measures, and Figure 13(d) shows the ROC curve of the method [12] produced by varying the value of the threshold parameter and estimating the corresponding measures.
(a)
(b)
(c)
(d)
Figure 14(a) shows the first image of the artificial image database used for the evaluation of the proposed method and the corresponding results of the segmentation for the proposed method (Figure 14(b)) and for the methods [6, 7] (Figure 14(c)), [11] (Figure 14(d)), and [12] (Figure 14(e)), respectively.
(a)
(b)
(c)
(d)
(e)
By observing the results presented in Figure 14 and in Table 1, it seems that the proposed method performs better than the methods presented in [6, 7, 11] in artificial finger image database in terms of sensitivity, specificity, and accuracy and has comparable performance with the method presented in [12] using the same evaluation criteria. In addition, the visual inspection of images shown in Figure 12 leads to the conclusion that the proposed method performs extremely better than the method [12] in real finger images. Thus, the proposed method, unlike the others compared, robustly extracts the finger vein network and preserves its connectivity against various conditions (shading, intensity variations, and noise distortion) both to real and to artificial finger images.
Apart from evaluating and comparing the proposed method to real and artificial finger vein images authors present also the results of the application of the proposed method and method [6, 7] in artificially distorted images using different types of noise and different levels of distortion. The study of the effect of noise led to an adaptation and a modification of the proposed method in order to robustly perform under extreme conditions. These conditions, although they seem to be unrealistic, may be produced by a low quality acquisition system and its noncareful setup.
Table 2 shows mean sensitivity, specificity, and accuracy of the proposed method and the method [6, 7] for artificial infrared images with different level of distortion. These results are presented in order to indicate the robustness of the proposed method especially in low quality images. Figure 15 shows the distorted artificial images and the corresponding finger vein patterns produced after the application of the proposed method while Figure 16 shows the distorted artificial images and the corresponding finger vein patterns produced after the application of the method presented in [6, 7].
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(i)
(j)
(k)
(l)
(m)
(n)
(o)
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
(i)
(j)
(k)
Comparing the results presented in Tables 1 and 2, it seems that the proposed method is superior to method [6, 7] in terms of sensitivity, specificity, and accuracy for the artificial finger image database. Moreover, it is worth noting that the results of the method [6, 7] are slightly different among different executions with the same parameters and images due to the randomness introduced in the selection of the current tracking pixel and the parameters and . Finally, regarding the computational complexity, the proposed method also outperforms the method [6, 7] as the number of iterations of line tracking increases.
By carefully observing the results in Table 2 and the images in Figure 16 it is obvious that the performance of the method presented in [6, 7] in low quality images is degraded and becomes unacceptable for images with high level of distortion. Thus, the proposed method outperforms line tracking method when applied to low quality image which may be produced as a result of a noncareful image acquisition setup.
As the above results show, the proposed method performs well in the majority of cases and achieves efficiently segmenting the finger vein images. However, no scientific method is perfect. Every method has also drawbacks. Thus, the main drawback of the proposed method is that its performance both in terms of segmentation accuracy and in terms of computational complexity is strongly related to the size of window neighbourhood. An appropriate selection of window neighbourhood size must be done in order to achieve meaningful results. On the other side, in case of an accurate image acquisition setup which will acquire images of specific resolution, the window size could be derived experimentally and could be then set at once.
4.8. Matching
In general, two methods are commonly used for matching of line-shaped patterns: structural matching [39] and template matching [40, 41]. As stated in [6, 7], structural matching requires additional extraction of feature points such as line endings and bifurcations. Since a finger vein pattern has few of these points, template matching based on comparison of pixel values is more appropriate for finger vein pattern matching. Thus, in this paper the robustness of the proposed method and the method [6, 7] for finger vein identification is evaluated by estimating the mismatch ratio between the registered and the input data for the artificial finger image database. In the matching process, the extracted finger vein pattern is converted into matching data, and these data are compared with the recorded raw data. For the case of artificial image database the ground truth data was used as recorded raw data for each image.
Mismatch ratio is calculated to examine whether or not two sets of data have a correlation with each other. The ratio is defined as the difference between two sets of data to be matched. and are the values at position of the registered and input matching data, and are the width and height of both sets of data, and are the distances in which motion in the vertical and horizontal directions, respectively, is required to adjust the displacement between the two sets of data, and the template data are defined as the rectangular region within whose upper left position is and lower right position is .
The value of mismatch , which is the difference between the registered and input data at the positions where overlaps with , is defined as follows:where and in consideration of the finger size in the captured image, and are set at and in order to adjust the finger position in the captured image by up to about 1 cm, and in (21) is a parameter that indicates whether a pixel labeled as part of the background region and a pixel labeled as part of a vein region overlapped with each other. When is defined as the pixel value of one pixel and is defined as the pixel value of the other pixel, can be described as follows:The minimum value of mismatch , which is the smallest calculated under the condition that the template overlaps with the input matching data at all positions, can be defined as follows:Using the definitions given above, the mismatch ratio is defined as follows:where and are and such that (23) is minimized. As is shown by (24), is described as the ratio between and the total number of pixels that are classified as belonging to the vein region in the two data sets.
As shown in Table 3, the average mismatch ratio for the artificial finger image database, estimated using (24), is 24.65% for the proposed method while for the method [6, 7] it is 43.64%. Although database contains twenty images and the results cannot be generalized, the proposed method seems to be more appropriate for finger vein identification purposes. The extracted patterns by the proposed method have significantly lower value of mismatch ratio than those extracted by the method [6, 7].
5. Conclusions
In this paper an efficient finger vein pattern extraction method is presented. The proposed method is based on the minimization of the objective function of a modified Mumford Shah Model and the local application of its results. This application produces two nonsmooth images where veins located in concave regions. The two images are then combined simply by addition. Detection of concave regions is achieved via a modified local entropy thresholding technique. The preliminary segmentation result was unsatisfactory due to the presence of some outliers (misclassifications).
Thus, a final morphological postprocessing step followed in order to clean the image from the misclassifications and to produce a robust finger vein pattern. Future work includes the improvement of our imaging device in order to acquire images with less shading and noise artefacts, something that will guarantee the successful application of our method in the majority of cases. In case of images with high quality the preprocessing and/or postprocessing step can be skipped.
The experimental evaluation of the proposed method shows that it can robustly segment the finger vessel network and that the extracted finger vein pattern is appropriate for finger vein identification purposes. Finally, the proposed method is robust against strong distortions met in the acquired images.
Appendix
Lemma A.1. The determinant of the symmetric matrix in the formcan be estimated using the following recursive formula:
Proof. From the definition of matrix it can be derived that
Lemma A.2. The matrix determinant is an increased sequence of positive real numbers for any .
Proof. If , thenIf , thenThe induction rule leads to a sequence of positive numbers for the determinant of :
Theorem A.3. The system of linear equations is solvable for any positive value of .
Proof. Based on Lemma A.2, .
Statement. Modified model produces the same results as the original model.
Proof. The minimization of the objective function regarding leads to the following equation:while the minimization of the two independent objective functions (one for the second order partial derivative in the -axis and one for the second directional derivative in the -axis) leads to the following equations:Adding (A.8) and (A.9) results inThe only difference between (A.7) and (A.10) is that in (A.10) parameter is multiplied by a factor of 2. This makes no difference because the parameter can be arbitrarily set and has no strong impact on the final results. In other words, (A.7) and (A.10) are equivalent and produce exactly the same results if the parameter set in (A.8) and (A.9) to have the half value from the value has the corresponding parameter in (A.7).
Local Entropy Thresholding. The cooccurrence matrix of the image is a dimensional matrix that gives an idea about the transition of intensities between adjacent pixels, indicating spatial structural information of an image. Depending upon the ways in which the gray level follows gray level different definitions of cooccurrence matrix are possible. Here, we made the cooccurrence matrix asymmetric by considering the horizontally right and vertically lower transitions. This choice has been made in order to decrease the computational cost and it is reasonable since the horizontally left and vertically upper transitions do not add more information to cooccurrence matrix. Thus, is defined as follows:The probability of cooccurrence of gray levels and can therefore be written asIf , , is a threshold, then can partition the cooccurrence matrix into 4 quadrants, namely, , , , and (Figure 17).
Let us define the following quantities:From the occurrence matrix, the corresponding probabilities within each individual quadrant must sum to one. Thus, we get the following cell probabilities for different quadrants:The second order entropy of the object can be defined asSimilarly, the second order entropy of the background can be written asHence, the total second order local entropy of the object and the background can be written asThe gray level corresponding to the maximum of gives the optimal threshold for object background classification.
In the first modification a different definition of the cooccurrence matrix is adopted increasing the local entropy values in vein structures. As mentioned, the cooccurrence matrix of an image shows the intensity transitions between adjacent pixels. The original cooccurrence matrix is asymmetric by considering the horizontally right and vertically lower transitions. They added some jittering effect to the cooccurrence matrix that tends to keep the similar spatial structure but with much less variations; that is, is computed as follows:One may wonder whether the modified cooccurrence matrix still will represent the original spatial structure. Actually, considering a smooth area in an image where and should be very close or identical, the above computation in (A.18) implicitly introduces certain smoothing effect and adds some structured noise to the cooccurrence matrix. The two matrices still share a similar structure that is important for the valid thresholding result. Also, the latter one has larger entropy with a much smaller standard deviation, which is more desirable for local entropy thresholding.
Secondly, they considered the sparse foreground in selecting the optimal threshold. The original threshold selection criterion aims to maximize the local entropy of foreground and background in a gray scale image without considering the small proportion of foreground. Therefore, they proposed selecting the optimal threshold that maximizes the local entropy of the binarized image that indicates the foreground/background ratio. The larger the local entropy the more balanced the ratio between foreground and background in the binary image is.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.