Abstract
Latent fingerprints are captured from the fingerprint impressions left unintentionally at the surfaces of the crime scene. They are often used as an important evidence to identify criminals in law enforcement agencies. Different from the widely used plain and rolled fingerprints, the latent fingerprints are usually of poor quality consisting of complex background with a lot of nonfingerprint patterns and various noises. Latent fingerprint segmentation is an important image processing step to separate fingerprint foreground from background for more accurate and efficient feature extraction and matching. Traditional methods are usually based on the local features such as gray scale variance and gradients, which are sensitive to noise and cannot work well for latent images. This paper proposes a latent fingerprint segmentation method based on combination of ridge density and orientation consistency, which are global and local features of fingerprints, respectively. First, a texture image is obtained by decomposition of latent image with a total variation model. Second, we propose to detect the ridge segments from the texture image, and then compute the density of ridge segments and ridge orientation consistency to characterize the global and local fingerprint patterns. Finally, fingerprint segmentation is performed by combining the ridge density and orientation consistency for latent images. The proposed method has been evaluated on NIST SD27 latent fingerprint database. Experimental results and comparison demonstrate the promising performance of the proposed method.
1. Introduction
Latent fingerprints are the finger skin impressions, which are unintentionally left on the surface of a crime scene by deposits of oils and/or perspiration from the finger [1]. This impression is usually invisible to the naked eye but it can be captured with some special techniques. Latent fingerprints have been used as an important evidence to identify criminals in law enforcement agencies for more than a century. Different from the plain and rolled fingerprints which are captured with good quality in the controlled environment and widely used in commercial area, latent fingerprints are usually of low image quality caused by unclear ridge structure, uneven image contrast, and various overlapping nonfingerprint patterns like lines, printed letters, handwritings, etc. as shown in Figure 1. Samples of rolled, plain, and latent fingerprint images are shown in Figures 1(a), 1(b), and 1(c), respectively. Due to the poor quality, the performance of automated latent fingerprint identification system (AFIS) (matching latent fingerprint to full template fingerprints) is much lower than that of matching full to full fingerprints [1]. Thus, it is necessary to perform the segmentation and enhancement processes for latent fingerprints before input to AFIS. However, the latent fingerprints are usually corrupted by noise and the valid fingerprint area is small like Figure 1(c). These problems cause the difficulties in segmenting the fingerprint ridge area from the latent image.
(a)
(b)
(c)
Fingerprint segmentation is an important processing step to segment the fingerprint foreground with the interleaved ridge and valley structure from the complex background with nonfingerprint patterns for more accurate and efficient feature extraction and identification. There are many methods proposed for fingerprint segmentation in the literature [5–8]. In these methods, the fingerprint image is uniformly divided into blocks and some local image features such as mean and variance of gray values and the orientation consistency are extracted to evaluate the fingerprint quality for segmentation. They can work well for plain and rolled fingerprints, but their performances are not satisfactory for latent fingerprints. The main difficulties may be due to weak ridge structure, and the complex overlapping irrelevant patterns and various types of noise in latent fingerprint images. The valid fingerprint region is small compared to the complex background. Thus, it is still challenging for segmentation of latent fingerprints.
There are some research efforts made to investigate new methods for improving the latent fingerprint segmentation [2–4, 9–15]. Zhang et al. [2] proposed an adaptive directional total variation (ADTV) model for latent fingerprint segmentation. This model combines both the anisotropic directional TV term and the spatially adaptive fidelity weight for more effective segmentation of latent fingerprint. Short et al. [9] proposed a method to apply an adaptable ridge template to determine a goodness of the fit score in a local region of fingerprint for segmentation. Choi et al. [10] proposed to combine both ridge orientation and frequency features for latent fingerprint segmentation. In that method, the orientation tensor is used to obtain the symmetric patterns of fingerprint ridge orientation and the local ridge frequency is estimated by the local Fourier analysis. These two features are combined for segmentation. Karimi-Ashtiani and Jay Kuo [11] proposed a latent fingerprint segmentation algorithm based on a quasi-global fingerprint model, which did not rely on the local gradients for estimation of orientations and frequencies and thus is robust to the gradient deviations. Cao et al. [12] proposed a method for latent fingerprint segmentation based on a ridge quality measure which was defined as the structural similarity between the fingerprint patch and its dictionary based reconstruction. A learning based method was proposed for latent fingerprint image segmentation based on fractal dimension features and weighted extreme learning machine ensemble [3]. The fractal dimension features were computed to characterize the texture patterns of image patches and persistent in geometric quantities. Instead of using the local image features, the linear density, which is a global feature, was computed by detection of line segments for latent fingerprint segmentation [4]. A clustering-based approach was investigated for latent fingerprints segmentation [13]. A latent fingerprint segmentation method was proposed based on orientation and frequency features [14]. Recently, a deep artificial neural network (DANN) model was proposed for segmentation of latent fingerprints by using a stack of restricted Boltzmann machines (RBMs) to learn the features of fingerprint image patches [15].
This paper proposes a latent fingerprint segmentation method based on combination of the ridge density and orientation consistency, which are extracted to characterize both the global and local texture patterns of fingerprint image. First, a total variation (TV) model is used to decompose the latent image into texture and cartoon components. The cartoon component is removed as the noise while the texture component consisting of the fingerprint pattern is used for further processing. Second, to characterize the interleaved ridge and valley structure of fingerprint, we propose to detect the ridge segments and then compute the ridge density and orientation consistency to evaluate the quality of fingerprint image. Finally, a segmentation mask is generated by thresholding and morphological processing for each latent image. The proposed method is tested on NIST SD27 latent fingerprint database and is compared to some existing methods to demonstrate its performance.
The rest of this paper is organized as follows. In Section 2, we will present the proposed latent fingerprint segmentation algorithm in detail. Section 3 provides the experimental results and comparisons. Finally, Section 4 concludes this paper.
2. The Proposed Method
In this section, we present the proposed segmentation algorithm for latent fingerprints in detail. The valid fingerprint pattern is often characterized as a number of parallel interleaved ridge and valley flows, while the nonfingerprint pattern consists of arch, lines, printed letters, speckle, handwritings, stain, and other noises. The main difference between the fingerprint and nonfingerprint patterns is the parallel interleaved ridge and valley flows. Thus, we propose to detect the ridge segments and the region with the dense distribution of parallel ridge segments is segmented as the foreground of fingerprint image. The general flowchart of the proposed latent fingerprint segmentation algorithm is demonstrated in Figure 2. It consists of four main steps, i.e., decomposition of latent image into cartoon and texture components, detection of ridge segments, computation of ridge density and orientation consistency, and latent fingerprint segmentation, which will be presented in the following subsections, accordingly.
2.1. Decomposition of Latent Fingerprint
In addition to the fingerprint pattern, latent images often contain various types of nonfingerprint patterns such as arch, line, character, speckle, handwritings, and stain, which usually have the smooth inner surface and sharp edges (see Figure 3(a)). The total variation (TV) model was proved to be an efficient model for regularizing images without smoothing the boundaries of targets [16]. To reduce these structured noises for latent fingerprint segmentation, the TV model is used to decompose the latent image into texture and cartoon components, i.e., . Texture component contains the oscillatory and noise component, while cartoon consists of the piecewise smooth component. Mathematically, the decomposition is obtained by minimization of total variation [16]:where and denote the image cartoon and texture components, respectively; denotes the total variation of ; and is a two norm. The parameter is used to balance the tradeoff between the fidelity of and the minimization of TV. Small λ will produce weak texture and strong cartoon image and vice versa. In our experiments, λ is set to 0.4 to decompose the texture component of latent image for fingerprint segmentation.
(a)
(b)
(c)
Figures 3(b) and 3(c) show the decomposed cartoon and texture components of the latent image shown in Figure 3(a), respectively. The cartoon component consists of most nonfingerprint patterns and is removed as noise while the texture image contains the weak fingerprint pattern. After decomposition, not only the structure noise but also the varying illuminations have been significantly reduced from the texture image, which will facilitate latent fingerprint segmentation.
2.2. Detection of Ridge Segments
After image decomposition by TV model, a texture image is obtained and used for further processing while the cartoon image with most of the structured noises is removed as nonfingerprint pattern. It will be easier to extract features from the texture image for fingerprint segmentation. The texture component of latent fingerprint is characterized by oscillatory patterns with interleaved and parallel ridge and valley structure. In this work, we propose to detect the ridge segments from the texture image to capture the fingerprint patterns. The ridge segment detection procedure can be divided into image binarization with noise removal, image thinning, and orientation computation for each ridge point, detailed as follows.
First, the texture image is a gray scale image of which is thresholded to a binary image with noise removal. For simplicity, the threshold is set to 128. In the binary image, the black points are the background pixels while the white points are the candidate points of ridge segments. Since the latent fingerprint is often corrupted by various types of noise, it is necessary to remove the noisy points from the binary image for more reliable detection of ridge segments. The noisy points usually occur as the isolated points and are removed according to the following formula:
Figure 4(a) shows the binary image after removing the noisy points from the thresholded binary image of texture image in Figure 3(c). In addition, to enhance the detection of ridge segments, we also reverse the thresholded binary image with the black points changed into white points and vice versa. The noisy points are removed with the same procedure as above. These two binary images with noise removal are generated for better characterization of the interleaved ridge and valley pattern. Figure 4(b) shows the reversed binary image after removing the noisy points.
(a)
(b)
(c)
Second, the above two binary images are thinned with the thinning method in [17] and these two thinned images are superimposed into one binary image to form ridge segment points. However, there are some noisy ridge segment points, which are usually characterized to be short, closed, and crossed. To remove the noisy ridge points for reliable detection of ridge segments, we delete the short ridge segments with the length smaller than 4 and delete the closed ridge circles with the number of end points less than 2. For each ridge point, we compute the number of ridge points in its 3 × 3 neighborhood. If it is larger than 3, the point is the crossed point, and this point and all the ridge segment points in its 3 × 3 neighborhood are changed to black points. In this way, we go through all ridge points and obtain a set of connected and continuous ridge segments, which are denoted as . Figure 4(c) shows the final detected ridge segment points from Figure 4(b).
Finally, we compute the orientation for each ridge point. Each ridge segment is composed of a set of connected points. Thus, the orientation of each ridge point is computed based on its neighboring connected ridge points. Assume that the ridge segment consists of connected points, denoted as , where , , and are the starting, middle, and ending points of the ridge segment , respectively. The orientation of point is computed as follows:where is the coordinates of point . Thus, we have obtained both the location and orientation for each ridge segment point of the binary image.
2.3. Computation of Ridge Density and Orientation Consistency
From Figure 4(c), we can see that the valid fingerprint area should have more parallel ridge segments than the nonfingerprint area. Thus, after detection of the ridge segments and computation of the orientation for each ridge point, we propose to compute a ridge density measure, which evaluates how densely the ridge points are distributed in a local region with the specific orientation. In addition to the locations, the orientations of ridge points are also considered because the neighboring ridge segments are parallel. Thus, the whole orientation range [0 π) is equally divided into 9 bins as , i.e., . To compute the ridge density, we divide the binary image with the ridge segment points into the blocks of 16 × 16 pixels. Let denote the total number of ridge points in each block and denote the number of ridge points in each block with the orientation in the bin . The interleaved ridge and valley flows of fingerprint change smoothly except for a few singular points. Thus the ridge points are densely distributed in a local fingerprint region with consistent orientations. The ridge density for block () is computed by sum of the weighted distribution of the ridge points with respect to different orientations as follows:where is the area of the block. The ridge density is high if the ridge points are densely distributed in a block and concentrated in one orientation bin. Otherwise, the ridge density is low.
In addition, the orientations of ridge segments usually change slowly in a local neighborhood. The orientation of each block is defined as the orientation with the maximum number of ridge points. We compute the orientation consistency to evaluate how consistent the orientation of each block is with its 8 neighboring orientations in the 3 × 3 neighborhood. The orientation consistency for block () is computed aswhere denotes the availability of orientation of block and . If the block orientation is consistent with its 8 neighboring orientations, is close to 1. Figures 5(a) and 5(b) show the ridge density map and the orientation consistency map of Figure 4(c), respectively.
(a)
(b)
(c)
2.4. Latent Fingerprint Segmentation
For latent fingerprint segmentation, the ridge density and orientation consistency are combined to generate an image quality measure, which is computed for each block asFigure 5(c) shows the image quality map of Figure 4(c). Thresholding is applied to generate a binary segmentation mask for each fingerprint. To fill the gaps and eliminate the islands, morphological operations are further applied to obtain the biggest and continuous binary mask. The convex hull of the biggest mask is computed as the final segmentation mask. Figures 6(a), 6(b), 6(c), and 6(d) show the fingerprint segmentation results by thresholding, the biggest mask, its convex hull, and the final mask overlapped with the texture image, respectively.
(a)
(b)
(c)
(d)
3. Experimental Results
In this section, we conduct experiments on the NIST SD27 latent fingerprint database to test the performances of the proposed algorithm. NIST SD27 consists of 258 latent fingerprints which are manually assigned into three different levels of qualities: “good”, “bad”, and “ugly” [18]. The numbers of “good”, “bad”, and “ugly” fingerprints are 88, 85, and 85, respectively. Our algorithm was implemented with MATLAB software on a computer of 3.1 GHz Intel Core i5-4440, 4 G RAM, and 64-bit Windows OS. For the latent fingerprint images in NIST SD 27, the average computation time of our algorithm is about 5 seconds for one image. The manual markups of ROI (Region of Interest) by experts provided in [19] are used as the ground truths for NIST SD 27 in our experiments. We compare the segmentation results of the proposed algorithm with the manual markups and other published methods [2–4, 10].
First, we test the proposed algorithm on the same sample latent fingerprints of different image qualities from NIST SD27 database as those in [3]. The segmentation results provided in [3] are directly used for comparison. The segmentation results of the method [2] are downloaded from the author’s website. Figures 7, 8, and 9 show the comparisons of the segmentation results by the manual markups, the method [2], the method [3], and the proposed method, for good, bad, and ugly latent fingerprint images, respectively. From these results, we can see that the method [2] usually generated a larger segmentation mask which includes the nonfingerprint patterns. There are some gaps and islands in the segmentation results by the method [3]. Our proposed method can perform well to separate the fingerprint foreground from the complex background for latent images of different qualities, which is closer to the manual markups.
(a)
(b)
(c)
(d)
(e)
(f)
(a)
(b)
(c)
(d)
(e)
(f)
(a)
(b)
(c)
(d)
(e)
(f)
Second, although the segmentation results of latent images can be evaluated visually, we compare the segmentation accuracy of our method with those of some published methods [2–4, 10]. Two measures, the Missed Detection Rate (MDR) and the False Detection Rate (FDR), are computed for performance comparison on NIST SD27. The manual segmentation masks are used as the ground truth. MDR is the average percentage of foreground pixels misclassified as background, while FDR is the average percentage of background pixels misclassified as foreground. Table 1 shows the comparison of the segmentation accuracy in the proposed method with those of the existing methods [2–4, 10]. The segmentation results are directly used from those reported in the paper. In the comparison, the computations of MDR and FDR and the ground truth of segmentation masks by the proposed method are same as those in [2, 4, 10]. The MDR and FDR of our method are 13.03% and 23.17%, respectively. These results show the effectiveness of our segmentation method compared to the other methods. It seems that the method [3] performed best in comparison. But it should be noted that the segmentation ground truth in method [3] was generated by the author of the paper and is different from our proposed method and the computations of MDR and FDR measures were based on image patches instead of pixels.
The ultimate goal of segmentation is to improve the performance of latent fingerprint identification. Thus we conduct the latent fingerprint identification experiments to test the effectiveness of the proposed segmentation algorithm finally. The foreground of segmented texture image is the input of latent image matching. The commercial software VeriFinger SDK 6.5 is integrated to perform the feature extraction and matching for latent fingerprint identification. To make the latent matching experiment more realistic and challenging, the template database includes both the 258 template fingerprints of NIST SD27 and the 27,000 fingerprints of NIST SD14. Each latent fingerprint is matched against all the fingerprints of template database and a matching score is generated between the latent and a template fingerprint. The matching scores are sorted in descending order and the identification performance is evaluated by Cumulative Match Characteristic (CMC) curve. A CMC curve plots the identification accuracy rates with respect to different rank-k ().
We compare the performances of the proposed algorithm with those of the linear density based method [4] and the manual markup as well as those without segmentation, which are denoted as “linear density based method”, “proposed method”, “manual markup”, and “without segmentation”, respectively. “Without segmentation” means the texture image is directly used as input for latent fingerprint identification. Figures 10(a), 10(b), 10(c), and 10(d) show the comparisons of CMC curves on all latent, “good”, “bad”, and “ugly” quality fingerprints of NIST SD27, respectively. We can see that the identification of latent fingerprints with segmentation can achieve much better performance than that without segmentation for all latent fingerprints. For bad latent images, the proposed algorithm performs better than the linear density based method [4] and even better than the manual markup. The performance is just slightly worse than the manual markup for latent fingerprints of good quality. For all fingerprints, the proposed algorithm has better performance than the linear density based method [4] and achieves comparable performance to the manual markup. These results and comparison demonstrate the effectiveness of the proposed segmentation algorithm for latent fingerprints.
(a)
(b)
(c)
(d)
4. Conclusion
This paper proposes a latent fingerprint segmentation algorithm based on combination of ridge density and orientation consistency. First, a texture image is generated by image decomposition with a TV model, and the structured noise and illumination variation is greatly reduced from the texture image. Second, we propose to detect the ridge segments and compute the ridge density and orientation consistency, which are combined for latent fingerprint segmentation. Finally, the proposed algorithm is tested on NIST SD27 and compared with other methods and the manual markups. Experimental results and comparisons demonstrate the effectiveness of the proposed segmentation algorithm for latent fingerprints.
Conflicts of Interest
There are no conflicts of interest related to this paper.
Acknowledgments
This work was supported in part by National Natural Science Foundation of China (NSFC) under Grants nos. 61773263 and 61375112 and The National Key Research and Development Program of China (no. 2016YFC0100903), SMC Excellent Young Faculty program of SJTU. This work was also supported in part by Science and Technology of Fujian Education Department under Grant no. JAT170365.