Abstract

Fingerprint registration and verification is an active area of research in the field of image processing. Usually, fingerprints are obtained from sensors; however, there is recent interest in using images of fingers obtained from digital cameras instead of scanners. An unaddressed issue in the processing of fingerprints extracted from digital images is the angle of the finger during image capture. To match a fingerprint with 100% accuracy, the angles of the matching features should be similar. This paper proposes a rotation and scale-invariant decision-making method for the intelligent registration and recognition of fingerprints. A digital image of a finger is taken as the input and compared with a reference image for derotation. Derotation is performed by applying binary segmentation on both images, followed by the application of speeded up robust feature (SURF) extraction and then feature matching. Potential inliers are extracted from matched features by applying the M-estimator. Matched inlier points are used to form a homography matrix, the difference in the rotation angles of the finger in both the input and reference images is calculated, and finally, derotation is performed. Input fingerprint features are extracted and compared or stored based on the decision support system required for the situation.

1. Introduction

Image processing has many applications [13], including biometric registration and verification. Biometric registration and verification are performed to restrict unauthorized people from entering a specific building or wing, to keep track of users of service by registering their fingerprints in a database, to take automatic attendance of employees, and to store data along with biometric features in a database for issuing passports or national identity cards.

Biometric verification and registration using image processing techniques such as face detection, fingerprint detection, and retina or iris detection have been widely researched [48]. Face detection-based techniques [9, 10] store facial features to be compared during the registration and verification process. Face verification and registration provides accuracy at the cost of high complexity and extensive allocation of system memory. Accuracy demands that issues such as scale difference, posture, illumination, and the opening and closing of the eyes and mouth be addressed, further increasing complexity.

Iris-based recognition and verification [7, 11, 12], on the other hand, is done by applying segmentation in the region of the iris to extract its radius. This is followed by feature extraction. These techniques are used along with fingerprint verification at airports and other high-risk buildings to provide accurate identification. Iris-based recognition and verification provide good accuracy, but the required equipment is costly.

Fingerprint verification and registration is a widely researched area [1323] in which work has focused on algorithms and techniques to store and compare fingerprint features obtained using a scanner. Fingerprint-based biometric verification and registration are widely used in banks [19, 20] and national databases for maintaining passport and national identity card information [24].

By using different approaches, such as combining fingerprint registration with odor analysis [21] and applying a ridgelet transform [22], researchers have improved existing algorithms. Complexity can be reduced by comparing a small number of features instead of entire images. However, these methods require the use of sensors (fingerprint readers) that are not always available. This can make performing the registration and verification process difficult.

Solutions that avoid the need of sensors have been proposed [2427]. These techniques focus on processing images captured using ordinary digital cameras. Hiew et al. [2426] applied skin color-based segmentation followed by morphological processing to extract regions of interest. Short-time Fourier transform analysis (STFT) is then applied, followed by ridge orientation calculation and core point detection. The detection and matching of features such as bifurcations and ridges are not discussed. However, detection and matching can be performed using the same fingerprint techniques used in the algorithms applied to input obtained from sensors.

Sankaran et al. [27] proposed a method similar to that discussed in this paper that performs skin color segmentation followed by the region of interest (ROI) extraction. Image enhancement is applied followed by ScatNet-based feature extraction. Finally, feature-based fingerprint registration/matching is performed.

When features are extracted from a digital image of a fingerprint, scanners are not needed. However, orientation alignment is an issue that is inadequately addressed in the algorithms proposed by Hiew et al. and Sankaran et al. The fingerprint in the input image must be vertically aligned when conventional methods are used. When rotation of the finger is present, a fingerprint mismatch will occur even if features of that person are present in the database.

Therefore, this paper proposes a rotation and scale-invariant technique that takes an image captured using an ordinary digital camera as the input and applies skin color-based segmentation to the input image and a reference image to extract regions of interest from both images. SURF [28] algorithm is applied to the segmented binary images, and the detected features from both images are matched. Inliers are extracted from the matched features using the M-estimator [29] for accurate calculation of the rotation angle using a homography matrix [30]. The input image is derotated to obtain an image with a vertically aligned finger. The vertically aligned input image is then subjected to image enhancement techniques, such as unsharp masking and contextual blockwise Fourier domain filtering. Otsu thresholding is applied for binarization. Then, nonmaximal suppression is used to obtain ridges having a thickness of one pixel. Features such as bifurcations and ridge endings are extracted from the thin ridges to be stored in a database for registration or compared with the features of registered fingerprints for verification. Simulation results show that the proposed method provides an accuracy of 100 percent if the suggested parameter range is applied.

This paper is organized as follows: Section 2 discusses the proposed methodology, Section 3 contains the results and discussion, Section 4 presents the conclusions and the direction of future work.

2. Methodology

The objective of this work is to propose a cost-effective rotation and scale-invariant alternative to the processing of fingerprint images obtained using scanners. A flowchart of the algorithm is shown in Figure 1. The major steps of the proposed algorithm are the derotation of the input image, image enhancement, binarization, thinning, minutiae extraction, and matching. The steps shown in the flowchart are explained in detail in this section.

2.1. Image Acquisition

In this study, image acquisition was performed using several smartphones: Samsung Galaxy S2, iPhone 3s, Galaxy S6, and iPhone 8. Images of fingers were captured with the focus on their ridges.

2.2. Image Derotation

A flowchart depicting the derotation process is shown in Figure 2. An image containing a finger aligned along the y-axis is selected as a reference image, as seen in Figure 3(a). Any input finger image requiring registration or verification is considered an input image and is subjected to the derotation process so that the fingerprint registration/matching process can be done smoothly. Figure 3(b) shows an example of an input image that requires rotational alignment for accurate feature matching.

2.2.1. Skin Color-Based Segmentation

Both the input and reference images are subjected to adaptive skin color-based image segmentation [31]. The images shown in Figures 3(c) and 3(d) show the results of this segmentation on the reference and input images. The input image shown in Figure 3(b) is particularly complex because the color of the background is similar to the color of the skin. Therefore, noise is detected as part of the finger. The white region in Figure 3(d) is the region of interest (ROI) and will be used in upcoming sections.

2.2.2. Input Image Scale Adjustment

Both input and reference images usually are of different dimensions. Therefore, the fingers in both images might have a huge scale difference. In that case, scale variance might produce problems while detecting and comparing features. In order to provide a scale-invariant derotation, scales of both input and reference fingers should nearly be the same. If the number of pixels belongs to the finger in the segmented reference image, it is denoted by and if the number of pixels belongs to ROI, it is denoted by . Scaling factor is calculated using (1) as

If , both the intensity and segmented input images are scaled down by a factor of , whereas if , the images are interpolated by a factor of using simple bilinear interpolation.

2.2.3. Feature Extraction and Matching

Intensity images are converted into binary images for time-efficient feature matching. Since finger shapes are all similar, features detected along their boundaries can provide better estimations of geometrical differences. For feature detection, the SURF [28] method is applied to the segmented binary images to obtain boundary features for both images. SURF is selected for its robustness and reasonable match rate for two same images with different angle of rotation [29]. After obtaining features, nearest neighbor-based feature matching by computing the pairwise distance between features is performed. Then, feature matching provides matched features that contain truly matched inliers as well as erroneously matched outliers. Figure 3(e) shows all the matched features including both inliers and outliers.

2.2.4. Inlier Extraction and Derotation of Image

To extract inliers from the matched points, an M-estimator [29] algorithm is applied. Figure 3(f) shows the inliers extracted from all matched features using the M-estimator. It can be seen that extracted inliers represent true matches that can be used to estimate the angle of rotation between accurately matched inlier points.

Once several potential inliers are extracted, the angle of rotation between the matched inliers must be determined. A homography matrix [30] is calculated for this purpose. For the two sets of matched points, for the reference image and for the input image. Then, the relation between them is as given in the following equation:where

The resultant matrix in (3) is of the form given in (4) as

Therefore, the angle of rotation can be calculated using or using (5) aswhere can be used to derotate the input image to obtain an image with a vertically aligned finger. Figure 3(g) shows the derotation of the input image after the calculation of . Figure 4 shows the alignment method applied to various input images showing fingers at different angles of rotation. It can be seen that the proposed algorithm predicted the angles of rotation accurately and that the fingers in the derotated images are aligned vertically irrespective of their initial alignment.

2.3. Image Enhancement

Edge-based image enhancement is applied to the grayscale image to improve the visibility of ridges on the finger. Edge-based enhancement is performed in two stages. First, unsharp masking [32] is applied to the image. This algorithm subtracts an image from a blurred version of itself and enhances those regions which provide a good response after subtraction. To calculate the blurred image, a Gaussian filter is convolved with the original grayscale image using the following equation:where

For the second stage of image enhancement, a contextual blockwise Fourier domain filter [33] is applied to the grayscale image by dividing it into a 16 × 16 overlapping block. These blocks are then filtered in the Fourier domain using a frequency and orientation-selective filter with parameters based on the estimated local ridge orientation and frequency. The outcome of the Fourier domain-based enhancement is another image denoted as . is multiplied with a skin-segmented binary image so that only the region of interest (ROI) is processed further.

Figure 5 shows the effects of this method on the derotated input image. The image in Figure 5(a) shows ridges that are not visible. The results of the unsharp masking can be seen in Figure 5(b) showing improved visibility of the ridges. Figure 5(c) shows the image once it has been converted to grayscale. Figure 5(d) shows that application of Fourier-based image enhancement further sharpens unclear edges and other details. Figure 5(e) shows the extracted ROI.

2.4. Image Binarization and Edge Thinning

Features such as ridge endings and bifurcations can be accurately extracted when ridges having a thickness of one pixel are detected. To detect thin ridges, all ridges are first detected by applying Otsu thresholding [34] to the enhanced region of interest. This will result in a binary image with ridge pixels represented in white and the remaining pixels in black. However, this will result in the detection of thick ridges. To limit the results to thin ridges, nonmaximal suppression [35] is applied. This method sets all the ridge pixels in a certain direction to zero if they do not have a maximum gradient value and retains ridge pixels with the maximal value. This results in conversion of the thick ridges to ridges with a thickness of only one pixel. Figure 6(a) shows Otsu thresholding being applied to the enhanced region of interest. The detected ridges are thick and thus cannot be used for feature detection. Application of nonmaximal suppression results in more desirable thin ridges, as seen in Figure 6(b).

2.5. Minutiae Extraction

Minutiae are features that are used for biometric verification. Instead of storing fingerprint images in the database, minutiae are stored and compared. The most common method used for minutiae extraction is the crossing number (CN) [36].

For extracting minutiae, the 3 × 3 window shown in Figure 7 is moved across each pixel to calculate its crossing number using the following equation:where denotes the pixels around the center pixel at location . Using , the ridge pixel can then be classified as a ridge ending, bifurcation, or isolated point using Table 1. Figure 6(c) shows bifurcations and ridge endings represented by blue and red squares, respectively.

For each bifurcation and ridge ending, the Euclidean distance between its spatial coordinate and , the normalized gradient value , and the gradient orientation are stored in and , respectively, for storage or comparison. Hence, every bifurcation and ridge ending point is stored in or in the form of 3D point and , respectively. The values donated by and are x-axis and y-axis points taken from two ROI points corresponding to the minimum x-axis and y-axis values. Figure 8 shows an example of calculation of that is represented by a green circle. This was determined using the minimum x-axis ROI point represented by a blue cross and the minimum y-axis ROI point represented by a red cross.

2.6. Fingerprint Registration and Matching

Extracted minutiae are either stored or compared with minutiae already existing in the database as required by the use case. Suppose that the ridge endings extracted from the input image are represented by the set and that bifurcations are represented by as follows:where and are sorted in an ascending order according to the calculated Euclidean distance such that and for all and . These sets must be compared with the stored features of a number of fingerprints stored in the database, as follows:where

and are also sorted in an ascending order according to the Euclidean distances and where and . Suppose that and are to be compared with the stored features of the ith fingerprint having feature sets and . First, , , , and are to be determined using the following equations:where and are the number of ridge endings extracted from the input image and the ith image features stored in the database and and are the number of extracted bifurcations. The input fingerprint features are compared with the ith fingerprint features if the condition in (19) is justified:

For matching, all 3D bifurcation and ridge ending points of the input fingerprint and the ith fingerprint from the database and input fingerprint are stored in , , , and respectively. is compared with , and is compared with . The steps for matching are(a)If has fewer bifurcations, store in and in and vice versa. Initialize bifurcation matching score to zero.(b)Compare each bifurcation 3D point in at the position with all bifurcation points at locations in where .(c)If is true for any value of t, then increment and move to the next bifurcation point for matching. Here, and are the magnitude and orientation of the normalized gradient of bifurcation of at position p, and and are the magnitude and orientation of at any of the eleven locations in Step b.(d)Repeat Steps a–c, comparing ridge endings and updating to track the number of ridges matched.(e)If , two fingerprints are said to be matched.

3. Results and Discussion

MATLAB 2018a was used in this study. A value of was used, and a Gaussian window of size , and was used throughout the work. There was no dataset available for evaluation purposes, so features from the fingerprints of 40 people were stored in a database. Features obtained from digital images of the fingers of 100 people, including those of registered people, were compared with the features already stored in the database in order to evaluate the performance of the proposed algorithm. Every person provided eight images of their finger at approximately , , , , , , , and .

The performance of the proposed method was evaluated by analyzing the performance of the derotation and matching methods. It was not possible to calculate the exact errors in the calculation of the rotation angles, so error calculation was done manually for ten random inputs for each set of rotation angles.

According to this manual evaluation, the errors fell in the range from to for all cases. The error range for the angles , , and was from to . For a rotation angle of , the error was approximately .

True positives, true negatives, false positives, and false negatives were calculated for different values of the matching threshold . True positives, true negatives, false positives, and false negatives are given in (20)–(23) as

Figure 9 shows a comparison of the percentages of true matches (TP) and mistaken mismatches (FN). It can be seen from Figure 9 that a value of successfully verified all 40 of the registered fingerprints irrespective of their angle of rotation. However, for values of , mistaken mismatches occurred in the range from . True mismatches (TN) occurred in 100% of the cases, and mistaken matches (FP) occurred in 0% of all cases.

4. Conclusions and Future Work

This paper proposes a fingerprint detection and verification method that uses a digital image of a finger as an input instead of a sensor. The input fingerprint image can be of any resolution and depends on the quality of the digital camera. Often, the finger in the image is not vertically aligned. Therefore, the input image is derotated by applying skin color-based binary segmentation, followed by downscaling or upscaling of the input image with a scale calculated using the segmented input image and a reference image. The angle of rotation of the input finger is then calculated by applying SURF-based feature extraction on both binary images. The features detected in both images are matched, and outliers are removed from the matched points using the M-estimator. The angle between the matched inliers is calculated using a homography matrix, and the input image is derotated by the calculated angle. After derotation, enhancement techniques including unsharp masking and Fourier-based blockwise enhancement are applied to increase the visibility of the ridges in the input image. Image enhancement is followed by the application of Otsu thresholding to segment the ridge regions, and then nonmaximal suppression is applied for edge thinning. A crossing number technique is used to extract ridge endings and bifurcations. This is followed by the proposed Euclidean distance-based matching to detect a fingerprint match or mismatch based on a matching score. The proposed method is a novel method, and simulation results show that if the matching threshold is kept within a suggested range, accuracy of 100% can be achieved.

Scar removal-based fingerprint detection using digital images and edge-preserving image interpolation techniques could extend the proposed method and is being considered for future work.

Abbreviations

STFT:Short-time Fourier transform analysis
ROI:Region of interest
SURF:Speeded up robust features
M-estimator:Maximum-likelihood estimator
CN:Crossing number
TP:True positives
TN:True negatives
FP:False positives
FN:False negatives.

Data Availability

The data used to support the findings of this study can be obtained by contacting the corresponding author through e-mail: [email protected].

Conflicts of Interest

The authors declare that they have no conflicts of interest.

Authors’ Contributions

Sajid Khan performed all the simulations and worked on the solution of most of the problems under the supervision of Dong-Ho Lee. Asif Khan provided helped in finalization of the content of manuscript and writing of the manuscript. Ahmad Waqas and Abdul Rehman Gilal collected the fingerprints for experiments and validation. Zahid Hussain Khand helped in the understanding of mathematical models and equations that are used in this research.