Abstract

The security problems of online transactions by smartphones reveal extreme demand for reliable identity authentication systems. With a lower risk of forgery, richer texture, and more comfortable acquisition mode, compared with face, fingerprint, and iris, palmprint is rarely adopted for identity authentication. In this paper, we develop an effective and full-function palmprint authentication system regarding the application on an Android smartphone, which bridges the algorithmic study and application of palmprint authentication. In more detail, an overall system framework is designed with complete functions, including palmprint acquisition, key points location, ROI segmentation, feature extraction, and feature coding. Basically, we develop a palmprint authentication system having user-friendly interfaces and good compatibility with the Android smartphone. Particularly, on the one hand, to guarantee the effectiveness and efficiency of the system, we exploit the practical Log-Gabor filter for feature extraction and discuss the impact of filtering direction, downsampling ratio, and discriminative feature coding to propose an improved algorithm. On the other hand, after exploring the hardware components of the smartphone and the technical development of the Android system, we provide an open technology to extend the biometric methods to real-world applications. On the public PolyU databases, simulation results suggest that the improved algorithm outperforms the original one with a promising accuracy of 100% and a good speed of 0.041 seconds. In real-world authentication, the developed system achieves an accuracy of 98.40% and a speed of 0.051 seconds. All the results verify the accuracy and timeliness of the developed system.

1. Introduction

Mobile Internet technologies have brought humans a new lifestyle. The emergence of smartphones has better accelerated people’s daily life with popular social networks such as online transactions [1]. However, frequent online transactions by smartphones often bring security problems. Specifically, to facilitate the payment, the “mobile wallet” should be linked with a cash card. Then, phenomena like Internet fraud and network “phishing” begin to spread extensively, which leads consumers to be still distrustful of the reliability of mobile payment. Moreover, when people trade online, they have to set different complex passwords on various occasions, which may be forgettable. These facts indicate that mobile payment still demands for reliable mechanisms extremely [2]. Over the last years, fingerprint authentication has drawn significant attention. However, fingerprint can be easily damaged due to physical work and is not informative enough for identity authentication. For iris authentication, users have to suffer from an uncomfortable acquisition mode.

Recently, many researchers focus on face biometrics. Nevertheless, face information is always exposed to the environment and can be easily stolen for identity forgery. As for hand geometry biometric, its physical uniqueness needs further discussion [1]. All in all, with the advantages of abundant textures, comfortable acquisition mode, and lower risk of forgery, palmprint is more acceptable to users in some situations, compared with the other biometrics [1].

The earliest palmprint authentication system was developed on an embedded device by Zhang and Shu team in 1999. They acquired palmprint by scanning the printed palmprint templates [3]. The offline operating mode renders the system not only time-consuming but also unacceptable to users. To make improvement, in 2003, they utilized CCD to acquire images and set pegs to restrain the palm position. However, the improved equipment is immovable for its large size and still unacceptable for its contacting acquisition mode [4]. In 2010, Zhang et al.’s team further developed a new palmprint device with a smaller size compared with the previous ones, but this device remains to suffer from the contact-based acquisition mode [5]. In 2015, Aykut and Ekinci developed a contactless palmprint authentication system that obtained good usage experience whereas it is inconvenient for mobile applications. In summary, the existing palmprint authentication systems are still not mature enough for general application and deserve in-depth research. Table 1 presents a total comparison among the abovementioned systems.

From Table 1, the contact-based palmprint authentication systems are unacceptable for the uncomfortable acquisition mode, while the contactless systems are not robust to the real disturbance [7]. Besides, the widely adopted feature extractor, Log-Gabor filter, in these systems has a high computation cost, which can greatly reduce the timeliness of the system [8]. Furthermore, the adopted Quadrant coding is unsuitable to make full play of the discrimination of the extracted features [9]. In current years, the Android operation system-based smartphones have occupied 76% of the smartphone market in the world. So, it is meaningful to further study an efficient contactless palmprint authentication system with the application on the Android smartphone.

According to the aforementioned analysis, we design an overall system framework with complete functions for palmprint authentication. Specifically, a vital acquisition scheme is established to ensure the image quality in the contactless acquisition process. Some improvements are introduced to the Log-Gabor filter and the original coding method to reduce the computational complexity and enhance the discriminability of the coding results. On the basis, a system with user-friendly interfaces and good compatibility is developed for the application on the Android smartphone.

Most systems only focus on the palmprint authentication method. Actually, as the primary work, it is crucial to establish an acquisition scheme. Thus, the image quality and accurate ROI segmentation can be ensured for more reliable identity authentication. Besides, the authentication algorithm is the core issue of our system. Considering the practicality and effectiveness, the Log-Gabor filter is the most suitable one for our system. However, to guarantee the timeliness of our system, we should consider reducing the computational complexity of the Log-Gabor filter. Moreover, a proper coding method should be exploited to enhance the discriminability of the extracted features, which helps the system distinguish one class from the others more easily. Finally, all the procedures should be programmed into modules and linked to the well-designed functional interfaces to facilitate the operation. Depending on the above motivations, the overall system framework proposed in this paper is displayed in Figure 1. Table 2 lists all the involved evaluation parameters and their definitions in advance. The main contributions of our work are summarized as follows:(i)A palmprint acquisition scheme is proposed against the potential disturbances such as the variation of palm posture and illumination. Then, we present a complete set of procedures for palmprint ROI extraction, such as key points location, palmprint angle correction, and the final ROI segmentation.(ii)To guarantee the efficiency of the system and obtain more discriminative feature coding results, we discuss the impact of filtering direction and downsampling to the performance of Log-Gabor filter and improve the original coding scheme [10, 11].(iii)A palmprint authentication framework with complete functions is designed. We explore and show the details of developing the corresponding system with user-friendly interfaces and good compatibility with Android smartphones.(iv)Our work provides an open technology to extend the palmprint authentication algorithms to real-world applications, which bridges the algorithmic studies and applications of biometrics.

The rest of the paper is organized as follows: Section 2 defines an acquisition scheme and provides a method for palmprint ROI extraction. Section 3 describes the improvements for palmprint authentication. Section 4 introduces the hardware components of the smartphone and the software realization of the palmprint authentication system. Section 5 performs experimental verifications. Section 6 concludes this paper.

2. Palmprint Acquisition and ROI Extraction

2.1. Acquisition Scheme

In our system, ROI segmentation is finished according to three key points among fingers. However, palm gesture and the position of the palm can affect the key point locating [12, 13]. Taking some cases into consideration. The palm deformation often produces visional textures. The large gaps among fingers can lead to imprecise locating results. A long distance between palm and camera can lead the acquired image to have too much background. All of these potential disturbances will bring difficulties for key points locating. So, it is necessary to define an acquisition scheme.

Considering the aforementioned potential disturbances, the acquisition scheme in this paper is determined as follows: five fingers extended and the other four fingers close to each other except for the thumb. A red line and a center point are marked on the interface to conduct the position of the finger root and palm center.

2.2. Key Point Location
2.2.1. Image Graying

The original palmprint image (540 × 600) shown in Figure 2(a) is converted into a gray image shown in Figure 2(b). To cut down the searching range, the upper portion (270 × 600) of the gray image is reserved, shown in Figure 2(c).

In the reserved subimage, the gaps between fingers root are deemed as the key points and denoted as , , and , respectively, as shown in Figure 2(c).

2.2.2. Columns of Key Points

The average gray value versus the column of the reserved subimage can be calculated bywhere and denote the gradient versus column and row, respectively. denotes the pixel coordinate. The three local minima are shown in Figure 3, which correspond to the column indexes of the three key points.

2.2.3. Rows of Key Points

The three subimages (100 × 20) containing three key points are, respectively, cropped from Figure 4(a) shown in Figure 4(b). For each subimage, the row index of the key point can also be obtained according to formula (1). The average gradient value versus the rows of the first subimage is shown in Figure 5. The X-coordinate of the maximal value corresponds to the row index of the key point.

2.3. ROI Segmentation

The acquired images often contain redundant information such as fingers and background. Palmprint ROI refers to the palm center that contains three flexor lines [14, 15]. Here, we segment ROI according to the above located key points.

Since the size of a smartphone screen is generally different from the size of a palmprint image and there exists rotation between them, it is necessary to correct the scale and angle of the palmprint image before ROI segmentation.

2.3.1. Scale Correction

The key point lying in the original image with a size of is denoted as . The corresponding position in the smartphone interface with a size of is denoted as . Then, the transformation can be defined as

2.3.2. Rotation Correction

The purpose of the rotation is to place and on a horizontal line to facilitate ROI segmentation. The calculation of the rotation angle is shown as follows:where and are the coordinates of and , respectively. The corresponding coordinates after rotation are denoted as . We define the rotation transformation as follows:where reserves the integer value, while the line is set as the x-axis, and its vertical line is set as the y-axis. Then, the coordinate system is established in the palmprint image. The length of is denoted as . Moving it down alongside the y-axis by , a square is built up with the side length of , as shown in Figure 6(a). To obtain a standard ROI, we scale the segmented square to a size of 128 × 128, shown in Figure 6(b).

3. Palmprint Authentication

In experiments, we find that image preprocessing has a little effect on ROI segmentation results. To shorten the running time of the system, we perform image preprocessing after ROI segmentation.

3.1. Image Preprocessing
3.1.1. Gray Enhancement

The pixel value of the original ROI image is denoted as , and the pixel value after the preprocessing is denoted as . The gray enhancement is defined as [16]where ,,, and are four given constants, according to which the original pixel value between is then transformed between .

3.1.2. Gray Normalization

To balance the uneven illumination, we normalize the gray image by the following formulas [17]:where and signify the mean function and the variance function, respectively. The preprocessed ROI images are shown in Figure 7(c), from which we observe that the three flexor lines and the surrounding details are more obvious and abundant.

3.2. Discussion on Log-Gabor Filter

The three principal lines in palmprint are not sufficient to signify the uniqueness of individuals. So, the feeble wrinkles around them should be amplified to provide more valuable information. What is more, direction information in palmprint should be characterized. The Log-Gabor filter is capable of supporting us to extract such a kind of features. By setting various factors and filtering directions, we can obtain more representative abundant features through the Log-Gabor filter [18, 19]. Feature coding is the other important process, in which a more discriminative feature coding method helps to further improve the performance of the system.

Note feature extraction corresponding multiple factors and directions can result in a large computation. To guarantee the timeliness and the effectiveness of our system, we will take the discussion on the original Log-Gabor filter and feature coding method (we call the original algorithm in the later content).

3.2.1. Log-Gabor Filter

The transfer function of the Log-Gabor filter in the linear frequency domain is defined as [20]where and denote the central frequency domain and the bandwidth, respectively. To maintain the shape of the filter, the ratio is usually fixed [21]. The output of the Log-Gabor filter is related to the filtering direction. Here, we set to discuss the influence of filtering direction to feature extraction results. The scale wavelength is set as 3, and the ratio is fixed by . The filtered results are shown in Figure 8. When , textures in all directions are well characterized except for the horizontal textures. When , textures in direction 135° is mostly represented. When , the horizontal textures are the clearest. When , textures in direction 45° are clearly represented. Thus, we conclude that the extracted textures are more representative when the filtered direction is perpendicular to texture.

3.2.2. Feature Extraction and Coding

Since the distribution of palmprint textures is complex, we set to cover the most directions. In addition, five scale factors, say , are chosen in each direction to extract more abundant information, with . Then, a feature matrix with the dimension of 4 × 5 × 128 × 128 = 327680 is obtained by the original algorithm, which leads to a computation disaster [7]. To this end, we bring some improvements to the Log-Gabor filter.

In detail, the ROI image and extracted features will be downsampled to get rid of the redundant information while reducing the computational time. To obtain distinguishable coding results for the final identity matching, we adopt the Gray code to replace Quadrant code to enhance the discriminability of features. The specifics are shown as follows:Step 1: ROI image (128 × 128) is downsampled to the size of 64 × 64. The pixel value before and after filtering is denoted as and , respectively. denotes the Log-Gabor filter with different filtering directions. The filtering process (feature extraction) is formulated as follows:Step 2: the features are downsampled with a ratio of 1 : 2 to obtain a feature matrix with a dimension of 4 × 32 × 32. Let and denote the pixel value before and after the feature downsampling. The feature downsampling can be performed by the following formula:Step 3: according to the real part and imaginary part of the features, Quadrants coding achieves two kinds of coding results, while Gray coding achieves four kinds of coding results. Hence, Gray coding is conducive to enhance the discriminability of features. Denoting the coding result as , then Gray coding can be defined as follows:

All the above steps are repeated regarding five scales to obtain a matrix with a dimension of 4 × 5 × 32 × 32 × 2 = 40960. The flow chart of the improved algorithm is shown in Figure 9.

3.2.3. Feature Matching

Hamming distance is exploited for feature matching to judge whom the acquired palmprint image belongs to. By the bitwise XOR operation on two feature vectors, the rate of the number “1” is defined as Hamming distance. If the samples come from the same user, the Hamming distance can be small enough. Otherwise, the Hamming distance can be large. The calculation of Hamming distance is defined aswhere and denote the th elements in two vectors and denotes the vector dimension. Theoretically, when and are exactly the same, . Otherwise, . There exist no palmprint images that are totally the same or different in practice, so generally . The final authentication is achieved by comparing the Hamming distance with the predetermined matching threshold.

4. System Development

4.1. Structure of the Hardware and Software
4.1.1. Hardware

In recent years, Android smartphones have occupied a large market share in the world. Among the most mainstream models in 2018, HUAWEI P20 is the most popular one that carries a host processor HiSilicon Kirin 970, 6 GB RAM, 128 GB ROM, and a 6.1-inch touchscreen with a resolution ratio of 2240 × 1080. Its operating system is Android 8.1. Among these, the camera and GPU are the key components. Four mainstream brands released in the same period with HUAWEI P20 are listed in Table 3 [22].

For the CMOS camera, the larger the camera numbers and the pixel size, the more information can be captured. The smaller the aperture size, the more light could be passed to enhance the image. What is more, G72MP12 is more powerful than Adreno540 and 418 for image processing. After the synthetic comparison, HUAWEI P20 is chosen as the platform of our system. The main hardware structure of HUAWEI P20 is shown in Figure 10.

4.1.2. Software

According to the designed framework in Figure 1, the system development can be finished through three steps: middle layer development, application layer development, and Android APK generation [23]. The entire development structure is displayed in Figure 11. The details are described as follows:(i)Underlying layer development: in the QT environment, we first develop executable files to call the hardware of the smartphone to finish each procedure in the designed framework. Procedures, like palmprint acquisition, ROI segmentation, feature extraction, feature coding, and feature matching, are all separately programmed into executable functional files by C++ language.(ii)Application layer development: using Android SDK, we develop interfaces to call the executable functional files. Through user-friendly interfaces, the user can operate the system in real time. The development in this stage is finished in Eclipse Java EE environment.(iii)Android APK generation: we use NDK to compile the functional files developed in the underlying layer. The compiled files are packaged with the JNI standard to facilitate the callback from the application layer. Then, dynamic link libraries are generated and packaged into an APK to install on the smartphone.

4.2. System Function Design

Activity is the functional unit in the application layer. Since an activity occupies a certain part of memory, to save the resource, we integrate the interrelated functions into one activity. It is also convenient for us to debug the code and maintain the software. Our APK is composed of four activities: main activity (main interface), database activity (database interface), authentication activity (authentication interface), and acquisition activity (acquisition interface).

4.2.1. Main Activity

The main activity provides entrances for the other activities. Additionally, the initialization and storage of the database are finished in this activity. The developed main interface is shown in Figure 12(a).

4.2.2. Database Activity

Users finish identity registration in this activity. They can append their identity information by clicking “Add” button and input their palmprint images for the later authentication (5 samples are necessary) by clicking “INPUT SAMPLES” button. Features extraction is finished by clicking “DATA GENERATION” button. Meanwhile, we design “Delete” and “Update” buttons to facilitate database management. The database interface is shown in Figure 12(b).

4.2.3. Authentication Activity

Identity authentication is achieved in this activity. After choosing the identity to be confirmed in the identity list, users can click the “Browse” button or “Take photo” button to select the test sample. When clicking the “Start” button, the system will display authentication result, elapsed time, and encourage users to evaluate the authentication result. Users can examine the correct authentication number and the accuracy by clicking the “Accuracy” button. Also, they can click the “ROI Compare” button to check the difference between two segmented ROI images. The authentication interface is shown in Figure 12(c).

4.2.4. Acquisition Activity

When users enter the real-time acquisition, the rear camera is driven to work. Two modes, “long press” and “click,” are designed for the “Take” button. When it is long pressed (about 2 seconds), the located key points and ROI are displayed on the screen. Tips for distance and angle adjustment are given on the top to guide the acquisition. Users can click the “Take” button (about 0.5 seconds) to finish the final acquisition. The developed acquisition interface is shown in Figure 12(d), where the red line and point are set to conduct the palm position.

5. Results and Discussion

5.1. Verification of the Improved Algorithm
5.1.1. Statistical Analysis

The blue spectrum samples in the PolyU multispectral palmprint database containing 6000 samples from 500 volunteers are chosen to verify the improved algorithm [18]. 12 samples of each volunteer are divided into two equal parts. The first part is regarded as the training samples, while the second part is used for the test.

Each volunteer is tested 6 times, and the total intraclass test times reach 3000. For interclass verification, test samples of each volunteer are, respectively, utilized to match each training sample of a selected volunteer. The minimum Hamming distance is deemed as the final matching result. If it is less than the threshold, they are regarded as the same class. Otherwise, they are regarded as different ones [24].

In our simulation, the intraclass Hamming distances distribute in interval [0.1317, 0.2265], while the interclass Hamming distances distribute in interval [0.3118, 0.5271]. The distribution density curves are shown in Figure 13.

Based on the above discussion, AA, FRR, and FAR are brought to evaluate the improved algorithm. We set the threshold , and the step length is 0.01. The FAR and FRR threshold curves are drawn in Figure 14. It can be seen that the FAR and FRR curves have no intersection, which means EER is 0. Note Figure 14 is more intuitive for us to find the optimum threshold interval, compared with the ROC curve [20].

A comprehensive comparison between the improved algorithm and the original one is listed in Table 4. The evaluation items include the intraclass and interclass Hamming distance, indicator L, feature dimension, and the authentication time. For Hamming distance and indicator L, compared with the original algorithm, the improved algorithm presents more obvious separation between the intraclass and interclass Hamming distance distributions, which greatly reduce the ERR. In addition, we find that the feature dimension is reduced by 6.25%. This guarantees the efficiency of our system.

5.1.2. Comparison with the Other Biometrics

Apart from palmprint, there are some other biometrics that can be used for identity authentication. This subsection carries out a comparison among the proposed system and some other biometrics-based methods. We particularly consider the new emerging biometrics, EEG [25] and ECG [2628]. The comparison results are shown in Table 5.

The collected data in Table 5 indicates that the methods based on the new biometrics all achieved promising accuracy. Recent years have witnessed the great application value of the EEG and ECG biometrics based on the brain wave and heartbeat pattern [25, 26]. Meanwhile, the good performance presented in [2528] also manifests the great potential of the deep neural networks to learn valuable pattern information in the complicated biometric signals. All of these facts motivate us to further study identity authentication using the new biometrics by deep neural networks.

5.1.3. Complexity Analysis

As is introduced in Subsections 3.2.2 and 3.2.3, given an ROI image , the entire authentication procedure includes three stages, that is, feature extraction, feature coding, and feature matching [29, 30]. The first stage involves image downsampling and Log-Gabor filtering. The computational complexity of image downsampling is N × M. After that, the image size becomes (N/2) × (M/2). According to formula (8), the filtering process should be repeated along 4 filtering directions, respectively, with 5 different scale factors [31, 32]. Hence, its computational complexity is 4 × 5 × (N/2) × (M/2). So, the operation complexity of the feature extraction stage is N × M + 5 × N × M. The feature coding stage involves feature downsampling and coding. Formula (9) indicates that the computational complexity of feature downsampling is (N/2) × (M/2). Then, the feature dimension becomes (N/4) × (M/4). From formula (10), we know the coding process has a complexity of 2 × (N/4) × (M/4). Thus, the feature coding stage has an operation complexity of N × M/4 + N × M/8. Finally, formula (11) indicates the operation complexity of the feature matching stage is 2 × (N/4) × (M/4). In summary, the total operation complexity of the entire authentication procedure is (N × M + 5 × N × M) + (N × M/4 + N × M/8) + N × M/8 = 13NM/2.

5.2. Verification of the Developed System
5.2.1. Test of Palmprint Acquisition

To demonstrate the effectiveness of the defined acquisition scheme, we acquire the palmprint samples during a random period in one day and randomly rotate the palm within a finite angle range [33]. The palmprint acquisition interface is shown in Figure 15.

Under the constraint of the acquisition scheme, most potential disturbances are suppressed. Meanwhile, with the conduction of the set marks, palm appears on the screen with a proper scale. After long pressing the “Take” button (about 2 seconds), key points locating and ROI segmentation are all finished. The side length and rotation angle of ROI are calculated and displayed on the top to suggest users to make the final acquisition.

5.2.2. Test of Palmprint Authentication

We recruited 500 samples acquired from 50 volunteers in the real world to verify the effectiveness of the developed system. At the beginning, each volunteer was required to register identity with 5 training samples. Then, they are required to implement 5 times intraclass real-time authentication and 5 times interclass real-time authentication, respectively. Since the palmprint database is established in real application [34], the variations of palm posture, background, and illumination commonly exist in the samples. Here, we set the threshold as 0.30. Specifically, after selecting the identity to be confirmed from the identity list, users can click the “Take photo” button to finish real-time acquisition for the test or click the “Browse” button to choose a previously acquired sample for the test. The system will display the storage path of the selected sample. The intraclass and interclass authentication operations are shown in Figures 16(a)16(d), respectively.

Users can click the “Start” button to trigger authentication. Then, the system will display authentication result and running time and encourage the user to make an evaluation. Additionally, we can check ROI of the test sample and training sample by clicking the “ROI compare” button. We can also view the total authentication times, correct authentication number, and authentication accuracy, by click the “Accuracy” button, as shown in Figure 17(c).

After a 250-time test, we find that the authentication accuracy of the developed system is close to that in MATLAB simulation. Due to the unavoidable disturbances in the acquired sample, the authentication accuracy in real-time acquisition is reduced to 98.40%. Note the authentication in MATLAB simulation takes about 0.0278 seconds for average, while it is 0.051 seconds on the smartphone. All the test results on MATLAB and smartphone are listed in Table 6.

Meanwhile, we find that if the acquisition process follows the defined constraint scheme, we can obtain more promising accuracy. Otherwise, the accuracy will be impacted. This indicates the importance of the established acquisition scheme. We also find that when the light intensity changes within a limited degree, the authentication accuracy can be nearly not reduced, which demonstrates that our system shows robustness to some extent.

6. Conclusion

In recent years, there emerges an extreme demand for reliable identity authentication system for secure mobile transactions. This paper presents an effective and efficient palmprint authentication framework towards the application on the Android smartphone. It contains all the required functions during palmprint authentication, including palmprint acquisition, ROI segmentation, feature extraction, and the final identity authentication. More importantly, our method provides an open technology to extend the palmprint authentication algorithms to real-world applications by exploring the development of a user-friendly palmprint authentication system based on the Android smartphone, which well bridges the algorithmic study and the real-world application of palmprint authentication.

This paper presents our exploratory work of propelling the application of palmprint authentication on the mobile smart terminals. Considering the timeliness of the system, we choose the practical and effective Log-Gabor filter for feature extraction. However, when the palmprint is subjected to corruption or occlusion, the Log-Gabor filter is unable to extract robust features anymore. In the next stage, we will manage to enhance the robustness of our system concerning more complicated application scenarios. Furthermore, we will also consider achieving identity authentication using the new emerging EEG and ECG biometrics with the powerful deep neural networks.

Data Availability

The adopted public PolyU multispectral palmprint database was downloaded from http://www.comp.polyu.edu.hk/∼biometrics/MultispectralPalmprint/MSP.html.

Disclosure

This paper is an extended version of an earlier conference paper: Research of Palmprint Authentication Arithmetic Based on Smartphone, in proceedings of the 2018 VII International Conference on Network, Communication and Computing, Taipei City, Taiwan, Dec. 14–16, 2018

Conflicts of Interest

The authors declare no conflicts of interest.

Acknowledgments

This work was supported in part by the National Natural Science Foundation under Grant 61673316 and in part by the project commissioned by the Sichuan Gas Turbine Research Institute of AVIC.