Abstract
Recent research proves that face recognition systems can achieve high-quality results even in non-ideal environments. Edge detection techniques and feature extraction methods are popular mechanisms used in face recognition systems. Edge detection can be used to construct the face map in the image efficiently, in which feature extraction techniques generate the most suitable features that can identify human faces. In this study, we present a new and efficient face recognition system that uses various gradient-and Laplacian-based operators with a new feature extraction method. Different edge detection operators are exploited to obtain the best image edges. The new and robust method based on the slope of the linear regression, called SLP, uses the estimated face lines in its feature extraction step. Artificial neural network (ANN) is used as a classifier. To determine the best scheme that gives the best performance, we test combinations of various techniques such as (Sobel filter (SF), SLP/principal component analysis (PCA), ANN), (Prewitt filter(PF), SLP/PCA, ANN), (Roberts filter (RF), SLP/PCA, ANN), (zero cross filter (ZF), SLP/PCA, ANN), (Laplacian of Gaussian filter (LG), SLP/PCA, ANN), and (Canny filter(CF), SLP/PCA, ANN). The BIO ID dataset is used in the training and testing phases for the proposed face recognition system combinations. Experimental results indicate that the proposed schemes achieve satisfactory results with high-accuracy classification. Notably, the combinations of (SF, SLP, ANN) and (ZF, SLP, ANN) gain the best results and outperform all the other algorithm combinations.
1. Introduction
Face recognition system has been widely used in the biometric technology area. The increasing commercial use of this system has increased its robustness and efficiency in the recent years. Furthermore, the development of face recognition algorithms in hardware systems has led to the relatively easy application and wide distribution of these systems. Subsequently, such technologies have been implemented in many system applications, such as face verification, face identification, surveillance, and monitoring system.
Face recognition has been studied in many works and research projects due to its importance. Reference [1] proposed face recognition based on polar fast Fourier transform. In this work, the face was detected using the Viola-Jones algorithm [2], whereby the image was converted into a greyscale image with a small size of 120120 pixel. Discrete Fourier transform was applied to the preprocessed image, and the dominant value of 2D-DFT quantities was calculated using polar Fourier transform method and characterized as 1D P-FF. A magnitude that denotes maximum value in 1D PFFT was considered a feature value. This extracted value was utilized to construct a symbolic object for epitomizing face images. The performance of the proposed scheme was tested using the AR database [3], and an average accuracy of 96.25% was achieved.
Reference [4] introduced another face recognition system using 1D transform domains. The authors benefitted from the combination of discrete wavelet transform and fast Fourier transform for feature extraction. 1D-DWT and 1D-FFT were compared using probe images and a database. The Euclidian distance was used to calculate the performance parameters, and matching processes were conducted to obtain the best results.
Reference [5] put forward the local appearance-based face recognition by using adaptive directional wavelet transform on the edges of extracted faces. The authors reported the importance of this algorithm by comparing this technique with efficient feature extraction algorithms such as linear discriminate analysis and existing non-adaptive multi-resolution. The popular Yale database was used to reveal the effectiveness of the adaptive directional wavelet transform-based subspace feature. The algorithms showed acceptable results compared with other face recognition algorithms.
Reference [6] established a new algorithm by modifying the eigenphase method for face recognition based on support vector machine (SVM). The authors performed two phases to increase accuracy. The first one was the local spatial domain phase, in which face images were segmented into equal size blocks. These blocks were concatenated as preparation for phase spectrum estimation. In the second phase, which was local frequency domain eigenphase, a similar special procedure was performed but in the frequency domain. The output of the two phases was fed to the SVM for classification. Accordingly, this algorithm could work better than the other compared algorithms in time and performance.
In [7], the authors proposed a new algorithm for face recognition using two feature extractors PCA technique and Wavelet decomposition vectors. This algorithm uses also two classifiers which are nearest distance and support vector machine. The proposed system was tested using the ORL database [8] and the experimental results showed that the new system enhanced the performance significantly. In another work, the authors suggested using a neural network instead of SVM to more enhance the performance.
Reference [7] proposed a new algorithm for face recognition using two feature extractors, namely, principal component analysis (PCA) technique and wavelet decomposition vectors. This algorithm used two classifiers: nearest distance and SVM. The proposed system was tested using the ORL database [8]. The experimental results showed that the new system enhanced the performance considerably.
Other papers have discussed face recognition issues using several techniques. Some works have depended on the appearance of the face only and used different combinations with the PCA algorithm (e.g. [9–11]. [12] proposed the Fisher face methods, which depended on the appearance of the face and was based on Fishers linear discriminant analysis. Other studies have focused on the geometry of the face and the relation between parts of the face and its properties such as nose, mouth, and eyes. Reference [13] used 16 and 30 face geometry features, respectively, and fed them into a classifier algorithm for classification. Reference [14] extracted the geometry of human face in detail using a 3D representation model. In many fields, researchers have tried to make hybridisation models between the appearance and geometry of the face, e.g., [15, 16]. Most of these approaches and techniques have limitations in terms of time for extracting features, the response time for the training dataset, and an update of the training dataset and hardware requirements.
This paper proposes a robust face recognition approach regardless of face image quality and illumination conditions. In this study, two groups of edge detection methods are used with the slope of the linear regression (SLP) operator to enhance the performance of the face recognition system. Edge detection methods include gradient-based operators (GOs) and Laplacian-based operators (LOs). The GOs include Sobel filter (SF), Prewitt filter (PF), and Roberts filter (RF). The LOs include Laplacian of Gaussian (LoG) filter (LF), Zero cross filter (ZF), and Canny filter (CF). For feature extraction, a new method called SLP is suggested. The proposed SLP uses a simple regression technique with the eigenfaces based on PCA. Artificial neural networks are utilized for classification. The BIO ID database [17] is used in our empirical experiments. This database includes 23 subjects with 1,502 face images. The results show that the proposed (SF, SLP, ANN) method can enhance the performance of a face recognition system and gains high-accuracy classification.
The rest of the paper is organized as follows. In Section 2, we briefly review the edge detection methods that will be used in this study, namely, PF, SF, RF, ZF, LF, and CF. Section 3 explains our proposed algorithm that includes the segmentation process, feature extraction methods, and ANN. Section 4 presents the experimental results and detailed discussion. Finally, Section 5 provides the conclusion and future work.
2. Edge Detection Algorithms
Edge detection is a fundamental step towards understanding image features because edges consist of feature data and contain important information related to face edges. Most images contain a certain amount of redundancy that can sometimes be deleted when edges are detected and replaced during reconstruction. This gap is where edge detection comes into play. The edge detection process aims to accurately detect the image edges, reduce the image size, and remove information that may be regarded as less relevant. In the proposed system, the goal of the edge detection operation is to construct a map of the face from the edges that can be used to extract features of face objects and prepare the features for the subsequent stages of the proposed system. We apply most of the edge detection filters (SF, PF, RF, CF, LF, and ZF) on the dataset images to perform this process and study the differences amongst the image edges [18, 19].
2.1. Sobel Filter
The SF is used to respond maximally to image edges that change in vertical and horizontal directions. Two masks are used in the Sobel operator. One mask works in the vertical direction and the other mask is in the horizontal direction of image pixels. These kernels of the SF can be used separately to obtain isolated estimations of the gradient in horizontal and vertical directions, Gx and Gy, respectively [20]. The two kernels can be joined together to find the value of the gradient at each pixel and the direction of the gradient as it can be seen in Figure 1. The following equations explain the magnitude and direction of the gradient.Typically, an approximate magnitude is computed using
which can be computed rapidly. The angle of orientation of the edge (relative to the pixel grid) giving rise to the spatial gradient is given by
2.2. Prewitt Filter
The PF works by convolving the image with a small, discrete, and integer-valued operator in horizontal and vertical directions. The algorithm ignores all edges that are not higher than the threshold. The threshold value is set automatically by the Prewitt algorithm. In the proposed algorithm, we use a range of threshold values (instead of standard value) to find the best edge pixels. In the Prewitt model, the image is treated using a kernel with eight values. All these values are sensitive to image edges in both directions. For each pixel, the local edge gradient magnitude is evaluated with the maximum reaction of all eight kernels at the considered pixel location, as shown in Figure 2. The gradient magnitude is given bywhere is the response of kernel at the specific pixel location and (n) is the number of convolution kernels. The local edge orientation is estimated with the orientation of the kernel that yields the maximum reaction [20, 21].
2.3. Roberts Filter
The RF applies an easy and rapid calculation called 2D spatial filter on an image. Regularly, the RF estimates the values (magnitude) of the gradient for every pixel point. The operator consists of a pair of 22 convolution kernels, as shown in Figure 3. One kernel is simply the other rotated by 90° [21]. The kernels can be applied separately to the input image to produce separate measurements of the gradient component in each orientation (Gx and Gy). Thereafter, both kernels can be combined to find the absolute magnitude of the gradient at each point and the orientation of that gradient. The gradient magnitude can be calculated by
The two components of the gradient are conveniently computed and added in a single pass over the input image by using the pseudoconvolution operator.
2.4. Zero Cross Filter
The zero cross edge detector determines zero crossing regions in images where the values of the Laplacian cross through zero regions. According to this operator, the edge point pixels are mostly represented by points that have a value of the Laplacian passing through zero. Hence, in these points, the Laplacian is changing its sign. For example, in face recognition algorithms, the points where the intensity of the face image edges changes suddenly are considered edge point candidates. Scientists have recommended dealing with this filter as a feature extractor operator because the output of the zero cross algorithm is usually a binary image with one-pixel thickness [22].
2.5. LoG Filter
The LoG algorithm is proposed to solve the noise sensitivity of the regular Laplacian filter. This algorithm removes noise by applying image smoothing using a Gaussian blurring technique to obtain the best performance. After applying the Laplacian method, a change in sign from positive to negative or vice-versa for any point (changes in sign) represents an edge. This algorithm supposes that important cross points will nearly be at edges in the images, that is, where the intensity values of the image regions change sharply. However, they can occur at places that are not as easy to allocate with image edges because some of these regions are very similar to one another, especially in greyscale images. The lines are detected by the LoG procedure with one-pixel thickness. The Laplacian of an image with pixel intensity values I (x, y) is given by
This step is an important pre-processing operation that decreases unwanted frequency noise prior to the differentiation operation. The LOG function is centered on zero and with Gaussian standard deviation in the following form:where x is the distance from the origin in the horizontal axis, y is the distance from the origin in the vertical axis, and is the standard deviation of the Gaussian distribution. The Gaussian smoothing filter and the Laplacian filter then convolve into this output result (hybrid filter) with the digital image to reach the desired output [22].
2.6. Canny Filter
Canny edge detection uses five different and sequential procedures to detect the edges. The first step is to smoothen the image to remove possible noise using a Gaussian filter. The second step is to compute gradient values in vertical and horizontal directions, which is the same procedure used in the Sobel algorithm. CF eliminates pixels that are not part of image edges by using a non-maximum suppression process. In the third step, threshold values are used to determine the potential edges. In the last step of the Canny algorithm, all edge pixels that have weak values or isolated from other pixels are eliminated and rejected [23, 24]. For further explanation, the mathematical model of the CF includes the following equations:where and represents the numbers of pixels between two points in horizontal and vertical directions.whereCompute gradient of g(i,j) using any of the gradient operators to getandIn the equations above, the threshold is chosen such that all edge elements are kept whilst most of the noise is suppressed.
All the previous edge detection filters are implemented and illustrated in Figures S1-S6.
3. Proposed Face Recognition Algorithm
The proposed algorithm consists of three main steps. In the first step, the face images are preprocessed with special filters, and one edge detection filter is selected and applied to the face image. This study uses six edge detection operators that will be tested with other combinations of our proposed algorithm. After that step, the resulting binary image from the edge detection method is segmented using the specified method, which will be explained later. In the second step, a feature extraction algorithm is applied to select the most important features of the segmented binary image. In this study, two feature extraction methods will be tested and used, namely, PCA and our new proposed SLP filter. In the last step, after extracting the features, all the gained features of faces will be used to train the artificial neural network for classification purposes. Figure 4 presents the steps of our proposed algorithm. This figure shows the steps of training the proposed method; the test steps are the same except that the ANN training step will not be conducted and the test dataset of face images will be used.
3.1. Segmentation
The idea behind dividing the face images into sub-images is inspired by the popular problem-solving technique Divide and Conquer presented in [25]. Dividing a problem into many parts to solve each of them separately can help in solving complex problems. Hence, the complexity of the problem is simplified. Complex problems, such as face recognition, can be solved with efficiency if we correctly adapt the problem-dividing technique. In our proposed algorithm, each training face image is segmented into 2D blocks with the same size. Each block is converted to the 1D feature vector, which will be the input of the subsequent algorithm step or feature extraction. Selecting the number of segments is an important and sensitive process because the choice can affect the overall face recognition system. In the next section, we will explain the steps of determining the optimal number of the segment for each face image experimentally.
3.2. Features Extractors
3.2.1. PCA
The eigenface finding based on PCA technique (also known as Kohonen-Loeve filter) is specifically used for face image data. In the PCA method, all dataset images are recognized with feature vectors that are created from projections of the training image to the basis in image space. PCA classifies dataset images according to calculated distances amongst feature vectors. Typical classifiers comprise nearest distance measure, Euclidean distance and nearest mean classification [26–29]. When PCA is used for the eigenface method, feature vectors identifying each image are shown in Figure 6. These vectors are obtained as follows:(1)Assume that we have N face images with (x) rows and (y) column. representing those images in column vectors with ()dimensions. We need to calculate a mean face image (Q), from these vectors like, where “Q” is the mean matrix (common properties of faces ).(2)The main data matrix is subtracted from , which can be represented as(3) column vectors are gathered in matrix: R = [L1, L2,..., ] with dimension () and a covariance matrix C is formed as Given high dimensionality and considering the linear algebra principle as well as great computational complexity we use A instead.(4)We calculate N eigenvalues and N eigenvectors () of to form the eigenface space. V = [, ,, ] represents a matrix including eigenvectors of (C) with a dimension of (). We can obtain eigenface space U=[,,, by
All row vectors of are eigenfaces of face images in the training set. Face images with high eigenvalues have great contributions to the eigenface space. For this reason, systems with low computational capability sort eigenvectors of face images according to their corresponding eigenvalues in decreasing order and choose eigenvectors to form a small eigenface space.(5) The matrix with (xx) dimension includes (N) column vectors corresponding to each face image in the training set. These vectors are called feature vectors and they represent each images specific characteristics. (W ) can be obtained After the eigenface space and feature vectors are obtained, we can compare a test image with the faces in the training set by projecting a test image into face space as follows:(6)“” is a column vector that represents our test image with dimension. At this stage, the distance of test image from mean face image should be calculated as column - vector,(7)After calculated, we must project the result on our eigenface space to obtain with (x 1) dimension.(8)(c) We need to find the similarity of in each in the matrix to determine which image in the training set resembles our test image. Various classifiers can be used at this stage [26–29].3.2.2. Slope Based Method
To extract the best features from edge images efficiently, a new feature extraction method based on the slope of the estimated curve of each small segment is used. The proposed SLP extraction method produces vectors of features that can identify each face successfully, as shown in Figure 5. The following steps explain in detail the process of extracting the features in the training phase:(1)We suppose we have N face images (, , ) with rows and columns, where is the number of images in the training set.(2)We convert the face images (, , ) to binary images by using one of the six edge detection filters discussed in Section 2. The binary images are denoted as (, , ).(3)We divide each binary image , into segments, where H is the number of rows and L is the number of columns in each segment. The segments of image face are denoted by (, ,...,), where k is the number of segments with () dimension.(4)For each segment of each face image , the value of pixels with ones is considered as points and a linear regression estimation method is used to find the best line that fit these points. Assume that the equation of the line is as where b is the Y-intercept and m is the slope of the estimated line.(5)Compute the slop of the estimated line of each segment using the following equation which is derived from Pearson’s correlation coefficient formula and linear regression: where m is the slop of segment, x is the x-coordinates, y is the y-coordinates and n is the length of x,y vector in the binary image. In our work, we select the linear regression techniques because it is the most widely used statistical technique to model a relationship between two sets of variables and it is widely used to estimate the line equation from a set of points. Slopes of segments are collected in a matrix with dimension () corresponding to each face image in the training set to create features vectors which will be the input to the Multi-layer Neural Network. The usage of the linear regression techniques in our steps makes the proposed algorithm works very well since it generates very robust features that are not affecting by outlier points or noise points that may exist in the binary edge images.
3.3. Feed Forward Neural Network
The last step in the proposed system is to train the neural network on the dataset. The input matrix of the training dataset consists of the feature vectors after we apply all the preprocessing operations, dimensionality reduction, and feature extraction on dataset images. ANN is a classifier that is used in the proposed face recognition system. The steps of this algorithm are summarized as follow:(1)The weighted sum of the input vectors (dot product between the input vector and weights that connect the input and the hidden layers) is calculated [30, 31].(2)The results from the weighted sum calculation are passed to the activation function to normalize the output. In the proposed neural network, we use the adaptive sigmoid function.(3)In the same way as above, we calculate the output of the final layer.(4)After the output is calculated, error evolution between actual and the desired output is started by using the least square error function and updating of the weights of layers according to the rate tolerance in errors.
4. Results and Discussion
As described earlier, the combinations of different face recognition systems are developed and will be tested in this section. The two steps in our proposed algorithm may use different schemes. The first step is change detection, which may be one of the six edge detection filters from the two groups. The first group is based on the gradient method and includes SF, PF, and RF. The second group includes LF, ZF, and CF that perform a Laplacian process before they begin edge detection. In the second step, which is feature extraction, we use an eigenvector-based method or the SLP method. ANN is the classifier that is used for the classification of the proposed algorithm combinations. The BIO ID database used for the training and the testing processes has 23 subjects (1,502 images). The face images are saved in a single file using the portable grey map (pgm) format. The pgm file contains a data header that has the image file information including image format, width and height and maximum allowed grey level value followed by the image data. Many experiments are carried out to determine the best parameters that we should use in each algorithm. One of the most important parameters is the number of segments that we should use in the first step of the proposed algorithm. Figure 7 shows the effects of using a different number of segments on the performance of our proposed algorithm. As noted from the results, the number of segments may considerably affect the performance of the algorithm. We also note that the best number of segments is 10. Hence, if we select values more or less than this value, then the results will not be acceptable. Therefore, in all our experiments, the face images are divided into 10 segments horizontally and vertically, thereby producing 100 segments.
The second most important parameter is the best number of hidden layers that should be used to classify the complex scenarios efficiently. For determining this parameter, another experiment is executed on different algorithms to extract the best value. Figure 8 shows the results of this experiment. As shown in the figure, the best number of neurons that should be used is 30. However, increasing the number of neurons is not always as good as the results show.
In addition to the number of neurons and several segments, the threshold values of the edge detection filters are important and considerably affect the resulting binary edge image and the classification process. To select the thresholds accurately, a separate experiment is performed for each of the considered six edge detection filters. Figures 9–14 show the results of using different threshold values for each edge detection method. In our experiments, the best threshold value that maximizes the performance is selected for each edge filter. Tables 1 and 2 show the results of comparing different combinations of the proposed algorithm by using the BIO ID database. Table 1 presents the results of using SF, PF, and RF for SLP and PCA feature extraction methods. For all algorithms, the multilayer perceptron neural network (MLPNN) is used. Table 2 presents the results of using LF, ZF, and CF for SLP and PCA feature extraction methods. For all algorithms, the MLPNN is also used.
The correct classification rate is used to measure the performance of the compared algorithms. This performance metric computes the average number of correctly classified faces as described in the following equation. To obtain the correct classification rate, the rates of BIO ID training and test data are set to four different values:(i)85 % of data set for training and 15% of the data set for testing.(ii)80 % of data set for training and 20% of the data set for testing.(iii)70 % of data set for training and 30% of the data set for testing.(iv)60 % of data set for training and 40% of the data set for testing.
As shown in Table 1, the SF, PF, and RF algorithms that use our new proposed SLP method outperform the other PCA versions. The best algorithm is the (RF, SLP, ANN) algorithm, which obtains the optimal mean value and most valuable results in two different percentages of training data. RF works effectively with binary images and is widely known as the best supporter kernel. The algorithms that use the PCA method obtain good results. For example, the (PF, PCA, ANN) algorithm obtains an average performance of 93%.
The results of the second-order filters in Table 2 confirm the performance of our new proposed combination with the new proposed SLP feature extraction method. As shown in Table 2, the LF, ZF and CF algorithms that work with our new proposed SLP method outperform the other PCA versions. In Table 2, the best algorithm is (ZF, SLP, ANN) with a correct classification of 95.7%.
The proposed SLP method can considerably enhance the performance of the face recognition system and performs better than the PCA mechanism. The proposed SLP method creates position matrixes of pixel values to calculate the slope of each point. Such a feature makes SLP robust against image environmental factors, such as light conditions and complex backgrounds. Moreover, SLP does not need complex normalization processes, such as dimensionality reduction, and additional execution time. Second-order filters such as LF, ZF, and CF generate a more suitable binary image for face recognition than the first-order filters such as SF, PF, and RF.
5. Conclusion
This paper presents a new and efficient face recognition system that uses a new and efficient feature extractor called the SLP method. The proposed algorithm uses an edge detection algorithm followed by the SLP mechanism to extract the important features of the face. We use combinations of various techniques such as (SF, SLP/PCA, ANN), (PF, SLP/PCA, ANN), (RF, SLP/PCA, ANN), (ZF, SLP/PCA, ANN), (LF, SLP/PCA, ANN), and (CF, SLP/PCA, ANN) to achieve the best combination. The BIO ID dataset is used in the training and testing phases for the proposed face recognition system combinations. The experimental results show that the proposed schemes achieve satisfactory results with high-accuracy classification. Notably, the combination of (ZF, SLP, ANN) gains the best results and outperforms all the other algorithm combinations.
Data Availability
The data used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
Supplementary Materials
Figures 1s-6s: effect of different edge operators (SF, PF, RF, ZF, LF, and CF) which were utilized during the edge detection process. Edge derived from BIO ID dataset images according to the chosen threshold values are present clearly in the main images of the main manuscript. Supplementary Figure 7s: example of the segmentation and SLP feature extraction method. Supplementary Figure 8s: example segmentation and PCA feature extraction method. Figures 9s-27s represent examples of some test results. (Supplementary Materials)