Mathematical Problems in Engineering

Volume 2018, Article ID 9801308, 18 pages

https://doi.org/10.1155/2018/9801308

## An Entropy-Histogram Approach for Image Similarity and Face Recognition

^{1}School of Computer Science and Technology, Huazhong University of Science and Technology, Wuhan 430074, China^{2}Faculty of Computer Science & Mathematics, University of Kufa, Najaf, Iraq^{3}School of Engineering, Edith Cowan University, Joondalup, WA, Australia^{4}Shenzhen Huazhong University of Science and Technology Research Institute, Shenzhen 518063, China

Correspondence should be addressed to Zahir M. Hussain; gro.eeei@niassuhmz

Received 4 March 2018; Revised 18 May 2018; Accepted 21 June 2018; Published 9 July 2018

Academic Editor: Mariko Nakano-Miyatake

Copyright © 2018 Mohammed Abdulameer Aljanabi et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

#### Abstract

Image similarity and image recognition are modern and rapidly growing technologies because of their wide use in the field of digital image processing. It is possible to recognize the face image of a specific person by finding the similarity between the images of the same person face and this is what we will address in detail in this paper. In this paper, we designed two new measures for image similarity and image recognition simultaneously. The proposed measures are based mainly on a combination of information theory and joint histogram. Information theory has a high capability to predict the relationship between image intensity values. The joint histogram is based mainly on selecting a set of local pixel features to construct a multidimensional histogram. The proposed approach incorporates the concepts of entropy and a modified 1D version of the 2D joint histogram of the two images under test. Two entropy measures were considered, Shannon and Renyi, giving a rise to two joint histogram-based, information-theoretic similarity measures: SHS and RSM. The proposed methods have been tested against powerful Zernike-moments approach with Euclidean and Minkowski distance metrics for image recognition and well-known statistical approaches for image similarity such as structural similarity index measure (SSIM), feature similarity index measure (FSIM) and feature-based structural measure (FSM). A comparison with a recent information-theoretic measure (ISSIM) has also been considered. A measure of recognition confidence is introduced in this work based on similarity distance between the best match and the second-best match in the face database during the face recognition process. Simulation results using AT&T and FEI face databases show that the proposed approaches outperform existing image recognition methods in terms of recognition confidence. TID2008 and IVC image databases show that SHS and RSM outperform existing similarity methods in terms of similarity confidence.

#### 1. Introduction

Facial recognition technology will change society in many ways. Face recognition system is available nowadays. Face recognition has been used in many commercial and law enforcement applications [1] such as some of the airport’s systems, and ATM and electronic payment started to test facial recognition in real events. This availability of efficient face recognition algorithms leads to the fact that it can be used in real-time security issues where there is nowhere to hide.

People can recognize each other by the spectacular diversity of facial features and this is essential to the formation of complex societies. The face has the capability to send emotional signals, either voluntarily or involuntarily. The current biometric system technology reads faces as efficiently as humans do. Holy places use face recognition to track the presence of worshipers; retailers also use facial recognition technology to monitor thieves or to arrest suspects. Face recognition technology helps to verify the ID of the ride-hailing driver, verify the permits of tourists to enter tourist places, and let people buy things with a smile [2].

Face recognition is falling within the similarity of images where it is possible to recognize the face by finding the similarity and dissimilarity between the image stored in the database and the current image of the same person [3, 4]. There are different approaches for face recognition, especially statistical and information-theoretic. In this paper, we focus on an information-theory approach to design a similarity measure capable of testing similarity for the purpose of face recognition.

The measurement of image similarity is a significant point in the applications of the real world and several fields like optical character recognition (OCR), identity authentication, human-computer interfacing, surveillance, and other pattern recognition tasks [5].

To measure the similarity between two digital images, there is a simple method to calculate the similarity which is mean squared error. The advantage of mean squared error is easy to calculate, but, at the same time, it is not accurate for pattern recognition.

It is possible to use information-theory approach in image processing for analysis if we consider the image is a two-dimensional random variable, giving rise to the use of information-theoretic measures (such as joint histogram) to define similarity and recognition measures between images [6]. There are two useful measures of information which are Shannon entropy and Renyi entropy measures. In this paper, we present an information-theoretic image similarity-recognition measures and show its superior performance versus the similarity measures SSIM, FSIM, and FSM and recognition measures ZMSIM and ZESIM.

This paper handles the information-theoretic approach and includes the following sections: Section 2 describes the related works that give high-performance measures; Section 3 presents the design of novel information-theoretic measures based on entropy, combined with the joint histogram; Section 4 shows simulation results and performance analysis; and Section 5 presents conclusions and future work.

#### 2. Related Works

There are several works that addressed the face recognition approach and images similarity measure by employing the information theory and entropy concepts. All previous work has solved a high level of challenges of the face recognition and image similarity to support this system to work in real time. The authors developed SSIM and explained the performance of SSIM by using some examples [7].

Zhang et al. (2011) presented a feature similarity index measure (FSIM) for image-quality assessment. Phase congruency was used as a primary feature in the feature similarity index measure, whereas the gradient magnitude was used as a secondary feature in the feature similarity index measure to compute the feature similarity index. Experiment results were on the six-benchmark image-quality assessment database. Later we will demonstrate that entropy metrics performance is much higher than FSIM [8].

The feature extraction using the discrete cosine transform (DCT) with the approach of illumination normalization in the domain of the logarithm is proposed by Arindam Kar et al. in 2013 [9]; then, in the second step, they applied the entropy measures on the discrete cosine transform coefficients. Finally, they applied the kernel entropy component analysis with an extension of arc cosine kernels on the extracted DCT coefficients; their system was tested on the four databases such as FERET, AR, FRAV2D, and AT&T.

In 2013 Darshana Mistry et al. used the concepts of entropy measure, joint entropy, and joint histogram to find the similarity between two digital images and test these measures on the brain images as a database [10]

In 2014, Lee et al. suggested a method for face recognition using Shannon entropy and fuzzy logic [11]. This method is based on combination of Shannon and fuzzy logic. In this work, the use of entropy is to calculate the element ratio between two faces images and the use of fuzzy logic is to calculate the entropy membership with one.

In 2015, Yulong Wang et al. introduced a MEEAR (Minimum Error Entropy-based Atomic Representation) framework for facial recognition system. MEEAR is based on the minimum error entropy (MEE) model to be more robust under noise condition [12]. MEEAR can be used for developing new classifiers. MEEAR can provide distinctive representation vector by reducing the atomic norm regularized Renyi’s entropy of the reconstruction error.

Images similarity index based on entropy function and group theory is proposed by Y. G. Suarez et al. in 2015 [13]. An algebraic group theory of images is considered in this image similarity index. Images subtraction is provided in this similarity index by inner law.

In 2016 Q. R. Zhang et al. proposed the Improved Relative Entropy (IRE) method for face recognition approach. The IRE method is based on Shannon entropy and it is more accurate than Linear Discriminant Analysis and Locality Preserving Projections methods. The experimental results of IRE using CMU PIE and YALE B databases showed the high performance of the IRE versus LDA and LPP [14].

The system of emotion recognition based on facial expression is proposed by Y. D. Zhang et al. in 2016 [15]. Seven different facial expressions are considered in this approach such as sad, happy, angry, surprised, disgust, neutral, and fear. To extract the features the biorthogonal wavelet entropy has been used and utilizes the fuzzy multiclass support vector machine as a classifier.

To improve the kernel entropy component analysis (KECA), X. Ruan et al. in 2017 [16] did this improvement in three stages. Extract the features of faces by using Gabor wavelets in the first stage. In the second stage use the algorithms of nonlinear dimension reduction. In the third and last stage, use the k-nearest neighbor to the final classification on the fusion of different weighted multiresolution image of a human face.

FRIQA (Full-Reference Image-Quality Assessment) is an algorithm proposed by Y. Ren et al. in 2017 [17]. In FRIQA algorithm, the local entropy of images is analyzed in the first step, and then it calculates the similarity of local entropy between two images (reference image and the distorted version of it). Finally, the quality is computed for distorted version of the reference image from local energy similarity.

In a recent development, the authors in [18] introduced state-of-the-art FSM which combines the SSIM and FSIM methods. Canny edge detector has been used in FSM. The performance of FSM is tested under Gaussian noise condition and a wide range of PSNR, using FEI and AT&T databases. Experimental results show that the proposed FSM is better than the SSIM and FSIM approaches in similarity and recognition of human faces.

#### 3. A Brief on Efficient Similarity and Recognition Measures

The distance between two sets of various data points based on a given norm is called a “similarity measure.” If we have a dataset and a function that gives a large distance between this set and members of a database, except probably one member, then we have a similarity algorithm that can detect similarity between given data and members of a database. In this paper, two information-theory measures are designed based on entropies combined with a joint histogram of two images. Performance comparison is considered with well-known similarity and recognition measures. All the methods of recognition of the face image depend on the extraction of certain features of the images; the similarity shows the features of the statistical correlation or informatics correlation. To find the similarity between two images, several approaches are utilized; some are used for face and facial expression recognition. Here we present a brief description of well-known similarity and recognition measures for the sake of performance comparison, which is overviewed as follows.

##### 3.1. Structural Similarity Index Measure (SSIM)

The structural similarity index measure (SSIM) is one of the most popular metrics used to find the similarity between two images. Zhou Wang et al. proposed this measure in 2004 [7]. SSIM has been widely utilized for many algorithms of digital image-processing systems and image-quality assessment. The technique used in structural similarity is based on using statistical measurements, and it has an ability to extract the statistical image features for image recognition purpose such as standard deviation and mean , to get a definition for a distance function that can measure the SSIM between a training image and a test image. The measure is given by this formula:where is a structural similarity measure of a statistical similarity between the test image and training image . The quantity is the statistical mean of pixels in image , is the statistical variance of pixels in the image , is the statistical mean of pixels in image , and is the statistical variance of pixels in image . The quantities and are constants: where is a small constant and is a maximum value of pixels; where .

##### 3.2. Feature Similarity Index Measure (FSIM)

In 2011, Zhang et al. [8] presented a feature-based similarity index for image-quality assessment (FSIM), which has become a very common measure to find the similarity in images. The phase congruency was used as a primary feature and the gradient magnitude was used as a secondary feature in feature similarity index measure to compute feature the similarity index. To calculate the similarity between images the FSIM definition is used: where means the whole image spatial domain, is a phase congruency, and is a similarity resulting from the combined similarity measure for phase congruency and similarity measure for gradient , as given by the formulaswhere and are parameters used to adjust the relative importance of phase congruency and gradient magnitude features. The phase congruency is given by the equationwhere is a small positive constant andwhere and , , and , noting that and are even and odd symmetric filters on scale and “” denotes convolution. The function is a 1D signal obtained after arranging pixels in different orientations. The local amplitudes are defined aswhere is the position on scale .

##### 3.3. FSM: A Feature-Based Rational Measure

In 2017 NA Shnain et al. [18] have proposed a new structure for image similarity measure. The new structure is a rational function of measure with different statistical properties. FSM combines the best features of the well-known SSIM and FSIM approaches, trading off between their performances for similar and dissimilar images. Canny’s edge detection in FSM is used as a distinctive structural feature, where (after processing by Canny’s edge filter) two binary images, and , are obtained from the original two images and . FSM can be given by [18]where stands for the feature similarity index measure (FSIM) and stands for the structural similarity measure (SSIM). The constants are given the values = 5, = 3, c = 7, and = 0.01, while is a correlative function given bywhere and represent the image means. This function is not applied here to the original images themselves but to their edge-detected versions using .

##### 3.4. Zernike-Moments Approach for Image Recognition

Zernike moments provide an efficient, rotation-invariant, and noise-resistant approach for image and face recognition, including the complicated effect of face expressions [19]. Zernike moments are rotation-independent as they are defined in polar coordinates , with the help of Zernike radial functions that are defined as follows [20]:where , are integers that satisfy the conditions: is even and In the 2-dimensional radial domain, Zernike moments are defined as follows:where indicates complex conjugation. In order to use these moments for image recognition, we should approximate them in the discrete Cartesian coordinate system. Therefore, we perform a linear transformation of the image Cartesian coordinates from the inside of the unit circle to the inside of the square as follows:withWe extract face features as various Zernike moments (which we call here Zernike domain) and then define a similarity measure after imposing a distance measure in this domain. In this work we will consider Euclidean and Minkowski distance metrics. Features of an image can be represented by a vector of selected Zernike moments, . These distance measures are applied to feature vectors , of two images in the Zernike domain. They are defined as follows.

###### 3.4.1. Euclidean Metric

###### 3.4.2. Minkowski Metric

In this work we will use for Minkowski metric. When these two metrics are applied to test similarity between the Zernike-domain image features , (for two images** x** and** y**), we call the two Zernike-based similarity measures Zernike-Euclidean Similarity (ZESIM) and Zernike-Minkowski Similarity (ZMSIM). From another viewpoint, we establish a comparison with an efficient, rotation-invariant method for face recognition based on Euclidean and Minkowski distance in the Zernike domain. We selected Zernike feature vector asAlso, we used for Minkowski metric. Comparison showed that the proposed measures outperform Zernike-Euclidean (ZESIM) and Zernike-Minkowski (ZMSIM) recognition approaches. This is so despite the fact that Zernike measures are so powerful that they apply to face expression recognition as well as face recognition.

##### 3.5. ISSIM: A Functionally Relative 2D Histogram-Based Similarity Measure

In [21] an efficient information-theoretic measure (called ISSIM) has been proposed. This measure used a functionally normalized error function based on 2D joint histogram between the two images , under testing as follows:where and represent elements of the joint histogram between two images, is a small positive number to avoid division by zero. Normalization has been done relative to the function (the histogram of the reference image) and the maximum pixel value . Other (scalar) normalization steps have been added to ensure that the proposed measure stays inside .

##### 3.6. The Proposed Measures

Researchers have proposed several similarity and recognition metrics used in image-processing field; each has its weaknesses and strengths. The most disturbing problem in image similarity for face recognition is the confusing high similarity given by a specific measure between the reference image and other images in the database.

In this paper, we propose novel information-theoretic similarity-recognition measures for image similarity and face recognition. The proposed measures reduce confusion when used in face recognition by giving a very small similarity between unrelated images. Information theory has already been applied to pattern recognition [22]; here we apply it to design two similarity-recognition measures that are useful for face recognition and image similarity. The two measures apply the concept of entropy (Shannon & Renyi) to image a joint histogram as a probabilistic distribution. The names Renyi Similarity Measure (RSM) and Shannon Similarity Measure (SHS) are given to the new measures, according to the use of Renyi and Shannon entropies. Performance tests have been applied against the popular metrics SSIM, FSIM, ZMSIM, ZESIM, and FSM. Additional tests also include comparisons with the information-theoretic ISSIM.

###### 3.6.1. Shannon and Renyi Entropies

Entropy is the expected value of the information. Entropy has several applications in statistical mechanics, coding theory, statistics, and related areas. Emerging fields have also used entropy, such as image similarity [23]. The most significant entropy in applications is Shannon entropy, whose mathematical formula is given bywhere represents the entropy, is discrete random variable , and is probability of event , Here the probabilistic events are the elements of the 2D joint histogram between two images (test and reference images).

Renyi entropy is another significant measure of information, given bywhere , , is a discrete random variable, and is corresponding probabilities for . This entropy is a mathematical generalization of Shannon entropy.

The main difference between Shannon entropy and Renyi entropy is the placement of the logarithm in the entropic equations, giving a flexible measure of the entropy as a result of the parameter , enabling several measurements of dissimilarity [24]. This entropy, if applied to a joint histogram, gives high performance for face recognition.

###### 3.6.2. A Joint Entropic-Histogram Similarity and Recognition Measure

In a huge database for digital images like a face database, there might be identical histograms for very different images. This fact will be a problem when researchers want to compare images using a histogram as a distinctive feature. To solve this problem, Pass et al. [25] proposed an alternative to the classical histogram, called a joint histogram, which includes additional information without losing the powerful feature of the histogram. The joint histogram is based mainly on selecting a set of local pixel features to construct a multidimensional histogram.

A 2D joint histogram entry for two images and represents the probability that a pixel intensity value from image cooccurs with pixel intensity value from image . The normalized joint histogram for two images and of size is defined here as follows:whereor

Now we apply the entropy to measure the information held in the joint histogram that represents the joint probability of pixel cooccurrence. Note that both and range from to . First, the Shannon entropy measure is applied to get Shannon-Histogram Similarity Measure (SHS) as follows:where reshapes the 2D joint histogram into a one-dimensional column vector via the colon operator, as defined in MATLAB, with a new dimension . Applying the Renyi entropy in this approach gives the Renyi-Histogram Similarity Measure (RSM) as follows:where ; . Using other entropies could be more helpful. However, this is beyond the scope of this paper at the moment and will be investigated in future works.

###### 3.6.3. Motivation

One of the most difficult challenges for researchers in measuring image similarity for face recognition is that there is a high level of scepticism about the similarity between the reference image and test image in the same database, particularly when the image has low resolution or distortion in terms of illumination or background changes.

The differences in facial expressions and head poses for human faces often give rise to scepticism. Official government security systems do not rely entirely on face recognition systems, because the latter still suffer from challenges such as different facial expressions, illumination, and changing shape with age. However, a face recognition system can be very supportive of current routine security systems.

In this work, we have contributed to reducing these challenges regarding similarity of images, especially for the purpose of face recognition. We proposed new image similarity measures that can be utilized in face recognition. These measures are built using an information-theory approach; they proved to be very accurate in finding similarity between face images with more confidence than existing images similarity and image recognition measures. Our method is motivated by the problem of finding image similarity in large databases, where reduced confidence may open the door for big confusion.

The aim of this work is to provide metrics to find similarity between images for the purpose of face recognition; also, this can be used in case of nonface images. High performance and accuracy are the main features of proposed measures as compared to existing measures. Although other measures may have the ability to find the similarity between images (even for face recognition), the proposed measures have high confidence by giving almost a near-zero value in case of different images, while other measures give a nontrivial amount of similarity when comparing different images.

#### 4. Experimental Results and Performance

We have implemented the proposed measures on MATLAB and tested their performance against other measures as follows.

##### 4.1. Test Environment: Image Databases

In this work, we used well-known face databases, AT&T and FEI [26, 27], as a test environment. AT&T database as shown in Figure 1 has 40 persons each, with 10 different poses (including facial expressions); hence the total number of AT&T face images used in this test is 400 images. FEI database, as shown in Figure 2, has 200 persons, each with 14 different poses (including facial expressions), and the total number of FEI face images used in this test is 700 images as part of it.