Abstract
Gait energy image (GEI) preserves the dynamic and static information of a gait sequence. The common static information includes the appearance and shape of the human body and the dynamic information includes the variation of frequency and phase. However, there is no consideration of the time that normalizes each silhouette within the GEI. As regards this problem, this paper proposed the accumulated frame difference energy image (AFDEI), which can reflect the time characteristics. The fusion of the moment invariants extracted from GEI and AFDEI was selected as the gait feature. Then, gait recognition was accomplished using the nearest neighbor classifier based on the Euclidean distance. Finally, to verify the performance, the proposed algorithm was compared with the GEI + 2D-PCA and SFDEI + HMM on the CASIA-B gait database. The experimental results have shown that the proposed algorithm performs better than GEI + 2D-PCA and SFDEI + HMM and meets the real-time requirements.
1. Introduction
As one of the biometrics, the gait recognition is typically used for identifying an individual using image sequences, which capture a person’s walk. Gait recognition has its advantages compared with traditional biometrics, such as nonaggression, noncontact, easy collection, and hard to hide and camouflage, which makes the gait recognition have a wide range of application of intelligent monitoring and medical diagnosis. More and more researchers are absorbed to pay attention to the gait recognition.
At present, the techniques of gait recognition can be commonly divided into two categories: model-based [1–3] and silhouette-based [4–8] approaches. Model-based approaches generally establish static or dynamic prior model, then the model parameters can be calculated by matching with the motion sequence, which are used to identify the individuals. Zeng et al. [1] present a new model-based approach for human gait recognition in the sagittal plane via deterministic learning (DL) theory. Side silhouette lower limb joint angles are chosen as the gait feature, which are extracted from the phase portrait of joint angles versus angular velocities. The obtained knowledge of the approximated gait system dynamics is stored in constant RBF networks. Finally, a bank of estimators is constructed using constant RBF networks to accomplish the gait recognition. Kovač and Peer [2] proposed a skeleton model-based gait recognition system focusing on modeling gait dynamics and eliminating the influence of subject’s appearance on recognition. Then, they tackled the problem of walking speed variation and proposed space transformation and feature fusion that mitigates its influence on recognition performance. Zhang et al. [3] proposed a model-based approach to gait recognition by employing a five-link biped locomotion human model. Firstly, they extract the gait features from image sequences using the Metropolis-Hasting method, and then Hidden Markov Models are trained based on the frequencies of these feature trajectories, from which recognition is performed. Silhouette-based approaches mainly extract the static or dynamic characteristics of the gait silhouette to recognize individuals through the moving information of image sequences without establishing prior model. Lee et al. [4] calculated the binomial distribution of every pixel in a gait cycle and organized the binomial distribution of all pixels in the gait image; therefore, the gait probability image (GPI) can be formed. Then, the symmetric Kullback-Leibler divergence is used to measure the information theoretical distance between gait signatures. This method is robust to slight variation in walking speed. Jeong and Cho [5] proposed a gait recognition method based on the multilinear tensor analysis. Firstly, they formed the accumulated silhouette from the gait image sequences and then described those as the tensor. Secondly, the method computed the core tensor which governed the interaction between factors organizing the original tensor using a multilinear tensor analysis. Finally, the computation of similarity based on the Euclidean distance is chosen to recognize the individual. Li et al. [6] regard the line quality vector of frame difference energy image as the feature of gait recognition. Continuous hidden Markov model (CHMM) was used to recognize the individual. Li and Wu [7] extracted unified Hu moments from the gait sequence as gait features and then used the SVM for training and recognition. Liu [8] proposed a novel feature fusion method. Firstly, the static characteristics were obtained from the Procrustes mean shape, then, the transform of Fan-Beam was carried out with GEI, and the dynamic characteristics of the gait sequences were obtained by using the two-dimensional principal component analysis to reduce the dimension of the feature space. Finally, the two features are fused to achieve the final recognition results. In regard that there is no consideration of the time that normalizes each silhouette in the GEI, this paper proposes a new class energy image which is denoted as the accumulated frame difference energy image (AFDEI), and then the fusion of the moment invariants extracted from GEI and AFDEI was selected as the gait feature. Then, we recognized the individual using the nearest neighbor classifier based on the Euclidean distance.
2. Gait Feature Extraction and Recognition
This paper combined the gait energy image (GEI) with the accumulated frame difference energy image (AFDEI). Furthermore, the moment invariants were extracted from GEI and AFDEI, and then the characteristics were fused as the gait feature. Finally, the nearest neighbor classifier was used to evaluate the gait recognition.
2.1. Gait Energy Image (GEI)
The gait energy image is used to reflect the gait sequence of a cycle in a simple energy image using the weighted average method. The gait sequences in a gait cycle are processed to align the binary silhouette. If the gait cycle image sequence is , gait energy image can be calculated by the following formula:where is the gait cycle image sequence, is the number of frames in a gait sequence of a cycle, and is the number of gait frames. Figure 1(a) shows the gait images of a cycle and Figure 1(b) shows the corresponding GEI.
(a) Gait images of a cycle
(b) Gait energy image (GEI)
The color or luminance of the pixels in the figure can indicate the size of the body’s parts when the person is walking. The white pixel points represent the parts which move slightly, such as head and trunk. Gray pixel points represent the parts which move significantly, such as the legs and arms. So, the gait energy image retains the static and dynamic characteristics of human walking, and it greatly reduces the amount of computation in the image processing.
2.2. Accumulated Frame Difference Energy Image (AFDEI)
Through the description of the gait energy image, there is no consideration of the time within the GEI. In order to improve the accuracy of gait recognition, this paper proposed the accumulated frame difference energy image (AFDEI).
Firstly, the frame difference energy image must be calculated. Frame difference energy image is calculated by combining forward frame difference image with backward frame difference image. Formula (2) shows how to calculate the forward frame difference image, formula (3) shows how to calculate the backward frame difference image, and formula (4) shows how to calculate the frame difference image. Considerwhere is the forward frame difference image. Considerwhere is the backward frame difference image. Considerwhere is the frame difference image.
By weighted average method, the accumulated frame difference energy image is obtained, which can reflect the time characteristic. Formula (5) shows how to calculate the accumulated frame difference image:
Because the purpose of the accumulated frame difference energy image is to reflect the time characteristic of human walking, in order to reduce the amount of computation, in this paper, the first six frames of a gait cycle are chosen as the information frames to obtain the frame difference energy image. Because of reflecting the time information of the gait pattern, the binary silhouette is not necessary aligned in order to reserve the dynamic changes of gait silhouette. Figure 2(a) shows the information frames, Figure 2(b) shows the frame difference energy image, and Figure 2(c) shows the accumulated frame difference energy image.
(a)
(b)
(c)
As shown in Figure 2(c), because the six frames of gait sequences are chosen as the information frames, each AFDEI contains the same time. The range of the human body and the brightness of pixel in the image can reflect the time characteristics such as step speed, cycle, stride length, and arm movement.
2.3. Feature Extraction and Fusion
2.3.1. Invariant Moments
M. K. Hu proposed the definition of invariant moments in 1962, according to the central moment; he proposed 7 invariant moments which have the invariance of rotation, translation, and scale. For 2D gray image , its origin moment and central moment are shown in the following formula:where is origin moment, is central moment, is the centroid coordinates of the image, , and .
normalized central moment is shown in the following formula:where .
The 7 invariant moments which are based on the normalized second-order central moment and the third-order central moment are as follows:
Tables 1 and 2 list the 10 invariant moments of the gait energy image and the accumulated frame difference energy image.
2.3.2. Feature Fusion
Through analysis of the gait pattern, the static and dynamic characteristics of the gait sequences can be characterized by the GEI, and the time characteristics can be characterized by the AFDEI. So we can get better recognition results by combining the two kinds of energy image. However, though some dynamic features can also be expressed in the AFDEI, the static characteristics are completely ignored, which will inevitably affect the accuracy of the identification. In order to sufficiently reflect the advantages of the multifeature fusion, according to the concept of trial and error, this paper has obtained the optimal weight of AFDEI and GEI in the process of feature fusion by a large number of experiments. Meanwhile, the interference of some useless information can be avoided when the useful information is obtained.
3. Experiment and Analysis
In this paper, the CASIA_B database provided by the automation of Chinese Academy of Sciences was used. The database contains 124 people: each person has 11 angles of view and three kinds of gait at the same angle of view. Three kinds of gait include 6 normal people, 2 wearing people, and 2 packaging people. In this paper, we choose the view which is divided into three groups of experiments: set A is normal walking sequence, set B is wearing walking sequence, and set C is packaging walking sequence. The principle of gait recognition is shown in Figure 3.
In this paper, the experiments were done with matlab 2012a on the computer with 2.5 GHz and the memory of 4 GB. Meanwhile, the proposed method was compared with GEI + 2D-PCA [9] and SFDEI + HMM [10]. Table 3 shows the result of experiment. The experimental result has shown that the proposed algorithm performs better than the other algorithms and meets the real-time requirements. The experimental results show that identification results have been greatly improved by the proposed gait recognition algorithm compared with the algorithms which only consider static or dynamic features. Especially in the case of wearing and packaging, the advantage of the time factor can be reflected better when the shape of the human body is affected. As a result of the slight change of walking speed, the recognition effect will be influenced. In the future studies, the most important work is to solve the problem of speed variation which can influence the recognition performance.
4. Conclusion
The gait energy image (GEI) preserves the dynamic and static information of a gait sequence; however, there is no consideration of the time. In regard to this problem, this paper proposed the accumulated frame difference energy image (AFDEI) to reflect the time characteristics. Furthermore, the moment invariants were extracted from GEI and AFDEI, and then the characteristics were fused, which obtains better recognition results than a single feature. Finally, the nearest neighbor classifier was used to evaluate the gait recognition. To testify the proposed algorithm, the CASIA_B database provided by the automation of Chinese Academy of Sciences was chosen to experiment. The experimental result shows that the proposed algorithm has a high recognition rate, which has a certain reference value for the study of gait recognition.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
The authors are grateful to the anonymous reviewers who made constructive comments. This work is supported by the National Natural Science Foundation of China (no. 61203302 and no. 61403277), the Tianjin Research Program of Application Foundation and Advanced Technology (14JCYBJC18900).