Abstract
Biometrics verification can be efficiently used for intrusion detection and intruder identification in video surveillance systems. Biometrics techniques can be largely divided into traditional and the so-called soft biometrics. Whereas traditional biometrics deals with physical characteristics such as face features, eye iris, and fingerprints, soft biometrics is concerned with such information as gender, national origin, and height. Traditional biometrics is versatile and highly accurate. But it is very difficult to get traditional biometric data from a distance and without personal cooperation. Soft biometrics, although featuring less accuracy, can be used much more freely though. Recently, many researchers have been made on human identification using soft biometrics data collected from a distance. In this paper, we use both traditional and soft biometrics for human identification and propose a framework for solving such problems as lighting, occlusion, and shadowing.
1. Introduction
Recently, with the increase of international terrorism and violence, the interest in identification technique using video surveillance has greatly increased. Also, with widespread of computers, biometric identification comes in demand in such fields as home automation and health care. Recently, it has come about through pattern recognition, computer vision, and image analysis automatically detecting physical presence and verifying oneβs identity.
Biometrics aims to recognize a person through physiological or behavioral attributes, such as face, fingerprint, iris, retina, and DNA [1]. Biometrical methods can be largely divided into traditional technique that deals with physical data such as face features and fingerprints, and the so called soft biometrics that is concerned about gender, ethnicity, height, tattoo, and signature as shown in Figure 1 [2]. Traditional biometrics has excellent accuracy and great versatility. However, it is difficult to collect physical data from a distance, and also cooperation is often required like with lifting fingerprint. On the other hand, soft biometrics has less accuracy, but it can be used in a large variety of environments and does not require cooperation. Since soft biometric data are not totally dependable, person identification is made based on multiple data. For example, only gender and ethnicity information is not enough to verify oneβs identity. Recently, multimodal biometric methods have been extensively researched where traditional and soft biometrics work together in order to ensure best results for a specific environment. One of the major advantages of a multimodal approach is that it is harder to circumvent or forge [3].
In this paper, we analyze how biometrics can be used for identification in video surveillance system and propose a framework to solve such problems as lighting, occlusion, and shadowing. Section 2 of this paper describes biometric identification using video surveillance system. Section 3 further proposes a framework for human identification from a distance. Future research directions and conclusion are presented in Section 4.
2. Biometrics
2.1. Traditional Biometrics
Broadly speaking, biometrics is about establishing personal identity using physical, physiological, and behavioral characteristics of the person. The main reason why it is so popular is security: with biometrics there is no risk something might be lost or stolen as is often the case with traditional IDs and passwords.
Especially, identification using face features and fingerprints has been extensively researched and is currently used in a wide variety of applications because of high accuracy rate. Then, attempts have been made to use face features and fingerprint in video surveillance systems that require, however, extra effort. On the one hand, identification using face features is very convenient for the people as recognition is made without physical contact [4]. On the other hand, this method is very sensitive to facial expression and changes in lighting. The accuracy also decreases as face features do change over the years. Besides, as the distance between the camera and the person increases, it becomes more difficult to extract face features needed for identification.
2.2. Identification Using Discrete Biometric Information
As discussed above, traditional biometrics methods are very accurate and versatile. However, for the most part they can be used only in controlled environment and in cooperation with the person being investigated. On the contrary, soft biometrics can be used in any environment and requires no cooperation.
Wayman [5] has suggested a method for filtering a large-scale biometric database containing such information as gender and age. Thus, the possible candidates can be screened depending on the specific feature. This method improves the speed of biometric system and the efficiency of search. But, it appeared that the elements like age, gender, ethnicity, and occupation can affect performance of biometric system [6]. For example, in young Asian women workers of the mines, the difficult identification problem occurs in biometric system. Therefore, recently the methods that could verify identification by assigning different weighted values to each of biometric features in a multimodal system have been researched. Jain et al. has proposed a multimodal biometric system that uses Bayes Theorem as shown in Figure 2 [7]. The Bayes Theorem used in the proposed system can be shown in where is the number of test subjects in the database, is the value of traditional biometric traits such as face and fingerprint, and is the value of soft biometric traits that can be used additionally.
When multimodal biometric data is used, each piece of data can contribute differently to identification. For example, ethnicity is much more informative than gender. In addition, in case that forgery is possible using makeup or heel, biometric information and soft biometric information have equal influence on identification, thus the recognition rate can be reduced. As shown in (2), different weighted values can be assigned to different biometric data. Lightweight values are assigned to soft biometric data in contrast to more accurate biometric information. The total of weighted values assigned to each of biometric information is 1, , and
Hossain and Chetty has used the face features and gait data together to determine the gender [8]. Before, the gender was determined by judging from face features only. By adding gait data, however, the accuracy has been greatly increased. Figure 3 shows a simple gender recognition workflow.
First, gait image and face image of the subject are obtained using background subtraction technique. Gait cycle is determined depending on the change in the number of pixels in the lower part of the silhouette (Figure 4) as shown where is the number of image frames and are the coordinates in the lower part of the silhouette (background removed).
Thereafter, the gender is checked based on correlation between the two images using canonical correlation analysis (CCA) and the database. Lastly, after going through the main identification step primarily using face information and gait information obtained from remote camera as shown in Figure 5, the recognition performance level was improved using in conjunction with soft biometric information obtained from the short distance camera.
2.3. Identification Using Continuous Biometric Information
Biometric identification is an important component of surveillance systems. There are, however, many constrains to use face recognition in real environments where biometric information should be obtained without interference [9]. For this, a variety of biometrics suitable for environment of surveillance system has been researched.
For example, in case of height the specificity is low but it is not oppressive and it obtains relatively accurate height from long distance as well as short distance. To determine the height, projective geometry method has being researched [10]. When vanishing line and vertical vanishing point on the standard plane and a reference height are given, oneβs height can be easily calculated.
The color of clothes can also be used to verify subject identity. First, quantization is used to distinguish clothing color. The octree-based color quantization can configure the similar palette to the pixel value obtained from image because its memory utilization is low if an appropriate octree depth is specified, the velocity of quantization is also fast and it configures the dynamic tree for input image [11]. Figure 6(a) shows input subject, Figure 6(b) shows quantified clothing area where the pixel value is 0 in the block, and Figure 6(c) shows the result of typical quantization color extracted from clothing area of input subject [12].
(a) Extracted human image
(b) Clothing area for quantization
(c) Extracted representative color
2.4. Soft Biometric Information Using Facial Mark
Soft biometric may include a variety of facial marks such as scars, tattoo, and freckles as shown in Figure 7 [13]. These biometric data can play an important role in establishing personal identity. Also with high resolution camera, increased database for facial image, and the development of image process and computer vision algorithm, the research to verify the identity using facial mark is increasing.
The research to improve facial recognition performance using facial mark properly which can be obtained from facial image and face is proceeding lately. Park and Jain suggested the identification technique using facial mark appeared on the face [14]. Figure 8 shows a schematic diagram of the proposed system. First, active appearance model (AAM) is used to extract the face. After producing a Mean Sharp using extracted facial image, it is mapped through barycentric coordinates. But, the mapped image has the problem due to the projected area such as eyes, nose, and mouth. This can be solved using Laplacian of Gaussian (LoG) or Difference of Gaussian (DoG) filter. After that, facial marks are extracted using the difference between the Mean Sharp image and the LOG image. Facial marks can be classified into 6 categories as shown in Figure 9.
3. Long-Distance Human Identification Framework
Biometric information used for identification in existing video surveillance systems includes face features and fingerprint. Such biometric information showed high recognition rate if the exact feature of the subject is extracted. However, with remote video surveillance there always such problems as lighting, occlusion, and shadowing that badly decrease recognition rate. Therefore, the research using soft biometric information is proceeding. In case of soft biometric information, identity can be verified in various environments but since its distinctiveness and permanence are low, it is possible to forge and falsify the information. Therefore, we propose a special framework for long-distance human identification as shown in Figure 10. The human identification system is divided into two subsystems shown in Figure 10. One subsystem is called the primary biometric system and it is based on traditional biometric identifiers like face and fingerprint. The other subsystem is called the secondary biometric system and it is based on soft biometric traits like height and clothing color. After that, information on height and clothing color obtained from video surveillance camera is stored in the database and it is used for secondary biometric information along with information on face and fingerprint for identification. The experimental environment of the proposed framework is assumed to be inside the building. Generally, for buildings requiring high level of security such as companies, libraries, or broadcasting stations, a single authentication system is not enough. Thus, both video surveillance camera and a fingerprint sensor are installed at the entrance of the building. But inside the building, the identity of the subject is further established from the distance, using facial information obtained from video surveillance cameras only. However, because of problems with lighting, shadowing, and occlusion, it is difficult to obtain accurate facial data.
The proposed framework obtains information on primary biometric traits like face and finger print and secondary biometric traits like height and clothing needed for identification from video surveillance camera and fingerprint sensor in short distance to determine the access of the subject at the entrance of the building shown in Figure 11(a). Although height and clothing color are not as permanent and reliable as the traditional biometric identifiers like face and fingerprint, they provide some information about the human identification that leads to higher accuracy in establishing the human identification system. Therefore, information on height and clothing color obtained from entrance camera is stored in the database and is used for additional biometric information along with information on face and fingerprint for identification. If the user is determined as unauthorized, the entry of the user will be controlled.
(a) Entrance
(b) Inside the building
If a subject is working inside the building where no fingerprint sensor is installed such as Figure 11(b), the fingerprint information cannot be obtained because the fingerprint sensor is not used like the environment of building entrance. So information on face, height and clothing color is obtained only by video surveillance camera. However, if facial data needed for identification cannot be obtained when the distance between the camera and the subject is too large or because of such problem as lighting, shadowing, or occlusion, the data about oneβs height, and clothing color are stored in the database at the entrance of the building and information on height and clothing color is obtained from the inside. If a person reenters the building, height and clothing color data can change. In this case, the identity can be verified by storing the new information on subjectβs height and clothing color in the database.
Therefore, the accuracy of object extraction required for identification was decreased in the existing video surveillance system due to the environmental factors including lighting, occlusion, and shadow, but the human identification system using proposed framework is expected to improve the recognition performance by using various biometric information even though the feature extraction is difficult due to the environmental factors such as lighting, shadow, and occlusion.
4. Conclusions
The research using biometric information for identification has been actively proceeded in video surveillance system. Typically, the traditional biometrics uses information on face and fingerprint. However, the traditional biometrics has the problem of decreased recognition rate because it needs cooperation with the user and low resolution image. Thus, the multimodal biometrics is researched using in conjunction with soft biometrics recently to verify the identity in nonoppressive and various environments. The multimodal biometrics using different biometrics is suitable for specific environment like video surveillance system compared to single biometrics and increases the recognition rate by maximizing the advantages of each biometric information.
In this paper, the identification technique using biometrics suitable for video surveillance system was analyzed. In addition, the framework was proposed to complement the problems of decreasing recognition performance due to lighting, occlusion, and shadow. However, no human identification system that satisfies various environments with the current technique is existed. Therefore, proposed framework limited the experimental environment to the inside of the building, but in the future we plan to complement the problems that can occur in various environments.
Acknowledgments
This work was supported by the National Research Foundation of Korea (NRF) Grant funded by the Korea government (MEST) (no. 2009-0086148) and by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Education, Science and Technology (2011-0023147).