Fast Cartoon-Texture Decomposition Filtering Based License Plate Detection Method

Abstract

Vehicle license plate detection is an important step in automatic license plate recognition, which is prone to be influenced by the background interference and complex environment conditions. It is known that cartoon-texture decomposition split an image into geometric cartoon and texture component, which can remove background interference away from the vehicle image. In this paper, we introduce a fast cartoon-texture decomposition filter into the detection process. Combining the edge detection, morphological filtering and Radon transform based tilt correction method, we formulate a new license plate detection algorithm. Experiment results confirm that the proposed algorithm can remove background interference away, inhibit the emergence of fake license plates, and improve the detection accuracy. Moreover, there is no inner loop iteration in the new algorithm, so it is fast and high-efficiency.

1. Introduction

It is well known that License Plate Detection (LPD) is the key technology and important step in Automatic License Plate Recognition (ALPR). ALPR is a system to recognize the license plate registration numbers automatically from digital images, and mainly involves four processes: image acquisition, LPD, character extraction and its recognition. In ALPR, LPD plays a pivotal role since that other processes are meaningless without the correct LPD. Hence, An efficient and accurate license plate detection algorithm under practical scenarios is much critical for a system to work in real-time [1–3].

During the past few decades, much research has been undertaken on LPD, which can be mainly divided into three categories: The first is based on color information [4–6]. According to license plate color features, this kind of algorithms construct color model for locating license plate. The advantage of these algorithms is that the color image does not need to be transformed to the gray image, thereby reducing the computation time, but it is sensitive to light conditions. The second kind of location algorithms are based on the edge information. As the license plate is rectangular in shape and has a known aspects ratio, it is possible to find all the rectangle in an image. These rectangles are usually detected by using edge information. The border of the license plate is detected as a rectangular by the edge detector as the color vicissitudes between the license plate character and its background [7]. Besides, there are also several works by combining the edge feature with other schemes [8–11]. They are fast and simple. However, discontinuities of edges or images with complex backgrounds may not provide the desired results. At last, there are many algorithms based on the texture features [12, 13]. License plate textures can be described in several ways, such as through the distribution of license plate edge, license plate shape, and license plate size. Authors in [14] proposed a license plate positioning algorithm based on tamura texture. They first find the candidate horizontal regions and then locate license plate exactly by using tamura texture features. This algorithm has high positioning accuracy and fast computation in high contrast and clear texture conditions. However, when this algorithm encounters interference similar to plate texture features in the process of texture analysis, such as the exhaust gate and bumper, the positioning rate is greatly decreased.

From the above, many license plate location algorithms are liable to be affected by the practical scenarios, such as the complex background, the light conditions and other complex environment conditions. To improve the reliability and the robustness of the algorithm, we design a new LPD method that performs better in the complex conditions. Specifically, the major contributions of this paper are summarized as follows.

(i) A fast filtering for cartoon-texture decomposition is introduced into the detection process, which split a vehicle image into cartoon component and texture component. Since that the texture component mainly includes small scale details with some periodic or oscillation characteristics, also the random noise. The background interferences can be effectively eliminated by separating texture from the vehicle image.

(ii) The cartoon-texture decomposition can help to optimize the image edge detection, and improve the accuracy of the LP detection. It is known that the cartoon component mainly contains geometric parts, such as piecewise-smooth regions and edge contours with large scale. So implant the decomposition into the edge detection process can enhance the quality of edge detection.

(iii) Combining the image decomposition, edge detection, mathematical morphology and Radon transform method, we formulate a new LPD algorithm. The process is verified to be high-efficiency and robust.

The remainder of the paper is organized as follows. In Section 2, we introduce a nonlinear filter for cartoon-texture decomposition. LPD algorithm is designed by combining the cartoon-texture decomposition, edge detection, morphological filtering and Radon transform. Section 3 presents the experimental results and the proposed LPD algorithm is verified to be effective. We conclude the paper and present some guidelines for future work in Section 4.

2. Fast Cartoon-Texture Decomposition Based LPD Algorithm

In this section, we first introduce a fast cartoon-texture decomposition method, which is just a nonlinear filter and do not need inner iteration, and can properly split an image into cartoon component and texture component. Based on which, we design a new LPD algorithm.

2.1. Fast Image Filtering for Cartoon-Texture Decomposition

The LPD accuracy is inclined to be affected by the complex backgrounds, which mainly include a lot of details, texture and noise and so on. In this paper, we introduce cartoon-texture decomposition to separate the texture and the noise component from a vehicle image. The most classical decomposition model is the Meyer’s models [18], which can force the cartoon part into the space of functions with bounded variation, and the texture part into a space of oscillatory distributions. Meyer’s models are simple and can perfectly describe cartoon component and texture component respectively. However, their numerical solution has proved challenging. There are many works thereafter in model variants and numerical attempts [15, 19–21]. Among those, authors in [22] developed an image filtering method, which retains the essential features of Meyer’s model and can be easily implemented. Inspired by [22], we design the decomposition filtering scheme for vehicle image. It is known that the Meyer’s models can be formulated aswhere may be selected as one of three norms: G-norm, F-norm, or E-norm, which are detailed described in [18]. While the most natural variational linear model associated with Meyer’s idea iswhere refers to the norm of in Sobolev space, and is the dual norm of norm, defined as . Using the Fourier transform in (2), it has and . Minimizing the functional in (2) in yields in Fourier the unique solution , whereis just a low-pass filter. To sum up, an image can be decomposed into its cartoon component and texture component by the linear filter pair:where denotes the identity operator. Vast experimental results show that the linear filtering method in (4) gives strikingly good results, but blurs slightly out edges in the cartoon component. The reason mainly lies in that a linear filter cannot keep the sharp edges in cartoon component. It is known that the total variation of a cartoon region does not decrease by low-pass filter, while a texture region possess high total variation due to its oscillation, and this total variation decreases very fast under low-pass filter, so it leads to a function at which is called local total variation (LTV):with this quantity, the local oscillatory behavior of an image can be evaluated by the relative reduction rate of LTV:If is close to , we have , i.e. the reduction under low-pass filter is significant, which means that the considered point belongs to a texture region. Instead, if is close to zero, then which means that there is little relative reduction of the local total variation by the low-pass filter. So the point is inclined to in a cartoon region. Therefore, the cartoon and texture components can be computed by weighted average of and depending on the relative reduction of , written asHere, is expected to be an increasing function that is equal zero near zero and equal to 1 near 1. One choice isIn equation (8), if is small, the function is non-oscillatory around and the function is cartoon around , thus . Otherwise, the function around belongs to the texture. Especially, we can adjust and to keep all step edges on the cartoon side, and put all fine structures on the texture side, as soon as they oscillate more than once. In this paper, we empirically set and .

To evaluate the decomposition quality, we introduce the correlation coefficient (“Corr” in short) [23] between the cartoon component and the texture component , defined aswhere and refer to the variance and covariance of given variables, respectively. Without loss of generality, we test the decomposition performance on the image “Barbara” and compare the performance of our method with that of [15]. We display the results in Figure 1, where Figure 1(a) is the original image, Figures 1(b) and 1(c) are the decomposition results of [15], and Figures 1(d) and 1(e) are the results of our method. It is clear that the filtering method (8) can depart cartoon and texture better and keep the main edge information in the cartoon component. We calculate the correlation coefficients of [15] and our method. They are and , respectively. It is concluded that our method outperform the algorithm in [15].

(a) Original image

(b) Cartoon of [15]

(c) Texture of [15]

(d) Cartoon of our method

(e) Texture of our method

Figure 1 

Image decomposition on “Barbara” image. (b-c) are the results of [15], the correlation coefficient between (b) and (c) is , (d-e) are the results of our filtering method (8), the correlation coefficient between (d) and (e) is .

2.2. Proposed LPD Algorithm

Our algorithm mainly comprises of four stages, which are summarized in Algorithm 1.

Input: Input car image (1) Gray scale conversion: converse RGB color image into gray-scale image (2) Improved edge detection: we first decompose image as , then detect the edge (3) LPD approach: we detect LP by morphological filtering and LP screening, (4) Tilt angle detection and correction: Output: Output license plate:

Algorithm 1 

Proposed LPD algorithm.

2.2.1. Gray Scale Conversion

We use simple digital camera for taking car images, which provides RGB color image as the input for the LPD. Most often a license plate have white characters on a blue background, most of the private-vehicle license plates have this formats. Moreover, there are also black characters on white background, black characters on yellow background, and so on. However, the color variations are not important for the LPD, because no color information is used as a feature for LP detection. The reason is that color information is highly sensitive to uneven illumination. So the proposed LPD uses gray scale or binary images. Conversion of an RGB color image into a gray scale one is performed as follows: where is pixel point, is a gray scale image and , , are the three color channels of an RGB color image [24].

2.2.2. Improved Edge Detection

Usually, a license plate is rectangular region, and has different color or gray from other regions. Therefore, edge information can be adopted to detection the license plate region. However, discontinuities of edges or images with complex backgrounds may not provide the desired results. In this paper, we improve the edge detection by utilize cartoon component of the vehicle image. since that the cartoon component mainly includes the geometric contours and the smooth regions, its edges will not have so many details and the weak edges. We substitute the edges of the original image with that of the cartoon image, which highlight the strong edges and overshadow the weak edges.

2.2.3. Morphological Filtering and LP Screening

Edge intensity images still have some relatively isolated and scattered points. Those points concentrated near the plate regions would negatively affect the location results. Thus, mathematical morphology operation [25, 26] can be applied to the edge intensity images to extract the license plate through the following steps.

(i) Obtaining candidate LP regions by morphological filtering method. Based on the prior information of the license plate, opening operation is conducted on the detected edges. Then they are jointed into connected domain using closing operation. As a result, the small connected domain are removed. By using the dilation operation, the tiny holes in the remaining object regions are filled using the dilation operation. Noting that, considering the length-width ratio of the license plate, the structural operator of the opening operation is selected as , the closing operation is , and the dilation operation is .

(ii) Selecting correct license plate by LP screening. After jointed them into connected areas, it needs to extract the information of each area, include the size, centroid, shape, position, the circumscribed rectangle and so on. Using the information, we judge whether the corresponding area is the one we need indeed. We determine the connected area we need and the bounding rectangle based on the characteristic of the license plate, exclude the false license plate area using the characteristic filter, and select the eligible area. Specifically speaking, we firstly label all the connected domain and extract each of them. Then divide them into separate area one by one. Finally, filter and select the eligible license plate according to the standard of the length-width ratio and the size we determined before. Experimental results show that the length-wide ratio varying in the interval, the wide of the license plate is less than one in three of the height of the image. According to this criterion, the correct LP can be detected.

2.2.4. Radon Transform Based Tilt Detection and Correction

In vehicle image acquisition process, it usually encounters the tilting problem for various reasons. This may affect the accuracy of subsequent process in license plate recognition system. So tilt angle detection is necessary and important.

In this paper, Radon transform is utilized to detect and correct the tilt problem of license plate. Radon transform of an image is the projection of the image intensity on a projection line having specific angle. Projection can be calculated by computing the line integrals from multiple sources along parallel paths, or beams, in a certain direction. In general, along some angle , the Radon transform of is the line integral of parallel to the -axis.where Figure 2 illustrates the geometry of the Radon transform.

(a) Original

(b) Cartoon 1

Figure 2 

Radon transform of along a projection line.

Take a vehicle image as example, Algorithm 1 is tested and the results are shown in Figure 3. Figure 3(b) is the cartoon component of Figure 3(a), the edge of the cartoon image is calculated by Sobel operator. By using morphological method, the candidate license plate regions are obtain and displayed in Figure 3(d), and the exact license plate in Figure 3(e). Taking Radon transform on Figure 3(e) we obtain the Radon image as Figure 3(f), from which its found the maximum value of the Radon transform are arrived at the angle . In the counterclockwise direction. we rotate the LP in Figure 3(e) by 4 degrees and obtain the correct LP as Figure 3(g). We crop the unnecessary upper and lower margins from the corrected LP. The final output of Algorithm 1 is shown as Figure 3(h). The experiment result confirms that Algorithm can successfully extract LP region from a vehicle image.

(a) Gray scale image

(b) Cartoon component

(c) Edge of

(d) Candidate regions of

(e) Exact LP of

(f) Radon transform of (e)

(g) Tilt correction of (e)

(h) Final output

Figure 3 

Example of the proposed detection algorithm. (a) is the vehicle image , (b) is its cartoon component , (c) is the edge image of , (d) includes the candidate LP regions, (e) is the detected LP, (f) is the Radon transformed image of (e), (g) is the tilt corrected LP and (h) is the final output.

3. Experimental Results

To validate the reliability of the proposed algorithm, we test on 400 photos of car image, which are obtained under different background, different light, and from different angle, to evaluate the algorithm from three aspects. We first evaluate the detection accuracy with or without cartoon-texture decomposition. Then test the performance of tilt correction. Finally, we compare the proposed method with two existing detection methods.

3.1. Effect of Cartoon-Texture Decomposition on LP Detection

Considering again the example in Figure 3(a), we compare the detection results of Algorithm 1 with the image decomposition step and the results without the image decomposition step. In Figure 4(a), the image decomposition is not considered. Five images from left to right are: gray scale image , the edge image of , the candidate LP regions, the detected license plate(s) by LP screening and the image of themselves. Six images in Figure 4(b) from left to right are: gray scale image , the cartoon component of , the edge image of , the candidate LP region, the detected license plate by LP screening and the image of itself. It is found that cartoon image involves only the large scale contour, so its edge profile is simpler than that of . With morphological filtering, the candidate LP regions of are less than that of . As a result, the detection results of include two license plates, and one of them is fake. While the detection result of is right.

(a) Detection process without the image decomposition step

(b) Detection process with the image decomposition step

Figure 4 

LP detection result with and without image decomposition.

3.2. Performance of Tilt Correction

Table 1 shows some tilt correction examples. The left column are detected LP regions without tilt correction. To make them correct, we apply radon transform on them. The middle column are the detected tilt angles. And the right column are the corrected LP with rotation transform. It is found that almost all the LP regions become horizontally straight in orientation after correction.

Table 1 

Examples of Tilt correction.

3.3. Comparison with Existing Methods

We compare the proposed LPD method with exiting algorithms [7, 16, 17]. Making full use of edge information, authors in [7] detected the license plate by combining edge detection and mathematical morphology method. The reason for selecting the algorithm in [16] is that it was also tested in hazardous condition, including different illumination and environment conditions, and from different angles. [17] is the first work which introduces the image decomposition method into the LPD. However, it was implemented by discrete difference equation which performs poorly in running speed. We implement them using MATLAB 2014a on personal computer, and apply on the sampled 200 vehicle images. The experimental results are reported in Table 2. In the table, the method [7] shows poor performance due to the complex background. No such special approach is taken in [7] to handle the complex environment conditions. Moreover, we report the detection results of one of the test images in Figure 5, including the running time in the captions. It is found that method [7] yields two plate regions, including a fake one. The methods in [16, 17] cost more time than ours.

Table 2 

Comparison with [7, 16].

(a) Test image

(b) Result of [7]:3.11s

(c) Result of [16]:1.53s

(d) Result of [17]:7.33s

(e) Result of ours:1.03s

Figure 5 

LP detection results of three methods. (a) is the vehicle image, (b-e) are the detecting results of [7, 16, 17] and the proposed method.

4. Conclusions

In this paper, a fast cartoon-texture decomposition filter is incorporated into license plate detection process. Combining with edge detection, morphologic filtering and Radon transform, we present a new LPD algorithm, which eliminates background and noise interference, such as small scale details and periodical change, and improves the accuracy of the recognition and location. This makes the algorithm has widely practicability. Moreover, the algorithm can be further developed by consider more complex situations, such as night, blurriness, rain and fog effects. This is the main direction of our future works.

Data Availability

The vehicle images used to support the ending of this study are composed of two parts. Part of the images is supplied by the database: https://sites.google.com/site/samicsemist/about-theteacher/mscthesisdetails. Another part of the images comes from our camera, requests for which after publication of this article will be considered by the corresponding author.

Conflicts of Interest

The authors declare that they have no conflicts of interest.

Acknowledgments

This work is supported by the National Natural Science Foundation of China under (Grant no. 61772389), Key Scientific Research Projects in Henan Colleges and Universities (19A110015), and Foundation of Shannxi Education Committee (16JK1603).