2D Barcode Image Decoding

Abstract

Barcode technology is one of the most important parts of Automatic Identification and Data Capture (AIDC). Quick Response code (QR code) is one of the most popular types of two-dimensional barcodes. How to decode various QR code images efficiently and accurately is a challenge. In this paper, we revise the traditional decoding procedure by proposing a serial of carefully designed preprocessing methods. The decoding procedure consists of image binarization, QR code extraction, perspective transformation and resampling, and error correction. By these steps, we can recognize different types of QR code images. The experiment results show that our method has better accuracy than Google open-source 1D/2D barcode image processing library Zxing-2.1. Moreover, we evaluate the execution time for different-size images. Our method can decode these images in real time.

1. Introduction

Barcode technology is one of the most important parts of Automatic Identification and Data Capture (AIDC); we can obtain decoded data through analysis of a barcode. According to the encoding type of barcodes, we divide barcode into two categories: one-dimensional barcode and two-dimensional barcode. One-dimensional barcode typically consists of varying the widths and spacings of parallel lines. Moreover, two-dimensional barcode is a graphical image that stores information both horizontally and vertically. Two-dimensional barcode compared with the one-dimensional barcode has the following advantages: (1) high data capacity; (2) no additional storages; (3) error correction ability.

Quick Response code (QR code) is one of the most popular types of two-dimensional barcodes developed in Japan by Denso Corporation in 1994. Although QR code was first designed for the automotive industry, QR codes are now used over much wider range of applications, including commercial tracking, transport ticketing, website Uniform Resource Locator (URL), and identity verification. QR code has several advantages. First, QR code has strong error correcting capability, which can restore 30% data for the maximum error correction level. Second, QR code can be scanned from any direction because the proportion of position detection patterns is not changed with the scanning direction. Third, QR code supports different encoding types and versions. We can choose an appropriate encoding type and version to reduce the size of QR code image.

Although there exist many commercial programs for QR code decoding, inventing more robust QR code decoding strategy still attracts many attentions from researchers. Many works are proposed recently. In [1, 2], Belussi and Hirata proposed a fast component-based two-stage approach for detecting QR codes in arbitrarily acquired images. Moreover Ohbuchi et al. [3] and Liu et al. [4] focused on enabling mobile phone to recognize QR code under poor conditions in real time. For decoding QR code image correctly and efficiently, we confront some challenges as follows. (1) When we capture QR code images using a phone camera in nonuniform background or uneven light, it might affect the decoded result. (2) The shooting angle might cause perspective distortion; to restore a distorted QR image is an important issue for decoding QR code images. (3) Decoding QR code images needs a sequence of sophisticated image processing. How to reduce the complexity of image processing and maintain the decoding accuracy rate is also a challenge.

In this paper, we focus on revising preprocessing methods in the traditional decoding procedure that is straightforward but still have much improving potential to overcome the above challenges. The decoding steps consist of image binarization, QR code extraction, perspective transformation and resampling, and error correction. In image binarization, we use a modified local threshold method to resolve the nonuniform background and uneven light problems. Our QR code extraction algorithm uses the position detection patterns of QR code efficiently and accurately to locate the QR code position. After locating the QR code position, we restore a distorted image through perspective transformation and resample. Finally, we get the corrected data by Reed-Solomon error correction algorithm.

2. Related Work

QR code recognition technology has been studied in past years. We can simply divide the recognition into two steps, image preprocessing and QR code extraction.

In image preprocessing, some researchers focus on image denoising [5] or camera shaking [6]. In order to improve the performance of low resolution QR-code recognition, a previous work [7] uses the super-resolution technique that generates a high resolution image from multiple low-resolution images. Moreover, some researchers use different binarizations to improve the nonuniform background and uneven light problems [8, 9].

In QR code extraction, researchers propose several different methods to locate and extract the QR code of images. Some researchers use the feature of finder pattern to find rough QR code position [10–12]. After estimating the QR code’s four corners using the rough QR code position, they effectively extract the QR code of images. In research [6, 13], they use edge detection to find possible rough barcode area. Then, morphological dilation and closing are used to generate more compact regions. Finally, the position of QR code of images can be detected.

Although these researches have their contributions, there are some shortcomings we can improve. In [5], researchers only focus on image denosing, but locating QR code position is an important part of QR decoding. They did not propose their method to process this problem. Previous works [6, 7, 10] need enormous calculation. Their methods are difficult to decode QR code image in real time. In research [9], their binarizaiton method needs to know the version of QR code in advance. In this paper, we propose a decoding method to improve the deficiencies in previous work.

3. Background

3.1. Perspective Transformation

QR code images may be distorted due to camera perspective projection. Therefore, we use perspective transformation to restore the distorted QR code images. The general representation of a perspective transformation [14] is 𝑥^′𝑦^′𝑤^′=𝑎𝑢𝑣1¹¹𝑎₁₂𝑎₁₃𝑎₂₁𝑎₂₂𝑎₂₃𝑎₃₁𝑎₃₂𝑎₃₃, (1) where 𝑥=𝑥^′/𝑤^′ and 𝑦=𝑦^′/𝑤^′. In (1), each original coordinate (𝑢, 𝑣) can be transformed to a new coordinate (𝑥, 𝑦) by using the 3 by 3 matrix. Based on (1), 𝑥 and 𝑦 can be written as 𝑥𝑥=^′𝑤^′=𝑎₁₁𝑢+𝑎₂₁𝑣+𝑎₃₁𝑎₁₃𝑢+𝑎₂₃𝑣+𝑎₃₃,𝑦𝑦=^′𝑤^′=𝑎₁₂𝑢+𝑎₂₂𝑣+𝑎₃₂𝑎₁₃𝑢+𝑎₂₃𝑣+𝑎₃₃. (2)

Without loss of generality, the 3 by 3 matrix can be normalized, so that𝑎₃₃=1. Equation (2) can be rewritten as 𝑥=𝑎₁₁𝑢+𝑎₂₁𝑣+𝑎₃₁−𝑎₁₃𝑢𝑥−𝑎₂₃𝑣𝑥,𝑦=𝑎₁₂𝑢+𝑎₂₂𝑣+𝑎₃₂−𝑎₁₃𝑢𝑦−𝑎₂₃𝑣𝑦. (3)

Applying four pairs of correspondence points to (3), we can get a linear system as follows: 𝑢₀𝑢₁𝑢₂𝑢₃0000𝑣₀𝑣₁𝑣₂𝑣₃0000111100000000𝑢₀𝑢₁𝑢₂𝑢₃0000𝑣₀𝑣₁𝑣₂𝑣₃00001111−𝑢₀𝑥₀−𝑢₁𝑥₁−𝑢₂𝑥₂−𝑢₃𝑥₃−𝑢₀𝑦₀−𝑢₁𝑦₁−𝑢₂𝑦₂−𝑢₃𝑦₃−𝑣₀𝑥₀−𝑣₁𝑥₁−𝑣₂𝑥₂−𝑣₃𝑥₃−𝑣₀𝑦₀−𝑣₁𝑦₁−𝑣₂𝑦₂−𝑣₃𝑦₃×𝑎₁₁𝑎₂₁𝑎₃₁𝑎₁₂𝑎₂₂𝑎₃₂𝑎₁₃𝑎₂₃=𝑥₀𝑥₁𝑥₂𝑥₃𝑦₀𝑦₁𝑦₂𝑦₃. (4)

The value of unknown coefficients will be determined by solving the linear system in (4).

3.2. Reed-Solomon Error Correction

Reed-Solomon codes are nonbinary cyclic error-correcting codes [15]. In QR code decoding procedures, we might obtain some error information because of distorted images or inappropriate binarization. Reed-Solomon codes help us to correct the error information. The general decoding algorithm [16] of RS codes consists of computing the syndromes, determining an error locator polynomial, and solving the error values.

For computing the syndromes, the received message is viewed as the coefficients of a polynomial 𝑆(𝑥): 𝑆(𝑥)=^𝑛−1_𝑖=0𝑟_𝑖𝑥^𝑖, (5) where 𝑟_𝑖 is the received message at position 𝑖. Therefore, the syndromes (𝑆₀,𝑆₁,…,𝑆_2𝑡−1) are defined as 𝑆₀=𝑆𝛼⁰,𝑆₁=𝑆𝛼¹,…,𝑆_2𝑡−1=𝑆𝛼^2𝑡−1, (6) where 2𝑡 is equal to the number of codewords available for error correction and 𝛼 is the primitive element on the Galois field. Syndrome polynomial is defined as𝑆(𝑥)=^2𝑡−1_𝑖=0𝑆_𝑖𝑥^𝑖. (7)

After computing syndrome polynomial, we use Euclidean algorithm to construct the error locator polynomial and error evaluator polynomial. Error locator polynomial can be used to find the error positions. Error evaluator polynomial is used to find the error values. The Euclidean algorithm is based on the key equation in Ω(𝑥)=𝑆(𝑥)Λ(𝑥)mod𝑥^2𝑡. (8)

Given only 𝑆(𝑥) and 𝑡, we want to obtain the error locator polynomialΛ(𝑥) and the error evaluator polynomial Ω(𝑥). Equation (8) means that 𝜃(𝑥)𝑥^2𝑡+𝑆(𝑥)Λ(𝑥)=Ω(𝑥) (9) for some polynomial 𝜃(𝑥). Finally, we can use Euclidean algorithm to generate the error locator polynomial and error evaluator polynomial.

For solving the error values, Forney's algorithm is selected. The error values for a Reed-Solomon code are computed by 𝑒_𝑖𝑘Ω𝑋=−^𝑘−1Λ^′𝑋_𝑘⁻¹, (10) where 𝑒_𝑖𝑘 is the corrected value at position 𝑖_𝑘; 𝑋_𝑘⁻¹ is the root ofΛ(𝑥) at position 𝑘; and Λ^′(𝑥) is the formal derivative ofΛ(𝑥). By (10), we obtain the corrected data of QR code images.

4. The Proposed Method

We design a QR code recognition procedure for decoding QR code accurately and rapidly. First, we propose a new local binarization method to overcome images with uneven light and nonuniform background problems. And then we use three finder patterns and an alignment pattern of QR code efficiently to locate the QR code position. After locating QR code position, we resample QR code image by perspective transformation. Finally, we get the corrected data by Reed-Solomon error correction algorithm. Figure 1 illustrates the procedure.

Figure 1 

QR code decoding procedure.

4.1. Binarization

Binarization is an important process for accurately recognizing black-and-white module in QR code images. Therefore, we propose a new binarization method to improve the recognition of images with nonuniform background and uneven light. Our method is a local threshold algorithm based on Sauvola’s method [8]. We decompose each image into a series of blocks and then calculate a threshold 𝑇₁ for each block using 𝑇₁×𝑥,𝑦=mean𝑥,𝑦1+standarddeviation𝑥,𝑦𝑅. (11)

The default 𝑅 is 1250. For each block, we convolve with the kernel in (12) to calculate a threshold 𝑇₂. By using the threshold 𝑇₂, we split each block into bright area and dark area. Figures 2 and 3 shows that our method can achieve the same performance with less computing cost, compared with modified Sauvola’s method [8]: 1𝐶=𝑓()=()−().10111121111,𝐴,𝐵,𝐶𝐵⋅𝑥−𝐴⋅𝑥𝐶⋅𝑦−𝐴⋅𝑦𝐵⋅𝑦−𝐴⋅𝑦𝐶⋅𝑥−𝐴⋅𝑥 (12)(13)

Figure 2 

An uneven light image [17].

Figure 3 

Binary image using our method.

4.2. Estimating Version

QR code extraction is responsible for searching QR code area in images. QR code extraction consists of searching finder patterns, estimating version, and searching alignment pattern. We use the proportion of finder patterns to locate the QR code position. After searching three finder patterns, we use the distance between these patterns to estimate version. Finally, we develop a connected-component method to search alignment pattern accurately.

4.2.1. Searching Finder Patterns

In searching finder patterns, we need to determine the upper left, lower left, and upper right finder patterns in the QR code. First, we scan the binary image horizontally to find all points matching the ratios 1 : 1 : 3 : 1 : 1. For each point matching the ratios, we check its vertical direction. Finally, we have collected all points in images matching the ratios 1 : 1 : 3 : 1 : 1 both horizontally and vertically.

After collecting points, we merge points in the same finder pattern. If the number of finder patterns is more than three, we filter out some inappropriate points. First, we calculate the angle between three points. If the maximum angle is greater than a threshold 𝑇₁ or the minimum angle is less than a threshold 𝑇₂, the points will be filtered. And then we identify the direction of finder patterns. The maximum angle corresponding to the finder pattern is the upper left finder pattern. Finally, we use cross product in (13) to determine the lower left and upper right finder patterns, where 𝐴 is the upper left finder pattern; 𝐵 and 𝐶 are the other finder patterns; and 𝐴⋅𝑥 and 𝐴⋅𝑦 represent the 𝑥 and 𝑦 coordinate of 𝐴. If 𝑓(𝐴,𝐵,𝐶)<0, then 𝐵 is upper right finder pattern, and 𝐶 is lower left finder pattern.

4.2.2. Estimating Version

After searching three finder patterns, we can estimate version using module size and the distance between two finder patterns. Module size can be calculated by Module_size=Widthoffinderpattern7. (14) We calculate version by =𝑓VersionEuclidean_distance₁,𝑓₂/Module_size−104, (15) where 𝑓₁ is upper left finder pattern and 𝑓₂ is upper right finder pattern. We can get rough version by (15), but (15) misjudges the version when QR code image is rotated. We overestimate the width of finder pattern in rotated image in Figure 4.

Figure 4 

Red line is the width of finder pattern, and blue line is the estimated width.

As a result, we underestimate the version of QR code image. Therefore, we revise (15) to be=𝑓VersionEuclidean_distance₁,𝑓₂/Module_size∗cos𝜃−104, (16) where 𝜃 is the angle between blue line and red line in Figure 4. We estimate version more accurately by (16) and thus before perspective transformation.

4.3. Searching Alignment Pattern

Low resolution image or inappropriate binarization method will result in the deformation of alignment pattern. Therefore, we propose a new searching alignment method to overcome the problem. First, we roughly locate the alignment pattern using three finder patterns. Moreover, we extract subimage with a specified radius in Figure 5. We calculate the connected component in the subimage, both black pixels and white pixels. After calculating the connected component, we get two images in Figures 6(b) and 6(c).

Figure 5 

We extract blue rectangle to find the alignment pattern (radius = 30 pixels).

(a)

(b)

(c)

Figure 6 

(a) Extracted image in Figure 5; (b) found connected components of black pixels; (c) found connected components of white pixels.

For each connected component of black pixel 𝐶_𝑖 in Figure 6(b), we check its border pixels. If all border pixels are adjacent to the same connected component of white pixel 𝐶^′_𝑗 in Figure 6(c), then we check all border pixels in 𝐶^′_𝑗. Once all border pixels in 𝐶^′_𝑗 are adjacent to the same black pixel connected component 𝐶_𝑘 in Figure 6(b), these components possibly contain alignment pattern. Finally, we calculate the centroid of 𝐶_𝑖 as the centroid of alignment pattern.

4.4. Perspective Transformation

After estimating the version and locating the centroid of three finder patterns and alignment pattern, we resample QR code image through perspective transformation. The eight coordinates can be presented as shown in Table 1.

Table 1 

Eight coordinates used for perspective transformation (𝐷=17+4^∗version).

To avoid the appearance of cracks and holes, we use inverse warping in perspective transformation. First, we calculate a transformation 𝑇 from normalized coordinate to original coordinate. For each pixel in normalize image, we calculate its corresponding location in original image using the transformation 𝑇. Finally, we copy the pixel value of original image to the normalized image. Table 2 shows our resampling results.

Table 2 

Result of resampling.

5. Experiment Result

Experimental environment:(1)CPU: Intel core i5-3210M CPU 2.50 GHz,(2)memory: 4 GB,(3)OS: Windows 7,(4)programming language: Java.

In this section, we conduct several experiments to evaluate the performance. First, we compare the accuracy between our QR code decoding algorithm and 2D barcode image processing library (Zxing) [17]. The experiment result shows that our decoding algorithm has better accuracy than Zxing. And then we use different sizes of images to evaluate the decoding time. The experiment results show our method decoding images in real time. Moreover, we implement a QR code scanner on Android phone. We test the decoding ability of some defaced images. The experiment result shows that our QR code scanner successfully decodes these images.

5.1. Image Source

Our test images in Table 3 are from Google open-source 1D/2D barcode image processing library Zxing-2.1 [17]. The images are classified into six data sets, a total of 178 images. The first data set contains low resolution QR code images. The second data set contains trademark QR code image. The skew QR code images are included in the third data set. The fourth data set contains distorted QR code images. Uneven light QR code images are included in the fifth data sets. The sixth data set contains rotated QR code images.

Table 3 

Test images.

5.2. Accuracy Analysis

We compare the accuracy between our decoding algorithm and Google open-source 1D/2D barcode image processing library Zxing-2.1 [17]. Table 4 shows the results.

Table 4 

Comparison between our method and Zxing-2.1.

In 178 test images, there are 8 images in Table 5 that we successfully recognize and Zxing-2.1 fails. Moreover, there are 3 images in Table 6 that Zxing-2.1 successfully recognizes and our method fails.

Table 5 

Images we successfully recognize and Zxing-2.1 fails.

Table 6 

Images Zxing-2.1 successfully recognizes and our method fails.

The accuracy of our method is about 90% and that of Zxing-2.1 is only 87%. Our method has better recognition rate in Sets 1 and 2. Zxing-2.1 cannot accurately locate the alignment pattern because of the low resolution or inappropriate binarization. Our method improves the problem, so that we have better result.

5.3. Efficiency Analysis

In this section, we show the execution time between our method and Zxing for decoding different sizes of QR code images. We use four different sizes of QR code images to calculate the execution time. Each size contains five images. Table 7 shows the results.

Table 7 

The execution time between our method and Zxing for decoding different-size images.

Our method needs more decoding time than Zxing, but our method still can decode QR code images in real time. We only need 448 milliseconds for decoding resolution 1024∗768 pixel images.

5.4. Comparison of Different Barcode Applications

We implement a QR code scanner on Android phone. Moreover, we collect four popular barcode scanners from Google Play and iOS market. We compare the decoding ability of some defaced images. Table 8 shows the result. Our application can successfully decode all test images. Our application has the best result of these applications in these images. The reason for our better results is that two steps of the binarization stage in our method will get a more appropriate threshold to split each block into bright area and dark area. Moreover compared with current applications, our new searching alignment method could handle a large deformation of alignment pattern as the result of inappropriate binarization and low resolution which can be observed in sample images.

Table 8 

Comparison of different barcode scanner applications.

6. Conclusion

In this paper, our carefully designed preprocessing methods improve the accuracy of the traditional decoding procedure. Although refined steps improve accuracy at the expense of increased decoding time [21], the QR code still can meet the real-time requirement. The decoding steps consist of image binarization, QR code extraction, perspective transformation, and error correction. We efficiently resolve the non-uniform background and uneven light problems by using a modified local threshold algorithm. In QR code extraction, we use the position detection patterns of QR code efficiently and accurately to locate the QR code position. After locating the QR code position, we restore a distorted image through perspective transformation and resampling. Finally, we get the corrected data by Reed-Solomon error correction algorithm.

In the experiment, we compare the accuracy between our method and Zxing. The experiment result shows our method to have better accuracy than Zxing. Moreover, we use different-size images to calculate the execution time. The experiment shows that our method can efficiently decode images. All images can be decoded in less than one second.