Target Image Matching Algorithm Based on Binocular CCD Ranging

Abstract

This paper proposed target image in a subpixel level matching algorithm for binocular CCD ranging, which is based on the principle of binocular CCD ranging. In the paper, firstly, we introduced the ranging principle of the binocular ranging system and deduced a binocular parallax formula. Secondly, we deduced the algorithm which was named improved cross-correlation matching algorithm and cubic surface fitting algorithm for target images matched, and it could achieve a subpixel level matching for binocular CCD ranging images. Lastly, through experiment we have analyzed and verified the actual CCD ranging images, then analyzed the errors of the experimental results and corrected the formula of calculating system errors. Experimental results showed that the actual measurement accuracy of a target within 3 km was higher than 0.52%, which meet the accuracy requirements of the high precision binocular ranging.

1. Introduction

Binocular CCD ranging is an optical distance measuring method, which imitates human beings’ distance perception using their eyes to obtain the image of the target object. It is a passive ranging in which the high precision ranging over a long distance can be achieved under natural conditions without sending out any signal to the target object. Binocular CCD ranging is characterized by concealment, which is especially suitable for military equipment ranging, aerial target ranging, and occasions needed to be kept secret and restricted by environment in industrial applications.

Image matching, the most core and key technique in binocular CCD ranging, has always been the research hotspot and difficulty for many experts and scholars. Many countries in the world have used weapons with image matching guidance technology, and the earliest one of the weapons is the cruise missile, which has very high concealment. The image matching technology aims to improve the guidance precision of long-range tactical, strategic missile and cruise missile so that the guidance precision is not limited by range distance. Due to the fact that the completion of target matching needs a wide range of image search and that the aircraft inertial navigation system has unstable errors, the matching speed and reliability become the key to matching positioning technology. Therefore, the research for image matching algorithm becomes the key to achieve good performance of guidance. Image matching is proposed by the United States when it was engaged in researches such as the terminal guidance of the projection system and auxiliary aircraft navigation systems in the 1970s and then slowly won the support and funding of the US army. With the rapid development of science and technology, the application of the image matching technology has gradually expanded from purely military purposes to medical image analysis, remote sensing image processing, machine vision, industrial control, and other fields. Therefore, the research for image matching technology has its important theoretical value and application prospect.

The so-called image matching is to find one or more mapping transformation in the changing space of the image and register in the space two or more images of the same target scene, under conditions of different time periods, sensors, and perspectives or recognize its corresponding mode in another target image based on the known target schema. In binocular CCD ranging system, the stereo matching of image is the key link, which determines the main precision of the whole system measurement.

Here is an outline of the paper. In Section 2, the target images are first acquired through the design of binocular CCD ranging system. The target features are extracted and more appropriate regions to the feature region size are chosen to match in Section 3. Section 3 intends to describe how to the subpixel level matching of the target images in conducted by adopting the improved cross-correlation algorithm and three-surface fitting method. To demonstrate the efficiency and reliability of the method, some numerical test examples with comparisons are given in Section 4. Section 5 concludes the paper.

2. The Principle of Binocular CCD Ranging

The principle of binocular CCD ranging is based on the binocular parallax, which is to determine the pixel coordinates position of the same target images on the right and the left images and then calculate their pixel difference, also called binocular parallax, so as to obtain the target distance based on binocular CCD ranging model. Figure 1 is the schematics of binocular CCD ranging. The baseline distance which is a distance that connect the two cameras’ projection center, expressed as 𝐵 [1, 2].

Figure 1 

Schematics of binocular CCD ranging.

When Target 𝐴 passes by the binocular ranging system formed by cameras on the right and left with two paralleled optical axes, the imaging, respectively, lies on the surface of the left CCD as Point 𝐴₁ and the right CCD as Point 𝐴₂. Its locations on the two focal planes are 𝑥_Left and 𝑥_Right. Since the focal lengths of both cameras are 𝑓, according to the principle of similar triangle, the measured distance 𝑙 can be deduced as [3] 𝑙=𝐵𝑓𝑥𝑥=𝑥_Left+𝑥_Right, (1) where 𝑥 is the position difference of the images that 𝐴’s images on the surface of the left CCD and the right one, also known as binocular parallax. It can be seen that under the ideal state that optical axis of binocular camera is strictly paralleled and the target object images are obtained as well, and through the image matching algorithm, the relevant position of the same target in CCD image is determined and the binocular parallax 𝑥 can be calculated. The focal length and baseline size having been known, the target distance 𝑙 can be concluded according to formula (1).

Binocular CCD ranging system is to use the principle of binocular stereo vision to realize the target image matching by imaging of the same target on different positions through different locations around the camera image on the right and the left. Image matching results directly affect the distance measurement results of binocular CCD ranging system, so as to ensure the accuracy of image matching is the key to the high precision binocular CCD ranging.

3. High Precision Image Matching

3.1. Fast Matching of Pixel Level

This paper adopts left and right CCD camera model, with the parameters maintaining consistency, the optical axis remaining basically paralleled, the binocular camera coordinate systems keeping in coplanar, and the axis being arranged in parallel. Therefore, the size and proportion of image acquired synchronously are in agreement, and the gray information of images can be less affected by noise. Therefore, the image matching algorithm based on template matching gray cross-correlation is adopted, which is most suitable for binocular CCD ranging system. Its principle is shown in Figure 2.

(a) Search graph 𝑆

(b) Template graph 𝑇

Figure 2 

Template and search image.

In Figure 2, graph 𝑆 is 𝑁_𝑥∗𝑁_𝑦, graph 𝑇 is 𝑀_𝑥∗𝑀_𝑦. When matching, template image traverses each pixel in the reference map for search. The search subgraph 𝑆^𝑖,𝑗, with the same size as template graph, has coordinate in the top-left (𝑖, 𝑗), and the value ranges of 𝑖 and 𝑗 are 0≤𝑖,𝑗≤𝑁−𝑀, respectively. Comparing the similarity of the gray value of each pixel in 𝑇 and 𝑆^𝑖,𝑗, if 𝑆^𝑖,𝑗 is in accordance with 𝑇, then they are phase matching.

This paper uses the normalized cross-correlation matching algorithm to compare the similarity of 𝑆^𝑖,𝑗 and 𝑇. The expression of cross-correlation function is 𝑁𝑖,𝑗=^𝑀𝑁_𝑚=1𝑛=1𝑇(𝑚,𝑛)−𝑇()×𝑆𝑚,𝑛_𝑖,𝑗(𝑚,𝑛)−𝑆^𝑖,𝑗()×𝑚,𝑛_{𝑀𝑁𝑚=1𝑛=1}𝑇(𝑚,𝑛)−𝑇(𝑚,𝑛)²×^𝑀𝑁_𝑚=1𝑛=1𝑆^𝑖,𝑗(𝑚,𝑛)−𝑆^𝑖,𝑗(𝑚,𝑛)^2−1/2,𝑇()=1𝑚,𝑛𝑀×𝑁^𝑀𝑁_𝑚=1𝑛=1𝑇(𝑚,𝑛),𝑆^𝑖,𝑗()=1𝑚,𝑛𝑀×𝑁^𝑀𝑁_𝑚=1𝑛=1𝑆^𝑖,𝑗(𝑚,𝑛), (2) where 𝑇(𝑚,𝑛) is the template image and its size is 𝑀×𝑁. 𝑇(𝑚,𝑛) is the average value of all the gray scales of pixels in 𝑇(𝑚,𝑛). On the reference image, the search area of the image using (𝑖,𝑗) as the top-left pixels is 𝑆^𝑖,𝑗(𝑚,𝑛). 𝑆^𝑖,𝑗(𝑚,𝑛) is the average value of all the gray scales of pixels in the search image. Image matching is to match the pixel at top-left corner of the template on the target image area. The values range of cross-correlation function 𝑁(𝑖,𝑗) is 0≤𝑁(𝑖,𝑗)≤1, and its value is determined by the matching degree of search image region and template image using (𝑖,𝑗) as the pixel at top-left corner. Due to the image difference, it is often impossible to get the pixel whose value of cross-correlation function is 1, while the greater the cross-correlation value of the pixel is, the higher the matching degree of the pixel will be. Therefore, the most matching pixel is the one with the biggest value of cross-correlation function [3].

In order to achieve the fast matching of images and improve the processing speed of the algorithm, fast image matching method described in [4] is used. The down sampling of image is made first and then the target image area is roughly matched with the reduced image size. After finding the rough matching point by using cross-correlation function, the matching point is restored to the corresponding position in the original image. The restored pixel location of the point is considered as the center for the accurate pixel matching point, which will be found by continuous cross-correlation operations in its surrounding pixels. Thus, it greatly reduces the amount of data of cross-correlation operation, saves the matching time, and realizes the accurate image matching pixel level.

3.2. Description of the Fast Matching Algorithm

The steps of improved fast matching algorithm are the following.

The first step is to extract the target area of the matching image. When the target image was captured by using binocular CCD camera of binocular ranging system, complete target image matching area “前” appeared on the left and right images. The extraction of target image area is conducted to the target image “前” on the left CCD image and that area is used as a template, preparing for finding out the matching area of the corresponding target image “前” on the CCD image. The actual size of the captured image is about 1300 × 1024 pixels. In order to obtain the complete target image, the template size of the matching target image is 200 × 200 pixels.

As shown in Figure 3, (a) is the image of left CCD camera and (b) is the target area extracted from the left CCD image as the template matching on the right image. The purpose is to find in the right CCD image the upper left corner matching point on the matching template area “前” extracted from the left CCD, the area of target image to be matched image.

Figure 3 

Extraction of the target image to be matched.

The second step is to downsample the right CCD image to be matched and target area template image extracted from the left CCD image. According to the principle of bilinear interpolation [5–7], 4 times downsample processing is taken to the template image and the right CCD image simultaneously. In Figure 4, (a) is the target template image extracted from the left CCD image and its size is 200 × 200 pixels; (c) is 800 × 600 pixels image, captured with the right CCD camera; (b) is the target template image after being downsampled four times which is lessened into a 50 × 50 pixels; (d) is the right image after being downsampled four times and reduced to 200 × 150 pixels.

(a) Target template image of the left one

(b) The 4 times sampling of Figure (a)

(c) The right CCD image

(d) The 4 times sampling of Figure (c)

Figure 4 

Downsampling of the target template image and the original left image.

The third step is to do rough matching operation on the right CCD image and the template image after being downsampled. As shown in Figure 4, the cross-correlation template matching operation is done between the target template image after being downsampled as (b) and the right image (d). Figure 4(b) can be used as the template to do the template search on (d), making cross-correlation operation for each pixel and finding the coordinate of cross-correlation peaks of the correlation function values (𝑚₀, 𝑛₀), which is the rough matching point of the upper left corner of the image coordinates of the target template.

The fourth step is to restore the rough matching point to its original image and select the appropriate area to search the pixel matching point. The rough matching point coordinates are (𝑚₀, 𝑛₀), which is found in the right image after downsampling in the previous step. When it is restored to its original size of the image, the matching point in the picture should be on the area with pixels coordinates (4𝑚₀, 4𝑛₀). Then the search of the exact matching point should be made on the appropriate area of the original size CCD image size of 160×80 pixels which is cut out of the central point (4𝑚₀, 4𝑛₀). The area selected can be neither too small nor too big, for it is very easy to exclude the precise matching if it is too small and increase the amount of calculation data if it is too big.

The fifth step is to use the original target template image extracted from the left CCD image to do the cross-correlation matching operation on 160×80 pixel area extracted from the right CCD image in the previous step and determine its correlation peak coordinates so that the precise matching point coordinates in the right CCD image of the target template image extracted from the left CCD image can be found.

By implementing these methods of fast matching images, it not only reduces the amount of data calculation and avoids the template matching operation of all pixels on the big size image, but also uses the method of performing the rough matching firstly and then locating precisely to find the relevant peak coordinates of the relevant cross-correlation function distribution and completing the pixel-level matching image.

3.3. The Subpixel Level Image Matching of CCD Ranging Image

The accuracy of image matching is the key to achieving the precision distance measuring of the binocular CCD ranging system [8]. Therefore, the accuracy of the image pixel level cannot meet the requirements of the binocular CCD ranging system. In this sense, this paper aims to realize the subpixel level matching of the target image.

In this paper, three-surface fitting method [9] is adopted to fit the subpixel level coordinates of the correlation peaks. The cross-correlation peak pixel coordinates of the distribution of the correlation function is taken as a central point; the pixel points around the area of 20×20 pixels are selected to fit the fitting points three times; then they are taken into the cross-correlation function with the binary representation of a cubic polynomial. The formula is just as follows: 𝑁𝑖,𝑗=^3𝑘_𝑘=0𝑛=0𝑎_𝑘𝑗𝑥^𝑗𝑦^𝑘−𝑗. (3)

Horizontal and vertical coordinates of about 21×21=441 pixels around the correlation peak and its surrounding can be brought to the binary cubic polynomial, using the least squares method [10] to obtain the expression of the coefficient 𝑎_𝑘𝑗 and get the complete expression of binary cubic fitting function. The following diagram in Figure 5 is the three-surface fitting. Using the expression of the binary cubic polynomial, the maximum value of the cubic surface fitting function can be figured out, which is the fitting matching subpixel coordinates and the target image subpixel level matching is completed.

Figure 5 

The result of three-surface fitting.

4. Experimental Results and Analysis of Image Matching

4.1. Experimental Results of Binocular CCD Ranging Experiment

The experiment adopts the CCD camera of OK-AC1300 model with a resolution of 1300×1024 pixels, two pixels size of 4.65 μm, and working output 8 bitRGB image and chooses the German deterrence Walimex500 mm F8-32 telephoto fixed focal length lens with 𝑓 = 500 mm. In the experiment, different targets on the distance of about 3 km plurality are measured. Ranging image groups are as follows.

The chosen target is “Energy Bureau of Jilin Province.” The images and the selected target in the experiment are shown as in Figure 6.

(a) The left CCD image

(b) The right CCD image

(c) The target image

Figure 6 

Ranging image groups for “Energy Bureau of Jilin Province.”

Figure 6 is a group of target images in the CCD ranging. The experiments use GPS navigation and positioning to measure the target shooting location and its latitude and longitude, a distance of 2.2631 km. To test the stability of the CCD ranging system and the running of the algorithm, several groups of target images are captured and matched during the experiment. The software interface of image matching is shown as in Figure 7.

Figure 7 

Software interface of image matching.

Combining the calibration parameters of the far-distant CCD binocular ranging system, the measurement results of experiments are obtained, which are shown in Table 1.

Table 1 

Measurement data of ranging image groups for “Energy Bureau of Jilin Province.”

4.2. The Error Analysis of Binocular CCD Ranging Results

In this paper, the formula of the binocular CCD ranging is modified and the two-dimensional case is discussed, pointing out that the actual three-dimensional imaging calculation is just the expanse of the two-dimensional calculation, which does not affect the calculation of the formula. Binocular vision system imaging model [11, 12] is shown as in Figure 8. 𝑂_𝐿 and 𝑂_𝑅 are the optical centers of the left camera and the right camera, respectively, 𝑂_𝐿𝑍_𝐿 and 𝑂_𝑅𝑍_𝑅 are the axes of the two cameras, respectively, and 𝑓_𝐿 and 𝑓_𝑅 are the focal lengths of the two cameras, respectively.

Figure 8 

Imaging model of binocular vision system.

In Figure 8, 𝑄₁ is setas a calibration point. It is assumed that the coordinates of this calibration point in the camera coordinate system are (x_1L, z_1L) and (x_1R, z_1R). According to a first-order radial distortion pinhole model, the ideal position of 𝑄₁ on the image plane is 𝑢₁₁, 𝑢₁₂, but its actual location is 𝑢^′₁₁, 𝑢^′₁₂. It is assumed that the two-dimensional position errors of each pixel are equal. They are both 𝐸, which is as follows: 𝐸=𝑋_𝑑−𝑋_𝑢²+𝑌_𝑑−𝑌_𝑢². (4)

According to the position relationship between 𝑄₁ and the point position of the two-dimensional image 𝑢^′₁₁, 𝑢^′₁₂, the position of the optical center of the camera 𝑂^′_𝐿 and 𝑂^′_𝑅 can be obtained with the linear method to obtain.

The coordinates of the 𝑢₁₁ and 𝑢₁₂ in the left and right cameras coordinate system which are obtained from the straight 𝑂_𝐿𝑄₁ and the intersection of the image plane are as follows: 𝑢₁₁=−𝑓_𝐿𝑥_1𝐿𝑧_1𝐿+𝐸,−𝑓_𝐿,𝑢₁₂=𝑓_𝑅𝑥_1𝑅𝑧_1𝑅,−𝑓_𝑅. (5)

It can infer 𝑢^′₁₁=−𝑓_𝐿𝑥_1𝐿𝑧_1𝐿+𝐸,−𝑓_𝐿,𝑢^′₁₂=𝑓_𝑅𝑥_1𝑅𝑧_1𝑅+𝐸,−𝑓_𝑅. (6)

Based on the line 𝑢^′₁₁𝑄^′₁, the coordinate of 𝑂^′_𝐿 in the coordinate system of the left camera is (𝑍_1𝐿𝐸/(𝑧_1𝐿+𝑓_𝐿),0). Similarly, the coordinate of 𝑂^′_𝑅 in the right camera coordinate system is (𝑡_𝑥−(𝑍_1𝐿𝐸/(𝑍_1𝐿+𝑓_𝐿)),0). Then the testing points were restored by using the obtained camera parameters. Assuming the actual coordinates of the test points for 𝑄₂ as (𝑥_2𝐿,𝑧_2𝐿) and (𝑥_2𝑅,𝑧_2𝑅), in theory testing image point 𝑄₂ should be 𝑢^′₂₁, 𝑢^′₂₂. Defined 𝑢^′₂₁ and 𝑢^′₂₂, respectively, are 𝑢^′₂₁=−𝑓_𝐿𝑥_2𝐿𝑍_2𝐿+𝐸,−𝑓_𝐿,𝑢^′₂₂=𝑡_𝑥−𝑓_𝑅𝑡_𝑥−𝑥_2𝐿𝑧_2𝐿−𝐸,−𝑓_𝑅. (7)

Due to the error effects caused by the matching, the corresponding image points of 𝑄₂, respectively, are detected as 𝑢^′′₂₁, 𝑢^′′₂₂ and the matching error is 𝐷; thus 𝑢^′′₂₂=𝑡_𝑥−𝑓_𝑅𝑡_𝑥−𝑥_2𝐿𝑧_𝑅−𝐸−𝐷,−𝑓_𝑅. (8) Therefore, the baseline length of 𝑂^′_𝐿, 𝑂^′_𝑅 is 𝑏=𝑡_𝑥𝑧−2^2𝐿𝐸𝑧_2𝐿+𝑓_𝐿. (9) And the parallax 𝑑 is calculated as 𝑓𝑑=^𝑅𝑡_𝑥−𝑥_2𝐿𝑧_2𝑅𝑓+𝐸+𝐷+^𝐿𝑥_2𝐿𝑧_2𝐿+𝐸−2𝑧_2𝐿𝐸𝑧_2𝐿+𝑓_𝐿. (10)

According to the CCD camera diagram, the depth value of 𝑄₂ can be obtained as follows: 𝑙^′=𝑡_𝑥−2𝑧_2𝐿𝐸/𝑧_2𝐿+𝑓_𝐿𝑓×𝑓_𝑅𝑡_𝑥−𝑥_2𝐿/𝑧_2𝑅+𝑓+𝐸+𝐷_𝐿𝑥_2𝐿/𝑧_2𝐿−+𝐸2𝑧_2𝐿𝐸/𝑧_2𝑙+𝑓_𝐿⁻¹. (11)

The system error is calculated as 𝑙𝑒=𝑙^′−𝑙,𝑙=𝑧_2𝐿. (12)

In this paper, when calculating the matching error, the system error 𝐷 is 0.05 pixel, and the calibration error 𝐸 is 0.5826 pixel. Therefore, this system error of the paper is about 0.52%.

5. Conclusions

In this paper, the proposed image matching algorithm is adopted to improve the accuracy of the binocular CCD passive ranging. During the measurement, the improved cross-correlation algorithm and three-surface fitting algorithm to the target image are applied to achieve the subpixel level image matching, improving the accuracy of ranging. Through the error analysis of measurement results, the paper modified the calculation formula of system errors. Experimental results show that the target measuring accuracy within 3 km is higher than 0.52%, which is of great practical value.

Conflict of Interests

The authors declare that there is no conflict of interests regarding the publication of this paper.

Acknowledgments

This project is supported by the scientific and technological research project of Jilin province Department of Education, China (no. 201363, no. 2013145), and basis study project supported by Jilin Provincial Science & Technology Department, China (no. 201105055).