Artificial Intelligence and Data Mining 2014View this Special Issue
Research Article | Open Access
Target Image Matching Algorithm Based on Binocular CCD Ranging
This paper proposed target image in a subpixel level matching algorithm for binocular CCD ranging, which is based on the principle of binocular CCD ranging. In the paper, firstly, we introduced the ranging principle of the binocular ranging system and deduced a binocular parallax formula. Secondly, we deduced the algorithm which was named improved cross-correlation matching algorithm and cubic surface fitting algorithm for target images matched, and it could achieve a subpixel level matching for binocular CCD ranging images. Lastly, through experiment we have analyzed and verified the actual CCD ranging images, then analyzed the errors of the experimental results and corrected the formula of calculating system errors. Experimental results showed that the actual measurement accuracy of a target within 3 km was higher than 0.52%, which meet the accuracy requirements of the high precision binocular ranging.
Binocular CCD ranging is an optical distance measuring method, which imitates human beings’ distance perception using their eyes to obtain the image of the target object. It is a passive ranging in which the high precision ranging over a long distance can be achieved under natural conditions without sending out any signal to the target object. Binocular CCD ranging is characterized by concealment, which is especially suitable for military equipment ranging, aerial target ranging, and occasions needed to be kept secret and restricted by environment in industrial applications.
Image matching, the most core and key technique in binocular CCD ranging, has always been the research hotspot and difficulty for many experts and scholars. Many countries in the world have used weapons with image matching guidance technology, and the earliest one of the weapons is the cruise missile, which has very high concealment. The image matching technology aims to improve the guidance precision of long-range tactical, strategic missile and cruise missile so that the guidance precision is not limited by range distance. Due to the fact that the completion of target matching needs a wide range of image search and that the aircraft inertial navigation system has unstable errors, the matching speed and reliability become the key to matching positioning technology. Therefore, the research for image matching algorithm becomes the key to achieve good performance of guidance. Image matching is proposed by the United States when it was engaged in researches such as the terminal guidance of the projection system and auxiliary aircraft navigation systems in the 1970s and then slowly won the support and funding of the US army. With the rapid development of science and technology, the application of the image matching technology has gradually expanded from purely military purposes to medical image analysis, remote sensing image processing, machine vision, industrial control, and other fields. Therefore, the research for image matching technology has its important theoretical value and application prospect.
The so-called image matching is to find one or more mapping transformation in the changing space of the image and register in the space two or more images of the same target scene, under conditions of different time periods, sensors, and perspectives or recognize its corresponding mode in another target image based on the known target schema. In binocular CCD ranging system, the stereo matching of image is the key link, which determines the main precision of the whole system measurement.
Here is an outline of the paper. In Section 2, the target images are first acquired through the design of binocular CCD ranging system. The target features are extracted and more appropriate regions to the feature region size are chosen to match in Section 3. Section 3 intends to describe how to the subpixel level matching of the target images in conducted by adopting the improved cross-correlation algorithm and three-surface fitting method. To demonstrate the efficiency and reliability of the method, some numerical test examples with comparisons are given in Section 4. Section 5 concludes the paper.
2. The Principle of Binocular CCD Ranging
The principle of binocular CCD ranging is based on the binocular parallax, which is to determine the pixel coordinates position of the same target images on the right and the left images and then calculate their pixel difference, also called binocular parallax, so as to obtain the target distance based on binocular CCD ranging model. Figure 1 is the schematics of binocular CCD ranging. The baseline distance which is a distance that connect the two cameras’ projection center, expressed as [1, 2].
When Target passes by the binocular ranging system formed by cameras on the right and left with two paralleled optical axes, the imaging, respectively, lies on the surface of the left CCD as Point and the right CCD as Point . Its locations on the two focal planes are and . Since the focal lengths of both cameras are , according to the principle of similar triangle, the measured distance can be deduced as  where is the position difference of the images that ’s images on the surface of the left CCD and the right one, also known as binocular parallax. It can be seen that under the ideal state that optical axis of binocular camera is strictly paralleled and the target object images are obtained as well, and through the image matching algorithm, the relevant position of the same target in CCD image is determined and the binocular parallax can be calculated. The focal length and baseline size having been known, the target distance can be concluded according to formula (1).
Binocular CCD ranging system is to use the principle of binocular stereo vision to realize the target image matching by imaging of the same target on different positions through different locations around the camera image on the right and the left. Image matching results directly affect the distance measurement results of binocular CCD ranging system, so as to ensure the accuracy of image matching is the key to the high precision binocular CCD ranging.
3. High Precision Image Matching
3.1. Fast Matching of Pixel Level
This paper adopts left and right CCD camera model, with the parameters maintaining consistency, the optical axis remaining basically paralleled, the binocular camera coordinate systems keeping in coplanar, and the axis being arranged in parallel. Therefore, the size and proportion of image acquired synchronously are in agreement, and the gray information of images can be less affected by noise. Therefore, the image matching algorithm based on template matching gray cross-correlation is adopted, which is most suitable for binocular CCD ranging system. Its principle is shown in Figure 2.
(a) Search graph
(b) Template graph
In Figure 2, graph is , graph is . When matching, template image traverses each pixel in the reference map for search. The search subgraph , with the same size as template graph, has coordinate in the top-left (, ), and the value ranges of and are , respectively. Comparing the similarity of the gray value of each pixel in and , if is in accordance with , then they are phase matching.
This paper uses the normalized cross-correlation matching algorithm to compare the similarity of and . The expression of cross-correlation function is where is the template image and its size is . is the average value of all the gray scales of pixels in . On the reference image, the search area of the image using as the top-left pixels is . is the average value of all the gray scales of pixels in the search image. Image matching is to match the pixel at top-left corner of the template on the target image area. The values range of cross-correlation function is , and its value is determined by the matching degree of search image region and template image using as the pixel at top-left corner. Due to the image difference, it is often impossible to get the pixel whose value of cross-correlation function is 1, while the greater the cross-correlation value of the pixel is, the higher the matching degree of the pixel will be. Therefore, the most matching pixel is the one with the biggest value of cross-correlation function .
In order to achieve the fast matching of images and improve the processing speed of the algorithm, fast image matching method described in  is used. The down sampling of image is made first and then the target image area is roughly matched with the reduced image size. After finding the rough matching point by using cross-correlation function, the matching point is restored to the corresponding position in the original image. The restored pixel location of the point is considered as the center for the accurate pixel matching point, which will be found by continuous cross-correlation operations in its surrounding pixels. Thus, it greatly reduces the amount of data of cross-correlation operation, saves the matching time, and realizes the accurate image matching pixel level.
3.2. Description of the Fast Matching Algorithm
The steps of improved fast matching algorithm are the following.
The first step is to extract the target area of the matching image. When the target image was captured by using binocular CCD camera of binocular ranging system, complete target image matching area “前” appeared on the left and right images. The extraction of target image area is conducted to the target image “前” on the left CCD image and that area is used as a template, preparing for finding out the matching area of the corresponding target image “前” on the CCD image. The actual size of the captured image is about 1300 × 1024 pixels. In order to obtain the complete target image, the template size of the matching target image is 200 × 200 pixels.
As shown in Figure 3, (a) is the image of left CCD camera and (b) is the target area extracted from the left CCD image as the template matching on the right image. The purpose is to find in the right CCD image the upper left corner matching point on the matching template area “前” extracted from the left CCD, the area of target image to be matched image.
The second step is to downsample the right CCD image to be matched and target area template image extracted from the left CCD image. According to the principle of bilinear interpolation [5–7], 4 times downsample processing is taken to the template image and the right CCD image simultaneously. In Figure 4, (a) is the target template image extracted from the left CCD image and its size is 200 × 200 pixels; (c) is 800 × 600 pixels image, captured with the right CCD camera; (b) is the target template image after being downsampled four times which is lessened into a 50 × 50 pixels; (d) is the right image after being downsampled four times and reduced to 200 × 150 pixels.
(a) Target template image of the left one
(b) The 4 times sampling of Figure (a)
(c) The right CCD image
(d) The 4 times sampling of Figure (c)
The third step is to do rough matching operation on the right CCD image and the template image after being downsampled. As shown in Figure 4, the cross-correlation template matching operation is done between the target template image after being downsampled as (b) and the right image (d). Figure 4(b) can be used as the template to do the template search on (d), making cross-correlation operation for each pixel and finding the coordinate of cross-correlation peaks of the correlation function values (, ), which is the rough matching point of the upper left corner of the image coordinates of the target template.
The fourth step is to restore the rough matching point to its original image and select the appropriate area to search the pixel matching point. The rough matching point coordinates are (, ), which is found in the right image after downsampling in the previous step. When it is restored to its original size of the image, the matching point in the picture should be on the area with pixels coordinates (4, ). Then the search of the exact matching point should be made on the appropriate area of the original size CCD image size of pixels which is cut out of the central point (4, ). The area selected can be neither too small nor too big, for it is very easy to exclude the precise matching if it is too small and increase the amount of calculation data if it is too big.
The fifth step is to use the original target template image extracted from the left CCD image to do the cross-correlation matching operation on pixel area extracted from the right CCD image in the previous step and determine its correlation peak coordinates so that the precise matching point coordinates in the right CCD image of the target template image extracted from the left CCD image can be found.
By implementing these methods of fast matching images, it not only reduces the amount of data calculation and avoids the template matching operation of all pixels on the big size image, but also uses the method of performing the rough matching firstly and then locating precisely to find the relevant peak coordinates of the relevant cross-correlation function distribution and completing the pixel-level matching image.
3.3. The Subpixel Level Image Matching of CCD Ranging Image
The accuracy of image matching is the key to achieving the precision distance measuring of the binocular CCD ranging system . Therefore, the accuracy of the image pixel level cannot meet the requirements of the binocular CCD ranging system. In this sense, this paper aims to realize the subpixel level matching of the target image.
In this paper, three-surface fitting method  is adopted to fit the subpixel level coordinates of the correlation peaks. The cross-correlation peak pixel coordinates of the distribution of the correlation function is taken as a central point; the pixel points around the area of pixels are selected to fit the fitting points three times; then they are taken into the cross-correlation function with the binary representation of a cubic polynomial. The formula is just as follows:
Horizontal and vertical coordinates of about pixels around the correlation peak and its surrounding can be brought to the binary cubic polynomial, using the least squares method  to obtain the expression of the coefficient and get the complete expression of binary cubic fitting function. The following diagram in Figure 5 is the three-surface fitting. Using the expression of the binary cubic polynomial, the maximum value of the cubic surface fitting function can be figured out, which is the fitting matching subpixel coordinates and the target image subpixel level matching is completed.
4. Experimental Results and Analysis of Image Matching
4.1. Experimental Results of Binocular CCD Ranging Experiment
The experiment adopts the CCD camera of OK-AC1300 model with a resolution of pixels, two pixels size of 4.65 μm, and working output 8 bitRGB image and chooses the German deterrence Walimex500 mm F8-32 telephoto fixed focal length lens with = 500 mm. In the experiment, different targets on the distance of about 3 km plurality are measured. Ranging image groups are as follows.
The chosen target is “Energy Bureau of Jilin Province.” The images and the selected target in the experiment are shown as in Figure 6.
(a) The left CCD image
(b) The right CCD image
(c) The target image
Figure 6 is a group of target images in the CCD ranging. The experiments use GPS navigation and positioning to measure the target shooting location and its latitude and longitude, a distance of 2.2631 km. To test the stability of the CCD ranging system and the running of the algorithm, several groups of target images are captured and matched during the experiment. The software interface of image matching is shown as in Figure 7.
Combining the calibration parameters of the far-distant CCD binocular ranging system, the measurement results of experiments are obtained, which are shown in Table 1.
4.2. The Error Analysis of Binocular CCD Ranging Results
In this paper, the formula of the binocular CCD ranging is modified and the two-dimensional case is discussed, pointing out that the actual three-dimensional imaging calculation is just the expanse of the two-dimensional calculation, which does not affect the calculation of the formula. Binocular vision system imaging model [11, 12] is shown as in Figure 8. and are the optical centers of the left camera and the right camera, respectively, and are the axes of the two cameras, respectively, and and are the focal lengths of the two cameras, respectively.
In Figure 8, is setas a calibration point. It is assumed that the coordinates of this calibration point in the camera coordinate system are (x1L, z1L) and (x1R, z1R). According to a first-order radial distortion pinhole model, the ideal position of on the image plane is , , but its actual location is , . It is assumed that the two-dimensional position errors of each pixel are equal. They are both , which is as follows:
According to the position relationship between and the point position of the two-dimensional image , , the position of the optical center of the camera and can be obtained with the linear method to obtain.
The coordinates of the and in the left and right cameras coordinate system which are obtained from the straight and the intersection of the image plane are as follows:
It can infer
Based on the line , the coordinate of in the coordinate system of the left camera is . Similarly, the coordinate of in the right camera coordinate system is . Then the testing points were restored by using the obtained camera parameters. Assuming the actual coordinates of the test points for as () and (), in theory testing image point should be , . Defined and , respectively, are
Due to the error effects caused by the matching, the corresponding image points of , respectively, are detected as , and the matching error is ; thus Therefore, the baseline length of , is And the parallax is calculated as
According to the CCD camera diagram, the depth value of can be obtained as follows:
The system error is calculated as
In this paper, when calculating the matching error, the system error is 0.05 pixel, and the calibration error is 0.5826 pixel. Therefore, this system error of the paper is about 0.52%.
In this paper, the proposed image matching algorithm is adopted to improve the accuracy of the binocular CCD passive ranging. During the measurement, the improved cross-correlation algorithm and three-surface fitting algorithm to the target image are applied to achieve the subpixel level image matching, improving the accuracy of ranging. Through the error analysis of measurement results, the paper modified the calculation formula of system errors. Experimental results show that the target measuring accuracy within 3 km is higher than 0.52%, which is of great practical value.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
This project is supported by the scientific and technological research project of Jilin province Department of Education, China (no. 201363, no. 2013145), and basis study project supported by Jilin Provincial Science & Technology Department, China (no. 201105055).
- J. M. Hakkarainen and H. Lee, “A CCD/CMOS absolute-value-of-difference processor for use in a stereo vision system,” IEEE Journal of Solid-State Circuits, vol. 28, no. 7, pp. 799–807, 1993.
- S. Vallerand, M. Kanbara, and N. Yokoya, “Binocular vision-based augmented reality system with an increased registration depth using dynamic correction of feature positions,” in Proceedings of the IEEE on Virtual Reality, pp. 271–272, 2003.
- F.-Q. Zhou, J.-G. Zhu, X.-Y. Yang, and S.-H. Ye, “A high speed CCD camera calibration technique,” Optics and Precision Engineering, vol. 8, pp. 96–100, 2000.
- S. Lin, Y. Jinhua, Z. Qun, and L. Peiping, “Research on measuring telephoto lens focal length by using image matching,” Journal of Electronic Measurement and Instrument, vol. 24, pp. 1043–1047, 2010.
- X. Jie, “Analyzing and improving the tsai camera calibration method in machined vision,” Computer Engineering & Science, vol. 32, pp. 46–47, 2010.
- L. Yong, Q. B. Feng, and L. Y. Qi, “Sub-pixel surface fitting algorithm in digital speckle correlation method,” in Proceedings of the 9th International Conference on Electronic Measurement & Instruments, vol. 4, pp. 959–962, 2009.
- D. G. Lowe, “Object recognition from local scale-invariant features,” in Proceedings of the 7th IEEE International Conference on Computer Vision (ICCV '99), vol. 9, pp. 1150–1157, Corfu, Greece, September 1999.
- D. G. Lowe, “Distinctive image features from scale-invariant keypoints,” International Journal of Computer Vision, vol. 60, no. 2, pp. 91–110, 2004.
- S. Lin, High Precision Ranging Based on Binocular CCD, Changchun University of Science and Technology, Changchun, China, 2011.
- F. Zou, X. Su, and M. Li, “Camera linear calibration method with a coplanar target,” Guangdian Gongcheng/Opto-Electronic Engineering, vol. 32, no. 4, pp. 70–74, 2005.
- Z. Zhigang, Study on High-Precision Camera Calibration and Robust Stereo Matching, University of Science and Technology of China, Beijing, China, 2008.
- P.-C. Hung and A. S. Voloshin, “In-plane strain measurement by digital image correlation,” Journal of the Brazilian Society of Mechanical Sciences and Engineering, vol. 25, no. 3, pp. 215–221, 2003.
Copyright © 2014 Dongming Li et al. This is an open access article distributed under the Creative Commons Attribution License, which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.