Abstract
A novel method for camera calibration is proposed based on an analysis of lens distortion in camera imaging. In the method, a line through the centre of concentric circles is used as a template in which orthogonal directions can be determined from an angle of circumference that corresponds to a diameter. By using three lines through the centre of concentric circles, based on the invariance of the cross-ratio, an image at the centre of the concentric circles can be used to obtain the vanishing point. The intrinsic parameters of the camera can be computed based on the constraints of the orthogonal vanishing points and the imaged absolute conic. The lens distortion causes points in the template to have a position offset. In the proposed method, we optimize the positions of the distortion points such that they gradually approach those of the ideal points. The simulated and real-world experiments demonstrate that the proposed method is efficient and feasible.
1. Introduction
Camera calibration is an important research topic in the field of pattern recognition because it is required for computer vision applications [1–3]. Meng and Hu [4] used a circle and several lines through the centre of the circle as a calibration template; however, a single circle contains little information. Wu et al. [5] proposed a method of camera calibration involving the affine invariance of parallel circles. If the intersection of two parallel circles is first computed to determine the circular points, then the intrinsic parameters can be determined. However, this method cannot be used to determine the centre of concentric circles and requires at least three images. And Bin [6] proposed a method to calculate the vanishing point by the theory of harmonic conjugate in projective geometry. The camera intrinsic parameters could be obtained by the relationship between the circular points and the image of absolute conic. In addition, lens distortion degrades the accuracy of the camera calibration [7]. Consequently, Ricolfe-Viala and Sánchez-Salmerón [8] proposed a nonlinear method that corrects the images based on the cross-ratio invariance, although this algorithm is more complex. To address the disadvantages in the above methods, we propose a method to compute the intrinsic parameters by employing a circle as a template, while the scale of the circle does not need to be known. Based on the property that an angle in a circular segment that corresponds to the diameter is 90°, if an image includes two pairs of orthogonal vanishing points, the intrinsic parameters can be calculated for the three images. This method reduces the complexity of camera calibration. We also propose a new method for correcting lens distortion, which corrects images using the least square method to fit a line that passes through the centre of concentric circles.
This paper is organized as follows. The underlying theory is introduced in Section 2. The camera calibration method is proposed in Section 3, and a method of determining the image of the circle centre is described using the concentric circles. The proposed method of correcting lens distortion is introduced in Section 4. In Section 5, the results of simulation experiments are presented to show whether the method described in Section 4 is valid. Then, an experiment that compares this method with other classic methods is conducted. Finally, Section 6 provides a summary of this paper.
2. Preliminaries
Let denote the homogeneous coordinates of a 3D point and denote the homogeneous coordinates of the corresponding image point. The projection relationship between these points iswhere is a nonzero scale factor and is a 3 × 4 matrix that is defined as the projection matrix, which can be expressed aswhere is a 3D rotation, is a translation vector, and is the intrinsic parameters matrix [1].
3. Use of the Orthogonal Vanishing Points to Solve K
3.1. Computing the Image of the Centre of Concentric Circles
The circle intersects the line at two points , and circle intersects the line at two points , as shown in Figure 1. It can be shown that the centre of the circles is at the mid-point of lines . Let represent the points in the direction of infinity along line , and denote line as a calibration line.
Proposition 1. In Figure 1, the corresponding points of , which are , , and in the image plane, are , , , and , respectively. Thus, the equations describing the image of the circle centre are
Proof. The circle centre is the mid-point of line segments , . Therefore, in projective geometry, the four points and the four points are harmonic conjugates, respectively. ThusBased on the invariance of the cross-ratio [9],
Let the coordinates of , and be , , and , respectively. Therefore, based on Proposition 1, the equation for solving the image of the centre of the concentric circles can be written as
3.2. Computing the Intrinsic Parameter Matrices
Proposition 2. In Figure 2, if the image of the circle centre is known, two pairs of vanishing points in orthogonal directions can be determined by three lines passing through the centre of the circle.
Proof. In Figure 2, from Proposition 1, the image of the centre of the circle and the image of the three lines passing through the centre of the circle can be obtained. Thus,where are the first set of orthogonal vanishing points and are the second set. In this way, two pairs of vanishing points in orthogonal directions can be obtained.
The image of absolute conic is , which can be represented by a symmetric matrix:In Hartley and Zisserman [10], the constraint equations between the image of absolute conic and the vanishing points have been established. Thus,If and , then (9) can be expressed by Because has six unknown parameters, it is necessary to take three pictures from different orientations. By computing the singular value decomposition (SVD) of the coefficient matrix, can be obtained. Then, can be determined by solving the inverse after the Cholesky decomposition of .
4. Correcting the Lens Distortion
4.1. Distortion Correction
In a pinhole camera, because of the lens distortion, a point between the template and its image has a positional offset. Therefore, we need to correct the distortion point in its original position. In Figure 1, we correct the image point on the line according to Proposition 3 and use the distortion model to calculate the radial distortion coefficients .
Proposition 3. The exercise of correcting the points in the image and calculating the radial distortion coefficients can be transformed into a calculation of the minimum of the objective function: , where , , and .
Proof. points from lines can be extracted in the image of the template, which include the intersection points between the circles and lines shown in Figure 1. The points can be numbered . The points for are in the lines for , which satisfywhere are the parameters of the lines for .
To ensure the points extracted from the image satisfy the line equation, the objective function (12) combined with the least-squares principle is used to solve for the minimum value of where is the number of lines in the image and is the number of points in each line.
In Wang et al. [7], the lens distortion of a camera was modelled as a combination of the radial and tangential distortions, and the relationship between the distortion point and the corresponding ideal point can be described by establishing the inverse model of distortion correction, as in where , and .
Assuming that and are the coordinates of the kth distortion point and the corresponding ideal point, respectively, in terms of (13), the objective functions are as follows:In order to correct the error from the distortion point to its ideal point, it is essential to minimize the global error. According to the above analysis, by combining (12) and (14), we obtain the final objective function:
In this way, the correction problem is converted into the calculation of the minimum of the final objective function . The minimization process can be accomplished using the Levenberg-Marquardt (LM) algorithm. The initial parameters of the line can be obtained using the distortion point, which is extracted from the image. During the iteration process, if the distortion point gradually approaches its ideal point, the iteration will complete and the linear equation will be determined. In this way, the minimum of the final objective function and the radial distortion coefficients can be obtained. At the same time, the lines and points in the image of the template can be corrected.
4.2. Steps in the Algorithm
Step 1. Use a camera to capture three images from different orientations.
Step 2. Extract the points on the image binary [11].
Step 3. After correcting the distortion points by applying Proposition 3, based on Proposition 1, the image of the centre of the concentric circle can be obtained using (6).
Step 4. Solve for the vanishing points using (7).
Step 5. Solve the resulting (10) using the SVD method for . Determine by solving the inverse after the Cholesky factorization of .
5. Experiments
In order to prove the effectiveness of the method and test the sensitivity of the method to noise, we performed both simulated and actual experiments. In this section, there are five real experiments as follows: Meng’s method [4], which uses one circle and a line that passes through the centre of the circle; Wu’s method [5], which uses parallel circles; our method; our method, which is applied after correcting the image using Ricolfe-Viala’s method [8] and calibrating the camera using our method; our method+, which is applied after correcting the images using the method proposed in this study and calibrating the camera using our method.
5.1. Simulation Experiment
Note that the lens distortion was not considered in the simulations. The intrinsic parameters of the camera were assumed to be . The goal of this simulation is to determine the effects of noise on the intrinsic parameters of the camera. We applied three methods: our method, Meng’s method, and Wu’s method. Each point includes Gaussian noise in which the variance ranges from 0 to 1.5. For each value of , twenty independent experiments were performed, and the average values of the camera intrinsic parameters were obtained. The results of the simulations of the three methods were compared and analyzed, and the results are shown in Figure 3. As shown in the figure, our method was stable.
(a)
(b)
(c)
(d)
5.2. Real Experiment
In the real experiment, the resolution of the camera was 1280 × 960 pixels. The images of the template containing two concentric circles and three lines passing through their centre are shown in Figures 4(a)–4(c). The corrected images are shown in Figures 4(d)–4(f).
(a)
(b)
(c)
(d)
(e)
(f)
After the original image was corrected, the image of the template was subjected to grey treatment. In Figure 5(a), the points extracted from the binary image are shown. Similarly, we selected two additional images with different orientations when extracting the points in Figures 5(b)-5(c). We then solved for the intrinsic parameters using the following methods: Meng’s method, Wu’s method, our method+, and our method. The results of the experiments are shown in Table 1.
(a)
(b)
(c)
To verify the validity of the intrinsic parameters of the camera in Table 1, the data in Table 1 were used to reconstruct the 3D information [12] of the checkerboard in Figure 6. Forty-five feature points corresponding to the five rows and nine columns of the checkerboard were selected from each image. The reconstruction results are shown in Figure 7. Then, the angle average value in parallel directions was calculated with the data from Figure 7. Similarly, the angle average value in orthogonal directions was obtained. Table 2 lists the angle results with real data as shown in Figure 7.
(a)
(b)
(a)
(b)
(c)
(d)
(e)
The real angles are 0° for the parallel lines and 90° in the orthogonal direction on the checkerboard. From Table 2, we can compare the absolute error of the experimental results with the real angle. It is clear that the absolute error of our method+ was smaller. Thus, our method+ was shown to be feasible and effective.
6. Conclusions
In this study, based on the theory of harmonic conjugates in projective geometry in combination with the vanishing point and the centre of concentric circles, a calibration method that uses circles and line is proposed. The image of the centre of the circle can be computed easily using the concentric circles, without requiring knowledge of the radius and the location of the centre. Three images can be taken from different orientations of the template, and the intrinsic parameters can be computed using the linear method. An advantage of the proposed process is that it does not require complicated calculations.
Conflicts of Interest
The authors declare that there are no conflicts of interest regarding the publication of this paper.
Acknowledgments
This work was supported in part by two grants from the National Natural Science Foundation of China (no. 61663048 and no. 11361074).