Abstract
Multisolution phenomenon is an important issue in P3P problem since, for many real applications, the question of how many solutions could possibly exist for a given P3P problem must at first be addressed before any real implementation. In this work we show that, given 3 control points, if the camera’s optical center is close to one of the 3 toroids generated by rotating the circumcircle of the control point triangle around each one of its 3 sides, there is always an additional solution with its corresponding optical center lying in a small neighborhood of one of the control points, in addition to the original solution. In other words, there always exist at least two solutions for the P3P problem in such cases. Since, for all such additional solutions, their corresponding optical centers must lie in a small neighborhood of control points, the 3 control points constitute the singular points of the P3P solutions. The above result could act as some theoretical guide for P3P practitioners besides its academic value.
1. Introduction
The Perspective-3-Point Problem, or P3P problem, is a single-view based pose estimation method. It was first introduced by Grunert [1] in 1841 and popularized in computer vision community a century later by mainly Fischler and Bolles’ work in 1981 [2]. Since it is the least number of points to have a finite number of solutions and no feature-matching across views is needed, it has been widely used in various fields [3–11], either for its minimal demand in restricted working environment, such as robotics and aeronautics, or for its computational efficiency acting as a minimum-set based solver repeatedly called in robust-statistics based iterative estimation framework, such as the well-known RANSAC-like framework, where the computational time increases exponentially with the cardinality of the minimum set; hence, the P3P problem is preferred due to its minimum requirement.
The P3P problem is defined as follows: Given the perspective projections of three control points with known coordinates in the world system and a calibrated camera, find the position and orientation of the camera in the world system. It is shown that the P3P problem could have 1, 2, 3, or at most 4 solutions depending on the configuration between the camera optical center and its 3 control points [12]. If the optical center and 3 control points happen to be concyclic, the problem becomes degenerated, and an infinitely large number of solutions could be possible.
Since, in many real applications, some basic questions must be answered before its real implementation (such as the following: Does it have a unique solution? If not, how many solutions could it have? Is the solution stable?), the multiple solution phenomenon in the P3P problem has been a focus of investigation since its very inception in the literature. Traditionally the multisolution phenomenon in P3P problem is analyzed by at first transforming its 3 quadratic constraints into a quartic equation and then roots of this quartic equation are located to derive possible solutions. For example, Haralick et al. summarized 6 different transformation methods [12]. Gao et al. [13] gave a complete solution classification. Gao and Tang [14] also gave an analysis on the solutions distribution from the probabilistic point of view. Recently, Rieck [15–17] gave a systematic analysis on the multisolution phenomenon, in particular, on the cases where the optical center of the P3P problem is close to the danger cylinder. From the geometric point of view, Zhang and Hu [18] showed that when the optical center lies on the danger cylinder, the corresponding P3P problem must have 3 distinct solutions, and one must be a double solution. Lowe [3] and Zhang and Hu [19] showed that when the optical center lies on the three vertical planes perpendicular to the control point plane and going through one of the 3 altitudes of the control point triangle, the P3P problem must have a pair of side-sharing solutions and a pair of point-sharing solutions. (As shown in [19], a pair of side-sharing solutions refers to two such solutions that if their optical centers are superposed, they must share two control points or share a common side of the control point triangle. Similarly, a pair of point-sharing solutions refers to two such solutions that if their optical centers are superposed, they must share a single common control point.) Sun and Wang [20] gave an interpretation of the solution changes at the intersecting lines of the above three vertical planes with the danger cylinder. Wu and Hu [21] gave a thorough investigation on the degenerate cases in their “The PnP Problem Revisited” work. Recently, Wang et al. [22] show that the side-sharing pair of solutions is usually accompanied by a point-sharing pair of solutions, or the two kinds of pairs are often companion pairs. They also show [23] that if the optical center is outside of all the 6 toroids, each positive root of Grunert’s quartic equation must correspond to a positive solution of the P3P problem. Their work shows that the root-solution-relationship is of one-to-one correspondence when the optical center is outside of all the 6 toroids but much more complicated when the optical center is inside of them.
In this work, we investigate the multisolution phenomenon in P3P problem for those cases where the camera optical center is very close to one of the 3 toroids. More specifically, based on Grunert’s derivation [12], we show that when the optical center of a given P3P problem is close to any one of the 3 toroids generated by rotating the circumcircle of the control point triangle around each of its 3 sides, in addition to its original solution, there is always an additional solution whose optical center always lies in a small neighborhood of control points. In other words, such a P3P problem always has at least two solutions. In addition, since, for all such additional solutions, their optical centers must be in a close neighborhood of control points, the 3 control points are singular points of the solutions’ optical centers distribution.
2. P3P Problem, 3 Toroids, and Grunert’s Quartic Equation
2.1. P3P Problem
As shown in Figure 1, , , and are the three control points with known distance: , , and ; is the camera optical center. Since the camera (under the pinhole model) is calibrated, the three subtended angles , , and of the projection rays can be considered as known entities and then, by the Law of Cosines, the following 3 basic constraints on the three unknowns , , and in (1) must hold: Hence, the P3P problem means to determine such positive triplets satisfying all the 3 basic constraints in (1). And the multisolution phenomenon refers to the existence of multiple such positive triplets all satisfying the 3 constraints in (1).
2.2. The 3 Toroids
As shown in Figure 1, by rotating the circumcircle of the triangle around one of its three sides by separately, 3 pairs of toroids (or 6 toroids) can be generated [24] as shown in Figure 2. For each toroid pair, one is generated by rotating the arc with the subtended angle (or , or ) and the other with its supplement angle () (or (), or ()). In this work, the 3 toroids refer to the 3 ones generated only with the subtended angle , , and , denoted by Toroid_, Toroid_, and Toroid_, respectively. Hence, when the optical center lies on Toroid_ (or Toroid_, or Toroid_), the subtended angle (or , or ) of the two projection rays () (or (), or ()), by definition, always has (or , or ).
(a)
(b)
2.3. Grunert’s Derivation
As shown in [12], by setting and , (1) can be transformed into a quartic equation in aswhere can be determined from the following quadratic equation:From (4), it can be seen that, given a positive root in (2), we can always obtain a positive . From (5), if a positive can be obtained, then the positive triplet (, , ) must be a positive solution of the P3P problem. Note that, given a positive root in (2), there could be positive roots of in (5). In other words, each positive could correspond to positive solutions of the P3P problem. In [23], the authors showed that if the optical center is outside of all the 6 toroids, each positive root must correspond to a unique positive solution of the P3P problem. In the next section, we show that when the optical center is close to one of the 3 toroids in Section 2.2, the corresponding P3P problem must have at least two different positive solutions, the one whose optical center is at the original position and the other whose optical center is in a small neighborhood around one of the 3 control points. Here “two different solutions” refer to “the two solutions with the same 3 control points, but their optical centers lie at a different position.”
3. Solution Singularity in P3P Problem
We have the following main result.
Proposition 1. As shown in Figures 1 and 2, if the optical center is close to Toroid_, then except for some special curves, there is always another optical center in a small neighborhood of control point such that is another positive solution of the P3P problem in addition to its original solution .
Proof. If the optical center lies on the Toroid_, . Hence, , and is a root of the quartic equation in (2).
Suppose the optical center is not on the Toroid_, but very close to it; then the three subtended angles from to the three control points , , and are as follows: , , , where is quite small since is very close to Toroid_. In fact, could be infinitesimally approaching 0 if the optical center is infinitesimally close to Toroid_.
Since is very small, all the 5 coefficients , can be approximated by its linear term at , , . In other words, we can keep, unchanged, but only is subject to a small change by at .
When , is a root of the quartic equation in (2). When , there should be a root of the linearized equation of (2) at , , . Since (2) is a quartic equation, we need only to consider the last two terms: that is,If when , and since when , then if , we haveLet us consider the condition of and that of when .
When , if , we haveSince , substituting it into the above equation and by some manipulations, the above equation can be simplified as means , or the control point triangle is a right triangle. In this case, the Toroid_ degenerates into a sphere.
If , indicates that the 3 subtended angles (, , of the optical center with the three control points , , and must satisfy the following relation: Since can be expressed asthen if ,
In conclusion, if , and if , always exists, and we can always make by choosing either or . Geometrically, means the optical center is inside of Toroid_, and means the optical center is outside of Toroid_.
Since when , from (5), . From , , a positive triplet can be obtained, which must be another solution of the P3P problem. Since is small, . Hence, when , , it indicates that the corresponding optical center of this additional solution must be close to the control point . In other words, is in a close neighborhood of control point . Hence, the proposition is proved.
Remark 2. When the optical center is close to Toroid_ (or Toroid_), by setting the variables , (or , ), we can similarly prove that there exists additionally a positive solution whose corresponding optical center lies in a close neighborhood of control point (or ).
Remark 3. When is a right angle, say , Toroid_ becomes a sphere, and this is quite a special case. As shown by Zhang and Hu [18] and Sun and Wang [20], the pair of side-sharing solutions degenerates into a double solution in this case.
Remark 4. In fact, “the closeness of neighborhood” in the above discussions can be explicitly quantified as , or a linear relationship between (the neighborhood size) and (the closeness magnitude of the optical center to the toroid).
Remark 5. From the above discussion, we can see that if the optical center is close to one of the 3 toroids (inside or outside of the toroid), except for some special curves, there is always a P3P positive solution whose optical center is in a close neighborhood of one of the three control points. This indicates that the 3 control points are singular points of the optical centers distribution of the P3P solutions.
4. Computer Simulation
Although the correctness of our proposition lies in its proof, we nevertheless also verify it by computer simulations. Our simulation procedure is as follows.
Generate 3 noncollinear control points , , and .
Repeat (1)–(4).(1)Choose at random an optical center close to one of the 3 toroids.(2)From , compute the three quadratic constraints in (1) and then transform them into a quartic equation in (2).(3)Compute the roots of (2) to determine a small root .(4)Compute the P3P solution corresponding to as shown in Section 3.
We generate about 10,000 triplets of control points, and for each triplet, about 10,000 different optical centers are simulated. All the simulations confirm our general theoretical conclusion; that is, the optical centers of all the additional P3P solutions are in a close neighborhood of control points. Figure 3 is an example of the resulting optical center distributions around the control point (for the visualization clarity purpose, only 100 points are shown). In Figure 3(a), the blue points are the optical centers near the Toroid_, the green point is the control point , the pink point is the control point , and the red points are the optical centers of those additional corresponding P3P solutions, which are all near the control point . Figure 3(b) is the enlarged part near the control point , and it shows that these optical centers are in a small neighborhood of . Each blue point in Figure 3(a) has a corresponding red point in Figure 3(b). In other words, a blue point and the 3 control points constitute a P3P problem whose optical center (the blue point) is close to Toroid_, and this P3P problem has another solution (the corresponding red point and the 3 control points) whose optical center (the red point) is close to the control point .
(a)
(b)
5. Conclusion
In this work, we find that, given 3 control points, if the camera optical center is close to any one of the 3 toroids, except for some special curves, there is always an additional positive solution for this P3P problem whose corresponding optical center always lies in a close neighborhood of one of the 3 control points. Hence, the 3 control points are singular points of the optical centers distribution of the P3P solutions. In other words, some singularity exists around the 3 control points for the P3P problem solutions.
Our result provides some new insights into the nature of multisolution phenomenon; in addition to its academic value, the result could also provide some theoretical guidance for P3P practitioners to properly arrange their control points to avoid unstable solutions for the P3P based applications.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgment
This work was supported by National Natural Science Foundation of China (NSFC) under Grant nos. 61402316 and 61403373.