Differential and Statistical Approach to Partial Model Matching

Abstract

Partial model matching approaches are important to target recognition. In this paper, aiming at a 3D model, a novel solution utilizing Gaussian curvature and mean curvature to represent the inherent structure of a spatial shape is proposed. Firstly, a Point-Pair Set is constructed by means of filtrating points with a similar inherent characteristic in the partial surface. Secondly, a Triangle-Pair Set is demonstrated after locating the spatial model by asymmetry triangle skeleton. Finally, after searching similar triangles in a Point-Pair Set, optimal transformation is obtained by computing the scoring function in a Triangle-Pair Set, and optimal matching is determined. Experiments show that this algorithm is suitable for partial model matching. Encouraging matching efficiency, speed, and running time complexity to irregular models are indicated in the study.

1. Introduction

Partial model matching has important applications in many cases such as the vision registration for automatic 3D models [1–3], content-based image retrieval from image database [4], assembly and restoration for fractured object [5], docking of proteins in molecular biology [6], and other applications. During the past decade, the problem of finding a partial match between models attracted considerable attention. In many cases, models may be only partially visible because the occlusion or the sensor usually cannot scan all the data, so the goal of partial model matching is to find a transformation of the sample model to overlap a large portion of the object model.

This paper focuses on the research of a 3D partial model matching. In 3D space, the models can be represented as 3D surfaces. Therefore, finding a Euclidean geometry rigid transformation of the sample surface to overlap a large portion of the object surface is the goal of 3D partial model matching. In recent years, there has been lots of research on 3D partial surface matching. The literature [7, 8] proposed a scoring schema for 3D partial surface matching, but this paper did not demonstrate a universal and high-efficiency algorithm for the constructing of a scoring factor. The literature [9] proposed an approach based on the Hausdorff and Frehet distance for geometry structure matching. This approach can be well applied in the matching of points and line segments set, but it is unstable and cannot perform well in the presence of noise, occlusion, and clutter. The literature [10] proposed a scoring schema for matching algorithm to a 3D incomplete object. The literature [11–13] proposed some matching algorithm using differential geometry to 3D partial surface matching. In addition, statistical descriptors [14], probabilistic framework [15], and curve analysis [16] were also applied to the recognition of 3D partial surface; these approaches indicated encouraging matching results for 3D points, line segments set, and mesh model; however, they would suffer from high computing complexity. It is worth mentioning that partial model matching has been successfully applied in video processing [17, 18].

Our work is motivated by the technique of scoring schema and statistical approaches. This technique, which uses the so-called scoring-function method, was originally introduced for 3D partial surface and volumetric matching. This idea has been applied in some pattern recognition problem [19] and achieved good results. The work proposed in this paper has been partly published in an international conference [20]. Here is a brief sketch of our algorithm.(1)Represent the inherent characteristic of points in sample and object surface utilizing differential geometry method. (2)Classify point pair with a similar inherent characteristic form sample and object surface. (3)Classify triangle pair with a similar shape feature and inherent characteristic based on point pairs and compute the geometry rigid transformation for every triangle pair. (4)Use scoring function to measure the goodness of every transformation through applying it to other triangle pairs. (5)Determine the optimal transformation with score of every transformation.

Experimental results show that our technique is most successful for the matching of irregular object surface. Encouraging matching efficiency, speed, and running time complexity to irregular models are indicated.

2. Compute and Store the Curvature

2.1. Represent the Spatial Surface Using Gaussian and Mean Curvature

According to differential geometry principle, curvature is the inherent characteristic of a spatial surface. Therefore, Gaussian curvature and mean curvature are employed for the representation of spatial surface in this paper.

In 3D Euclidean space, given a parametric surface defined as 𝑆𝑥,𝑦=𝑥𝑦𝑓𝑥,𝑦^𝑇,𝑥,𝑦∈𝐷, (1) where 𝑋-𝑌 is the reference plane in 3D space, 𝐷 is projection region of the surface to 𝑋-𝑌 plane, and 𝑓(𝑥,𝑦) represents the distance from the surface to point (𝑥,𝑦) in 𝑋-𝑌 plane.

Gaussian curvature 𝐾 and mean curvature 𝐻 can be computed according to the following formulas [18]: 𝑓𝐾=^𝑥𝑥𝑓_𝑦𝑦−𝑓²_𝑥𝑦1+𝑓²_𝑥+𝑓²_𝑦²,𝐻=1+𝑓²_𝑥𝑓_𝑦𝑦+1+𝑓²_𝑦𝑓_𝑥𝑥−2𝑓_𝑥𝑓_𝑦𝑓_𝑥𝑦21+𝑓²_𝑥+𝑓²_𝑦^3/2. (2)

For a digital range image surface, approximations can be computed by a local polynomial fitting approach, and 𝑛×𝑛 operator is usually utilized to the convolution operation with the original range image: 𝑓_𝑥=𝐷_𝑥∗𝑓,𝑓_𝑦=𝐷_𝑦𝑓∗𝑓_𝑥𝑥=𝐷_𝑥𝑥∗𝑓,𝑓_𝑥𝑦=𝐷_𝑥𝑦∗𝑓,𝑓_𝑦𝑦=𝐷_𝑦𝑦∗𝑓, (3) where 𝐷 is 𝑛×𝑛 operator. For 𝑛=7, the parameters can be computed as follows: 𝐷_𝑥=𝑑₀𝑑^𝑇₁,𝐷_𝑦=𝑑₁𝑑^𝑇₀,𝐷_𝑥𝑥=𝑑₀𝑑^𝑇₂,𝐷_𝑦𝑦=𝑑₂𝑑^𝑇₀,𝐷_𝑥𝑦=𝑑₁𝑑^𝑇₁𝑑₀=171111111^𝑇𝑑₁=128−3−2−10123^𝑇𝑑₂=18450−3−4−305^𝑇, (4) where 𝑑₀, 𝑑₁, and 𝑑₂ are column vectors for window operator computing.

2.2. Generate the Point-Pair Set

After presenting the acquisition of Gaussian curvature 𝐾 and mean curvature 𝐻, in our algorithm, point pairs with similar curvature from sample and object surface will be classified, and then the point pairs will be stored into a Point-Pair Set. In order to measure the similarity between two points from sample and object surface, the definition of curvature distance is introduced as follows:

Definition 1. ∀𝑝¹_𝑖∈𝑆₁,𝑝²_𝑗∈𝑆₂, curvature distance DC between point 𝑝¹_𝑖,𝑝²_𝑗 is DC𝑝¹_𝑖,𝑝²_𝑗=12×𝐾𝑝¹_𝑖−𝐾𝑝²_𝑗𝐾𝑝¹_𝑖+𝐾𝑝²_𝑗+𝐻𝑝¹_𝑖−𝐻𝑝²_𝑗𝐻𝑝¹_𝑖+𝐻𝑝²_𝑗. (5)
However, in many cases, a lot of plane points, whose Gaussian curvature and mean curvature are zero, can exist in the surface. They cannot contribute to the matching but will occupy large computing and reduce the matching efficiency, even lead to an error matching. So that these plane points will be discarded before matching.

Definition 2. A Point-Pair Set named PS is a set defined as PS=𝑝¹_𝑖,𝑝²_𝑗∣𝑝¹_𝑖∈𝑆₁,𝑝²_𝑗∈𝑆₂,DC𝑝¹_𝑖,𝑝²_𝑗𝜀𝑐,𝐾𝑝¹_𝑖𝜎𝑝𝜀𝑐=maxDC¹_𝑖,𝑝²_𝑗+min𝐷𝐶𝑝¹_𝑖,𝑝²_𝑗21𝜎=𝑛^𝑛_𝑖=1𝐾𝑝¹_𝑖. (6) Where 𝜀𝑐 is an average threshold to guarantee the similarity of 𝑝¹_𝑖 and 𝑝²_𝑗, 𝜎 is to discard plane points, and 𝑁 is the number of the points.
The cardinality of PS could be large in special cases. For instance, cardinality of PS will be |𝑆₁|×|𝑆₂| when the sample and object surface are congruent sphere surface. A large Point-Pair Set will occupy a large amount of the storage space. Considering the quantity of curvature categories cannot exceed the quantity of pixel points in surface, a list can be employed to store the set.

3. Match Algorithm

3.1. Measure Similarity for Spatial Triangles

In order to perform the matching, the further work is to separately seek two asymmetrical triangles with similar shape features from the sample and object surface, and then we will design a set named “Triangle-Pair Set” to store these triangle pairs. A plane can be determined by three spatial points, so the asymmetrical triangle can be employed as the skeleton to uniquely confirm the location of a spatial surface. Therefore, we arbitrarily select three points from the object surface to form an asymmetrical triangle and then seek three corresponding points from sample surface to form another asymmetrical triangle, whose features of corresponding vertices and edges are similar to the previous triangle. The similarity of space triangles is measured by the definition as follows.

Definition 3. Consider ∀(𝑝¹₁,𝑝²₁),(𝑝¹₁,𝑝²₁),(𝑝¹₁,𝑝²₁)∈PS, |𝑝^𝑚₁𝑝^𝑚₂|<|𝑝^𝑚₁𝑝^𝑚₃|<|𝑝^𝑚₂𝑝^𝑚₃|, 𝑚∈{1,2}. The similar distance between Δ𝑝¹₁𝑝¹₂𝑝¹₃ and Δ𝑝²₁𝑝²₂𝑝²₃ is 1DT=3×_{1≤𝑖<𝑗≤3}𝑝¹_𝑖𝑝¹_𝑗−𝑝²_𝑖𝑝²_𝑗𝑝¹_𝑖𝑝¹_𝑗+𝑝²_𝑖𝑝²_𝑗. (7)
The absolute distance between Δ𝑝¹₁𝑝¹₂𝑝¹₃ and Δ𝑝²₁𝑝²₂𝑝²₃ is ∑DA=2×^3𝑖=1𝑝¹_𝑖𝑝²_𝑖∑³_{1≤𝑖<𝑗≤3}𝑝¹_𝑖𝑝¹_𝑗+𝑝²_𝑖𝑝²_𝑗. (8)

3.2. Compute Geometry Rigid Transformation

Suppose that Δ𝑝¹₁𝑝¹₂𝑝¹₃ and Δ𝑝²₁𝑝²₂𝑝²₃ are two similar asymmetrical triangles, (𝑝¹_𝑖,𝑝²_𝑖),and𝑖∈{1,2,3} are corresponding points, we represent the transformation 𝑇 between Δ𝑝¹₁𝑝¹₂𝑝¹₃ and Δ𝑝²₁𝑝²₂𝑝²₃ into a translation transformation 𝑁 and a rotation transformation 𝐴: 𝑇Δ𝑝₁𝑝₂𝑝₃=𝐴𝑝₁𝑝₂𝑝₃+𝑁, (9) where 𝐴=(𝑛₁𝑛₂𝑛₃)^𝑇 is a rotation matrix and 𝑁 is a translation matrix.

We firstly shift the triangle centers to the origin of coordinate, so the transformation 𝑇 satisfies 𝑝²₁𝑝²₂𝑝²₃−𝑁₂=𝐴𝑝¹₁𝑝¹₂𝑝¹₃−𝑁₁, (10) where 𝑁₁, 𝑁₂ is the center location matrix of Δ𝑝¹₁𝑝¹₂𝑝¹₃ and Δ𝑝²₁𝑝²₂𝑝²₃.

Therefore, rotation matrix can be deduced as in the following formula: 𝐴=𝑝²₁𝑝²₂𝑝²₃−𝑁₂𝑝¹₁𝑝¹₂𝑝¹₃−𝑁₁⁻¹. (11)

3.3. Generate Triangle-Pair Set

Similar triangle pairs from two surfaces will be stored in Triangle-Pair Set defined as follows:

Definition 4. A Triangle-Pair Set named TS is a set TS=Δ𝑝¹₁𝑝¹₂𝑝¹₃,Δ𝑝²₁𝑝²₂𝑝²₃,𝑇∣𝑝¹_𝑖𝑝²_𝑖()()∈PS,DT<𝜀𝑡𝜀𝑡=maxDT+minDT2, (12) where 𝜀𝑡 is an average threshold to guarantee the similarity of Δ𝑝¹₁𝑝¹₂𝑝¹₃ and Δ𝑝²₁𝑝²₂𝑝²₃. 𝑇 is the transformation from Δ𝑝²₁𝑝²₂𝑝²₃ to Δ𝑝¹₁𝑝¹₂𝑝¹₃. So we can design the logic structure of Triangle-Pair Set as Figure 1.

Figure 1 

Logic structure of the Triangle-Pair Set.

3.4. Determine Optimal Transformation

The validity of every transformation will be tested through applying it to every other triangle pair in TS. For a specified transformation 𝑇₀ in PS, we define the scoring function as follows: Score𝑇₀=𝑆Δ𝑝¹₁𝑝¹₂𝑝¹₃,𝑝²₁𝑝²₂𝑝²₃,𝑇₀. (13)

Consider (Δ𝑝¹₁𝑝¹₂𝑝¹₃,Δ𝑝²₁𝑝²₂𝑝²₃) is an arbitrary similar triangle pair in PS, and function 𝑆 is defined as follows: 𝑆Δ𝑝¹₁𝑝¹₂𝑝¹₃,Δ𝑝²₁𝑝²₂𝑝²₃,𝑇₀=1DA𝑇₀Δ𝑝¹₁𝑝¹₂𝑝¹₃,Δ𝑝²₁𝑝²₂𝑝²₃()()<𝜀𝑎0others𝜀𝑎=maxDA+minDA2, (14) where 𝜀𝑎 is an average threshold to determine whether Δ𝑝¹₁𝑝¹₂𝑝¹₃ can be transformed to Δ𝑝²₁𝑝²₂𝑝²₃ by 𝑇₀. An optimal transformation 𝑇_𝑃 is the transformation with maximum score satisfying Score𝑇_𝑃=max(Score(𝑇)), (15) where 𝑇 is an arbitrary transformation in TS.

4. Experimental Results and Complexity Analysis

4.1. Detect Partially Occluded Object

The goal of the first experiment is to detect the partially occluded object from a range image using the algorithm proposed in this paper. The image data is established referred to [21].

In Figures 2(a) and 2(b), two original objects (Duck and Venusm) in 3D space are demonstrated, and the surface is formed by pixel points.

(a) Original object

(b) 3D model of original object

(c) Duck is partially occluded

(d) 3D model of the scene

Figure 2 

Two 3D original and partially occluded models.

In Figure 2(c), the Duck is partially occluded by the Venusm. In order to find the location of the occluded object (Duck) in the 3D scene, we must compute an optimal transformation from the scene to the occluded object, then apply the transformation to the occluded object and reconstruct it in the 3D scene in Figure 2(d).

In this experimentation, the computing results are demonstrated in Table 1.

Table 1 

Parameters and computing results.

Optimal triangle in original object is demonstrated in Figure 3(a), and the optimal triangle in occluded object is illustrated in Figure 3(b). According to the result in Table 1, we apply the transformation to original object. The reconstruction result is shown in Figure 3(c).

(a) Optimal triangle in original object

(b) Optimal triangle in occluded object

(c) Reconstruction result to occluded object

Figure 3 

Detect partially occluded object.

4.2. Partial Surface Matching for Two Similar Objects

The second experimentation demonstrates our algorithm applied in a universal case. The two surfaces are partially similar, and the goal is to find a transformation to match the most similar part of the two surfaces.

The two surfaces with missing data in Figures 4(a) and 4(b) are generated by a Matlab function. In order to match the most similar part of the two surfaces, an optimal transformation from surface 1 to surface 2 needs to be computed.

(a) Surface 1

(b) Surface 2

Figure 4 

Two partially similar surfaces.

The computing results in this experimentation are demonstrated in Table 2.

Table 2 

Parameters and computing result.

Optimal triangle in surface 1 is demonstrated in Figure 5(a), and the optimal triangle in surface 2 is illustrated in Figure 5(b). According to the results shown in Table 2, the transformation is applied to surface 1 and the matching and reconstruction result is demonstrated in Figure 5(c).

(a) Optimal triangle in surface 1

(b) Optimal triangle in surface 2

(c) Matching and reconstruction result

Figure 5 

Partial surface matching of two similar surfaces.

4.3. Complexity Analysis

Suppose that the two input sets have comparable sizes, we measure the computing complexity of the algorithm as a function of 𝑁 (the cardinality of the input sets); 𝑘 is the size of Point-Pair Set. Computing the curvature of every point takes 𝑂(𝑁) time. Preparing the Point-Pair Set can be executed with expected 𝑂(𝑁log𝑁) running time because we can sort the points based on curvature. In addition, seeking triangle pairs, computing and testing transformation based on Point-Pair Set will cost 𝑂(𝑘³) running time. In total, the running time of the whole algorithm is 𝑂(𝑁+𝑁log𝑁+𝑘³).

Obviously, the computing efficiency is mainly determined by the size of Point-Pair Set 𝑘. The quantity of point pairs will be smaller when the surface processes many different curvatures; in this case, our algorithm will cost less time at this time. So our algorithm is very suitable for the matching of irregular objects.

In order to explicitly describe the performance of our algorithm, we perform experimentations for objects with different shapes. Running times of some algorithms for 𝑁=900 are demonstrated in Table 3 (CPU: PIV2.0 GHZ, RAM: 1 GB, Software: MATLAB 7.0).

Table 3 

Performance comparison of some algorithms.

We can see from Table 3 that our algorithm cannot indicate an encouraging matching efficiency for regular objects such as sphere surfaces.

4.4. Noise Robustness

The proposed work depends crucially on the Gaussian curvature and means curvature computation, which is notoriously sensitive to noise and local perturbation. In order to analyze the affection of noise to the match results, we perform an experimentation to verify the noise robustness of this algorithm. In this experimentation, we generate 100 similar surfaces by Matlab function, and Gaussian noise (𝑁(0,𝜎)) is added to each image in the database and 𝜎 changes from 0 mm to 2.0 mm. We design the classification rate to represent the percentage of correct matching quantity in the database; the classification rates are shown in Figure 6 for various 𝜎 values.

Figure 6 

Classification rate when Gaussian noise increases.

The result shows that the algorithm in this paper does not have good noisy robustness. At present, our approach can get encouraging matching efficiency and running time complexity in case of high signal to noise ratio. Therefore, the future work will concentrate on this problem.

5. Conclusion

In this paper, an approach for solving the 3D partial model matching is proposed. Firstly we utilize Gaussian curvature and mean curvature to represent the two object surfaces. Then curvature distance is defined to classify the point pair with similar curvature, and Point-Pair Set is employed to store point pairs. Secondly, we classify triangle pair with similar shape features from the two surfaces based on Point-Pair Set, similar distance is employed to measure the similarity of two triangles, and then the Triangle-Pair Set is generated. Finally, absolute distance and score function are proposed for measuring the goodness of a given transformation in Triangle-Pair Set, and a successful matching is determined based on the score of optimal transformation between the objects. Experimental results indicated our algorithm can be suitable for the matching of natural objects.

However, the efficiency of this approach would be reduced when the shape is regular and the robustness to noise is not very good. In future work, we will plan to explore several improvements of the algorithm, including the improvement of noise robustness, design of better distance definition, experimentation with other scoring functions, and further study of statistical approaches.

Acknowledgments

This work is supported by Research Fund for the Doctoral Program of Higher Education of China (20090162120069), Science and Technology Plan of Hunan (2009FJ3016), Postdoctoral Fund of Central South University, China Postdoctoral Science Foundation (2012M521552), and NSFC (61202341).