Abstract

To improve the accurate and sufficient recognition of abnormal points on the workpiece, a multidimensional anomaly point identification approach based on an improved eigenvalue method is proposed in this paper. Whether a point is normal or not depends on the angle between the two adjacent vectors which consisted of four adjacent points around the current focus. The comprehensive judgment is carried out by multidimensional approximation. The numerical simulation and actual experiment validate the efficiency of the proposed method to quickly and accurately identify the abnormal point cloud in the surface point cloud data.

1. Introduction

As a typical mechatronics equipment, industrial robots have been widely used in modern industrial production. Combined with robot vision and artificial intelligence technology, the intelligent production lines are formed by industrial robots to replace some humans for heavy and boring works and boring and repetitive tasks. Earlier industrial robots were mainly used for assembly, welding, handling, spraying, and other precision requirements. With the improvement of precision and rigidity of industrial robots, more and more industrial robots have been directly applied to grinding and drilling. In the case of cutting, milling, etc., it gradually replaces the traditional manual operation with harsh working environment, low processing efficiency, and poor consistency of surface polishing quality [1].

Robotic grinding has been widely used in industries such as automotive manufacturing, glass processing, and phased array radar and aircraft manufacturing. In automotive body grinding, the body normal vector is directly obtained by mathematical models; however, due to factors such as machining, assembly errors, and stress concentration of the body parts, the actual shape and theoretical shape of the body often deviate. Grinding the surface normal in the mathematical model will lead to defects in the quality of the body grinding. At the same time, the body shape of each car is often affected by unpredictable factors such as machining error, assembly error, and material deformation of the components, which often leads to different deviations between the actual shape and the theoretical shape of the same part of the same workpiece. The point cloud normal grinding in the model will result in grinding quality defects. As a single car, it is only necessary to ensure that the car body surface has a high degree of smoothness, and it is not necessary to ensure that each car has an absolute unity smoothness at the same position. Therefore, in the process of industrial robot grinding, it is necessary (i) to solve the body normal vector according to the actual shape of the grinding part, (ii) to adjust the corresponding posture of the end effector, and (iii) to adjust the grinding of each grinding according to the data of the specific abnormal point.

The typical size of the workpieces is large, and hence the corresponding surface to be ground can be approximated as a plane feature within a certain range (10 mm  10 mm). By extracting these plane features using a laser sensor, it is possible to calculate the normal vector of each point. The grinding parameters can also be determined during subsequent grinding, and hence the surface feature extraction is an important part of robotic vision grinding. At present, there are many methods for fitting surface points to surface equations, and many have studied the recognition and extraction of abnormal point clouds. However, there are only few studies on combining surface anomaly point cloud recognition with industrial robot grinding.

Recently, many experts and scholars have carried out a lot of research on the identification of abnormal points obtained from the robot visual grinding system. Lindner et al. [2] studied a novel robotic vision system that uses laser dynamic triangulation to determine the three-dimensional coordinates of an object of observation, thereby improving the applicability of the vision system. Wu et al. [3] proposed a kind of footwear tracking method based on industrial robots. The proposed method mainly includes two steps of three-dimensional point cloud reconstruction and feature curve extraction. Princely and Selvaraj [4] proposed a vision-guided robot system (VGRS) to remove burrs to eliminate the weakness of “teaching” or “offline” programming. This system collects each artifact as a two-dimensional graphic data and automatically generates robot programs based on workpiece graphic data and finishing parameters. The robot system proposed by Ji et al. [5] consists of a structured light-based 3D scanner and a robot arm UR5 fixed on a mobile platform. The 3D scanner is used to identify wall defects and measure the roughness. Muja et al. proposed a fast method to find the nearest neighbor search packet (FLANN) based on the existing research results [69], including layer k-means tree and multiple random k-dimensional tree [10]. The methods described above do not have the ability to detect and accurately identify abnormal points and feature edges, resulting in an algorithm that is not practical. In this paper, a reliable point cloud data plane fitting method is proposed, which is called as the improved eigenvalue method. This method eliminates outliers in point cloud data by the multidimensional improved eigenvalue method to achieve robust identification of abnormal point cloud and feature edge.

2. Application of Improved Eigenvalue Method in Robot Milling

2.1. Expression of Deterministic Surfaces

The point cloud data of the grinding surface are obtained by a visual recognition device. When an abnormal point cloud is recognized in the whole point cloud data obtained, the robot may move to that abnormal point cloud performing an incorrect grinding operation. Commonly, acquisition methods of surface point cloud data include visual measurement sensors, contact measurement sensors, and noncontact measurement sensors. This study used the widely used noncontact measurement sensor—point laser sensor (PLS) [11].

B-spline curves and surfaces have local modifiability and strong convexity, which can successfully solve the description problem of free-curve surfaces. This paper describes the deterministic surface model by using bicubic B-spline surfaces commonly used in engineering.

Given (n + 1) × (m + 1) points Qij (i = 0, 1, …, n; j = 0, 1, …, m) in space, an n × m number of Bézier patches can be approximated, as shown in Figure 1. The expression of multiple cubic B-spline patches is expressed as follows [1217]:where Ui (u) and Vi () are the B-spline basis functions, Cij is the control point, and the number of patches is h × l. The surface model is the linear combination of the tensor product of the control point and the basis function. Therefore, the above formula is expressed aswhere Xk is the tensor product of the B-spline basis function, Ck = (xck, yck, zck)T is the corresponding control point, and N = (h + 3)(l + 3) is the number of control points.

The surface to be ground is considered as a plane perpendicular to the normal of the point in a certain area. For the shell of airplanes, high-speed trains, and automobile sheet metal, the surface curvature changes gently and the curvature is small. The parts to be ground is approximated as plane treatment. On this basis, the analysis and calculation of the measurement data are simplified accordingly. Proof can meet the needs of the project [18, 19]. For general surfaces, the normal vector is calculated by the vector method. To simplify the problem without loss of generality, we approximate the surface in a small area to a plane, and the normal vector at point is

Therefore, the grinding surface is regarded as plane grinding within a certain range. There are many methods for fitting a point cloud to obtain a plane using a point laser sensor. The commonly used methods are the least squares method and the eigenvalue method. The above two methods do not take into consideration the possible outliers in the acquired planar point cloud data. For this reason, an improved planar fitting method is adopted in this study based on the eigenvalue method.

2.2. Eigenvalue Method

Define as the measurement points of the machined part, where Ti is the element of the point.

The equation of the space plane is expressed aswhere A, B, and C are the unit normal vectors of the points to be grounded, i.e., , and di is the distance from the coordinate origin to the plane, di 0; to determine the plane characteristics, the key is to determine the four parameters A, B, C, and D [20, 21].

It is assumed that a certain plane is scanned, and n data points {(xi, yi, zi), i = 1, 2, …, n} are obtained. Using equation (4) to represent the plane equation, any data point (xi, yi, zi) the distance to the plane is

To obtain the best fit plane, the following formula should be satisfied under the condition :

The extremum of the function is calculated by the Lagrangian multiplier method which is expressed as

Deriving the above formula for d and letting the derivative be zero,

Then, there is

Therefore, equation (5) is expressed as follows:among them

Let equation (7) be derived for A and let the derivative be zero:where Similarly, for formula (4) for B and C, let the derivative be equal to 0 and the following is obtained:

The characteristic equations of (11a)–(11c) are formed to obtain the following equation:

From equation (8), the problem of solving A, B, and C is transformed into the solution of matrix eigenvalues and eigenvectors.

Make

Obviously, the matrix M is a 3  3 real symmetric matrix, which is known from the matrix knowledge. For a real symmetric matrix, its eigenvalue can be obtained by using the following formula:where (,) represents the dot product of two vectors.

Expand (14) and consider that , i.e., (x, x) = 1, then we get

Therefore, the minimum value of is the minimum eigenvalue of a, and the eigenvector value corresponding to the smallest eigenvalue is A, B, and C.

The eigenvalues of the real symmetric matrix M can be obtained by using the following equation:

M is a 3 × 3 real symmetric matrix, so the matrix has at most 3 real eigenvalues, and is the smallest one. After is obtained, the corresponding homogeneous linear equation is obtained:

The obtained nonzero solution is the eigenvector corresponding to .which are the parameters A, B, and C of the plane equation. is the true unit normal vector of the measurement site.

2.3. Multidimensional Improved Eigenvalue Method for Identifying Abnormal Point Clouds

When using the laser scanning system to scan the research object due to various factors, such as occlusion and dust, there are abnormal points in the acquired point cloud data. The eigenvalue method does not consider the existence of these abnormal points. It directly uses all the data points in the point cloud to perform plane fitting, which results in the plane parameters A, B, and C being not optimal, that is, the fitted plane does not reflect the true shape of the scanned surface. Based on the eigenvalue method, this paper obtains the estimated values of the smooth plane parameters A, B, and C by identifying the anomalous points in the point cloud, which is called the improved eigenvalue method [2225]. The specific algorithm is as follows.

The modified eigenvalue method is used to derive the normal parameters of the local tiny parts. On this basis, the abnormal point cloud can be identified. In the direction of the normal vector, the angle between the point and the adjacent two points is calculated by judging the abnormal point to determine whether the point is an abnormal point. If is smaller than the predetermined value, the point cloud is considered as an abnormal point. The tool is processed by the cutting tool, otherwise it is normal. As shown in Figures 24, let and be the adjacent three points in the point cloud data, and the judgment process of whether is an abnormal point is as follows:

(1)Calculate the corresponding unit vector and :(2)Calculate the angle between and :(3)Determine whether is less than a preset allowable angle (according to experience, generally take ). If t then the point cloud is abnormal and may need to be grounded, otherwise it corresponds to normal point cloud data of the workpiece contour that needs to be retained.(4)If the point is judged to be an abnormal point in the tool running direction, pass and . The crossover direction is re-evaluated using the method of steps 1–3.As seen from Figure 4, the vertical distance of to the surface is expressed as follows:The occurrence of the abnormal point has a precursor. As shown in the figure, , , and are the data obtained by the path measurement, where is an abnormal point. Because the small step size (0.2–0.5 mm, empirical value) is used for sampling, and are close to 180° when judged by the angle method, is the previous point of the abnormal point , and will decrease by a certain extent compared with , but is generally not less than 90°. The angle corresponding to the abnormal point is generally less than 90°. From the above analysis, we can see that before the occurrence of the abnormal point, will shrink to a certain extent. For a special point cloud, the measuring device can judge from all directions. If the angle of the point cloud data in each direction is smaller than the set value, then the point must be an abnormal point. If only one direction is judged to be an abnormal point, the point may be either a normal point or an abnormal point. This method can be applied to the process of judging the abnormal point of a complex surface, and the abnormal point can be accurately determined. Subsequent processing may happen at this step.(5)Recalculate A, B, and C using all retained point cloud data.(6)Repeat steps 2–5 until all the remaining points are less than the preset allowable angle .(7)Calculate the best A, B, and C values.(8)Using equation (5) to calculate d, the best plane fitting equation can be obtained.

The abnormal point cloud identification process is shown in Figure 5.

3. Simulation Analysis

For the identification of the abnormal point cloud on the surface for the simulated improved eigenvalue method, use MATLAB to fit the function . If the surface can be regarded as a plane within a certain range, the extraction plane must satisfy . Randomly extract 10,000 points on this surface, as shown in Figure 6, artificially make 5 of them become abnormal points, 2 of which are on the edge strips, and simulate the edge strips on the car sheet metal, then the other 4 points are on the normal surface. As shown in Figure 7, the circle indicates an abnormal point [26, 27].

Using MATLAB tools, the least square method, eigenvalue method, and improved eigenvalue method were used to estimate the parameters of the above simulation data. Tables 1 and 2 show the calculation results.

Table 1 shows the calculation results of the parameters of the three methods without abnormal values.

When an abnormal point cloud is added to the point cloud data, the calculation results by three different methods are shown in Table 2.

From the data of the two tables, when there is no abnormal value in the point cloud data, the calculation results of the three methods are consistent; but when there is an abnormal point, the eigenvalue method and the proposed improved eigenvalue method based on this method are most accurate for the identification of abnormal point clouds. At the same time, when there are edge strips on the surface, the eigenvalue method is not ready to identify the point cloud data of the edge strips. The improved recognition algorithm can identify the multidimensional comprehensive judgment method. The point cloud is the characteristic edge strip to be ground by the abnormal point cloud or the curved surface, which reduces the occurrence of misjudgment during the grinding process.

4. Experimental Verification

The robot automatic grinding system is mainly composed of tooling equipment, measuring instrument equipment, control equipment, and software system. The tooling equipment is composed of a robot, a mobile platform, and a terminal actuator; the measuring instrument is mainly a laser tracking range finder. The software system is composed of a robot offline programming and simulation system, a robot control system, an automatic grinding control system, and a laser tracking measurement system. This study belongs to the abnormal point cloud identification of the measurement part. The integrated laser point cloud measurement module is used for measurement. The laser ranging module emits a laser beam to illuminate the surface of the workpiece and receives the reflected laser light for distance measurement. The laser ranging module can output a standard signal, or can add a postprocessing unit to directly output the distance value. The laser noncontact measurement has high precision, is less affected by the material of the workpiece, and is less prone to collision. The application range is relatively wide. The measurement equipment and point cloud data are as shown in Figure 8 [2831].

Figure 9 shows the 3D point cloud data obtained by scanning the car rearview mirror, converting the data format and importing it into MATLAB. As can be seen from Figures 10 and 11, the size of the rearview mirror burrs is uneven, showing uneven and discontinuous distribution. The eigenvalue method and the multidimensional improved eigenvalue method (MIEM) were used to identify the anomaly points. It was found that the eigenvalue method can identify the anomalous points in the plane area as well as abnormal points on the side strips, but the identification accuracy is low. In fact, the feature edge strip is correctly external with respect to the profile (see Figure 10, the line highlighted in blue), but it is identified as abnormal. This is showed that this method is not suited to correctly identify abnormal points that are at the boundaries of the profile (i.e., between the edge strip and the plane). On the other hand, the multidimensional improved eigenvalue method (MIEM) can not only accurately identify the abnormal points in the plane area, but can also prepare to recognize the surface. This is because the improved eigenvalue method (MIEM) is a comprehensive judgment from multiple dimensions, the identified abnormal points are better than the previous two methods, and the recognition of feature edges on complex surfaces is particularly accurate.

5. Conclusion

Because the previous surface fitting algorithm only recognizes abnormal points from one dimension, this paper proposes an improved multidimensional identification method, which is validated through numerical simulation and actual experiment. The results prove the multidimensional improved eigenvalue method’s (MIEM) feasibility and robustness. This method can accurately obtain the normal vector direction near the scanning point and then judge the abnormal point cloud from multiple dimensions by the angle method, which lays a theoretical foundation for the subsequent determination of grinding thickness and grinding method.

Data Availability

The data used to support the findings of this study have not been made available because the data of this study is mainly the surface point cloud data, which can be scanned by the network or other equipment. This study mainly proposes a method.

Conflicts of Interest

The authors declare that there are no conflicts of interest regarding this paper.

Acknowledgments

This work was supported by the National Natural Science Foundations (grant no. 51175419), Project of Institute of Higher Vocational Education, Nanjing Institute of Technology (grant nos. GJ18–05 and GJ19-09Z), and Shaanxi Key Laboratory of Machinery Manufacturing Equipment Construction Project, which are highly appreciated by the authors.