Abstract
In view of the problem of “jumping points” in the phase unwrapping of the multifrequency heterodyne principle, this paper proposes a novel method to improve the multifrequency heterodyne. By solving the root-mean-square error of the original frequency function, it includes the relationship between the error and the adjacent phase in the condition of constrained phase unwrapping, and it compensates phase of the skip point. To ensure the accuracy of phase unwrapping, the function with a “jump point” after each phase unwrapping and the absolute phase curve of the principal value function are used to establish the threshold judgment model of the least square method, and the initial phase unwrapping of the principal value function with different frequencies is carried out continuously. The simulation analysis of phase compensation with the four-step phase-shifting method shows that the error is reduced to 36% under the set environment. The experimental result of 3D reconstruction by measuring the flatness of the plate shows that the error decreases by 41% after phase compensation compared with before phase compensation. The three-dimensional reconstruction experiment of pitch measurement with a nut shows that the nut after phase compensation is smooth without noise, and the pitch error is 0.033 mm, which verified the method is workable and effective.
1. Introduction
Based on triangulation, the structured light three-dimensional measurement method has been widely used in various fields due to its high-precision and noncontact measurement [1–3]. For example, the measurement of three-dimensional morphology is applied to reverse engineering, biomedical treatment, heritage acquisition, architecture and computer animation, etc. [4–6]. To get a high-precision three-dimensional profile, we need to unwrap the phase accurately. The four-step phase-shifting method is adopted in the 3D reconstruction process. The arctangent function is involved in the derivation process, so the phase is wrapped between . Therefore, the wrapped phase needs to be unwrapped continuously in the global field [7].
At present, the principle of multifrequency heterodyne is one of the widely used methods. Multifrequency heterodyne is a time phase unwrapping method [8, 9]. The calculation of phase unwrapping at different frequencies is independent in the process of phase unwrapping. Therefore, the algorithm can better suppress interference. However, there are some problems in phase unwrapping of multifrequency heterodyne, such as jumping error [10]. Jiang et al. [11] proposed a series of constraints to compensate the phase of multifrequency heterodyne. Zhang et al. proposed a look-up table method to build a preestablished storage table between the input gray and the output gray to compensate for the nonlinear phase error in the projection and acquisition system, but this method took a lot of time to calculate [12]. Cai et al. used the Hilbert algorithm to compensate the reverse error, but this method was unable to measure the profile of complex objects, and the measurement speed was slow [13].
In order to solve the problem of jumping error in the principle of multifrequency heterodyne, this paper proposes a novel method to improve multifrequency heterodyne [14–16]. The innovation of this paper is described below. The root-mean-square error of the principal value function with different frequencies is solved, and the phase unwrapping is carried out with this error as the corresponding constraint condition. Finally, the least square threshold judgment model is established for the function with a “jump point” after each phase unwrapping and the absolute phase curve of the principal value function. According to the threshold value, whether to enter the next iteration is judged. If the error is greater than the set value, then the error is taken as the constraint condition for phase unwrapping again until the threshold condition is met.
In this way, the jumping error is eliminated when the phase of the initial principal value function of different frequencies is unwrapped. Through the simulation of phase unwrapping of sinusoidal gratings with three different frequencies of noise, there is a “jump point” in phase unwrapping before phase compensation, and a continuous phase is got after phase compensation; in the experimental part, the method is applied to 3D profile reconstruction, and the experimental results show that the 3D profile after phase compensation is more smooth without a jump point.
In this paper, the contributions of the phase error compensation method are as follows. Firstly, the constraints of the phase error compensation method in this paper are more concise and easy to calculate. Secondly, this paper compensates the wrapping phase of various frequencies, respectively, and the phase becomes more continuous after phase unwrapping. Thirdly, compared with single-frequency wrapping phase error compensation, the multifrequency heterodyne method is more fault-tolerant and accurate in 3D reconstruction of objects because of the independent phase unwrapping process of various frequencies. The second part of this paper involves the description of the principle of multifrequency heterodyne phase unwrapping; the third part is a novel method of phase compensation; we divide the fourth part into simulation verification of this method and experimental verification of the feasibility of this method; the fifth part is the summary.
2. The Principle of the Multifrequency Heterodyne Method
The principle of the heterodyne method is to stack the principal values of sine grating functions with different frequencies and finally get the entire image with the frequency of 1 Hz, where absolute phase of its expression is [17] where is an integer. is the principal value function of the phase of the sine grating functions. The key to phase unwrapping is to find the value of .
Different frequencies of and correspond to phase function and , respectively. The wrapped phase maps of and will be unwrapped, where the expression of absolute phase is [18]
The schematic diagram of multifrequency heterodyne shows phase unwrapping in Figure 1.
The fringe series satisfy
The integer fringe numbers of the fringe patterns of different frequencies on the same point satisfy
The heterodyne phase value is
From Equations (2)–(5), we get
The frequency value after superposition of different frequencies is
From Equations (6) and (7), the absolute phase value of fringe gratings with different frequencies is
From Equations (2) and (8), the key value is
Finally, the absolute phase value can be got [19]:
In fact, there will be some “jumping points” in the process of multifrequency heterodyne phase unwrapping. Therefore, it is necessary to compensate the errors to improve the precision of phase unwrapping.
3. Multifrequency Heterodyne Phase Error Compensation
There are three kinds of gratings with different frequencies. The projector projects the gratings with fringe periods of 70, 64, and 59. After collecting by a CCD camera, they carry the phase unwrapping processing out. Finally, the noise problem shown in the red circle area in Figure 2 will appear in the result of global phase unwrapping. After analysis, it is a jump error. Therefore, it is necessary to compensate the phase error before 3D reconstruction of the object, so as to avoid “jumping points” in the reconstructed model.
With the multifrequency heterodyne method, there are some problems such as jumping error. First, calculate the root-mean-square error (RMSE) after the phase unwrapping of sine grating functions with different frequencies. According to the principle of multifrequency heterodyne in the upper section and the formula below, the root-mean-square error (RMSE) can be got: where defined as where is the absolute phase value of a certain frequency grating, i.e., the value after phase unwrapping of fringes; is the frequency of the fringes; and is the total frequency. The wrapped phase error is analyzed, and the error is a Gaussian distribution, and the root-mean-square error (RMSE) is calculated as , where , , are the principal value function , , and Gaussian standard deviations. To deal with the “ jump point” error, add the constraint conditions to the original algorithm. The constraints are where is the wrapped phase after dual-frequency correction, computing and over and , and select . In this way, avoid the jumping error caused by the noise in the wrapped phase. The phase unwrapping function after correction is
For the heterodyne function , the constraints are where is the wrapped phase after dual-frequency correction. Two wrapped phase maps of and have Gaussian standard deviation and , respectively, computing and over and , and select . is using a similar method. Compensation for jumping errors in the wrapped phase map is computed by where is the wrapped phase after compensation. At this moment, the value of is determined by . where and are the phase unwrapping function after correction. After phase compensation is completed, the model relationship between the compensated phase unwrapping mean and the compensated frequency phase unwinding phase function value is established. The model relation is given by formula (18). After phase compensation, there may be a “jump point” function and its absolute phase curve of its principal value function to establish the least square method threshold judgment model and determine whether the error is less than the set threshold value. If not, return the error to Equations (13)–(17) as the condition of constrained phase unwrapping and then carry out phase compensation until the threshold condition is met. The judgment equation is
The multifrequency heterodyne completes phase compensation.
4. Simulation and Experimental Analysis
4.1. Simulation Analysis of 3D Reconstruction before and after Phase Error Compensation
Three sinusoidal gratings with different frequencies are selected for phase compensation. The three frequencies are 70, 64, and 59, respectively. The simulation environment is a nonideal sinusoidal grating with a resolution of . In Figure 3, it can be seen that the environment noise before compensation will cause the “jump points” of the last phase unwrapping. First, the root-mean-square error (RMSE) of the noise is calculated as , which is the constraint condition of Equations (13) and (15) in this paper. Functions and , through judging the and relationship, determine whether a jump occurs, and finally, the phase of the jump point is carried out to add or subtract . The green curves in Figures 3(a) and 3(b) represent the principal value functions and of the wrapped phase, respectively; the red curves are and ; the blue curves are and after the phase unwrapping. It can be seen that the function shows “jumping points” because of noise, and the pixel jump reaches 30-40 pixels at the beginning and end. After the algorithm proposed in this paper is used for compensation, as shown in Figures 3(c) and 3(d), the pixel jump drops to 5 pixels after the phase of the blue curve is unwrapped, indicating that the phase is compensated.
(a)
(b)
(c)
(d)
Figures 4(a), 4(b), and 4(c), respectively, correspond to the phase compensation front curve of functions , , and . The initial noise also exists in these three principal value functions. It is transferred to the principal value function after superposition frequency by the heterodyne method. The basic steps of the heterodyne method are as follows: step 1: get the phase value of the principal value function first; step 2: get an entire image with the phase frequency of 1 Hz by using the principle of the heterodyne method; and step 3: reverse . According to the heterodyne method, the frequency is got by superposition, and the sequence of phase unwrapping is the principal value functions , , , , and , while the final purpose is to obtain the last three different frequency principal value functions, the continuous phase unwrapping of , , and . In the process of phase unwrapping, when the errors of Figures 3(c) and 3(d) pass through the next phase unwrapping, because the errors of the jumping points of the principal value functions and are already tiny, the relationship of constraint conditions and may not meet the conditions of or , which is not enough to affect the phase unwrapping of , , and , so the phase unwrapping no longer has a jump. In this paper, a decision condition of the least square method is added to the method. The error is still large, but after many iterations, it is found that the phase unwrapping of , , and in Figures 4(d), 4(e), 4(f), respectively, does not have jumping points.
(a)
(b)
(c)
(d)
(e)
(f)
The experimental results show that the errors of the principal value function before phase compensation (phase wrapping) and after phase compensation (phase unwrapping) can be ignored, and the errors of the principal value function of heterodyne can be reduced by 36%. Table 1 shows the calculated standard deviation before and after the phase compensation, which shows that the algorithm in this paper has a significant improvement on the accuracy.
In order to further prove the accuracy and better visualization effect of the method proposed in this paper, combined with MATLAB, the hemispherical radius is simulated by the four-step phase-shifting method, and the grating resolution ratio is . Gaussian noise with standard deviation of 0.01 is added. Figures 5(a–f) display the three-dimensional and two-dimensional images of the phase unwrapping, respectively. Through the comparison of the two sets of data, we can see the recovery morphology of the hemisphere after the phase compensation, which shows that the compensation method proposed in this paper is accurate and workable.
(a)
(b)
(c)
(d)
(e)
(f)
4.2. Experimental Analysis of 3D Reconstruction before and after Phase Error Compensation
In the experiment, the flat plate and nut are scanned for verification. Figure 6(a) is for scanning and collecting the plate data. The experimental platform mainly includes a screw rod, a two-dimensional storage platform, a projector and camera installed on the screw rod, and a two-dimensional storage platform controlled by a step motor, which can move along the horizontal -axis and -axis. Figure 6(b) is to collect the nut data. The place where the nut of the experimental platform is placed is a two-freedom-degree rotating platform, which can swing the arm forward and backward, and the fixed axis can rotate in a circle. In the experiment, the projector resolution ratio is pixels, the camera resolution ratio is pixels, and the number of fringe periods is 64, 60, and 57. The upper computer of the two platforms is a Windows 10 system, and the CPU is Intel i5. The camera collects the data of the flat plate and the nut, respectively, gets the data, and compares the effect of phase compensation and noncompensation for the data.
(a)
(b)
First, collect the plate data, and the number of effective points collected is. In this paper, 3D reconstruction is carried out by the four-step phase-shifting method. Figure 7(a) shows the three-dimensional image before phase compensation, in which there are jump points when the pixel is close to 64 pixels. Figure 7(b) carries out phase compensation in this method, and it can be seen that it eliminates the “jump points.” To show the more obvious change, Figure 7(c) shows the variation of the cross-section data, data before phase compensation (red), and data after phase compensation (blue); the front and back error is reduced from the maximum error of 0.5 mm to 0.1 mm; and the error is reduced by 40%, verifying the feasibility of the phase compensation method.
(a)
(b)
(c)
To analyze the accuracy of the phase error compensation method proposed in this paper, the nuts which are widely used in industry are used for reconstruction. The pitch standard is easy to query and can be analyzed quantitatively. Therefore, the pitch value is taken as another standard to evaluate the accuracy of the phase error compensation method in this paper. The four-step phase-shifting method is used to collect nut data for 3D reconstruction. Figure 8 shows the reconstruction model of the nut before and after phase compensation.
(a)
(b)
By calculating the thread pitch of the nut (unit: mm) as the measurement standard, analyze the distance between ten adjacent threads of the screw, among which the specification of the screw is M12, and the standard pitch is 1.75 mm. For the acquired data, the distance between adjacent ten threads is recorded as shown in Table 2.
According to the table, Figures 8(a) and 8(b) show that the surface of the nut after compensation is flat and there is no jumping error. It shows that the novel method of multifrequency heterodyne phase error compensation proposed in this paper can effectively eliminate the “jump point” in the actual 3D reconstruction and ensure the high reconstruction accuracy.
5. Conclusion
To eliminate the jumping error in the process of phase unwrapping, this paper proposes a novel method of multifrequency heterodyne phase compensation, which solves the problem of “jumping points” in the actual situation. In this paper, the root-mean-square error (RMSE) of the principal value function with different frequencies after phase unwrapping is calculated first, and they include it in the constraint condition of phase compensation, to suppress the jump error. The least square threshold judgment condition is added to verify whether there is still a “jump point” with large error after compensation, which affects the global unwrapping of the last phase. If it is greater than the threshold value, it will carry a simple reiteration to meet the threshold requirements. In this paper, the simulation analysis shows that the error is reduced by 36% in the set environment, and the phase unwrapping after the error compensation can effectively eliminate the “jumping points” and get the continuous phase. The experimental verification of flatness measurement shows that the error after phase compensation is reduced by 41% compared with that before compensation, and the three-dimensional reconstruction of the nut can also achieve high accuracy. After the error compensation, the 3D reconstruction object surface becomes smoother and more continuous. Simulation and experiment show that the improved multifrequency heterodyne phase compensation method is workable.
The future direction of the work in the paper can be divided into the following points. First, it can be seen from the experimental data before and after the compensation of the plate that there is still room for correction of the error of the plate, and it is not clear whether the formula proposed in this paper is the best for the reconstruction error compensation. Second, this paper is to carry out 3D reconstruction of objects without mirror reflection. How to achieve high-precision 3D reconstruction of mirror reflection objects is still a problem to be solved.
Data Availability
The data used to support the findings of this study are available from the corresponding authors upon request.
Conflicts of Interest
The authors declare that there is no conflict of interest regarding the publication of this paper.
Acknowledgments
This work was supported in part by the National Natural Science Foundation of China under Grant 61701296 and Grant U1831133, in part by the Natural Science Foundation of Shanghai under Grant 17ZR1443500, and in part by the Shanghai Aerospace Science and Engineering Fund under Grant SAST2017-062.