Wireless Sensors based on the Internet of ThingsView this Special Issue
Blind Crosstalk Correction for Color Fringe Projection Profilometry Combining the Sine and Cosine Phase-Shifting Patterns
Color crosstalk is challenging for color fringe projection profilometry, which will cause color imbalance and phase-shift deviation, thus resulting in phase error. To address this challenge, this paper presents a blind crosstalk correction method for color phase-shifting fringe projection profilometry without crosstalk calibration. Three channels of the color fringe are firstly normalized to resolve color imbalance. The phase error induced by phase-shift deviation between three channels doubles the frequency of the color fringe. Therefore, the original cosine color fringe is shifted by quarter period to generate another sine color fringe to compensate the periodic phase error. Both simulations and experiments were carried out, and their results confirmed the effectiveness and accuracy of the proposed method.
Fringe projection profilometry (FPP) based on triangulation principle is one of the most commonly used methods in optical three-dimensional (3D) shape measurement due to its low cost, fast speed, and high-resolution capabilities [1–6]. Normally, a FPP system contains a computer, a projector, and a camera. The projector and the camera are directed along different directions to the measured objects, all of which are controlled by the computer. Firstly, the fringe patterns should be designed and generated by the computer. Then, these patterns are projected onto the measured objects by the projector; at the same time, the deformed patterns are recorded by the camera. Finally, these captured images are transferred to the computer and analyzed by phase extraction algorithms to obtain the geometric information of the measured objects . There are several phase extraction algorithms, including Fourier transform , wavelet transform , and phase-shifting profilometry (PSP) . Among them, PSP is the most popular method, because it can achieve pixel-wise phase extraction, which can reconstruct the measured objects with abrupt changes more robustly. However, it generally requires more fringe patterns (normally at least three) than transform methods .
In order to overcome this problem, color phase-shifting technique has been developed [12–19]. Three phase-shifting patterns are assigned to red, green, and blue channels, respectively. Thus, only one color fringe pattern is required to reconstruct 3D shapes. However, the spectra of red, green, and blue channels are usually made to have some overlaps so that there would be no color-blind areas on the spectrum [20–22]. Thence, phase error will be introduced if conventional phase-shifting algorithms are employed directly. To suppress the phase error caused by color crosstalk, Huang et al.  proposed a method to compensate the coupling effects between channels. Since the degrees of overlaps are fixed, they projected pure red, green, and blue fringe patterns onto a white-painted flat board to calculate the degrees of coupling effects between three channels and then compensated it. This method has been explored when different numbers of fringes are projected on each channel by Zhang . On the other hand, Wan et al.  presented an active phase error suppression method based on hue precorrection. The hue is precorrected to be linear by measuring the function of the combined effect and modifying the computer-generated hue according to the source-distorted relationship. Then, the new modified color fringe pattern can be generated for measuring objects. However, all these techniques need preprocessing before measurement, which makes it inconvenient to use. Hu et al.  proposed an algorithm for estimating the color remixing matrix based on the color fringe patterns captured from the reference plane or the surface of objects. Ma et al.  proposed a blind phase error suppression approach based on isotropic -dimensional fringe pattern normalization  and carrier squeezing interferometry . Wu et al.  proposed a point-to-point compensation method for color coupling and imbalance. The proposed method achieves point-to-point compensation based on the correspondence between camera pixels and projector pixels. Recently, Wang et al.  found the phase error introduced by crosstalk is approximately twice the frequency of color fringe. Therefore, Hilbert transform was applied on the color fringe to compensate the phase error. However, due to the fact that Hilbert transform belongs to frequency domain methods, this approach cannot handle the measured objects that contain discontinuities or separations well.
In this paper, we present a blind crosstalk correction method for color phase-shifting fringe projection profilometry. Due to the fact that phase error introduced by color crosstalk is twice the frequency of color fringe, the other color fringe with phase-shift is used to compensate the phase error. In addition, the fringe normalization was also designed to resolve the color imbalance problem. We describe the principle of this method in Section 2. Simulations and experiments are illustrated in Sections 3 and 4, respectively. Finally, Section 5 summarizes this work.
2.1. Color Phase-Shifting Fringe
Phase-shifting algorithms are widely used in optical metrology because of their high accuracy, among which three-step phase-shifting algorithm is the most commonly chosen algorithm since it requires the least number of patterns. Three phase-shifting patterns can be separately encoded into three channels of one color fringe. Ideally, the three channels of the color fringe can be written as
where denotes the image pixel, is the average intensity, is the intensity modulation, and is the wrapped phase. Simultaneously solving the above equations will lead to
The arctangent function gives the wrapped phase ranging from to with phase jumps. Therefore, phase unwrapping should be performed to recover the absolute phase map. The relationship between the wrapped phase and the absolute phase can be described as where represents the fringe order, which can be determined by phase unwrapping algorithms.
2.2. Color Crosstalk Model
Most color cameras and projectors are designed to have spectrum overlaps between three channels to avoid color-blind regions in the spectrum. As a result, it is difficult to completely separate the three encoded fringes because of color crosstalk. In general, the linear color crosstalk can be described as  where are the RGB channels of the captured fringe pattern, represents the crosstalk matrix, and are crosstalk factors. In general, the crosstalk between red and blue channels is smaller than that between red and green channels or between blue and green channels. Thus, crosstalk factors generally satisfy the following constraints:
Taking the red channel as an example, can be further expressed as
The above equations can be rewritten as
Since , is positive and usually very small. The phase-shift deviations of green channel and blue channel can be analyzed in the equivalent way; and are also usually very small and is negative. As a result, the green channel and the blue channel can be further expressed as
Through above analysis, it can be known that the values of , , and are different (the same for , , and ), which will cause color imbalance between three channels of color fringe. To avoid introducing phase error caused by color imbalance, fringe normalization should be performed, which will be discussed in Section 2.5. After fringe normalization, the three channels of the color fringe can be rewritten as
2.3. Phase Error Analysis
Although the color imbalance problem is addressed after fringe normalization, there are still phase-shift deviations between three channels of color fringe due to the existence of . If we continue to calculate the phase as
Then, phase error will be introduced, which can be expressed as where and ; . It can be seen that highly relates to . To simplify the above equation, we assume that and . Based on this assumption, will be simplified as
Therefore, Equation (15) can be further approximated as 
This indicates that the phase error approximately doubles the frequency of the color fringe.
2.4. Phase Error Compensation
Based on this observation, if the original phase map is shifted by quarter period or , the shifted phase map can be used to compensate the quasisine phase error by averaging it with the original phase map. Therefore, the other color fringe with phase shift to the original one is also obtained as
Then, we can calculate the carried phase as
The phase error can be expressed as
Clearly, and have the same amplitude but opposite distributions. Thence, we can compensate the phase error as
2.5. Fringe Normalization
As mentioned above, two color fringes with phase shift are used to compensate the phase error which caused by phase-shift deviation. And the color crosstalk matrix of two color fringes can be considered the same. Namely, for the same channel, the values of and of two color fringes are equal. Thus, a background pattern is utilized to obtain the values of . By subtracting of each channel, the fringe intensities of two color fringes can be rewritten as
Then, they can be normalized by the following equations:
As shown in Figure 1, the framework of the proposed method contains three steps. Firstly, a background pattern is used to remove color imbalance in different channels of cosine and sine fringes. Then, two phase maps with phase errors caused by phase-shift deviation are extracted, respectively. Finally, the phase error caused by phase-shift deviation is compensated by averaging two phase maps.
In this section, simulations were carried out to verify the performance of the proposed method. The first simulated object is called peaks function in MATLAB, and its resolution is pixels. The fringe pitch of the patterns we used is pixels. As mentioned in Section 2.2, the crosstalk matrix should satisfy Equation (5). Therefore, we choose the crosstalk matrix as
Meanwhile, to simulate the uneven illumination, a 2D Gaussian function with a size of pixels and a standard deviation of 100 pixels was applied on the simulated patterns. And the uneven illumination can be expressed as
Figures 2(a) and 2(b) plot the 300th cross sections of cosine and sine fringes before fringe normalization, respectively. Obviously, the problem of color imbalance was existing in different channels. Then, the color imbalance problem was eliminated by fringe normalization, and the fringes after fringe normalization are shown in Figures 2(c) and 2(d). Figure 3(a) shows the 3D shape obtained before fringe normalization; periodic errors can be seen clearly. As shown in Figure 3(b), these errors were decreased after fringe normalization, but the errors caused by phase-shift deviation were still existing. Figure 3(c) shows the 3D shape retrieved after phase error compensation. Apparently, periodic errors were effectively suppressed by the proposed method. The reconstructed surface is very smooth, with almost no visible errors. In addition, their corresponding root mean square errors (RMSE) are shown in Figures 3(d)–3(f). Before fringe normalization, the RMSE in Figure 3(d) is 0.1298 rad. Figure 3(e) shows that the RMSE was reduced to 0.1099 rad after fringe normalization. And the RMSE, as shown in Figure 3(f), was reduced to 0.0550 rad after phase error compensation. This simulation illustrates the effectiveness of the proposed method.
To compare the proposed method with Wang’s method  which uses frequency domain method, a staircase with four steps was simulated. The 3D shapes reconstructed by the proposed method and Wang’s method are shown in Figures 4(a) and 4(b), respectively. Obviously, Wang’s method performed worse than the proposed method because it uses frequency domain method, which is not suitable for processing objects with abrupt surfaces. Figure 5 illustrates a part of the 300th cross section, which contains the abrupt change between the third and fourth steps. The third and fourth stairs are marked in green and yellow in Figure 4, respectively. The distortion of the abrupt change retrieved by Wang’s method is more serious than the proposed method. This simulation demonstrates that the proposed method can handle objects with abrupt surfaces well.
To further verify the proposed method, a FPP system was set up comprising a digital-light-processing (DLP) projector (Light-Crafter 4500) and a complementary-metal-oxide-semiconductor (COMS) camera (Point grey Chameleon3). The resolution of the camera is pixels, and the optical lens (Kowa LM16JCM) mounted on the camera has a focal length of 16 mm. The resolution of the projector is pixels. It is well known that high-frequency fringes can effectively resist noise and provide high-precision measurement . However, using high-frequency fringes also introduces phase ambiguities in the reconstruction process that need to be addressed by means of phase unwrapping. In these experiments, three-frequency method  was employed to unwrap the wrapped phase. Here, , , and represent periods of high-, medium-, and low-frequency fringes, respectively. And they satisfy the following equation: where is a constant. Corresponding wrapped phases of three frequency are , , and . In addition, is usually greater than the projector’s horizontal and vertical resolution to ensure complete elimination of phase ambiguity, which means the wrapped phase of low-frequency fringes has no phase ambiguity. It can be mathematically expressed as where is the absolute phase of low-frequency fringes. Then, the fringe order and absolute phase of medium-frequency fringes can be calculated as where returns the closest integer. After determining , the fringe order and absolute phase of high-frequency fringes can be obtained similarly:
Certainly, the errors caused by noise will inevitably be introduced in the actual measurement. Let us assume that the noise errors of low- and medium-frequency fringes are and , respectively. Then, the actual fringe orders obtained are
Obviously, if the value of is too large, the fringe orders of medium- and high-frequency fringes will be calculated wrong, which will result in unwrapping errors. In our measurement system, is a suitable value. As mentioned above, the period of low-frequency fringes should be greater than the projector’s horizontal and vertical resolution. Therefore, the period of high-frequency fringes should satisfy
In this paper, we choose pixels. Thus, pixels and pixels.
In the first experiment, a ball was measured. Total seven patterns were projected onto the ball and captured by the camera, as shown in Figure 6. Figures 6(a)–6(c) show the deformed cosine fringe patterns with periods of 18, 144, and 1152 pixels, respectively. Their corresponding sine fringe patterns are shown in Figures 6(d)–6(f). Figure 6(g) shows the background pattern that is used for fringe normalization. Before phase error compensation, we need to normalize these fringes to address the color imbalance problem. Figure 7 shows the 500th cross sections of six color patterns. Obviously, the imbalance problem existing in Figures 7(a)–7(f) was effectively resolved after fringe normalization, and the normalized fringes are shown in Figures 7(g)–7(l). Afterwards, the reconstructed 3D shapes are shown in Figure 8. Among them, Figure 8(a) shows the 3D shape reconstruction before fringe normalization; periodic phase errors are clearly visible. Then, the fringe normalization method was employed, and the 3D shape retrieved after fringe normalization is shown in Figure 8(b). The phase errors caused by color imbalance were suppressed, but the phase errors caused by phase-shift deviation were still existing. Thus, Wang’s  and the proposed methods were both applied to compensate these errors, and the reconstructions obtained are shown in Figures 8(c) and 8(d), respectively. Visually, both Wang’s and the proposed methods effectively suppressed the phase errors in the center of the ball. However, the frequency domain method used by Wang caused large errors in the edge areas, especially the upper edge of the ball. In order to quantify errors, we also used 18-step phase-shifting method  to measure this ball and compared the measurement result with the above four processing results. The corresponding deviation maps are shown in Figures 8(e)–8(h). The RMSE of the reconstruction before fringe normalization is 0.5682 rad, and it is reduced to 0.1255 rad after fringe normalization. As shown in Figures 8(g) and 8(h), after phase error compensation, the RMSEs of Wang’s and the proposed methods are 0.0550 and 0.0368 rad, respectively. And the maximum phase error of Wang’s method reaches 0.8 rad in edge areas which is 5.3 times that of the proposed method. In addition, Figure 9 plots the 800th cross section of the center of the ball. This experiment shows that both Wang’s method and the proposed method are reliable in processing smooth object, but the proposed method handles edge areas better than Wang’s method.
In order to further compare the proposed method with Wang’s method in dealing with object with abrupt surface, a staircase with four steps was measured. All captured patterns are shown in Figure 10. Similarly, the 18-step phase-shifting method was used as a reference. Figures 11(a)–11(c) show the 3D reconstructions recovered by 18-step phase-shifting, Wang’s, and the proposed methods, respectively. Visually, the 18-step phase-shifting method performed best while Wang’s method worked the worst. Due to the application of frequency domain method, the abrupt changes between steps were smoothed, resulting in incorrect measurement. The abrupt changes between steps reconstructed by the proposed method are obviously better than Wang’s method. Furthermore, we performed plane fitting on each step of the reconstruction retrieved by 18-step phase-shifting method. Figures 11(d)–11(f) show the deviation maps between the three reconstructions and the fitting results. Markedly, the accuracy of the proposed method is similar to that of the 18-step phase-shifting method, with only a difference of 0.01 rad. However, the RMSE of Wang’s method is approximately twice that of the proposed method. Figure 12 plots the 800th cross section of the staircase. The proposed method almost coincides with 18-step phase-shifting method, while Wang’s method performs poorly. This experiment confirms that the proposed method is more robust and accurate than Wang’s method in dealing with object with abrupt surfaces.
Two isolated sculptures were also measured to verify the actual measurement effect of the proposed method. All captured images are shown in Figure 13. Figure 14 shows the reconstructed 3D shapes, and the reconstruction in Figure 14(c) is significantly better than that in Figures 14(a) and 14(b). Figures 14(d)–14(f) show their corresponding deviation maps. Compared with the result before fringe normalization, the proposed method significantly improved the measurement accuracy, and the RMSE of the proposed method is only one-fifth of that before fringe normalization. This experiment demonstrates that the proposed method has sufficient robustness and accuracy in the actual measurement.
To further validate the dynamic measurement performance of the proposed method, one moving hand was measured. Figures 15(a)–15(d) show the 3D shapes reconstructed without color crosstalk compensation, and phase errors can be seen clearly. In contrast, the corresponding frames processed by the proposed method have no obvious errors, as shown in Figures 15(e)–15(h). This experimental result further confirms that the proposed method can effectively compensate the phase errors and improve measurement accuracy in dynamic measurement.
This paper presents an effective color crosstalk correction method for color FPP without crosstalk calibration. Due to the fact that phase error induced by color crosstalk is approximately twice the frequency of color fringe, the other sine color fringe was used to compensate the phase error. Both simulations and experiments confirmed the effectiveness of the proposed method. Experimental data shows that the proposed method has the ability to deal with objects with abrupt surfaces, and phase error caused by color crosstalk can be reduced to one-fifth after applying the proposed method.
The data used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that there is no conflict of interest regarding the publication of this paper.
X. Su and Q. Zhang, “Dynamic 3-D shape measurement method: a review,” Optics and Lasers in Engineering, vol. 48, no. 2, pp. 191–204, 2010.View at: Publisher Site | Google Scholar
S. Zhang, “High-speed 3D shape measurement with structured light methods: a review,” Optics and Lasers in Engineering, vol. 106, pp. 119–131, 2018.View at: Publisher Site | Google Scholar
S. Zhang, “Absolute phase retrieval methods for digital fringe projection profilometry: a review,” Optics and Lasers in Engineering, vol. 107, pp. 28–37, 2018.View at: Publisher Site | Google Scholar
Y. Wang, L. Liu, J. Wu, X. Chen, and Y. Wang, “Enhanced phase-coding method for three-dimensional shape measurement with half-period codeword,” Applied Optics, vol. 58, no. 27, p. 7359, 2019.View at: Publisher Site | Google Scholar
J. Deng, J. Li, H. Feng et al., “Efficient intensity-based fringe projection profilometry method resistant to global illumination,” Optics Express, vol. 28, no. 24, pp. 36346–36360, 2020.View at: Publisher Site | Google Scholar
J. Deng, J. Li, S. Ding et al., “Fringe projection decamouflaging,” Optics and Lasers in Engineering, vol. 134, article 106201, 2020.View at: Publisher Site | Google Scholar
C. Zuo, L. Huang, M. Zhang, Q. Chen, and A. Asundi, “Temporal phase unwrapping algorithms for fringe projection profilometry: a comparative review,” Optics and Lasers in Engineering, vol. 85, pp. 84–103, 2016.View at: Publisher Site | Google Scholar
M. Takeda, “Fourier fringe analysis and its application to metrology of extreme physical phenomena a review,” Applied Optics, vol. 52, pp. 20–29, 2012.View at: Google Scholar
J. Zhong and J. Weng, “Spatial carrier-fringe pattern analysis by means of wavelet transform wavelet transform profilometry,” Applied Optics, vol. 43, no. 26, pp. 4993–4998, 2004.View at: Publisher Site | Google Scholar
C. Zuo, S. Feng, L. Huang, T. Tao, W. Yin, and Q. Chen, “Phase shifting algorithms for fringe projection profilometry: a review,” Optics and Lasers in Engineering, vol. 109, pp. 23–59, 2018.View at: Publisher Site | Google Scholar
Y. Wang, L. Liu, J. Wu, X. Song, X. Chen, and Y. Wang, “Dynamic three-dimensional shape measurement with a complementary phase-coding method,” Optics and Lasers in Engineering, vol. 127, p. 105982, 2020.View at: Publisher Site | Google Scholar
L. Kinell, “Multichannel method for absolute shape measurement using projected fringes,” Optics and Lasers in Engineering, vol. 41, no. 1, pp. 57–71, 2004.View at: Publisher Site | Google Scholar
B. Li, Y. Fu, J. Zhang, H. Wu, G. Xia, and G. Jiang, “A fast three-dimensional shape measurement method based on color phase coding,” Optik, vol. 127, no. 3, pp. 1011–1015, 2016.View at: Publisher Site | Google Scholar
Y.-Y. Wan, Y.-P. Cao, C. Chen, and K. Peng, “An online triple-frequency color-encoded fringe projection profilometry for discontinuous object,” Journal of Modern Optics, vol. 63, no. 14, pp. 1420–1427, 2016.View at: Publisher Site | Google Scholar
Y. Ding, J. Xi, Y. Yu, F. Deng, and J. Cheng, “Recovering the absolute phase maps of three selected spatial-frequency fringes with multi-color channels,” Neurocomputing, vol. 252, pp. 17–23, 2017.View at: Publisher Site | Google Scholar
C. K. Yee and K. S. Yen, “Single frame profilometry with rapid phase demodulation on colour-coded fringes,” Optics Communications, vol. 397, pp. 44–50, 2017.View at: Publisher Site | Google Scholar
Y. Wan, Y. Cao, X. Liu, T. Tao, and J. Kofman, “High-frequency color-encoded fringe-projection profilometry based on geometry constraint for large depth range,” Optics Express, vol. 28, no. 9, pp. 13043–13058, 2020.View at: Publisher Site | Google Scholar
J. Wang, Y. Zhou, and Y. Yang, “Rapid 3D measurement technique for colorful objects employing RGB color light projection,” Applied Optics, vol. 59, no. 7, pp. 1907–1915, 2020.View at: Publisher Site | Google Scholar
X. Chen, C. Lu, M. Ma, X. Mao, and T. Mei, “Color-coding and phase-shift method for absolute phase measurement,” Optics Communications, vol. 298-299, pp. 54–58, 2013.View at: Publisher Site | Google Scholar
P. S. Huang, “Color phase-shifting technique for three-dimensional shape measurement,” Optical Engineering, vol. 45, no. 1, p. 013602, 2006.View at: Publisher Site | Google Scholar
S. Ma, C. Quan, C. J. Tay, R. Zhu, and B. Li, “Phase error correction for digital fringe projection profilometry,” Physics Procedia, vol. 19, pp. 227–232, 2011.View at: Publisher Site | Google Scholar
L. Song, Z. Li, C. Chen, J. Xi, Q. Guo, and X. Li, “A correction method of color projection fringes in 3D contour measurement,” Opyoelectronics Letters, vol. 11, no. 4, pp. 303–306, 2015.View at: Publisher Site | Google Scholar
P. S. Huang, Q. Hu, F. Jin, and F.-P. Chiang, “Color-encoded digital fringe projection technique for high-speed three-dimensional surface contouring,” Optical Engineering, vol. 38, no. 6, pp. 1065–1071, 1999.View at: Publisher Site | Google Scholar
Z. Zhang, “Time efficient color fringe projection system for 3D shape and color using optimum 3-frequency selection,” Optics Express, vol. 14, no. 14, pp. 6444–6455, 2006.View at: Publisher Site | Google Scholar
Y. Wan, Y. Cao, C. Chen et al., “Active phase error suppression for color phase-shifting fringe projection based on hue pre-correction,” Optics & Laser Technology, vol. 118, pp. 102–108, 2019.View at: Publisher Site | Google Scholar
Y. Hu, J. Xi, J. Chicharo, and Z. Yang, “Blind color isolation for color-channel-based fringe pattern profilometry using digital projection,” Journal of the Optical Society of America. A, vol. 24, no. 8, pp. 2372–2382, 2007.View at: Publisher Site | Google Scholar
S. Ma, R. Zhu, C. Quan, B. Li, C. J. Tay, and L. Chen, “Blind phase error suppression for color-encoded digital fringe projection profilometry,” Optics Communications, vol. 285, no. 7, pp. 1662–1668, 2012.View at: Publisher Site | Google Scholar
J. Antonio Quiroga and M. Servin, “Isotropic -dimensional fringe pattern normalization,” Optics Communications, vol. 224, no. 4-6, pp. 221–227, 2003.View at: Publisher Site | Google Scholar
B. Li, L. Chen, W. Tuya, S. Ma, and R. Zhu, “Carrier squeezing interferometry suppressing phase errors from the inaccurate phase shift,” Optics Letters, vol. 36, no. 6, pp. 996–998, 2011.View at: Publisher Site | Google Scholar
M. Wu, G. Wu, S. Zhang, Y. Wu, F. Liu, and N. Fan, “Point-to-point compensation method for color coupling and imbalance in color-fringe pattern profilometry,” Journal of Optics, vol. 22, no. 9, p. 095702, 2020.View at: Publisher Site | Google Scholar
Y. Wang, L. Liu, J. Wu, X. Chen, and Y. Wang, “Hilbert transform-based crosstalk compensation for color fringe projection profilometry,” Optics Letters, vol. 45, no. 21, pp. 6110–6113, 2020.View at: Publisher Site | Google Scholar
D. Caspi, N. Kiryati, and J. Shamir, “Range imaging with adaptive color structured light,” IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 20, no. 5, pp. 470–480, 1998.View at: Publisher Site | Google Scholar