Computational Ghost Imaging with Multiplexed Time-Varying Signals
This study proposes two methods of optical watermarking based on multiplexed time-varying signals for computational ghost imaging using the Hadamard matrices. The proposed methods can realize image fusion and dual optical encryption. The time-varying signal is encoded into a specific Hadamard coefficient in advance and hidden in the light source of the transmitting end as a multiplicative factor or loaded at the receiving end as an additive factor. Theory and experiments confirm the feasibility of this scheme. Moreover, the scheme is highly scalable and has potential applications in multispectral single-pixel imaging.
Ghost imaging (GI) [1–3] is a nonlocal imaging method. In particular, the object is illuminated by structural light, and then the image is collected using a detector without a spatial resolution. Unlike conventional methods, in which a multipixel array detector (e.g., a charge-coupled device (CCD)) is used, GI calculates the spatial information of the reconstructed target by reflecting the total light intensity transmitted from or through the target under pseudorandom or structural illumination. Using the intensity correlation between the signal and reference beams, GI can nonlocally produce the image of the target. However, the optical path required for traditional GI is a two-arm system, in which one arm is the signal beam and the other arm is the reference beam. Till date, GI has been used in various frequency bands [4–8] and is particularly beneficial  at certain nonvisible wavelength ranges. Moreover, this technology has been used in the fields of microscopy , remote sensing [9, 10], three-dimensional (3D) imaging [11, 12], multispectral imaging , imaging through complex media [14, 15], tomography [16, 17], optical encryption [18–21], solving image distortion caused by uneven spatial distribution , and introduction calculation time .
To simplify the optical path and reduce acquisition time, researchers have proposed computational GI (CGI)  or single-pixel imaging [25–27], which uses spatial light modulation or digital micromirror devices (DMDs) to manually control the speckle mode while using a single-pixel detector for fast acquisition. The reference beam can be computed in CGI instead of detection by experimental measures using a CCD, thus rendering the GI system considerably simpler and more efficient. The potential of this method is that it can recover multipixel images faster than traditional scan-based methods, and it has broad application prospects in certain frequency bands. The single-pixel detector used herein provides many improvements in imaging performance. For example, it can improve the imaging efficiency, images in low light, and exhibit faster timing-response capabilities. Finally, single-pixel cameras can perform compressed sensing during the data-acquisition step, thereby reducing the pressure of storing and transmitting large amounts of data, which is an effective approach to solve high-dimensional problems.
In this article, we use a homemade digital-to-analog converter (DAC) to output a premade time-varying voltage signal. This signal drives a light emitting diode (LED) to produce a time-varying light source. We refer to this as a TV-CGI system. It multiplexes the time-varying signal from the original idle light source. When we particularly encode the time-varying signal, we can achieve single-pixel optical-image watermarking and fusion and the signal can also be used for optical-image encryption. Moreover, this system is highly scalable and can be transplanted into Fourier or multispectral single-pixel imaging, as proposed earlier.
2. TV-CGI Methodology
2.1. CGI Using the Hadamard Matrix
CGI requires a series of structured light patterns to illuminate the target and simultaneously detect the relative light intensity using a photodetector (PD). The target is then recovered by some operation using these light intensity values and corresponding illumination patterns. To date, many improvements have been made to the basic GI algorithm to recover images [28–33], such as differential GI , normalized GI (NGI) , corresponding imaging , pseudoreverse GI , sinusoidal GI , and GI using the Hadamard basis . These enhancements were mainly established to ensure that the reference beam of the optical path of the GI system effectively eliminates the influence of light source volatility using differential or normalized GI schemes. Thus, this study focuses on CGI with only one arm, and the corresponding algorithms are based on this scenario. Furthermore, the Hadamard matrix is used to create illumination patterns because the orthogonality of this matrix effectively improves the signal-to-noise ratio (SNR) and imaging efficiency of this imaging system.
Figure 1 shows a schematic of the generation of a series of mask patterns based on the Hadamard matrix. We use matrix h to represent the Hadamard basis and use the following equation to generate a series of mask patterns:where and represent the i-th column and j-th row of matrix h, respectively. We describe this physical process in the form of a matrix to facilitate the mathematical representation. We reshape all mask patterns into column vectors and form N such column vectors in the measurement matrix H. The Hadamard matrix is a type of a special orthogonal matrix that comprises only +1 and −1 elements. We use a differential scheme to overcome this obstacle, and this differential detection helps improve the SNR of this system. Based on the positive and negative values, we divide H into two matrices, and , to satisfy
Correspondingly, we use the N-dimensional row vector O to represent the imaging target, where N represents the resolution we want to restore. Then, the physical process of illuminating the target using the structured light pattern and collecting the intensity value using the PD can be expressed usingwhere the N-dimensional row vector B represents the N light intensity values measured using the PD. In fact, B also represents the Hadamard coefficient of the imaging target. Similarly, we use two row vectors and to represent the sequence of light intensities measured using and , respectively. Because of the orthogonality of the Hadamard matrix (i.e., , where represents the transpose of H and I represents the unit matrix of the corresponding order and represents a real constant), we can efficiently recover the target usingwhere represents the estimated solution of O.
2.2. Introduction of Two Types of TV Signals
Figure 2 shows a simplified schematic of the CGI system with two TV signal configurations. Here, the TV signal is loaded on the receiving or transmitting ends. The TV signal loaded at the receiving end can be regarded as an additive factor and that at the transmitting end can be regarded as a multiplicative factor. In this system, the addition of the TV signal is synchronous with single-pixel detection; thus, we use the N-dimensional row vector F to represent the average of the TV signal corresponding to the series of mask patterns over a single duration. Similar to the N-dimensional row vector B, the following equation can be obtained:
We first discuss output 1, which is obtained using the additive factor, as shown in Figure 2. It is incorporated in a way that is similar to that of the additive white Gaussian noise. We use an N-dimensional row vector to represent the value collected using the PD, which yieldswhere and are the weighting factors of both matrices. As can be seen from (6), the TV perturbation added at the receiving end causes the introduction of additive factor into the original image. Matrix F is often seen as noise in conventional CGI systems; however, if we carefully modulate F, we obtain promising results. As discussed earlier, B is actually the Hadamard coefficient of O. Similarly, F can be also regarded as the Hadamard coefficient of object . Clearly, we can use the digital-to-analog conversion circuit to make F output the Hadamard coefficient of a specific object and adjust and to coordinate the ratio of two objects to achieve image fusion or optical watermarking. Then, the following equation can be obtained:
When we load the TV signal onto the light source, it multiplies the signal acquisition. Similarly, it can be expressed using the following:
Here, o is the Hadamard product, which represents the product of the corresponding elements in the two vectors. According to and , the following can be obtained:
According to the duality of H, the following can be also obtained:where G is the N-dimensional column vector in which all elements are 1. By substituting (10) into (9), the following equation is obtained:
In (11), and are used to express the results of and , which are all real constants. The physical meaning of and is the sum of the relative information amounts of objects O and OF, respectively.
According to the orthogonality of H, (12) can be obtained:
Equation (13) shows that the multiplicative factor introduced at the transmitting end renders promising results in imaging; that is, the information of the two objects O and is weighted together. One of the weighting factors of O is the sum of the information amount of , and one of the weighting factors of is the sum of the information amount of O. In (13), and are the two additive terms; they can cause interference, so it is necessary to discuss them. Because is known, and are correspondingly unchanged. Figure 3 shows the 3D map of matrices and . We observe that they are significantly similar to the identity matrix I, and the only difference is the elements in the first column. The first column in comprises a series of negative values, and the first column in comprises a series of positive values. We can predict that the first pixel value of the two images will be contaminated, while the other pixel values will remain uncontaminated. In fact, if the first pixel values of the two images are similar, interference in the entire image can be ignored. For the convenience of representation, we use the symbol to represent the operation scheme in (13); thus,
3. Experiments and Application of the TV-CGI System
Figure 4 shows a schematic of the experimental setup of the TV-CGI system. The light emitted by the LED (Thorlabs M850L3 850 nm) is collimated using the lens (Thorlabs ACL50832U-B) to illuminate the object. The lens images the scene onto DMD (Texas Instruments DLPA008A), which is used to quickly generate a sequence of binary structured light patterns. These masks patterns pass or selectively block the scene in space. Finally, the light intensity of the corresponding mask patterns is captured using the PD (Thorlabs PDA100A2) through a converging lens. Unlike other path settings for CGI, the light source used here is adjustable. The LED is controlled by an additional modulator (Thorlabs LEDD1B) to vary the input current using an out-of-band triggering DAC (16 bit, homemade), which can be considered as a signal generator. Moreover, the signal generated by the DMD at the start of mask patterns display synchronizes the output signal of the DAC. The signal of the DAC can be hidden into the light source as a multiplicative factor at the transmitting end, or it can be placed as an additive factor at the receiving end, that is, directly connected to the DAQ (National Instruments, PCI-6220) in parallel with the signal collected using the PD. The collected composite signals are used for data processing and image reconstruction on the host computer.
The DMD comprise micromirrors. We activate the middle micromirrors as the region of interest and employ micromirrors per unit; thus, we can generate a mask pattern with a resolution of . The frame rate of the DMD is set to 1,000 frames per second, which is used to match that of the homemade DAC, and its code width can be accurately stabilized at 1 ms. The analog output voltage of DAC is adjustable from 0 to 3 V, wherein the output light of the LED exhibits good linear behavior. Although the voltage accuracy of the homemade DAC can reach 16 bits, with the introduction of noise (the measured noise is approximately 1 mV), the DAC can reach an accuracy of 11–12 bits. The sampling rate of DAQ was set to 200 kS/s, and the amplification gain of the PD was set to 20 dB. The voltage amplitude detected by the PD in the case of no light was approximately 10 mV. When the signal was generated, the amplitude of the voltage was approximately 5 V; thus, the SNR of this system can be calculated as 54 dB (, where is the amplitude of the signal and is the amplitude of the noise).
This section is divided into three parts. The first part describes the application and experimental results of the TV-CGI system in optical watermarking and image fusion. The second part describes the application and experimental results of the TV-CGI system in optical encryption of one-dimensional time signals and two-dimensional images. In the third part, we discuss the high scalability of this system for light-mode modulation. Because additive TV signals are placed at the receiving end, the concealment is not strong. It can be digitally processed, which has been discussed in many studies [34, 35]. Therefore, we mainly discuss the multiplicative TV signals hidden in the light source.
3.1. Optical Watermarking and Image Fusion
Figure 5 shows the experimental results of optical watermarking in the TV-CGI system. We calculate the Hadamard coefficients of the watermark patterns on the computer in advance and normalize them to a set of 16-bit binary numbers. These numbers are inputted to the DAC, which converts them to voltages to drive the LED. Figure 5(a) shows the imaging result () of CGI using the Hadamard matrix without a watermark. The results indicate that we can adjust the intensity of the watermark pattern in the composite image by changing the amount of information in the watermark pattern. The experimental results confirm the feasibility of this method. We change the amount of information by changing the DC component of the watermark pattern in Figure 5(b); in Figure 5(d), we change the contrast of the watermark pattern. Similarly, the watermark pattern can be removed.
To better illustrate the practicality of this scheme, we select various grayscale objects (“Zodiac” symbols) as the target of image fusion, as shown in Figure 6.
3.2. Dual Optical Encryption
The TV-CGI system can be applied to optical encryption. As shown in Figure 7, we can construct a randomly fluctuating multiplicative TV signal and hide it in the light source. The wave distribution is used to create a cipher pattern as a key. The decryption method is also relatively simple. In particular, the receiving end is synchronized by the grayscale value corresponding to the cipher pattern. Only the recipient with the cipher pattern can decrypt the correct image. The high randomness of the key provides strong security to the system. Moreover, users can use other styles of patterns as keys. Because this encryption method is placed in the light source, it is highly concealed and is theoretically compatible with previously proposed encryption schemes [22–25, 36]; thus, the proposed approach can be used to achieve dual optical encryption. It is inevitable that the decrypted image will have some introduced noise because the noise induced by the encryption process at the transmitting end and decryption process at the receiving end is multiplicatively superimposed, which can deteriorate the SNR of this process. Therefore, this encryption method can be limited to practical use in systems with high SNR.
3.3. Scalability of the System
In the first three sections of this study (from the theory to the experiments), the TV-CGI system was shown to realize the application of optical watermarking, image fusion, and optical-image encryption. In the last section, we discuss certain further applications.
In CGI or a single-pixel imaging system, the use of DMD is not the only approach of generating illumination patterns and is more suitable for quickly generating binary illumination patterns. Spatial light modulators or digital projectors can be used in this field to generate grayscale illumination patterns. The abovementioned scheme fluctuates the light source by adjusting the input voltage. When using grayscale modulation, we can modify the grayscale value of the original illumination pattern to load specific TV signals into the illumination pattern itself. However, the depth of grayscale modulation is generally 8 bits or less. This will further compromise the SNR of the imaging system. As shown in Figure 8, we performed a complementary experiment of color watermarking on two different 3D scenes using a digital projector. We calculated the Hadamard coefficients of the three channels of the watermark pattern (see Figure 8(a)) in advance and loaded them into the corresponding Hadamard-based illumination patterns. Then, we separately displayed the three RGB channel illumination patterns for three independent acquisitions to finally synthesize the three channels into one color image. The collection method in this experiment is similar to that described in Section 3. However, because the diffused light collected by the PD is weak, the gain of the PD is set at 60 dB; thus, the SNR of the system is approximately 31.5 dB ( and ). The imaging results show that we achieved the addition of a color watermark to the 3D color scene. However, we acknowledge that there is some distortion in the color that can be corrected by digital processing. It is observed that the watermark in Scene 1 is much weaker than that in Scene 2, which also agrees with the theory described in Section 2. The amount of information in Scene 2 is stronger than that in Scene 1; thus, the relative intensity of the watermark in Scene 2 is stronger than that in Scene 1 for the same watermark pattern. Moreover, the imaging result is noisy, which we believe mainly occurs because the modulation depth of the digital projector is insufficient. We believe that this strategy can be improved and has the potential to match multispectral single-pixel imaging [17, 37, 38].
4. Discussion and Conclusion
Conclusively, this study proposes a new scheme, called the TV-CGI system, which was confirmed through theory and experimentation. First, the Hadamard matrix is used as the illumination mode, and the differential detection method is used to improve the SNR and imaging efficiency of the imaging system. Second, with respect to watermarking and image fusion, the watermark intensity in the synthesized pattern can be adjusted by changing the DC component and contrast of the watermark pattern. Third, regarding encryption, the multiplicative time-varying signal is hidden in the light source, which provides high concealment. Moreover, the signal used has high randomness, which facilitates high security for the encryption, and the decryption method is relatively simple compared with other methods. This system can be applied to optical-image fusion, optical watermarking, and dual optical encryption. The proposed approach is highly scalable and has potential applications in multispectral single-pixel imaging systems.
The data used to support the findings of this study are available from the corresponding author upon request.
Conflicts of Interest
The authors declare that they have no conflicts of interest.
This study was funded by the National Natural Science Foundation of China (grant numbers 61575131, 61675138, and 61575130) and General Projects of Science and Technology Programs of Beijing Municipal Commission of Education (grant number SQKM201810028004).
B. Sun, M. P. Edgar, R. Bowman et al., “3D computational ghost imaging,” Photonics Conference, vol. 8899, Article ID 889902, 2014.View at: Google Scholar
S. M. Khamoushi, Y. Nosrati, and S. H. Tavassoli, “Sinusoidal ghost imaging,” Optics Letters, vol. 40, no. 15, pp. 3452–3455, 2015.View at: Google Scholar