Abstract
In this work, we propose a structured illumination (SI) method based on a two-photon excitation (TPE) scanning laser beam. Advantages of TPE methods include optical sectioning, low phototoxicity, and robustness in the face of sample-induced scattering. We designed a novel multispot point spread function (PSF) for a fast, two-photon scanning SIM microscope. Our multispot PSF is generated with a phase retrieval algorithm. We show how to obtain the phase distribution and then simulate the effect of this distribution on a spatial light modulator (SLM), which produces the multispot PSF in the object plane of the microscope. We produce simulations that show the viability of this method. The results are simulated and a multispot PSF scanning SIM microscope is proposed.
1. Introduction
Many important features in biological microscopy are on the order of tens to hundreds of nanometers [1]. To image these tiny features, scanning electron microscopy can be used, but this is undesirable since that class of microscopes has strict requirements on sample preparation. The primary disadvantage, from a biological point of view, is that the samples are fixed. On the other hand, if a light microscope is used properly, biologists obtain in vivo imaging. When compared to electron microscopes, the main disadvantage of the light microscope is its resolving power. The wave nature of light fundamentally limits a microscope’s resolution, which is around half the wavelength of excitation light [2]. To overcome this limit, superresolution imaging (SR) has been implemented. Another problem for biological imaging is sample-induced aberration. The way to overcome this barrier is through use of adaptive optics (AO).
In the past two decades, there have been many SR microscopes invented, and this field is growing with each year. Stimulated emission depletion microscopy (STED) was invented in 1994 and works by exciting a small fluorescent area in a sample and then depleting a tight area around the centroid of excitation, confining emission light to a very small region [3]. Stochastic methods include photoactivated localization microscopy (PALM) and stochastic optical reconstruction microscopy (STORM) that operate under the principle of statistically locating fluorescent markers that are photoswitchable or photoactivatable [4, 5].
Structured illumination microscopy (SIM) is yet another SR technique [7]. Linear SIM only gives a modest superresolution factor of 2 [8] but is less invasive and faster than the other SR microscopes. SIM works by projecting illumination patterns onto the sample such that high-spatial frequency information which is beyond the microscope’s resolving power is aliased into the passband of the microscope. A convenient illumination pattern is therefore sinusoidal, because this has a simple Fourier domain representation. SIM is typically applied to wide-field imaging, so image quality is often degraded by out of focus light, aberration, and scattering. This makes imaging a densely labeled, thick sample all but impossible.
Two-photon excitation (TPE) microscopy rejects out of focus light inherently [9]. Recently, a new SIM-based method of microscopy that incorporates TPE has been implemented [8]. In this method, called scanning SIM, illumination patterns are scanned onto the object plane. In that work, the authors use a TPE method with an intensity modulation that emulates a sinusoidal pattern in the object plane. Emission light from the sample is then collected through both a photomultiplier tube (PMT) and a CCD array. An image is built up by summing individual contributions from the excitation light. The price to pay when using this technique is that it is slow when compared to conventional SIM. In conventional SIM, a typical setup requires 9 wide-field images for a single reconstruction. If an image is to be built up into pixels, then the number of PMT readings is increased by a factor of . If one were to attempt to reduce the image acquisition time for scanning SIM, the factor would be a good place to start. Faster scanning SIM utilizes wide-field imaging with a CCD array. TPE scanning SIM has been implemented in [10], with wide-field image acquisition. Later on, the same group, went on to build a TPE scanning SIM microscope which used a pinhole array coupled with a microlens array [11], a setup which allowed several excitation spots to be imaged at once.
Scanning SIM is attractive because the PSF of the microscope is contained within a small region in the object plane, which opens up the technique to adopting AO. In this study, we propose an alternative way to build up the scanning SIM illumination pattern that is both fast and suitable for AO. With the phase retrieval algorithm given in [12], we can create a multispot PSF in the object plane by using a spatial light modulator (SLM). We propose a TPE microscope since it rejects out of focus light, reduces photo bleaching and phototoxic effects, and is good for optical sectioning. We also propose a multispot PSF in order to speed up the imaging process. We simulate an illumination pattern that is suitable for a SIM reconstruction with TPE. Furthermore, since the multispot PSF is contained within a small region in the object plane, we can use adaptive optics to make local wavefront corrections within the sample. This will allow SIM to be used in deep tissue imaging.
The rest of the paper is organized as follows. In Section 2, we discuss the mathematical framework of SIM and also method of obtaining our simulations. In Section 3, we discuss the results of our simulations and the implications for using this method for scanning SIM. We conclude with Section 4, with a summary of the present work and discussion of future work.
2. Methods
2.1. SIM Framework
We begin with the framework for SIM, which follows, roughly, the work done in [7, 13, 14]. In this discussion, we will refer to three different images. The first is the image which we would like to see but are not able to, because of the low-pass filtering action of microscope objective lens. We will refer to this image as the ground truth, the sample under observation which contains unobservable, high frequency information. The image that we can observe with the microscope is called just that: the observable image. The third image is called the illumination pattern. This is the structured pattern that will be used to heterodyne the high frequency information into the passband of the microscope system. The Fourier transform (FT) of the PSF is known as the optical transfer function (OTF).
For any point in real space, , we denote the ground truth by and the observable image by . These functions are related to the OTF by the following (for brevity we have combined this equation with its Fourier dual):where denotes the convolution operator and is a point in reciprocal space.
Next, we address the illumination pattern, . If we make this illumination pattern a sinusoid, then its illumination intensity can be given bywhere is the spatial frequency vector of the illumination pattern and is a phase shift. For conventional SIM reconstructions, ranges from 1 to 3.
Another convenience of this illumination pattern is seen in its spatial frequency domain representation which is a sum of three of delta functions by Lal et al. [6]:and if we mix (or multiply) (2) with the ground truth image, then we obtain
The convolution theorem allows us to write the spectrum of asso we see that, under the illumination pattern , the spectrum of the observable image is a linear, phase-shifted combination of the ground truth. Here we note that lower spatial frequencies of already fall within the passband of the OTF, , but do not lie within the passband.
If we use different phases, then we can have a series of measurements that can be represented in matrix form asHowever, in this work, we will use only three phases, 0, , and , as this is the minimum number of known variables needed to solve for three unknown equations.
It is the second two rows of the rightmost matrix in (6), the terms with , that contain the superresolution information. By heterodyning these frequencies into the passband of the microscope, the SR information can be recovered. In the reconstruction step, we move these SR spatial frequency components, , back to their original location in reciprocal space. Since this shift operation involves overlapping regions, there may be more than one estimate for a given point in reciprocal space. A weighted average can be implemented to obtain an overall Fourier representation of the SR image. A Wiener filter is also typically used to suppress numerical instability, and artifacts may appear. In this work, we consider white additive noise applied to the raw SIM images. Additive noise is a process that is well understood and incorporated into SIM reconstruction algorithms [6, 13].
This mathematical framework demonstrates how SIM effectively increases the passband of the OTF. After recovering the enlarged Fourier domain representation of the object, we perform an inverse Fourier transform. In this way, we can recover the high-spatial frequency information that was originally outside the passband of the microscope OTF. For more information about the SIM framework, the reader is directed to [7, 14].
2.2. Simulation Framework
Conventional SIM is usually a wide-field microscopy technique. The robustness of the reconstruction depends on the fidelity of the illumination patterns (cf. (2)). That is to say, any deviation from the ideal sinusoidal illumination pattern, the spectral leakage in the frequency domain will degrade the reconstruction, since (5) will be violated. Therefore, a clean, high-contrast illumination pattern is essential for a SIM reconstruction. Another source of error in SIM reconstruction is an incorrect estimation of phase and spatial frequency of the illumination patterns. In this work, we estimate spatial frequency using Fourier domain techniques. Phase can be estimated using an autocorrelation maximization routine, as in [6].
Pattern projection onto the object can be challenging in and of itself, but when imaging deep into a sample, sample-induced aberrations blur the illumination pattern, making SIM reconstruction difficult. Use of adaptive optics (AO) can help, but wavefront correction is limited to a small area known as the isoplanatic patch, which typically does not cover the entire field of view [15]. For a typical image of a drosophila embryo, the isoplanatic patch has been found to be around 20 m [16]. Therefore, we aim to keep our PSF contained within a region of this size in the object plane.
In two-photon excitation (TPE) microscopy, the excitation light is focused into a small volume in the object plane, similar to a confocal laser scanning microscope. The theoretical PSF of a single photon confocal system is modeled by a Bessel function of the first kind [17]. The TPE system is nonlinear, and its PSF is proportional to the square of the intensity of the single photon PSF [18]. Therefore, our multispot PSF must be squared in our simulations.
We use the Gerchberg-Saxton phase retrieval algorithm from [12], implemented in Matlab. We begin by proposing a desired PSF that meets our requirements for scanning SIM. Next we perform the phase retrieval algorithm to derive a phase distribution for the SLM. We then generate a PSF from the phase distribution in order to emulate the scanning operation in software. Figure 3 shows a block diagram of the iterative loop. In a typical SIM reconstruction, there are three phases of illuminations patterns and at least two rotations. With our method, the illumination pattern can be phase-shifted by changing the scanning path with galvo mirrors. Likewise, to rotate the illumination pattern, the SLM distribution can be rotated and the beam steered accordingly.
3. Results and Discussion
To illustrate the simulation process, we propose a multispot PSF as shown in Figure 1. The proposed PSF is simply an array of five points arranged in a linear fashion. In making this proposed PSF, the hope is that the derived phase map would produce a set of points that have an even intensity distribution. In Figure 1(c) we can see that this is not the case. If we were to use this generated PSF for a scanning SIM reconstruction, the illumination pattern would be roughly sinusoidal, but with additional frequency components that make the pattern unsuitable for a SIM reconstruction. The uneven intensity distribution among the five spots is likely a result of coherent interference of the laser source. Thus, a better proposed PSF should increase the distance between the points so that coherent interference is decreased.
(a) Target PSF 1
(b) Phase solution
(c) PSF in object plane
In order to create a more even distribution among the PSF spots, some spacing should be added in between the pixels in the proposed PSF. This is what we see in Figure 2(a). We add some offset in the vertical direction in order to mitigate the effect of coherent interference. We can also add some variation in intensity to the points of the proposed PSF so that the intensities of the points in the generated PSF are more uniform. The offset in the vertical direction of the proposed PSF introduces an asymmetry, but this will not be an issue due to the scanning operation. In fact, we can illustrate this point by simulating this very operation. In Figure 4, we can see a small sample of the effective illumination pattern by scanning the generated PSF from Figure 2(c). It is apparent that this illumination pattern is very close to sinusoidal, although there is a ghosting effect due to the presence of side lobes. In practice, this effect will not be present since the illumination pattern can be made arbitrarily large by the scanning operation. We can scan an area that is larger than the region of interest and then crop the SIM images before performing the SIM reconstruction algorithm.
(a) Target PSF 2
(b) Phase solution
(c) PSF in object plane
In Figure 5, we show the 1D Fourier representation of our 2D sinusoidal pattern. If our generated pattern were truly sinusoidal, then only the peaks present at would be present. However, we note that noise is present in the areas denoted by the red ovals. In practice, we would continue to refine the multispot PSF using the phase retrieval algorithm to come up with a better PSF.
In Figure 6, we illustrate a SIM reconstruction. The test image (Figure 6(a)) is blurred with the PSF of the microscope system and with white noise at 20% of peak image intensity (Figure 6(b)). The simulation shows a robust reconstruction of the TPE scanning SIM method even when noise is added to the SIM images in the frequency domain (Figure 6(c)). The robustness is in part due to the normalizing effect of the Wiener filter that is part of the SIM reconstruction algorithm. It is also due to the apodization function that we apply to the final image. In either case, the present work has shown that the conventional SIM reconstruction algorithm can be applied to the images obtained with the TPE scanning SIM method.
(a) Ground truth
(b) Blurred image with noise
(c) Reconstruction
4. Conclusion
In this work we simulated a scanning SIM reconstruction method based on PSF shaping in the object plane. The PSF was constructed with a phase retrieval algorithm, which in turn generated a real PSF in the object plane. This generated PSF can be written into in the object plane of a microscope with a spatial light modulator. The PSF is large enough to offer an increase in the speed of scanning SIM yet small enough to be fully contained within an isoplanatic patch in an AO system. We plan on carrying out this work in a TPE microscope with adaptive optics for obtaining SIM reconstructions in thick, densely labeled samples, such as in a drosophila embryo.
Competing Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
The results presented herein were obtained at the W. M. Keck Center for Adaptive Optical Microscopy (CfAOM) at University of California, Santa Cruz. The CfAOM was made possible by the generous financial support of the W. M. Keck Foundation. This material is also based upon work supported by the National Science Foundation under Grant nos. 1353461 and 1429810. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation.