Abstract
Multilevel image segmentation is demonstrated as a rapid and accurate method of quantitative analysis for nanoparticle assembly in TEM images. The procedure incorporating K-means clustering algorithm and watershed transform is tested on transmission electron microscope (TEM) images of FePt-based nanoparticles whose diameters are less than 5 nm. By solving the nanoparticle segmentation and separation problems, this unsupervised method is useful not only in the nonoverlapping case but also for agglomerated nanoparticles. Furthermore, the method exhibits scale invariance based on comparable results from images of different magnifications.
1. Introduction
The controls of size and spacing of nanoparticles are necessary in the implementation of nanoparticle assembly in devices and other developing technologies. The methods and systems that can rapidly assess dispersed nanoparticle populations are highly desirable [1]. To understand properties of nanoparticles and their characteristic nanoscale applications, transmission electron microscopy (TEM) commonly provides morphological illustrations for further quantitative analysis. In order to enhance a number of nanoparticles commonly limited by human inspections, image processing techniques have been implemented on TEM images [2]. Images of higher magnifications from atomic resolution TEM [3], scanning transmission electron microscopy (STEM) [4], and atomic force microscopy (AFM) [5] have also successfully been analyzed. The majority of the analysis segments the images into foreground and background pixels. The most frequently reported techniques used for nanoscale applications are based on traditional image processing approaches that utilize boundary and edge information. Some existing methods rely on the well-known Otsu’s thresholding method [6] or edge detection [7]. Such techniques are suitable for high contrast images but less effective in the case of agglomerated nanoparticles without clear boundaries [8].
In order to resolve the aggregates, some methods have been studied using model-based reconstruction [9–11]. These methods employ a deformable shape template to map the intensity profile in the vertical or spatial directions. However, they rely on predefined shape templates such as spherical or elliptical templates and are often incapable of characterizing overlapping nanoparticles [10, 11]. In addition, a recently proposed approach segments Au nanoparticles which are partially overlapping [12]. The morphological measurement discussed in this work depends almost exclusively on an initial thresholding. Any form of particle measurement must then assume that the particles existing in the “foreground” are of the same type, while in reality, they may belong to overlapping types with different intensity profiles or shapes.
Presently, multilevel considerations have received little attention in the field. In the current work, we draw motivation to adopt an unsupervised method for the automatic implementation of multilevel segmentation by mining natural homogeneities from the data. Compared to traditional image segmentation approaches based on predefined thresholding [2–7, 12], the current work adopts an unsupervised method [13, 14] for multilevel segmentation, allowing a more accurate analysis of foreground regions. This allows for possible successive operations to enhance the capability of segmentation in the case of agglomerated nanoparticles. In this work, iron-platinum- (FePt-) based nanoparticle is selected as a test case to prove this issue. Ferromagnetic FePt nanoparticles are under development for novel permanent magnets and bit-patterned media due to their high magnetic anisotropy and chemical stability [15, 16]. In addition, superparamagnetic nanoparticles have been applied in biomedical science thanks to their comparable size to those of biological counterparts. Magnetic fields are used to manipulate these nanoparticles in magnetic separation, hyperthermia treatment, drug delivery, and magnetic resonance imaging (MRI) contrast enhancement [15, 17]. Their size and spatial distributions dictate the magnetic properties. The long-range order of self-assembled nanoparticles with rather uniform size and spacing can be implemented in devices if the quantitative analysis of nanoscale images is obtained. This proposed multilevel segmentation is particularly useful in the case of varying degrees of particle overlapping, like magnetic nanoparticles which exhibit both self-assembly behavior and tendency to agglomerate. The procedure is also applied on TEM images with different magnifications to examine the scale invariant feature.
2. Materials and Methods
2.1. Nanoparticles and TEM Imaging
FePt-based nanoparticles, referred to as Samples A–D, were, respectively, synthesized by using Fe(tmhd)3, Fe(dbm)3, Fe(hfac)3, and Fe(acac)3 as Fe sources. The synthesis was detailed and the effects of starting materials on the structural and magnetic properties of these self-assembled nanoparticles were discussed in the previous work [18]. Images of nanoparticles with a resolution of 512 × 533 pixels were taken by TEM (FEI model Tecnai G2 20) operating at an accelerating voltage of 200 kV. The nanoparticles were dried on carbon-coated copper grids overnight at room temperature before the imaging.
2.2. Multilevel Image Segmentation
The grayscale of TEM images often fluctuates greatly in intensity making it difficult to segment the particles from the background using the thresholding. In the traditional edge detector, the edge of the particles can be detected by careful selection of threshold values. The automatic selection of threshold values for the Canny edge detector is incapable of obtaining edge information in this case, as demonstrated in Figure 1(a). The image can be segmented into a binary image using Otsu’s method, followed by the Canny edge detection. The result shows some improvement for the detection of the particle, as shown in Figure 1(b). However, the boundary of the particle was not precisely determined. Alternatively, unsupervised learning methods can partition datasets into disjoint groups [19] and we adopt the -means clustering algorithm described in the literature [13, 14] for this TEM image analysis. The results obtained by -means are shown in Figures 1(c)-1(d). All image pixels are classified into classes, so that pixels with similar gray level are quantized into the same clusters. Depending on the quality of the centroids generated, the particle is clearly highlighted in the quantized image in Figure 1(c) and its boundary can be distinguished from the background. The important characteristic of a segmented image is the highlighting of its foreground region according to different levels (blue and dark blue color in Figure 1(c)). This information is used for particle separation in the case of overlapping particles.
(a)
(b)
(c)
(d)
In order to separate overlapping particles, we labeled the foreground region as base image and applied the well-known watershed transform [20, 21] to the particles within the base image. However, the direct application of watershed transform usually results in oversegmentation. To overcome this problem, we formed a top image from the quantized pixels within the foreground regions, by using the output of the -means. The top image was defined such that it contained pixels whose gray values were darker than the pixels around the border of the foreground region. Figure 2 shows the extraction of top and base images from the output of -means and then the application of watershed transform in two channels. The representation of the top image contains small regions with emphasis on the overlapping particles and the center of the individual particles. These regions are considered the depth area, according to the principle of the watered transform. Thus, as shown by the first channel in Figure 2, the application of distance transform followed by the watershed transform to the top image can avoid oversegmentation. The region representing the center of the particle may not be separated since the distances from the region center to each point of the region’s boundary are quite similar. In comparison, the regions representing the overlapping particles have asymmetry shape, and thus, the distances from their center to the boundary have a high standard deviation. Thus, the output of the distance transformation will guide the watershed transform to separate these regions, as shown by the regions around the center of the top image in Figure 2.
In comparison, as shown by the result from the second channel, the watershed transform of the base image results in oversegmentation. At the final stage, the outputs of the two segmentation channels are combined, in such a way that the segmented regions from the base image are merged if their boundaries pass through the same region of the top image. This region merging technique assumes that the regions of the top image are showing the area around the center of the particles and not the boundaries. The union operator combines the oversegmented regions of the base image to produce a more meaningful region as shown in Figure 2. In the experiment, we have observed that the performance of the multilevel segmentation was depending on the value of . We found that was suitable for our experimental image data. In addition, the top image was suitably obtained by 2-3 levels of quantized gray scales at the dark side.
3. Results and Discussion
In the preprocessing stage, the grayscale input images were passed through a histogram enhancement step to reduce the effects of different lighting conditions at the background. Then, the opening and closing operators [22] with structural elements of size 2 × 2 pixels were applied as low-pass filters. In the image segmentation, the number of class was selected from the numbers in the range [2, 23], according to the characteristics of the gray levels of the input images, in order to preserve the resolution of the segmented particles. The requirement of predetermination of is a limitation of the -means algorithm, whereas other unsupervised methods can obtain the number of classes automatically [19]. The watershed transform was applied to the top and the base images for separation of overlapping particles, as discussed in Figure 2.
For the original Image A1 shown in Figure 3(a), its segmentation result is shown in Figure 3(b) and Table 1 summarises the statistics of nanoparticle populations. As the particles detected at the border of each image were truncated by the image acquisition system, they were not included in these results. With the image processing method, the multilevel segmentation method can accurately detect all particles in Image A1 where the particles do not overlap. The method can separate the particles without oversegmentation. These 99 particles yield an average diameter of nm.
(a)
(b)
The scale invariant feature of the measurement method was also studied by applying the method to images of Sample A taken at lower magnifications. Images A2 and A3, respectively, shown in Figures 4(a) and 5(a), provide wider views of nanoparticle assembly with sampling size over one hundred. Figures 4(b) and 5(b) show the results after applying the multilevel segmentation. Their computed diameter and area are compared in Table 1. Evidently, the proposed method offers high accuracy measurement of particle dimensions with consistent results obtained from Images A1–A3. Regardless of resolution, the diameter and standard deviation are similar with only slight differences at the first decimal. The proposed method is thus highly desirable since, in practice, the array of nanoparticles in material research should be examined at different spatial locations and resolutions.
(a)
(b)
(a)
(b)
Images of three other samples with particle agglomeration are analyzed. From visual inspections, Image B3 (Figure 6) obviously differs from Image A3 by containing densely packed particles. The method produces some over-segmentation when the particles have a higher degree of overlapping. However, most agglomerations can be separated resulting in 1598 particles detected. The numbers of detection are reduced in Images C3 (Figure 7) and D3 (Figure 8). With the exception of some agglomerated particles in the former, the distances between the particles were comparable to those in Images A3 and the procedure. By contrast, nanoparticles tend to cluster in certain positions leaving large voids on the substrate in Image D. As a result, some large agglomerated particles could not be separated and the standard deviation in size determination becomes the largest.
(a)
(b)
(a)
(b)
(a)
(b)
Since agglomerated particles shown in Images B3 and D3 are difficult to separate through visual inspection, the image processing can replace the subjective with tedious inspections. With a large number of particles taken into account, the accuracy is greatly improved from the previous analysis based on sampling size of 50 particles by Image J program [18]. Whereas each average diameter obtained from Image J compared in Table 1 is larger, both methods nevertheless give rise to the same size ranking of nanoparticles in Images A3–D3. Overall, the multilevel segmentation is effective in nanoparticle counting and capable of handling over 1000 agglomerated particles in images with substantial overlapping.
4. Conclusions
Multilevel segmentation procedure was implemented in a rapid and accurate assessment of nanoparticle assembly. To demonstrate its performance, TEM images of FePt-based nanoparticles with varying magnifications and degrees of overlapping were tested. The original images required separations of the particles from the substrate background and this was accomplished by the multilevel segmentation. By applying the -means clustering algorithm and watershed transform, average particle size can be determined. The proposed method offered the scale invariant feature applied to images with different magnifications, yielding comparable average size. Furthermore, this image processing method successfully characterized agglomerated nanoparticles in the TEM images.
Conflict of Interests
The authors certify that there is no conflict of interests with any financial organization regarding the material discussed in this paper.
Acknowledgments
The authors would like to thank Dr. Komkrich Chokprasombat, Assistant Professor Dr. Pimphaka Harding, and Professor Dr. Thomas Randall Lee for their contributions and suggestions regarding the synthesis of nanoparticles. TEM images were taken with the assistance of Pikul Ratphonsan at the Khon Kaen University.