Abstract
Pathological image enhancement is a significant topic in the field of pathological image processing. This paper proposes a high dynamic range (HDR) pathological image enhancement method based on improved bias field correction and guided image filter (GIF). Firstly, a preprocessing including stain normalization and wavelet denoising is performed for Haematoxylin and Eosin (H and E) stained pathological image. Then, an improved bias field correction model is developed to enhance the influence of light for high-frequency part in image and correct the intensity inhomogeneity and detail discontinuity of image. Next, HDR pathological image is generated based on least square method using low dynamic range (LDR) image, H and E channel images. Finally, the fine enhanced image is acquired after the detail enhancement process. Experiments with 140 pathological images demonstrate the performance advantages of our proposed method as compared with related work.
1. Introduction
Pathological image is an important basis for computer aided diagnosis and is regarded as the gold standard in disease diagnosis. Cell segmentation and identification are critical steps in various medical diagnoses; it is difficult to acquire accurate cell segmentation results because of low contrast and noise of image. To address this issue, it is necessary to enhance pathological image before cell segmentation process. The enhancement of pathological image can improve image quality and contrast, and it can provide more objective and reliable data support for doctors. This is of great significance and strong application value as it improves detection efficiency and medical diagnostic accuracy, meanwhile, reducing manual diagnosis error and human costs.
There are a variety of image enhancement methods and frameworks to improve image quality. Traditional histogram equalization- (HE-) based methods [1–3] are widely used for image enhancement owing to their simplicity. Besides, many new models and algorithms are proposed to process image. Reference [4] proposed an efficient transformation-free approach for color image enhancement, which manipulates pixels value directly in source RGB color space. Reference [5] presented a color image enhancement method using daubechies wavelet transform and HIS color space. Reference [6] enhanced the pathological anatomy images based on superresolution to improve medical diagnosis. In recent years, the Retinex theory is widely used in the medical image processing [7]. Many algorithms based on Retinex theory such as single-scale Retinex (SSR) [8], multiscale Retinex (MSR) [9], McCann99 [10], and Frackle-McCann [11] have been developed for image enhancement. There are also many other methods to enhance high dynamic range (HDR) images [12–14]. Reference [15] put forward a Bilateral Filtering-Dynamic Range Partitioning (BF-DRP) algorithm which can use bilateral filter (BF) on image to extract a coarse component and a details component which are processed independently and then recombined together to obtain final enhanced image. Reference [16] improved the BF-DRP algorithm by adding an adaptive Gaussian filter to smooth the imbalanced variation of gradient. Reference [17] proposed guided image filter (GIF) and [18] applied that in HDR image denoising and enhancement, which can avoid gradient flipping artifacts.
However, all these methods mainly target natural images or infrared images; the effect is unsatisfactory when they are used on pathological images. It remains a challenging task to obtain a good enhancement result for color pathological images because of three main issues: intensity inhomogeneity of image, detail discontinuity in tissue structures, and lower dynamic range of image. To address those issues, in this paper, we propose a HDR pathological image enhancement method based on GIF and improved bias field correction model. The main contributions of this paper are as follows. First, a new pathological image operational process is designed to denoise and enhance the source low dynamic range (LDR) image. Second, an improved bias field correction model is proposed to correct the intensity inhomogeneity and detail discontinuity of image. Third, a new method to generate HDR pathological image is presented, using LDR image and Haematoxylin (H) and Eosin (E) channel image after stain separation.
The remainder of the paper is organized as follows. In Section 2, we introduce the related work. Our proposed pathological image enhancement method is described in Section 3. Section 4 shows our experiments results and compares them with other enhancement methods. Finally, conclusions are summarized in Section 5.
2. Related Work
2.1. Bias Field Correction Model
The intensity inhomogeneity is a common phenomenon of medical images, which is attributed to many factors, such as nonuniform illumination, imaging equipment defect, and the complexity of human tissues. Intensity inhomogeneity is a critical factor that affects some image processing, because it will influence the true intensity region of different tissues and then lead to the errors of image segmentation or other image analysis processes. Therefore, it is one necessary step to remove the intensity inhomogeneity from the image. The bias field is a popular mathematical assumption of image intensity inhomogeneity, which is generally accepted at present, and it manifested as the smoothly varying of intensity within the same tissue of the image. This assumption can be represented by the following mathematical model [19]: where is the observed image with intensity inhomogeneity, is the true image, is the bias field, and is Gaussian noise with zero mean which can be ignored after denoising process and then get . There are usually two assumptions for the above model [20]:(1)The bias field is smoothly varying. That is, the bias field approximates a constant in a small neighborhood of every pixel in the observed image.(2)The true image describes the physical property of tissues and the value of this property should be the same in the same tissue. Thus we assume that the pixel value within every tissue of the true image is a constant.
The bias field correction procedure is used to remove the bias field from image and finally to obtain the corrected image . There are many bias field correction methods, and [20] proposed a new algorithm called multiplicative intrinsic component optimization (MICO) for bias field estimation, which achieved a good result. In that paper, the bias field is represented by a linear combination of a group of smooth basis functions as follows:where is a column vector valued function composed of basis functions, is the transpose operator, and is pixel point in the image, and is a column vector of the coefficients. Therefore, bias field estimation can be viewed as to find the optimal coefficients of the linear combination .
In addition, the true image is formulated as the following model:where is the constant of the th tissue and there are tissues in the image, is the percentage of the th tissue being in the pixel , there are for and for , and is the range of th tissue.
In order to calculate the bias field , that paper proposed an energy minimization function as follows:where is the whole image domain. After merging equations (2), (3), and (4), the energy function can be expressed asWe can see that the energy is the function about variables . The minimization of can be achieved by alternately solving one variable with the other two fixed. And we can finally obtain the bias field corrected image after solving the bias field estimation.
2.2. Guided Image Filter
GIF filters the input image by considering the guidance image. It is a smoothing operator which can smooth filtering, preserve the edge details, and avoid the artifacts effectively. It is fast and easy to implement and can obtain a nice visual quality. GIF is derived from a local linear transformation model considering the content of a guidance image. The filtering process at a pixel can be formulated as follows [17]:where is the guidance image and the is the linear transform of in window centered at pixel . are the local linear coefficients, the calculating formula of which is as follows:where and are the mean value and variance separately of image in window , is the number of pixels within , is the input image, and is the mean of in window . is the regularization parameter also called smooth factor and used to prevent being too large.
However, a pixel is involved in more than one window that covers when computing (2) and has different value in different windows. Therefore, we can get the final value by averaging all and thus the output of the filter can be represented as follows:Here, the average local linear coefficients are and .
3. Proposed Pathological Image Enhancement Method
As shown in Figure 1, the generic process of proposed pathological image enhancement method is introduced.
3.1. Image Preprocessing
There are always some undesirable problems such as hypochromasia, hyperchromasia, and color variations in the pathological images acquired from stained tissue due to different staining solutions of manufactures, different scanners or microscopes, and different staining protocols of labs, which can block the image observation and image interpretation [21]. To improve image quality, we use Reinhard’s stain normalization method [22] to bring the pathological image into a better color appearance of a target image selected by pathologists.
There is much additive noise in the pathological image after stain normalization; thus we adopt denoising method to smooth the image. Some denoising methods often cannot take into account both denoising and preserving image detail and lead to edge blur problem. To solve this problem, the wavelet denoising method [23] is performed, which can preserve the image edge and other feature information while denoising.
3.2. Improved Bias Field Correction Model
The traditional bias field model concerns the intensity inhomogeneity caused by bias field in the low frequency part but ignores the detail discontinuity of high-frequency part caused by different light. Therefore, we improve the bias field model through including the detail discontinuity: where is the factor of detail discontinuity.
We have the following assumptions about factor :(1) is mainly high-frequency information and only exists in the edge region of image.(2) is slowly varying in the edge region and has no significant gradient variation.
After denoising, (9) becomes . At the base of MICO algorithm [20], we use high-pass filter to remove low frequency information and then superimpose high-frequency information to enhance image details and to remove detail discontinuity.
3.3. Generate HDR Pathological Image
The dynamic range of pathological image acquired by digital image acquisition devices is very limited. Generally, the luminance range of digital image described by RGB color model is about two orders of magnitude (256) and there are always some overexposure and underexposure parts in the image. However, the dynamic range of a real scene is much wider than that of the image and the range that can be observed by human eyes is also much larger. In medical diagnosis, the dynamic range of sensors can reach 104, even 105, but LDR image cannot record all information. On the contrary, HDR image can record both RGB information and actual luminance information of pixels and can describe image details better. We could provide better images with clearer details and more reliable information for subsequent cell segmentation and identification if using LDR images to generate HDR image. It is noted that HDR images need to be compressed because they cannot be displayed on traditional screen directly, which is the tone mapping operator (TMO) of HDR image [24]. Therefore, HDR image enhancement can change the dynamic range of luminance of original image, meanwhile acquiring more details of the real scene.
We usually need many image sequences with different exposure of the same scene to generate HDR image. But in this paper, we generate HDR image sequences based on stain separation method [25]. In this case, we cannot determine the exposure time. Here one sample strategy is that we estimate the exposure time by calculating the ratio of mean values of the sequential images.
The steps of generating HDR image are as follows:(1)Separate the corrected pathology image into H channel stained by Haematoxylin and E channel stained by Eosin based on the stain separation method.(2)Estimate the ratio of exposure time of adjacent images based on their means of pixels for both corrected pathology images and H and E channel images. Then fit curve of pixel relationship based on least square method.(3)Calculate camera response function based on Mitsunaga algorithm [26] by replacing the pixel value with pixel relationship curve and then generate HDR pathological image.
3.4. Detail Enhancement Based on GIF
The enhancement method based on BF [27] can reduce the loss of image detail information, but it has gradient reverse problem. In this paper we replace BF with GIF, because GIF can avoid image gradient reverse and reduce calculation cost. In order to prove the superiority of GIF, we design five groups of images with different sizes. And there are 10 color pathological images with the same size in each group. Every image is processed with BF and GIF, respectively, and the run time is recorded successively. Finally we calculate the average run time for every group. The comparison result is shown in Table 1.
It is clear that GIF takes much less run time than BF when processing the same image. Therefore in our image enhancement method, we firstly transform the HDR pathology image into YCbCr color space (the type of sampling is 4 : 4 : 4), then divide the luminance component into base layer and detail layer by GIF rather than BF, next compress the dynamic range for base layer using histogram mapping to reduce noise and enhance detail information, meanwhile enhance the detail layer by adaptive masking, and finally combine the enhanced luminance component and color component together. The basic procedure is shown in Figure 2.
The equation of hierarchical processing for image based on GIF is shown as follows:where is the GIF, is the logarithm of luminance after transforming the HDR image into YCbCr color space, is the base layer of image including the grey information of texture region in image, and is the detail layer including high-frequency detail information such as edges. describe the coordinate of the pixel.
The result of GIF is related to the radius of filter window and the regular parameter (smooth factor) . Figure 3 shows how these two parameters affect the hierarchical results for pathology image. Figure 3(a) is the original pathological image and (b) is the Y channel image by transforming the original image into YCbCr color space. (c), (e), and (g) show the base layer with , , , , and , , respectively. (d), (f), and (h) show the detail layer of image corresponding to (c), (e), and (g), respectively. It is seen that the detail layer with larger window radius and regular parameter can obtain much more detail information.
(a) Original image
(b) Y channel image
(c) Base layer image (, )
(d) Detail layer image (, )
(e) Base layer image (, )
(f) Detail layer image (, )
(g) Base layer image (, )
(h) Detail layer image (, )
4. Experiments and Comparison
In this paper, all the experiments are implemented using Matlab R2014a development tool in the 64-bit Windows 7 operating system (8-core CPU, 3.40 GHz, 8 G memory), and our experimental data is actual clinical pathological images which are RGB color images stained by Haematoxylin and Eosin (H and E). The picture format is TIFF and spatial resolution is . The data includes 123 liver tissue images and 17 lung tissue images, that is, 140 images in total. Besides, the parameters of the GIF are , . The enhancement result of our proposed method is shown in Figure 4. Figure 4(a) is the original pathological image and Figure 4(b) is the reference image which has good stain quality and is used in the stain normalization process. Figure 4(c) is the normalized image after stain normalization. Figure 4(d) is the result after denoising and correcting by improved bias field model. Figures 4(e) and 4(f) show the E channel and H channel images after stain separation of Figure 4(d), respectively. Figure 4(g) is the HDR pathological image generated with Figures 4(c), 4(e), and 4(f), displayed by TMO. Figure 4(h) is the final detail enhanced HDR pathological image.
(a) Original image
(b) Reference image for stain normalization
(c) Normalized image
(d) Corrected image by improved bias field model
(e) E channel image
(f) H channel image
(g) HDR image after tone mapping
(h) Detail enhanced HDR image
It is observed that our proposed method can obtain a good image enhancement effect in Figure 4. This method can improve image luminance and contrast; meanwhile it can preserve the image detail structure well. In order to verify the effectiveness of our algorithm, we compare our method with various Retinex-based methods and some HE-based algorithms, namely, Frankle-McCann Retinex [11], SSR [28], MSR [29], Dualistic Sub-Image Histogram Equalization (DSIHE) [30], Minimum Mean Brightness Error Bi-Histogram Equalization (MMBEBHE) [31], Recursive Mean-Separate Histogram Equalization (RMSHE) [32], and Recursive Sub-Image Histogram Equalization (RSIHE) [33]. All these methods are applied to our 140 pathological images successively and we finally get 140 groups of enhanced images. There are 9 images in every group: one original image and 8 result images enhanced by 8 image enhancement algorithms. Figure 5 demonstrates one of the groups, where Figure 5(a) is the original image, Figure 5(b)–Figure 5(h) are the enhanced image by the above comparison algorithms, respectively, and Figure 5(i) is our result.
(a) Original image
(b) Frankle-McCann Retinex
(c) SSR
(d) MSR
(e) DSIHE
(f) MMBEBHE
(g) RMSHE
(h) RSIHE
(i) Proposed
We can see from Figure 5 that all the above methods can enhance the pathological images to some extent. But the differences between the results are a little big. To evaluate correctly the enhancement results of different algorithms, this paper analyze the corresponding images from both the subjective and objective aspects.
4.1. Subjective Evaluation
Pathological image is the gold standard in disease diagnosis. In order to verify the practicability of our algorithm, we extract 50 groups of images randomly and invite three pathologists to choose one best image from every group. The selected image should be the optimal in visual quality and the most helpful to clinical application from the perspective of a pathologist. The final result is shown in Table 2.
As Table 2 shows, the results of HE-based algorithms are basically not recognized by pathologists, because there is serious distortion problem in the enhanced image of those algorithms. Most of the detail information in those images is lost and the cell regions cannot be distinguished. Although it is possible to obtain the optimal enhancement effect for every one of the other algorithms, our proposed method is better statistically, having the maximum amount of the selected best image. So from a pathologist’s viewpoint, our pathological image enhancement method can improve the image quality a lot.
4.2. Objective Evaluation
On the other hand, we also analyze the experiment results quantitatively in five well-known metrics, namely, peak signal noise ratio (PSNR) [34], standard deviation (SD), mean, measure of enhancement (EME) [35], and information entropy [34].
PSNR is widely used to evaluate the quality of image. The higher PSNR value denotes that the image could suppress noise better. The calculation equation of PSNR is as follows:whereand where and are input image and output image, respectively, is the total number of image pixels, and is the dynamic range of pixel values. The unit of PSNR is decibels (dB).
The mean value is used to evaluate the average luminance of image. The higher mean value represents that the image is brighter. SD is also a popular metric in image enhancement and is used to estimate the contrast of image. The image contrast is grater if the SD value is higher. For mean and SD, we compute the average value of three channels of the color pathological image in our experiments.
EME is one well-known blind-reference image quality assessment (IQA) metric. It gives a quality score to each image based on the image contrast. The larger EME value represents the more detail information and more obvious variation in local region. The definition of EME is as follows:where the test image is divided into small blocks and and represent the maximum and minimum values of pixel, respectively, in block .
Information entropy is an important metric to measure the content of image. And the higher value indicates an image with richer details. The equation of entropy is as follows:where is the gray levels of image and is the probability of gray level in the whole image.
All the objective evaluation result data is shown in Table 3 and all the data is an average value of our 140 pathological images. We can see from Table 3 that all these algorithms could enhance image to some extent. The SD values of HE-based methods are much higher than other methods. But their contribution is not outstanding when taking the serious distortion problem of those enhanced images into account. In addition, the PSNR value of our algorithm is not as good as the Retinex-based algorithms, but we obtain the best results in all the other metrics of mean, EME, and entropy. And considering that we just use the original image and the enhanced image to replace the noise-free image and test image in the definition of PSNR, the contribution of PSNR is limited. So in conclusion, our method is superior to others.
Moreover, in order to indicate the complexity of our algorithm, we compare the run times of all the algorithms, running in the same platform and using the same image as described above. The final average result is shown in the last column of Table 3. It is obvious that our algorithm is much faster than Retinex-based algorithms. Although the run time of HE-based algorithm is the least, the time cost of our algorithm is worthy in view of the quality of enhanced images.
4.3. Cell Segmentation
We also use a simple automatic cell segmentation method based on morphology to verify that our image enhancement algorithm can improve subsequent image segmentation and other image analysis processes. In this cell segmentation method, firstly transform the original RGB image into gray-scale image and then conduct image opening operation, image reconstruction, image binarization, image erosion and dilation operations, and image denoising [36]. We select 20 groups of enhanced images in which the cell regions are relatively obvious to do test. Figure 6 demonstrates one group segmentation result, where Figure 6(a) is the original RGB pathological image, Figure 6(b) is the ground truth segmented manually under the guidance of pathologists, and Figure 6(c)–Figure 6(j) are obtained from segmenting the corresponding enhanced images.
(a) Original image
(b) Ground truth
(c) Frankle-McCann
(d) SSR
(e) MSR
(f) DSIHE
(g) MMBEBHE
(h) RMSHE
(i) RSIHE
(j) Proposed
It is clear that the cell segmentation result of our algorithm is more close to the ground truth. In order to compare the segmentation results quantitatively, we adopt two standard segmentation metrics to evaluate, namely, dice coefficient [37] and Jaccard index [38]. The definitions of them are as follows:where is the segmentation region, is the true region of the target, and is the number of relevant pixel points. The average results of performance metrics are displayed in Table 4. We can see that the result of our proposed method is the best. Our HDR pathological image enhancement method is superior to the comparison algorithms as we can improve the image segmentation and pathological analysis better.
In sum, we proposed that HDR pathological image enhancement method obtains a better result according to both pathologists’ subjective evaluation and quantitative analysis in data, and the cell segmentation method also proves the better performance of our method, as the quality and detail of original image are both improved.
5. Conclusions
This paper proposes new HDR pathological image enhancement methods based on GIF and improved bias field correction model. First, stain normalization and wavelet denoising operations are used in image preprocessing. And the improved bias field model is introduced to correct the intensity inhomogeneity and detail discontinuity of image. Then the HDR pathological image is generated using LDR image and H and E channel images. Next, the component of HDR image is separated into base layer and detail layer by GIF and the two layers are enhanced separately. Finally, the fine enhanced image is acquired after combining the component and the color components. To verify the effectiveness of the proposed method, we perform the enhancement experiments using 140 pathological images. The experiment results and comparisons with related work demonstrate that our proposed method improves the image quality in terms of human vision, PSNR, SD, mean, EME, information entropy, and cell segmentation.
Competing Interests
The authors declare that there were no competing interests regarding the publication of this article.
Acknowledgments
This work was supported by the National Natural Science Foundation of China (no. 61472073).