Abstract
Digital image contrast enhancement methods that are based on histogram equalization technique are still useful for the use in consumer electronic products due to their simple implementation. However, almost all the suggested enhancement methods are using global processing technique, which does not emphasize local contents. Therefore, this paper proposes a new local image contrast enhancement method, based on histogram equalization technique, which not only enhances the contrast, but also increases the sharpness of the image. Besides, this method is also able to preserve the mean brightness of the image. In order to limit the noise amplification, this newly proposed method utilizes local mean-separation, and clipped histogram bins methodologies. Based on nine test color images and the benchmark with other three histogram equalization based methods, the proposed technique shows the best overall performance.
1. Introduction
Contrast enhancement is currently still one of the active research branches in digital image processing discipline. The main aim of this research branch is to improve the appearance of the acquired image [1]. Although both digital image enhancement and image restoration share the same aim, unlike image restoration, the results obtained from digital image enhancement mostly are evaluate subjectively, where the degree of image quality perceived might be different from person to person [2]. Yet, recent researches in digital image contrast enhancement also include some objective measures in order to evaluate the performance of the algorithms.
Histogram equalization (HE) is one of the popular digital image contrast enhancement methods [3]. This method is simple and easy to be implemented. HE enhances the given input image by using a monotonic mapping function, where this mapping function is derived from the intensity histogram of the input image. The mapping function also can be derived from the probability density function (pdf), which is basically the normalized version of the histogram [4]. In order to produce an overall contrast enhancement, HE stretches the dynamic range of the image’s histogram.
In spite of its simple implementation and popularity, global HE (GHE), which is HE method that uses only one transfer function calculated from the whole pixels in the image, has several drawbacks. These drawbacks include intensity saturation artifacts and washed out appearance. Fortunately, Kim [5] has discovered that these undesired problems can be reduced by simply maintaining the average brightness level (i.e., average intensity level) of the input image into the corresponding output image. As a consequence from this important rule, many variations of GHE have been proposed.
Many of these GHE variations work by equalizing two subhistograms independently in order to preserve the mean brightness. Within this framework, the method splits the input image’s histogram into two sections, which is upper and lower subhistograms, by using one splitting point. This splitting point has normally been calculated by using the input image’s statistics, such as the average intensity value [5] or the median intensity value [6].
In order to further improve the performance of the contrast enhancement method in terms of the mean brightness preservation performance, several GHE based methods split the input image’s histogram into more than two sections. This multiple separated histogram is obtained by using more than one splitting point. These separating points are typically calculated based on each section’s average intensity values [7], median intensity values [8], and local minimum or local maximum values of histogram’s bin [9].
Yet, several other GHE based contrast enhancement methods do not put any constrain on preserving the mean brightness. These methods avoid the appearance of unwanted artifacts in their output images by controlling the enhancement rate, which relates to the slope of the intensity mapping function, by modifying the shape of the input histogram. Such modifications are normally been carried out by clipping the histogram [4], applying a weighting function to the histogram’s bins [10], or creating a complementary histogram [11].
However, GHE and its derivative methods process the image globally, which mean that these methods use only one monotonically increasing transformation function for each input image. As mentioned in [11], the calculation for this discrete transformation function is done by using discrete values. Therefore, the contrast enhancement is obtained by increasing the spaces between consecutive histogram bins. Yet, mostly the histogram bins with small values will be merged together, which lead to the deterioration of contrast for intensity levels with low probability values.
Local HE, also known as adaptive HE, is still not been fully explored. Unlike GHE based methods, local HE methods generally produce sharper images. As local HE is able to introduce new intensity levels into the output image, they may flatten the image histogram and thus increase the information content (i.e., which normally can be measured by using entropy value) [12]. One of the basic local HE methods is the block overlapped HE (BOHE). BOHE works by using a sliding window of size , which defines a local contextual region on the image [1]. This local region is treated as a subimage by BOHE, and thus a similar procedure to GHE is applied in order to determine the local intensity mapping function. However, for each windowing operation, only the center pixel of the contextual region is assigned the output value. As a consequence, for a given input image with dimensions pixels, BOHE needs MN local intensity mapping functions, which leads to high computational time.
Because the computational requirements by BOHE are very high, Pizer et al. [13] have proposed the interpolated adaptive HE (IAHE). IAHE does not use the exact local intensity mapping function to most of the pixels, but it uses the approximation of the function by utilizing bilinear interpolations. Another approximation of BOHE is the method developed by Kim et al. [14] which is called the partially overlapped subblock histogram equalization (POSHE). Unfortunately, the blocking effect reduction filter (BERF) used by POSHE is still not effective on removing some blocking effect due to the significant difference of intensity mapping functions of neighbouring subblocks.
Another approach to reduce the high complexity of BOHE is by optimizing the way of creating the local intensity histograms [12]. Yet, there is still a major problem associated with BOHE, which is the noise amplification, thus causing the existence of speckle noise in the resultant image. Therefore, a few BOHE based methods that give constrain on noise amplification, such as the contrast clipped adaptive histogram equalization (CLAHE) [15] and multiple layers block overlapped histogram equalization (MLBOHE) [12], were designed.
In this paper, a new variation to BOHE, which is called the local contrast enhancement utilizing bidirectional switching equalization of separated and clipped subhistograms (LCE-BSESCS), has been proposed. This paper is divided into four sections. After this introductory section, Section 2 will explain the methodology of LCE-BSESCS. The next section, which is Section 3, will present the experimental results. Finally, Section 4 will present the conclusion from this research work.
2. Methodology
The main objective of LCE-BSESCS is to increase the image contrast and sharpness, especially to images that suffer from uneven illumination. LCE-BSESCS also aims to preserve the mean brightness of the input image, so that intensity saturation problem can be avoided. The basic ideas of LCE-BSESCS are listed as follows.(i)Reducing saturation artefact: LCE-BSESCS uses bidirectional equalization, where the intensity mapping moves towards the local mean value.(ii)Reducing noise amplification: LCE-BSESCS utilizes clipped subhistograms in order to avoid abrupt intensity changes.(iii)Reducing the processing time: switching approach has been used, so that instead of calculating transfer functions from both subhistograms, LCE-BSESCS calculates one transfer function from one subhistogram only.The methodology for LCE-BSESCS is explained in the following subsections.
2.1. Define the Contextual Region
Similar to BOHE, the operation of LCE-BSESCS uses a sliding window of size pixels, where both and are odd integer numbers (i.e., and ). Here, and are integer values, presenting the distance from the center pixel of the sliding window to the window’s border in vertical direction and horizontal direction, respectively.
This sliding window is used to define the local image, also known as the contextual region (CR), for local processing. Let us assume as the spatial coordinates of the pixel, and represents the spatial coordinates for the center pixel of the current CR. Using the sliding window, CR will only occupy the area with coordinates in the range of and .
In the implementation of LCE-BSESCS, truncation of the sliding window has been applied for the pixels located near the image border. For example, when coordinates are at position , the corresponding CR is truncated to the size of pixels, instead of pixels. At this position, the “center pixel” of CR is not located at the middle of CR but located at the top left. By using truncation approach, the size of the original input image can be maintained and we do not need to use any padding operation (e.g., zero padding, padding with global average value, or padding with global median value) toward the input image. We believe that the sampled intensity values from the truncation approach is more accurate than the padding approach, as the sample will not be contaminated by artificially added values.
2.2. Create Local Histogram
As mentioned in previous subsection, at every “center pixel” , one CR is defined. If the input image, denoted as , has the intensity levels within the range (i.e., ), the subimage defined by CR, , also has the intensity levels . Using this CR, a local intensity histogram is defined as where is the th intensity level and is the number of pixels within CR with intensity level .
2.3. Separate Histogram into Two Subhistograms
With the aim of maintaining the local mean brightness within CR, local mean-separation methodology has been used. The mean-separation methodology was first introduced by Kim [5] in his method known as brightness preserving bi-HE (BBHE). Unlike BBHE that uses global average intensity value as the splitting point, the local splitting point in LCE-BSESCS is the average intensity value from the samples within CR. This local separating point is defined as where is the floor function.
2.4. Clip the Corresponding Subhistogram
A special consideration is given in order to limit the amplification of speckle noise which may degrade the appearance of the resultant contrast enhanced image. In LCE-BSESCS, histogram clipping approach has been utilized. By clipping histogram bins which are exceeding certain threshold value, we can control the enhancement rate defined by the local intensity mapping function. Therefore, the enhanced pixel value at will not deviate too much from its surrounding homogenous region. Adapted from bi-HE with a plateau limit (BHEPL) [16], the threshold value for the histogram clipping process is obtained by using the average value from subhistogram. However, unlike BHEPL, a switching approach has been used. The value for is defined as Where
By using this threshold value, LCE-BSESCS clips the histogram bins as follows: Next, subhistogram from the clipped histogram needs to be normalized, so that pdf of that histogram section can be obtained. In order to do this, the total number of pixels from the modified subhistogram is calculated
2.5. Create the Bidirectional Intensity Mapping Function
After the corresponding normalized clipped subhistogram has been found, intensity mapping function for the use in LCE-BSESCS is defined. Function is defined as Where
It is worthnoting that the equalization process defined by (8) propagates from the left to the right side of the subhistogram. On the other hand, equalization process given by (9) propagates in opposite direction. There are two main advantages by using bidirectional equalization. First, by using this design, the transformed intensities with high histogram values will be given values similar to the local mean value . Therefore, this condition will enable LCE-BSESCS to have a better brightness preservation ability.
The second benefit that can be obtained from bidirectional equalization is the prevention from intensity saturation artifact. In normal local histogram equalization based methods, equalization process always propagates from the left side to the right side of the histogram. Thus, if the “center pixel” has the highest intensity value within CR, the corresponding output pixel will be assigned the highest intensity level (i.e., intensity ) that normally corresponds to the whitest pixel. As a consequence, the output image might suffer from intensity saturation artifact. However, this is not a problem to LCE-BSESCS as the equalization process by this method is moving towards instead of or .
2.6. Map the Center Pixel
Finally, the output intensity value that corresponds to the “center pixel” is given as: After the value of is determined, the sliding window slides to the next adjacent pixel and repeats the same procedure until all the pixels in output image get their corresponding contrast enhanced values.
As compared with GHE based contrast enhancement methods, which use one monotonic intensity mapping function for each input image, LCE-BSESCS uses intensity mapping functions. In LCE-BSESCS, each pixel has its own transformation function. As a consequence, LCE-BSESCS is able to introduce new intensity levels into the output image. In this method, the histogram bins can be merged, as well as being broken up into several bins. Therefore, similar to other local HE methods, most of the histogram bins will be filled up with values. Hence, the utilization of the dynamic range in LCE-BSESCS is more effective than GHE based method. It is expected that LCE-BSESCS will produce a larger entropy value.
3. Experimental Results
Nine digital images of size pixels (i.e., 7.99-megapixel images) are used in this experiment in order to evaluate the performance of LCE-BSESCS. All of these images are 24-bit-per-pixel color images and were acquired using the camera from the same smartphone. This images are shown in Figure 1. In order to process these color images, in this experiment, we treat each color channel (i.e., red (R), green (G), and blue (B) color channel) as an independent 8-bit-per-pixel grayscale image. Therefore, in order to process one color image, LCE-BSESCS will be executed three times, where each execution corresponds to one color channel. Thus, in this experiment, the level of quantization is set to .
(a) Image 1
(b) Image 2
(c) Image 3
(d) Image 4
(e) Image 5
(f) Image 6
(g) Image 7
(h) Image 8
(i) Image 9
One experiment has been carried out in order to investigate the suitable filter size to be used by LCE-BSESCS. One test image, which is Image 1, has been used for this purpose. In this experiment, we has limit the shape of the filter to be square by setting to be equal to . The enhanced versions of Image 1 by using different filter sizes are shown in Figure 2. As shown by this figure, LCE-BSESCS successfully sharpens the input image. However, we can see that LCE-BSESCS with smaller filter size does not enhance much contrast as compared to LCE-BSESCS with bigger filter size.
(a) Filter size 3 × 3
(b) Filter size 5 × 5
(c) Filter size 9 × 9
(d) Filter size 17 × 17
(e) Filter size 33 × 33
(f) Filter size 65 × 65
(g) Filter size 129 × 129
(h) Filter size 257 × 257
(i) Filter size 513 × 513
In order to understand the reason for this situation, we have investigated the number of nonempty histogram bins on local histograms versus the size of LCE-BSESCS filter. The statistics from this investigation are presented as box and whisker plots in Figure 3. It is known that for local HE, the contrast of the output image is depending much on the number of nonempty histogram bins [12]. From the graph, it is shown that when the size of the filter is small, the number of nonempty histogram bins is also small, and thus majority of the local histograms have many empty bins. The number of nonempty histogram bins is increasing when the filter size increases. Based on both Figures 2 and 3, we think that LCE-BSESCS with size pixels is adequate for the enhancement. With this window size, almost half of the local histograms (i.e., based on the median value) has 128 or more nonempty histogram bins (i.e., more than intensity levels), and the output image is well enhanced.
Then, in order to bench-mark the performance of LCE-BSESCS, three other common HE based contrast enhancement methods have been implemented. These three methods are GHE [1], BOHE [12], and CLAHE [15]. In this implementations, the filter size for both BOHE and CLAHE is set to be exactly the same with the size of the filter used by LCE-BSESCS, which is pixels. The plateau limit used in CLAHE is set as from the maximum histogram bin value. Examples of the results are shown in Figures 4, 5, and 6.
(a) Image 1
(b) Enlarged version from (a)
(c) Histogram of (a)
(d) Result from GHE
(e) Enlarged version from (d)
(f) Histogram of (d)
(g) Result from BOHE
(h) Enlarged version from (g)
(i) Histogram of (g)
(j) Result from CLAHE
(k) Enlarged version from (j)
(l) Histogram of (j)
(m) Result from LCE-BSESCS
(n) Enlarged version from (m)
(o) Histogram of (m)
(a) Image 5
(b) Enlarged version from (a)
(c) Histogram of (a)
(d) Result from GHE
(e) Enlarged version from (d)
(f) Histogram of (d)
(g) Result from BOHE
(h) Enlarged version from (g)
(i) Histogram of (g)
(j) Result from CLAHE
(k) Enlarged version from (j)
(l) Histogram of (j)
(m) Result from LCE-BSESCS
(n) Enlarged version from (m)
(o) Histogram of (m)
(a) Image 9
(b) Enlarged version from (a)
(c) Histogram of (a)
(d) Result from GHE
(e) Enlarged version from (d)
(f) Histogram of (d)
(g) Result from BOHE
(h) Enlarged version from (g)
(i) Histogram of (g)
(j) Result from CLAHE
(k) Enlarged version from (j)
(l) Histogram of (j)
(m) Result from LCE-BSESCS
(n) Enlarged version from (m)
(o) Histogram of (m)
Two image quality measures has been used for the evaluation process. As the images are color images, in this research work, the measurements will be carried out on the luminance value of the image. The luminance value of the pixel at position is calculated by using the following equation [1]: where is the red color component, is the green color component, and is the blue color component.
The first measure is the mean brightness error (MBE) [5]. MBE has been widely used as a quality measure for histogram equalization based methods. MBE is defined as: where is the height of the image, is the width of the image, is the average luminance value of the enhanced image, and is the average luminance of the original input image. It is expected that a good contrast enhancement method can preserve the mean luminance and therefore has small value of MBE.
The second measure is the speckle noise strength (SNS), which is defined as follows: where is the approximation to a clean, enhanced image. In this work, is obtained from the version of that has been filtered by a median filter of size pixels. This filter size is still relatively small when compared with the size of the input image (i.e., and ) and works well even when the image has been corrupted by high density of impulsive noise. Good enhancement method is expected to give small value of SNS.
Tables 1 and 2 show the measured MBE and SNS values, respectively: The bold font in these tables refers to the best result. As shown by Table 1, the proposed method, which is LCE-BSESCS, has the lowest average MBE value. Its average MBE value is significantly lower than others. This outcome shows that LCE-BSESCS has the highest ability to maintain the mean brightness of the image. The mean brightness of the input image is well maintained in the corresponding output image by LCE-BSESCS.
Table 2 shows that all the contrast enhancement methods evaluated in this research increase the strength of the speckle noise. Yet, among these tested methods, LCE-BSESCS produces the lowest average SNS. Although CLAHE also uses clipping histogram approach similar to LCE-BSESCS, LCE-BSESCS has a better performance in SNS. This indicates that a better restriction for noise amplification can be obtained by combining the clipped histogram approach with bidirectional equalization using subhistograms.
Qualitative evaluation, by inspecting the appearance of the output images visually, has also been carried out in this research. Some of the output images are shown in Figure 4 to Figure 6. As shown by this figures, BOHE and CLAHE tend to enhance the edges too much, which make the resultant images lost their global contrast. On the other hand, images produced by LCE-BSESCS appear to be more natural. Furthermore, as also shown by these figures, LCE-BSESCS maintains the basic shape of the original histogram.
4. Conclusions
In this research work, a new contrast enhancement method has been proposed. This method is based on local HE method, where the method emphasizes on maintaining mean brightness, while improving contrast of small details on the image. The degree of enhancement can be adjusted by changing the filter size, which is the only parameter used for LCE-BSESCS. As shown by the results, the proposed method has the best performance in terms of preserving the mean brightness and avoiding significant noise amplification. Qualitative evaluation shows that the proposed method produces relatively more natural images. Therefore, it can be concluded that the proposed method shows the best overall performance.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgment
This work was supported in part by the Universiti Sains Malaysia’s Research University Individual (RUI) with Account no. 1001/PELECT/814169.