Abstract
A novel enhancement method of global brightness modulation and local contrast enhancement combined with the improved fuzzy set theory is proposed for color image contrast enhancement. The proposed method consists of three stages. Firstly, putting forward nonlinear global brightness mapping model adjusts dynamic range of images for luminance component of color space. Secondly, membership function is established in stages to adjust local contrast of image details nonlinearly based on fuzzy set theory. Finally, the enhanced images are transformed from color space into color space. The experiments further show that the proposed method has the shortest processing time, the highest AIC values, and the least NIQE values among the other four conventional methods. It has excellent effect, which can enhance the global brightness and local contrast, and advance visibility of low illumination images.
1. Introduction
With rapid development of information science and technology, digital image processing technology is more and more widely applied in military, medical, and multimedia technology and so forth. Image enhancement technology is an indispensable technique and essential method to improve image quality and visual effect in digital image processing technology. Nowadays, there exist unsaturated, low signal-to-noise ratio and low contrast in color images, due to camera defocusing, nonuniform illumination, and atmospheric disturbances. However, low contrast is one of the most important key factors for visual effects of color images. Hence, low illumination color image enhancement research has been a hot issue in recent years. It is especially important to enhance the visual effect and highlight the image details. The purposes for enhancement are denoising and improvement of brightness and contrast; meanwhile, color information also should be maintained. In brief, the purpose is to make low illumination color images brighter and more natural.
Traditional image enhancement methods can be mainly divided into two categories: the global methods: including logarithmic transformation, Gamma correction, piecewise linear transformation, and histogram equalization [1–4]; without the brightness difference among neighborhood, these methods changed the gray levels of pixels by one-to-one mapping; when low illumination images were enhanced only by global methods, the contrast was effectively improved but the local details features were ignored easily; finally, enhanced images showed fuzzy phenomenon; the local methods: enhancing local contrast based on the local features and mainly including multiscale Retinex with color restoration and gradient domain methods [5–7]. The Retinex theory has been widely paid attention to. For example, the global dynamic ranges of original images were preliminarily adjusted with the Retinex theory, and the local characteristics were adjusted by multiscale in the method of the literature [5]; based on demands of human eyes perception, images’ local gradient field was adjusted nonlinearly in method of literature [7]. The Retinex algorithm needs to calculate different scale image for processing result and large amount of calculation. It is vulnerable to brightness image effects of the other area. In literature [8], Zhou et al. introduced an adaptive enhancement algorithm for low illumination color image, which is overall brightness and local contrast adaptive enhancement (OBLCAE). It is based on color space. However, the enhanced images by OBLCAE show halo problem at the light and shade contrast, and areas of the distortion are serious and of poor clarity. These methods can make enhanced images degenerate phenomenon: namely, bright area details are weakened; then, dark area details cannot be enhanced effectively. Recent studies [9–12] stress on the importance and necessity of having automatic methods for contrast enhancement and suggest that the Gray Level Grouping (GLG) and fuzzy-logic based methods are better suited for automatic contrast enhancement of images.
The basic objective of GLG is to achieve a uniform histogram for a low contrast color image. In GLG, histogram components in different segments of grayscale can be grouped using different criteria, so they can be redistributed differently over the grayscale to meet specific purposes; for example, certain applications may require different parts of histogram to be enhanced to different extents. The drawback of GLG is that it is not computationally efficient compared to fuzzy-based methods [13]. To sum up, it is difficult and necessary to propose an enhancement method for low illumination images.
Fuzzy rules efficiently process data by mimicking human decision-making. Jaya and Gopikakumari [14] introduced fuzzy-rule based enhancement in the Sequency by Mapped Real Transform (SMRT) domain. SMRT has been an integer transform and computationally efficient. Hence, the fuzzy set theory can analyze human behavior system, such as judgment, perception, and recognition, which had been more and more widely applied in image enhancement algorithms and achieved good results.
With development of the fuzzy set theory, it was firstly applied in image enhancement and got breakthrough results in the mid of the 1981s [15]. The method of literature [16] introduced fuzzy entropy to revise grayscale, fuzzy information, and neighborhood statistics of images based on human eyes perception. This method not only can effectively enhance contrast of the local images and improve images’ visual effect but also can enhance image edges and restrain noise. For contrast enhancement and brightness preserving, Raju and Nair [17] proposed an enhancement algorithm based on fuzzy-logic and histogram in color space.
The proposed method can naturally apperceive the global scenario content and clearly extrude the local details at the same time. On the premise of adjusting global brightness, the proposed method can enhance local contrast, restore image details, and meet the demands of human eyes perception in different visual areas with the fuzzy set theory.
In this paper, the proposed method based on color space is proposed for low illumination color image enhancement. It mainly includes three steps: the first is global brightness adaptive iterative adjustment; the second is adaptive local contrast enhancement based on the fuzzy set theory; and the last is color restoration. The experiments show that the proposed method can not only adaptively enhance global brightness but also adaptively enhance local contrast and make the enhanced image details more clear and vivid.
The rest of this letter is organized as follows. In Section 2, the theory of fuzzy-logic enhancement is introduced. In Section 3, the details of the proposed method are presented to enhance images under nonuniform and uniform illumination condition, including global brightness modulation and local contrast adaptive enhancement with improved fuzzy set theory. The experimental results are given in Section 4. Finally, a conclusion is drawn in Section 5.
2. Fuzzy Image Enhancement
Fuzzy-logic is being efficiently applied in different areas of image processing. In recent years, fuzzy algorithms for image enhancement have been developed with better performance compared to conventional and other advanced techniques like GLG. Fuzzy image enhancement is done by mapping image gray level intensities into a fuzzy plane using membership function. According the classical fuzzy enhancement methods by Pal and King [15], fuzzy processing consists of mainly three stages: image fuzzification (image coding), fuzzy enhancement, and, if necessary, image defuzzification (decoding of results).
2.1. Image Fuzzification (Image Coding)
According to (1), the image data are transformed from gray level domain to the fuzzy membership domain based on the membership functionwhere and are conversion coefficients. denotes the maximal gray value. denotes the gray level of the th pixel.
2.2. Constructing Fuzzy Membership Function
This step is the main power of fuzzy enhancement to modify the membership values. The membership values were modified into membership plane with a suitable fuzzy approach:where is defined as successive applications of , which can be expressed as
2.3. Image Defuzzification (Decoding of Results)
To achieve modified gray levels, the output of membership plane should be decoded. It means that the membership values are retransformed into the gray level plane:
According to these three steps, each gray level has been assigned to a degree of membership depending on its location in histogram. Generally, dark pixels were assigned as low membership values, and bright pixels are assigned as high membership values.
As everyone knows, in digital image processing, fuzzy set theory is mainly through membership function making original image from spatial domain to fuzzy domain; then we modify grayscale pixels in fuzzy domain. At last, inverse transformation goes back to spatial domain. The edge information is enhanced. The key is the selection of membership function in the procedure. The segmented membership function can improve speed and quality of contrast enhancement. Compared with other fuzzy enhancement methods, using segmented membership function modified the image pixels, namely, adjusting membership function based on the histogram distributions of the original images.
3. Proposed Work
This letter includes the global brightness modulation with adaptive sine function and the local contrast enhancement combined with the improved fuzzy set theory. In order to better understand the proposed method, flow chart of algorithm is shown in Figure 1. color space has higher color correlation, converting image from space to space with small color correlation. Get luminance component , and luminance component by using transformation equation (5):where , and denote three components values, respectively, of color image and are given by
3.1. Global Brightness Adaptive Modulation
The key to the global brightness modulation of the low illumination images is setting up a suitable nonlinear normalization of monotone increasing function [8, 18]. Commonly, we use nonlinear mapping functions, including logarithmic function, exponential function, S-shaped hyperbolic tangent function, and compound function. They meet constraint conditions and , , where denotes the normalized component value of low illumination image pixels. Literature [19–21] designed, respectively, the following three functions for luminance nonlinear mapping:
It has been pointed out the nonlinear function of literature [19] (7) is better than the other two functions (8) and (9), because there is no need for setting parameters to adaptively adjust the brightness for different low illumination images. This letter puts forward a new nonlinear global brightness mapping model, namely, the sine function, as shown in where denotes the luminance component of original images and denotes the luminance component of enhanced images by the global brightness modulation; parameter denotes modulation parameter. According to human eyes perception, the letter modifies sine function by setting parameter and ensures that the resulting images would not overflow. Modified sine function is very flexible mapping curve, which could influence the course of mapping curve under different values of the parameter , suitable for processing different characteristics of images, as shown in Figure 2.
From the curves diagram, it is an obvious advantage to process global brightness by function (10). With setting the parameter values, the brighter area of images will not be enhanced in excess, at the same time, the darker area will not be excessively weakened. In Figure 2, the curves are all over function (7), which has better brightness modulation ability than function (7). The image enhancement effect is better with function (10). Meanwhile, fix on the adaptive modulation parameter values of by where denotes the gray levels. As 30% or more of pixels gray levels fall below 60, the image is very dark and parameter value of is set to ; when 80% or more of pixel gray level is greater than 180, the image is bright and parameter value of is set to 1.2. In other cases, the parameter of is set to 1.4. Therefore, in view of different low illumination image, image brightness dynamic range can be adaptively adjusted through iterative calculation by function (10) and make the enhanced image more satisfying to human eyes.
3.2. Local Contrast Adaptive Enhancement Based on Fuzzy Theory Set
Through adjusting brightness, the dark area of images gets brighter, at the same time, the dynamic range of images is compressed and the image contrast declines. Based on the research of human visual system, it is sensitive to process local contrast enhancement for human eyes perception. However, the fuzzy set theory is on the basis of specific grayscale pixel features, and this letter proposes a novel local contrast adaptive enhancement method based on it.
3.2.1. Image Fuzzification
In image fuzzification, gray level intensities are transformed to fuzzy plane whose value ranges between 0 and 1. The original image of size has pixels gray level in the range of , considered as a collection of fuzzy singletons in the fuzzy set notation. The fuzzy matrix corresponding to this image can be expressed ashere is the gray level of th pixel and is its membership value, called the fuzzy property plane of the image; indicates the membership degree of relative to ; indicates dark, and indicates bright.
3.2.2. Constructing Fuzzy Membership Function
The method of constructing fuzzy membership function is proposed. We calculate histogram of original image, and indicates number of pixels gray level . Meanwhile, weighted average can be calculated from the histogram as follows:
The parameter divides the histogram into two parts based on the relationship between the image pixels and the weighted average. The first part contains pixels values in range of and the second part contains values in range of . The content of component is performed by two kinds of fuzzy membership functions and .
To calculate enhanced stretching intensity it needs to meet two rules as follows.
(1) If the Difference between and Is Larger, the Intensity of Stretching Should Be Smaller in the Part . That is to say, the pixels values closer to will be enhanced higher whereas values farther from will be extended lesser. The membership values can be used:
(2) If the Difference between and Maximum Value Is Larger, the Intensity of Stretching Should Be Larger in the Part . That is to say, the pixels values closer to will be enhanced lesser whereas values farther from will be extended higher. The membership values can be used:
In order to reduce the amount of image fuzziness, contrast intensification is applied to the fuzzy set to generate another new fuzzy set , the membership function of which is expressed as (2) and (3).
By (2) and (3), calculate the two parts, and , to obtain the new fuzzy set.
3.2.3. Image Defuzzification
Image defuzzification is the inverse of fuzzification. The method transformes the fuzzy plane back into the gray level intensities. Finally, the enhanced image can be obtained by following inverse transformation, respectively, between the parts and :
Here, enhancing control parameter decides the stretching intensity. It computed the enhanced intensity levels in the two parts. From the experimental analysis, fix value 128 for , which gave better results for the low illumination color images [16].
This enhanced information can be replaced by . Then, can be combined with the Hue and Saturation components of original color space to obtain enhanced image which is finally converted to image.
3.3. Color Space to Restore
This proposed method obtains enhanced color images by adjusting global brightness and local contrast. The color information of the enhanced image is linearly restored, and the proportion relationship of three-component is consistent. Therefore, it can better reserve the color information of the original image. The recovery method is shown as follows:where denotes enhancement factor at the th pixel; denotes the luminance component of original images, and denotes the luminance component of enhanced images; , denotes , and three-component values of the original image; , denotes , and three-component values of the enhanced image.
4. Experimental Results and Analysis
A lot of experiments are performed with the proposed method and compared with several other conventional methods such as Retinex algorithm [22], fuzzy enhancement (FE) [15], Gray Level Grouping (GLG), overall brightness, and local contrast adaptive enhancement (OBLCAE) [8]. These methods have been tested with several images under nonuniform and uniform illumination conditions. The test images used for the experiments are available on the website http://live.ece.utexas.edu/research/Quality/index.htm [23].
Test 1. Figure 3 shows processing results of the five algorithms under nonuniform illumination condition. Figures 3(a) and 3(g) show the two original images and their histograms. Figures 3(b), 3(c), 3(d), and 3(e) show processing results by Retinex algorithm, FE algorithm, GLG, and OBLCAE, respectively. Figure 3(f) is the result by the proposed method. Figure 3(b) shows the enhanced images by the Retinex algorithm which is an algorithm of the NASA research for Langley Retinex Center and has been successfully applied to the Photo Flair software in the Truview company. This algorithm needs to calculate different scale image for processing result and large amount of calculation. Because of all scales with a certain weight added to the center pixel values, it is vulnerable to brightness image effects of the other area. The resulting images by FE algorithm are not clear and overall dark, as shown in Figure 3(c). The resulting images by GLG algorithm show fuzzy phenomenon, as shown in Figure 3(d). Figure 3(e) shows the enhanced images by OBLCAE. The processing results show halo problem at the light and shade contrast, and the areas of the distortion are serious and of poor clarity. Figure 3(f) shows the enhanced images by the proposed method. It can improve image resolution and, at the same time, remove halo problems of light and shade contrast. The enhancement effects are more natural. The output histograms of proposed method achieve a smoother distribution between low and high gray level, as shown in Figure 3(h); their results are more natural looking. The histograms of the proposed method are successfully distributed evenly to its dynamic range as compared to the original images, as shown in Figure 3(g).
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
Test 2. Figure 4 shows processing results of the five algorithms under uniform illumination condition. Figures 4(a) and 4(g) show two original images and their histograms. Figures 4(b), 4(c), 4(d), and 4(e) show processing results by Retinex algorithm, FE algorithm, GLG, and OBLCAE, respectively. The Retinex algorithm did filter processing for three components of , and , respectively, and combined together, as shown in Figure 4(b). The enhanced images by FE algorithm are overall dark, losing their original color bright characteristics, as shown in Figure 4(c). From Figures 4(d) and 4(e), the processing results are excessively enhanced, so the distortion phenomena are serious. The proposed method retained the color information of original image, only to process the brightness of the images, making them more clear and natural, which have a better visual effect than the other four algorithms. Compared with the histograms distribution in Figures 4(g) and 4(h), the proposed method expanded the dynamic ranges of the gray scale distribution and then changed the contrast of the images.
(a)
(b)
(c)
(d)
(e)
(f)
(g)
(h)
Test 3. Test the algorithms’ processing time, comparing the real-time performance among the five algorithms. The CPU for test of the personal computer is Intel Core (TM) 2 Duo Q9550, whose frequency is 2.83 GHz, memory is 3.5 GB, and programming platform is Microsoft Visual C++ 6.0. There are three kinds of test images, whose resolutions are 256 pixels × 256 pixels, 512 pixels × 512 pixels, and 720 pixels × 576 pixels, respectively. Test results are shown in Table 1, and the proposed method has a good real-time performance.
A subjective assessment for the visual quality of the enhanced images is carried out. Average information contents (AIC) [24] and natural image quality evaluator (NIQE) [25, 26] index are used to assess the effectiveness of the proposed method.
The AIC is used to measure the content of an image. Higher value of the AIC indicates that more information is brought out. It is defined aswhere is the probability density function of the th gray level.
The NIQE is used for measuring image quality. It is based on construction of a “quality aware” collection of statistical feature which can be got by a simple and successful space domain natural scene statistic (NSS) model. The quality of distorted image is expressed as the distance between the quality aware NSS feature model and the multivariate Gaussian (MVG) model fit to the features extracted from the distorted image. Smaller NIQE indicates better image quality. The NIQE is defined aswhere and and and are the mean vectors and covariance matrices of the natural MVG model and the distorted image’s MVG model.
Figures 5 and 6 show the comparisons of average AIC and NIQE values between the proposed method and other conventional methods for images under nonuniform and uniform illumination conditions. Further, the qualities of the test images measured in terms of AIC and NIQE which are enhanced using the aforementioned techniques are given in Tables 2 and 3. According to Table 2, the proposed method produces the highest AIC and shows more details. Table 3 shows the natural image quality evaluator index of tested images. It is found that the proposed method produces lowest NIQE values as compared to the others. Hence, the proposed method makes enhanced images more natural. Based on qualitative and quantitative analyses, the proposed method has been found effective in enhancing contrast of images in comparison to a few existing methods. In addition, combined with global brightness modulation of adaptive sine function and the local contrast enhancement based on the improved fuzzy set theory, the proposed method makes full dynamic range of the pixel values for maximum contrast. Experimental results show that the proposed method not only outperforms all of the other methods in contrast enhancement but also provides good visual representation in visual comparison. Hence, the proposed method gives better visual quality images.
5. Conclusion
In this letter, based on human visual system, a novel enhancement method combined with the global brightness modulation and local contrast enhancement is proposed. It has effectively solved the problems about brightness and contrast of the illumination images. Aiming at the problem of brightness, the proposed method combines with nonlinear sine function for global brightness modulation, through multiple iterations, to determine the best images; at the same time, the letter proposes the local contrast enhancement based on fuzzy set theory. Finally, we compare the proposed method and the other four algorithms with two performance measure values under two kinds of conditions, nonuniform illumination and uniform illumination. Meanwhile, time of the proposed method is the shortest for different sizes of images and has a good real-time performance. The experimental results show that image contrast and visual quality by the proposed method are better than the other enhancement algorithms. Further, we optimize the algorithm, shorten the consumption, and improve the real-time performance. For future work, we plan to improve the proposed method’s robustness and apply it to images obtained from optoelectronic theodolite to improve image visibility and clarity. Meanwhile, this method is simple and suitable for consumer electronic products.
Competing Interests
The authors declare that they have no competing interests.
Acknowledgments
This work was supported by the National Natural Science Foundation of China (NSFC) (Grant no. 61405191) and the Jilin Province Science Foundation for Youths of China (Grant no. 20150520102JH).