Abstract
The development of high dynamic range (HDR) display arouses the research of inverse tone mapping methods, which expand dynamic range of the low dynamic range (LDR) image to match that of HDR monitor. This paper proposed a novel physiological approach, which could avoid artifacts occurred in most existing algorithms. Inspired by the property of the human visual system (HVS), this dynamic range expansion scheme performs with a low computational complexity and a limited number of parameters and obtains high-quality HDR results. Comparisons with three recent algorithms in the literature also show that the proposed method reveals more important image details and produces less contrast loss and distortion.
1. Introduction
Recently, high dynamic range (HDR) image is a considerable topic in image processing fields [1]. The newly developed HDR monitors have greatly extended the limited dynamic range of conventional displays and can simultaneously present bright highlights and dark shadows, so they have gained significant interests in industry. At the same time, the large existing base of low dynamic range (LDR) images makes it necessary to solve how to show LDR images and videos on HDR monitors. This arouses a number of algorithms to expand LDR content to HDR content, which is called inverse tone mapping (ITM) and has been developed as reviewed by Banterle et al. [2]. Among these algorithms, Akyüz et al. [3] indicated that LDR image does not necessarily require sophisticated treatment to produce a compelling HDR experience. Simply boosting the range of an LDR image linearly to fit the HDR display can equal or even surpass the appearance of a true HDR image. This method works well under the hypothesis that the input image is high quality. Masia et al. [4] presented an exponential expansion method with as exponent, which focuses on images with large saturated regions.
The more sophisticated algorithms detect saturated areas in image and expand them by intricate processing or boost them largely. Meylan et al. [5] proposed a piecewise linear mapping function that allocates more range to those highlights in image. Banterle et al. [6] gave a general framework to map LDR content with saturated areas. The LDR content is first mapped to middle dynamic range by iTMO (inverse tone mapping operator); then, an expand map is computed to reconstruct lost luminance profiles in saturated areas of the image. Rempel et al. [7] performed a similar method; a brightness enhancement map is computed to scale the contrast which has been linearly extended to middle range.
Although the algorithms described above produce appealing results for a wide range of LDR contents, the linear expansion may not work well for images with lower quality. The exponential expansion based on transmission is unsuitable for images with low key value. Other algorithms perform sophisticated treatment to saturated areas or boost them largely. This introduces the possibility of making the image appear worse than before processing through the introduction of objectionable artifacts, and the large boosting to the bright image areas sometimes results in contouring artifacts for bright object [4].
In fact, because of the large difference between the luminance ranges of these two formats, the faithful reproduction of the HDR content from the LDR content is not possible in general. However, studies on human visual system (HVS) showed that the perceived brightness of each point in a scene is not simply determined by its absolute luminance; instead, the electric signal generated by the retina is transmitted through different layers of cells that introduce a complex, and not yet fully understood, sequence of spatial interactions, nonlinear mappings, and feedback mechanisms [8]. One useful consequence of these mechanisms is that, in order to reproduce an image, it is not necessary to generate an identical or proportional luminance on the display; by exploiting the characteristics of human vision, it is possible to process the image to amplify its dynamic range without producing a significant change in the visual sensation experienced by the observers [8].
In this paper, based on the retina response, a physiological ITM scheme is proposed, which is able to produce high-quality results with a very low computational complexity and a limited number of parameters. The main novel contribution consists in the design of the local adaptive response of retina and its inverse, which comply with the physiological perception procedure to light and minimize the formation of artifacts.
2. Proposed Method Presentation
Firstly, the proposed method deduces the local retina response and then inverses it, finally based on the inversed model to expand the dynamic range of LDR images.
2.1. Algorithm Description
Compared to the dynamic range of the real-world scene, the response of photoreceptors in retina has a narrow dynamic range, thanks to the adaptation mechanism in human vision, which makes the eyes first adapt to some luminance value and then perceive images in a rather small dynamic range around this luminance value. Thus, the basic process of human vision is a global tone mapping to the entire scene. This global function can be described by the relationship between retina response and stimulus light intensity [9] as where () is the retina response to the light intensity and is the maximum response. σ is the global adaptation level; it represents the intensity required to generate a response that is one-half the amplitude of . The parameter is a sensitivity control exponent.
It has been proven that the absolute brightness information is of secondary importance to HVS and tends to be largely discarded on very early stages of visual processing through mechanisms of brightness constancy. Local contrasts are used instead to convey the wealth of information about the scene [10]. So, the more proper description of retina response should be a local mapping. The local adaptation property can be realized by changing the global adaptation level σ to the local adaptation level of pixel . Let be the intensity of pixel in real-world scene; the response of the retina is regarded as the intensity of the LDR image perceived by eyes which can be described as where means the maximum value of the LDR output.
For inverse tone mapping, the needs to be solved from (2); the result is as
To avoid zero denominator, a small positive number δ is added to (3):
The and can be obtained from LDR image directly. The following will describe how to set parameters and .
2.2. Parameters Setting
The sensitivity parameter was discussed in the literature [9] that has a value generally between 0.7 (long test flashes) and 1.0 (short test flashes). After carrying out lots of experiments by increasing the value of gradually from 0.7 to 1.0, the results suggest that is better for the test images used in this paper.
The is local adaptation level; it describes the surrounding intensity information of a pixel. In general, the arithmetic average, the geometric average, or a Gaussian blurred version within a local region of the image can be used for determining this parameter. Here, the local surrounding intensity of pixel in the HDR image is used to represent the . With only the LDR image, based on the assumption that the maximum luminance 255 of the LDR image is mapped to the maximum luminance of the HDR display, the algorithm first computes of the LDR image and then multiples it by the ratio between the maximum luminance of HDR display and 255.
There may be various ways for computing . The bilateral filter introduced by Durand and Dorsey [11] is used here. The proposed inverse tone mapping operator is summarized as
The output of the bilateral filter for a pixel is where is a normalization factor: where is the standard deviation for a Gaussian in the spatial domain such as where is the standard deviation for a Gaussian in the luminance domain. is a normalization factor and Ω is the whole image. In the proposed algorithm, and are set empirically to 16 and 0.3, respectively.
3. Experiments and Evaluations
The proposed algorithm is implemented on a PC (i5-2520, 2.5 GHz). The maximum luminance of the HDR monitor is set to 3000 cd/m2, according to the most popular HDR monitor “Brightside’s 37.” Figure 1 shows a subset of the test images. These images represent various lighting conditions, from very dark to very bright.
In order to test and validate the performance of the proposed scheme, the other three inverse tone mapping operators are also implemented: Banterle et al.’s inverse photographic mapping (iPG) [6], LDR2HDR by Rempel et al. [7], and expansion by Masia et al. [4]. The image quality metric presented by Aydin et al. [12] is used to assess the quality of the generated HDR images. The metric generates a summary image with red, green, and blue pixels. Red pixels indicate contrast reversal (the contrast polarity is reversed in the test image with respect to the reference image), green pixels indicate loss of visible contrast (visible contrast in the reference image becomes invisible in the test image), and blue pixels indicate amplification of invisible contrast (not visible in the reference image and visible in the test image). In the experiments, the reference image is the original LDR image; the test image is the generated HDR image.
Figure 2 shows some images generated by the image quality metric [12]. By computing the percentage of red, green, and blue pixels (the ratio of the red, green, and blue pixel number to the total number of images) of these metric images, the algorithms also can be compared numerically. Table 1 displays the percentage values corresponding to images of subset in Figure 1. Because of limitations of the print medium, HDR images cannot be showed here directly. A part of tone mapped images of four algorithms is presented in Figure 3, with Reinhard et al.’s photographic tone mapping operator [13].
(a)
(b)
(c)
(d)
(a)
(b)
(c)
(d)
Several results of Masia et al.’s algorithm are unpleasant; it is due to the small key values of the LDR images which result in negative values. Table 1 indicates that the proposed algorithm has larger blue percentage and smaller red and green percentages, which means that it can disclose more details with little negligible contrast loss and reversal than the other three algorithms; this is visually testified by Figure 2. Figure 3 also shows that the proposed algorithm works well and obtains more pleasing images. In brief, the simulation results prove that, thanks to its physiology and local adaptive properties, the proposed algorithm has good performance for various light conditions or incorrectly exposed images.
4. Conclusion
This paper presented a compact ITM algorithm designed for legacy LDR images. It has low computational complexity and can obtain high quality HDR images with a few parameters compared to other recent methods. The imitation of the HVS property and the utilization of adaptive local luminance and the utilization of adaptive local luminance help the algorithm to realize two main goals of an ITM operator: preserving global image details and enhancing local contrasts. The algorithm works well for images with incorrectly exposed areas thanks to its physiological property and enhances more details with little contrast loss and reversal than the methods which largely boost or sophisticatedly deal with the saturated regions. The computational efficiency combined with the high visual quality of the results makes the proposed scheme attractive.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
This work was supported in part by the Sichuan Science and Technology Pillar Program under Grant no. 2012GZ0038 (Science and Technology Department of Sichuan Province) and by the Fundamental Research Funds for the Central Universities under Grant no. ZYGX2011J004 (UESTC).