Abstract
Thumbnail image with blurry edge information has attracted much attention in recent years. Thumbnail image, which is a tiny version of the original image, enables user to scan through a large proportion of image data for elimination of blurry image or picking up a sharp and a nice image in user’s perspective. The thumbnail image is being utilized commonly in camera, smart phone, and other computing devices. In this paper, a method to embed blurry edge information in thumbnail images is proposed. This method is straight forward and simple to be implemented in electronic products. The image will undergo edge width measurement process by finding the local maximum and local minimum locations based on its edge magnitude. This is obtained after the implementation of horizontal and vertical first order derivatives of the original high resolution input image. The blur edges will be emphasized by utilizing edge width information during downsampling process to enable users to identify blurry edge image distinctively. Experimental results show a satisfactory outcome in embedding blurry characteristics of the original image to thumbnail image on the proposed method.
1. Introduction
Photo capturing by digital camera is very convenient for most consumers, be it the process of photo taking or viewing the photo after the image is captured [1]. Today, the digital camera has become a routine tool, especially when it is integrated into the smart phone with social media applications. Furthermore, digital image becomes easily accessible through consumer electronics such as laptop and tablet. In year 2011, Americans have captured 80 billion photographs and more than 250 million photographs are being posted daily on Facebook. This number is expected to be increased each year [2]. While the camera manufacturers continue to improve photo quality and increase the resolution of the image captured by their products, the consumers are drowning in digital visual contents [3]. Thus, there is a need to determine ways to review and check the quality of the captured digital photograph.
Thumbnail images are normally being used in electronic devices, such as computer, and handheld electronic product, such as a smart phone. Thumbnail image, which is a smaller version of the original image, is normally being displayed in batches. It enables users to scroll through a bunch of images more effectively. How a blurry image is displayed by the thumbnail image is important, so that the users can conveniently filter out those unwanted blur images in a batch of thumbnails [4].
Thumbnail image enables the user to judge the original image in an effective way; whether an image quality or intrinsic value is worth keeping. Unfortunately, most of the image thumbnail technologies are unable to depict the real blur information from the original image. This drawback makes the quality judgment of an image through thumbnail image hard and not accurate. As a consequence, this situation creates inconveniences and leads to frustrations, especially when the user made a mistake in judging blurry image as a sharp, clear image [5].
Currently, there are several ways to create thumbnail images. The most common method is pixel-based downsampling with antialiasing filter (PDAF). PDAF uses low-pass filter to bandlimit the frequency components of the original image, so that aliasing problem after signal resampling process can be avoided. However, this method is not able to retain noise information very well. This is because the noise mostly occupies high frequency image components. Therefore, thumbnail image which resulted from PDAF always appears to be rather clean from the noise [6, 7].
Direct pixel-based downsampling (DPD) is a method of generating thumbnail image without the need of any filtering. In this method, direct downsampling is employed, and this means that the pixel value of the thumbnail image takes one of the corresponding pixel values from the original high resolution image. The DPD thumbnail image is sharper than PDAF thumbnail image because it contains higher frequency components [8]. Another thumbnail image method is direct subpixel-based downsampling (DSD). DSD uses a similar concept to DPD, but instead of using information from just one pixel, one thumbnail image’s pixel by DSD uses three pixel values from the original image. These three pixels correspond to the red (R), green (G), and blue (B) color channel. Therefore, DSD is only applicable for color images [9].
It is worth noting that none of the downsampling algorithms mentioned above are designed to embed the blur information from the original high resolution image to the thumbnail image. Therefore, thumbnail with blur information (TBI) is proposed in this paper. The purpose of this new algorithm is to assist the user to perceptually identify an image of good quality based on the thumbnail image. From there, the user will be able to select wanted or unwanted blur images in a bulky image folder.
This paper is arranged as follows. Section 2 describes the preliminary concept of blur detection. This useful concept is used in TBI in order to emphasize the blur. Next, Section 3 describes the processes involved in TBI in detail. Section 4 presents the experimental results where the performance of the proposal TBI method is compared with a few other well-known thumbnail image algorithms. This paper ends with Section 5 which summarizes the findings.
2. Preliminary Concept of Blur Detection
One of the concepts used in the proposed method, TBI, is presented in Figure 1. This figure presents an example to show the effect of blur towards the slope of the object’s edges in the image. Sharp images as shown by Figures 1(a) and 1(b) contain step edges. However, when the image becomes blurred, the step edges are becoming ramp edges. The degree of blurring is increasing from Figure 1(a) to Figure 1(c) to Figure 1(e). Hence, the resultant intensity profiles are generated from these three grayscale images decreasing from sharper image (i.e., Figure 1(a)) to the most blurred image (i.e., Figure 1(e)). Noting that the “merging effect” between the edges at distance 600 along the profile is getting more serious when the image is getting more blurred. This phenomenon is more prevalent when the edges are located near to each other. Figure 1 shows that the slope of the edges is depending on the degree of blurring. Therefore, the measure of blurring can be estimated by using the information from the object’s edges [10].
(a)
(b)
(c)
(d)
(e)
(f)
The proposed TBI method uses the estimated edge width to estimate the blur extent. This proposed method has been categorized into few processes, which are as follows.(i)Stage 1: Calculate horizontal and vertical derivative of the image.(ii)Stage 2: Find local maximum and local minimum of edge magnitude.(iii)Stage 3: Determine the direction of blur and calculate the edge width.(iv)Stage 4: Generate the downsampled image.
3. Thumbnail with Blur Information
Figure 2 shows the methodology of the TBI. Gradient maps are obtained after the horizontal and vertical first order derivatives are performed to the input image. The edge magnitude calculation is done using the data obtained in the first order derivative. After the edge magnitude map is obtained, edge blur direction map is generated. With the combination of the local maximum, local minimum, and edge magnitude data, the blur width map in the image can be estimated.
3.1. Horizontal and Vertical Derivative of Image
At the initial stage, the gradient, , of the input image at any pixel is calculated by applying 2D directional derivative as given by where is first order derivative in direction, is first order derivative of direction, is the original image, and are the spatial coordinates of the pixel.
In the implementation of TBI, Sobel filter [11] is used at this stage in order to find the 2D directional derivatives. Sobel filter is used to calculate the gradient magnitude at each point in each 2D image channel. Sobel operator is chosen as it is less sensitive to the noise as compared to the Roberts cross filter or Laplacian filter [10]. Thus, this gives an advantage to the TBI method because a Sobel filter will enable TBI to give more emphasis on blur issue, which is the main concern for this method. Derivative in both and directions is executed separately for each R, G, and B color channel of the original image. Equation (2) is the equations for calculating first order derivatives using Sobel horizontal and Sobel vertical derivative kernel [12] as follows: where represents a convolution process.
3.2. Edge Magnitude, Local Maximum, and Local Minimum
Edge magnitude, , is calculated using data obtained from and by Edge magnitude is needed in TBI in order to find local maximum and local minimum. To find the local maximum and minimum, we use the concept of peak and valley analysis from the edge magnitude data. The usage of the local minimum and local maximum point will be utilized in defining the edge width in the following subsection. Equation (4) summarizes how the local minimum and maximum are defined based on edge magnitude values as follows: where & is logical “AND” and and are the local minimum and local maximum map, respectively.
Notice that (4) only shows the processes in direction. A similar process to define the local maximum and local minimum in direction needs to be performed using the same concept. All the local maximum and minimum will be embedded in a new masking map, and , respectively. Local maximum, , will be used to denote the initial edge location in this method, whereas local minimum, , will be defined as the ending point. Both local maximum and local minimum will be used to estimate edge width in the next stage.
3.3. Determine the Edge Blur Direction and Edge Width
Edge blur direction that indicates the direction of the normal to the edge point with respect to the -axes is defined using atan2 function as in The data obtained from the atan2 in (5), whose values are in the range of to , is used in TBI to denote the direction of the blur edge in the width calculation process. The edge blurs direction is bidirectional, which means that . TBI uses the idea from Chung et al. [10] to determine the edge width. The process of calculating the edge width starts at a local maximum. At each local maximum point, the algorithm traverses, guided by , until it found a local minimum point on the “left side” of the edge. Then, the algorithm starts again at the same local maximum point until it found another local minimum point on the “right side” of the edge. The width of the image blur is then defined as the distance between these two local minimums.
Figure 3 shows an example of how the edge width is being determined. In this example, we use an image of size 4 × 4 pixels, where the coordinates are located at the left top corner. The value shown inside each pixel presents the gradient direction . The white box at coordinates presents the local maximum point, whereas the black boxes at coordinates and present the local minimum points. The local maximum point acts as the initial starting position for the calculation of the edge width. The explanations of this example are as follows.(a)The value of will guide the algorithm to traverse into one of the eight neighboring pixels. In this case, at position , the value of is equal to . Therefore, the algorithm moves to coordinates . Count “1” will be mapped at this location as .(b)The pixel at is not a local minimum. Thus, this pixel becomes the current pixel. At this position, the value of is equal to . The algorithm moves to coordinates . The distance between these coordinates with the starting point is 2 pixels, and thus, count “2” will be mapped at this location as .(c)The current pixel is now at coordinates . The value of at this position is . Thus, the algorithm moves to coordinates . Count “3” will be mapped at this location as . Because pixel at coordinates is a local minimum point, this indicates the termination point for the blur edge on the “right side.”(d)The algorithm needs to find the width of the blur edge located on the “left side.” Thus, the algorithm starts again at coordinates . But, by using bidirectional property, is now equal to . Therefore, it will move to coordinates . Then, at coordinates , is equal to , which leads to the local minimum point at . Thus, the traversing process terminates at this position.(e)The resultant from this process is a counting map. From this counting map, the width of the edges can be calculated. In this example, the width of the edge is equal to 3 (i.e., the width of the “right side”) + 2 (i.e., the width of the “left side”) + 1 (i.e., the local maximum point) = 6 pixels.(f)The edge width map will be generated by the algorithm by putting the edge width value to every pixel that was involved with the traversing process.
(a)
(b)
(c)
(d)
(e)
(f)
3.4. TBI Downsampling of Image
Let assume that the original image is being downsampled using a factor . Thus, the size of the downsampling window for this process is pixels. If the local edge width values contained in this downsampling window area are then the average blur edge width value is defined as
TBI downsampling process uses either the first pixel value (i.e., using the same method as DPD method) or the average intensity value, depending on the value of the local average blur width . The thumbnail image will use the average pixel value when fulfills the half factor rule. The half factor rule is a process implemented when the average width of a certain region is greater than half of the downsampling factor. This average value is obtained from an averaging process of a 2D matrix that is initiated on the image thumbnail itself, using a window of size 3 × 3 pixels. On the other hand, the first pixel value is used when the average blurs width is less or equal to half of the downsampling factor.
4. Experimental Results
In order to evaluate the performance of TBI, thumbnail images from TBI are compared with the outputs from PDAF, DPD, and DSD methods. Three test images have been used. These images are color images and in JPEG format. All images have been acquired using a phone camera, Samsung GT-I9100. The dimensions of the acquired original images are 2448 × 3264 pixels. The downsampling factor used is set to , and therefore, the thumbnail images are at size 432 × 288 pixels.
Figure 4 shows the results using one of the test images. This test image “The Boat” shows an image of a boat, captured during night time under dim light. This image appears yellowish because the scene was illuminated by yellow street lights. As shown by Figure 5(a), there is a slightly blurred region of the boat’s engine, as indicated by a red rectangle. In this figure, the results show that DPD and DSD produce sharper thumbnail images, when compared to the output from PDAF and also the proposed TBI. Yet, these methods failed to emphasize the blurs on the boat’s engine. The proposed method produces a much better thumbnail image, as it carries the blur information of the original image in the most accurate way.
(a)
(b)
(c)
(d)
(e)
(a)
(b)
(c)
(d)
(e)
Figure 5(a) shows the test image that we named as “The Boy.” The red rectangle on this image, which covers a part of a pillow, indicates the region that suffers from motion blur. From Figure 5, we can observe that DPD, DSD, and PDAF produce almost the same thumbnail images. On the other hand, the proposed TBI method successfully embeds distinctive blur information into its thumbnail image.
Figure 6(a) is a test image, which shows an image of a cat. This test image suffers from out-of-focus blur. The blur region is indicated by a red rectangle. From Figure 6, we can notice that DPD and DSD methods produce much sharper thumbnail images, as compared with PDAF and TBI. But DPD and DSD failed to give emphasis on the blur region. PDAF and the proposed method successfully embed blur characteristic into their thumbnail images. By comparing Figure 6(d) with Figure 6(e), we can see that TBI is more effective in embedding blur information.
(a)
(b)
(c)
(d)
(e)
5. Conclusion
In this paper, a new thumbnail image algorithm (TBI) has been proposed. Experimental results by using three test images suggest that the proposed method has successfully embeded blur information into thumbnail image. The complexity of TBI is low, and the method is simple. Therefore, TBI can be suggested as one of the digital display options for consumer electronics. TBI potentially can prevent the hassle in filtering away low quality or unwanted pictures.
Conflict of Interests
The authors declare that there is no conflict of interests regarding the publication of this paper.
Acknowledgments
The authors would like to thank the reviewers for their constructive comments. They also would like to thank Mr. Rostam Affendi Hamzah and Miss Kam Chai Mei for helping them in proofreading this paper. This work was supported in part by the Universiti Sains Malaysia’s Research University Individual (RUI) Grant with account no. 1001/PELECT/814169.