Color-Texture-Based Image Retrieval System Using Gaussian Markov Random Field Model

Abstract

The techniques of 𝐾-means algorithm and Gaussian Markov random field model are integrated to provide a Gaussian Markov random field model (GMRFM) feature which can describe the texture information of different pixel colors in an image. Based on this feature, an image retrieval method is also provided to seek the database images most similar to a given query image. In this paper, a genetic-based parameter detector is presented to decide the fittest parameters used by the proposed image retrieval method, as well. The experimental results manifested that the image retrieval method is insensitive to the rotation, translation, distortion, noise, scale, hue, light, and contrast variations, especially distortion, hue, and contrast variations.

1. Introduction

Much attention has been devoted to the design of image databases over the past few years [1–8]. Image retrieval is an important task in many image database applications, such as office automation, medical image archiving, digital library, multimedia publishing, computer-aided design, and geographic information systems.

Most traditional and common methods of image retrieval utilize some method of adding metadata such as captioning, keywords, or descriptions to the images so that retrieval can be performed over the annotation words [4]. As these systems are built upon a large database, the textual-feature-based method becomes not only cumbersome but also inadequate to represent image contents. Hence, many content-based image retrieval (CBIR) systems have been proposed in the related literatures [1–8]. The CBIR aims at avoiding the use of textual descriptions and instead retrieves images based on their visual similarity to a user-supplied query image or user-specified image features.

Low-level features, such as colors [9, 10], shapes [7, 8], and textures [11, 12], are extensively used to index the image features of the CBIR systems. The shape-based image retrieval system searches for images containing the objects similar to the objects specified by a query image. However, locating and recognizing objects from images is a real challenge. One of the difficulties is to separate the objects from background. These difficulties come from discretization, occlusions, poor contrast, viewing conditions, noise, complicated objects, complicated background, and so forth. Moreover, the shape-based image retrieval method can deal with only the images with simple object shapes efficiently. For complex object shapes, the region-based method has to build a binary sequence by using smaller grid cells so that more accurate results can be obtained; nevertheless, the storage of indices and retrieval time may increase tremendously.

The color and texture attributes have been very successfully used in retrieving images with similar feature distributions. The color-based image retrieval method, such as the conventional color histogram (CCH) [13] and the fuzzy color histogram (FCH) [14], measures the similarity of two images with their distance in color space. The extraction of the color features follows a similar progression in each of the four methods: (1) selection of the color space, (2) quantization of the color space, (3) extraction of the color feature, (4) derivation of an appropriate distance function [13]. Color attribute may avoid object identification and extraction in image retrieval. Color may provide multiple measurements at a single pixel of the image, and often enable the classification to be done without complex object segmentation. Vu et al. [7] proposed a color-based image retrieval method, called a SamMatch method, to process the query by sampling-based matching approach. The SamMatch method has the benefits of region-based techniques without reliance on the highly variable accuracy of segmentation methods. However, this method requires the user to point out the object areas at the time of the query.

Unfortunately, color-based image retrieval systems often fail to retrieve the images taken from the same scene but under different time or conditions, for example, the images of a countryside taken at dawn, dusk, or noon under a clear or a cloudy sky. In another scenario where a same scene is imaged by different devices, using one image taken by one device as the query example may fail to find the same scene taken by other devices.

Texture attribute depicts the “surface” of an object. Intuitively, this term refers to properties such as smoothness, coarseness, and regularity of an object. Generally, the structural homogeneity does not come from the presence of a single color or intensity, but requires the interaction of various intensities within a region. Texture similarity is often useful in distinguishing objects with similar colors, such as sky and sea as well as leaves and grass. Making texture analysis is a real challenge. One of the ways to perform content-based image retrieval using texture as the cue is to segment an image into a number of different texture regions and perform a texture analysis on each region. However, segmentation can sometimes be problematic for image retrieval. In addition, texture is quite difficult to describe and subject to the difference of human perception. No satisfactory quantitative definition of texture is available at this time.

The notion of texture appears to depend upon three ingredients [15]. First, some local “order” is repeated over a region which is large in comparison to the order's size. Second, the order consists in the nonrandom arrangement of elementary parts. Third, there are roughly uniform entities having approximately the same dimension everywhere within the textured region. Liu and Picard [16], and Niblack et al. [17, 18] used contrast, coarseness and directionality models to achieve texture classification and recognition. Huang and Dai [19] proposed a texture-based image retrieval method. The method associates one coarse and one fine feature descriptors with each image. Both descriptors are derived from the coefficients of wavelet transform of the original image. The coarse feature descriptor is used at the first stage to quickly screen out nonpromising images; the fine feature descriptor is subsequently employed to find the truly matched images. However, Huang’s method cannot give a high accuracy rate of precisely seeking the desired image.

To solve the problems mentioned above, this paper proposes a color-texture-based image retrieval system (CTBIR method). The CTBIR method integrates the color and texture attributes of an image. It employs 𝐾-means algorithm [20] to classify the colors in all database images into clusters, then calculates the average of the pixel colors in each cluster, and uses this average color to represent the colors of the pixels in the cluster. Next, the CTBIR method utilizes Gaussian Markov random field model (GMRFM) to analyze the texture of each representative color in an image. A genetic algorithm is also provided to decide the most suitable parameters used in the CTBIR method. Besides, the performance of the CTBIR method is investigated by experiments. The effects of various image variations, such as rotation, translation, distortion, noise, scale, light, hue, and contrast on the performance of the CTBIR method are explored as well.

2. Related Works

This section will give a brief introduction of Gaussian Markov Random Field Model, genetic algorithm, and other techniques to be used in the CTBIR method.

2.1. Gaussian Markov Random Field Model

Texture plays a very significant role in analyzing image characteristics. Chellappa and Chatterjee [11] adopted GMRFM to describe the relationship between each pixel and its surrounding pixels. Generally, there exist some similarities and correlations in gray-level variance between adjacent pixels. In GMRFM, the varying degree of texture in images is statistically calculated mainly on the basis of the property. While Chellappa and Chatterjee used GMRFM to describe the textures of objects on an gray-scale image and segment the objects from the image, in this paper it will be used to characterize the textures of images in order to retrieve the images from a database which are most similar to the given query image.

In GMRFM [12, 21–23], all the pixels in each 𝑚_𝐺×𝑚_𝐺 block 𝐵 are classified according to their relationship with the central pixel 𝑥_0,0. Figure 1 illustrates an example of GMRFM where 𝑚_𝐺=5 and (𝑖,𝑗) indicates the neighboring pixels 𝑥_𝑖,𝑗 of 𝑥_0,0, where 𝑥_𝑖,𝑗 is the pixel belonging to the jth category in the ith rank relative to 𝑥_0,0. A set of eigenvalues 𝜃_𝑖,𝑗’s are employed to describe the relationship between each 𝑥_𝑖,𝑗 and 𝑥_0,0 where 𝑥_0,0 is calculated as [11] 𝑥_0,0=_{(𝑖,𝑗)∈𝑁𝑥0,0}𝜃_𝑖,𝑗𝑥_𝑖,𝑗+𝑒. (2.1) In this formula, 𝑒 indicates a Gaussian distribution of noises. 𝑁_𝑥0,0 denotes the neighbors of 𝑥_0,0 shown in Figure 1. Let 𝑋 and 𝜃 be the column vectors composed, respectively, of 𝑥_𝑖,𝑗 and 𝜃_𝑖,𝑗 excluding 𝑥_0,0and 𝜃_0,0. Then, one can calculate 𝜃 as follows:𝜃=⌊𝑋𝑋^𝑇⌋⁻¹⌊𝑥_0,0𝑋⌋, (2.2) which can be used to depict the texture of 𝐵. The larger most of the values in vector 𝜃 is, the rougher 𝐵 is; otherwise, 𝐵 is smoother. Yue [8] also uses 𝜃 to describe the texture of a telemeter image of glacier and divides the image into some regions according to their textures.

Figure 1 

The neighborhood of 𝑥0,0 with a block of 5×5 pixels.

2.2. Genetic Algorithm

Seo [24] applied a genetic algorithm for selection and feature combination in pattern recognition applications. Ballerini et al. [25] used a genetic algorithm to select a subset of texture features from all the features extracted. Lai and Chen [26] used the interactive genetic algorithm to tune human judgment results on similarity of images. Joshi and Tapaswi [2] applied a genetic algorithm to decide the most plausible matching between two images in image retrieval. Stejić et al. [6] proposed an image retrieval system using local similarity patterns. The system used a genetic algorithm to find an optimal assignment of similarity criteria to image regions. Chan and King [1] provided a trademark image retrieval system which employed a genetic algorithm to find the weighting factors in the dissimilarity function by integrating five shape features. This paper will adopt a genetic algorithm to decide the fittest parameters which will be used by the CTBIR method.

Genetic algorithm (GA) [27] is a heuristic optimization method that operates through a determined and randomized search. The set of possible solutions for an optimization problem is considered as a population of individuals, and the degree of adaptation of an individual to its environment is specified by its fitness. A chromosome, essentially a set of character strings, represents the coordinate of an individual in the search space. A gene is a subsection of a chromosome that encodes the value of a single parameter that is being optimized. Typical encoding for a gene can be binary or integer.

Derived from the biological model of evolution, genetic algorithms operate on the Darwinian principle of natural selection, which holds that, given a certain population, only the individuals that adapt well to their environment can survive and transmit their characteristics to their descendants. A genetic algorithm consists of three major operations: selection, crossover, and mutation. Selection evaluates all individuals, and only those best fitted to their environment survive. Crossover recombines the genetic material of two individuals to form new combinations with the potential for better performance. Mutation induces changes in a small number of chromosomal units with the goal of maintaining sufficient population diversity during the optimization process.

2.3. System Evaluation

Whenever a certain image 𝑓_𝑞 is selected as a query image, the image retrieval system delivers the user 𝑅 database images whose matching distances to 𝑓_𝑞 are shortest. If the desired database image is one of these 𝑅 transmitted images, we say the system correctly finds out the desired image [9, 10]. Otherwise, the system fails to respond to the desired image. In the following experiments, the accuracy rate of querying for a system will be explained with ACC.

The other evaluation, normalized modified retrieval rank (NMRR), proposed by MPEG-7 [28] is usually used as a benchmark for system evaluation. NMRR not only indicates how many of the correct items are retrieved, but also how highly they are ranked among the retrieved items. NMRR is defined byNMRR(𝑞)=(∑^NG(𝑞)_𝑘=1(Rank(𝑘)/NG(𝑞)))−0.5−(NG(𝑞)/2)𝐾(𝑞)+0.5−0.5×NG(𝑞), (2.3) where NG(𝑞) is the size of the ground truth set for a query image 𝑞, Rank(𝑘) is the ranking of the ground truth images by the retrieval algorithm, and 𝐾(𝑞) specifies the “relevance rank” for each query. As the size of the ground truth set is normally unequal, a suitable 𝐾(𝑞) is determined by𝐾(𝑞)=min(4×NG(𝑞),2×GTM), (2.4) where GTM is the maximum of NG(𝑞) for all queries. If Rank(𝑘)>𝐾(𝑞), then Rank(𝑘) is changed into 𝐾(𝑞)+1. The NMRR is in the range of [0,1], and the smaller NMRR is, the better the retrieval performance will be. ANMRR is defined as the average NMRR over a range of queries, and is given by1ANMRR=NQ^NQ𝑞=1NMRR(𝑞), (2.5) where NQ is the number of query images.

3. Color-Texture-Based Image Retrieval System

This paper proposes a color-texture-based image retrieval system (CTBIR method) on the basis of the color and texture attributes of images. The system contains two phases—feature extraction and image matching. The feature extraction phase is to extract the GMRFM feature of an image. The image matching phase is to compare the GMRFM features of the given query image and each database image and then, deliver the most similar database image relative to the query image to the user.

3.1. Feature Extraction

For large sized blocks, the GMRFM feature needs high feature dimensions to precisely describe the texture of an image. The CTBIR method will require much memory space to hold the feature of the image and much time to compare the feature with that of each database image. In this paper, every gray-scale image is hence divided into many overlapping blocks of 4×4 pixels, as shown in Figure 2(a), and the relationship of the positions for the 16 pixels in a block is depicted in Figure 2(b). Afterwards, each block B of 4×4 pixels is further partitioned into four overlapping subblocks, 𝐵_1,1, 𝐵_1,2, 𝐵_2,1, and 𝐵_2,2, of 3×3 pixels where each subblock 𝐵_𝑖,𝑗 includes nine pixels: 𝑥_{𝑖−1,𝑗−1}, 𝑥_{𝑖−1,𝑗}, 𝑥_{𝑖−1,𝑗+1}, 𝑥_{𝑖,𝑗−1}, 𝑥_𝑖,𝑗, 𝑥_𝑖,𝑗+1, 𝑥_{𝑖+1,𝑗−1}, 𝑥_𝑖+1,𝑗, 𝑥_{𝑖+1,𝑗+1}. Let 𝜃_𝑖,𝑗 in the GMRFM 𝜃 be the jth category in the ith rank in B. 𝜃_𝑖,𝑗 is defined as follows: 𝜃_𝑖,𝑗=∑²_ℎ=1∑²_𝑘=1𝜃^{𝐵𝑖,𝑗}_ℎ,𝑘4, (3.1) where 𝜃^{𝐵𝑖,𝑗}_ℎ,𝑘, for h, 𝑘={1,2}, are the texture estimates and are computed as follows: 𝜃^{𝐵𝑖,𝑗}_1,1=𝑥_𝑖,𝑗×(𝑥_{𝑖,𝑗−1}+𝑥_𝑖,𝑗+1)2×((∑¹_ℎ=−1∑¹_𝑘=−1𝑥²_{𝑖+ℎ,𝑗+𝑘})−𝑥²_𝑖,𝑗),𝜃^{𝐵𝑖,𝑗}_1,2=𝑥_𝑖,𝑗×(𝑥_{𝑖−1,𝑗}+𝑥_𝑖+1,𝑗)2×((∑¹_ℎ=−1∑¹_𝑘=−1𝑥²_{𝑖+ℎ,𝑗+𝑘})−𝑥²_𝑖,𝑗),𝜃^{𝐵𝑖,𝑗}_2,1=𝑥_𝑖,𝑗×(𝑥_{𝑖−1,𝑗+1}+𝑥_{𝑖+1,𝑗−1})2×((∑¹_ℎ=−1∑¹_𝑘=−1𝑥²_{𝑖+ℎ,𝑗+𝑘})−𝑥²_𝑖,𝑗),𝜃^{𝐵𝑖,𝑗}_2,2=𝑥_𝑖,𝑗×(𝑥_{𝑖−1,𝑗−1}+𝑥_{𝑖+1,𝑗+1})2×((∑¹_ℎ=−1∑¹_𝑘=−1𝑥²_{𝑖+ℎ,𝑗+𝑘})−𝑥²_𝑖,𝑗). (3.2) As a result, each block B comprises four feature values 𝜃_1,1, 𝜃_1,2, 𝜃_2,1, and 𝜃_2,2.

(a) The way to cut overlapped blocks of 4 × 4

(b) The relationship of positions for pixels in a block

Figure 2 

The cutting way for GMRFM feature and the relationship of positions.

Each pixel in a full color image 𝐼 consists of three color components: R, G, and B. The CTBIR method uses the three color components to construct three gray-scale images of the same size as 𝐼: 𝐼_𝑟, 𝐼_𝑔, and 𝐼_𝑏, and each of which is entirely composed of R, G, and B color components from 𝐼, respectively. Next, 𝐼, 𝐼_𝑟, 𝐼_𝑔, and 𝐼_𝑏 are all divided into overlapping blocks of 4×4 pixels and the attributes 𝜃’s in each block with regard to 𝐼_𝑟, 𝐼_𝑔, and 𝐼_𝑏 are calculated, which are, respectively, notated as 𝜃^𝑟_1,1, 𝜃^𝑟_1,2, 𝜃^𝑟_2,1, 𝜃^𝑟_2,2, 𝜃^𝑔_1,1, 𝜃^𝑔_1,2, 𝜃^𝑔_2,1, 𝜃^𝑔_2,2, 𝜃^𝑏_1,1, 𝜃^𝑏_1,2, 𝜃^𝑏_2,1, 𝜃^𝑏_2,2.

In a full color image, a pixel is generally described by a 24-bit memory space; there are 2²⁴ possible colors in an image. However, it is impossible for most images to use all of the possible colors. In order to reduce the memory space required for storing image features and querying time, the CTBIR method employs 𝐾-means algorithm to group the pixels in all database images into 𝐾 groups according to their colors. Moreover, the average color of all the pixels in one group is considered to be a representative color of all images (including database images and query image). The 𝐾 representative colors comprise a common palette (CP).

In every color image 𝐼, the ith color C_i in CP is given a set of variables: 𝜃^{𝐶𝑃𝑖,𝑟}_1,1, 𝜃^{𝐶𝑃𝑖,𝑟}_1,2, 𝜃^{𝐶𝑃𝑖,𝑟}_2,1, 𝜃^{𝐶𝑃𝑖,𝑟}_2,2, 𝜃^{𝐶𝑃𝑖,𝑔}_1,1, 𝜃^{𝐶𝑃𝑖,𝑔}_1,2, 𝜃^{𝐶𝑃𝑖,𝑔}_2,1, 𝜃^{𝐶𝑃𝑖,𝑔}_2,2, 𝜃^{𝐶𝑃𝑖,𝑏}_1,1, 𝜃^{𝐶𝑃𝑖,𝑏}_1,2, 𝜃^{𝐶𝑃𝑖,𝑏}_2,1, 𝜃^{𝐶𝑃𝑖,𝑏}_2,2, and A. For each block B of 4 × 4 pixels in I, the average C_a of all the pixel colors in B is calculated. If C_a is similar to C_i in CP, the blocks 𝜃^𝑟_1,1, 𝜃^𝑟_1,2, 𝜃^𝑟_2,1, 𝜃^𝑟_2,2, 𝜃^𝑔_1,1, 𝜃^𝑔_1,2, 𝜃^𝑔_2,1, 𝜃^𝑔_2,2, 𝜃^𝑏_1,1, 𝜃^𝑏_1,2, 𝜃^𝑏_2,1, and 𝜃^𝑏_2,2 are, respectively, added to 𝜃^{𝐶𝑃𝑖,𝑟}_1,1, 𝜃^{𝐶𝑃𝑖,𝑟}_1,2, 𝜃^{𝐶𝑃𝑖,𝑟}_2,1, 𝜃^{𝐶𝑃𝑖,𝑟}_2,2, 𝜃^{𝐶𝑃𝑖,𝑔}_1,1, 𝜃^{𝐶𝑃𝑖,𝑔}_1,2, 𝜃^{𝐶𝑃𝑖,𝑔}_2,1, 𝜃^{𝐶𝑃𝑖,𝑔}_2,2, 𝜃^{𝐶𝑃𝑖,𝑏}_1,1, 𝜃^{𝐶𝑃𝑖,𝑏}_1,2, 𝜃^{𝐶𝑃𝑖,𝑏}_2,1, and 𝜃^{𝐶𝑃𝑖,𝑏}_2,2, as well as 𝐴_𝑖 is increased by 1. Then, the CTBIR method adopts these 𝜃’s and A’s of all colors in CP as the feature of 𝐼, where we call it the GMRFM feature of 𝐼. The GMRFM feature of 𝐼 hence contains 12×𝐾 of 𝜃’s values and 𝐾 of A’s values, where 𝐴_𝑖 is the number of 4×4 blocks whose average pixel color is similar to one color 𝐶_𝑖 of CP in 𝐼. Hence, A’s can be used to describe the distribution of pixels colors in 𝐼, while the textures can be described by means of 𝜃’s. On the whole, the distribution of the textures for different colors in I can be explained by A’s and 𝜃’s.

The GMRF model assumes that the texture field is stochastic, stationary and satisfies a conditional independence assumption. It is able to capture the local (spatial) contextual variations in an image and is effective for the analysis and synthesis of random looking textures. For each color image I, the CTBIR method generates a set of 𝜃^{𝐶𝑃𝑘,𝑟}_𝑖,𝑗, 𝜃^{𝐶𝑃𝑘,𝑔}_𝑖,𝑗, 𝜃^{𝐶𝑃𝑘,𝑏}_𝑖,𝑗, and 𝐴_𝑖 for 𝑖, 𝑗={1,2} and 1≤𝑘≤𝐾. Here, (𝑖,𝑗) illustrates the orientation of GMRFM feature of 𝐼; 𝐶𝑃_𝑘,𝑟, 𝐶𝑃_𝑘,𝑔, and 𝐶𝑃_𝑘,𝑏 are the three color components of the 𝑘th color in 𝐶𝑃. 𝐴_𝑖 describes the color distribution of 𝐼. Hence, the GMRFM feature also can describe the texture orientation and color information of 𝐼.

3.2. Image Matching

Minkowski distance is widely used to calculate the distance between two multidimension vectors [5]. Suppose that (𝑥₁,𝑥₂,…,𝑥_𝐾) and (𝑥^′₁,𝑥^′₂,…,𝑥^′_𝐾) are two 𝐾-dimension vectors. The Minkowski distance 𝐷_𝑟 of the two vectors can be defined by𝐷_𝑟=(^𝐾_𝑖=1|𝑥_𝑖−𝑥^′_𝑖|^𝑟)^1/𝑟, (3.3) where 𝑟 is a user-defined constant. 𝐷_𝑟 is a Manhattan distance if 𝑟=1 and while 𝐷_𝑟 is a Euclidean distance in case of 𝑟=2.

Let 𝜃^{𝐶𝑃𝑘,𝐶}_𝑖,𝑗 and 𝜃^{′𝐶𝑃𝑘,𝐶}_𝑖,𝑗, respectively, be the 𝜃^𝐶𝑃𝐶_𝑖,𝑗 of 𝐼 and 𝐼^′ with regard to the 𝑘th color in CP, where 𝐶=𝑟, 𝑔, or 𝑏; 𝑖 and 𝑗 are 1 or 2 while 𝐴_𝑘 and 𝐴^′_𝑘 stand for 𝐴, respectively, of 𝐼 and 𝐼^′with regard to the 𝑘th color in CP. The image matching distance D between 𝐼 and 𝐼^′ is defined as follows:𝐷=^𝐾_{𝑘=1𝐶=𝑟,g,𝑏}|𝐴_𝑘−𝐴^′_𝑘|^𝑟1(²_𝑖=1²_𝑗=1|𝜃^{𝐶𝑃𝑘,𝐶}_𝑖,𝑗−𝜃^{′𝐶𝑃𝑘,𝐶}_𝑖,𝑗|^𝑟2)^1/𝑟2, (3.4) where 𝑟₁ and 𝑟₂ are two user-defined constants.

4. Genetic Algorithm-Based Parameter Detector

The performance of CTBIR method is significantly affected by 𝑟₁ and 𝑟₂. In this paper, a genetic algorithm-based parameter detector (GBPD) is used to determine the most suitable 𝑟₁ and 𝑟₂. The GBPD makes use of a binary string composed of 𝑚₁+𝑚₂ binary bits to represent a chromosome 𝐶ℎ in which the first 𝑚₁ bits and the remaining 𝑚₂ bits are used to describe 𝑟₁ and 𝑟₂, respectively. For each chromosome Ch, 𝑟₂ and 𝑟₂ are encoded as 𝑟₁=gap₁×𝑛₁ and 𝑟₂=gap₂×𝑛₂, where 𝑛₁ is the number of 1 bit in the first 𝑚₁ bits of 𝐶ℎ, and 𝑛₂ is the number of 1 bit in the remaining 𝑚₂ bits of 𝐶ℎ; gap₁ and gap₂ are the maximum estimated errors of 𝑟₁ and 𝑟₂.

For a certain application, one can accumulate some of its historic images (including database images and query images). After that, he can apply the accumulated historic images to train the most appropriate values for 𝑟₁ and 𝑟₂ via the genetic algorithm. In this genetic algorithm, we define the accuracy rate (ACC) obtained by the CTBIR method based on the accumulated historic images and the values of 𝑟₁ and 𝑟₂ encoded by a chromosome as the fitness of the chromosome.

GBPD first randomly generates N chromosomes, each with 𝑚₁+𝑚₂ binary bits. To evolve the best solution, the genetic algorithm repeatedly executes mutation, crossover, and selection operations until the relative fitness of the reserved chromosomes is very similar.

In the mutation operation, for each of the 𝑁 reserved chromosomes, the GBPD uses a random number generator to specify one bit 𝑏₁ from the first 𝑚₁ bits and the other bit 𝑏₂ from the remaining 𝑚₂ bits in the chromosome. Later, 𝑏₁ and 𝑏₂ are replaced by⌝𝑏₁ and⌝𝑏₂ to generate a new chromosome, where ⌝ signifies the operator “NOT.”

In the crossover operation, GBPD similarly uses a random number generator to designate 𝑁^′ pairs of chromosomes from the 𝑁 reserved chromosomes. Let 𝐶ℎ[𝑖⋯𝑗] be the substring consisting of the 𝑖th to 𝑗th bits in 𝐶ℎ. For each chromosome pair (𝐶ℎ₁,𝐶ℎ₂), the genetic algorithm concatenates 𝐶ℎ₁𝑚[1⋯⌊¹2⌋],𝐶ℎ₂𝑚[(⌊¹2⌋+1)⋯𝑚₁],𝐶ℎ₁[(𝑚₁+1)⋯(𝑚₁𝑚+⌊²2⌋)],𝐶ℎ₂[(𝑚₁𝑚+⌊¹2⌋+1)⋯(𝑚₁+𝑚₂)] (4.1) into a new chromosome, and𝐶ℎ₂𝑚[1⋯⌊¹2⌋],𝐶ℎ₁𝑚[(⌊¹2⌋+1)⋯𝑚₁],𝐶ℎ₂[(𝑚₁+1)⋯(𝑚₁𝑚+⌊²2⌋)],𝐶ℎ₁[(𝑚₁𝑚+⌊¹2⌋+1)⋯(𝑚₁+𝑚₂)] (4.2) into other new chromosome.

In the selection operation, 𝑁 optimal chromosomes are selected from the N chromosomes reserved in the previous iteration along with the 𝑁 and 2×𝑁^′ chromosomes created in the mutation and crossover operations according to their fitness. GBPD continuously performs the mutation, crossover, and selection operations, until the related fitness of the reserved 𝑁 chromosomes is very close or the number of iterations is equal to the specified maximum number of generations (in this paper, the maximum number of generations is set to be 100). Finally, GBPD multiplies gap₁ (resp., gap₂) by the numbers of 1-bits in 𝐶ℎ[1⋯𝑚₁] (resp., 𝐶ℎ[𝑚₁+1⋯𝑚₁+𝑚₂]) to get 𝑟₁ (resp., 𝑟₂), where 𝐶ℎ is the chromosome having the best fitness in the 𝑁 reserved chromosomes.

Let 𝑚₁=𝑚₂=14, gap₁=gap₂=0.1, and let Figure 3(a) be a chromosome Ch. Derived from Ch, 𝑟₁=0.1×4=0.4, and 𝑟₂=0.1×6=0.6. Figure 3(b) demonstrates a new chromosome created from Ch by the mutation operator, where the underlined bits are the randomly selected 𝑏₁ and 𝑏₂. Figure 3(c) displays two new chromosomes 𝐶ℎ^′₁ and 𝐶ℎ^′₂, generated from the two chromosomes 𝐶ℎ₁ and 𝐶ℎ₂ through the crossover operator.

(a) A chromosome Ch

(b) A new chromosome created from Ch by mutation operator

(c) Two new chromosomes 𝐶ℎ′1 and 𝐶ℎ′2derived from 𝐶ℎ1 and 𝐶ℎ2 by mutation operator

Figure 3 

An example of GBPD.

5. Experiments

The purpose of this section is to evaluate the performances of the CTBIR method by experiments. Let 𝑆𝑒𝑡𝐷={𝑓^𝑑₁,𝑓^𝑑₂,𝑓^𝑑₃,…,𝑓^𝑑₁₀₈₇} and 𝑆𝑒𝑡𝑄={𝑓^𝑞₁,𝑓^𝑞₂,𝑓^𝑞₃,…,𝑓^𝑞₁₀₈₇} be two image sets, each of which contains 1087 full color images. The images in SetD are employed as the database images and those in SetQ are used as the query images. Some of them are captured from animations, where each image pair (𝑓^𝑑_𝑖,𝑓^𝑞_𝑖) is randomly picked from a same animation. Most of the animations were downloaded from http://www.mcsh.kh.edu.tw. The rest of the images was scanned from natural images and trademark pictures.

In SetD and SetQ, the former 500 images are almost cartoon images and the remainders are natural ones. Figure 4(a) shows parts of the query and database images in the former 500 ones of SetD and SetQ, and Figure 4(b) demonstrates parts of those in the latter 587 images of SetD and SetQ. In the following experiments, the parameters 𝑚₁, 𝑚₂, N, 𝑁^′, gap₁, and gap₂ are given to be 40, 40, 20, 10, 0.05, and 0.05 to determine the most suitable 𝑟₁ and 𝑟₂ via the genetic algorithm. Moreover, the size 𝐾 of CP is set to 16.

(a) Partial cartoon images

(b) Partial natural images

Figure 4 

Partial testing images.

The first experiment is to investigate the performance of the CTBIR method in retrieving cartoon images. First, the images {𝑓^𝑑₁,𝑓^𝑑₂,…,𝑓^𝑑₁₀₀} and {𝑓^𝑞₁,𝑓^𝑞₂,…,𝑓^𝑞₁₀₀} are, respectively, specified to be the database images and the query images, and the GBPD is applied to determine the most suitable 𝑟₁ and 𝑟₂. The experimental results obtained by the GBPD tell that the most suitable 𝑟₁ and 𝑟₂ are 1.60 and 0.40. Then, the images {𝑓^𝑑₁,𝑓^𝑑₂,…,𝑓^𝑑₅₀₀} and {𝑓^𝑞₁,𝑓^𝑞₂,…,𝑓^𝑞₅₀₀} are used as the database images and the query images based on the most suitable 𝑟₁=1.60 and 𝑟₂=0.40.

The second experiment is to scrutinize the performance of the CTBIR method in retrieving natural images. In this experiment, {𝑓^𝑑₅₀₁,𝑓^𝑑₅₀₂,…,𝑓^𝑑₆₀₀} and {𝑓^𝑞₅₀₁,𝑓^𝑞₅₀₂,…,𝑓^𝑞₆₀₀} are first taken as the database images and the query images, respectively, and the GBPD is employed to find out the most suitable 𝑟₁=1.35 and 𝑟₂=0.85. In the wake, {𝑓^𝑑₅₀₁,𝑓^𝑑₅₀₂,…,𝑓^𝑑₁₀₈₇}and {𝑓^𝑞₅₀₁,𝑓^𝑞₅₀₂,…,𝑓^𝑞₁₀₈₇} are used as the database images and the query images, and the CTBIR method is assigned to retrieve the expected database images with 𝑟₁=1.35 and 𝑟₂=0.85. Figure 5 shows the obtained 𝑟₁, and 𝑟₂, in 100 iterations obtained by the GBPD in experiments 1 and 2. This experimental result explains that a bigger r₁is required to describe the cartoon images, since the color distributions between most natural images are more varied compared to those between most cartoon images even though the colors of both images are reduced to reserve only a few principal colors.

(a) Cartoon images

(b) Nature images

Figure 5 

The 𝑟1 and 𝑟2 obtained by the GBPD in 100 iterations.

In the third experiment, the images {𝑓^𝑑₁,𝑓^𝑑₂,…,𝑓^𝑑₁₀₀,𝑓^𝑑₅₀₁,𝑓^𝑑₅₀₂,…,𝑓^𝑑₆₀₀} and {𝑓^𝑞₁,𝑓^𝑞₂,…,𝑓^𝑞₁₀₀,𝑓^𝑞₅₀₁,𝑓^𝑞₅₀₂,…,𝑓^𝑞₆₀₀} are considered as the database images and the query images for determining the most suitable 𝑟₁=1.65 and 𝑟₂=0.50 by using the GBPD. Then, the images in 𝑆𝑒𝑡𝐷={𝑓^𝑑₁,𝑓^𝑑₂,𝑓^𝑑₃,…,𝑓^𝑑₁₀₈₇} and 𝑆𝑒𝑡𝑄={𝑓^𝑞₁,𝑓^𝑞₂,𝑓^𝑞₃,…,𝑓^𝑞₁₀₈₇} are used as the database images and the query images based on the most suitable 𝑟₁=1.65 and 𝑟₂=0.50. Table 1 displays the results of the first to third experiments.

Table 1 

The results of the first to third experiments.

The fourth experiment is to compare the performances of the CTBIR, Huang’s and SamMatch methods by using the images in 𝑆𝑒𝑡𝐷={𝑓^𝑑₁,𝑓^𝑑₂,𝑓^𝑑₃,…,𝑓^𝑑₁₀₈₇} and 𝑆𝑒𝑡𝑄={𝑓^𝑞₁,𝑓^𝑞₂,𝑓^𝑞₃,…,𝑓^𝑞₁₀₈₇}. Table 2 demonstrates the experimental results where 𝑟₁ and 𝑟₂ are given to be 1.65 and 0.50. The experimental results indicate that the CTBIR method gives much better ANMRR than Huang’s and SamMatch methods, especially for cartoon images. The CTBIR method totally spends 110.4 seconds to run the 1087 queries in this experiment. In this experiment, CP consists of 𝐾=16 colors each of which corresponds to twelve 𝜃’s and one 𝐴; each value is held in a 4-byte memory space. Hence, the CTBIR method takes 1087×16×13×4=904,384 bytes to hold the whole GMRFM features of the 1087 database images.

Table 2 

The performances (ANMRR) obtained in Experiment 3.

The SamMatch method resizes each image into a 256×256 image, reduces all pixel colors in the image into only 256 colors, and divides the image into 16×16 blocks and uses the average colors of the blocks as the feature of the image. Hence, 113 samples evenly spread out in each image, and each dimension is held by a 4-byte memory space. It consumes 491.324 (=1087×113×4) bytes to hold the features of all database images. In addition, the SamMatch method takes 245.02 seconds to execute the 1087 queries. The Huang’s method consumes 620,928 bytes memory space to save the features of 1087 database images, and spends 125.6 seconds in running the 1087 queries.

When one takes some pictures, the lens may be adjusted to different positions or different directions. Moving the position of certain objects or pixels on an image is called the shift variation of the image, rotating the objects in certain degree is called the rotation variation of the image, and being added some noises in the image is called the noise variation of the image. An image may be enlarged or reduced because of different camera resolution setups. We call this phenomenon a scale variation of the image. In the real world, the distortion variation is a very common phenomenon too. For example, putting a picture on an uneven plane may deform the objects on the picture. Besides that, the hue, contrast, and light variations, such as the image pairs in Figure 6, may be often generated. These variations may make the image regarded as a different image from the original one by an image retrieval system. An excellent image retrieval method should be insensitive to these variations.

(a) Two scale variant images

(b) Two Shift variant images

(c) Two rotation variant images

(d) Two noise variant images

(e) Two distortion variant images

(f) Two lightness variant images

(g) Two hue variant images

(h) Two contrast variant images

Figure 6 

Some variant image pairs.

The next experiment is designed to explore the capacities of the CTBIR method for resisting the variations of rotation, distortion, noise, scale, hue, luminance, and contrast in images. 100 full color images I_q,1, I_q,2,…, I_q,100 are used as the query images. These 100 full color images comprise an image set S_q. Besides, this paper employs the rotation, distortion, noise, scale, hue, luminance, and contrast functions in ADOBE PHOTOSHOP 7.0 to process each I_q,i, and, respectively, generates the variant images I_r,i, I_d,i, I_n,i, I_s,i, I_h,i, I_l,i, and I_c,i. The group of images I_α,1, I_α,2, …, I_α,100 forms an image set S_α, which is produced by the same function where α = r, d, n, s, h, l, and c. Figure 7 shows some images in S_q, S_r, S_d, S_n, S_s, S_h, S_l, and 𝑆_𝑐, respectively, where i is the image number in S_q, S_r, S_d, S_n, S_s, S_h, S_l, and S_c.

Figure 7 

The ith images in sets 𝑆𝑞, 𝑆𝑟, 𝑆𝑑, 𝑆𝑛, 𝑆𝑠, 𝑆ℎ, 𝑆𝑙, and 𝑆𝑐.

In this experiment, all the images in S_r, S_d, S_n, S_s, S_h, S_l, and S_c are employed as the database images, and the images in S_q as the query images. For the ith query, I_q,i is used as the query image and the CTBIR, Huang’s, and SamMatch methods are applied to retrieve the similar database images for the query image. Table 3 shows the experimental results. The ANMRR_total in Table 3 is the accuracy (ANMRR) which the experiment obtains. Furthermore, the ANMRR_r, ANMRR_d, ANMRR_n, ANMRR_s, ANMRR_h, ANMRR_l, and ANMRR_c in Table 3 are the accuracies (ANMRR) obtained by the experiments which take the images, respectively, in S_r, S_d, S_n, S_s, S_h, S_l, and S_c as the database images.

Table 3 

The results of the five experiment.

The experimental results show that the color feature is more significant than the texture feature in distinguishing different cartoon images; hence, a bigger r₁ and smaller r₂ should be provided. However, a smaller r₁ and a bigger r₂ should be assigned for recognizing natural images. The experimental results also indicate that the CTBIR method can offer impressive performances in resisting the rotation, translation, distortion, noise, scale, hue, light, and contrast variations, especially for distortion, hue, and contrast variations because it is indifferent to the shape, color, and light variations when describing the texture features of an image.

6. Conclusions

In this paper, a CTBIR method is proposed for image retrieval. The GMRFM feature is used to describe the texture and color information of an image. Based on this feature, the CTBIR method is used to deliver the database images, which are most similar to the given query image, to the user. The GBPD is offered to decide the fittest parameters of r₁ and r₂, as well. The experimental results show that the CTBIR method is indifferent to the rotation, translation, distortion, noise, scale, hue, light, and contrast variations, especially distortion, hue, and contrast variations. Moreover, the CTBIR method could give a much better performances than those of the Huang’s and SamMatch methods. In the future, a subimage retrieval system based on the GMRFM feature will be developed. With a given query image I_Q, the subimage retrieval system is to respond the images I_D from a database so that I_Q is very similar to a region on I_D.