Content-Based Image Retrieval using Color Quantization and Angle Representation

An efficient non-uniform color quantization and similarity measurement methods are proposed to enhance the content-based image retrieval (CBIR) applications. The HSV color space is selected because it is close to human visual perception system, and a non-uniform color method is proposed to quantize an image into 37 colors. The marker histogram (MH) vector of size 296 values is generated by segmenting the quantized image into 8 regions (multiplication of 45  ) and count the occurrences of the quantized colors in their particular angles. To cope with rotated images, an incremental displacement to the MH is applied 7 times. To find similar images, we proposed a new similarity measurement and other 4 existing metrics. A uniform color quantization of related work is implemented too and compared to our quantization method. One-hundred test images are selected from the Corel-1000 images database. Our experimental results conclude high retrieving precision ratios compared to other techniques.


INTRODUCTION
The increasing number on accessing the image database hosts by Internet users pay more attention to the researchers in developing different algorithms for the image retrieval problem, which is a branch of information retrieval. The performance of the retrieval process is evaluated by the distance or similarity measurements between the query image and the image collection.
Text based image retrieval (TBIR) and content based image retrieval (CBIR) are the main two approaches of the image retrieval problems. Image retrieval is widely used in important applications, such as search engines for image searching, indexing, and retrieving, criminal issues for searching suspected photos, medical diagnosis to discover certain diseases by searching for similar cases, album archives organization, remote sensing systems, motion tracking and crime prevention, etc.
In TBIR systems, the query/input is text and the result of searching using the Google's search engine is a collection of apple fruit and Apple company logo as shown in Figure 1(a). The results of the retrieval process cannot distinguish between the two objects of the same category, because of their names similarity. In CBIR systems, the query/input is an image and the result of searching using the Google's search engine is a collection of only apple fruit as shown in Figure  1(b).

Fig 1: (a) TBIR for the query text "apple". (b) CBIR for the query image
In this research, our focus is on CBIR only that has the ability of extracting the useful information (such as color, shape, and texture features) from the image by analyzing its content to be able to find the most relevant and similar images. Moreover, the CBIR has a powerful availability in search engines like Google, Yahoo and AltaVista [1].
The main process of the CBIR approaches is depicted in Figure 2, the features such as color, texture, shape, or combination are extracted from both the query image and the image database to be processed later, resulting into a marker vector (value or series of values) for each single image called marker histogram (MH). The ole into the CBIR performance [2].

Fig 2: CBIR system process illustration
The proposed CBIR system process starts by converting the image from the RGB color space into the HSV color space, then quantizing the HSV image using the non-uniform color approach into 37 colors. In the second step, the quantized image is divided into 8 regions and each color is counted inside the 8 regions to produce the marker histogram(MH). After the generation of MH, it is rotated into 45, 90, 135, 180, 225, 270, and 315 resulting into another 7 MHs. A new similarity measurement technique and others such as Cosine, Euclidean, Canberra, and Manhattan are used to measure the similarity between the MH of the query image and MHs of the image database. The results of our similarity measurement are mapped into the period [0, 1]. The proposed system is expected to be resistant against rotations and scalability issues compared to the approaches presented in [3,4].
The advantages of using MHs are the ability to overcome the problem of depending on the color histogram that retrieves irrelevant images because of similarity in the histograms, and of dividing the image into 8 regions to find rotated images.

RELATED WORK
In color based approaches, the histograms are widely used to count the occurrences of the image's colors and applied to measure the similarity between two images [5,6], but as mentioned in the previous section, different images can be similar in their histograms and are irrelevant from the human visual system as shown in Figure 3. However, the precision of histogram based approaches depend on the image's complexity and its type.

Fig 3: Two different images with identical histograms
The extraction of shape feature is used to measure similarity too, which relies on segmenting the image into different regions, and then the main shape of each particular region is detected [7,8]. The main problem is the existence of a huge number of objects within the image and is time consuming. The texture is another interesting feature to find similarity of the visual patterns in the image in addition to the spatial distribution of these textures within the image [9-12. The main problem in the texture feature is its weakness against scale, rotation and translation, because the texture features such as direction, periodicity, and coarseness are potential subjects for change. Other approaches used the combination of the color and texture features to get a much more powerful similarity measurement such as the multi-texton histogram [13], integrative co-occurrence matrix [14], texton co-occurrences matrix [15] and color auto-correlograms [16]. Other researchers used the structure features to find similarity between images [17,18].

System Description
Two main problems are addressed in the proposed CBIR system workflow, the color quantization and the representation of an image into a marker histogram vector or series of representative values. Color quantization is the starting step of the process of any CBIR system to reduce the dimensionality space into a dozen of colors. The two common approaches used for the color quantization are the uniform and non-uniform. To generate a marker, the image is divided into 8 regions and each quantized color is counted in the divided regions. The proposed similarity formula is used to measure the similarity between two markers. Image rotation is considered by shifting the original marker 7 times resulting in a total of 8 markers (each marker represents a multiplication of 45 rotated image for all quantized colors) to enhance the performance of the proposed CBIR system. The block diagram of the proposed CBIR system is depicted in Figure 4.

Non-Uniform Color Quantization
The depth of the color per pixel reflects the pixel's resolution, and the total number of colors in the image depends on the image type, color system and the pixel's resolution, for example if the HSV (Hue, Saturation, and Value) system is used, with 8-bit depth for each color, then, the image includes 2 24 or 16 million possible colors. Therefore, high computations are required to process a huge number of colors in image causes a major problem. Another problem is the distinction between two similar colors with two different saturation values looks different for the computer vision applications, but for the human vision system they might look as one color. The overall solution for both problems is to use quantization.
The quantization is the process that improves the performance of the CBIR system by decreasing the huge number of colors from millions to dozens in terms of replacing similar or close colors by one quantized color. In the RGB (Red, Green, and Blue) color space, assume the criteria is to quantize any pixel value (R ≤ 40, G ≤ 40, and B ≤ 40) into a black pixel and (R > 215, G > 215, and B > 215) into a white pixel. However, the RGB color space is successfully used in computer vision systems, but the human vision system is close to the HSV color space in terms of the color perception and its saturation. For this reason, the HSV color space is selected in our proposed CBIR approach.
To convert an image from the RGB color space into HSV color space, the R, G, and B values are divided by 255 to change the range from 0 .. 255 to 0 .. 1, then: The value of H ranges from 0 to 359, S from 0 to 1, and V from 0 to 1 as shown in Figure 5(a). The human perception system can see clearly that the Hue separates the distribution of the colors and similar colors are appeared adjacent to each other (i.e, red and orange, or orange and yellow), low Saturation looks like white or close to white color, and low Value looks like black or close to black color. Moreover, the distribution of the colors across the Hue plane are non-equal (non-uniformly distributed) and some colors (i.e., red, green, and blue) occupy more than others in the Hue plane as depicted in Figure 5 In this research, the idea of the work of [19] for color quantization is modified by partitioning the Hue space non-uniformly. The non-uniform color quantization process is described in the Equation (4), where Q is the quantized vector. Figure 6 illustrates the implementation of Equation (4).

The Marker Histogram
By knowing the height and width of the image, it's an easy to get the center of the image (reference point). From the reference point, the image is partitioned into 8 regions as depicted in Figure 7. Each region represents a multiplication of 45° (i.e., region 2 covers all angles between 45° and 89°).
where, (x,y) is the index of any quantized pixel in the image.
Dividing all values of the generated MH by the maximum one (in our example, is 10047) enables the MH invariant to scale changes. In addition, to make MH robust against rotations, a shift process is made 7 times to the MH vector, as for example, assume that the third rotation (135) of the MH vector (10047, 3341, 6765, 7503, 3258, 2508, 1394, and 5842) is (2508, 1394, 5842, 10047, 3341, 6765, 7503, and 3258). The same operation is repeated to other rotated angles.

Similarity Measurement
There exist many similarity methods to measure the similarity between two objects. If the similarity value is 1 means the two objects are identical and 0 if both are completely different. In this research, we have implemented five similarity measurements which are: the proposed similarity measurement, Cosine, Euclidean, Canberra, and Manhattan (Equation 6 -Equation 10), respectively. Let S is the similarity value between both vectors X and Y and is calculated by the following formulas: , cos(  where n is the size of MH. The advantage of implementing the Equation (6) makes low importance for small numbers like 0.1 and 0.2 when it has to be compared with 0.88 and 0.92 for example, and to give less importance for comparisons between 0.9 and 0.07 for example. Therefore, comparisons between the abovementioned similarity measurements are discussed in Section 4 and the results will show that our similarity measurement provides an overall better accuracy than others.

EXPERIMENTAL RESULTS AND DISCUSSIONS
This research is implemented on MATLAB 2009a, core i3 -2.20 GHz processor, and Windows 7 operating system. The Corel-1000 image database is selected in our experiments that consists of 10 categories (Africa , Beaches, Building, Buses, Dinosaurs, Elephants, Flowers, Horses, Mountains and Foods), and each category consists of 100 different true color images. From each category, ten images are selected to test the uniform and non-uniform color quantization by using the mentioned similarity measurement techniques in subsection 3.4.
Retrieved images are ranked according to their similarity values between the query image and the Corol-1000 image database, and four choices are selected to display the best 10, 15, 20, and 25 scores. Initially, all images are transformed from the RGB color space into HSV color space. Two color quantization methods are implemented, the proposed nonuniform color quantization maps the image into 37 colors and the uniform color quantization maps the query image into 72 colors as shown in Figure 9(b,c), respectively.
Assume that the query image is the image no. 878 is selected from the Corel-1000 image database, Figure 10 and Figure  11 illustrate the implementation of our similarity method on the non-uniform and uniform quantized image, respectively. As shown in the Figure 10 and Figure 11, each retrieved image is displayed with its associated similarity value appears above it, and the wrong retrieved image is bounded by a red rectangle. A snapshot of the precision results of non-uniform and uniform color quantization for particular category (i.e., category 8) are summarized in Table 1 and Table 2, respectively. Our similarity measurement presents higher precision values among other similarity/distance metrics. However, the precision value of the retrieval process is proportional to the total number of retrieved images displayed.  The overall performance of this research is summarized in Table 3 and Table 4. The results obtained ensures that the non-uniform color quantization concludes better results than uniform color quantization, and our proposed similarity measurement as well.

CONCLUSIONS
In this research the HSV color space is chosen in our experiments because it's close to our visual perception system, and the implementation of a non-uniform color quantization is our approach. The Hue plane is non-uniformly divided into 7 blocks of different sizes including threshold criteria imposed on Saturation and Value planes, to create an image of 37 quantized colors. The quantized image is divided into 8 regions and each region is a multiplication of 45. By counting the occurrences of the 37 quantized colors in the 8 regions, a marker histogram (MH) vector of size 296 values is generated.
To cope with rotated images, the non-rotated MH is rotated 7 times by shifting the occurrences of each quantized color in multiplications of 45.
Therefore, The 8 generated MH vectors of the query image are compared with a MH vector of each image in the Corel-1000 image database using our proposed similarity measurement and other existing distance/similarity measurements such as Cosine, Euclidean, Canberra, and Manhattan. The overall precision obtained shows that our similarity measurement offers better results than other metrics.