Ghost-Free High Dynamic Range Imaging Using Histogram Separation and Edge Preserving Denoising

In this paper, we introduce a ghost-free High Dynamic Range imaging algorithm for obtaining ghost-free high dynamic range (HDR) images. The multiple image fusion based HDR method work only on condition that there is no movement of camera and object when capturing multiple, differently exposed low dynamic range (LDR) images. The proposed algorithm makes three LDR images from a single input image to remove such an unrealistic condition. For this purpose a histogram separation method is proposed in the algorithm for generating three LDR images by stretching each separated histogram. An edge-preserving denoising technique is also proposed in the algorithm to suppress the noise that is amplified in the stretching process. In the proposed algorithm final HDR image free from ghost artifacts in dynamic environment because it self-generates three LDR images from a single input image. Therefore, the proposed algorithm can be use in mobile phone camera and a consumer compact camera to provide the ghost artifacts free HDR images in the form of either inbuilt or post-processing software application.


INTRODUCTION
Real world scenes capturing becomes easy for non-experts since high-quality imaging devices are increase in consumer electronics market. Real world scenes acquisition includes three essential factors: i) high spatial resolution, ii) true color reproduction, and iii) high dynamic range (HDR). Since the HDR imaging method has been emerged it played an important role in digital imaging [1]. Digital imaging devices have a limited dynamic range because of the finite number of bits for pixel intensity values. To overcome the limited dynamic range, HDR imaging technique has been introduced, that capture multiple differently exposed low dynamic range (LDR) images of the same scene and combine them to obtain the extended dynamic range image [2] [3]. Whereas the multiple image-based methods can successfully provide HDR images in the static scene, ghost artifacts are often generated when the scene contains moving objects. Moreover, it is even more difficult for mobile camera with limited computational power to acquire motion-free multiple images. For this reason, consumer mobile imaging devices needed single image based HDR algorithm.
In order to make ghost free HDR image, several approaches have been introduce in the literature. Kao has proposed image fusion with global and local motion stabilization to fuse two registered differently exposed images [4]. Choi has proposed a multi-exposure camera system [5] which captures three consecutive images from the same scene with different exposure times. This system acquires HDR images by combining the object-shape information from the underexposed image and the color information from the two blurred over-exposed images. Wu has proposed a method [6] for automatically generating HDR images in consideration of camera moment and dynamic scenes. This method includes the camera response function that corrects the LDR images using the original image. To minimize the errors among the camera response function and image alignment, the corrected images are processed using image inpainting. Im has proposed LDR image registration in [7], where the camera motion is compensated to align multiple LDR images. A modified elastic registration method is also used for accurate alignment. Since all the methods introduce above generate an HDR image using multiple LDR images, so they cannot be applied to limited memory mobile imaging devices. These multiple image-based HDR method also require high computational complexity and long processing time in image registration and fusion steps, which makes consumer applications impossible. While Bilcu has proposed an HDR method that can be useful in mobile imaging applications [8], this method also produces ghost artifacts because it does not think about camera motion during the exposure time. So all the above method produces ghost artifacts in dynamic condition. Now to tackle the problem of ghost artifacts, we proposed single frame based high dynamic range imaging algorithm. The existing single frame algorithm use weighted histogram separation for histogram separation and generate HDR image that loss some information. The proposed algorithm self generate three LDR images from a single input image. The proposed algorithm generate multiple LDR images that has exactly the same geometric coordinate and combine multiple LDR images to generate HDR image that is free from ghost artifacts and also reduce loss of information . The proposed algorithm significantly improves the performance against the computational complexity because it does not need any image stabilization and image registration. The structure of the paper is prepared as follow. W e present the proposed method, results analysis and finally conclude the paper.

PROPOSED METHOD
In this section, we present the proposed HDR algorithm that consists of histogram separation method, histogram stretching, edge-preserving denoising, and LDR image fusion.The proposed algorithm as shown in the dotted box in Fig.1. UEL-under-exposed LDR image, NEL-normal-exposed LDR image, OEL-over-exposed LDR image The proposed algorithm uses the concept of genetic algorithm [9] and entropy maximization technique [10] as a histogram separation method to specify stretching region that also reduce the loss of information in the final HDR image. Then stretches each separated histogram by using histogram equalization [11] and generate under-exposed, normalexposed and over-exposed image. We then use an edge-preserving denoising algorithm [12] to suppress noise amplified in the process of histogram stretching.

Histogram separation method
In this section, we proposed histogram separation method that uses the concept of entropy based histogram separation method and genetic algorithm, to specify stretching region. These two concepts are given below. Proposed Algorithm J a n u a r y 0 4 , 2 0 1 4

Entropy based histogram separation
Kapur has developed the algorithm [10] for image histogram separation into two subsets that is based on the entropy of the gray level. Here entropy represents randomness of the gray levels. The histogram separation based on entropy can be described as follows: Now suppose in an input LDR image there be L gray levels and these gray levels are in the range {0, 1, 2,……, (L-1)}. Then we can define probability of occurrence of pixel Pi = h (i)/N (0 ≤ i ≤ (L-1)), where h (i) denote the number of pixels for the corresponding gray-level L and N denote total number of pixels in the image which is equal to In Kapur image histogram separation algorithm we consider gray level entropy of two subsets for exact histogram partitioning. The two subsets entropy can be computed as follows: H0 -Subset 0 entropy and H1-Subset 1 entropy H0 calculate the entropy of the first subset that has 0 to  gray levels. H1 calculate the entropy of the second subset that has  to 255 gray levels. The optimal threshold is the gray level that maximizes Equation (3). If the objective function has a maximum value about a gray level point then we get exact histogram partitioning.
Then the objective function.
Now we use optimization technique that can find the gray level point in quick time. So we employ a genetic algorithm for finding a gray level point that maximizes the objective function.

Genetic algorithm
In this section we use Genetic algorithm [9] to determine the gray level point that maximizes the Equation (3). So the Equation (3) is the fitness function for genetic algorithm and the aim is to maximize the fitness function.
The procedure of the Genetic Algorithms is given as.
Here T represent the number of generation (0, 1, 2…1000) Step.1. In this step of genetic algorithm initial population of size 64 is generated by randomly selecting the gray level point from all the gray level point of an image.
Step.2. Fitness function that is The gray level point that has highest fitness function value is selected as an optimum threshold gray level point for that generation T.
Step.3. Perform the GA's reproduction, and mutation operator and create the next generation T+1.
Step.4. Get the final optimal threshold gray level point  , if the predetermined number of generations is reached or the optimal threshold remains same for 10 generations, else return to Step (2).
At the end of the genetic algorithm, we get the gray level point  for which objective function has maximum value. Using the specified gray level point  , we split image histogram dataset D into two subsets D0 and D1, which are defined Now we split histogram D into two subsets by using  , then each subset contains gray levels from 0 to  , and from  to 255. Therefore, the proposed algorithm generate under exposed and over exposed LDR images by stretching two subsets using histogram equalization. The proposed algorithm include color space transforms from RGB color space to the hue, saturation, and value (HSV) color space to preserve the chromatic information. Then stretching of the histogram along the V channel can be done in the following manner.
J a n u a r y 0 4 , 2 0 1 4 Where Si(x, y) represent the pixel intensity at (x, y) in subset image i. The histogram stretching perform along the V channel, together with the unprocessed H and S channels, is then final image transformed back to the RGB color space. The stretched histogram D0 belongs to the under-exposed image and D1 belongs to the over-exposed image. Finally, we can use the image fusion to combine these three LDR images, the reference, under-exposed, and over-exposed images, to generate the HDR image.

Edge-Preserving denoising algorithm
In the proposed algorithm noise is amplified that degrades the quality of the HDR image together with the brightness levels during the histogram equalization process [11]. The simplest way we use an averaging filter to reduce this noise. However, averaging filter removes not only noise but also essential details such as edges. The proposed algorithm employs an edge-preserving denoising algorithm [12] that preserves the edges and removes noise. It takes detailed high-frequency regions from a noisy LDR image and takes flat regions from the result of averaging filter with an appropriate amount of weighting between the two as follow: Where favg is the output of averaging filter. The parameter β is a weighting factor that is given by Where δ represents a tuning parameter to make β distributed as uniformly as possible in the range of [0, 1] and σ 2 represents the local variance. Fig. 2(a) shows the histogram stretched image with amplified noise and Fig. 2(b) shows the corresponding color mapped edge image, where white regions correspond flat, non-edge regions, and black regions represent edge that are same in all three color channels. Each RGB histogram stretched image is processed by the denoising algorithm given in equation (6).

Image fusion
In mobile imaging devices available memory and, computational power are limited, which create multi-frame HDR imaging difficult. For efficient implementation, we proposed LDR image fusion process [4] for the limited number of images. LDR image fusion is needed to combine the three LDR images that are generated by the proposed algorithm and create an HDR image. In the image fusion technique, we employ only simple arithmetic operator that fuses the same coordinate pixel of three LDR images and produce the final HDR image that have same coordinate pixel that have LDR images. So the final HDR image free from the ghost artifact. The fusion of three LDR can be describe as follows Where HC (i, j) represents the final ghost free HDR image, and where p is a normalizing constant.

RESULT ANALYSIS
In this section, we show the result of our proposed Ghost-free HDR image generation algorithm that generates Ghost free HDR image using the single input image. The proposed algorithm use only single input image for an HDR image generation so it can be used in a dynamic environment where the ghost artifact in the capturing image occur frequently. The existing Debevec's [5], Im's [8], and Bilcu's [2] methods use multiple differently exposed LDR images for HDR image generation so it produce ghost in a dynamic environment. The existing method need extra time for ghost detecting and in ghost removal process so it is time consuming compared to the proposed ghost free HDR image generation algorithm.
The set of images called Bottle, Café, and Berlin used as an example of differently exposed LDR images that is used in the existing method can be shown in Fig. 3 (a, b, c). Fig. 3 (a, b,

c) Three set of LDR images acquired by different exposure values (EVs) in the dynamic environment.
In the set of Bottle images pedestrians are walking, and the camera is shaking as shown in Fig. 3(a). In the set of Café images both the vehicle and the camera are moving in different directions as shown in Fig. 3(b). In the set of Berlin images the camera is shaking under backlighting environment as shown in Fig. 3(c).So these Set of differently exposed LDR images produced ghost in the final HDR image that is generated using existing Debevec's [3], Im's [7], and Bilcu's [8] methods. Fig. 4 (a, b, c) Three input images for proposed Ghost free HDR method.

(c) Berlin Sequence
Now our proposed algorithm generate HDR image using single input image that is show in Fig. 4. The proposed algorithm self generated three LDR images that are under-exposed, normal-exposed and over-exposed that is Show in Fig. 5. The final Ghost free HDR image also shows in Fig. 5.

Fig. 5 Results of the proposed ghost free HDR methods (Bottle Sequence); (a) Input image, (b) Under-exposed LDR image, (c) Normal-exposed LDR image, (d) Over-exposed LDR image, and (e) the final HDR image.
In result analysis we take five LDR images as an input image and generate HDR image using the proposed algorithm and existing algorithm [13]. Fig. 6 show the five input LDR images (a, b, c, d and e), output HDR image of the existing algorithm and the output HDR image of the proposed algorithm.

Output HDR images
Image (a-e) Show the output HDR images respectively as input images in the existing algorithm.

Input LDR images
(c) Normal-exposed image (d) Over-exposed image (b) Under-exposed image (e) HDR image Input LDR image (a) J a n u a r y 0 4 , 2 0 1 4   Table 1 and Table 2 show the Comparison of mean squared error (MSE), Peak Signal to Noise Ratio (PSNR), Normalized Cross Correlation (NCC) and Normalized Absolute Error (NAE) values of the proposed method with existing method. The large value of MSE indicates poor quality of the image. In our proposed work the value of Mean Squared Error is reduced so the image quality has been improved. The higher value of PSNR indicates quality of the image. In our proposed work the value of Peak Signal to Noise Ratio is increased so the image quality has been improved. The higher value of NCC indicates quality of the image. In our proposed work the value of Normalized Cross Correlation is increased so the image quality has been improved. The large value of the NAE indicates poor quality of the image. In our proposed work the value of Normalized Absolute Error is reduced so the image quality has been improved. Image (a-e) Show the output HDR images respectively as input images of the proposed algorithm. J a n u a r y 0 4 , 2 0 1 4 In the result of proposed method mean square error (MSE) and normalized absolute error (NAE) is reduced as compare to the existing method, correspondingly peak signal to noise ratio (PSNR) and normalized cross co-relation (NCC) is improved as compared to the existing methods. For these reasons the quality of the HDR image is better in the proposed algorithm.

CONCLUSION
We proposed single image-based ghost-free HDR imaging algorithm using histogram separation method and edge preserving denoising technique. Because the existing multiple image-based HDR image generation method work only on condition that there is no camera and object movement during the acquisition of several differently exposed LDR image. Ghost artifact is unavoidable in the dynamic environment when acquiring multiple, differently exposed LDR images. To solving this problem, the proposed algorithm self generates three LDR images from a single input image. For this, we use techniques of histogram equalization. To remove noise amplification during the process of histogram equalization, we also use edge preserving noise suppression technology. We can then generate HDR image by fusing three LDR images. The proposed HDR method generates ghost artifact-free HDR images using a single input image. For this reason the proposed method provides easy acquisition using a camera without using a tripod for acquiring LDR images. In the future it can be used as function in mobile phone camera in the form of integrated algorithm or post-treatment to provide the ghost artifacts-free HDR image.