Single image dehazing by approximating and eliminating the additional airlight component
Introduction
Images captured by the camera, are often affected by haze due to several atmospheric conditions such as fog, smoke, multiple light sources, the scattering of light, etc. Presence of haze in an image obscures the clarity and hence needs to be restored. The flux of light per unit area received by the camera from the scene is attenuated along the line of sight. This redirection of light lessens the immediate scene transmission and replaces with a layer of scattered light known as airlight. Such scattering of light due to rigid particles floating in the air reduces the visibility of the scene. To enhance the quality of images is essential in various applications of computer vision, for instance, object recognition, scene analysis and traffic observation. Notwithstanding, outdoor images inevitably encounter the ill effects of awful climate conditions that straightforwardly cause the degradation of the image quality. The typical effects that result from the haze, fog, mist and smoke incredibly decay the visibility of the image. Haze/mist can massively reduce the color contrast, as appeared in Fig. 1(a). Fig. 1(b) shows the eventual outcome of image dehazing on Fig. 1(a).
Presence of haze in an image obscure the visibility of the image and hence needs to be restored. The flux of light per unit area received by the camera from the scene is attenuated along the line of sight. This redirection of light lessens the immediate scene transmission and replaces with a layer of scattered light known as airlight. Such scattering of light due to rigid particles floating in the air reduces the visibility of the scene. Since the local non-uniform density of haze relies upon unknown scene depth data, dehazing remain an under-constrained issue for a hazy/foggy image. The effect of the scattering at any pixel depends on the depth of the pixel, and the degradation of the image due to haze is spatial-variant.
Traditional strategies for haze removal primarily depend on additional depth information or various multiple observations of a similar scene. The conventional methods exploited multiple images to overcome the problem. For example, the method of Narasimhan and Nayar [1] requires multiple pictures of a similar scene taken under various climate conditions. The method of Shwartz et al. [2] requires images with the various level of polarization. However, images taken in various climatic conditions may be unavailable in reality. Recently researches are attempting for single image dehazing, where reference images for the given hazy images are not required.
Under the assumption that haze-free images contain a considerably higher contrast compared to hazy images, Tan [3] proposed a dehazing approach that maximizes local contrast primarily based on a Markov random field model. Fattal proposed another dehazing technique based on unbiased component analysis under the assumption that surface shading and medium transmission are locally uncorrelated [4]. He et al. [5] proposed a straightforward and effective dehazing technique that depends on the dark channel prior (DCP). The DCP-based approaches produce good result, but cannot handle locales where shading segment does not shift fundamentally contrasted with noise. Another approach was introduced by Fattal in [6], where the color line was introduced for dehazing. The color line based methods approximate transmission map based on the shift of the direction of atmospheric light from the origin. However, both the approaches fail to retain the original color of the objects in the hazy image.
Recently, some deep learning based methods [7], [8], [9], [10], [11], [12] have been proposed, producing much better results compared to the handcrafted features like DCP and color-line. However, in case of images where sky background is present, the deep learning based methods fail to dehaze properly. The amount of haze at high depth area (where distance of the object from the camera is very high, such as sky area) is naturally higher compared to the low depth area of the image. Number of images with a significantly large sky region is very less in the datasets we use. Moreover, the number of patches in the images including sky regions is even less. Hence, being data-greedy techniques, deep learning based techniques fail to learn the amount of haze at high depth regions, due to the lack of adequate patches including high depth regions.
In this paper, we propose a novel approach to address the problem of single image dehazing, preserving the texture information. The proposed method relies on the pixels with moderate intensities of small image patches and exhibit concordant distributions in RGB space. The distribution of color is captured from the Y channel of YCbCr color map to consider the texture. Our second contribution is to substitute the commonly used soft matting technique [13] in assessing the transmission map for haze removal, by assuming that color of the haze-free pixels in the image is approximated by a hundred discrete colors. Based on this assumption we discover the haze-free pixels and apply the Nearest-Neighbor (NN) regularization to predict the value of hazy pixels. The proposed method replaces the traditional softmatting technique for smoothening the transmission map of the image, by NN regularization technique which reduces the execution time drastically. Finally, unlike the state-of-the-art methods, we do not consider and estimate the transmission of the medium. Alternatively, we approximate the additional airlight present in the image patch and eliminate that to clear the haze.
Unlike Ancuti et al. [14], where the concept of estimating the airlight in local image patches was introduced, the proposed method applies the approximation of airlight in local patches of YCbCr representation of images, to preserve the original texture of the image. Moreover, unlike the state-of-the-art methods, we apply NN regularization on the DCP image, to smoothen the DCP. We interject and regularize the fractional evaluations of pixel intensity values into an entire transmission map, and outline an Adaptive Naive Bayes Classification approach to deal with pixels with a significant depth (such as sky region). At the sky regions of the hazy image, the amount of haze becomes uneven due to the atmospheric scattering. The uneven haze leads to estimate an uneven color at the flat sky region. In adaptive filtering scheme, the patch size is automatically increased at the flat regions of the image in order to get information from more pixels in the neighborhood, which results in estimation of flat intensity values at the flat sky regions. Contrary to the conventional field models which comprise of normal coupling between adjacent pixels, we settle the transmission in isolated locales by fluctuating the number of pixels. Specifically, the proposed adaptive filter uses smaller patches at textured regions of the image, whereas, we take the likelihood of more number of pixels in a bigger patch, to get the suitable pixel intensity value.
In this paper, we have extended the idea of our arxived paper [15]. In [15] we combine two different approaches. We initially compute the DCP and then apply an NN regularization technique to obtain a smooth transmission map of the hazy image. We consider the effect of airlight on the image by using the color line model to assess the commitment of airlight in each patch of the image and interpolate at the local neighborhood where the estimate is unreliable. In addition to [15], the proposed method relies on the Y channel of the YCbCr representation of the image, in order to preserve the texture information of the image. Second, the noise and compression artifact that is imperceptible in the hazy image will be intensified, particularly in the area of low transmission esteems (sky regions). Hence, we implement the regularization not to damage the image content in the nearby locale yet strengthen the smoothing impact in the outlying area.
The rest of the paper is organized as follows. Section 2 describes the problem and a survey on the existing literature. Section 3 displays the mathematical and numerical foundation for the proposed strategy for dehazing. Section 4 details the experiments conducted with the proposed algorithm and results obtained by the proposed method compared to the state-of-the-art. Concluding comments are given in Section 5.
Section snippets
Background and related work
Single image dehazing is a challenging task which draws significant attention from researchers of the fields of Computer Graphics and Image Processing [16]. The physical model often used to characterize the haze formation that causes degradation of an image, is known as Koschmieder’s atmospheric scattering model [17]:where I(x) is the observed intensity value of pixel x in image I; J(x) is the scene radiance of a haze-free image at x and t(x) is the medium transmission
Proposed approach
In several restoration-based approaches, strong visual effect regions (i.e., regions with high contrast and high reflectance) in fog images can unavoidably be selected to assess atmospheric light A, which prompts reflection issues. In the regions with a heavy fog, the pixels in the image will have higher color intensity. Hence, we can induce that the intensity channel of an image invariably contains the ambiguous luminance information.
In this section, the proposed method is described as a
Results
We have performed extensive experiments on a variety of images and the results are compared with the state-of-the-art image defogging algorithms. Here, we utilize both subjective comparisons and objective quality assessments to examine the effectiveness of the proposed dehazing algorithm.
Conclusions
We have proposed a novel mathematical technique for haze removal emphasizing the generic regularity across pixels representing the same regions of the image. The proposed technique can minimize the haze, without affecting the original color of the image. The experiments on benchmark real and synthetic image datasets show that the proposed method performs much better than the state-of-the-art non-learning based techniques. Moreover, the proposed method shows comparable results with the recent
CRediT authorship contribution statement
Kushal Borkar: Conceptualization, Methodology, Data curation, Software. Snehasis Mukherjee: Writing - original draft, Visualization, Investigation, Supervision, Writing - review & editing.
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Kushal Borkar completed his Undergraduate course in Electronics and Communication Engineering from the Indian Institute of Information Technology SriCity (IIIT SriCity) with Honours in 2019. His research interest includes Computer Vision, Image Processing and Graphics.
References (53)
- et al.
Haze visibility enhancement: a survey and quantitative benchmarking
Comput. Vis. Image Underst.
(2017) - et al.
A novel dehazing model for remote sensing images
Comput. Electr. Eng.
(2018) - et al.
Infimal convolution type regularization of TGV and shearlet transform for image restoration
Comput. Vis. Image Underst.
(2019) - et al.
Blind video denoising via texture-aware noise estimation
Comput. Vis. Image Underst.
(2018) - et al.
Fast single image dehazing based on a regression model
Neurocomputing
(2017) - et al.
A multi-scale fusion scheme based on haze-relevant features for single image dehazing
Neurocomputing
(2018) - et al.
Vision and the atmosphere
Int. J. Comput. Vis.
(2002) - et al.
Blind haze separation
Proceedings of the 2006 CVPR
(2006) Visibility in bad weather from a single image
Proceedings of the 2008 CVPR
(2008)Single image dehazing
ACM Trans. Graph.
(2008)
Single image haze removal using dark channel prior
Proceedings of the 2009 CVPR
Dehazing using color-lines
ACM Trans. Graph.
Single image dehazing via multi-scale convolutional neural networks
Proceedings of the 2016 ECCV
DehazeNet: an end-to-end system for single image haze removal
IEEE Trans. Image Process.
Aod-net: all-in-one dehazing network
Proceedings of the 2017 ICCV
Edge-enhanced GAN for remote sensing image superresolution
IEEE Trans. Geosci. Remote Sens.
Deep distillation recursive network for remote sensing imagery super-resolution
Remote Sens.
Single image haze removal using dark channel prior
IEEE Trans. Pattern Anal. Mach. Intell.
Night-time dehazing by fusion
Proceedings of the IEEE International Conference on Image Processing (ICIP)
Benchmarking single-image dehazing and beyond
IEEE Trans. Image Process.
Theorie der Horizontalen Sichtweite: Kontrast und Sichtweite
Fast visibility restoration from a single color or gray level image
Proceedings of the 2009 ICCV
Bayesian defogging
Int. J. Comput. Vis.
Modified gain intervention filter based dehazing technique
J. Mod. Opt.
Cited by (0)
Kushal Borkar completed his Undergraduate course in Electronics and Communication Engineering from the Indian Institute of Information Technology SriCity (IIIT SriCity) with Honours in 2019. His research interest includes Computer Vision, Image Processing and Graphics.
Snehasis Mukherjee has obtained his Ph.D. in Computer Science from the Indian Statistical Institute in 2012. Before doctoral study, he has completed his Bachelors degree in Mathematics from the University of Calcutta and Masters degree in Computer Applications from the Vidyasagar University. He did his Post Doctoral Research works at the National Institute of Standards and Technology (NIST), Gaithersburg, Maryland, USA. Currently he is working as an Assistant Professor in the Computer Vision Group of the Indian Institute of Information Technology SriCity (IIIT SriCity). He has written several peer-reviewed research papers (in reputed journals and conferences). His research area includes Computer Vision, Machine Learning, Image and Video Processing.