Method for enhancing visibility of hazy images based on polarimetric imaging

Jian Liang; Liyong Ren; Enshi Qu; Bingliang Hu; Yingli Wang

doi:10.1364/PRJ.2.000038

Abstract

A novel polarimetric dehazing method is proposed based on three linear polarization images (0°, 45°, and 90°). The polarization orientation angle of the light scattered by the haze particles is introduced in the algorithm. No additional image-processing algorithm is needed in the postprocessing. It is found that the dehazed image suffers from little noise and the details of the objects close to the observer can be preserved well. In addition, this algorithm is also proved to be useful for preserving image colors. Experimental results demonstrate that such an algorithm has some universality in handling all kinds of haze. We think that this robust algorithm might be very suitable for real-time dehazing.

1. INTRODUCTION

Over the past decades, there has been great interest in polarimetric imaging since it has been widely used in many imaging applications ^[1–3]. Several kinds of polarimetric cameras have been proposed to satisfy different usages, such as wideband cameras ^[2], 3-Stokes parameters imaging cameras ^[4,5], and underwater cameras ^[6]. Moreover, polarimetric cameras that can simultaneously obtain full-Stokes parameters are under study by the techniques of liquid crystals ^[7], plasmonic lenses ^[8], and Wollaston prisms ^[9].

Nowadays, hazy weather appears more and more frequently as a result of pollution, and we all know that the visibility of an image taken directly in the haze is usually very poor. This may cause some inconvenience in daily life. To solve this problem, polarimetric imaging has been found to be a useful method for effectively improving the visibility of the image. In 2001, Schechner et al. ^[10] first demonstrated that one can enhance the quality of images taken in poor visibility weather by using polarimetric imaging. The basic principle includes two aspects. The first one is to estimate the intensity of the air light (scattered by the haze particles) according to its partial-polarized property; the second one is to remove the above-estimated part from the hazy image and thus to obtain the visibility-enhanced image. In following years, his group made much effort to discuss and perfect this theory ^[11–13]. Especially in 2009, they accomplished the experiment of underwater descattering ^[6], and the outcome of the descattered image is pretty good. More recently, Mudge and Virgen published their real-time polarization dehazing results using their own polarimetric camera ^[14]. The dehazing algorithm they used is almost the same as Schechner’s. Meanwhile, another dehazing method based on multiresolution image fusion of color and near-infrared information has also been reported for haze-degraded images ^[15], although it is much more costly than methods based on the polarimetric imaging. Note that, in addition to the above-mentioned physics-based dehazing methods, there still exist some other image-processing methods for the same purpose ^[16,17]. These methods can also enhance the contrast of hazy images. However, some information of the image is inevitably lost, since all these methods estimate the haze of one pixel only according to several surrounding pixels. In ^[17], the authors compared their result with Schechner’s, and we can easily see that the latter one is better.

The polarization-based algorithm is very effective in dehazing; however, there still exist some drawbacks to be overcome. For one thing, this algorithm is based on two orthogonal images, which should be the “brightest” and the “darkest,” respectively, and then adds them together as the intensity image. This step needs the two images to be exactly the same; otherwise, there may be ghost shadows in the dehazed image and significant details may be submerged in the background. For another, due to the uncertainty of the CCD pixels, there always exists the response difference for the same intensity with two snapshots in one pixel or one snapshot in different pixels. This may result in terrible noise in the dehazed image, which needs to be eliminated by complex imaging-processing algorithms. However, such imaging-processing algorithms may cause some information to be lost.

Sign up for Photonics Research TOC. Get the latest issue of Photonics Research delivered right to you！Sign up now

In this paper, we propose a new polarization dehazing method. The polarization orientation angle is introduced to estimate the air light in the image, and it is proved to be quite useful in eliminating the blur in the dehazed image. Also, the noise in the sky area can be eliminated without any imaging-processing algorithm. This algorithm might be much more convenient and reliable in real-time dehazing.

2. TECHNIQUE BACKGROUND

The main idea of the polarization-based dehazing theory is expounded in ^[18]. To clearly understand our work, let us briefly review some important parameters and relations. The light incident upon the camera contains the air light scattered from the haze particles and the direct light reflected from the objects. We use $A$ and $D$ , respectively, to represent their intensities. Therefore, the total intensity $I$ is the combination of them: As usual, the air light is assumed to be governed by the haze particle’s single scattering model, and $A$ can be expressed by where $A_{\infty}$ is the air light from an object at infinite distance and $β$ is the extinction coefficient. Similarly, the direct light $D$ decays exponentially with the propagation distance, being expressed by where $L$ is the object radiance, i.e., the real information we care about from the scene. Obviously, haze particles cause severe degradation of both the intensity and the polarization of the direct light. This is why the direct light from a distant object can be regarded as nonpolarized light.

By combining Eqs. (1)–(3), we finally obtain the expression of $L$ : The method of estimating the parameter $A_{\infty}$ can be found in ^[12,13]. If the image includes the sky, one can easily regard the intensity of the horizontal sky as $A_{\infty}$ ^[18]. For simplicity, all the hazy images used in this paper are images with sky in them.

Mount a polarizer in front of the camera lens, and take three photos $I (0)$ , $I (45)$ , and $I (90)$ by rotating the polarizer to the angles of 0°, 45°, and 90°, respectively. Therefore, the Stokes parameters can be written as Note that, in fact, $S_{0}$ is the same as the total intensity $I$ . According to Eq. (5), one can give the expression of the orientation angle $θ$ and the degree of polarization (DOP) $p$ , respectively, as As mentioned above, the direct light from far distant objects reaching the camera can be assumed to be nonpolarized light. Thus, according to Eq. (5), although $S_{0}$ contains both the direct light and the air light, there is nearly no contribution of direct light in parameters $S_{1}$ and $S_{2}$ . As a result, $θ$ provides us a much more accurate value than $p$ , as can be seen from Eqs. (6) and (7). Thus, it is a benefit for us to use parameter $θ$ to estimate the intensity of the polarized part of the air light. Note that the results of Eqs. (6) and (7) are matrices, and the value of $θ$ and $p$ can be estimated from the sky area of the images. Because we have already found the pixel that can represent $A_{\infty}$ when estimating it, we use the value of the same pixel in those matrices to represent $θ$ and $p$ for simplicity.

Our algorithm can be introduced as follows. Let us define the rotating angle 0° of the polarizer as the $x$ axis, and $θ$ as the included angle between the direction of the polarization and the $x$ axis, as shown in Fig. 1.

Defining the axes according to the rotating angle of the polarizer. The x and y axes represent 0° and 90°, respectively. θ is the included angle between the direction of the polarization and the x axis. The arrow represents the polarized direction of the air light.

Figure 1.Defining the axes according to the rotating angle of the polarizer. The $x$ and $y$ axes represent 0° and 90°, respectively. $θ$ is the included angle between the direction of the polarization and the $x$ axis. The arrow represents the polarized direction of the air light.

First, we use $I (0)$ and the Stokes parameters obtained previously to estimate the polarized part of the air light. $θ$ and $p$ of the polarized air light can be estimated from the very upper part of the image, and, according to Fig. 1, the intensity of the light in the $θ$ direction [denoted by $I_{∥} (θ)$ ] can be obtained from the raw image by Thus, the intensity of the light perpendicular to the $θ$ direction [denoted by $I_{⊥} (θ)$ ] can be written as Note that, although in principle, both the direct light and the air light are partially polarized light, their orientation angles are different. As a result, $I_{∥} (θ)$ contains all of the polarized air light, part of the polarized direct light, and half of the nonpolarized light. Correspondingly, $I_{⊥} (θ)$ contains part of the polarized direct light and half of the nonpolarized light, but none of the polarized air light. That is to say, the difference between $I_{∥} (θ)$ and $I_{⊥} (θ)$ can approximately represent the polarized air light (denoted by $A_{p}$ ) since the nonpolarized light is fully counteracted and the polarized direct light is partially counteracted. Therefore, $A_{p}$ can be written as It should be pointed out that, in Schechner’s theory, the intensity difference between the “brightest” image and the “darkest” one is considered to be $A_{p}$ . However, the “brightest” and the “darkest” images are difficult to find, thus resulting in less precision in the estimated $A_{p}$ . Using such an $A_{p}$ to do the dehazing may cause blurring of the image, as mentioned in ^[11]. So the estimation of $A_{p}$ in our algorithms is more precise compared to Schechner’s algorithm, and the details of the objects in the dehazed image can be better retrieved.

Second, the total intensity of the air light can be expressed as ^[18] where $p$ can be estimated from Eq. (7), and $ε_{1}$ is a bias factor used to balance the error of the estimated $p$ . $ε_{1}$ is likely to be chosen in the range of 1 to $1 / p$ ^[18]. Note that, in order to obtain the optimum dehazing effect, $ε_{1}$ might be chosen as different values for different kinds of haze, just as the value is chosen as 1.09 in ^[18] and 1.5 in ^[14]. As usual, the value of $ε_{1}$ should be large enough to guarantee that $A$ is always smaller than $S_{0}$ , although the visibility enhancement might be limited by the excessive $ε_{1}$ . In our experiment, we found that $ε_{1} = 2.5$ can ensure that $A$ is always smaller than $S_{0}$ in all different hazes, while the visibility of the dehazed image can be substantially enhanced.

Finally, by substituting Eq. (11) into Eq. (4), the light radiance from the objects $L$ can be obtained. In fact, $A_{p}$ is determined by $I_{∥} (θ)$ while $I_{∥} (θ)$ is calculated by $I (0)$ , so it is easy to know that the characteristic of air light $A$ relies on $I (0)$ from Eq. (11). So the result of $I - A$ [i.e., $I (0) + I (90) - A$ ] is probably dominated by the property of $I (90)$ . Therefore, in Eq. (4) we can see that $L$ primarily represents the dehazed image of $I (90)$ . Compared with the dehazed image from $S_{0}$ , this algorithm can effectively suppress the noise in the dehazed image.

Additionally, we squared the dehazed image to further improve its quality. This is a well-known simple but effective treatment, since it enhances the contrast of the image without losing any information. This treatment can be regarded as prolonging the exposure time of the dehazed image in a sense. For this purpose and to distinguish from $L$ expressed by Eq. (4), from now on we will use another parameter, $L_{Squ}$ , to represent the object radiance through the relation where $ε_{2}$ can be called a correction factor, which is used to ensure that all the intensity values of the pixels are in the range of display. In our experiment, we set $ε_{2} = 1.5 \times L_{Squ}^{\max} / I {(90)}_{\max}$ , where $L_{Squ}^{\max}$ and $I {(90)}_{\max}$ represent the maximum values of the pixels in $L_{Squ}$ and $I (90)$ , respectively. Here, without loss of generality, we add a constant 1.5 into the expression of $ε_{2}$ to make the values of the pixels more suitable for displaying. Besides, it should be noted that the exponent in Eq. (12) should not be chosen to be large; otherwise, parts of the image may become too bright, while other parts become too dark. Consequently, after the confirmation of $ε_{1}$ and $ε_{2}$ , the proposed dehazed algorithm can be preceded automatically when three original polarized images are given. As a result, this algorithm is much more appropriate for real-time dehazing processing.

3. EXPERIMENTAL RESULTS AND DISCUSSION

In our experiments, an ordinary 8-bit black and white industrial camera is first used to demonstrate the feasibility of our dehazing algorithm. The dynamic range of the camera is 58 dB. We mount a polarizer in front of the camera lens, and snap three shots as the rotating angles are set to 0°, 45°, and 90°, as shown in Figs. 2(a)–2(c). Figure 2(d) is the image dehazed by using our algorithm, without any additional image-processing algorithms. We can see that it has a better visual effect, with less noise and no ghost shadows appearing in the dehazed image. Also, the edges of the buildings are well preserved. Also, owing to the fact that the polarized part of the direct light is partly preserved in the algorithm, the details of the objects close to the observer are clearer. We use histograms to show the improvement of our method, as shown in Figs. 2(e) and 2(f). Note that the histograms of Figs. 2(a)–2(c) are almost the same, so we take the histogram of Fig. 2(a) as an example to compare with that of Fig. 2(d). From the histograms we can see that the distribution of gray level in Fig. 2(d) is much broader than that of Fig. 2(a), which implies that more details can be displayed in Fig. 2(d).

Figure 2.(a)–(c) are the images directly snapped by the black-and-white industrial camera, where the rotating angle of the polarizer is set to 0°, 45°, and 90°, respectively. (d) is the dehazed image using our algorithm without any additional image-processing algorithms. (e) and (f) are the histograms of (a) and (c), respectively.

In order to compare our dehazing algorithm with Schechner’s, we purposely set the rotating angles as 0° and 90°, which are exactly the “darkest” angle and the “brightest” one, respectively. Therefore, we dehaze the image using Schechner’s algorithm based on Figs. 2(a) and 2(c). The result is shown in Fig. 3. Similarly, no additional image-processing algorithm is used. The dehazed image also shows great improvement compared with the raw images. However, if we look at the image carefully, we will find that in the sky area there exists some noise, and at the edges of the buildings there are many dark noise spots. The former problem is mainly because of the error of the camera itself and the latter one may be caused by the mismatch of the two raw images. The same problems also happened in ^[18] and are comprehensively discussed in ^[11]. Although they have handled these problems well by using several image-processing algorithms, apparently our method is much simpler. In addition, the details of the objects close to the observer are not as clear as in Fig. 2(d).

Figure 3.Dehazed image based on Schechner’s algorithm without any additional image-processing algorithm.

In the following, we focus on color image dehazing. We use an ordinary color industrial camera together with a polarizer to snap the 0°, 45°, and 90° raw images, as shown in Figs. 4(a)–4(c). First, the difference between the color dehazing algorithm and the black-and-white dehazing algorithm should be discussed. As mentioned in ^[18], the color image dehazing algorithm is a little more complicated. We all know that in Rayleigh scattering, the scattering factor is 4 times inversely proportional to the wavelength. Although scattering in haze does not fit into the Rayleigh model, both the DOP and the intensity of air light are still dependent on the wavelength. A color image snapped by an ordinary color camera saves three color channels, including the red, green, and blue components, which we call the RGB components for short. Therefore, we can split the RGB apart and apply the dehazing algorithm separately. However, if we directly combine them as a color dehazed image, we will find that it suffers from terrible blueshift. So we demonstrate a simple way to preserve the original color weight of RGB values in the dehazed image, without using an image-processing algorithm. From Eq. (2), we can see that the air light becomes weaker as the distance decreases. Thus in the very short distance, the direct light from the objects is much stronger than the air light. Simultaneously, the scattering factor here can be ignored, and the RGB values here can be regarded as the real response from the objects. Based on this fact, in experiment, first, we get the black-and-white image of the original image $S_{0}$ . Second, we search the brightest pixel in the lower part of the black-and-white image and calculate the color weights (i.e., each monochromatic concentration among RGB colors) of this pixel in the color image. Here, the reason that we choose the brightest pixel is that it suffers less effect caused by the detecting error of the pixel itself. Third, we regulate the dehazed monochromatic image by the corresponding color weight. Finally, we combine the three monochromatic images and work out a color dehazed image, which can be seen in Fig. 4(d). We can see in the picture that the effect of the haze is relieved and the color is preserved.

Figure 4.(a)–(c) are the images directly snapped by the color industrial camera, where the rotating angle of the polarizer is set to 0°, 45°, and 90°, respectively. (d) is the dehazed image using our algorithm without any additional image-processing algorithm.

To further illustrate that the values of $ε_{1}$ and $ε_{2}$ can fit for many different levels of haze, we snap two groups of images, one of which contains slight haze, while the other contains dense haze, as shown in Figs. 5(a)–5(c) and Figs. 6(a)–6(c), respectively. Figure 5 primarily demonstrates the situation in which the haze is slight. The characteristic is that the polarization of the air light is large. This can be proved from Figs. 5(a) and 5(c), where we can see that the intensity changes a lot while the angle of the polarizer is rotated. Figure 5(d) is the dehazed image. We can see that the contour of the mountains and the electric wires are much clearer. In addition, the color is preserved well.

Figure 5.(a)–(c) are the images directly snapped in light hazy weather, where the rotating angle of the polarizer is set to 0°, 45°, and 90°, respectively. (d) is the dehazed image using our algorithm without any image-processing algorithm.

Figure 6.(a)–(c) are the images directly snapped in dense hazy weather, where the rotating angle of the polarizer is set to 0°, 45°, and 90°, respectively. (d) is the dehazed image using our algorithm without any image-processing algorithm.

Compared with Fig. 5, Figs. 6(a)–6(c) are snapped in much more dense hazy weather. The intensity difference can hardly be distinguished by the human eye. Figure 6(d) is the dehazed image. We can see that the quality of the dehazed image is much better. In particular, although the smoke in the former three images is different, the smoke in the dehazed image reflects only the property of the smoke in Fig. 6(c). If necessary, the dehazed images of Figs. 6(a) and 6(b) can also be obtained by a similar dehazing algorithm.

4. CONCLUSIONS

In this paper, we demonstrate a new dehazing algorithm by use of the first three Stokes parameters. To do that, three images are snapped by simply rotating the polarizer angle to 0°, 45°, and 90°. Experimental results demonstrate that this dehazing algorithm shows several advantages compared with others reported so far. First, the dehazed image handled by this algorithm suffers from little noise and no ghost shadows. Second, the details of the objects close to the observer can also be preserved very well. Third, a simple method to estimate the RGB color weight can be derived from this algorithm, which is very useful for preventing color shift. Fourth, the factors used in the algorithm are also verified experimentally to be applicable to other haze situations, meaning that this dehazing algorithm has some universality, and if a polarimeter can capture these three photos simultaneously, it might be very suitable for real-time dehazing.

References

[1] F. A. Sadjadi, C. S. L. Chun. Remote sensing using passive infrared Stokes parameters. Opt. Eng., 43, 2283-2291(2004).

[2] D. A. Lavigne, M. Breton, G. Fournier, J.-F. Charette, M. Pichette, V. Rivet, A.-P. Bernier. Target discrimination of man-made objects using passive polarimetric signatures acquired in the visible and infrared spectral bands. Proc. SPIE, 8160, 816007(2011).

[3] T. Novikova, A. Pierangelo, A. De Martino, A. Benali, P. Validire. Polarimetric imaging for cancer diagnosis and staging. Opt. Photon. News, 23, 26-33(2012).

[4] N. Lefaudeux, N. Lecochinski, S. Breugnot, P. Clemenceau. Compact and robust linear Stokes polarization camera. Proc. SPIE, 6972, 69720B(2008).

[5] J. Mudge, M. Virgen. Near-infrared simultaneous Stokes imaging polarimeter: integration, field acquisitions, and instrument error estimation. Proc. SPIE, 8160, 81600B(2011).

[6] T. Treibitz, Y. Y. Schechner. Active polarization descattering. IEEE Trans. Pattern Anal. Mach. Intell., 31, 385-399(2009).

[7] G. Myhre, W. L. Hsu, A. Peinado, C. LaCasse, N. Brock, R. A. Chipman, S. Pau. Liquid crystal polymer full-stokes division of focal plane polarimeter. Opt. Express, 20, 27393-27409(2012).

[8] K. A. Bachman, J. J. Peltzer, P. D. Flammer, T. E. Furtak, R. T. Collins, R. E. Hollingsworth. Spiral plasmonic nanoantennas as circular polarization transmission filters. Opt. Express, 20, 1308-1319(2012).

[9] J. D. Perreault. Triple Wollaston-prism complete-Stokes imaging polarimeter. Opt. Lett., 38, 3874-3877(2013).

[10] Y. Y. Schechner, S. G. Narasimhan, S. K. Nayar. Instant dehazing of images using polarization. Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 1, 325-332(2001).

[11] E. Namer, Y. Y. Schechner. Advanced visibility improvement based on polarization filtered images. Proc. SPIE, 5888, 588805(2005).

[12] S. Shwartz, E. Namer, Y. Y. Schechner. Blind haze separation. Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, 2, 1984-1991(2006).

[13] E. Namer, S. Shwartz, Y. Y. Schechner. Skyless polarimetric calibration and visibility enhancement. Opt. Express, 17, 472-493(2009).

[14] J. Mudge, M. Virgen. Real time polarimetric dehazing. Appl. Opt., 52, 1932-1938(2013).

[15] L. Schaul, C. Fredembach, S. Süsstrunk. Color image dehazing using the near-infrared. 16th IEEE International Conference on Image Processing (ICIP), 1629-1632(2009).

[16] K. M. He, J. Sun, X. Tang. Single image haze removal using dark channel prior. IEEE Trans. Pattern Anal. Mach. Intell., 33, 2341-2353(2011).

[17] K. Nishino, L. Kratz, S. Lombardi. Bayesian defogging. Int. J. Comput. Vis., 98, 263-278(2012).

[18] Y. Y. Schechner, S. G. Narasimhan, K. Nayar. Polarization-based vision through haze. Appl. Opt., 42, 511-525(2003).