Skip to main content
SpringerLink
Log in

A guiding teaching and dual adversarial learning framework for a single image dehazing

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

In most existing deep learning-based image dehazing methods, the haze-free source images are only used as the ground truth for the design of the loss function, whereas the guiding role that the source image should play on different feature levels has been ignored. This will result in a sub-optimal dehazing output. To address this issue, inspired by the knowledge distillation, a guiding teaching framework is designed for single image dehazing in an end-to-end manner, where the features of the haze-free source image at different levels are completely used to promoting the restoration of the hazy image. Specifically, the framework consists of a two-stream convolutional neural network termed teacher stream (TS) and student stream (SS), respectively. The input of the former is a haze-free image while the output is the desired image after reconstruction. The input of the latter is the hazy image, and the output is the restored image. Moreover, a dual adversarial strategy is designed to further improve the ability of SS to imitate teacher stream. In this process, the output results of SS are divided into two categories according to their hazy intensity levels. Then a thick light discriminator is introduced and made against the SS pit, such that the images with better dehazing effects can be used to deal with the ones poorly dehazed. A second discriminator termed light clear discriminator (LCD) is further introduced and a minimax game between the LCD and the SS is defined to drive the final result produced by SS closer to the reconstruction result of the TS. Experimental results show that the proposed method outperforms several latest methods applied to both artificial hazy images and the hazy images from the real scene.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5

Similar content being viewed by others

References

  1. Algabri, M., Mathkour, H., Bencherif, M.A., Alsulaiman, M., Mekhtiche, M.A.: Towards deep object detection techniques for phoneme recognition. IEEE Access 8, 54663–54680 (2020)

    Article  Google Scholar 

  2. Ba, L.J., Caruana, R.: Do deep nets really need to be deep? In: Proceedings of the 27th International Conference on Neural Information Processing Systems, vol 2, pp 2654–2662 (2014)

  3. Berman, D., Treibitz, T., Avidan, S.: Non-local image dehazing. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1674–1682. Las Vegas, NV (2016)

  4. Cai, B., Xu, X., Jia, K., Qing, C., Tao, D.: Dehazenet: an end-to-end system for single image haze removal. IEEE Trans. Image Process. 25(11), 5187–5198 (2016)

    Article  MathSciNet  MATH  Google Scholar 

  5. Chen, D., He, M., Fan, Q., Liao, J., Zhang, L., Hou, D., Yuan, L., Hua, G.: Gated context aggregation network for image dehazing and deraining. In: IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 1375–1383. IEEE (2019)

  6. Chen, J., Chen, J., Chao, H., Yang, M.: Image blind denoising with generative adversarial network based noise modeling. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3155–3164. IEEE, Salt Lake City, UT (2018)

  7. Chen, Z., Hu, Z., Sheng, B., Li, P., Kim, J., Wu, E.: Simplified non-locally dense network for single-image dehazing. Vis. Comput. 36(2), 2189–2200 (2020)

    Article  Google Scholar 

  8. Choi, L.K., You, J., Bovik, A.C.: Referenceless prediction of perceptual fog density and perceptual image defogging. IEEE Trans. Image Process. 24(11), 3888–3901 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  9. Dong, Y., Liu, Y., Zhang, H., Chen, S., Qiao, Y.: Fd-gan: generative adversarial networks with fusion-discriminator for single image dehazing. arXiv:2001.06968 (2020)

  10. Engin, D., Genc, A., Ekenel, H.K.: Cycle-dehaze: enhanced cyclegan for single image dehazing. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 938–9388. IEEE, Venice (2018)

  11. Feng, J., Wu, X., Chen, J., Zhang, X., Tang, X., Li, D.: Joint multilayer spatial–spectral classification of hyperspectral images based on CNN and convlstm. In: IGARSS 2019–2019 IEEE International Geoscience and Remote Sensing Symposium, pp. 588–591. IEEE, Japan (2019)

  12. Ge, S., Zhao, S., Li, C., Li, J.: Low-resolution face recognition in the wild via selective knowledge distillation. IEEE Trans. Image Process. 28(4), 2051–2062 (2019)

    Article  MathSciNet  Google Scholar 

  13. Ge, S., Zhao, S., Li, C., Zhang, Y., Li, J.: Efficient low-resolution face recognition via bridge distillation. IEEE Trans. Image Process. 29(4), 6898–6980 (2020)

    Article  MATH  Google Scholar 

  14. Guo, F., Zhao, X., Tang, J., Huipeng, L.L., Zou, B.: Single image dehazing based on fusion strategy. Neurocomputing 378, 9–23 (2020)

    Article  Google Scholar 

  15. Ha, E., Lim, H., Yu, S., Paik, J.: Low-light image enhancement using dual convolutional neural networks for vehicular imaging systems. In: 2020 IEEE International Conference on Consumer Electronics (ICCE), pp. 1–2. IEEE, Las Vegas, NV (2020)

  16. He, K., Sun, J., Tang, X.: Single image haze removal using dark channel prior. IEEE Trans. Pattern Anal. Mach. Intell. 33(12), 2341–2353 (2011)

    Article  Google Scholar 

  17. Heo, B., Kim, J., Yun, S., Park, H., Kwak, N., Choi, J.Y.: A comprehensive overhaul of feature distillation. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 1921–1930. IEEE, Seoul, Korea (South) (2019)

  18. Hinton, G., Vinyals, O., Dean, J.: Distilling the knowledge in a neural network. arXiv:1503.02531 (2015)

  19. Hong, M., Xie, Y., Li, C., Qu, Y.: Distilling image dehazing with heterogeneous task imitation. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 3459–3468. IEEE, Seattle (2020)

  20. Ignatov, A., Kobyshev, N., Timofte, R., Vanhoey, K.: DSLR-quality photos on mobile devices with deep convolutional networks. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 3297–3305. IEEE, Venice (2017)

  21. Justin, J., Alexandre, A., Li, F.F.: Perceptual losses for real-time style transfer and super-resolution. In: 2016 European Conference on Computer Vision (ECCV), pp. 694–711 (2016)

  22. Khmag, A., Al-Haddad, S.A.R., Ramli, A.R., Kalantar, B.: Single image dehazing using second-generation wavelet transforms and the mean vector l2-norm. Vis. Comput. 34(5), 675–688 (2018)

    Article  Google Scholar 

  23. Kupyn, O., Budzan, V., Mykhailych, M., Mishkin, D., Matas, J.: Deblurgan: blind motion deblurring using conditional adversarial networks. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8183–8192. IEEE, Salt Lake City, UT (2018)

  24. Ledig, C., Theis, L., Huszár, F., Caballero, J., Cunningham, A., Acosta, A., Aitken, A., Tejani, A., Totz, J., Wang, Z., Shi, W.: Photo-realistic single image super-resolution using a generative adversarial network. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 105–114. IEEE, Honolulu, HI (2017)

  25. Li, B., Peng, X., Wang, Z., Xu, J., Feng, D.: An all-in-one network for dehazing and beyond. ICCV 2017, 4770–4778 (2017a)

    Google Scholar 

  26. Li, B., Peng, X., Wang, Z., Xu, J., Feng, D.: Aod-net: All-in-one dehazing network. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 4780–4788 (2017b)

  27. Li, B., Ren, W., Fu, D., Tao, D., Feng, D., Zeng, W., Wang, Z.: Benchmarking single-image dehazing and beyond. IEEE Trans. Image Process. 28(1), 492–505 (2019)

    Article  MathSciNet  MATH  Google Scholar 

  28. Li, H., He, X., Tao, D., Tang, Y., Wang, R.: Joint medical image fusion, denoising and enhancement via discriminative low-rank sparse dictionaries learning. Pattern Recognit. 79, 130–146 (2018a)

    Article  Google Scholar 

  29. Li, H., He, X., Yu, Z., Luo, J.: Noise-robust image fusion with low-rank sparse decomposition guided by external patch prior. Inf. Sci. 523, 14–37 (2020a)

    Article  MathSciNet  MATH  Google Scholar 

  30. Li, H., Wang, Y., Yang, Z., Wang, R., Li, X., Tao, D.: Discriminative dictionary learning-based multiple component decomposition for detail-preserving noisy image fusion. IEEE Trans. Instrum. Meas. 69(4), 1082–1102 (2020b)

    Article  Google Scholar 

  31. Li, Q., Bi, D., Xu, Y., Zha, Y.: Haze degraded image scene rendition. Acta Autom. Sin. 40(4), 744–750 (2014)

    Google Scholar 

  32. Li, R., Pan, J., Li, Z., Tang, J.: Single image dehazing via conditional generative adversarial network. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8202–8211. Salt Lake City, UT (2018b)

  33. Lim, B., Son, S., Kim, H., Nah, S., Lee, K.M.: Enhanced deep residual networks for single image super-resolution. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1132–1140. IEEE, Honolulu, HI (2017)

  34. Liu, R., Fan, X., Hou, M., Jiang, Z., Luo, Z., Zhang, L.: Learning aggregated transmission propagation networks for haze removal and beyond. IEEE Trans. Neural Netw. Learn. Syst. 30(10), 2973–2986 (2019a)

    Article  Google Scholar 

  35. Liu, X., Ma, Y., Shi, Z., Chen, J.: Griddehazenet: attention-based multi-scale network for image dehazing. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 7313–7322. IEEE, Seoul (2019b)

  36. Mannos, J., Sakrison, D.: The effects of a visual fidelity criterion of the encoding of images. IEEE Trans. Inf. Theory 20(4), 525–536 (1974)

    Article  MATH  Google Scholar 

  37. Mehta, A., Sinha, H., Narang, P., Mandal, M.: Hidegan: A hyperspectral-guided image dehazing gan. In: The IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 846–856 (2020)

  38. Meng, G., Wang, Y., Duan, J., Xiang, S., Pan, C.: Efficient image dehazing with boundary constraint and contextual regularization. In: 2013 IEEE International Conference on Computer Vision (ICCV), pp. 617–624. NSW, Sydney (2013)

  39. Nah, S., Kim, T.H., Lee, K.M.: Deep multi-scale convolutional neural network for dynamic scene deblurring. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 257–265. IEEE, Honolulu, HI (2017)

  40. Pan, Y., He, F., Yu, H.: A novel enhanced collaborative autoencoder with knowledge distillation for top-n recommender systems. Neurocomputing 332, 137–148 (2019)

    Article  Google Scholar 

  41. Qin, X., Wang, Z., Bai, Y., Xie, X.: FFA-net: feature fusion attention network for single image dehazing. Proc. AAAI Conf. Artif. Intell. 34, 11908–11915 (2020)

    Google Scholar 

  42. Ren, W., Liu, S., Zhang, H., Pan, J., Cao, X., Yang, M.H.: Single image dehazing via multi-scale convolutional neural networks. In: 2016 European Conference on Computer Vision (ECCV), vol. 9906, pp. 154–169 (2016)

  43. Ren, W., Ma, L., Zhang, J., Pan, J., Cao, X., Liu, W., Yang, M.H.: Gated fusion network for single image dehazing. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3253–3261. Salt Lake City, UT (2018)

  44. Romero, A., Ballas, N., Kahou, S.E., Chassang, A., Gatta, C., Bengio, Y.: Fitnets: hints for thin deep nets. arXiv:1412.6550 (2014)

  45. Ronneberger, O., Fischer, P., Brox, T.: U-net: Convolutional networks for biomedical image segmentation. In: Medical Image Computing and Computer-Assisted Intervention—MICCAI 2015, pp. 234–241. Springer International Publishing (2015)

  46. Shao, Y., Li, L., Ren, W., Gao, C., Sang, N.: Domain adaptation for image dehazing. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 2805–2814. IEEE, Seattle (2020)

  47. Song, Y., Li, J., Wang, X., Chen, X.: Single image dehazing using ranking convolutional neural network. IEEE Trans. Multimed. 20(6), 1548–1560 (2018)

    Article  Google Scholar 

  48. Swami, K., Das, S.K.: Candy: Conditional adversarial networks based fully end-to-end system for single image haze removal. In: 2018 24th International Conference on Pattern Recognition (ICPR), pp. 3061–3067. Beijing (2018)

  49. Tang, G., Zhao, L., Jiang, R., Zhang, X.: Single image dehazing via lightweight multi-scale networks. In: 2019 IEEE International Conference on Big Data (Big Data), pp. 5062–5069. IEEE, Los Angeles, CA (2019)

  50. Tong, S., Dong, G., Wei, Z., Chunhua, S., Tao, M.: Regularizing proxies with multi-adversarial training for unsupervised domain-adaptive semantic segmentation. arXiv:1907.12282 (2019)

  51. Wang, J.B., He, N., Zhang, L.L., Lu, K.: Single image dehazing with a physical model and dark channel prior. Neurocomputing 149, 718–728 (2015)

    Article  Google Scholar 

  52. Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)

    Article  Google Scholar 

  53. Wu, H., Liu, J., Xie, Y., Qu, Y., Ma, L.: Knowledge transfer dehazing network for nonhomogeneous dehazing. In: 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1975–1983. IEEE, Seattle (2020)

  54. Xie, M., Zhou, Z., Zhang, Y.: Joint framework for image fusion and super-resolution via multicomponent analysis and residual compensation. IEEE Access 7, 174092–174107 (2019)

    Article  Google Scholar 

  55. Yang, C., Xie, L., Qiao, S., Yuille, A.: Knowledge distillation in generations: more tolerant teachers educate better students. arXiv:1805.05551 (2018a)

  56. Yang, D., Sun, J.: Proximal dehaze-net: a prior learning-based deep network for single image dehazing. In: 2018 European Conference on Computer Vision, vol. 11211, pp. 729–746 (2018)

  57. Yang, F., Zhang, Q.: Depth aware image dehazing. Vis. Comput. https://doi.org/10.1007/s00371-021-02089-3 (2021)

  58. Yang, X., Xu, Z., Luo, J.: Towards perceptual image dehazing by physics-based disentanglement and adversarial training. In: 2018 AAAI, New Orleans, pp. 7478–7485 (2018b)

  59. Yim, J., Joo, D., Bae, J., Kim, J.: A gift from knowledge distillation: Fast optimization, network minimization and transfer learning. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7130–7138. IEEE, Honolulu, HI (2017)

  60. Zagoruyko, S., Komodakis, N.: Paying more attention to attention: improving the performance of convolutional neural networks via attention transfer. In: 5th International Conference on Learning Representations, ICLR 2017 (2017)

  61. Zhang, H., Patel, V.M.: Densely connected pyramid dehazing network. In: 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3194–3203. Salt Lake City, UT (2018)

  62. Zhang, H., Sindagi, V., Patel, V.M.: Image de-raining using a conditional generative adversarial network. IEEE Trans. Circ. Syst. Video Technol. 1–1 (2019a)

  63. Zhang, J., Cao, Y., Wang, Y., Wen, C., Chen, C.W.: Fully point-wise convolutional neural network for modeling statistical regularities in natural images. In: 2018 ACM Multimedia Conference (2018)

  64. Zhang, S., He, F.: Drcdn: learning deep residual convolutional dehazing networks. Vis. Comput. https://doi.org/10.1007/s00371-019-01774-8 (2020)

  65. Zhang, S., He, F., Ren, W.: NLDN: non-local dehazing network for dense haze removal. Neurocomputing 410, 363–373 (2020a)

    Article  Google Scholar 

  66. Zhang, S., He, F., Ren, W., Yao, J.: Joint learning of image detail and transmission map for single image dehazing. Vis. Comput. 36(2), 305–316 (2020b)

    Article  Google Scholar 

  67. Zhang, W., Liu, Y., Dong, C., Qiao, Y.: Ranksrgan: generative adversarial networks with ranker for image super-resolution. In: 2019 IEEE/CVF International Conference on Computer Vision (ICCV), pp. 3096–3105. IEEE, Seoul, Korea (South) (2019b)

  68. Zhang, Y., Yang, M., Li, N., Yu, Z.: Analysis-synthesis dictionary pair learning and patch saliency measure for image fusion. Signal Process. 167, 107327 (2020c)

    Article  Google Scholar 

  69. Zheng, M., Qi, G., Zhu, Z., Li, Y., Wei, H., Liu, Y.: Image dehazing by an artificial image fusion method based on adaptive structure decomposition. IEEE Sens. J. 20(4), 8062–8072 (2020)

    Article  Google Scholar 

  70. Zhu, J.Y., Park, T., Isola, P., Efros, A.A.: Unpaired image-to-image translation using cycle-consistent adversarial networks. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 2242–2251. IEEE, Venice (2017)

  71. Zhu, Z., Wei, H., Hu, G., Li, Y., Qi, G., Mazur, N.: A novel fast single image dehazing algorithm based on artificial multi-exposure image fusion. IEEE Trans. Instrum. Meas. (2020). https://doi.org/10.1109/TIM.2020.3024335

    Article  Google Scholar 

Download references

Acknowledgements

This work is partly supported by the Science and Technology Project of Yunnan Power Grid Co., Ltd. (No. YNKJXM 20190729) and the National Key Research and Development Plan Project (Nos. 2018YFC0830105 and 2018YFC0830100).

Author information

Authors and Affiliations

  1. College of Land Resource Engineering, Kunming University of Science and Technology, Kunming, 650500, China

    Zhengyun Fang

  2. Electric Power Research Institute of Yunnan Power Grid Co., Ltd., Kunming, 650217, China

    Ming Zhao, Meiyu Li & Yong Yang

  3. Faculty of Information Engineering and Automation, Kunming University of Science and Technology, Kunming, 650500, China

    Zhengtao Yu

  4. Yunnan Key Laboratory of Artificial Intelligence, Kunming University of Science and Technology, Kunming, 650500, China

    Zhengtao Yu

Authors
  1. Zhengyun Fang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ming Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhengtao Yu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Meiyu Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Yong Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zhengtao Yu.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Fang, Z., Zhao, M., Yu, Z. et al. A guiding teaching and dual adversarial learning framework for a single image dehazing. Vis Comput 38, 3563–3575 (2022). https://doi.org/10.1007/s00371-021-02184-5

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-021-02184-5

Keywords

Search

Navigation