Skip to main content
Log in

MFFormer: multi-level boosted transformer expanded by feature interaction block

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

The objective of image dehazing algorithms is to extract latent clear image details from degraded images obscured by inhomogeneous haze, improving the quality of the degraded images. In the image dehazing task, transformer-based models demonstrate effective spatial aggregation capability through multi-head self-attention. However, those models tend to overlook the effective interaction between channel information and lack the utilization of feature information of other layers in the image reconstruction stage. This paper proposes a novel dehazing model called MFFormer, which comprises a feature interaction block (FIB) and a multi-level feature-boosted module (MFBF). Specifically, the FIB provides the model with global channel feature information and depth spatial feature information through channel interaction operation, allowing the model to efficiently model degraded regions of images. The MFBF integrates shallow feature information and deep feature information at different scales, enhancing the role of shallow features in the image reconstruction stage. Furthermore, we propose a content-guided contrastive regularization that focus on optimizing the model with shallow hidden features to recover more image details. Experimental results on synthetic and real-world datasets demonstrate that the proposed MFFormer achieves superior dehazing results with a smaller number of parameters compared to the state-of-the-art models.The code is released in https://github.com/Dudragon1/MFFormer.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Data Availability

No datasets were generated or analysed during the current study.

References

  1. Nayar, S.K., Narasimhan, S.G.: Vision in bad weather. In: Proceedings of the seventh IEEE international conference on computer vision. IEEE, 2: 820–827 (1999)

  2. Narasimhan, S.G., Nayar, S.K.: Vision and the atmosphere. Int. J. Comput. Vision 48, 233–254 (2002)

    Article  MATH  Google Scholar 

  3. McCartney, E.J.: Optics of the atmosphere: scattering by molecules and particles. New York, (1976)

  4. He, K., Sun, J., Tang, X.: Single image haze removal using dark channel prior. IEEE Trans. Pattern Anal. Mach. Intell. 33(12), 2341–2353 (2010)

    MATH  Google Scholar 

  5. Zhu, Q., Mai, J., Shao, L.: A fast single image haze removal algorithm using color attenuation prior. IEEE Trans. Image Process. 24(11), 3522–3533 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  6. Berman, D., Avidan, S.: Non-local image dehazing. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 1674-1682 (2016)

  7. Tang, K., Yang, J., Wang, J.: Investigating haze-relevant features in a learning framework for image dehazing. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2995–3000 (2014)

  8. Qu, Y., Chen, Y., Huang, J., et al.: Enhanced pix2pix dehazing network. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 8160–8168 (2019)

  9. Li, B., Peng, X., Wang, Z., et al.: Aod-net: All-in-one dehazing network. In: Proceedings of the IEEE international conference on computer vision. 4770–4778 (2017)

  10. Zhao, S., Zhang, L., Shen, Y., et al.: RefineDNet: a weakly supervised refinement framework for single image dehazing[J]. IEEE Trans. Image Process. 30, 3391–3404 (2021)

    Article  MATH  Google Scholar 

  11. Liu, X., Ma, Y., Shi, Z., et al.: Griddehazenet: Attention-based multi-scale network for image dehazing. In: Proceedings of the IEEE/CVF international conference on computer vision. 7314-7323 (2019)

  12. Qian, W., Zhou, C., Zhang, D.: FAOD-Net: a fast AOD-Net for dehazing single image[J]. Math. Probl. Eng. 2020, 1–11 (2020)

    MATH  Google Scholar 

  13. Su, Y.Z., He, C., Cui, Z.G., et al.: Physical model and image translation fused network for single-image dehazing[J]. Pattern Recogn. 142, 109700 (2023)

    Article  MATH  Google Scholar 

  14. Wang, N., Cui, Z., Su, Y., et al.: Multiscale supervision-guided context aggregation network for single image dehazing. IEEE Signal Process. Lett. 29, 70–74 (2021)

    Article  MATH  Google Scholar 

  15. Dong, H., Pan, J., Xiang, L., et al.: Multi-scale boosted dehazing network with dense feature fusion. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 2157-2167 (2020)

  16. Hu, G., Tan, A., He, L., et al.: Pyramid feature boosted network for single image dehazing. Int. J. Mach. Learn. Cybern. 14(6), 2099–2110 (2023)

    Article  MATH  Google Scholar 

  17. Ye, T., Jiang, M., Zhang, Y., et al.: Perceiving and modeling density is all you need for image dehazing. arxiv preprint arxiv:2111.09733, (2021)

  18. Song, Y., He, Z., Qian, H., et al.: Vision transformers for single image dehazing. IEEE Trans. Image Process. 32, 1927–1941 (2023)

    Article  MATH  Google Scholar 

  19. Qiu, Y., Zhang, K., Wang, C., et al.: MB-TaylorFormer: Multi-branch efficient transformer expanded by Taylor formula for image dehazing. In: Proceedings of the IEEE/CVF International Conference on Computer Vision. 12802-12813 (2023)

  20. Liu, J., Yuan, H., Yuan, Z., et al.: Visual transformer with stable prior and patch-level attention for single image dehazing[J]. Neurocomputing 551, 126535 (2023)

    Article  MATH  Google Scholar 

  21. Guo, C.L., Yan, Q., Anwar, S., et al.: Image dehazing transformer with transmission-aware 3d position embedding. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 5812-5820 (2022)

  22. Cui, Y., Knoll, A.: Exploring the potential of channel interactions for image restoration[J]. Knowl.-Based Syst. 282, 111156 (2023)

    Article  MATH  Google Scholar 

  23. Zamir, S.W., Arora, A., Khan, S.: Learning enriched features for real image restoration and enhancement. In: Computer Vision-ECCV, et al.: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXV 16. Springer International Publishing 2020, 492–511 (2020)

  24. Wu, H., Qu, Y., Lin, S., et al.: Contrastive learning for compact single image dehazing. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 10551-10560 (2021)

  25. Qin, X., Wang, Z., Bai, Y., et al.: FFA-Net: Feature fusion attention network for single image dehazing. In: Proceedings of the AAAI conference on artificial intelligence. 34(07): 11908-11915 (2020)

  26. Dong, J., Pan, J., Physics-based feature dehazing networks. In: Computer Vision-ECCV,: 16th European Conference, Glasgow, UK, August 23–28, 2020, Proceedings, Part XXX 16. Springer International Publishing 2020, 188–204 (2020)

  27. Wang, N., Cui, Z., Su, Y., et al.: Prior-guided multiscale network for single-image dehazing. IET Image Proc. 15(13), 3368–3379 (2021)

    Article  MATH  Google Scholar 

  28. Cui, Z., Wang, N., Su, Y., et al.: ECANet: enhanced context aggregation network for single image dehazing. SIViP 17(2), 471–479 (2023)

    Article  MATH  Google Scholar 

  29. Lan, Y., Cui, Z., Su, Y., et al.: Online knowledge distillation network for single image dehazing. Sci. Rep. 12(1), 14927 (2022)

    Article  MATH  Google Scholar 

  30. Tu, Z., Talebi, H., Zhang, H., et al.: Maxim: Multi-axis mlp for image processing. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition. 5769-5780 (2022)

  31. Dosovitskiy, A., Beyer, L., Kolesnikov, A., et al.: An image is worth 16x16 words: Transformers for image recognition at scale. arxiv preprint arxiv:2010.11929, (2020)

  32. Liu, Z., Lin, Y., Cao, Y., et al.: Swin transformer: Hierarchical vision transformer using shifted windows. In: Proceedings of the IEEE/CVF international conference on computer vision. 10012-10022 (2021)

  33. Park, N., Kim, S.: How do vision transformers work? arxiv preprint arxiv:2202.06709, (2022)

  34. Hu, J., Shen, L., Sun, G.: Squeeze-and-excitation networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 7132-7141 (2018)

  35. Chen, Z., He, Z., Lu, Z. M.: DEA-Net: Single image dehazing based on detail-enhanced convolution and content-guided attention. IEEE Transactions on Image Processing, (2024)

  36. Sun, H., Li, B., Dan, Z., et al.: Multi-level feature interaction and efficient non-local information enhanced channel attention for image dehazing[J]. Neural Netw. 163, 10–27 (2023)

    Article  MATH  Google Scholar 

  37. Ronneberger, O., Fischer, P., Brox, T., U-net: Convolutional networks for biomedical image segmentation. Medical image computing and computer-assisted intervention-MICCAI,: 18th international conference, Munich, Germany, October 5–9, 2015, proceedings, part III 18. Springer International Publishing 2015, 234–241 (2015)

  38. Lin, T.Y., Dollár, P., Girshick, R., et al.: Feature pyramid networks for object detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition. 2117-2125 (2017)

  39. Romano, Y., Elad, M.: Boosting of image denoising algorithms. SIAM J. Imag. Sci. 8(2), 1187–1219 (2015)

    Article  MathSciNet  MATH  Google Scholar 

  40. Li, B., Ren, W., Fu, D., et al.: Benchmarking single-image dehazing and beyond. IEEE Trans. Image Process. 28(1), 492–505 (2018)

    Article  MathSciNet  MATH  Google Scholar 

  41. Ancuti, C.O., Ancuti, C., Sbert, M., Dense-haze: A benchmark for image dehazing with dense-haze and haze-free images[C], et al.: IEEE international conference on image processing (ICIP). IEEE 2019, 1014–1018 (2019)

  42. Ancuti, C.O., Ancuti, C., Timofte, R.: NH-HAZE: An image dehazing benchmark with non-homogeneous hazy and haze-free images. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition workshops. 444-445 (2020)

  43. Bai, H., Pan, J., Xiang, X., et al.: Self-guided image dehazing using progressive feature fusion. IEEE Trans. Image Process. 31, 1217–1229 (2022)

    Article  MATH  Google Scholar 

  44. Lu, L.P., Xiong, Q., Chu, D.F., et al.: MixDehazeNet: Mix structure block for image dehazing network. arxiv preprint arxiv:2305.17654, (2023)

  45. Luo, Z., Gustafsson, F.K., Zhao, Z., et al.: Photo-Realistic Image Restoration in the Wild with Controlled Vision-Language Models. arxiv preprint arxiv:2404.09732, (2024)

Download references

Author information

Authors and Affiliations

Authors

Contributions

X.G. contributed to the conception of the study; D.H. performed the experiments and contributed significantly to analysis and manuscript preparation; Z.Z. helped perform the analysis of implementation methods. All authors reviewed the manuscript.

Corresponding author

Correspondence to Xiaolin Gong.

Ethics declarations

Conflict of interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Gong, X., Du, H. & Zheng, Z. MFFormer: multi-level boosted transformer expanded by feature interaction block. SIViP 19, 96 (2025). https://doi.org/10.1007/s11760-024-03665-5

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11760-024-03665-5

Keywords

Navigation