Abstract
Haze classification plays a crucial role in air quality and visibility assessment. In contrast to traditional image classification, haze classification requires the classifier to capture the characteristics of different levels of haze. However, existing methods primarily focus on feature extraction while neglecting the interference of background information. To address this issue, this paper proposes a hard attention infused network (HAINet) for haze classification, consisting of an unsupervised segmentation module (USM) and a hybrid information fusion module (HIF). The USM is used to extract haze area information in an unsupervised manner, generating various forms of haze images. The HIA selects different various forms of haze images, as a hard attention mechanism, to reduce the impact of background and improve classification performance. We conduct experiments on two datasets, Hazel-level and Haze-Wild, in terms of performance comparison, ablation study, and case studies. The results show that our method effectively reduces the impact of background noise in haze images and consistently improves the classification performance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Alami, S., Ezzine, A., Elhassouni, F.: Local fog detection based on saturation and RGB-correlation. In: 2016 13th International Conference on Computer Graphics, Imaging and Visualization (CGiV), pp. 1–5. IEEE (2016)
Chakma, A., Vizena, B., Cao, T., Lin, J., Zhang, J.: Image-based air quality analysis using deep convolutional neural network. In: IEEE ICIP, pp. 3949–3952 (2017)
Chen, Y., Wang, J., Li, S., Wang, W.: Multi-feature based foggy image classification. IOP Conf. Ser.: Earth Environ. Sci. 234, 012089 (2019)
Dosovitskiy, A., Beyer, L., Kolesnikov, A., et al.: An image is worth 16x16 words: transformers for image recognition at scale. arXiv preprint arXiv:2010.11929
Godard, C., Mac Aodha, O., Firman, M., Brostow, G.J.: Digging into self-supervised monocular depth prediction (2019)
Guan, Q.L., Zheng, Y., Meng, L., Dong, L.Q., Hao, Q.: Improving the generalization of visual classification models across IoT cameras via cross-modal inference and fusion. IEEE Internet Things J. PP, 1 (2023)
Guo, L., et al.: Haze image classification method based on Alexnet network transfer model. J. Phys.: Conf. Ser. 1176, 032011 (2019). IOP Publishing (2019)
He, K., Sun, J., Tang, X.: Single image haze removal using dark channel prior. IEEE Trans. Pattern Anal. Mach. Intell. 33, 2341–2353 (2010)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the CVPR, pp. 770–778 (2016)
Jeong, K., Choi, K., Kim, D., Song, B.C.: Fast fog detection for de-fogging of road driving images. IEICE Trans. Inf. Syst. E101.D, 473–480 (2018)
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)
Li, J., Ma, H., Li, X., Meng, L., Meng, X.: Unsupervised contrastive masking for visual haze classification. In: Proceedings of ICMR (2022)
Li, R., Liu, X., Li, X.: Estimation of the PM\(_{2. 5}\) pollution levels in Beijing based on nighttime light data from the defense meteorological satellite program-operational Linescan system. Atmosphere 6(5), 607–622 (2015)
Li, X., Wu, L., Chen, X., Meng, L., Meng, X.: DSE-Net: artistic font image synthesis via disentangled style encoding. In: 2022 IEEE International Conference on Multimedia and Expo (ICME), pp. 1–6. IEEE (2022)
Li, X., Wu, L., Wang, C., Meng, L., Meng, X.: Compositional zero-shot artistic font synthesis. In: Proceedings of IJCAI (2023)
Li, X., Ma, H., Meng, L., Meng, X.: Comparative study of adversarial training methods for long-tailed classification. In: Proceedings of the 1st International Workshop on Adversarial Learning for Multimedia, pp. 1–7 (2021)
Li, X., Zheng, Y., Ma, H., Qi, Z., Meng, X., Meng, L.: Cross-modal learning using privileged information for long-tailed image classification. In: CVM (2023)
Li, Y., Huang, J., Luo, J.: Using user generated online photos to estimate and monitor air pollution in major cities. In: Proceedings of the 7th International Conference on Internet Multimedia Computing and Service, pp. 1–5 (2015)
Liu, C., Tsow, F., Zou, Y., Tao, N.: Particle pollution estimation based on image analysis. PLoS ONE 11(2), e0145955 (2016)
Liu, J., et al.: Prompt learning with cross-modal feature alignment for visual domain adaptation. In: CAAI (2022)
Liu, T., Qi, Z., Chen, Z., Meng, X., Meng, L.: Cross-training with prototypical distillation for improving the generalization of federated learning. In: ICME (2023)
Ma, H., Li, X., Meng, L., Meng, X.: Comparative study of adversarial training methods for cold-start recommendation. In: Proceedings of ADVM (2021)
Ma, H., Qi, Z., Dong, X., Li, X., Zheng, Y., Meng, X.M.L.: Cross-modal content inference and feature enrichment for cold-start recommendation. In: IJCNN (2023)
Ma, H., et al.: Exploring false hard negative sample in cross-domain recommendation. In: RecSys (2023)
Ma, H., et al.: Triple sequence learning for cross-domain recommendation. arXiv preprint arXiv:2304.05027 (2023)
Ma, J., Li, K., Han, Y., Yang, J.: Image-based air pollution estimation using hybrid convolutional neural network. In: ICPR, pp. 471–476. IEEE (2018)
Mao, J., Phommasak, U., Watanabe, S., Shioya, H.: Detecting foggy images and estimating the haze degree factor. J. Comput. Sci. Syst. Biol. 7(6), 226–228 (2014)
Mei, S., Li, H., Fan, J., Zhu, X., Dyer, C.R.: Inferring air pollution by sniffing social media. In: 2014 IEEE/ACM International Conference on Advances in Social Networks Analysis and Mining (ASONAM 2014), pp. 534–539. IEEE (2014)
Meng, L., et al.: Learning using privileged information for food recognition. In: Proceedings of the 27th ACM International Conference on Multimedia, pp. 557–565 (2019)
Meng, L., Feng, F., He, X., Gao, X., Chua, T.S.: Heterogeneous fusion of semantic and collaborative information for visually-aware food recommendation. In: Proceedings of MM (2020)
Qi, Z., Chen, X.: A novel density-based outlier detection method using key attributes. Intell. Data Anal. 26(6), 1431–1449 (2022)
Qi, Z., Wang, Y., Chen, Z., Wang, R., Meng, X., Meng, L.: Clustering-based curriculum construction for sample-balanced federated learning. In: Fang, L., Povey, D., Zhai, G., Mei, T., Wang, R. (eds.) Artificial Intelligence. CICAI 2022. LNCS, vol. 13606. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-20503-3_13
Rijal, N., Gutta, R.T., Cao, T., Lin, J., Bo, Q., Zhang, J.: Ensemble of deep neural networks for estimating particulate matter from images. In: ICIVC (2018)
Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad-CAM: visual explanations from deep networks via gradient-based localization. In: Proceedings of ICCV, pp. 618–626 (2017)
Sun, W., et al.: Sequential fusion of multi-view video frames for 3D scene generation. In: Fang, L., Povey, D., Zhai, G., Mei, T., Wang, R. (eds.) Artificial Intelligence. CICAI 2022. LNCS, vol. 13604. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-20497-5_49
Wang, H., Yuan, X., Wang, X., Zhang, Y., Dai, Q.: Real-time air quality estimation based on color image processing. In: 2014 IEEE Visual Communications and Image Processing Conference, pp. 326–329. IEEE (2014)
Wang, X., et al.: Feature enhancement and fusion for image-based particle matter estimation with F-MSE loss. In: IEEE ICIP, pp. 768–772. IEEE (2020)
Wang, Y., Li, X., Ma, H., Qi, Z., Meng, X., Meng, L.: Causal inference with sample balancing for out-of-distribution detection in visual classification. In: Fang, L., Povey, D., Zhai, G., Mei, T., Wang, R. (eds.) Artificial Intelligence. CICAI 2022. LNCS, vol. 13604. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-20497-5_47
Wang, Y., et al.: Meta-causal feature learning for out-of-distribution generalization. In: Karlinsky, L., Michaeli, T., Nishino, K. (eds.) Computer Vision – ECCV 2022 Workshops. ECCV 2022. LNCS, vol. 13806. Springer, Cham (2022). https://doi.org/10.1007/978-3-031-25075-0_36
Wang, Y., Qi, Z., Li, X., Liu, J., Meng, X., Meng, L.: Multi-channel attentive weighting of visual frames for multimodal video classification. In: IJCNN (2023)
Zhang, C., Yan, J., Li, C., Rui, X., Liu, L., Bie, R.: On estimating air pollution from photos using convolutional neural network. In: Proceedings of MM (2016)
Zhang, C., Yan, J., Li, C., Wu, H., Bie, R.: End-to-end learning for image-based air quality level estimation. Mach. Vis. Appl. 29(4), 601–615 (2018)
Zhang, Y., Sun, G., Ren, Q., Zhao, D.: Foggy images classification based on features extraction and SVM. In: Proceeding of 2013 International Conference on Software Engineering and Computer Science, pp. 14–142 (2013)
Zhang, Z., Ma, H., Fu, H., Liu, L., Zhang, C.: Outdoor air quality level inference via surveillance cameras. Mobile Inf. Syst. 2016, 1–10 (2016)
Zhang, Z., Ma, H., Fu, H., Wang, X.: Outdoor air quality inference from single image. In: International Conference on Multimedia Modeling (2015)
Zhao, X., Jiang, J., Feng, K., Wu, B., Luan, J., Ji, M.: The method of classifying fog level of outdoor video images based on convolutional neural networks. J. Indian Soc. Remote Sens. 49(9), 2261–2271 (2021)
Zhao, X., Zhang, T., Chen, W., Wu, W.: Image dehazing based on haze degree classification. In: CAC, pp. 4186–4191. IEEE (2020)
Acknowledgements
This work is supported by TaiShan Scholars Program (Grant no. tsqn202211289) and Excellent Youth Scholars Program of Shandong Province (Grant no. 2022HWYQ-048).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Li, J., Ma, H., Li, X., Qi, Z., Meng, X., Meng, L. (2023). Unsupervised Segmentation of Haze Regions as Hard Attention for Haze Classification. In: Lu, H., et al. Image and Graphics . ICIG 2023. Lecture Notes in Computer Science, vol 14358. Springer, Cham. https://doi.org/10.1007/978-3-031-46314-3_28
Download citation
DOI: https://doi.org/10.1007/978-3-031-46314-3_28
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-46313-6
Online ISBN: 978-3-031-46314-3
eBook Packages: Computer ScienceComputer Science (R0)