Abstract
Accurate determination of weather from images or video is of prime significance in applications such as autonomous vehicles drive or atmospheric pollution estimation. In contrast to image classification approaches generally adopted in the literature based on neural networks acting directly on the input, we propose a novel joint learning approach combining features extracted from the input image, sensitive to the human visual system (HVS) and those generated by the CNN model trained on benchmark datasets. The features work in a joint collaboration that is able to detect the presence of weather features during learning. The novel approach outperforms many state-of-the-art methods which use only CNN-based extracted features in order to classify the images. Experimental results with publicly available benchmark datasets establish the robustness and effectiveness of the proposed method in multi-weather classification.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
An, J., Chen, Y., Shin, H.: Weather classification using convolutional neural networks. In: 2018 International SoC Design Conference (ISOCC), pp. 245–246 (2018). https://doi.org/10.1109/ISOCC.2018.8649921
Ancuti, C., Ancuti, C.O., De Vleeschouwer, C.: D-hazy: a dataset to evaluate quantitatively dehazing algorithms. In: 2016 IEEE International Conference on Image Processing (ICIP), pp. 2226–2230 (2016). https://doi.org/10.1109/ICIP.2016.7532754
Bhandari, H., Palit, S., Chowdhury, S., Dey, P.: Can a camera tell the weather? In: 2021 36th International Conference on Image and Vision Computing New Zealand (IVCNZ), pp. 1–6 (2021). https://doi.org/10.1109/IVCNZ54163.2021.9653246
Chen, Z., Yang, F., Lindner, A., Barrenetxea, G., Vetterli, M.: Howis the weather: automatic inference from images. In: 2012 19th IEEE International Conference on Image Processing, pp. 1853–1856 (2012). https://doi.org/10.1109/ICIP.2012.6467244
Comaniciu, D., Meer, P.: Mean shift: a robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24(5), 603–619 (2002). https://doi.org/10.1109/34.1000236
Derpanis, K.G., Lecce, M., Daniilidis, K., Wildes, R.P.: Dynamic scene understanding: the role of orientation features in space and time in scene classification. In: 2012 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1306–1313 (2012). https://doi.org/10.1109/CVPR.2012.6247815
Elhoseiny, M., Huang, S., Elgammal, A.: Weather classification with deep convolutional neural networks. In: 2015 IEEE International Conference on Image Processing (ICIP), pp. 3349–3353 (2015). https://doi.org/10.1109/ICIP.2015.7351424
Groen, I.I.A., Ghebreab, S., Prins, H., Lamme, V.A.F., Scholte, H.S.: From image statistics to scene gist: evoked neural activity reveals transition from low-level natural image structure to scene category. J. Neurosci. 33, 18814–18824 (2013)
Kurihata, H., et al.: Rainy weather recognition from in-vehicle camera images for driver assistance. In: IEEE Proceedings Intelligent Vehicles Symposium 2005, pp. 205–210 (2005). https://doi.org/10.1109/IVS.2005.1505103
Laffont, P.Y., Ren, Z., Tao, X., Qian, C., Hays, J.: Transient attributes for high-level understanding and editing of outdoor scenes. ACM Trans. Graph. 33, 149:1–149:11 (2014). https://doi.org/10.1145/2601097.2601101
Li, Z., Li, Y., Zhong, J., Chen, Y.: Multi-class weather classification based on multi-feature weighted fusion method. IOP Conf. Ser. Earth Environ. Sci. 558(4), 042038 (2020). https://doi.org/10.1088/1755-1315/558/4/042038
Lin, D., Lu, C., Liao, R., Jia, J.: Learning important spatial pooling regions for scene classification. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp. 3726–3733 (2014). https://doi.org/10.1109/CVPR.2014.476
Liu, Y.F., Jaw, D., Yeh, W.C., Hwang, J.N.: DesnowNet: context-aware deep network for snow removal. IEEE Trans. Image Process. 27(6), 3064–3073 (2018). https://doi.org/10.1109/TIP.2018.2806202
Lu, C., Lin, D., Jia, J., Tang, C.K.: Two-class weather classification. IEEE Trans. Pattern Anal. Mach. Intell. 39(12), 2510–2524 (2017). https://doi.org/10.1109/TPAMI.2016.2640295
Narasimhan, S., Nayar, S.: Shedding light on the weather. In: 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition 2003 Proceedings, vol. 1, p. I (2003). https://doi.org/10.1109/CVPR.2003.1211417
Roser, M., Moosmann, F.: Classification of weather situations on single color images. In: 2008 IEEE Intelligent Vehicles Symposium, pp. 798–803 (2008). https://doi.org/10.1109/IVS.2008.4621205
Shen, L., Tan, P.: Photometric stereo and weather estimation using internet images. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1850–1857 (2009). https://doi.org/10.1109/CVPR.2009.5206732
Song, H., Chen, Y., Gao, Y.: Weather condition recognition based on feature extraction and K-NN. In: Sun, F., Hu, D., Liu, H. (eds.) Foundations and Practical Applications of Cognitive Systems and Information Processing. AISC, vol. 215, pp. 199–210. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-642-37835-5_18
Yan, X., Luo, Y., Zheng, X.: Weather recognition based on images captured by vision system in vehicle. In: Yu, W., He, H., Zhang, N. (eds.) ISNN 2009. LNCS, vol. 5553, pp. 390–398. Springer, Heidelberg (2009). https://doi.org/10.1007/978-3-642-01513-7_42
Zhang, Z., Ma, H.: Multi-class weather classification on single images. In: 2015 IEEE International Conference on Image Processing (ICIP), pp. 4396–4400 (2015). https://doi.org/10.1109/ICIP.2015.7351637
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Bhandari, H., Chowdhury, S., Palit, S. (2023). A Hybrid Human-Machine System for Image-Based Multi-weather Detection. In: Yan, W.Q., Nguyen, M., Stommel, M. (eds) Image and Vision Computing. IVCNZ 2022. Lecture Notes in Computer Science, vol 13836. Springer, Cham. https://doi.org/10.1007/978-3-031-25825-1_23
Download citation
DOI: https://doi.org/10.1007/978-3-031-25825-1_23
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-25824-4
Online ISBN: 978-3-031-25825-1
eBook Packages: Computer ScienceComputer Science (R0)