Abstract
Weather classification from single images plays an important role in many outdoor computer vision applications, while it has not been thoroughly studied. Despite existing methods have achieved great success under the supervision of weather labels, they are hardly applicable to real-world applications due to the reliance on extensive human-annotated data. In this paper, we make the first attempt to view weather classification as an unsupervised task, i.e., classifying weather conditions from single images without labels. Specifically, a two-step unsupervised approach, where weather feature learning and weather clustering are decoupled, is proposed to automatically group images into weather clusters. In weather feature learning, we employ a self-supervised task to learn the semantically meaningful weather features. To ensure weather features invariant to image transformations and extract discriminative weather features, we also introduce online triplet mining into the task. In weather clustering, a learnable clustering method is designed by mining the nearest neighbors as a prior and enforcing the consistent predictions of each image and it’s nearest neighbors. Experimental results on two public benchmark datasets indicate that our approach achieves promising performance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Lu, C., Lin, D., Jia, J., Tang, C.-K.: Two-class weather classification. IEEE Trans. Pattern Anal. Mach. Intell. 39(12), 2510–2524 (2017)
Zhao, B., Li, X., Lu, X., Wang, Z.: A CNN-RNN architecture for multi-label weather recognition. Neurocomputing 322, 47–57 (2018)
Ohn-Bar, E., Trivedi, M.M.: Are all objects equal? deep spatio-temporal importance prediction in driving videos. Pattern Recognit. 64, 425–436 (2017)
Loncomilla, P., Ruiz-del-Solar, J., Martínez, L.: Object recognition using local invariant features for robotic applications: a survey. Pattern Recognit. 60, 499–514 (2016)
Zhang, Z. Ma, H.: Multi-class weather classification on single images. In: 2015 IEEE International Conference on Image Processing (ICIP), pp. 4396–4400 (2015)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances Neural Information Processing System, vol. 25, pp. 1097–1105 (2012)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. ArXiv Prepr. ArXiv14091556 (2014)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Lu, C., Lin, D., Jia, J., Tang, C.-K.: Two-class weather classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3718–3725 (2014)
Caron, M., Bojanowski, P., Joulin, A., Douze, M.: Deep clustering for unsupervised learning of visual features. In Proceedings of the European Conference on Computer Vision, pp. 132–149 (2018)
Van Gansbeke, W., Vandenhende, S., Georgoulis, S., Proesmans, M., Van Gool, L.: Scan: learning to classify images without labels. In: European Conference on Computer Vision, pp. 268–285 (2020)
Qin, Q., Huang, L., Wei, Z., Xie, K., Zhang, W.: Unsupervised deep multi-similarity hashing with semantic structure for image retrieval. IEEE Trans. Circuits Syst. Video Technol. 31, 2852–2865 (2020)
Zhang, W., Huang, L., Wei, Z., Nie, J.: Appearance feature enhancement for person re-identification. Expert Syst. Appl. 163, 113771 (2021)
Elhoseiny, M., Huang, S., Elgammal, A.: Weather classification with deep convolutional neural networks. In: 2015 IEEE International Conference on Image Processing (ICIP), pp. 3349–3353 (2015)
Shi, Y., Li, Y., Liu, J., Liu, X., Murphey, Y.L.: Weather recognition based on edge deterioration and convolutional neural networks. In: 2018 24th International Conference on Pattern Recognition (ICPR), pp. 2438–2443 (2018)
Lin, D., Lu, C., Huang, H., Jia, J.: RSCM: region selection and concurrency model for multi-class weather recognition. IEEE Trans. Image Process. 26(9), 4154–4167 (2017)
Zhao, B., Hua, L., Li, X., Lu, X., Wang, Z.: Weather recognition via classification labels and weather-cue maps. Pattern Recognit. 95, 272–284 (2019)
Xie, K., Wei, Z., Huang, L., Qin, Q., Zhang, W.: Graph convolutional networks with attention for multi-label weather recognition. Neural Comput. Appl. 33(17), 11107–11123 (2021). https://doi.org/10.1007/s00521-020-05650-8
Gidaris, S., Singh, P., Komodakis, N.: Unsupervised representation learning by predicting image rotations. ArXiv Prepr. ArXiv180307728 (2018)
Pathak, D., Krahenbuhl, P., Donahue, J., Darrell, T., Efros, A.A.: Context encoders: feature learning by inpainting. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2536–2544 (2016)
Chen, T., Kornblith, S., Norouzi, M., Hinton, G.: A simple framework for contrastive learning of visual representations. In: International Conference on Machine Learning, pp. 1597–1607 (2020)
Xie, J., Girshick, R., Farhadi, A.: Unsupervised deep embedding for clustering analysis. In: International Conference on Machine Learning, pp. 478–487 (2016)
Chang, J., Wang, L., Meng, G., Xiang, S., Pan, C.: Deep adaptive image clustering. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 5879–5887 (2017)
Hu, W., Miyato, T., Tokui, S., Matsumoto, E., Sugiyama, M.: Learning discrete representations via information maximizing self-augmented training. In: International Conference on Machine Learning, pp. 1558–1567 (2017)
Ji, X., Henriques, J.F., Vedaldi, A.: Invariant information clustering for unsupervised image classification and segmentation. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 9865–9874 (2019)
Zhang, Z., Ma, H., Fu, H., Zhang, C.: Scene-free multi-class weather classification on single images. Neurocomputing 207, 365–373 (2016)
Caron, M., Misra, I., Mairal, J., Goyal, P., Bojanowski, P., Joulin, A.: Unsupervised learning of visual features by contrasting cluster assignments. ArXiv Prepr. ArXiv200609882 (2020)
Wu, J., et al.: Deep comprehensive correlation mining for image clustering. In: Proceedings of the IEEE/CVF International Conference on Computer Vision, pp. 8150–8159 (2019)
Niu, C., Zhang, J., Wang, G., Liang, J.: Gatcluster: self-supervised gaussian-attention network for image clustering. In: European Conference on Computer Vision, pp. 735–751 (2020)
Acknowledgements
This work is supported by the National Natural Science Foundation of China (No. 61872326, No. 62072418); Qingdao Independent Innovation Major Project (20-3-2-2-hy, 20-3-2-12-xx). This work got the data service from the Marine Environment Data Service System which supported by the National Key R&D Program of China (2019YFC1408405).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Xie, K., Huang, L., Zhang, W., Qin, Q., Wei, Z. (2022). Learning to Classify Weather Conditions from Single Images Without Labels. In: Þór Jónsson, B., et al. MultiMedia Modeling. MMM 2022. Lecture Notes in Computer Science, vol 13141. Springer, Cham. https://doi.org/10.1007/978-3-030-98358-1_5
Download citation
DOI: https://doi.org/10.1007/978-3-030-98358-1_5
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-98357-4
Online ISBN: 978-3-030-98358-1
eBook Packages: Computer ScienceComputer Science (R0)