skip to main content
10.1145/3557915.3560937acmconferencesArticle/Chapter ViewAbstractPublication PagesgisConference Proceedingsconference-collections
research-article
Public Access

Clustering augmented self-supervised learning: an application to land cover mapping

Published: 22 November 2022 Publication History

Abstract

Collecting large annotated datasets in Remote Sensing is often expensive and thus can become a significant obstacle for training advanced machine learning models. Standard techniques for addressing this issue, based on the underlying idea of pre-training the Deep Neural Networks (DNN) on freely available large datasets, cannot be used for Remote Sensing due to the unavailability of such large-scale labeled datasets and the heterogeneity of data sources caused by the varying spatial and spectral resolution of different sensors. Self-supervised learning is an alternative approach that learns feature representation from unlabeled images without human annotations. In this paper, we introduce a new method for land cover mapping by using a clustering-based pretext task for self-supervised learning. We demonstrate the method's effectiveness in two societally relevant applications from the aspect of segmentation performance, discriminative feature representation learning, and the underlying cluster structure. We also show the effectiveness of the active sampling using the clusters obtained from our method in improving the mapping accuracy given a limited budget for annotating. Finally, a real-world application of the developed framework in identifying intra-class categories of well-managed and poorly-managed plantations demonstrates its utility in a problem of considerable societal importance.

References

[1]
Vijay Badrinarayanan, Alex Kendall, and Roberto Cipolla. 2017. Segnet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE transactions on pattern analysis and machine intelligence 39, 12 (2017), 2481--2495.
[2]
CDL 2021. USDA Cropland Data Layer. https://www.nass.usda.gov/Research_and_Science/Cropland/SARS1a.php.
[3]
Jia Deng, Wei Dong, Richard Socher, Li-Jia Li, Kai Li, and Li Fei-Fei. 2009. Imagenet: A large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition. Ieee, 248--255.
[4]
Rahul Ghosh, Xiaowei Jia, and Vipin Kumar. 2021. Land cover mapping in limited labels scenario: A survey. arXiv preprint arXiv:2103.02429 (2021).
[5]
Rahul Ghosh, Praveen Ravirathinam, Xiaowei Jia, Chenxi Lin, Zhenong Jin, and Vipin Kumar. 2021. Attention-augmented Spatio-Temporal Segmentation for Land Cover Mapping. In 2021 IEEE International Conference on Big Data (Big Data). IEEE, 1399--1408.
[6]
Spyros Gidaris, Praveer Singh, and Nikos Komodakis. 2018. Unsupervised representation learning by predicting image rotations. arXiv preprint arXiv:1803.07728 (2018).
[7]
Cassandra Handan-Nader and Daniel E Ho. 2019. Deep learning to map concentrated animal feeding operations. Nature Sustainability (2019).
[8]
Yunfeng Hu et al. 2018. A deep convolution neural network method for land cover mapping: a case study of qinhuangdao, China. Remote Sensing (2018).
[9]
Minyoung Huh, Pulkit Agrawal, and Alexei A Efros. 2016. What makes ImageNet good for transfer learning? arXiv preprint arXiv:1608.08614 (2016).
[10]
Neal Jean et al. 2019. Tile2vec: Unsupervised representation learning for spatially distributed data. In Proceedings of the AAAI Conference on Artificial Intelligence.
[11]
Xiaowei Jia et al. 2017. Incremental dual-memory lstm in land cover prediction. In Proceedings of the 23rd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining.
[12]
Longlong Jing and Yingli Tian. 2020. Self-supervised visual feature learning with deep neural networks: A survey. IEEE Transactions on Pattern Analysis and Machine Intelligence (2020).
[13]
Anuj Karpatne et al. 2016. Monitoring land-cover changes: A machine-learning perspective. IEEE Geoscience and Remote Sensing Magazine (2016).
[14]
Gustav Larsson, Michael Maire, and Gregory Shakhnarovich. 2016. Learning representations for automatic colorization. In European conference on computer vision. Springer, 577--593.
[15]
Gustav Larsson, Michael Maire, and Gregory Shakhnarovich. 2017. Colorization as a proxy task for visual understanding. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 6874--6883.
[16]
Quoc V Le. 2013. Building high-level features using large scale unsupervised learning. In 2013 IEEE international conference on acoustics, speech and signal processing. IEEE, 8595--8598.
[17]
Honglak Lee, Roger Grosse, Rajesh Ranganath, and Andrew Y Ng. 2009. Convolutional deep belief networks for scalable unsupervised learning of hierarchical representations. In Proceedings of the 26th annual international conference on machine learning. 609--616.
[18]
Jonathan Long, Evan Shelhamer, and Trevor Darrell. 2015. Fully convolutional networks for semantic segmentation. In Proceedings of the IEEE conference on computer vision and pattern recognition. 3431--3440.
[19]
Laurens van der Maaten and Geoffrey Hinton. 2008. Visualizing data using t-SNE. Journal of machine learning research 9, Nov (2008), 2579--2605.
[20]
Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg S Corrado, and Jeff Dean. 2013. Distributed representations of words and phrases and their compositionality. In Advances in neural information processing systems. 3111--3119.
[21]
Guruprasad Nayak, Rahul Ghosh, Xiaowei Jia, Varun Mithafi, and Vipin Kumar. 2020. Semi-supervised Classification using Attention-based Regularization on Coarse-resolution Data. In Proceedings of the 2020 SIAM International Conference on Data Mining.
[22]
Maxim Neumann, Andre Susano Pinto, Xiaohua Zhai, and Neil Houlsby. 2019. In-domain representation learning for remote sensing. arXiv preprint arXiv:1911.06721 (2019).
[23]
Hyeonwoo Noh, Seunghoon Hong, and Bohyung Han. 2015. Learning deconvolution network for semantic segmentation. In Proceedings of the IEEE international conference on computer vision. 1520--1528.
[24]
Mehdi Noroozi and Paolo Favaro. 2016. Unsupervised learning of visual representations by solving jigsaw puzzles. In European Conference on Computer Vision. Springer, 69--84.
[25]
Mehdi Noroozi, Hamed Pirsiavash, and Paolo Favaro. 2017. Representation learning by learning to count. In Proceedings of the IEEE International Conference on Computer Vision. 5898--5906.
[26]
Deepak Pathak, Philipp Krahenbuhl, Jeff Donahue, Trevor Darrell, and Alexei A Efros. 2016. Context encoders: Feature learning by inpainting. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2536--2544.
[27]
Jeffrey Pennington, Richard Socher, and Christopher D Manning. 2014. Glove: Global vectors for word representation. In Proceedings of the 2014 conference on empirical methods in natural language processing (EMNLP). 1532--1543.
[28]
Marc'Aurelio Ranzato, Fu Jie Huang, Y-Lan Boureau, and Yann LeCun. 2007. Unsupervised learning of invariant feature hierarchies with applications to object recognition. In 2007 IEEE conference on computer vision and pattern recognition.
[29]
Olaf Ronneberger, Philipp Fischer, and Thomas Brox. 2015. U-net: Convolutional networks for biomedical image segmentation. In International Conference on Medical image computing and computer-assisted intervention. Springer, 234--241.
[30]
Ekrem Saralioglu and Oguz Gungor. 2020. Semantic segmentation of land cover from high resolution multispectral satellite images by spectral-spatial convolutional neural network. Geocarto International (2020).
[31]
Andrei Stoian et al. 2019. Land cover maps production with high resolution satellite image time series and convolutional neural networks: Adaptations and limits for operational systems. Remote Sensing (2019).
[32]
Renee Su and Rong Chen. 2019. Land Cover Change Detection via Semantic Segmentation. arXiv preprint arXiv:1911.12903 (2019).
[33]
Priit Ulmas and Innar Liiv. 2020. Segmentation of Satellite Imagery using U-Net Models for Land Cover Classification. arXiv preprint arXiv:2003.02899 (2020).
[34]
Pascal Vincent, Hugo Larochelle, Yoshua Bengio, and Pierre-Antoine Manzagol. 2008. Extracting and composing robust features with denoising autoencoders. In Proceedings of the 25th international conference on Machine learning. 1096--1103.
[35]
Stefano Vincenzi, Angelo Porrello, Pietro Buzzega, Marco Cipriano, Pietro Fronte, Roberto Cuccu, Carla Ippoliti, Annamaria Conte, and Simone Calderara. 2020. The color out of space: learning self-supervised representations for Earth Observation imagery. arXiv preprint arXiv:2006.12119 (2020).
[36]
Junyuan Xie, Ross Girshick, and Ali Farhadi. 2016. Unsupervised deep embedding for clustering analysis. In International conference on machine learning. 478--487.
[37]
Michael Xie et al. 2015. Transfer Learning from Deep Features for Remote Sensing and Poverty Mapping. arXiv:1510.00098 [cs.CV]
[38]
Jianwei Yang, Devi Parikh, and Dhruv Batra. 2016. Joint unsupervised learning of deep representations and image clusters. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 5147--5156.
[39]
Richard Zhang, Phillip Isola, and Alexei A Efros. 2017. Split-brain autoencoders: Unsupervised learning by cross-channel prediction. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 1058--1067.
[40]
L Zhou, X Yang, et al. 2008. Use of neural networks for land cover classification from remotely sensed imagery. The International Archives of the Photogrammetry, Remote Sensing and Spatial Information Sciences (2008).

Cited By

View all
  • (2023)Detection and Tracking of Dynamic Ocean Carbon Uptake Regimes Built Upon Spatial Target-Driver Relationships via Adaptive Hierarchical Clustering2023 IEEE 19th International Conference on e-Science (e-Science)10.1109/e-Science58273.2023.10254820(1-10)Online publication date: 9-Oct-2023

Index Terms

  1. Clustering augmented self-supervised learning: an application to land cover mapping

      Recommendations

      Comments

      Information & Contributors

      Information

      Published In

      cover image ACM Conferences
      SIGSPATIAL '22: Proceedings of the 30th International Conference on Advances in Geographic Information Systems
      November 2022
      806 pages
      ISBN:9781450395298
      DOI:10.1145/3557915
      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

      Sponsors

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      Published: 22 November 2022

      Permissions

      Request permissions for this article.

      Check for updates

      Author Tags

      1. land cover mapping
      2. self-supervised learning
      3. semantic segmentation

      Qualifiers

      • Research-article

      Funding Sources

      • National Aeronautics and Space Administration (NASA)
      • NSF

      Conference

      SIGSPATIAL '22
      Sponsor:

      Acceptance Rates

      Overall Acceptance Rate 257 of 1,238 submissions, 21%

      Contributors

      Other Metrics

      Bibliometrics & Citations

      Bibliometrics

      Article Metrics

      • Downloads (Last 12 months)139
      • Downloads (Last 6 weeks)12
      Reflects downloads up to 20 Feb 2025

      Other Metrics

      Citations

      Cited By

      View all
      • (2023)Detection and Tracking of Dynamic Ocean Carbon Uptake Regimes Built Upon Spatial Target-Driver Relationships via Adaptive Hierarchical Clustering2023 IEEE 19th International Conference on e-Science (e-Science)10.1109/e-Science58273.2023.10254820(1-10)Online publication date: 9-Oct-2023

      View Options

      View options

      PDF

      View or Download as a PDF file.

      PDF

      eReader

      View online with eReader.

      eReader

      Login options

      Figures

      Tables

      Media

      Share

      Share

      Share this Publication link

      Share on social media