Land cover classification using geo-referenced photos

Abstract

We investigate publicly available geo-referenced photo collections for land cover classification. Mapping land cover is a fundamental task in the geographic sciences and is typically done using remote sensing (overhead) imagery through manual annotation. We here propose a novel alternate approach based on proximate sensing. The goal of proximate sensing is to map what-is-where on the surface of the Earth using ground level images of objects and scenes. It has the potential to map phenomena not observable through remote sensing. We perform an extensive case study on using ground level images for binary land cover classification into developed and undeveloped regions. We investigate visual features and text annotations to label images or sets of images with these two classes. Knowing the location of the images allows us to generate land cover maps which we quantitatively evaluate using ground truth maps. We apply our approach to two photo collections, Flickr, the popular photo sharing website, and the Geograph project, whose goal is to collect geographically informative photos. Comparing these two collections allows us to measure the impact of photographer intent. We utilize a weakly supervised learning framework which eliminates the need for manually labeled training data. We also investigate methods for filtering images that are unlikely to be geographically informative. Our results are promising and validate proximate sensing as a novel alternate approach to geographic discovery.