Delineation of cultivated land parcels based on deep convolutional networks and geographical thematic scene division of remotely sensed images

https://doi.org/10.1016/j.compag.2021.106611Get rights and content

Highlights

  • A stratified framework is proposed to extract information from large area of image.

  • The theoretical concept of geographical thematic scene is firstly presented.

  • A deep learning network is proposed to fast divide the geographical thematic scene.

  • A well-performed boundary delineation accuracy assessment method is proposed.

Abstract

Extraction of cultivated land information from high spatial resolution remote sensing images is increasingly becoming an important approach to digitization and informatization in modern agriculture. The continuous development of deep learning technology has made it possible to extract information of cultivated land parcels by an intelligent way. Aiming at fine extraction of cultivated land parcels within large areas, this article builds a framework of geographical thematic scene division according to the rule of territorial differentiation in geography. A deep learning semantic segmentation network, improved U-net with depthwise separable convolution (DSCUnet), is proposed to achieve the division of the whole image. Then, an extended multichannel richer convolutional features (RCF) network is involved to delineate the boundaries of cultivated land parcels from agricultural functional scenes obtained by the former step. In order to testify the feasibility and effectiveness of the proposed methods, this article implemented experiments using Gaofen-2 images with different spatial resolution. The results show an outstanding performance using methods proposed in this article in both dividing agricultural functional scenes and delineating cultivated land parcels compared with other commonly used methods. Meanwhile, the extraction results have the highest accuracy in both the traditional evaluation indices (like Precision, Recall, F1, and IoU) and geometric boundary precision of cultivated land parcels. The methods in this article can provide a feasible solution to the problem of finely extracting cultivated land parcels information within large areas and complex landscape conditions in practical applications.

Introduction

Cultivated land parcels (CLPs) are the basic units on which humans engage in agricultural activities using land resources (Sun et al., 2020). Accurate information of CLPs is the basis and core information required for digitalization and informatization in modern agriculture (García-Pedrero et al., 2017, García-Pedrero et al., 2018). The acquisition of information on the spatial morphology of CLPs is a prerequisite for extracting other information (e.g., use types or cropping systems of CLPs) (Wu et al., 2019). Therefore, it is important to extract accurate spatial morphological information of CLPs (i.e., CLPs boundary information) which has vital value for practical applications (Masoud et al., 2020).

With the construction of the Global Navigation Satellite System (GNSS), it has become possible to obtain CLPs boundary information by using equipment or sensors placed in the fields (Chen et al., 2021, Sarri et al., 2017). Although the method using GNSS is very accurate, it requires a large amount of positioning equipment. Thus, this is expensive and time-consuming for large area agriculture monitoring. As a technologically feasible way, remote sensing provides a comprehensive, timely, and rapid record of various geographical phenomena, patterns, and processes on the earth's surface. Thanks to the development of sensor technology, the spatial resolution of remote sensing images has gradually increased, reaching the meter level and even the sub-meter level. It is common to use high spatial resolution remote sensing images as data sources to obtain boundary information of CLPs in a lot of research (Valero et al., 2016). Apart from the traditional manual visual interpretation, automatic information extraction technology with computers has become a popular way to outline CLPs boundaries (Waldner and Diakogiannis, 2020, Zhang et al., 2020). The state-of-the-art of CLPs boundaries delineation with remote sensing images is as follows.

  • Boundaries delineation of CLPs using edge based image segmentation (edge detection methods)

Early edge detection methods mainly refer to differential operators, like Roberts (Roberts, 1963), Prewitt (Prewitt, 1970), Sobel (Sobel, 1978), Canny (Canny, 1986), and so on. These operators are theoretically capable of detecting all kinds of edges from the image. With the improvement of algorithms, other methods (e.g., wavelet transform, Hough line detection, and etc.) can also detect image edges in different ways. For boundaries delineation of CLPs, Wagner (2020) used the Sobel operator to detect edges from Sentinel-2 imagery (Wagner and Oppelt, 2020b). Robb (2020) used Canny and phase congruency edge detection algorithms to delineate edges from unmanned aerial system imagery (Robb et al., 2020). Although the algorithms are simple and easy to be implemented, the methods are relatively sensitive to noise, and there are some discontinuous lines in the extracted results. In other words, the methods do not guarantee closed polygons (North et al., 2019). Thus, some post-processing methods need to be utilized after edge detection. For example, Wagner (2020) proposed an improved parametric activity contour model and a graph-based contour line extraction method to extract parcel polygons (Wagner and Oppelt, 2020b), and Robb (2020) used Hough line detection and active contour model to derive CLPs boundaries based on edges obtained by Canny (Robb et al., 2020).

  • Boundaries delineation of CLPs using region based image segmentation (region segmentation methods)

This method can be subdivided into two methods: region growth and region split-merge. Region growth segmentation is a bottom-up approach. Multiresolution segmentation (MRS) based on fractal net evolution approach and mean shift (MS) segmentation (Ma et al., 2014, Ming et al., 2012, Ming et al., 2015) are representative methods for region growth segmentation. Specifically, MRS has been integrated in eCognition software, which is famous for object based image analysis (OBIA). In contrast, region split-merge segmentation operates images from top to down. For CLPs boundary information extraction, O’Connell (2015) extracted parcels by MRS using airborne remote sensing images with two strategies (i.e., single scale and multiscale) (O’Connell et al., 2015). Ming (2016) delineated CLPs using scale pre-estimation based MS segmentation from SPOT-5 images (Ming et al., 2016). These methods can relieve sensitivity to noise to a certain extent, but they are complex compared to edge detection methods. Region based segmentation methods are usually unsupervised, the segmentation results also need post-processing, such as image classification, to select the CLPs. In addition, the choice of segmentation parameters (especially the scale parameter) is one of the important issues that cannot be avoided (Ma et al., 2015, Ming et al., 2018).

The former two methods have their own strengths and weaknesses. Thus, researchers want to combine the advantages of both edge-based and region-based approaches. For example, Mueller (2004) pointed out that the traditional region-based image segmentation method has the problem that the boundaries of adjacent parcels are mixed and cannot be distinguished (Mueller et al., 2004). Therefore, shape information, as a kind of edge feature, is involved into the region-based segmentation method, so that the boundary of CLPs could be extracted accurately.

  • Boundaries delineation of CLPs based on image classification with deep learning methods

Considering the disadvantage of image segmentation methods, binary classification (i.e., boundaries of CLPs and non-boundaries) is involved in the extraction of CLPs boundaries as well. Thanks to deep learning networks, image deep features can be extracted. Image classification with deep learning methods perform better than traditional statistical classifiers in a lot of remote sensing applications (Ma et al., 2019). For delineation of CLPs boundaries, Wagner (2020) extracted a variety of shallow features, such as spectra and textures of images, and used them as input of the neural network for CLPs boundaries delineation (Wagner and Oppelt, 2020a). Taravat (2021) used improved U-Net to extract the boundaries of CLPs (Taravat et al., 2021). In addition to convolutional neural networks (CNN) based on the patch and semantic segmentation networks, some networks converting the traditional hard classification into soft classification have been proposed to extract edge or boundary features of images specially, for example, HED (Xie and Tu, 2017), DeepEdge (Bertasius et al., 2015), RCF (Liu et al., 2019, Liu et al., 2017), CASENet (Yu et al., 2017), CEDN (Yang et al., 2016), and etc. Thus, the outcomes of these networks are usually edge or boundary probability maps, which the ground objects with clear boundaries, such as roads, buildings, coastal lines, and so on, can be easily extracted. Liu (2020) employed RCF to delineate farmland parcels with high spatial resolution remote sensing optical images and time series synthetic aperture radar (SAR) data, which worked well in mountainous areas (Liu et al., 2020). Though deep learning methods require plenty of labelled images for model training, they are seen as potential approaches to delineating CLPs because of the deep abstract features they used.

Though great achievements have been made, a common problem still exists that whatever kind of method is used in the previous studies mentioned above, the experimental data they used only covered limited areas and contained very few types of ground objects. However, CLPs information with large areas and complex landscape conditions is urgently desired in agricultural practical applications. Thus, these large covered images need to be processed before information extraction. One of the necessary processing between traditional image pre-processing and information extraction is image region division (Xu et al., 2019, Zhou et al., 2018). According to the first law of geography (Tobler, 1970) and the rule of territorial differentiation, it can be concluded that there is a clustering effect (spatial homogeneity within the region) for the same type of ground objects, and there is regional heterogeneity for different types of ground objects. Therefore, region division, an application of classical geographic idea, is increasingly becoming an effective solution for solving the problem oriented to large areas and complex landscapes.

In recent years, researchers have also studied the issue of image region division to varying degrees in the remote sensing community. In this article, the methods of image region division are categorized according to different division indicators.

  • Image region division by auxiliary features

The division indicators of this method are always from auxiliary data, which associate with the study area. The auxiliary data include vector data (Wu et al., 2020), digital elevation model (DEM) (Wu et al., 2021), or other remote sensing images (Huang et al., 2021). For example, DEM was used to divide the study area into mountain regions, hilly regions, plain regions, and so on (Liu et al., 2020). Then, information extraction was followed in different subregions. Zhou (2019) employed point cloud data acquired by light detection and ranging (LiDAR) technique to generate the standard normalized digital surface model which was further used as a kind of auxiliary feature to divide images into different regions (Zhou et al., 2019). In the urban research field, image region division was always used to identify the basic units of urban functional zones. For example, Zhang (2017) used vector data of city roads to derive different districts (Zhang et al., 2017). Huang (2021) utilized nighttime light and daytime multi-view imagery to obtain the image regions (Huang et al., 2021). However, this kind of method by auxiliary features highly requires auxiliary data which is sometimes unavailable. In addition, transferring the scale or registering the auxiliary data to the target remote sensing images is often required.

  • Image region division by image shallow features

The fundamental shallow features of remote sensing images are spectral features and texture features. These two kinds of features are always used as the basis for image segmentation and classification. Similarly, they are also used as indicators for image region division. For example, Zheng (2013) carried out over-segmentation with the spectral feature of images (Zheng et al., 2013) and then used improved Markov random field model to combine the over-segments as post-processing (Zheng et al., 2019, Zheng et al., 2017). Considering OBIA and the idea of stratified processing, Xu (2019) segmented the image on rough scale based on the fused spectral and texture features to achieve image region division. After that the information of farmland could be derived from the rural region (Xu et al., 2019). In addition to spectral and texture features, there are also some other specific indicators retrieved from remote sensing images for image region division. Some indicators also have a clear physical meaning, such as surface temperature, vegetation indices (e.g., normalized difference vegetation index, NDVI), and so on. Contrasted with the method based on auxiliary features, the method using image shallow features can relieve the limitation of data dependency. However, the results of this method are binary classification commonly. It is not good at generalizing image information on a large scale because of the broken regions obtained by this method. Additionally, it is needed to indicate that image segmentation is a necessary process whatever for method based on auxiliary features or based on shallow features, thus scale selection issues are often inevitably involved in image region division by using these two kinds of methods above.

  • Image region division by deep features of images

Semantic segmentation networks by deep learning are the core of this method. In computer vision, semantic segmentation is a classic task. Semantic segmentation is a way to classify the image pixel by pixel, which is different from traditional segmentation in the remote sensing community. At an early stage, semantic segmentation based on graph theory (for example, grab cut, N-cut, and etc.) merely classified the image into two categories, and semantic information was not clear. With the reemergence of deep learning and CNN, the development of semantic segmentation has been greatly promoted. Long (2015) proposed fully convolutional network (FCN) by deleting fully connected layers of traditional CNN (Long et al., 2015, Shelhamer et al., 2017), which has made great progress on semantic segmentation. After that, researchers proposed other kinds of semantic segmentation networks based on FCN, including SegNet (Badrinarayanan et al., 2017), U-Net (Ronneberger et al., 2015), DeepLab (Chen et al., 2018), and so on. These networks extract deep features of images on multiscale and are capable of generalizing the image on a large scale by continuously stacking convolutional layers and pooling layers. The deep learning networks can yield more satisfactory image semantic segmentation results compared with the graph-based methods.

Inspired by the big progress of semantic segmentation in computer vision, deep learning based semantic segmentation has been introduced into the remote sensing community and have made some achievements in the field of image processing and analysis (Ma et al., 2019). Semantic segmentation is currently used for two tasks, namely, binary classification (thematic objects extraction) and full objects classification. Although image region division belongs to the latter, there is few studies on image region division. This is because the existing semantic segmentation datasets are mostly for thematic objects extraction and image classification tasks. There is a lack of remote sensing image datasets specifically for image region division, resulting in limitation of the related research to a certain extent. However, the method of image region division using semantic segmentation networks is theoretically feasible and advantageous.

In conclusion, image region division is a necessary image processing and gradually becomes an effective method to solve the problem of information extraction from remote sensing images with large areas and complex landscape conditions. Therefore, aiming at fine extraction of CLPs, this article proposed a stratified extraction framework with two steps. Firstly, image region division using a semantic segmentation network on a coarse scale, and then, delineation of CLPs boundaries using a boundary detection deep learning network on a fine scale. Experiments proves that the proposed framework can improve the accuracy and efficiency in CLPs boundaries extraction.

The remaining parts of this article are settled as follows. The description of the proposed methods is introduced in Section 2. Section 3 shows the experiments and results in detail. The effectiveness of image region division and the performance of the proposed methods are discussed in Section 4. Finally, the conclusion of this article is summarized in Section 5.

Section snippets

Methodology

The flowchart of CLPs boundaries delineation in this article is shown in Fig. 1. First of all, this article proposed a novel theory of image region division based on geographical thematic scene division according to the rule of territorial differentiation. The definition of geographical thematic scenes and their division system are described in Section 2.1. Secondly, the dataset of geographical thematic scene division and the dataset of CLPs boundaries are built by using pre-processed remote

Study area and datasets

In this article, two study areas have been chosen to carry out experiments. Fig. 7 shows the true color synthesis of the study areas. Study area A is located in Guangping County, Hebei Province, China. The experimental data is a Gaofen-2 fusion image with the panchromatic band and the multispectral image. The image was photographed on 25 February 2017, with the size of 6000 × 6000 pixels and a spatial resolution of 1 m, namely, 36 square kilometers. The main type of crop in study area A is

Effectiveness of image region division based on geographical thematic scenes

Focusing on practical agriculture monitoring in a large area, fast CLPs boundaries delineation is the fundamental step. As the expansion of the study area, the types of objects contained in the study area increase and get complex besides CLPs. It is difficult to accurately delineate CLPs information within the large area image covered by complex landscapes. Therefore, according to the first law of geography and the rule of territorial differentiation, this article proposes a novel method to

Conclusion

The purpose of this article is to delineate CLPs boundaries from high spatial remote sensing images by using deep convolutional networks. The proposed method is capable of extracting CLPs within large area covered by complex landscapes, which can meet the need of practical applications. The main contributions are summarized below.

(1). Image region division gradually become a potential solution for information extraction of large area remote sensing images with complex landscapes. According to

Funding

This work was supported in part by the National Natural Science of China under Grant 41671369, in part by the National Key Research and Development Program under Grant 2017YFB0503600-05, in part by 2021 Graduate Innovation Fund Project of China University of Geosciences, Beijing, under Grant ZD2021YC054, and in part by the Fundamental Research Funds for the Central Universities.

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

References (60)

  • D. Sarri et al.

    Development of a prototype of telemetry system for monitoring the spraying operation in vineyards

    Comput. Electron. Agric.

    (2017)
  • I. Sobel

    Neighborhood coding of binary images for fast contour following and general binary array processing

    Comput. Graph. Image Process.

    (1978)
  • S. Valero et al.

    Production of a dynamic cropland mask by processing remote sensing image series at high temporal and spatial resolutions

    Remote Sensing

    (2016)
  • F. Waldner et al.

    Deep learning on edge: Extracting field boundaries from satellite images with a convolutional neural network

    Remote Sens. Environ.

    (2020)
  • L. Xu et al.

    Farmland Extraction from high spatial resolution remote sensing images based on stratified scale pre-estimation

    Remote Sensing

    (2019)
  • D. Zhang et al.

    A generalized approach based on convolutional neural networks for large area cropland mapping at very high resolution

    Remote Sens. Environ.

    (2020)
  • X. Zhang et al.

    Hierarchical semantic cognition for urban functional zones with VHR satellite images and POI data

    ISPRS J. Photogramm. Remote Sens.

    (2017)
  • W. Zhao et al.

    Deeply synergistic optical and SAR time series for crop dynamic monitoring

    Remote Sens. Environ.

    (2020)
  • W. Zhou et al.

    SO–CNN based urban functional zone fine division with VHR remote sensing image

    Remote Sens. Environ.

    (2020)
  • V. Badrinarayanan et al.

    SegNet: a deep convolutional encoder-decoder architecture for image segmentation

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2017)
  • Bertasius, G., Shi, J., Torresani, L., 2015. DeepEdge: A multi-scale bifurcated deep network for top-down contour...
  • Canny, J., 1986. A Computational Approach to Edge Detection. IEEE Transactions on Pattern Analysis and Machine...
  • L.C. Chen et al.

    DeepLab: semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2018)
  • F. Chollet

    Xception: Deep Learning with Depthwise Separable Convolutions

  • A. García-Pedrero et al.

    A machine learning approach for agricultural parcel delineation through agglomerative segmentation

    Int. J. Remote Sens.

    (2017)
  • A. García-Pedrero et al.

    The outlining of agricultural plots based on spatiotemporal consensus segmentation

    Remote Sensing

    (2018)
  • Howard., A.G., Zhu, M., Chen, B., Kalenichenko, D., Wang, W., Andreetto, T.W.M., Adam, H., 2017. arXiv preprint,...
  • W. Liu et al.

    Farmland Parcel Mapping in Mountain Areas Using Time-Series SAR Data and VHR Optical Images

    Remote Sensing

    (2020)
  • Y. Liu et al.

    Richer Convolutional Features for Edge Detection

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2019)
  • Y. Liu et al.

    Richer Convolutional Features for Edge Detection

  • Cited by (0)

    View full text