SegOptim—A new R package for optimizing object-based image analyses of high-spatial resolution remotely-sensed data

https://doi.org/10.1016/j.jag.2018.11.011Get rights and content

Highlights

  • Inappropriate image segmentation parameters often lead to sub-optimal results.

  • The new SegOptim R package allows to optimize image segmentation parameters.

  • SegOptim was tested in very high - high spatial resolution images in six test sites.

  • Genetic Algorithms optimization improves Geographic Object-based Image Analysis.

  • Integration between image segmentation and supervised classification is improved.

Abstract

Geographic Object-based Image Analysis (GEOBIA) is increasingly used to process high-spatial resolution imagery, with applications ranging from single species detection to habitat and land cover mapping. Image segmentation plays a key role in GEOBIA workflows, allowing to partition images into homogenous and mutually exclusive regions. Nonetheless, segmentation techniques require a robust parameterization to achieve the best results. Frequently, inappropriate parameterization leads to sub-optimal results and difficulties in comparing distinct methods.

Here, we present an approach based on Genetic Algorithms (GA) to optimize image segmentation parameters by using the performance scores from object-based classification, thus allowing to assess the adequacy of a segmented image in relation to the classification problem. This approach was implemented in a new R package called SegOptim, in which several segmentation algorithms are interfaced, mostly from open-source software (GRASS GIS, Orfeo Toolbox, RSGISLib, SAGA GIS, TerraLib), but also from proprietary software (ESRI ArcGIS). SegOptim also provides access to several machine-learning classification algorithms currently available in R, including Gradient Boosted Modelling, Support Vector Machines, and Random Forest.

We tested our approach using very-high to high spatial resolution images collected from an Unmanned Aerial Vehicle (0.03 – 0.10 m), WorldView-2 (2 m), RapidEye (5 m) and Sentinel-2 (10 – 20 m) in six different test sites located in northern Portugal with varying environmental conditions and for different purposes, including invasive species detection and land cover mapping. The results highlight the added value of our novel comparison of image segmentation and classification algorithms. Overall classification performances (assessed through cross-validation with the Kappa index) ranged from 0.85 to 1.00. Pilot-tests show that our GA-based approach is capable of providing sound results for optimizing the parameters of different segmentation algorithms, with benefits for classification accuracy and for comparison across techniques. We also verified that no particular combination of an image segmentation and a classification algorithm is suited for all the tasks/objectives. Consequently, it is crucial to compare and optimize available methods to understand which one is more suited for a certain objective.

Our approach allows a closer integration between the segmentation and classification stages, which is of high importance for GEOBIA workflows. The results from our tests confirm that this integration has benefits for comparing and optimizing both processes. We discuss some limitations of the SegOptim approach (and potential solutions) as well as a future roadmap to expand its current functionalities.

Introduction

Geographic Object-based Image Analysis (GEOBIA) is a recent sub-discipline of Geographic Information Science that, according to Hay and Castilla (2008), is “(…) devoted to developing automated methods to partition remote sensing imagery into meaningful image-objects, and assessing their characteristics through spatial, spectral and temporal scales, so as to generate new geographic information (...)”. More specifically, object-based analysis of Earth Observation (EO) data allows to describe the imaged reality using spectral, textural, spatial, contextual and topological features in a multi-scalar and integrated fashion (Lang, 2008). GEOBIA benefitted from the availability of (very-)high spatial resolution imagery, from vast progresses in image segmentation as well as software bridging image processing and GIS functionalities in an object-based environment ready for exploration and analysis (Blaschke, 2010).

GEOBIA concepts and methods have been used in several contexts and reported in a large and growing body of literature. Image segmentation plays an important role in GEOBIA (Blaschke, 2010; Lang, 2008). Independently of the method used, segmentation provides the elementary blocks used for object-based image analysis (Blaschke, 2010), interpretation, classification and modelling. Segmentation involves the partitioning of an image into a set of jointly exhaustive and mutually disjoint regions (a.k.a. segments, composed by multiple image pixels), that are internally more homogeneous and similar, compared to adjacent ones. Image segments are then related to geographic objects of interest (e.g., forests, agricultural or urban areas) through some form of object-based classification (Castilla and Hay, 2008). Image segments are created on the basis of one or more homogeneity (and merging) criteria in one or more dimensions of a feature space (Blaschke, 2010). As such, these segments have additional information related to the properties and moments of the distribution of spectral data from each individual contained pixel as well as contextual, morphological and spatial information (Blaschke, 2010; Hay and Castilla, 2008; van der Werff and van der Meer, 2008). Image segments should be meaningful in respect to a particular task and their properties should allow to convert them into useful geographic objects. This is especially challenging because widely different segmentation results can be obtained by varying parameter values of existing algorithms (Dragut et al., 2014; Liu et al., 2012). Therefore, defining objective criteria is needed to address this problem, allowing to identify which image segmentation algorithms and parameterization may provide optimal solutions in each case.

In order to better understand how image segmentation parameters are typically defined by users, we performed a semi-systematic, sample-based review of the current literature (see details in Supporting Information – Appendix S1). We found that 44% of 72 randomly selected papers (out of 1067 retrieved in our search) did not explicitly mentioned how image segmentation parameters were defined or tuned (Supp. Info. Appendix S1/Figure S1–1a). From publications that did mentioned this, 28% corresponded to specific procedures, 19% used a visual interpretation/trial-and-error approach and 9% used the ESP tool (Dragut et al., 2014) available only for eCognition multi-resolution segmentation (Baatz and Schape, 2000). Only very few examples (ca. 7%) explicitly integrated both image segmentation and classification steps for tuning parameters.

Although several works did not state or used trial-and-error approaches for defining image segmentation parameters, there are rigorous alternatives to perform this task. For example, ESP is an automatic tool for eCogniton® proprietary software capable of determining a set of suitable ‘scale’ parameters for multi-resolution image segmentation (Dragut et al., 2014). This unsupervised approach uses changes in local variance to detect scale transitions in spatial data. Other works have employed a supervised approach to this problem, comparing a set of reference objects with those obtained by image segmentation. Clinton et al. (2010) proposed several discrepancy measures to assess segmentation quality between sets based on area, location, or the combination of both aspects. Based on this work, Liu et al. (2012) devised a supervised discrepancy measure named Euclidean Distance 2 (ED2) to evaluate segmentation quality based on geometric and arithmetic similarity and used it to define the best parameters for segmentation (Liu et al., 2012; Novelli et al., 2017). Räsänen et al. (2013) compared several discrepancy measures for mapping boreal forests and concluded that it is crucial to state the objectives of image segmentation and that discrepancy evaluation measures should be used with care. Moreover, some of the above methods are software or algorithm-specific (such as ESP tool) or focus on geometric, positional or areal similarity between segments and reference data and thus, are not integrated with classification procedures.

In order to overcome some of these limitations and to generalize the process of comparing different segmentation algorithms and optimizing their parameters, we developed a solution that integrates both processing steps (segmentation and supervised classification) in a single and unified workflow. Genetic Algorithms (GA) are then used to optimize image segmentation parameter values in relation to the classification problem. GA’s are broadly included in the class of evolutionary algorithms and constitute a type of computational search method used to find exact and approximate solutions to a given optimization problem (Scrucca, 2013). GA’s are employed here due to their flexible and multi-purpose nature as a stochastic search technique capable of solving optimization problems both for continuous and discrete functions, by mimicking the biological principles of evolution and natural selection (Haupt and Haupt, 2004; Scrucca, 2013).

Our approach, based on GA optimization, was implemented in a new open-source R package named SegOptim which can be used to tune image segmentation parameters in the context of supervised classification of EO images. For assessing the proposed approach and toolkit we devised a set of tests using high (10–20 m), very-high (2–5 m) and ‘ultra’-high (0.03-0.10 m) spatial resolution images, collected from distinct EO platforms in six test sites in northern Portugal with different environmental and landscape conditions, and in which image segmentation was used for different purposes. SegOptim package is directed towards users interested in applying, comparing and optimizing object-based algorithms to image segmentation and classification of EO data. We provide access to the package source code through a Bitbucket repository: https://bitbucket.org/joao_goncalves/segoptim, and a tutorial describing the package functionalities: https://segoptim.bitbucket.io/docs/ (in prep).

Section snippets

Implementing the optimization approach in R

The R environment and language for statistical computing (R Development Core Team, 2017) provided the computational environment to implement SegOptim – the R package implementing the optimized GA-based approach to multi-technique image segmentation and GEOBIA supervised classification.

R has been gaining more popularity among the Remote Sensing (RS)/EO and GIS communities due to its ability to handle raster datasets (raster package (Hijmans, 2016)) and (pre-)process EO data (packages such as

Classification performance across test sites

Supervised classification performance for 5-fold cross-validation, based on optimized image segmentation, attained very good to excellent results across the six test sites (see Table 5 and Fig. 5). Kappa values ranged from 0.96 (E1) to 1.00 (E3) for single-class tests and, between 0.85 (E2) to 0.94 (E6) for multi-class tests (see also Supp. Info. – Appendix S4 with confusion matrices and producer/user accuracies).

Overall, no particular image segmentation algorithm outperformed its competitors

Main contributions to improve GEOBIA workflow

In this study we developed a generic approach to object-based analysis comparison and optimization which integrates two crucial steps in GEOBIA workflows typically exhibiting mutual dependencies: image segmentation and classification (Baatz et al., 2008). Due to their flexibility and power, we selected GA as a suitable method to optimize and fine-tune segmentation parameter values. Overall, the use of GA for optimizing image segmentation parameters with the aim of maximizing the thematic

Acknowledgments

J. Gonçalves, I. Pôças and B. Marcos were financially supported by FCT (Portuguese Foundation for Science and Technology), through grants SFRH/BD/90112/2012, SFRH/BPD/79767/2011 and SFRH/BD/99469/2014, respectively, funded by national and European funds (ESF), through POPH-QREN (2007-2013) and/or CSF (2014-2020).

References (54)

  • H.M.A. van der Werff et al.

    Shape-based classification of spectrally identical objects

    Isprs J. Photogramm. Remote. Sens.

    (2008)
  • R. Adams et al.

    Seeded region growing

    IEEE Trans. Pattern Anal. Mach. Intell.

    (1994)
  • M. Baatz et al.

    Progressing from object-based to object-oriented image analysis

  • M. Baatz et al.

    multiresolution segmentation: an optimization approach for High quality multi-scale image segmentation

  • B. Bechtel et al.

    Segmentation for object extraction of trees using MATLAB and SAGA

  • L. Breiman

    Random forests

    Mach. Learn.

    (2001)
  • G. Castilla et al.

    Image objects and geographic objects

  • D. Clewley et al.

    A python-based open source system for geographic object-based image analysis (GEOBIA) utilizing raster attribute tables

    Remote Sens. (Basel)

    (2014)
  • N. Clinton et al.

    Accuracy assessment measures for object-based image segmentation goodness

    Anglais

    (2010)
  • D. Comaniciu et al.

    Mean shift: a robust approach toward feature space analysis

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2002)
  • M. Fernández-Delgado et al.

    Do we Need Hundreds of Classifiers to Solve Real World Classification Problems?

    J. Mach. Learn. Res.

    (2014)
  • J. Gonçalves et al.

    Evaluating an unmanned aerial vehicle-based approach for assessing habitat extent and condition in fine-scale early successional mountain mosaics

    Appl. Veg. Sci.

    (2016)
  • Goslee, S.C., 2011. Analyzing Remote Sensing Data in R: The landsat Package. 43,...
  • H. Grybas et al.

    A comparison of unsupervised segmentation parameter optimization approaches using moderate- and high-resolution imagery

    GIsci. Remote Sens.

    (2017)
  • P.N. Happ et al.

    Interseg: A Distributed Image Segmentation Tool

    (2016)
  • R.M. Haralick

    Statistical and structural approaches to texture

    Proceedings of the IEEE

    (1979)
  • R.L. Haupt et al.

    Practical Genetic Algorithms

    (2004)
  • Cited by (0)

    View full text