International Journal of Applied Earth Observation and Geoinformation
SegOptim—A new R package for optimizing object-based image analyses of high-spatial resolution remotely-sensed data
Introduction
Geographic Object-based Image Analysis (GEOBIA) is a recent sub-discipline of Geographic Information Science that, according to Hay and Castilla (2008), is “(…) devoted to developing automated methods to partition remote sensing imagery into meaningful image-objects, and assessing their characteristics through spatial, spectral and temporal scales, so as to generate new geographic information (...)”. More specifically, object-based analysis of Earth Observation (EO) data allows to describe the imaged reality using spectral, textural, spatial, contextual and topological features in a multi-scalar and integrated fashion (Lang, 2008). GEOBIA benefitted from the availability of (very-)high spatial resolution imagery, from vast progresses in image segmentation as well as software bridging image processing and GIS functionalities in an object-based environment ready for exploration and analysis (Blaschke, 2010).
GEOBIA concepts and methods have been used in several contexts and reported in a large and growing body of literature. Image segmentation plays an important role in GEOBIA (Blaschke, 2010; Lang, 2008). Independently of the method used, segmentation provides the elementary blocks used for object-based image analysis (Blaschke, 2010), interpretation, classification and modelling. Segmentation involves the partitioning of an image into a set of jointly exhaustive and mutually disjoint regions (a.k.a. segments, composed by multiple image pixels), that are internally more homogeneous and similar, compared to adjacent ones. Image segments are then related to geographic objects of interest (e.g., forests, agricultural or urban areas) through some form of object-based classification (Castilla and Hay, 2008). Image segments are created on the basis of one or more homogeneity (and merging) criteria in one or more dimensions of a feature space (Blaschke, 2010). As such, these segments have additional information related to the properties and moments of the distribution of spectral data from each individual contained pixel as well as contextual, morphological and spatial information (Blaschke, 2010; Hay and Castilla, 2008; van der Werff and van der Meer, 2008). Image segments should be meaningful in respect to a particular task and their properties should allow to convert them into useful geographic objects. This is especially challenging because widely different segmentation results can be obtained by varying parameter values of existing algorithms (Dragut et al., 2014; Liu et al., 2012). Therefore, defining objective criteria is needed to address this problem, allowing to identify which image segmentation algorithms and parameterization may provide optimal solutions in each case.
In order to better understand how image segmentation parameters are typically defined by users, we performed a semi-systematic, sample-based review of the current literature (see details in Supporting Information – Appendix S1). We found that 44% of 72 randomly selected papers (out of 1067 retrieved in our search) did not explicitly mentioned how image segmentation parameters were defined or tuned (Supp. Info. Appendix S1/Figure S1–1a). From publications that did mentioned this, 28% corresponded to specific procedures, 19% used a visual interpretation/trial-and-error approach and 9% used the ESP tool (Dragut et al., 2014) available only for eCognition multi-resolution segmentation (Baatz and Schape, 2000). Only very few examples (ca. 7%) explicitly integrated both image segmentation and classification steps for tuning parameters.
Although several works did not state or used trial-and-error approaches for defining image segmentation parameters, there are rigorous alternatives to perform this task. For example, ESP is an automatic tool for eCogniton® proprietary software capable of determining a set of suitable ‘scale’ parameters for multi-resolution image segmentation (Dragut et al., 2014). This unsupervised approach uses changes in local variance to detect scale transitions in spatial data. Other works have employed a supervised approach to this problem, comparing a set of reference objects with those obtained by image segmentation. Clinton et al. (2010) proposed several discrepancy measures to assess segmentation quality between sets based on area, location, or the combination of both aspects. Based on this work, Liu et al. (2012) devised a supervised discrepancy measure named Euclidean Distance 2 (ED2) to evaluate segmentation quality based on geometric and arithmetic similarity and used it to define the best parameters for segmentation (Liu et al., 2012; Novelli et al., 2017). Räsänen et al. (2013) compared several discrepancy measures for mapping boreal forests and concluded that it is crucial to state the objectives of image segmentation and that discrepancy evaluation measures should be used with care. Moreover, some of the above methods are software or algorithm-specific (such as ESP tool) or focus on geometric, positional or areal similarity between segments and reference data and thus, are not integrated with classification procedures.
In order to overcome some of these limitations and to generalize the process of comparing different segmentation algorithms and optimizing their parameters, we developed a solution that integrates both processing steps (segmentation and supervised classification) in a single and unified workflow. Genetic Algorithms (GA) are then used to optimize image segmentation parameter values in relation to the classification problem. GA’s are broadly included in the class of evolutionary algorithms and constitute a type of computational search method used to find exact and approximate solutions to a given optimization problem (Scrucca, 2013). GA’s are employed here due to their flexible and multi-purpose nature as a stochastic search technique capable of solving optimization problems both for continuous and discrete functions, by mimicking the biological principles of evolution and natural selection (Haupt and Haupt, 2004; Scrucca, 2013).
Our approach, based on GA optimization, was implemented in a new open-source R package named SegOptim which can be used to tune image segmentation parameters in the context of supervised classification of EO images. For assessing the proposed approach and toolkit we devised a set of tests using high (10–20 m), very-high (2–5 m) and ‘ultra’-high (0.03-0.10 m) spatial resolution images, collected from distinct EO platforms in six test sites in northern Portugal with different environmental and landscape conditions, and in which image segmentation was used for different purposes. SegOptim package is directed towards users interested in applying, comparing and optimizing object-based algorithms to image segmentation and classification of EO data. We provide access to the package source code through a Bitbucket repository: https://bitbucket.org/joao_goncalves/segoptim, and a tutorial describing the package functionalities: https://segoptim.bitbucket.io/docs/ (in prep).
Section snippets
Implementing the optimization approach in R
The R environment and language for statistical computing (R Development Core Team, 2017) provided the computational environment to implement SegOptim – the R package implementing the optimized GA-based approach to multi-technique image segmentation and GEOBIA supervised classification.
R has been gaining more popularity among the Remote Sensing (RS)/EO and GIS communities due to its ability to handle raster datasets (raster package (Hijmans, 2016)) and (pre-)process EO data (packages such as
Classification performance across test sites
Supervised classification performance for 5-fold cross-validation, based on optimized image segmentation, attained very good to excellent results across the six test sites (see Table 5 and Fig. 5). Kappa values ranged from 0.96 (E1) to 1.00 (E3) for single-class tests and, between 0.85 (E2) to 0.94 (E6) for multi-class tests (see also Supp. Info. – Appendix S4 with confusion matrices and producer/user accuracies).
Overall, no particular image segmentation algorithm outperformed its competitors
Main contributions to improve GEOBIA workflow
In this study we developed a generic approach to object-based analysis comparison and optimization which integrates two crucial steps in GEOBIA workflows typically exhibiting mutual dependencies: image segmentation and classification (Baatz et al., 2008). Due to their flexibility and power, we selected GA as a suitable method to optimize and fine-tune segmentation parameter values. Overall, the use of GA for optimizing image segmentation parameters with the aim of maximizing the thematic
Acknowledgments
J. Gonçalves, I. Pôças and B. Marcos were financially supported by FCT (Portuguese Foundation for Science and Technology), through grants SFRH/BD/90112/2012, SFRH/BPD/79767/2011 and SFRH/BD/99469/2014, respectively, funded by national and European funds (ESF), through POPH-QREN (2007-2013) and/or CSF (2014-2020).
References (54)
- et al.
A multi-scale looping approach to predict spatially dynamic patterns of functional species richness in changing landscapes
Ecol. Indic.
(2016) Object based image analysis for remote sensing
Isprs J. Photogramm. Remote. Sens.
(2010)- et al.
Geographic stacking: decision fusion to increase global land cover map accuracy
Isprs J. Photogramm. Remote. Sens.
(2015) - et al.
Automated parameterisation for multi-scale image segmentation on multiple layers
ISPRS J. Photogramm. Remote Sens.
(2014) - et al.
A systematic comparison of different object-based classification techniques using high spatial resolution imagery in agricultural environments
Int. J. Appl. Earth Obs. Geoinf.
(2016) - et al.
Discrepancy measures for selecting optimal combination of parameter values in object-based image analysis
Isprs J. Photogramm. Remote. Sens.
(2012) - et al.
Decision fusion and non-parametric classifiers for land use mapping using multi-temporal RapidEye data
Isprs J. Photogramm. Remote. Sens.
(2015) - et al.
A review of supervised object-based land-cover image classification
Isprs J. Photogramm. Remote. Sens.
(2017) - et al.
Classifier ensembles: select real-world applications
Inf. Fusion
(2008) - et al.
A region growing and merging algorithm to color segmentation
Pattern Recognit.
(1997)