Residential area extraction based on saliency analysis for high spatial resolution remote sensing images

https://doi.org/10.1016/j.jvcir.2015.09.019Get rights and content

Highlights

  • We propose a residential area extraction method based on saliency analysis.

  • The ADP-LWT is utilized for orientation feature extraction.

  • The logarithm co-occurrence histogram is used to compute the intensity features.

  • The color opponency and diagram objection are applied to capture the color features.

  • The saliency map is obtained through a weighted combination of the three features.

Abstract

Traditional residential area extraction methods for remote sensing image depend on classification, segmentation and prior knowledge which are time-consuming and difficult to build. In this paper, an efficient, saliency analysis-based residential area extraction method is proposed. In the proposed model, an adaptive directional prediction-based lifting wavelet transform (ADP-LWT) is introduced to obtain the orientation feature. A logarithm co-occurrence histogram is employed to compute the intensity feature. The color opponency and diagram objection based on the information are proposed to extract color feature from the contrast in the red–green opponent channel. The saliency map is obtained through a weighted combination based on the feature competition and the residential area is extracted by saliency map threshold segmentation. The experimental results reveal that the residential area extracted by our model has more demarcated boundaries and better performance in background subtraction.

Introduction

With an increase of the spatial resolution in remote sensing images, remote sensing image analysis has become much more complicated and important than before [1], [2]. Residential area extraction is an important application in remote sensing images processing such as change detection, land utilization and disaster warning.

As a product of the interaction between man and nature, residential area is the central place for human daily life and activities. Currently, researchers have proposed a variety of automatic and semi-automatic method based on the classification and segmentation to extract residential area.

Considering the difference in course of the classification, the methods based on the classification and segmentation can be divided into two categories, the supervised methods and un-supervised methods. The supervised methods make use of the prior knowledge to classify the residential area and backgrounds, such as neural network model [3], support vector machine method (SVM) [4], [5] and decision tree method [6]. Zhicheng and Itti extracted saliency feature and gist feature from satellite images and employed SVM to detect and classify targets automatically [7]. In those methods, the classification is implemented on the basis of the selection and learning of various training samples which may causes high computational complexity and makes the methods vulnerable to human factors. The un-supervised methods extract the initial feature to implement the classification with certain rules [8], such as k-means method and fuzzy clustering method. Chen proposed an improved 2D Otsu segmentation method [9]. The improved method calculates probabilities of diagonal quadrants in 2D histogram separately and obtains better performance of segmentation than the traditional Otsu method. Zhang employed an improved 2-D gradient histogram and minimum mean absolute deviation (MMAD) model to segment the roads and residential area from vegetation area in remote sensing images [10]. Comparing to the supervised methods, the unsupervised methods have little dependence on artificial factors. However, it extracts regions with relatively lower classification precision.

Residential area can also be seen as a kind of significant ROI and can be obtained using the method of ROI extraction. In recent years, saliency analysis based on image feature and visual attention mechanism has been widely researched and is the most efficient way to obtain ROI for natural scene images. Saliency analysis doesn’t need to build the prior knowledge library as well as conduct segmentation and classification in the whole image. And it has lower computation complexity.

Saliency analysis is usually based on the biological models that simulate the visual attention mechanism in the human visual system (HSV) [11], [12], [13], [14], such as ITTI model and GBVS model. The ITTI model [15] is one of the most famous biological models. This method generates saliency maps based on intensity, color and orientation features. Harel’s graph-based visual saliency method (GBVS) [16] is proposed on the basis of the ITTI model. The GBVS model obtains feature maps using the approach of the ITTI method and takes advantage of graph theory when it comes to the generation of the saliency map. More importantly, the GBVS model assumes the unique existence of the salient regions in the image.

In recent years, computational model has been proposed for a faster calculation of the saliency. The spectral residuals method (SR) [17], was inspired by Shannon information theory and it generates saliency maps through the extraction of spectral residuals in the frequency domain without considering the color features. Achanta et al. proposed a frequency-tuned approach (FT) [18] that transforms the input images from RGB color space to CIE Lab color space to utilize the color feature. Imamoglu’s wavelet transform-based method (WT) [19] utilizes low-level features obtained from the wavelet transform domain. Some researchers also propose a combination of biological model and computation model in order to get better saliency detection results. Yan et al. introduced a hierarchical model (H) [20] which introduces a multi-layer approach and computes saliency in each layer. The final saliency map is obtained using a tree-structure graphical model. The HC and RC models proposed by Cheng et al. introduced a regional contrast based salient object extraction algorithm, which simultaneously evaluates global contrast differences and spatial weighted coherence scores [21].

In high resolution satellite images, residential area is often very rich in textures, which are derived from the fact that these areas are composed of heterogeneous objects with different natures, including man-made objects with various materials and natural objects, such as vegetation and soils [22]. Besides, residential area is different to the surrounding area with respect to color and intensity, which makes it salient to human visual system. In contrast, the non-salient regions are often relatively uniform, with less variation in the image scene [23]. Hence, some people come up with the design to utilize visual saliency. Zhang and Yang propose a frequency domain analysis and salient region detection model (FDA-SRD) which extracts the residential area in HSI space based on quaternion Fourier transform and obtain the shape information of the residential area employing an adaptive threshold segmentation algorithm based on Gaussian Pyramids [24]. The MFF model processes the input image along intensity feature and orientation feature. Then the model introduced an across-scale fusion method to obtain the saliency map [25]. However these models provide inaccurate boundaries for the residential area.

In this paper, a novel residential area extraction method for remote sensing images based on saliency analysis is presented. The proposed model implemented saliency analysis through orientation feature, intensity feature and color feature extraction. The orientation feature is extracted through the adaptive directional prediction-based lifting wavelet transform (ADP-LWT). The intensity feature is obtained by the logarithm co-occurrence histogram. The color opponency and diagram objection based on the information are employed to extract color features from the contrast of the red–green opponent channel in multi-spectral images. Finally, the orientation, color and intensity feature maps are combined to obtain the final map through feature competition. Experimental results reveal that our method has better performance in providing residential area with more demarcated boundaries and better background subtraction.

The remainder of this paper is organized as follows. Section 2 illustrates the proposed model in this paper. Section 3 focuses on the experimental results in high spatial resolution remote sensing images. Section 4 provides conclusions.

Section snippets

Methodology

In this paper, a novel model is proposed to improve the performance of residential area extraction for high spatial resolution remote sensing images. Our model is conducted in four steps:

Firstly, orientation feature maps are obtained by the adaptive directional prediction-based lifting wavelet transform in the high frequency sub-bands of panchromatic images.

Secondly, we obtain intensity feature maps based on logarithm co-occurrence histogram in the low frequency sub-bands of panchromatic images.

Experimental results

To compare the results of the existing models, extensive experiments are designed for qualitative and quantitative evaluations. We choose the saliency analysis based models, ITTI, GBVS, FT, WT, SR, H, FDA-SRD, HC, RC and MFF models and residential area extraction models to compare with our model using a number of high spatial resolution remote sensing images obtained by the SPOT 5 satellite with the size of 1024 × 1024 pixels.

Conclusion

In this paper, a model based on saliency analysis for high spatial resolution remote sensing images is proposed. The orientation, intensity and color features are used to get the saliency map in remote sensing images. The orientation feature is obtained by a high frequency approach in the adaptive directional prediction-based lifting wavelet transform (ADP-LWT). The logarithm co-occurrence histogram is employed to compute the intensity feature. Considering the spectral characteristics of remote

Acknowledgments

This work was sponsored by the National Natural Science Foundation of China (Nos. 61571050 and 61071103) and by the Fundamental Research Funds for the Central Universities (2012LYB50).

References (33)

  • D. Arvor et al.

    Advances in geographic object-based image analysis with ontologies: a review of main contributions and limitations from a remote sensing perspective

    ISPRS J. Photogramm. Remote Sens.

    (2013)
  • W. Sweldens

    The lifting scheme: a custom-design construction of biorthogonal wavelets

    Appl. Comput. Harmonic Anal.

    (1996)
  • Y. Zhong et al.

    A hybrid object-oriented conditional random field classification framework for high spatial resolution remote sensing imagery

    IEEE Trans. Geosci. Remote Sens.

    (2014)
  • H. Murai et al.

    Remote sensing image analysis using a neural network and knowledge-based processing

    Int. J. Remote Sens.

    (1997)
  • T. Xiaohua et al.

    Urban land cover classification with airborne hyperspectral data: what features to use?

    IEEE J. Sel. Topics Appl. Earth Observations Remote Sens.

    (2014)
  • C.-C. Chang et al.

    LIBSVM: a library for support vector machines

    ACM Trans. Intell. Syst. Technol.

    (2011)
  • J.-n. Xiong, S. Liu, The decision tree algorithm of automatically extracting residential information from tm image, in:...
  • L. Zhicheng et al.

    Saliency and gist features for target detection in satellite images

    IEEE Trans. Image Process.

    (2011)
  • S. Dahiya, P.K. Garg, M.K. Jat, Object oriented approach for building extraction from high resolution satellite images,...
  • Q. Chen

    Modified two-dimensional Otsu image segmentation algorithm and fast realisation

    IET Image Process.

    (2012)
  • L. Zhang

    Remote sensing image segmentation based on an improved 2-D gradient histogram and MMAD model

    Geosci. Remote Sens. Lett. IEEE

    (2014)
  • A. Borji et al.

    State-of-the-art in visual attention modeling

    IEEE Trans. Pattern Anal. Mach. Intell.

    (2013)
  • A. Borji et al.

    Salient object detection: a benchmark

  • C. Huang et al.

    Regions of interest extraction from color image based on visual saliency

    J. Supercomput.

    (2011)
  • Z. Wang et al.

    A two-stage approach to saliency detection in images

    (2008)
  • L. Itti et al.

    A model of saliency-based visual attention for rapid scene analysis

    IEEE Trans. Pattern Anal. Mach. Intell.

    (1998)
  • Cited by (21)

    • Saliency detection and region of interest extraction based on multi-image common saliency analysis in satellite images

      2018, Neurocomputing
      Citation Excerpt :

      Experience suggests that salient regions always have a recognizable, obvious color. Additionally, color has been proven to be a useful and robust cue for distinguishing various objects [34]. As we know, color is a kind of spectrum.

    • Salient object detection via boosting object-level distinctiveness and saliency refinement

      2017, Journal of Visual Communication and Image Representation
      Citation Excerpt :

      Our approach belongs to the second category. The detection of salient objects has a wide range of applications, such as image and video compression [3], image classification [4], object/region detection and recognition [5], and image re-targeting [6], to name a few. Although the performance of salient object detection approaches has dramatically improved in the last few years, unsolved problems remain.

    View all citing articles on Scopus

    This paper has been recommended for acceptance by Yehoshua Zeevi.

    View full text