Elsevier

Neurocomputing

Volume 134, 25 June 2014, Pages 79-91
Neurocomputing

A sub-pixel mapping method based on an attraction model for multiple shifted remotely sensed images

https://doi.org/10.1016/j.neucom.2012.12.078Get rights and content

Abstract

Sub-pixel mapping is a technique designed to obtain the spatial distribution of different classes in mixed pixels at the sub-pixel scale by transforming fraction images into a classification map. Traditional sub-pixel mapping algorithms only utilize a low-resolution image, and sub-pixel mapping is an ill-posed problem as information in a single low-resolution image is not enough to obtain a high-resolution land-cover map. The accuracy can be improved by incorporating auxiliary datasets to provide more land-cover information, such as multiple shifted images from the same area. In this paper, an attraction model is used to utilize multiple shifted remotely sensed images which have complementary information to each other at the sub-pixel scale. For multiple shifted images, one is selected as the base image, and the shifts of the other images and the base image can be calculated. The improved attraction model can obtain the impacts of the base image and auxiliary images, respectively, and integrate them to achieve a better result. The proposed algorithm was tested on synthetic real imagery, and the experimental results demonstrate that the proposed approach outperforms two traditional sub-pixel mapping algorithms based on a single image, and the another multiple shifted images based sub-pixel mapping method.

Introduction

Remotely sensed images usually contain mixed pixels, especially for images with coarse spatial resolution in which most pixels contain more than one class on the ground. Soft classification techniques are used to solve the problem of mixed pixels by generating a number of fraction images in which the abundance of different land-cover classes can be obtained to avoid losing information [1], [2]. However, the spatial attribution of different classes in a pixel is still unknown.

To obtain more details in a pixel, sub-pixel mapping was first introduced by Atkinson [3]. The sub-pixel mapping technique divides a coarse pixel into sub-pixels and assigns a land-cover class to each sub-pixel, with the constraint that the total number of sub-pixels of a given class is directly proportional to the percentage cover of that class for the original larger pixel [3]. With this technique, soft classification outputs can be converted to a hard classification map with a finer resolution.

The key problem in sub-pixel mapping is determining the most likely location of each land-cover class within the coarse pixel [4]. Many different techniques have been proposed to tackle this issue: Hopfield neural networks [5], [6], BP neural networks [7], [8], [9], the linear optimization technique [4], the spatial attraction model [10], [11], the pixel swapping algorithm [12], genetic algorithms [13], artificial immune systems [14], Markov random fields [15], [16], [17], geostatistics [18], [19], and differential evolution [20]. However, most of the traditional methods only utilize the soft-classified proportion of the data of a single image at the pixel level, and are based on the spatial dependence assumption [21].

In general, super-resolution mapping can be formulated as an inverse problem which is under-determined by reconstructing a fine spatial resolution map of land-cover class labels from a set of class fractions provided by a low-resolution image [22]. Consequently, sub-pixel mapping is also an ill-posed problem that transforms a low-resolution fraction image into a high-resolution classification map. In addition, traditional methods based on a single image have a limit to the detail and accuracy of the resulting thematic map. Therefore, additional supplementary datasets should be used. Many types of auxiliary datasets are available, such as LIDAR [23], fused image [21], and panchromatic imagery [24]. However, such suitable datasets are always hard to obtain. Another kind of auxiliary dataset comprises multi-date or multi-angle images which are taken from the same area of land surface but have a shift with each other at the sub-pixel scale because of the slight orbit-translation caused by orbit swing. Fraction maps derived from these images can therefore provide additional land-cover information at the sub-pixel scale, which can theoretically be used to improve the accuracy of super-resolution mapping [22].

However, traditional sub-pixel mapping methods are based on a single remotely sensed image, and the integration of multiple images is not considered. In this paper, to utilize these multiple shifted fraction images synchronously to improve the sub-pixel mapping accuracy, a multiple shifted images based attraction model (MSAM) is proposed to integrate the complementary information in the multiple images. The attraction values of the base image and auxiliary images are first calculated by the proposed model, and the two values are then integrated to obtain an integrated attraction value, given different weights. Lastly, the distribution of different classes can be determined, with the constraints of the fraction values and the integrated attraction value. The proposed method was tested using three synthetic remotely sensed images. The experimental results demonstrate that the proposed approach gives a better result than two traditional sub-pixel mapping methods which are based on a single image and the another multiple shifted images based method.

The rest of this paper is organized as follows: Section 2 gives a detailed description of the proposed sub-pixel mapping model. Section 3 gives the experimental results of the algorithm, which are compared with the other three sub-pixel mapping algorithms. Finally, conclusions are drawn in Section 4.

Section snippets

Sub-pixel mapping

The key issue in sub-pixel mapping is determining the optimal distribution in the mixed pixel. The universal criterion is spatial dependence [3], as proposed by Atkinson in 1997, which refers to the tendency for spatially proximate observations of a given property to be more alike than more distant observations. It evolves from the so-called Tobler׳s First Law [25], i.e., everything is related to everything else, but near things are more related than distant things. To implement a sub-pixel

Experiments and analysis

To simulate a real-world situation, the original high-resolution image was shifted at the pixel scale in the x and y directions and degraded to obtain a low-resolution image by applying an averaging filter, given the resize factor. The low-resolution image was then unmixed with the fully constrained least squares (FCLS) [26] method to obtain a fraction image. This procedure is performed many times, according to the number of images used for the proposed algorithm. In this way, the original

Conclusion

A new mapping method based on an attraction model which integrates the information of multiple shifted images, namely, the multiple shifted images based attraction model (MSAM) algorithm, is proposed for sub-pixel mapping. In MSAM, auxiliary information about the distribution of land cover in a pixel can be obtained from multiple shifted images, and it is merged into the original attraction model. The experimental results in this paper consistently show that the proposed MSAM gives better

Acknowledgments

This work was supported by National Basic Research Program of China (973 Program) under Grant no. 2011CB707105, the National Natural Science Foundation of China under Grant nos. 41371344, 41201426, and A Foundation for the Author of National Excellent Doctoral Dissertation of PR China (FANEDD) under Grant no. 201052, and the Program for New Century Excellent Talents in University under Grant no. NECT-10-0624. In addition, the authors gratefully acknowledge the help of Dr. F. Ling of the

Xiong Xu received the B.S. degree in photogrammetry, in June 2008, and the Ph.D. degree in Photogrammetry and Remote Sensing, in June 2013, from Wuhan University, Wuhan, China. He is currently a Research Assistant with the College of Surveying and Geo-Informatics, Tongji University, China. His current research interests include Multi- and Hyper- spectral image processing, Artificial neural network, and Pattern recognition.

References (27)

  • K. Wu et al.

    Super-resolution land-cover mapping based on the selective endmember spectral mixture model in hyperspectral imagery

    Opt. Eng.

    (2011)
  • D. Nigussie et al.

    Possibilities and limitations of artificial neural networks for subpixel mapping of land cover

    Int. J. Remote Sens.

    (2011)
  • K.C. Mertens et al.

    A sub-pixel mapping algorithm based on sub-pixel/pixel spatial attraction models

    Int. J. Remote Sens.

    (2006)
  • Cited by (26)

    • CNN based sub-pixel mapping for hyperspectral images

      2018, Neurocomputing
      Citation Excerpt :

      Also, the endmembers are computed based on a global convex hull assumption which is not universally binding. Most of the sub-pixel mapping techniques iteratively optimize an initially computed spatial distribution of sub-pixel labels so as to maximize the spatial contiguity [25,36,46,49,51–53,59]. Different unsupervised learning techniques such as Genetic algorithm, Hopfield network, Markov random field etc. have been explored for this purpose [2,3,40,48,60].

    • The effect of the point spread function on sub-pixel mapping

      2017, Remote Sensing of Environment
      Citation Excerpt :

      As the coarse proportions are fixed for a given pixel, iterations (as in the first method type) are not necessarily involved and SPM solutions can be produced more quickly. Methods falling into this type include sub-pixel/pixel spatial attraction model (SPSAM) (Mahmood et al., 2013; Mertens et al., 2006; Xu et al., 2014), back-propagation neural network-based algorithm (Gu et al., 2008; Zhang et al., 2008), learning-based algorithm (Zhang et al., 2014), kriging (Verhoeye and Wulf, 2002; Boucher and Kyriakidis, 2006) and radial basis function (RBF) (Wang et al., 2014a) interpolation. They can also be summarized as the soft-then-hard SPM (STHSPM) algorithms, a concept proposed in our previous work (Wang et al., 2014b; Chen et al., 2015).

    • Modified sparse representation based image super-resolution reconstruction method

      2017, Neurocomputing
      Citation Excerpt :

      In the image reconstruction stage, for input LR image patches, those HR image patches highly similar to LR image patches are found in dictionaries and used as high components of LR image patches, and then the HR image patches can be obtained by adding these high components and LR image patches [28–30]. Common learning based methods include Markov random field (MRF) based ones [30,31], locally linear embedding (LLE) based ones [32,33], sparse representation based ones [34,35] and so on. Although MRF based methods can improve the edge effect of reconstructed images, they need millions of HR and LR image patches in order to get better reconstruction effect.

    • Adaptive MAP sub-pixel mapping model based on regularization curve for multiple shifted hyperspectral imagery

      2014, ISPRS Journal of Photogrammetry and Remote Sensing
      Citation Excerpt :

      As one of the earlier approaches, artificial neural networks (ANN) have been proved to be effective for sub-pixel mapping (Mertens et al., 2004b), e.g., the Hopfield neural network (HNN) (Tatem et al., 2001a,b, 2002, 2003; Nguyen et al., 2006; Muad and Foody, 2010; Su et al., 2012), the back-propagation (BP) neural network (Mertens et al., 2003a, 2004b; Wang et al., 2006; Zhang et al., 2008), and the multi-layer perceptron neural network (Shao and Lunetta, 2011). Methods based on spatial attraction models (Mertens et al., 2006; Xu et al., 2014a; Ling et al., 2013), pixel-swapping algorithms (Atkinson, 2005; Thornton et al., 2006; Xu and Huang, 2014), and a linear optimization technique (Verhoeye and Wulf, 2002) have also been proposed for sub-pixel mapping. Recently, to obtain optimal sub-pixel mapping results, sub-pixel mapping algorithms based on computational intelligence have been successfully utilized, e.g., genetic algorithms (Mertens et al., 2003b), differential evolution (Zhong and Zhang, 2012), particle swarm optimization (Wang et al., 2012), artificial immune systems (Zhong and Zhang, 2013), and multi-agent systems (Xu et al., 2014b).

    View all citing articles on Scopus

    Xiong Xu received the B.S. degree in photogrammetry, in June 2008, and the Ph.D. degree in Photogrammetry and Remote Sensing, in June 2013, from Wuhan University, Wuhan, China. He is currently a Research Assistant with the College of Surveying and Geo-Informatics, Tongji University, China. His current research interests include Multi- and Hyper- spectral image processing, Artificial neural network, and Pattern recognition.

    Yanfei Zhong received the B.S. degree in information engineering and the Ph.D. degree in photogrammetry and remote sensing from Wuhan University, China, in 2002 and 2007, respectively.

    He has been with the State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University since 2007 and is currently a Professor. His research interests include multi- and hyperspectral remote sensing data processing, high resolution image processing and scene analysis, and computational intelligence. He has published more than 60 research papers, including more than 25 peer-reviewed articles in international journals such as IEEE Transactions on Geoscience and Remote Sensing and IEEE Transactions on Systems, Man and Cybernetics Part B, and Pattern Recognition.

    Dr. Zhong was the recipient of the National Excellent Doctoral Dissertation Award of China (2009) and New Century Excellent Talents in University of China (2009). He was a Referee of IEEE Transactions on Cybernetics, IEEE Transactions on Geoscience and Remote Sensing, IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, IEEE Geoscience and Remote Sensing Letters, and Pattern Recognition.

    Liangpei Zhang received the B.S. degree in physics from Hunan Normal University, ChangSha, China, in 1982, the M.S. degree in optics from the Xi׳an Institute of Optics and Precision Mechanics of Chinese Academy of Sciences, Xi׳an, China, in 1988, and the Ph.D. degree in Photogrammetry and Remote Sensing from Wuhan University, Wuhan, China, in 1998.

    He is currently with the State Key Laboratory of Information Engineering in Surveying, Mapping and Remote Sensing, Wuhan University, as the head of the Remote Sensing Division. He is also a “Chang-Jiang Scholar” Chair Professor appointed by the Ministry of Education, China. He is currently the Principal Scientist for the China State Key Basic Research Project (2011–2016) appointed by the Ministry of National Science and Technology of China to lead the remote sensing program in China. He is an Executive Member (Board of Governor) of the China National Committee of International Geosphere-Biosphere Programme. He also serves as an Associate Editor of International Journal of Ambient Computing and Intelligence, International Journal of Image and Graphics, International Journal of Digital Multimedia Broadcasting, Journal of Geo-spatial Information Science, and the Journal of Remote Sensing. He has more than 300 research papers and is the holder of 5 patents. His research interests include hyperspectral remote sensing, high resolution remote sensing, image processing and artificial intelligence.

    Dr. Zhang is a fellow of the Institution of Electrical Engineers, an executive Member for the China Society of Image and Graphics, and others. He regularly serves as a Cochair of the series SPIE Conferences on Multispectral Image Processing and Pattern Recognition, Conference on Asia Remote Sensing, and many other conferences. He edits several conference proceedings, issues, and the Geoinformatics Symposiums. He is currently serving as an Associate Editor for the IEEE Transactions on Geoscience and Remote Sensing.

    View full text