Skip to main content
SpringerLink
Log in

Combining morphological filtering, anisotropic diffusion and block-based data replication for automatically detecting and recovering unscanned gaps in remote sensing images

  • Research Article
  • Published:
Earth Science Informatics Aims and scope Submit manuscript

Abstract

Filling damaged pixels in satellite images is a key task present in many Remote Sensing applications. As a representative example of image restoration issue, we can refer to the failure of the Scan Line Corrector (SLC) on board the Landsat Enhanced Thematic Mapper Plus (ETM +) sensor, in which 22% of the scanned pixels in the SLC-off images were missed, thus creating unexpected stipe-type gaps in the scenes. In order to improve the usability of ETM + SLC-off data in a straightforward manner, in this paper we propose a unified methodology that automatically segments and repairs Landsat-7 scenes occluded by stripes. The proposed framework combines Morphology-based filtering, anisotropic diffusion and block-based pixel replication as an effective, fully unsupervised restoration methodology designed to cope with different gap sizes in Landsat images. Our approach does not require having as input data any prior gap mask, side reference image or time-dependent frames of the same scene to work properly. As shown in the experimental results, the current methodology performs adequately for a variety of multispectral remote sensing images with different stripe-size thicknesses and heterogeneous segments. We attest to the accuracy and robustness of our end-to-end framework throughout a variety of qualitative and quantitative evaluations involving state-of-the-art restoration methods.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

References

  • Aghamohamadnia M, Abedini A (2014) A morphology-stitching method to improve landsat slc-off images with stripes. Geod Geodyn 5(1):27–33

    Article  Google Scholar 

  • Ali L, Kasetkasem T, Khan W, Chanwimaluang T, Nakahara H (2017) Performance evaluation of different inpainting algorithms for remotely sensed images. In: Asian conference on defence technology (ACDT), pp 43–48

  • Barcelos CAZ, Boaventura M, Silva EC (2003) A well-balanced flow equation for noise removal and edge detection. IEEE Trans Image Process 12(7):751–763

    Article  Google Scholar 

  • Biehl L, Landgrebe D (2002) Multispec—a tool for multispectral–hyperspectral image data analysis. Comput Geosci 28(10):1153–1159

    Article  Google Scholar 

  • Biehl L, Maud AR, Hsu WK, Yeh TT (2020) Multispec (purdue research foundation) - version 3.5.1 (setp. 2020). https://engineering.purdue.edu/biehl/MultiSpec/index.html accessed 10 Octuber 2020

  • Bradley D, Roth G (2007) Adaptive thresholding using the integral image. J Graphics Tools 12(2):13–21

    Article  Google Scholar 

  • Buyssens P, Daisy M, Tschumperlé D, Lézoray O (2015) Exemplar-based inpainting: Technical review and new heuristics for better geometric reconstructions. IEEE Trans Image Process 24(6):1809–1824

    Google Scholar 

  • Cardim GP, da Silva E, Dias MA, Bravo I, Gardel A (2020) A nonrecursive gr algorithm to extract road networks in high-resolution images from remote sensing. Earth Sci Inf 13(4):1187–1199

    Article  Google Scholar 

  • Casaca W, Boaventura M, de Almeida MP, Nonato LG (2014) Combining anisotropic diffusion, transport equation and texture synthesis for inpainting textured images. Pattern Recogn Lett 36:36–45

    Article  Google Scholar 

  • Casaca W, Gois JP, Batagelo HC, Taubin G, Nonato LG (2020) Laplacian coordinates: Theory and methods for seeded image segmentation. In: IEEE Transactions on Pattern Analysis and Machine Intelligence. https://doi.org/10.1109/TPAMI.2020.2974475

  • Cheng Q, Shen H, Zhang L, Li P (2014) Inpainting for remotely sensed images with a multichannel nonlocal total variation model. IEEE Trans Geosci Remote Sens 52(1):175–187

    Article  Google Scholar 

  • Congalton RG, Green K (2019) Assessing the accuracy of remotely sensed data. CRC Press, Boca Raton

    Book  Google Scholar 

  • de Azevedo SC, Silva E, Colnago M, Negri RG, Casaca W (2019) Shadow detection using object area-based and morphological filtering for very high-resolution satellite imagery of urban areas. J Appl Remote Sens 13(3):1–16

    Article  Google Scholar 

  • Ehsan S, Clark A, Rehman N, McDonald-Maier K (2015) Integral images: Efficient algorithms for their computation and storage in resource-constrained embedded vision systems. Sensors 15(7):16804–16830

    Article  Google Scholar 

  • El Fellah S, Rziza M, El Haziti M (2017) An efficient approach for filling gaps in landsat 7 satellite images. IEEE Geosc Remote Sens Lett 14(1):62–66

    Article  Google Scholar 

  • Elharrouss O, Almaadeed N, Al-Maadeed S, Akbari Y (2020) Image inpainting: A review. Neural Process Lett 51:2007–2028

    Article  Google Scholar 

  • Gao J (2009) Digital analysis of remotely sensed imagery. McGraw-Hill Professional Publishing, New York

    Google Scholar 

  • Haas TC (1990) Kriging and automated variogram modeling within a moving window. Atmos Environ Part A Gen Top 24(7):1759–1769

    Article  Google Scholar 

  • Jamali A (2020) Land use land cover mapping using advanced machine learning classifiers: A case study of shiraz city, iran. Earth Sci Infs 13(1):1015–1030

    Article  Google Scholar 

  • Lechner AM, Foody GM, Boyd DS (2020) Applications in remote sensing to forest ecology and management. One Earth 2(5):405–412

    Article  Google Scholar 

  • Li J, Pei Y, Zhao S, Xiao R, Sang X, Zhang C (2020) A review of remote sensing for environmental monitoring in china. Remote Sens 12(7):1130

    Article  Google Scholar 

  • Li X, Shen H, Li H, Zhang L (2016) Patch matching-based multitemporal group sparse representation for the missing information reconstruction of remote-sensing images. IEEE J Sel Top Appl Earth Obs Remote Sens 9(8):3629–3641

    Article  Google Scholar 

  • Meng F, Yang X, Zhou C, Li Z, Liu B (2018) Multiscale adaptive reconstruction of missing information for remotely sensed data using sparse representation. Remote Sens Lett 9(5):457–466

    Article  Google Scholar 

  • Miao J, Zhou X, Huang T, Zhang T, Zhou Z (2019) A novel inpainting algorithm for recovering landsat-7 etm+ slc-off images based on the low-rank approximate regularization method of dictionary learning with nonlocal and nonconvex models. IEEE Trans Geosci Remote Sens 57(9):6741–6754

    Article  Google Scholar 

  • Miranda V, Pina P, Heleno S, Vieira G, Mora C, Schaefer CE (2020) Monitoring recent changes of vegetation in fildes peninsula (king george island, antarctica) through satellite imagery guided by uav surveys. Sci Total Environ 704:135295

    Article  Google Scholar 

  • Nair V, Kosal Ram PG, Sundararaman S (2019) Shadow detection and removal from images using machine learning and morphological operations. J Eng 2019(1):11–18

    Article  Google Scholar 

  • Najman L, Talbot H (2013) Mathematical Morphology: From Theory to Applications. Wiley, New York

    Book  Google Scholar 

  • Nasi R, Viljanen N, Kaivosoja J, Hakala T, Pandžić M, Markelin L, Honkavaara E (2017) Assessment of various remote sensing technologies in biomass and nitrogen content estimation using an agricultural test field. Int Arch Photogramm Remote Sens Spatial Inf Sci (ISPRS) 42W3:137–141

    Article  Google Scholar 

  • Negri RG, Silva E, Casaca W (2018) Inducing contextual classifications with kernel functions into support vector machines. IEEE Geosci Remote Sens Lett 15(6):962–966

    Article  Google Scholar 

  • Negri RG, Frery AC, Casaca W, Azevedo S, Dias MA, Silva E, Alcântara EH (2020) Spectral-spatial-aware unsupervised change detection with stochastic distances and support vector machines. IEEE Trans Geosci Remote Sens 1(1):1–14

    Google Scholar 

  • Nikolakopoulos KG, Raptis I (2014) Open quarry monitoring using gap-filled landsat 7 etm slc-off imagery. In: Earth resources and environmental remote sensing/GIS applications V, vol 9245, pp 32–42

  • Oliveira HC, Guizilini VC, Nunes IP, Souza JR (2018) Failure detection in row crops from uav images using morphological operators. IEEE Geosci Remote Sens Lett 15(7):991–995

    Article  Google Scholar 

  • Patel A, Prajapati R, Dharpure JK, Mani S, Chauhan D (2019) Mapping and monitoring of glacier areal changes using multispectral and elevation data: A case study over chhota-shigri glacier. Earth Sci Inf 12:489–499

    Article  Google Scholar 

  • Rawat JS, Kumar M (2015) Monitoring land use/cover change using remote sensing and gis techniques: A case study of hawalbagh block, district almora, uttarakhand, india. Egyptian J Remote Sens Space Sci 18(1):77–84

    Article  Google Scholar 

  • Romero-Sanchez ME, Ponce-Hernandez R, Franklin SE, Aguirre-Salado CA (2015) Comparison of data gap-filling methods for landsat etm + slc-off imagery for monitoring forest degradation in a semi-deciduous tropical forest in mexico. Int J Remote Sens 36(11):2786–2799

    Article  Google Scholar 

  • Rudin LI, Osher S, Fatemi E (1992) Nonlinear total variation based noise removal algorithms. Physica D 60(1–4):259–268

    Article  Google Scholar 

  • Sadiq A, Sulong G, Edwar L (2016) Recovering defective landsat 7 enhanced thematic mapper plus images via multiple linear regression model. IET Comput Vis 10(8):788–797

    Article  Google Scholar 

  • Sadiq A, Edwar L, Sulong G (2017) Recovering the large gaps in landsat 7 slc-off imagery using weighted multiple linear regression (wmlr). Arab J Geosci 10(403):1–14

    Google Scholar 

  • Schönlieb CB (2015) Second-Order Diffusion Equations for Inpainting. Cambridge University Press, Cambridge, pp 32–62. Partial Differential Equation Methods for Image Inpainting

    Google Scholar 

  • Seshadrinathan K, Pappas TN, Safranek RJ, Chen J, Wang Z, Sheikh HR, Bovik AC (2009) Chapter 21 - image quality assessment. In: Bovik A (ed) The Essential Guide to Image Processing. Academic Press, Boston, pp 553–595

  • Shen H, Li X, Cheng Q, Zeng C, Yang G, Li H, Zhang L (2015) Missing information reconstruction of remote sensing data: A technical review. IEEE Geosci Remote Sens Magaz 3(3):61–85

    Article  Google Scholar 

  • Shen J, Chan TF (2002) Mathematical models for local nontexture inpaintings. SIAM J Appl Math 62(3):1019–1043

    Article  Google Scholar 

  • Shroff M, Bombaywala SR (2019) A qualitative study of exemplar based image inpainting. SN Appl Sci 1:1730

    Article  Google Scholar 

  • Tai SC, Chen PY, Chao CY (2016) Missing pixels restoration for remote sensing images using adaptive search window and linear regression. J Elect Imaging 25(4):1–8

    Article  Google Scholar 

  • Thanh DNH, Prasath VBS, Dvoenko S, Hieu LM (2021) An adaptive image inpainting method based on euler’s elastica with adaptive parameters estimation and the discrete gradient method. Signal Process 178:107797

    Article  Google Scholar 

  • Wang Y, Yang Y, Chen T (2017) Spectral-spatial adaptive and well-balanced flow-based anisotropic diffusion for multispectral image denoising. J Vis Commun Image Represent 43:185–197

    Article  Google Scholar 

  • Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612

    Article  Google Scholar 

  • Zeng C, Shen H, Zhang L (2013) Recovering missing pixels for landsat etm + slc-off imagery using multi-temporal regression analysis and a regularization method. Remote Sens Environ 131:182–194

    Article  Google Scholar 

  • Zhao R, Peng GH, dong Yan W, Pan LL, Wang LY (2020) Change detection in sar images based on superpixel segmentation and image regression. Earth Sci Inf 14:69–79

    Article  Google Scholar 

Download references

Acknowledgements

This research has been supported by São Paulo Research Foundation (FAPESP), grants #2019/24259-0 and #2018/06756-3, and by National Council for Scientific and Technological Development (CNPq), grant #427915/2018-0. The authors declare no conflicts of interest.

Author information

Authors and Affiliations

  1. Department of Energy Engineering, São Paulo State University, Rosana, SP, Brazil

    Dayara Basso, Marilaine Colnago & Wallace Casaca

  2. Natural Resources Institute, Federal University of Itajubá, Itajubá, MG, Brazil

    Samara Azevedo

  3. Department of Cartography, São Paulo State University, Presidente Prudente, SP, Brazil

    Erivaldo Silva

  4. CERENA, IST, University of Lisbon, Lisbon, Portugal

    Pedro Pina

Authors
  1. Dayara Basso
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Marilaine Colnago
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Samara Azevedo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Erivaldo Silva
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Pedro Pina
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Wallace Casaca
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Wallace Casaca.

Additional information

Communicated by: H. Babaie

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Basso, D., Colnago, M., Azevedo, S. et al. Combining morphological filtering, anisotropic diffusion and block-based data replication for automatically detecting and recovering unscanned gaps in remote sensing images. Earth Sci Inform 14, 1145–1158 (2021). https://doi.org/10.1007/s12145-021-00613-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12145-021-00613-6

Keywords

Search

Navigation