Skip to main content
SpringerLink
Log in

Significant full reference image segmentation evaluation: a survey in remote sensing field

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Image segmentation is a crucial step in remote sensing application, as it breaks down a larger image into smaller chunks, which contains useful information. Although there are several image segmentation algorithms available, evaluation of the algorithms is challenging. Furthermore, the evaluation of image segmentation can elucidate the best image segmentation algorithm for a single image or a group of image or a whole class of image. This paper explores and evaluates the benefits and the drawbacks of various qualitative and quantitative image segmentation evaluation metrics used in remote sensing applications. For all the metrics, a quantitative set of values for good and bad segmentation is provided. In addition, some image segmentation algorithms such as Multi Otsu Threshold, K Means clustering, Fuzzy C Means clustering, Improved K Means clustering (IFCM), Improved Fuzzy C Means clustering, Naïve Bayes classifier, K Nearest Neighbor, Decision Tree (DT) and Random Forest classifier are used in the experimental comparison of metrics. The qualitative and quantitative satellite image segmentation evaluation using the mentioned algorithms is measured. The results are analyzed to strengthen the impact of different metrics on the segmentation algorithms. In both qualitative and quantitative analysis, the IFCM outperformed the other unsupervised machine learning algorithms and the DT outperformed the other supervised machine learning algorithms. The effectiveness of the provided metrics in the remote sensing field is validated.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Andrefouet S, Claereboudt M (2000) Objective class definitions using correlation of similarities between remotely sensed and environmental data. Int J Remote Sens 21(9):1925–1930. https://doi.org/10.1080/014311600209832

  2. Athi (2021) Image Quality Measures. (https://www.mathworks.com/matlabcentral/fileexchange/25005-image-quality-measures), MATLAB Central File Exchange. Retrieved May 12, 2021

  3. Bezdek JC, Moshtaghi M, Runkler T, Leckie C (2016) The generalized C index for internal fuzzy cluster validity. IEEE Trans Fuzzy Syst 24(6):1500–1512. https://doi.org/10.1109/TFUZZ.2016.2540063

    Article  Google Scholar 

  4. Chen Z, Zhu H (2019) Visual quality evaluation for semantic segmentation: subjective assessment database and objective assessment measure. IEEE Trans Image Process 28(12):5785–5796. https://doi.org/10.1109/TIP.2019.2922072

    Article  MathSciNet  MATH  Google Scholar 

  5. Chen G-H, Yang C-L, Po L-M, Xie S-L (2006) Edge-based structural similarity for image quality assessment. In: IEEE International Conference on Acoustics Speech and Signal Processing Proceedings, Toulouse, France. https://doi.org/10.1109/ICASSP.2006.1660497

    Chapter  Google Scholar 

  6. Choudhry MS, Kapoor R (2016) Performance analysis of fuzzy C-means clustering methods for MRI image segmentation. Procedia Computer Science 89:749–758. https://doi.org/10.1016/j.procs.2016.06.052

    Article  Google Scholar 

  7. Dey V, Zhang Y, Zhong ZM (2010) A review on image segmentation techniques with remote sensing perspective. In: Proc. ISPRS TC 7th Symp.—100 years ISPRS. Vienna Univ. of Technology, Vienna, Austria, pp 31–42

    Google Scholar 

  8. Draper C, Reichle R, de Jeu R, Naeimi V, Parinussa R, Wagner W (2013) Estimating root mean square errors in remotely sensed soil moisture over continental scale domains. Remote Sens Environ 137:288–298. https://doi.org/10.1016/j.rse.2013.06.013

    Article  Google Scholar 

  9. Eskicioglu AM, Fisher PS (1995) Image quality measures and their performance. IEEE Transactions on Communications 43(12):2959–2965. https://doi.org/10.1109/26.477498

    Article  Google Scholar 

  10. Gao B, Wang J (2015) Multi-objective fuzzy clustering for synthetic aperture radar imagery. IEEE Geosci Remote Sens Lett 12(11):2341–2345. https://doi.org/10.1109/LGRS.2015.2477500

    Article  Google Scholar 

  11. Gao H, Tang Y, Jing L, Li H, Ding H (2017) A novel unsupervised segmentation quality evaluation method for remote sensing images. Sensors (Basel, Switzerland) 17(10):2427. https://doi.org/10.3390/s17102427

    Article  Google Scholar 

  12. Gonzalez-Audicana M, Saleta JL, Catalan RG, Garcia R (2004) Fusion of multispectral and panchromatic images using improved IHS and PCA mergers based on wavelet decomposition. IEEE Trans Geosci Remote Sens 42(6):1291–1299. https://doi.org/10.1109/TGRS.2004.825593

    Article  Google Scholar 

  13. Hai YZ, Kun W, Jianfeng J, Lihua Z (2006) Improvements of the definition of image fidelity. In: 12th International Multi-Media Modelling Conference. https://doi.org/10.1109/MMMC.2006.1651364

    Chapter  Google Scholar 

  14. Halkidi M, Vazirgiannis M (2001) Clustering validity assessment: finding the optimal partitioning of a data set. In: Proceedings 2001 IEEE international conference on data mining, pp 187–194. https://doi.org/10.1109/ICDM.2001.989517

    Chapter  MATH  Google Scholar 

  15. Hyndman RJ, Koehler AB (2006) Another look at measures of forecast accuracy. Int J Forecast 22(4):679–688. https://doi.org/10.1016/j.ijforecast.2006.03.001

    Article  Google Scholar 

  16. Hyndman RJ, Koehler AB (2006) Another look at measures of forecast accuracy. Int J Forecast 22(4):679–688. https://doi.org/10.1016/j.ijforecast.2006.03.001

    Article  Google Scholar 

  17. Imamura K, Kuroda H, Fujimura M (2011) Image content detection method using correlation coefficient between pixel value histograms. Signal Processing, Image Processing and Pattern Recognition. https://doi.org/10.1007/978-3-642-27183-0_1

  18. Khair U, Fahmi H, Hakim S, Rahim R (2017) Forecasting error calculation with mean absolute deviation and mean absolute percentage error. J Phys Conf Ser 930:012002. https://doi.org/10.1088/1742-6596/930/1/012002

    Article  Google Scholar 

  19. Kragic D, Christensen HI (2015) Chapter 5.3 - Robust Visual Servoing1. In: Xu Y, Qian H, Wu X (eds) Household Service Robotics. Academic Press, pp 397–427. https://doi.org/10.1016/B978-0-12-8008812.00018-9

    Chapter  Google Scholar 

  20. Land DD, Bouldin DW (1979) A Cluster Separation Measure. IEEE Transactions on Pattern Analysis and Machine Intelligence PAMI 1(2):224–227. https://doi.org/10.1109/TPAMI.1979.4766909

    Article  Google Scholar 

  21. Li W, Zou Z, Shi Z (2020) Deep matting for cloud detection in remote sensing images. IEEE Trans Geosci Remote Sens 58(12):8490–8502. https://doi.org/10.1109/TGRS.2020.2988265

    Article  Google Scholar 

  22. Liu X, Yang C (2013) A kernel spectral angle mapper algorithm for remote sensing image classification. In: 6th international congress on image and signal processing (CISP), pp 814–818. https://doi.org/10.1109/CISP.2013.6745277

    Chapter  Google Scholar 

  23. Lu T, Wang J, Zhang Y, Wang Z, Jiang J (2019) Satellite image super-resolution via multi-scale residual deep neural network. Remote Sens 11(13):1588. https://doi.org/10.3390/rs11131588

    Article  Google Scholar 

  24. Matkovi K, Neumann L, Neumann A, Psik T, Purgathofer W (2005) Global contrast factor—a new approach to image contrast. In: Proc. of the first Eurographics conference on computational aesthetics in graphics, visualization and imaging; Girona, Spain. Eurographics Association, Aire-la-Ville, Switzerland, pp 159–167. https://doi.org/10.5555/2381219.2381242

    Chapter  Google Scholar 

  25. Meilă M (2003) Comparing Clusterings by the variation of information. In: Lecture Notes in Computer Science. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-45167-9_14

    Chapter  Google Scholar 

  26. Memon F, Unar MA, Memon S (2015) Image quality assessment for performance evaluation of focus measure operators. Mehran University Research Journal of Engineering & Technology 34(4):379–386 https://arxiv.org/abs/1604.00546

    Google Scholar 

  27. Mignotte M (2008) Segmentation by fusion of histogram-based $K$-means clusters in different color spaces. IEEE Trans Image Process 17(5):780–787. https://doi.org/10.1109/TIP.2008.920761

  28. Neto AM, Victorino AC, Fantoni I, Zampieri DE, Ferreira JV, Lima DA (2013) Image processing using Pearson's correlation coefficient: applications on autonomous robotics. In: 13th international conference on autonomous robot systems, pp 1–6. https://doi.org/10.1109/Robotica.2013.6623521

    Chapter  Google Scholar 

  29. Ossama O, Mokhtar HMO, El-Sharkawi ME (2011) An extended k-means technique for clustering moving objects. Egyptian Informatics Journal 12(1):45–51. https://doi.org/10.1016/j.eij.2011.02.007

    Article  Google Scholar 

  30. Pont-Tuset J, Marques F (2013) Measures and meta-measures for the supervised evaluation of image segmentation. In: IEEE conference on computer vision and pattern recognition, pp 2131–2138. https://doi.org/10.1109/CVPR.2013.277

    Chapter  Google Scholar 

  31. Surya Prabha D, Satheesh Kumar J (2016) Performance Evaluation of Image Segmentation using Objective Methods. Indian Journal of Science and Technology 9(8):1–8. https://doi.org/10.17485/ijst/2016/v9i8/87907

    Article  Google Scholar 

  32. Pradhan B, Al-Najjar HAH, Sameen MI, Mezaal M, Alamri M (2020) Landslide detection using a saliency feature enhancement technique from LiDAR-derived DEM and Orthophotos. IEEE Access 8:121942–121954. https://doi.org/10.1109/ACCESS.2020.3006914

    Article  Google Scholar 

  33. Pugazhenthi A, Kumar LS (2020) Automatic cloud segmentation from INSAT-3D satellite image via IKM and IFCM clustering. IET Image Process 14:1273–1280. https://doi.org/10.1049/iet-ipr.2018.5271

    Article  Google Scholar 

  34. Pugazhenthi A, Singhai J (2014) Automatic centroids selection in K-means clustering based image segmentation. In: International conference on communication and signal processing, pp 1279–1284. https://doi.org/10.1109/ICCSP.2014.6950057

    Chapter  Google Scholar 

  35. Rohith G, Kumar LS (2021) Paradigm shifts in super-resolution techniques for remote sensing applications. Vis Comput 37:1965–2008. https://doi.org/10.1007/s00371-020-01957-8

    Article  Google Scholar 

  36. Rohlf FJ (1974) Methods of comparing classifications. Annu Rev Ecol Syst 5:101–113. https://doi.org/10.1146/annurev.es.05.110174.000533

    Article  Google Scholar 

  37. Sharma N, Aggarwal LM (2010) Automated medical image segmentation techniques. Journal of Medical Physics 35(1):3–14. https://doi.org/10.4103/0971-6203.58777

    Article  Google Scholar 

  38. Unnikrishnan R, Pantofaru C, Hebert M (2007) Toward objective evaluation of image segmentation algorithms. IEEE Trans Pattern Anal Mach Intell 29(6):929–944. https://doi.org/10.1109/TPAMI.2007.1046

    Article  Google Scholar 

  39. Wang Z, Bovik AC (2002) A universal image quality index. IEEE Signal Processing Letters 9(3):81–84. https://doi.org/10.1109/97.995823

    Article  Google Scholar 

  40. Wang Z, Bovik AC, Sheikh HR, Simoncelli EP (2004) Image quality assessment: from error visibility to structural similarity. IEEE Trans Image Process 13(4):600–612. https://doi.org/10.1109/TIP.2003.819861

    Article  Google Scholar 

  41. Wang Z, Wang E, Zhu Y (2020) Image segmentation evaluation: a survey of methods. Artif Intell Rev 53:5637–5674. https://doi.org/10.1007/s10462-020-09830-9

    Article  Google Scholar 

  42. Wang P, Wang P, Wang C, Yuan Y, Wang D (2020) A center location algorithm for tropical cyclone in satellite infrared images. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 13:2161–2172. https://doi.org/10.1109/JSTARS.2020.2995158

    Article  Google Scholar 

  43. Wei SD, Lai SH (2008) Fast template matching based on normalized cross correlation with adaptive multilevel winner update. IEEE Trans Image Process 17(11):2227–2235. https://doi.org/10.1109/TIP.2008.2004615

    Article  MathSciNet  MATH  Google Scholar 

  44. Wen F, Zhang Y, Gao Z, Ling X (2018) Two-pass robust component analysis for cloud removal in satellite image sequence. IEEE Geosci Remote Sens Lett 15(7):1090–1094. https://doi.org/10.1109/LGRS.2018.2829028

    Article  Google Scholar 

  45. Xu Z (2012) Intuitionistic fuzzy aggregation techniques. In: Studies in fuzziness and soft computing. Springer, Berlin, Heidelberg, p 279. https://doi.org/10.1007/978-3-642-28406-9_1

    Chapter  Google Scholar 

  46. Žalik KR, Žalik B (2011) Validity index for clusters of different sizes and densities. Pattern Recogn Lett 32(2):221–234. https://doi.org/10.1016/j.patrec.2010.08.007

    Article  MATH  Google Scholar 

  47. Žalik KR, Žalik B (2011) Validity index for clusters of different sizes and densities. Pattern Recogn Lett 32(2):221–234. https://doi.org/10.1016/j.patrec.2010.08.007

    Article  MATH  Google Scholar 

  48. Zhang H, Fritts JE, Sally A, Goldman (2008) Image segmentation evaluation: a survey of unsupervised methods. Comput Vis Image Underst 110(2):260–280. https://doi.org/10.1016/j.cviu.2007.08.003

    Article  Google Scholar 

  49. Zhang L, Zhang L, Mou X, Zhang D (2011) FSIM: a feature similarity index for image quality assessment. IEEE Trans Image Process 20(8):2378–2386. https://doi.org/10.1109/TIP.2011.2109730

    Article  MathSciNet  MATH  Google Scholar 

  50. Zhao Y, Karypis G (2004) Empirical and theoretical comparisons of selected criterion functions for document clustering. Mach Learn 55:311–331. https://doi.org/10.1023/B:MACH.0000027785.44527.d6

    Article  MATH  Google Scholar 

Download references

Acknowledgments

The authors thank Indian Meteorological Department for providing the Radiosonde Data and the Rain Data.

Author information

Authors and Affiliations

  1. Department of Electronics and Communication Engineering, National Institute of Technology Puducherry, Puducherry, India

    Pugazhenthi A, Sruthy Sebastian, G. Rohith & Lakshmi Sutha Kumar

Authors
  1. Pugazhenthi A
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sruthy Sebastian
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. G. Rohith
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Lakshmi Sutha Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Pugazhenthi A.

Ethics declarations

Conflict of interest and authorship conformation form

Please check the following as appropriate:

° All authors have participated in (a) conception and design, or analysis and interpretation of the data; (b) drafting the article or revising it critically for important intellectual content; and (c) approval of the final version.

° This manuscript has not been submitted to, nor is under review at, another journal or other publishing venue.

°The authors have no affiliation with any organization with a direct or indirect financial interest in the subject matter discussed in the manuscript.

°The following authors have affiliations with organizations with direct or indirect financial interest in the subject matter discussed in the manuscript:

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

A, P., Sebastian, S., Rohith, G. et al. Significant full reference image segmentation evaluation: a survey in remote sensing field. Multimed Tools Appl 81, 17959–17987 (2022). https://doi.org/10.1007/s11042-022-12769-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-12769-4

Keywords

Search

Navigation