Skip to main content
SpringerLink
Log in

Interactive image segmentation based on multi-layer random forest classifiers

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Since fully automatic image segmentation methods often fail for most complex images, researchers turn to the interactive segmentation paradigm to achieve better segmentation performance. However, many interactive image segmentation algorithms are highly dependent on user interactive information. This paper presents a novel interactive image segmentation algorithm based on multi-layer random forests. Given a small amount of user input markers, region merging is done according to the merging rule, in which both the color histogram and gradient orientation histogram of the region are included to avoid the merging error. To speed up the calculation of gradient orientation histogram, breadth-first search is used to determine the intersection of two adjacent regions. Then, we relabel the training samples with k-means algorithm and Silhouette index and further perform the first layer random forest classification. Next, we reconstruct the training samples with the adjacent superpixel pairs and use the second layer random forest classifiers to classify the superpixels whose prediction confidence is lower than the threshold after the first layer random forest classification. Experiments on real natural images are conducted to demonstrate the performance of the proposed algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Algorithm 1
Fig. 3
Algorithm 2
Fig. 4
Algorithm 3
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

Notes

  1. The MATLAB source code is publicly available at https://github.com/SYiLin/Paper_MRFIS.git.

References

  1. Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Süsstrunk S (2012) SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans Pattern Anal Mach Intell 34(11):2274–2282

    Article  Google Scholar 

  2. Acuna D, Ling H, Kar A, Fidler S (2018) Efficient interactive annotation of segmentation datasets with polygon-rnn++. In proceedings of the IEEE conference on computer vision and pattern recognition. p 859–868

  3. Breiman L (2001) Random forests. Mach Learn 45(1):5–32

    Article  MATH  Google Scholar 

  4. Breve F (2019) Interactive image segmentation using label propagation through complex networks. Expert Syst Appl 123:18–33

    Article  Google Scholar 

  5. Castrejon L, Kundu K, Urtasun R, Fidler S (2017) Annotating object instances with a polygon-rnn. In proceedings of the IEEE conference on computer vision and pattern recognition. p 5230–5238

  6. Comaniciu D, Meer P (2002) Mean shift: a robust approach toward feature space analysis. IEEE Trans Pattern Anal Mach Intell 24(5):603–619

    Article  Google Scholar 

  7. Cordts M, Omran M, Ramos S, Rehfeld T, Enzweiler M, Benenson R, Franke U, Roth S, Schiele B (2016) The cityscapes dataset for semantic urban scene understanding. In proceedings of the IEEE conference on computer vision and pattern recognition. p 3213–3223

  8. Csillik O (2017) Fast segmentation and classification of very high resolution remote sensing data using SLIC superpixels. Remote Sens 9(3):243

    Article  Google Scholar 

  9. Eramian M, Power C, Rau S, Khandelwal P (2020) Benchmarking human performance in semi-automated image segmentation. Interact Comput 32(3):233–245

    Article  Google Scholar 

  10. Gu Y, Si B, Liu B (2021) A novel hierarchical model in ensemble environment for road detection application. Remote Sens 13(6):1213

    Article  Google Scholar 

  11. Hariharan B, Arbeláez P, Bourdev L, Maji S, Malik J (2011) Semantic contours from inverse detectors. In 2011 international conference on computer vision. IEEE, p 991–998

  12. Hu Z, Shi T, Wang C, Li Q, Wu G (2021) Scale-sets image classification with hierarchical sample enriching and automatic scale selection. Int J Appl Earth Obs Geoinf 105:102605

    Google Scholar 

  13. Jian M, Jung C (2016) Interactive image segmentation using adaptive constraint propagation. IEEE Trans Image Process 25(3):1301–1311

    Article  MathSciNet  MATH  Google Scholar 

  14. Jiang Q, Tawose OT, Pei S, Chen X, Jiang L, Wang J, Zhao D (2019) Weakly-supervised image semantic segmentation based on superpixel region merging. Big Data Cognitive Comput 3(2):31

    Article  Google Scholar 

  15. Krähenbühl P, Koltun V (2011) Efficient inference in fully connected crfs with gaussian edge potentials. Adv Neural Inf Proces Syst 24:109–117

    Google Scholar 

  16. Li Z, Chen Q, Koltun V (2018) Interactive image segmentation with latent diversity. In proceedings of the IEEE conference on computer vision and pattern recognition p 577–585

  17. Li M, Chen D, Liu S, Guo D (2021) Online learning method based on support vector machine for metallographic image segmentation. SIViP 15(3):571–578

    Article  Google Scholar 

  18. Li Y, Sun R, Liu Y, Yang Y, Ma S, Liu Y (2019) Interactive foreground segmentation and shape reconstruction from RGBD images. Comput Electr Eng 79:106463

    Article  Google Scholar 

  19. Li Y, Sun J, Tang C-K, Shum H-Y (2004) Lazy snapping. ACM Trans Graphics (ToG) 23(3):303–308

    Article  Google Scholar 

  20. Lin T-Y, Maire M, Belongie S, Hays J, Perona P, Ramanan D, Dollár P, Zitnick CL (2014) Microsoft coco: common objects in context. In European conference on computer vision. Springer, p 740–755

  21. Liu Y, Li Z, Xiong H, Gao X, Wu J, Wu S (2013) Understanding and enhancement of internal clustering validation measures. IEEE Trans Cybern 43(3):982–994. https://doi.org/10.1109/TSMCB.2012.2220543

    Article  Google Scholar 

  22. Liu X, Song M, Tao D, Bu J, Chen C (2015) Random geometric prior forest for multiclass object segmentation. IEEE Trans Image Process 24(10):3060–3070

    Article  MathSciNet  MATH  Google Scholar 

  23. Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In proceedings of the IEEE conference on computer vision and pattern recognition. p 3431–3440

  24. Martin D, Fowlkes C, Tal D, Malik J (2001) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In proceedings eighth IEEE international conference on computer vision. ICCV 2001. IEEE, p 416–423

  25. Ning J, Zhang L, Zhang D, Wu C (2010) Interactive image segmentation by maximal similarity based region merging. Pattern Recogn 43(2):445–456

    Article  MATH  Google Scholar 

  26. Peng Z, Qu S, Li Q (2019) Interactive image segmentation using geodesic appearance overlap graph cut. Signal Process Image Commun 78:159–170

    Article  Google Scholar 

  27. Peng B, Zhang L, Zhang D (2011) Automatic image segmentation by dynamic region merging. IEEE Trans Image Process 20(12):3592–3605

    Article  MathSciNet  MATH  Google Scholar 

  28. Perazzi F, Pont-Tuset J, McWilliams B, Van Gool L, Gross M, Sorkine-Hornung A (2016) A benchmark dataset and evaluation methodology for video object segmentation. In proceedings of the IEEE conference on computer vision and pattern recognition. p 724–732

  29. Pinto A, Pereira S, Rasteiro D, Silva CA (2018) Hierarchical brain tumour segmentation using extremely randomized trees. Pattern Recogn 82:105–117

    Article  Google Scholar 

  30. Prinke P, Haueisen J, Klee S, Rizqie MQ, Supriyanto E, König K, Breunig HG, Piątek Ł (2021) Automatic segmentation of skin cells in multiphoton data using multi-stage merging. Sci Rep 11(1):1–19

    Article  Google Scholar 

  31. Ramadan H, Lachqar C, Tairi H (2020) A survey of recent interactive image segmentation methods. Comput Vis Media 6:1–30

    Google Scholar 

  32. Ren X, Malik J (2003) Learning a classification model for segmentation. In IEEE international conference on computer vision. IEEE computer society, p 1–8

  33. Rother C, Kolmogorov V, Blake A (2004) " GrabCut" interactive foreground extraction using iterated graph cuts. ACM Trans Graphics (TOG) 23(3):309–314

    Article  Google Scholar 

  34. Shi J, Malik J (2000) Normalized cuts and image segmentation. IEEE Trans Pattern Anal Mach Intell 22(8):888–905

    Article  Google Scholar 

  35. Tang M, Gorelick L, Veksler O, Boykov Y (2013) Grabcut in one cut. In proceedings of the IEEE international conference on computer vision. p 1769–1776

  36. Vincent L, Soille P (1991) Watersheds in digital spaces: an efficient algorithm based on immersion simulations. IEEE Comput Archit Lett 13(6):583–598

    Google Scholar 

  37. Wang X-Y, Wu Z-F, Chen L, Zheng H-L, Yang H-Y (2016) Pixel classification based color image segmentation using quaternion exponent moments. Neural Netw 74:1–13

    Article  Google Scholar 

  38. Yu H, Zhou Y, Qian H, Xian M, Wang S (2017) Loosecut: interactive image segmentation with loosely bounded boxes. In 2017 IEEE international conference on image processing (ICIP). IEEE, p 3335–3339

  39. Zhao B, Cao Z, Wang S (2017) Lung vessel segmentation based on random forests. Electron Lett 53(4):220–222

    Article  Google Scholar 

  40. Zheng Q, Li H, Fan B, Wu S, Xu J (2018) Integrating support vector machine and graph cuts for medical image segmentation. J Vis Commun Image Represent 55:157–165

    Article  Google Scholar 

Download references

Acknowledgements

This work is partially supported by the National Natural Science Foundation of China (Grant no. 61373004).

Author information

Authors and Affiliations

  1. College of Information, Mechanical and Electrical Engineering, Shanghai Normal University, Shanghai, China

    Yilin Shan, Yan Ma, Yuan Liao, Hui Huang & Bin Wang

Authors
  1. Yilin Shan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yan Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yuan Liao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hui Huang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Bin Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yan Ma.

Ethics declarations

Conflict of interests

The authors declare that there is no conflict of interests regarding the publication of this article.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Shan, Y., Ma, Y., Liao, Y. et al. Interactive image segmentation based on multi-layer random forest classifiers. Multimed Tools Appl 82, 22469–22495 (2023). https://doi.org/10.1007/s11042-022-14199-8

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-14199-8

Keywords

Search

Navigation