Skip to main content
SpringerLink
Log in

Multi-scale region composition of hierarchical image segmentation

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Hierarchical image segmentation is a prominent trend in the literature as a way to improve the segmentation quality. Generally, meaningful objects in an image are described by segments from different scales. Thus, one may spend extra effort on searching for the best representation of objects in the hierarchical segmentation result. In this paper, a novel algorithm is proposed to optimally select the segmentation scale, which leads to a composite segmentation as the output. To this end, the quality of regions from different scales of the hierarchical segmentation is evaluated. Then, a graphical model is constructed as a set of nodes. The weights among nodes are computed according to the segmentation quality of regions in multiple levels. In order to optimize the labeling of each node in the graph, the composition process is performed twice with two sampling intervals. Comprehensive experiments are conducted on different datasets for popular hierarchical image segmentation algorithms. The results show that the output of the proposed algorithm can improve the quality of hierarchical segmentation in a single scale at a low cost of computation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Notes

  1. https://www.kaggle.com/kmader/finding-lungs-in-ct-data

References

  1. Al-Huda Z, Peng B, Yang Y, Ahmed M (2019) Object scale selection of hierarchical image segmentation using reliable regions. In: IEEE international conference on intelligent systems and knowledge engineering, pp 1171–1178

  2. Alpert S, Galun M, Basri R, Brandt A (2012) Image segmentation by probabilistic bottom-up aggregation and cue integration. IEEE Trans Pattern Anal Mach Intell 34(2):315–326

    Article  Google Scholar 

  3. Arbeláez P, Maire M, Fowlkes C, Malik J (2011) Contour detection and hierarchical image segmentation. IEEE Trans Pattern Anal Mach Intell 33(5):898–916

    Article  Google Scholar 

  4. Bertasius G, Shi J, Torresani L (2015) Deepedge: a multi-scale bifurcated deep network for top-down contour detection. In: IEEE conference on computer vision and pattern recognition, pp 4380–4389

  5. Boykov Y, Veksler O, Zabih R (2001) Fast approximate energy minimization via graph cuts. IEEE Trans Pattern Anal Mach Intell 23:1222–1239

    Article  Google Scholar 

  6. Cai Q, Liu H, Zhou S, Sun J, Li J (2018) An adaptive-scale active contour model for inhomogeneous image segmentation and bias field estimation. Pattern Recognit 82:79–93

    Article  Google Scholar 

  7. Calderero F, Marques F (2010) Region merging techniques using information theory statistical measures. IEEE Trans Image Process 19(6):1567–1586

    Article  MathSciNet  MATH  Google Scholar 

  8. Chen Y, Dai D, Pont-Tuset J, Gool LV (2016) Scale-aware alignment of hierarchical image segmentation. In: IEEE conference on computer vision and pattern recognition, pp 364–372

  9. Comaniciu D, Meer P (2002) Mean shift: a robust approach toward feature space analysis. IEEE Trans Pattern Anal Mach Intell 24:603–619

    Article  Google Scholar 

  10. Diwakar M, Kumar M (2018) Ct image denoising using locally adaptive shrinkage rule in tetrolet domain. J King Saud Univ—Comput Inf Sci 30(1):41–50

    Article  Google Scholar 

  11. Diwakar M, Kumar M (2018) Ct image denoising using nlm and correlation-based wavelet packet thresholding. IET Image Process 12(5):708–715

    Article  Google Scholar 

  12. Diwakar M, Kumar M (2018) A review on ct image noise and its denoising. Biomed Signal Process Control 42:73–88

    Article  Google Scholar 

  13. Diwakar M, Singh P (2020) Ct image denoising using multivariate model and its method noise thresholding in non-subsampled shearlet domain. Biomed Signal Process Control 57:101754

    Article  Google Scholar 

  14. Everingham M, Gool LV, Williams C, Winn J, Zisserman A (2009) The pascal visual object classes (voc) challenge. Int J Comput Vis 88 (2):303–338

    Article  Google Scholar 

  15. Felzenszwalb P, Huttenlocher D (2004) Efficient graph based image segmentation. Int J Comput Vis 59(2):167–181

    Article  Google Scholar 

  16. Franek L, Abdala D, Vega-Pons S, Jiang X (2010) Image segmentation fusion using general ensemble clustering methods. In: Asian conference of computer vision, pp 373–384

  17. Golodetz S, Voiculescu I, Cameron S (2017) Simpler editing of graph-based segmentation hierarchies using zipping algorithms. Pattern Recognit 70:44–59

    Article  Google Scholar 

  18. Gould S, Fulton R, Koller D (2009) Decomposing a scene into geometric and semantically consistent regions. In: IEEE conference on computer vision and pattern recognition, pp 1–8

  19. He J, Zhang S, Yang M, Shan Y, Huang T (2019) Bi-directional cascade network for perceptual edge detection. In: IEEE conference on computer vision and pattern recognition

  20. Huang Q, Dom B (1995) Quantitative methods of evaluating image segmentation. In: International conference on image processing, p 3053

  21. Isola P, Zoran D, Krishnan D, Adelson EH (2014) Crisp boundary detection using pointwise mutual information. In: European conference on computer vision, pp 799–814

  22. Khoreva A, Benenson R, Hosang J, Hein M, Schiele B (2017) Simple does i: weakly supervised instance and semantic segmentation. In: IEEE conference on computer vision and pattern recognition, pp 1665–1674

  23. Kim TH, Lee KM, Lee SU (2013) Learning full pairwise affinities for spectral segmentation. IEEE Trans Pattern Anal Mach Intell 35(7):1690–1703

    Article  Google Scholar 

  24. Kumar M, Diwakar M (2016) A new exponentially directional weighted function based ct image denoising using total variation. J King Saud Univ Comput Inf Sci S1319157816301458

  25. Leung T, Malik J (2001) Representing and recognizing the visual appearance of materials using three-dimensional textons. Int J Comput Vis 43(1):29–44

    Article  MATH  Google Scholar 

  26. Li Y, Feng X (2016) A multiscale image segmentation method. Pattern Recognit 52:332–345

    Article  Google Scholar 

  27. Li H, Cai J, Nguyen T, Zheng J (2013) A bechmark for sematic image segmentation. In: IEEE international conference on multimedia and expo

  28. Li K, Tao W, Liu X, Liu L (2018) Iterative image segmentation with feature driven heuristic four-color labeling. Pattern Recognit 76:69–79. https://doi.org/10.1016/j.patcog.2017.10.023

    Article  Google Scholar 

  29. Liao X, Li K, Yin J (2017) Separable data hiding in encrypted image based on compressive sensing and discrete fourier transform. Multimed Tools Appl 76:20739–20753

    Article  Google Scholar 

  30. Liao X, Zheng Q, Li P (2017) Data embedding in digital images using critical functions. Signal Process: Image Commun 58:146–156

    Article  Google Scholar 

  31. Liu Y, Cheng M, Hu X, Bian J, Zhang L, Bai X, Tang J (2019) Richer convolutional features for edge detection. IEEE Trans Pattern Anal Mach Intell 41(8):1939–1946

    Article  Google Scholar 

  32. Long J, Shelhamer E, Darrell T (2017) Fully convolutional networks for semantic segmentation. IEEE Trans Pattern Anal Mach Intell 39(4):640–651

    Article  Google Scholar 

  33. Ma X, Peng B, Gong X, Zheng Y, Li T (2019) Hierarchical region merging for multi-scale image segmentation. In: IEEE international conference on intelligent systems and knowledge engineering, pp 1027–1034

  34. Malisiewicz T, Efros A (2007) Improving spatial support for objects via multiple segmentations. In: The British machine vision conference

  35. Maninis KK, Pont-Tuset J, Arbeláez P, Gool LV (2016) Convolutional oriented boundaries. In: European conference on computer vision, pp 580–596

  36. Martin D (2002) An empirical approach to grouping and segmentation. Ph.D. thesis, EECS Department, University of California, Berkeley

  37. Martin D, Fowlkes C, Tal D, Malik J (2001) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: IEEE international conference on computer vision, pp 416–424

  38. Meila M (2005) Comparing clusterings: an axiomatic view. In: International conference on machine learning, pp 577–584

  39. Min X, Zhai G, Gu K, Zhu Y, Zhou J, Guo G, Yang X, Guan X, Zhang W (2019) Quality evaluation of image dehazing methods using synthetic hazy images. IEEE Trans Multimed 21(9):2319–2333

    Article  Google Scholar 

  40. Pagnutti G, Zanuttigh P (2018) Joint segmentation of color and depth data based on splitting and merging driven by surface fitting. Image Vis Comput 70:21–31

    Article  Google Scholar 

  41. Pont-Tuset J, Marques F (2010) Supervised assessment of segmentation hierarchies. In: European conference on computer vision, pp 814–827

  42. Pont-Tuset J, Arbeláez P, Barron J, Marques F, Malik J (2016) Multiscale combinatorial grouping for image segmentation and object proposal generation. IEEE Trans Pattern Anal Mach Intell 39(1):128–140

    Article  Google Scholar 

  43. Ren Z, Shakhnarovich G (2013) Image segmentation by cascaded region agglomeration. In: IEEE conference on computer vision and pattern recognition, pp 2011–2018

  44. Russell B, Freeman W, Efros A, Sivic J, Zisserman A (2006) Using multiple segmentations to discover objects and their extent in image collections. In: IEEE conference on computer vision and pattern recognition, pp 1605–1614

  45. Salembier P, Garrido L (2014) Binary partition tree as an efficient representation for image processing, segmentation, and information retrieval. IEEE Trans Image Process 71:561–576

    Google Scholar 

  46. Schroff F, Kalenichenko D, Philbin J (2015) Facenet: a unified embedding for face recognition and clustering. In: IEEE conference on computer vision and pattern recognition, pp 815–823

  47. Shen W, Wang X, Wang Y, Bai X, Zhang Z (2015) Deepcontour: a deep convolutional feature learned by positive-sharing loss for contour detection. In: IEEE conference on computer vision and pattern recognition, pp 3982–3991

  48. Shi J, Malik J (1997) Normalized cuts and image segmentation. In: IEEE conference on computer vision and pattern recognition, p 731

  49. Shi J, Malik J (2000) Normalized cuts and image segmentation. IEEE Trans Pattern Anal Mach Intell 22(8):888–905

    Article  Google Scholar 

  50. Shimoda W, Yanai K (2018) Weakly supervised semantic segmentation using distinct class specific saliency maps. In: Computer vision and image understanding. https://doi.org/10.1016/j.cviu.2018.08.006

  51. Shotton J, Winn J, Rother C, Criminisi A (2006) Textonboost: joint appearance, shape and context modeling for multi-class object recognition and segmentation. In: European conference on computer vision, pp 1–15

  52. Syu J, Wang S, Wang L (2017) Hierarchical image segmentation based on iterative contraction and merging. IEEE Trans Image Process 26 (5):2246–2260

    Article  MathSciNet  MATH  Google Scholar 

  53. Unnikrishnan R, Hebert M (2005) Measures of similarity. In: IEEE workshop computer vision applications, pp 394–394

  54. Unnikrishnan R, Pantofaru C, Hebert M (2007) Toward objective evaluation of image segmentation algorithms. IEEE Trans Pattern Anal Mach Intell 29(6):929–944

    Article  Google Scholar 

  55. Wang H, Zhang Y, Nie R, Yang Y, Peng B, Li T (2014) Bayesian image segmentation fusion. Knowl-Based Syst 71:162–168

    Article  Google Scholar 

  56. Xie S, Tu Z (2017) Holistically-nested edge detection. Int J Comput Vis 125(1-3):3–18

    Article  MathSciNet  Google Scholar 

  57. Xu C, Whitt S, Corso J (2013) Flattening supervoxel hierarchies by the uniform entropy slice. In: IEEE international conference on computer vision, pp 2240–2247

  58. Yin S, Qian Y, Gong M (2017) Unsupervised hierarchical image segmentation through fuzzy entropy maximization. Pattern Recognit 68:245–259. https://doi.org/10.1016/j.patcog.2017.03.012

    Article  Google Scholar 

  59. Yu Z, Feng C, Liu M, Ramalingam S (2017) Casenet: deep category-aware semantic edge detection. In: IEEE conference on computer vision and pattern recognition, pp 1761–1770

  60. Zhao Q (2015) Segmenting natural images with the least effort as humans. In: The British machine vision conference, pp 110.1–110.12

Download references

Author information

Authors and Affiliations

  1. School of Information Science and Technology, Southwest Jiaotong University, Chengdu, 611756, China

    Bo Peng, Zaid Al-Huda & Zhuyang Xie

  2. National Engineering Laboratory of Integrated Transportation Big Data Application Technology, Chengdu, China

    Bo Peng, Zaid Al-Huda & Zhuyang Xie

  3. School of Computer Science, Chengdu University of Information Technology, Chengdu, 610225, China

    Xi Wu

Authors
  1. Bo Peng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Zaid Al-Huda
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhuyang Xie
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xi Wu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bo Peng.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work was supported by the National Natural Science Foundation of China under Grant 61772435, Sichuan Highway Science and Technology Project under Grant 2019-01.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Peng, B., Al-Huda, Z., Xie, Z. et al. Multi-scale region composition of hierarchical image segmentation. Multimed Tools Appl 79, 32833–32855 (2020). https://doi.org/10.1007/s11042-020-09346-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-020-09346-y

Keywords

Search

Navigation