Skip to main content
SpringerLink
Log in

Spatial color histogram-based image segmentation using texture-aware region merging

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

We propose a new image segmentation method using spatial-color histograms that include the color and spatial information of a given image. Previous methods used a histogram with only the color information of the image or did not effectively suppress the texture components of the same object to form segmented regions, and they frequently led to the false merging of two different regions. Thus, these methods caused an over-segmentation result in the same object or an under-segmentation result in the regional boundary between two different objects. To resolve these problems, the proposed method performs a clustering that considers both color and spatial information of the image in the histogram domain and texture-aware region merging. Moreover, using a total variation-based regularizer that can remove the texture components in the same object and preserve the edge components between different objects, we improve the accuracy of region merging process that is applied to the result of the proposed histogram-based segmentation. Compared to the best results obtained using previous histogram-based methods, the proposed method achieved improvements of 0.02335 (2.910%), 0.0195 (3.977%), 0.05515 (2.431%), and 0.9639 (9.250%) in probability rand index, segmentation covering, variation of information, and boundary displacement error, which are the most widely used for segmentation evaluation metrics, respectively. Further, when compared to the state-of-the-art methods, which use the superpixel, iterative contraction and merging, and deep learning-based methods, the proposed method provides promising segmentation quality with fast operation speed.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  1. Achanta R, Shaji A, Smith K, Lucchi A, Fua P, Susstrunk S (2012) SLIC superpixels compared to state-of-the-art superpixel methods. IEEE Trans Pattern Anal March Intell 34(11):2274–2282. https://doi.org/10.1109/TPAMI.2012.120

    Article  Google Scholar 

  2. Arbelaez P, Maire M, Fowlkes C, Malik J (2011) Contour detection and hierarchical image segmentation. IEEE Trans Pattern Anal Mach Intell 33(5):898–916. https://doi.org/10.1109/TPAMI.2010.161

    Article  Google Scholar 

  3. Chen T-W, Chen Y-L, Chien S-Y (2008) Fast image segmentation based on K-means clustering with histograms in HSV color space. In: Proc IEEE 10th workshop on Multimedia Signal Process, pp 322–325. https://doi.org/10.1109/MMSP.2008.4665097

    Chapter  Google Scholar 

  4. Cheng C-C, Li C-T, Chen L-G (2010) A novel 2D-to-3D conversion system using edge information. IEEE Trans Consum Electron 56(3):1739–1745. https://doi.org/10.1109/TCE.2010.5606320

    Article  Google Scholar 

  5. Cho H, Kang S-J, Cho SI, Kim YH (2014) Image segmentation using linked mean-shift vectors and its implementation on GPU. IEEE Trans Conum Electron 60(4):719–727. https://doi.org/10.1109/TCE.2014.7027348

    Article  Google Scholar 

  6. Cho SI, Kang S-J, Kim YH (2014) Human perception-based image segmentation using optimizing of colour quantisation. IET Image Process 8(12):761–770. https://doi.org/10.1049/iet-ipr.2013.0602

    Article  Google Scholar 

  7. Christoudias CM, Georgescu B, Meer P (2002) Synergism in low level vision, in proc. In: 16th Int Conf Pattern Recognit, pp, pp 150–155. https://doi.org/10.1109/ICPR.2002.1047421

  8. Comaniciu D, Meer P (2002) Mean shift: a robust approach toward feature space analysis. IEEE Trans Pattern Anal Mach Intell 24(5):603–619. https://doi.org/10.1109/34.1000236

    Article  Google Scholar 

  9. Comaniciu D, Ramesh V, Meer P (2001) The variable bandwidth mean shift and data-driven scale selection. In: Proc. 8th IEEE Int Conf Comput Vis, pp 438–455. https://doi.org/10.1109/ICCV.2001.937550

    Chapter  Google Scholar 

  10. Cour T, Benezit F, Shi J (2005) Spectral segmentation with multiscale graph decomposition. In: Proc IEEE Conf Comput Vis. Pattern Recognit (CVPR), pp 1124–1131. https://doi.org/10.1109/CVPR.2005.332

  11. Derefeldt G, Swartling T (1995) Color concept retrieval by free color naming. Displays 16(2):69–77. https://doi.org/10.1016/0141-9382(95)91176-3

    Article  Google Scholar 

  12. Felzenszwalb PF, Huttenlocher DP (2004) Efficient graph-based image segmentation. Int J Comput Vis 59(2):167–181. https://doi.org/10.1023/B:VISI.0000022288.19776.77

    Article  MATH  Google Scholar 

  13. Freixenet J, Muñoz X, Raba D, Martí J, Cufí X (2002) Yet another survey on image segmentation: region and boundary information integration. In: Proc 7th Eur Conf Comput Vis, pp 408–422. https://doi.org/10.1007/3-540-47977-5_27

  14. Fu X, Wang C-Y, Chen C, Wang C, Jay Kuo C-C (2015) Robust image segmentation using contour-guided color palettes. In: Proc IEEE Int Conf Comput Vis (ICCV), pp 1618–1625. https://doi.org/10.1109/ICCV.2015.189

    Chapter  Google Scholar 

  15. Gass T, Szekely G, Goksel O (2014) Simultaneous segmentation and multiresolution nonrigid atlas registration. IEEE Trans Image Process 23(7):2931–2943. https://doi.org/10.1109/TIP.2014.2322447

    Article  MathSciNet  MATH  Google Scholar 

  16. Ghamisi P, Couceiro MS, Benediktsson JA, Ferreira NMF (2012) An efficient method for segmentation of images based on fractional calculus and natural selection. Expert Syst Appl 39(16):12407–12417. https://doi.org/10.1016/j.eswa.2012.04.078

    Article  Google Scholar 

  17. Gould S, Fulton R, Koller D (2009) Decomposing a Scene into Geometric and Semantically Consistent Regions. In: Proc IEEE Int Conf Comput Vis (ICCV), pp 1–8. https://doi.org/10.1109/ICCV.2009.5459211

    Chapter  Google Scholar 

  18. He L, Chao Y, Suzuki K, Wu K (2009) Fast connected-component labeling. Pattern Recogn 42(9):1977–1987. https://doi.org/10.1016/j.patcog.2008.10.013

    Article  MATH  Google Scholar 

  19. Hill B, Roger T, Vorhagen F (1997) W (1997) comparative analysis of the quantization of color spaces on the basis of the CIELAB color-difference formula. ACM Trans Graph 16(2):109–154. https://doi.org/10.1145/248210.248212

    Article  Google Scholar 

  20. Jeong S-G, Lee C, Kim CS (2013) Motion-compensated frame interpolation based on multihypothesis motion estimation and texture optimization. IEEE Trans Image Process 22(11):4497–4509. https://doi.org/10.1109/TIP.2013.2274731

    Article  MathSciNet  MATH  Google Scholar 

  21. Kanezaki A (2018) Unsupervised image segmentation by backpropagation. In: Proc IEEE Int Conf Acoust Speech Signal Process (ICASSP), 1543–1547. https://doi.org/10.1109/ICASSP.2018.8462533

  22. Kang S-J, Bae S (2015) Fast segmentation-based backlight dimming. J Display Tech 11(5):399–402. https://doi.org/10.1109/JDT.2015.2416171

    Article  Google Scholar 

  23. Kim S, Nowozin S, Kohli P, Yoo CD (2011) Higher-order correlation clustering for image segmentation. In: Proc Neural Inform Process Syst, pp 1530–1538. https://doi.org/10.5555/2986459.2986630

    Chapter  Google Scholar 

  24. Kim TH, Lee KM, Lee SU (2013) Learning full pairwise affinities for spectral segmentation. IEEE Trans Pattern Anal March Intell 35(7):1690–1703. https://doi.org/10.1109/TPAMI.2012.237

    Article  Google Scholar 

  25. Kim W, Kanezaki A, Tanaka M (2020) Unsupervised learning of image segmentation based on differentiable feature clustering. IEEE Trans Image Process 29:8055–8068. https://doi.org/10.1109/TIP.2020.3011269

    Article  Google Scholar 

  26. Lee J, Lee D-K, Park R-H (2012) Robust exemplar-based inpainting algorithm using region segmentation. IEEE Trans Consum Electron 58(2):553–561. https://doi.org/10.1109/TIP.2014.2300823

    Article  Google Scholar 

  27. Li Z, Chen J (2015) Superpixel segmentation using linear spectral clustering. In: Proc IEEE Conf Comput Vis Pattern Recognit, pp 1356–1363. https://doi.org/10.1109/CVPR.2015.7298741

    Chapter  Google Scholar 

  28. Li Z, Wu X-M, Chang S-F (2012) Segmentation using superpixels: A bipartite graph partitioning approach. In: Proc IEEE Conf Comput Vis Pattern Recognit (CVPR), pp 789–796. https://doi.org/10.1109/CVPR.2012.6247750

    Chapter  Google Scholar 

  29. Malcolm J, Rathi Y, Tannenbaum (2007) A Graph cut segmentation with nonlinear shape priors. In: Proc IEEE Int Conf Image Processing, pp 365–368. https://doi.org/10.1109/ICIP.2007.4380030

    Chapter  Google Scholar 

  30. Martin D, Fowlkes C, Tal D, Malik J (2001) A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proc IEEE 8th Int Conf Comput Vis, pp 416–423. https://doi.org/10.1109/ICCV.2001.937655

    Chapter  Google Scholar 

  31. Meila M (2005) Comparing clustering: an axiomatic view. In: Proc Int Conf Mach Learning (ICML), pp 577–584. https://doi.org/10.1145/1102351.1102424

    Chapter  Google Scholar 

  32. Mezaris V, Kompatsiaris I, Strintzis M. G (2004) Still image segmentation tools for object-based multimedia applications. Int J Pattern Recognit Artif Intell 18 (4): 701–725. https://doi.org/10.1142/S0218001404003393

  33. Otsu N (1979) A threshold selection method from gray-level histogram. IEEE Trans Syst Man Cybern 9(1):62–66. https://doi.org/10.1109/TSMC.1979.4310076

    Article  Google Scholar 

  34. Reinhard E, Ward G, Pattanaik S, Debevec P, Heidrich W, Myszkowski K (2010) High dynamic range imaging: acquisition display and image-based lighting. Morgan Kaufmann, Waltham

    Google Scholar 

  35. Shi J, Malik J (2000) Normalized cuts and image segmentation. IEEE Trans Pattern Anal Mach Intell 22(8):888–905. https://doi.org/10.1109/34.868688

    Article  Google Scholar 

  36. Syu J-H, Wang S-J, Wang L-C (2017) Hierarchical image segmentation based on iterative contraction and merging. IEEE Trans Image Process 26(5):2246–2260. https://doi.org/10.1109/TIP.2017.2651395

    Article  MathSciNet  MATH  Google Scholar 

  37. Tan KS, Isa NAM (2011) Color image segmentation using histogram thresholding – fuzzy C-means hybrid approach. Pattern Recogn 44(1):1–15. https://doi.org/10.1016/j.patcog.2010.07.013

    Article  MATH  Google Scholar 

  38. Tan KS, Isa NAM, Lim WH (2013) Color image segmentation using adaptive unsupervised clustering approach. Appl Soft Comput. 13(4):2017–2036. https://doi.org/10.1016/j.asoc.2012.11.038

    Article  Google Scholar 

  39. Tan KS, Lim WH, Isa NAM (2013) Novel initialization scheme for fuzzy C-means algorithm on color image segmentation. Appl Soft Comput 13(4):1832–1853. https://doi.org/10.1016/j.asoc.2012.12.022

    Article  Google Scholar 

  40. Tao W, Jin H, Zhang Y (2007) Color image segmentation based on mean shift and normalized cuts. IEEE Trans Systems Man Cybern 37(5):1382–1389. https://doi.org/10.1109/TSMCB.2007.902249

    Article  Google Scholar 

  41. Unnikrishnan R, Pantofaru C, Hebert M (2007) Toward objective evaluation of image segmentation algorithms. IEEE Trans Pattern Anal Mach Intell 26(6):929–944. https://doi.org/10.1109/TPAMI.2007.1046

    Article  Google Scholar 

  42. Xu L, Yan Q, Xia Y, Jia J (2013) Structure extraction from texture via relative total variation. ACM Trans Graph 31(6):139:1–139:10. https://doi.org/10.1145/2366145.2366158

    Article  Google Scholar 

  43. Yang F, Lu H, Yang M-H (2014) Robust Superpixel tracking. IEEE Trans Image Process 23(4):1639–1651. https://doi.org/10.1109/TIP.2014.2300823

    Article  MathSciNet  MATH  Google Scholar 

  44. Zhang L, Song M, Liu Z, Liu X, Bu J, Chen C (2013) Probabilistic graphlet cut: exploiting spatial structure cue for weakly supervised image segmentation, in proc. IEEE Conf Comput Vis Pattern Recognit:1908–1915. https://doi.org/10.1109/CVPR.2013.249

  45. Zhao Q, Yang Z, Tao H, Liu W (2009) Evolving mean shift with adaptive bandwidth: a fast and noise robust approach, in proc. 9th Asian Conf Comput Vis:258–268. https://doi.org/10.1007/978-3-642-12307-8_24

Download references

Acknowledgement

This research was supported by the National Research Foundation of Korea (NRF) grant funded by the Korea government (MSIT) (No. 2020R1G1A1102163).

Author information

Authors and Affiliations

  1. Department of Electrical Engineering, Daegu University, Gyeonsan, 38453, Republic of Korea

    Ho Sub Lee

  2. Department of Multimedia Engineering, Dongguk University, Seoul, 04620, Republic of Korea

    Sung In Cho

Authors
  1. Ho Sub Lee
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Sung In Cho
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Sung In Cho.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Lee, H.S., In Cho, S. Spatial color histogram-based image segmentation using texture-aware region merging. Multimed Tools Appl 81, 24573–24600 (2022). https://doi.org/10.1007/s11042-022-11983-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-022-11983-4

Keywords

Search

Navigation