Skip to main content
SpringerLink
Log in

Spectral Image Segmentation Using Image Decomposition and Inner Product-Based Metric

  • Published:
Journal of Mathematical Imaging and Vision Aims and scope Submit manuscript

Abstract

Image segmentation is an indispensable tool in computer vision applications, such as recognition, detection and tracking. In this work, we introduce a novel user-assisted image segmentation technique which combines image decomposition, inner product-based similarity metric, and spectral graph theory into a concise and unified framework. First, we perform an image decomposition to split the image into texture and cartoon components. Then, an affinity graph is generated and the weights are assigned to its edges according to a gradient-based inner-product function. From the eigenstructure of the affinity graph, the image is partitioned through the spectral cut of the underlying graph. The computational effort of our framework is alleviated by an image coarsening process, which reduces the graph size considerably. Moreover, the image partitioning can be improved by interactively changing the graph weights by sketching. Finally, a coarse-to-fine interpolation is applied in order to assemble the partition back onto the original image. The efficiency of the proposed methodology is attested by comparisons with state-of-art spectral segmentation methods through a qualitative and quantitative analysis of the results.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13

Similar content being viewed by others

References

  1. Andrews, S., Hamarneh, G., Saad, A.: Fast random walker with priors using precomputation for interactive medical image segmentation. In: Lec. Notes in Comput. Sci., pp. 1–8 (2010)

    Google Scholar 

  2. Arbelaez, P., Maire, M., Fowlkes, C., Malik, J.: Contour detection and hierarchical image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 33, 898–916 (2011)

    Article  Google Scholar 

  3. Bergo, F.P., Falcão, A.X., Miranda, P.A., Rocha, L.M.: Automatic image segmentation by tree pruning. J. Math. Imaging Vis. 29, 141–162 (2007)

    Article  Google Scholar 

  4. Biyikoglu, T., Leydold, J., Stadler, P.F.: Laplacian Eigenvectors of Graphs: Perron-Frobenius and Faber-Krahn Type Theorems. Springer, Berlin (2007)

    Book  MATH  Google Scholar 

  5. Bollobás, B.: Modern Graph Theory. Springer, Berlin (1998)

    Book  MATH  Google Scholar 

  6. Boykov, Y., Funka-Lea, G.: Graph cuts and efficient n−d image segmentation. Int. J. Comput. Vis. 70, 109–131 (2006)

    Article  Google Scholar 

  7. Cai, W., Chung, A.C.S.: Shape-based image segmentation using normalized cuts. In: Proc. of ICIP’06, pp. 1101–1104 (2006)

    Google Scholar 

  8. Carvalho, M., Costa, A., Ferreira, A., Junior, R.C.: Image segmentation using component tree and normalized cut. In: Proc. of Sibgrapi 2010 (23th Conference on Graphics, Patterns and Images), pp. 317–322 (2010)

    Chapter  Google Scholar 

  9. Carvalho, M., Ferreira, A., Costa, A.: Image segmentation using quadtree-based similarity graph and normalized cut. In: Lect. Notes Comput. Sci., vol. 6419, pp. 329–337 (2010)

    Google Scholar 

  10. Casaca, W., Boaventura, M.: A decomposition and noise removal method combining diffusion equation and wave atoms for textured images. Math. Probl. Eng. 2010, 1–21 (2010)

    Article  MathSciNet  Google Scholar 

  11. Casaca, W., Paiva, A., Nonato, L.G.: Spectral segmentation using cartoon-texture decomposition and inner product-based metric. In: Proc. of Sibgrapi 2011 (24th Conference on Graphics, Patterns and Images), pp. 266–273 (2011)

    Chapter  Google Scholar 

  12. Chung, F.R.K.: Spectral Graph Theory. AMS, Providence (1997)

    MATH  Google Scholar 

  13. Cour, T., Benezit, F., Shi, J.: Spectral segmentation with multiscale graph decomposition 2005. In: Proc. of CVPR, pp. 1124–1131 (2005)

    Google Scholar 

  14. Demanet, L., Ying, L.: Curvelets and wave atoms for mirror-extended images. In: Wavelets XII. Proc. of SPIE, vol. 6701, p. 67010J (2007)

    Chapter  Google Scholar 

  15. Demanet, L., Ying, L.: Wave atoms and sparsity of oscillatory patterns. Appl. Comput. Harmon. Anal. 23, 368–387 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  16. Diaz, J., Petit, J., Serna, M.: A survey of graph layout problems. ACM Comput. Surv. 34, 313–356 (2002)

    Article  Google Scholar 

  17. Estrada, F.J., Jepson, A.D.: Benchmarking image segmentation algorithms. Int. J. Comput. Vis. 85(2), 167–181 (2009)

    Article  Google Scholar 

  18. Falcão, A., Stolfi, J., Lotufo, R.: The image foresting transform: theory, algorithms, and applications. IEEE Trans. Pattern Anal. Mach. Intell. 26, 19–29 (2004)

    Article  Google Scholar 

  19. Fiedler, M.: Algebraic connectivity of graphs. Czechoslov. Math. J. 23, 298–305 (1973)

    MathSciNet  Google Scholar 

  20. Grady, L.: Multilabel random walker image segmentation using prior models. In: Proc. of IEEE CVPR 2005, pp. 763–770 (2005)

    Google Scholar 

  21. Grady, L., Schwartz, E.L.: Isoperimetric graph partitioning for image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 28, 469–475 (2006)

    Article  Google Scholar 

  22. Grady, L., Sinop, A.K.: Fast approximate random walker segmentation using eigenvector precomputation. In: Proc. of IEEE CVPR 2008 (2008)

    Google Scholar 

  23. Jameson, C., Jujuunashvili, A., Knyazev, A.: Modern Eigenvalue Solvers for Spectral Image Segmentation (2008). http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.122.3624

  24. Koutis, I., Miller, G.L., Tolliver, D.: Combinatorial preconditioners and multilevel solvers for problems in computer vision and image processing. In: Int’l Symp. on Visual Computing, pp. 1067–1078 (2009)

    Chapter  Google Scholar 

  25. Ma, X., Wan, W., Yao, J.: Texture image segmentation on improved watershed and multiway spectral clustering. In: Proc. of ICALIP’08, pp. 1693–1697 (2008)

    Google Scholar 

  26. Maji, S., Vishnhoi, N., Malik, J.: Biased normalized cuts. In: Proc. of CVPR 2011, pp. 2057–2064 (2011)

    Chapter  Google Scholar 

  27. Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proc. 8th Int’l Conf. Computer Vision, vol. 2, pp. 416–423 (2001)

    Google Scholar 

  28. Meyer, Y.: Oscillating Patterns in Image Processing and Nonlinear Evolution Equations. AMS, Providence (2001)

    MATH  Google Scholar 

  29. Mohar, B., Juvan, M.: Some applications of Laplace eigenvalues of graphs. In: Graph Symetric: Algebraic Methods and Applications, vol. 497, pp. 227–275 (1997)

    Google Scholar 

  30. Perona, P., Malik, J.: Scale-space and edge detection using anisotropic diffusion. IEEE Trans. Pattern Anal. Mach. Intell. 12, 629–639 (1990)

    Article  Google Scholar 

  31. Pratt, W.: Digital Image Processing. Wiley, New York (2008)

    Google Scholar 

  32. Sarkar, S., Soundararajan, P.: Supervised learning of large perceptual organization: graph spectral partitioning and learning automata. IEEE Trans. Pattern Anal. Mach. Intell. 22, 504–525 (2000)

    Article  Google Scholar 

  33. Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 22, 888–905 (2000)

    Article  Google Scholar 

  34. Shuai, Y., Masahide, A., Akira, T., Masayuki, K.: High accuracy bicubic interpolation using image local features. IEICE Trans. Fundam. Electron. Commun. Comput. Sci. E90-A, 1611–1615 (2007)

    Article  Google Scholar 

  35. Soundararajan, P., Sarkar, S.: Analysis of MinCut, average cut and normalized cut measures. In: Workshop on Perceptual Organization in Computer Vision (2001)

    Google Scholar 

  36. Spielman, D.A.: Spectral graph theory and its applications. In: Proc. of the 48th Annual IEEE Symposium on Foundations of Computer Science, pp. 29–38 (2007)

    Google Scholar 

  37. Sun, F., He, J.P.: A normalized cuts based image segmentation method. In: Proc. of ICIC’09, pp. 333–336 (2009)

    Google Scholar 

  38. Tao, W., Jin, H., Zhang, Y.: Color image segmentation based on mean shift and normalized cuts. IEEE Trans. Syst. Man Cybern. 37, 1382–1389 (2007)

    Google Scholar 

  39. Tolliver, D., Miller, G.L.: Graph partitioning by spectral rounding: applications in image segmentation and clustering. In: Proc. of CVPR 2006, vol. 1, pp. 1053–1060 (2006)

    Google Scholar 

  40. Vese, L., Osher, S.: Modeling textures with total variation minimization and oscillating patters in image processing. J. Sci. Comput. 19, 553–572 (2003)

    Article  MathSciNet  MATH  Google Scholar 

  41. Vese, L., Osher, S.: Color texture modeling and color image decomposition in a variational-PDE approach. In: Proc. of the 8th Int’l Symp. on Symbolic and Numeric Algorithms for Scientific Computing, pp. 103–110 (2006)

    Google Scholar 

Download references

Acknowledgements

We would like to thank the anonymous reviewers for their useful comments to improve the quality of this work and Shawn Andrews for kindly providing us with the implementation of RWS-EP and RWS-EPP. This research has been funded by FAPESP-Brazil, INCT-MACC and CNPq-Brazil.

Author information

Authors and Affiliations

  1. Instituto de Ciências Matemáticas e de Computação – ICMC, Universidade de São Paulo – USP, São Carlos, São Paulo, Brazil

    Wallace Casaca, Afonso Paiva, Erick Gomez-Nieto, Paulo Joia & Luis Gustavo Nonato

Authors
  1. Wallace Casaca
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Afonso Paiva
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Erick Gomez-Nieto
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Paulo Joia
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Luis Gustavo Nonato
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Afonso Paiva.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Casaca, W., Paiva, A., Gomez-Nieto, E. et al. Spectral Image Segmentation Using Image Decomposition and Inner Product-Based Metric. J Math Imaging Vis 45, 227–238 (2013). https://doi.org/10.1007/s10851-012-0359-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10851-012-0359-6

Keywords

Search

Navigation