Skip to main content
SpringerLink
Log in

Image Segmentation by Relaxed Deep Extreme Cut with Connected Extreme Points

  • Conference paper
  • First Online:
Discrete Geometry and Mathematical Morphology (DGMM 2021)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12708))

Abstract

In this work, we propose a hybrid method for image segmentation based on the selection of four extreme points (leftmost, rightmost, top and bottom pixels at the object boundary), combining Deep Extreme Cut, a connectivity constraint for the extreme points, a marker-based color classifier from automatically estimated markers and a final relaxation procedure with the boundary polarity constraint, which is related to the extension of Random Walks to directed graphs as proposed by Singaraju et al. Its second constituent element presents theoretical contributions on how to optimally convert the 4 point boundary-based selection into connected region-based markers for image segmentation. The proposed method is able to correct imperfections from Deep Extreme Cut, leading to considerably improved results, in public datasets of natural images, with minimal user intervention (only four mouse clicks).

Thanks to CNPq (308985/2015-0, 313554/2018-8, 465446/2014-0), CAPES (88887.136422/2017-00) and FAPESP (2014/12236-1, 2014/50937-1, 2016/21591-5).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Circularity was measured by the isoperimetric quotient. That is, the ratio of the object area to the area of a circle with the same perimeter.

  2. 2.

    The squared EDT values are quantized into 256 levels (8 bits) prior to the weighted mean computation.

  3. 3.

    The sharpness measure is the complement of fuzziness as defined in [15].

  4. 4.

    The source code is available on the website: http://www.vision.ime.usp.br/~pmiranda/downloads.html.

  5. 5.

    The error rate is the percentage of misclassified pixels within the bounding boxes.

  6. 6.

    The datasets are available on the website: http://www.vision.ime.usp.br/~pmiranda/downloads.html.

  7. 7.

    https://cvlsegmentation.github.io/dextr/.

References

  1. Boykov, Y., Veksler, O., Zabih, R.: Fast approximate energy minimization via graph cuts. IEEE Trans. Pattern Anal. Mach. Intell. 23(11), 1222–1239 (2001)

    Article  Google Scholar 

  2. Boykov, Y., Jolly, M.P.: Interactive graph cuts for optimal boundary & region segmentation of objects in N-D images. Int. Conf. Comput. Vision (ICCV) 1, 105–112 (2001)

    Article  Google Scholar 

  3. Bragantini, J., Martins, S.B., Castelo-Fernandez, C., Falcão, A.X.: Graph-based image segmentation using dynamic trees. In: Vera-Rodriguez, R., Fierrez, J., Morales, A. (eds.) CIARP 2018. LNCS, vol. 11401, pp. 470–478. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-13469-3_55

    Chapter  Google Scholar 

  4. Cappabianco, F.A.M., Ribeiro, P.F.O., de Miranda, P.A.V., Udupa, J.K.: A general and balanced region-based metric for evaluating medical image segmentation algorithms. In: 2019 IEEE International Conference on Image Processing (ICIP), pp. 1525–1529, September 2019. https://doi.org/10.1109/ICIP.2019.8803083

  5. Ciesielski, K., Falcão, A., Miranda, P.: Path-value functions for which Dijkstra’s algorithm returns optimal mapping. J. Math. Imag. Vision 60(7), 1025–1036 (2018)

    Article  MathSciNet  Google Scholar 

  6. Demario, C.L., Miranda, P.A.V.: Relaxed oriented image foresting transform for seeded image segmentation. In: 2019 IEEE International Conference on Image Processing (ICIP), pp. 1520–1524 (2019)

    Google Scholar 

  7. Dice, L.: Measures of the amount of ecologic association between species. Ecology 26, 297–302 (1945)

    Article  Google Scholar 

  8. Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The PASCAL Visual Object Classes Challenge 2012 (VOC2012) Results. http://www.pascal-network.org/challenges/VOC/voc2012/workshop/index.html

  9. Falcão, A., Stolfi, J., Lotufo, R.: The image foresting transform: theory, algorithms, and applications. Trans. PAMI 26(1), 19–29 (2004)

    Article  Google Scholar 

  10. Grady, L.: Random walks for image segmentation. Trans. PAMI 28(11), 1768–1783 (2006)

    Article  Google Scholar 

  11. Gulshan, V., Rother, C., Criminisi, A., Blake, A., Zisserman, A.: Geodesic star convexity for interactive image segmentation. In: Proceedings of Computer Vision and Pattern Recognition, pp. 3129–3136 (2010)

    Google Scholar 

  12. Hariharan, B., Arbelaez, P., Bourdev, L., Maji, S., Malik, J.: Semantic contours from inverse detectors. In: International Conference on Computer Vision (ICCV) (2011)

    Google Scholar 

  13. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778, June 2016. https://doi.org/10.1109/CVPR.2016.90

  14. Lempitsky, V., Kohli, P., Rother, C., Sharp, T.: Image segmentation with a bounding box prior. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 277–284 (2009)

    Google Scholar 

  15. Malmberg, F., Nyström, I., Mehnert, A., Engstrom, C., Bengtsson, E.: Relaxed image foresting transforms for interactive volume image segmentation. In: Proceedings of SPIE, vol. 7623, pp. 7623–7623-11 (2010)

    Google Scholar 

  16. Maninis, K.K., Caelles, S., Pont-Tuset, J., Gool, L.V.: Deep extreme cut: from extreme points to object segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 616–625 (2018)

    Google Scholar 

  17. Mansilla, L.A.C., Miranda, P.A.V.: Oriented image foresting transform segmentation: Connectivity constraints with adjustable width. In: 2016 29th SIBGRAPI Conference on Graphics, Patterns and Images (SIBGRAPI), pp. 289–296, October 2016. https://doi.org/10.1109/SIBGRAPI.2016.047

  18. Mansilla, L.A.C., Miranda, P.A.V., Cappabianco, F.A.M.: Oriented image foresting transform segmentation with connectivity constraints. In: 2016 IEEE International Conference on Image Processing (ICIP), pp. 2554–2558, September 2016

    Google Scholar 

  19. Mansilla, L., Miranda, P.: Image segmentation by oriented image foresting transform: handling ties and colored images. In: 18th International Conference on Digital Signal Processing, pp. 1–6. Santorini, GR, July 2013

    Google Scholar 

  20. Miranda, P., Falcão, A., Spina, T.: Riverbed: a novel user-steered image segmentation method based on optimum boundary tracking. IEEE Trans. Image Process. 21(6), 3042–3052 (2012)

    Article  MathSciNet  Google Scholar 

  21. Miranda, P., Falcão, A., Udupa, J.: Synergistic arc-weight estimation for interactive image segmentation using graphs. Comput. Vision Image Underst. 114(1), 85–99 (2010)

    Article  Google Scholar 

  22. Miranda, P., Mansilla, L.: Oriented image foresting transform segmentation by seed competition. IEEE Trans. Image Process. 23(1), 389–398 (2014)

    Article  MathSciNet  Google Scholar 

  23. Noh, H., Hong, S., Han, B.: Learning deconvolution network for semantic segmentation. In: 2015 IEEE International Conference on Computer Vision (ICCV), pp. 1520–1528 (2015)

    Google Scholar 

  24. Papadopoulos, D.P., Uijlings, J.R.R., Keller, F., Ferrari, V.: Extreme clicking for efficient object annotation. In: 2017 IEEE International Conference on Computer Vision (ICCV), pp. 4940–4949 (2017)

    Google Scholar 

  25. Rother, C., Kolmogorov, V., Blake, A.: “grabcut’’: interactive foreground extraction using iterated graph cuts. ACM Trans. Gr. 23(3), 309–314 (2004)

    Article  Google Scholar 

  26. Singaraju, D., Grady, L., Vidal, R.: Interactive image segmentation via minimization of quadratic energies on directed graphs. In: International Conference on Computer Vision and Pattern Recognition, pp. 1–8, June 2008

    Google Scholar 

  27. Tang, M., Ayed, I.B., Marin, D., Boykov, Y.: Secrets of grabcut and kernel k-means. In: 2015 IEEE International Conference on Computer Vision (ICCV), pp. 1555–1563 (2015)

    Google Scholar 

  28. Tang, M., Gorelick, L., Veksler, O., Boykov, Y.: Grabcut in one cut. In: 2013 IEEE International Conference on Computer Vision, pp. 1769–1776 (2013)

    Google Scholar 

  29. Wu, J., Zhao, Y., Zhu, J., Luo, S., Tu, Z.: Milcut: a sweeping line multiple instance learning paradigm for interactive image segmentation. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp. 256–263 (2014)

    Google Scholar 

  30. Xu, N., Price, B., Cohen, S., Yang, J., Huang, T.: Deep grabcut for object selection. In: Kim, T.-K., Stefanos Zafeiriou, G.B., Mikolajczyk, K. (eds.) Proceedings of the British Machine Vision Conference (BMVC), pp. 182.1-182.12. BMVA Press, September 2017

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Mathematics and Statistics, University of São Paulo, CEP 05508–090, São Paulo, SP, Brazil

    Débora E. C. Oliveira, Caio L. Demario & Paulo A. V. Miranda

Authors
  1. Débora E. C. Oliveira
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Caio L. Demario
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Paulo A. V. Miranda
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Paulo A. V. Miranda .

Editor information

Editors and Affiliations

  1. Uppsala University, Uppsala, Sweden

    Joakim Lindblad

  2. Uppsala University, Uppsala, Sweden

    Filip Malmberg

  3. Uppsala University, Uppsala, Sweden

    Nataša Sladoje

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Oliveira, D.E.C., Demario, C.L., Miranda, P.A.V. (2021). Image Segmentation by Relaxed Deep Extreme Cut with Connected Extreme Points. In: Lindblad, J., Malmberg, F., Sladoje, N. (eds) Discrete Geometry and Mathematical Morphology. DGMM 2021. Lecture Notes in Computer Science(), vol 12708. Springer, Cham. https://doi.org/10.1007/978-3-030-76657-3_32

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-76657-3_32

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-76656-6

  • Online ISBN: 978-3-030-76657-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation