Skip to main content
SpringerLink
Log in

GridIIS: Grid Based Interactive Image Segmentation

  • Conference paper
  • First Online:
Pattern Recognition and Computer Vision (PRCV 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14435))

Included in the following conference series:

  • 354 Accesses

Abstract

Interactive segmentation enables users to specify the object of interest (OOI) via various interaction strategies to obtain accurate segmentation results. An ideal interactive method should efficiently and accurately express users’ segmentation intentions. However, the existing methods can only use a single interactive mode, ignoring the differences in scale and shape between OOIs, resulting in an inflexibility labeling process. In this paper, we propose a grid-based interactive image segmentation method (GridIIS). Specifically, GridIIS overlays grids on the image, and users can specify the location and shape of the OOI by selecting the grid areas as the interactive guidance. Users can choose the appropriate grid selection method and size considering the OOI’s scale, shape, and boundary clarity to obtain guidance. We accordingly propose a novel grid sampling strategy, that considers the OOI’s scale and shapes to adaptively estimate the grid size and area. Experiments on several datasets from different domains (street views, medical images, scene texts, etc.) show that our method achieves superior performance with fewer interaction rounds and exhibits strong generalization ability in cross-domain datasets.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 109.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 139.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Bai, J., Wu, X.: Error-tolerant scribbles based interactive image segmentation. In: CVPR, pp. 392–399 (2014)

    Google Scholar 

  2. Bai, X., Sapiro, G.: A geodesic framework for fast interactive image and video segmentation and matting. In: ICCV, pp. 1–8. IEEE (2007)

    Google Scholar 

  3. Boykov, Y.Y., Jolly, M.P.: Interactive graph cuts for optimal boundary & region segmentation of objects in ND images. In: ICCV, vol. 1, pp. 105–112. IEEE (2001)

    Google Scholar 

  4. Caesar, H., Uijlings, J., Ferrari, V.: Coco-stuff: thing and stuff classes in context. In: CVPR, pp. 1209–1218 (2018)

    Google Scholar 

  5. Chen, X., et al.: Conditional diffusion for interactive segmentation. In: ICCV, pp. 7345–7354 (2021)

    Google Scholar 

  6. Chen, X., et al.: FocalClick: towards practical interactive image segmentation. In: CVPR, pp. 1300–1309 (2022)

    Google Scholar 

  7. Cheng, M.-M., et al.: Intelligent visual media processing: when graphics meets vision. JCST 32(1), 110–121 (2017)

    Google Scholar 

  8. Cordts, M., et al.: The cityscapes dataset for semantic urban scene understanding (2016). arXiv: 1604.01685 [cs.CV]

  9. Ding, Z., et al.: A dual-stream framework guided by adaptive gaussian maps for interactive image segmentation. Knowl.-Based Syst. 223, 107033 (2021)

    Article  Google Scholar 

  10. Everingham, M., et al.: The pascal visual object classes (VOC) challenge. IJCV 88(2), 303–338 (2010)

    Article  Google Scholar 

  11. Gerhard, S., et al.: Segmented anisotropic ssTEM dataset of neural tissue. figshare (2013)

    Google Scholar 

  12. Grady, L.: Random walks for image segmentation. PAMI 28(11), 1768–1783 (2006)

    Article  Google Scholar 

  13. Jang, W.D., Kim, C.S.: Interactive image segmentation via backpropagating refinement scheme. In: CVPR, pp. 5297–5306 (2019)

    Google Scholar 

  14. Kim, T.H., Lee, K.M., Lee, S.U.: Generative image segmentation using random walks with restart. In: Forsyth, D., Torr, P., Zisserman, A. (eds.) ECCV 2008. LNCS, vol. 5304, pp. 264–275. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-88690-7_20

    Chapter  Google Scholar 

  15. Koohbanani, N.A., et al.: NuClick: a deep learning framework for interactive segmentation of microscopic images. MIA 65, 101771 (2020)

    Google Scholar 

  16. Li, Y., et al.: Lazy snapping. ACM Trans. Graph. (ToG) 23(3), 303–308 (2004)

    Article  Google Scholar 

  17. Liew, J.H., et al.: Regional interactive image segmentation networks. In: ICCV. IEEE Computer Society, pp. 2746–2754 (2017)

    Google Scholar 

  18. Lin, D., et al.: Scribblesup: scribble-supervised convolutional networks for semantic segmentation. In: CVPR, pp. 3159–3167 (2016)

    Google Scholar 

  19. Lin, T.Y., et al.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10602-1_48

    Chapter  Google Scholar 

  20. Lin, Z., et al.: Interactive image segmentation with first click attention. In: CVPR, pp. 13339–13348 (2020)

    Google Scholar 

  21. Ling, H., et al.: Fast interactive object annotation with Curve-GCN. In: CVPR, pp. 5257–5266 (2019)

    Google Scholar 

  22. Luo, X., et al.: MIDeepSeg: minimally interactive segmentation of unseen objects from medical images using deep learning. MIA 72, 102102 (2021)

    Google Scholar 

  23. Mahadevan, S., Voigtlaender, P., Leibe, B.: Iteratively trained interactive segmentation. arXiv preprint: arXiv:1805.04398 (2018)

  24. Majumder, S., Yao, A.: Content-aware multi-level guidance for interactive instance segmentation. In: CVPR, pp. 11602–11611 (2019)

    Google Scholar 

  25. Maninis, K.-K. et al.: Deep extreme cut: from extreme points to object segmentation. In: CVPR, pp. 616–625 (2018)

    Google Scholar 

  26. McGuinness, K., O’connor, N.E.: A comparative evaluation of interactive segmentation algorithms. Pattern Recogn. 43, 434–444 (2010)

    Article  Google Scholar 

  27. Rother, C., Kolmogorov, V., Blake, A.: “GrabCut’’ interactive foreground extraction using iterated graph cuts. ACM Trans. Graph. (TOG) 23(3), 309–314 (2004)

    Article  Google Scholar 

  28. Sofiiuk, K., Petrov, I.A., Konushin, A.: Reviving iterative training with mask guidance for interactive segmentation. arXiv preprint: arXiv:2102.06583 (2021)

  29. Sofiiuk, K., et al.: F-BRS: rethinking backpropagating refinement for interactive segmentation. In: CVPR, pp. 8623–8632 (2020)

    Google Scholar 

  30. Wang, G., et al.: DeepIGeoS: a deep interactive geodesic framework for medical image segmentation. PAMI 41(7), 1559–1572 (2018)

    Article  Google Scholar 

  31. Wu, J., et al.: Milcut: a sweeping line multiple instance learning paradigm for interactive image segmentation. In: CVPR, pp. 256–263 (2014)

    Google Scholar 

  32. Xu, N., et al.: Deep GrabCut for object selection. arXiv preprint: arXiv:1707.00243 (2017)

  33. Xu, N., et al.: Deep interactive object selection. In: CVPR, pp. 373–381 (2016)

    Google Scholar 

  34. Yuliang, L., et al.: Detecting curve text in the wild: new dataset and new solution. arXiv preprint: arXiv:1712.02170 (2017)

  35. Zhang, C., et al.: Intention-aware feature propagation network for interactive segmentation. arXiv preprint: arXiv:2203.05145 (2022)

  36. Zhang, K., Zhuang, X.: CycleMix: a holistic strategy for medical image segmentation from scribble supervision. In: CVPR, pp. 11656–11665 (2022)

    Google Scholar 

  37. Zhang, S., et al.: Interactive object segmentation with inside-outside guidance. In: CVPR, pp. 12234–12244 (2020)

    Google Scholar 

  38. Zhuang, X.: Multivariate mixture model for cardiac segmentation from multi-sequence MRI. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 581–588. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46723-8_67

    Chapter  Google Scholar 

Download references

Acknowledgements

This work is supported by National Natural Science Foundation of China (No. 61773325), Industry-University Cooperation Project of Fujian Science and Technology Department (No. 2021H6035), Fujian Key Technological Innovation and Industrialization Projects (No. 2023XQ023), and Fu-Xia-Quan National Independent Innovation Demonstration Project (No. 2022FX4).

Author information

Authors and Affiliations

  1. School of Computer and Information Engineering, Xiaman University of Technology, Xiamen, China

    Pengqi Zhu, Da-Han Wang & Shunzhi Zhu

  2. Fujian Key Laboratory of Pattern Recognition and Image Understanding, Xiamen, China

    Pengqi Zhu, Da-Han Wang & Shunzhi Zhu

Authors
  1. Pengqi Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Da-Han Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shunzhi Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Da-Han Wang .

Editor information

Editors and Affiliations

  1. Nanjing University of Information Science and Technology, Nanjing, China

    Qingshan Liu

  2. Xiamen University, Xiamen, China

    Hanzi Wang

  3. Beijing University of Posts and Telecommunications, Beijing, China

    Zhanyu Ma

  4. Sun Yat-sen University, Guangzhou, China

    Weishi Zheng

  5. Peking University, Beijing, China

    Hongbin Zha

  6. Chinese Academy of Sciences, Beijing, China

    Xilin Chen

  7. Chinese Academy of Sciences, Beijing, China

    Liang Wang

  8. Xiamen University, Xiamen, China

    Rongrong Ji

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zhu, P., Wang, DH., Zhu, S. (2024). GridIIS: Grid Based Interactive Image Segmentation. In: Liu, Q., et al. Pattern Recognition and Computer Vision. PRCV 2023. Lecture Notes in Computer Science, vol 14435. Springer, Singapore. https://doi.org/10.1007/978-981-99-8552-4_28

Download citation

  • DOI: https://doi.org/10.1007/978-981-99-8552-4_28

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-99-8551-7

  • Online ISBN: 978-981-99-8552-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation