Skip to main content
SpringerLink
Log in
Book cover

International Conference on Medical Image Computing and Computer-Assisted Intervention

MICCAI 2022: Medical Image Computing and Computer Assisted Intervention – MICCAI 2022 pp 308–318Cite as

End-to-End Segmentation of Medical Images via Patch-Wise Polygons Prediction

  • Conference paper
  • First Online:

  • 7529 Accesses

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13435))

Abstract

The leading medical image segmentation methods represent the output map as a pixel grid. We present an alternative in which the object edges are modeled, per image patch, as a polygon with k vertices that is coupled with per-patch label probabilities. The vertices are optimized by employing a differentiable neural renderer to create a raster image. The delineated region is then compared with the ground truth segmentation. Our method obtains multiple state-of-the-art results for the Gland segmentation dataset (Glas), the Nucleus challenges (MoNuSeg), and multiple polyp segmentation datasets, as well as for non-medical benchmarks, including Cityscapes, CUB, and Vaihingen. Our code for training and reproducing these results is attached as a supplement.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Acuna, D., Ling, H., Kar, A., Fidler, S.: Efficient interactive annotation of segmentation datasets with polygon-rnn++. In: CVPR (2018)

    Google Scholar 

  2. Badrinarayanan, V., Kendall, A., Cipolla, R.: SegNet: a deep convolutional encoder-decoder architecture for image segmentation. Trans. Pattern Anal. Mach. Intell. 39(12) (2017)

    Google Scholar 

  3. Bernal, J., Sánchez, F.J., Fernández-Esparrach, G., et al.: WM-DOVA maps for accurate polyp highlighting in colonoscopy: validation vs. saliency maps from physicians. Comput. Med. Imag. Graph. 43 (2015)

    Google Scholar 

  4. Chao, P., Kao, C.Y., Ruan, Y.S., Huang, C.H., Lin, Y.L.: HardNet: a low memory traffic network. In: ICCV (2019)

    Google Scholar 

  5. Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: Semantic image segmentation with deep convolutional nets and fully connected CRFs. arXiv (2014)

    Google Scholar 

  6. Chen, L.C., Papandreou, G., Kokkinos, I., Murphy, K., Yuille, A.L.: Deeplab: Semantic image segmentation with deep convolutional nets, atrous convolution, and fully connected CRFs. Trans. Pattern Anal. Mach. Intell. 40(4) (2017)

    Google Scholar 

  7. Cheng, D., Liao, R., Fidler, S., Urtasun, R.: DarNet: deep active ray network for building segmentation. In: CVPR (2019)

    Google Scholar 

  8. Cohen, L.D.: On active contour models and balloons. CVGIP Image Underst. 53(2) (1991)

    Google Scholar 

  9. Cordts, M., et al.: The cityscapes dataset for semantic urban scene understanding. In: CVPR (2016)

    Google Scholar 

  10. Dong, B., Wang, W., Fan, D.P., Li, J., Fu, H., Shao, L.: Polyp-PVT: polyp segmentation with pyramid vision transformers. arXiv (2021)

    Google Scholar 

  11. Fan, D.P., et al.: PraNet: parallel reverse attention network for polyp segmentation. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12266, pp. 263–273. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59725-2_26

    Chapter  Google Scholar 

  12. Fang, Y., Chen, C., et al.: Selective feature aggregation network with area-boundary constraints for polyp segmentation. In: MICCAI (2019)

    Google Scholar 

  13. Gur, S., Shaharabany, T., Wolf, L.: End to end trainable active contours via differentiable rendering. In: ICLR (2019)

    Google Scholar 

  14. Hatamizadeh, A., Sengupta, D., Terzopoulos, D.: End-to-end trainable deep active contour models for automated image segmentation: delineating buildings in aerial imagery. In: ECCV (2020)

    Google Scholar 

  15. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: CVPR (2016)

    Google Scholar 

  16. Huang, C.H., Wu, H.Y., Lin, Y.L.: HarDNet-MSEG: a simple encoder-decoder polyp segmentation neural network that achieves over 0.9 mean dice and 86 fps. arXiv (2021)

    Google Scholar 

  17. Jha, D., et al.: Kvasir-SEG: a segmented polyp dataset. In: Ro, Y.M., et al. (eds.) MMM 2020. LNCS, vol. 11962, pp. 451–462. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-37734-2_37

    Chapter  Google Scholar 

  18. Kass, M., Witkin, A., Terzopoulos, D.: Snakes: active contour models. IJCV (1988)

    Google Scholar 

  19. Kato, H., Ushiku, Y., Harada, T.: Neural 3d mesh renderer. In: CVPR (2018)

    Google Scholar 

  20. Kumar, N., Verma, R., Anand, D., Zhou, Y., Onder, O.F., et al.: A multi-organ nucleus segmentation challenge. IEEE Trans. Med. Imaging 39(5), 1380–1391 (2019)

    Google Scholar 

  21. Ling, H., Gao, J., Kar, A., Chen, W., Fidler, S.: Fast interactive object annotation with curve-GCN. In: CVPR (2019)

    Google Scholar 

  22. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: CVPR (2015)

    Google Scholar 

  23. Loper, M.M., Black, M.J.: OpenDR: an approximate differentiable renderer. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8695, pp. 154–169. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10584-0_11

    Chapter  Google Scholar 

  24. Marcos, D., et al.: Learning deep structured active contours end-to-end. In: CVPR (2018)

    Google Scholar 

  25. Patel, K., Bur, A.M., Wang, G.: Enhanced u-net: a feature enhancement network for polyp segmentation. In: CRV. IEEE (2021)

    Google Scholar 

  26. Peng, C., Zhang, X., Yu, G., Luo, G., Sun, J.: Large kernel matters-improve semantic segmentation by global convolutional network. In: CVPR (2017)

    Google Scholar 

  27. Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: MICCAI (2015)

    Google Scholar 

  28. Rottensteiner, F., et al.: International society for photogrammetry and remote sensing, 2D semantic labeling contest (2022)

    Google Scholar 

  29. Rupprecht, C., Huaroc, E., et al.: Deep active contours. arXiv (2016)

    Google Scholar 

  30. Silva, J., Histace, A., Romain, O., et al.: Toward embedded detection of polyps in WCE images for early diagnosis of colorectal cancer. Int. J. CARS 9, 283–293 (2014)

    Google Scholar 

  31. Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv (2014)

    Google Scholar 

  32. Sirinukunwattana, K., et al.: Gland segmentation in colon histology images: the GLAS challenge contest. Med. Image Aanal. 35 (2017)

    Google Scholar 

  33. Sudre, C.H., Li, W., et al.: Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations. In: DLMIA (2017)

    Google Scholar 

  34. Sun, X., Zhang, P., Wang, D., Cao, Y., Liu, B.: Colorectal polyp segmentation by u-net with dilation convolution. In: ICMLA, pp. 851–858. IEEE (2019)

    Google Scholar 

  35. Szegedy, C., et al.: Going deeper with convolutions. In: CVPR (2015)

    Google Scholar 

  36. Tajbakhsh, N., Gurudu, S.R., Liang, J.: Automated polyp detection in colonoscopy videos using shape and context information. IEEE Trans. Med. Image 35(2) (2015)

    Google Scholar 

  37. Valanarasu, J.M.J., Oza, P., Hacihaliloglu, I., Patel, V.M.: Medical transformer: gated axial-attention for medical image segmentation. arXiv (2021)

    Google Scholar 

  38. Wah, C., Branson, S., Welinder, P., Perona, P., Belongie, S.: The Caltech-UCSD birds-200-2011 dataset (2011)

    Google Scholar 

  39. Wang, D., et al.: AFP-mask: anchor-free polyp instance segmentation in colonoscopy. IEEE J. Biomed. Health Inform. 26(7):2995–3006 (2022)

    Google Scholar 

  40. Wang, H., Cao, P., Wang, J., Zaiane, O.R.: Uctransnet: rethinking the skip connections in u-net from a channel-wise perspective with transformer. arXiv (2021)

    Google Scholar 

  41. Wang, H., et al.: Axial-DeepLab: stand-alone axial-attention for panoptic segmentation. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12349, pp. 108–126. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58548-8_7

    Chapter  Google Scholar 

  42. Wei, J., Hu, Y., Zhang, R., Li, Z., Zhou, S.K., Cui, S.: Shallow attention network for polyp segmentation. In: de Bruijne, M., et al. (eds.) MICCAI 2021. LNCS, vol. 12901, pp. 699–708. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87193-2_66

    Chapter  Google Scholar 

  43. Xiao, X., Lian, S., Luo, Z., Li, S.: Weighted res-UNet for high-quality retina vessel segmentation. In: ITME. IEEE (2018)

    Google Scholar 

  44. Yin, Z., Liang, K., Ma, Z., Guo, J.: Duplex contextual relation network for polyp segmentation. arXiv (2021)

    Google Scholar 

  45. Yu, F., Koltun, V.: Multi-scale context aggregation by dilated convolutions. arXiv (2015)

    Google Scholar 

  46. Zhang, H., et al.: Context encoding for semantic segmentation. In: CVPR (2018)

    Google Scholar 

  47. Zhang, R., Li, G., Li, Z., Cui, S., Qian, D., Yu, Y.: Adaptive context selection for polyp segmentation. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12266, pp. 253–262. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59725-2_25

    Chapter  Google Scholar 

  48. Zhou, Z., Rahman Siddiquee, M.M., Tajbakhsh, N., Liang, J.: UNet++: a nested U-Net architecture for medical image segmentation. In: Stoyanov, D., et al. (eds.) DLMIA/ML-CDS -2018. LNCS, vol. 11045, pp. 3–11. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00889-5_1

    Chapter  Google Scholar 

Download references

Acknowledgments

This project has received funding from the European Research Council (ERC) under the European Union’s Horizon 2020 research and innovation programme (grant ERC CoG 725974).

Author information

Authors and Affiliations

  1. Tel-Aviv University, Tel Aviv-Yafo, Israel

    Tal Shaharabany & Lior Wolf

Authors
  1. Tal Shaharabany
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lior Wolf
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tal Shaharabany .

Editor information

Editors and Affiliations

  1. Rochester Institute of Technology, Rochester, NY, USA

    Linwei Wang

  2. Chinese University of Hong Kong, Hong Kong, Hong Kong

    Qi Dou

  3. University of Virginia, Charlottesville, VA, USA

    P. Thomas Fletcher

  4. National Center for Tumor Diseases (NCT/UCC), Dresden, Germany

    Stefanie Speidel

  5. Case Western Reserve University, Cleveland, OH, USA

    Shuo Li

1 Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (pdf 1460 KB)

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Shaharabany, T., Wolf, L. (2022). End-to-End Segmentation of Medical Images via Patch-Wise Polygons Prediction. In: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (eds) Medical Image Computing and Computer Assisted Intervention – MICCAI 2022. MICCAI 2022. Lecture Notes in Computer Science, vol 13435. Springer, Cham. https://doi.org/10.1007/978-3-031-16443-9_30

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-16443-9_30

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-16442-2

  • Online ISBN: 978-3-031-16443-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation