Skip to main content

Deep MCEM for Weakly-Supervised Learning to Jointly Segment and Recognize Objects Using Very Few Expert Segmentations

  • Conference paper
  • First Online:
Information Processing in Medical Imaging (IPMI 2021)

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12729))

Included in the following conference series:

  • 4121 Accesses

Abstract

Typical methods for semantic image segmentation rely on large training sets comprising pixel-level segmentations and pixel-level classifications. In medical applications, a large number of training images with per-pixel segmentations are difficult to obtain. In addition, many applications involve images or image tiles containing a single object/region of interest, where the image/tile-level information about object/region class is readily available. We propose a novel deep-neural-network (DNN) framework for joint segmentation and recognition of objects relying on weakly-supervised learning from training sets having very few expert segmentations, but with object-class labels available for all images/tiles. For weakly-supervised learning, we propose a variational-learning framework relying on Monte Carlo expectation maximization (MCEM), inferring a posterior distribution on the missing segmentations. We design an effective Metropolis-Hastings posterior sampler coupled with sample reparametrizations to enable end-to-end learning. Our DNN first produces probabilistic segmentations of objects, and then their probabilistic classifications. Results on two publicly available real-world datasets show the benefits of our strategies of (i) joint object segmentation and recognition as well as (ii) weakly-supervised MCEM-based learning.

The authors are grateful for support from the Infrastructure Facility for Advanced Research and Education in Diagnostics grant funded by Department of Biotechnology (DBT), Government of India (BT/INF/22/SP23026/2017).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Ahn, J., Cho, S., Kwak, S.: Weakly supervised learning of instance segmentation with inter-pixel relations. In: IEEE Computer Vision and Pattern Recognition, pp. 2204–2213 (2019)

    Google Scholar 

  2. Chen, L., Zhu, Y., Papandreou, G., Schroff, F., Adam, H.: Encoder-decoder with atrous separable convolution for semantic image segmentation. In: European Conference on Computer Vision, pp. 833–851 (2018)

    Google Scholar 

  3. Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: IEEE Computer Vision and Pattern Recognition, pp. 580–587 (2014)

    Google Scholar 

  4. Guo, H., Xu, M., Chi, Y., Zhang, L., Hua, X.-S.: Weakly supervised organ localization with attention maps regularized by local area reconstruction. In: Martel, A.L., et al. (eds.) MICCAI 2020, Part I. LNCS, vol. 12261, pp. 243–252. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59710-8_24

    Chapter  Google Scholar 

  5. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: IEEE Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  6. Hong, S., Noh, H., Han, B.: Decoupled deep neural network for semi-supervised semantic segmentation. In: Neural Information Processing Systems, pp. 1495–1503 (2015)

    Google Scholar 

  7. Hung, W., Tsai, Y., Liou, Y., Linand, Y., Yang, M.: Adversarial learning for semi-supervised semantic segmentation. In: British Machine Vision Conference, p. 65 (2018)

    Google Scholar 

  8. Kervadec, H., Dolz, J., Tang, M., Granger, E., Boykov, Y., Ayed, I.: Constrained-CNN losses for weakly supervised segmentation. Med. Image Ana. 54, 88–99 (2019)

    Article  Google Scholar 

  9. LeCun, Y., Bottou, L., Orr, G.B., Müller, K.-R.: Efficient backprop. In: Orr, G.B., Müller, K.-R. (eds.) Neural Networks: Tricks of the Trade. LNCS, vol. 1524, pp. 9–50. Springer, Heidelberg (1998). https://doi.org/10.1007/3-540-49430-8_2

    Chapter  Google Scholar 

  10. Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: IEEE Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)

    Google Scholar 

  11. Papandreou, G., Chen, L., Murphy, K., Yuille, A.: Weakly-and semi-supervised learning of a deep convolutional network for semantic image segmentation. In: International Conference on Computer Vision, pp. 1742–1750 (2015)

    Google Scholar 

  12. Reinhard, E., Ashikhmin, M., Gooch, B., Shirley, P.: Color transfer between images. IEEE Comp. Gra. App. 21(5), 34–41 (2001)

    Article  Google Scholar 

  13. Robbins, H., Monro, S.: A stochastic approximation method. Anna. Math. Stat. 22, 400–407 (1951)

    Article  MathSciNet  Google Scholar 

  14. Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015, Part III. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28

    Chapter  Google Scholar 

  15. Souly, N., Spampinato, C., Shah, M.: Semi supervised semantic segmentation using generative adversarial network. In: International Conference on Computer Vision, pp. 5688–5696 (2017)

    Google Scholar 

  16. Tardy, M., Mateus, D.: Looking for abnormalities in mammograms with self-and weakly supervised reconstruction. IEEE Trans. Med. Imaging PP, 1 (2021)

    Google Scholar 

  17. Zhou, Y., et al.: Collaborative learning of semi-supervised segmentation and classification for medical images. In: IEEE Computer Vision and Pattern Recognition, pp. 2088–2079 (2019)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding authors

Correspondence to Akshay V. Gaikwad or Suyash P. Awate .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Gaikwad, A.V., Awate, S.P. (2021). Deep MCEM for Weakly-Supervised Learning to Jointly Segment and Recognize Objects Using Very Few Expert Segmentations. In: Feragen, A., Sommer, S., Schnabel, J., Nielsen, M. (eds) Information Processing in Medical Imaging. IPMI 2021. Lecture Notes in Computer Science(), vol 12729. Springer, Cham. https://doi.org/10.1007/978-3-030-78191-0_48

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-78191-0_48

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-78190-3

  • Online ISBN: 978-3-030-78191-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics