Abstract
Typical methods for semantic image segmentation rely on large training sets comprising pixel-level segmentations and pixel-level classifications. In medical applications, a large number of training images with per-pixel segmentations are difficult to obtain. In addition, many applications involve images or image tiles containing a single object/region of interest, where the image/tile-level information about object/region class is readily available. We propose a novel deep-neural-network (DNN) framework for joint segmentation and recognition of objects relying on weakly-supervised learning from training sets having very few expert segmentations, but with object-class labels available for all images/tiles. For weakly-supervised learning, we propose a variational-learning framework relying on Monte Carlo expectation maximization (MCEM), inferring a posterior distribution on the missing segmentations. We design an effective Metropolis-Hastings posterior sampler coupled with sample reparametrizations to enable end-to-end learning. Our DNN first produces probabilistic segmentations of objects, and then their probabilistic classifications. Results on two publicly available real-world datasets show the benefits of our strategies of (i) joint object segmentation and recognition as well as (ii) weakly-supervised MCEM-based learning.
The authors are grateful for support from the Infrastructure Facility for Advanced Research and Education in Diagnostics grant funded by Department of Biotechnology (DBT), Government of India (BT/INF/22/SP23026/2017).
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Ahn, J., Cho, S., Kwak, S.: Weakly supervised learning of instance segmentation with inter-pixel relations. In: IEEE Computer Vision and Pattern Recognition, pp. 2204–2213 (2019)
Chen, L., Zhu, Y., Papandreou, G., Schroff, F., Adam, H.: Encoder-decoder with atrous separable convolution for semantic image segmentation. In: European Conference on Computer Vision, pp. 833–851 (2018)
Girshick, R., Donahue, J., Darrell, T., Malik, J.: Rich feature hierarchies for accurate object detection and semantic segmentation. In: IEEE Computer Vision and Pattern Recognition, pp. 580–587 (2014)
Guo, H., Xu, M., Chi, Y., Zhang, L., Hua, X.-S.: Weakly supervised organ localization with attention maps regularized by local area reconstruction. In: Martel, A.L., et al. (eds.) MICCAI 2020, Part I. LNCS, vol. 12261, pp. 243–252. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59710-8_24
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: IEEE Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Hong, S., Noh, H., Han, B.: Decoupled deep neural network for semi-supervised semantic segmentation. In: Neural Information Processing Systems, pp. 1495–1503 (2015)
Hung, W., Tsai, Y., Liou, Y., Linand, Y., Yang, M.: Adversarial learning for semi-supervised semantic segmentation. In: British Machine Vision Conference, p. 65 (2018)
Kervadec, H., Dolz, J., Tang, M., Granger, E., Boykov, Y., Ayed, I.: Constrained-CNN losses for weakly supervised segmentation. Med. Image Ana. 54, 88–99 (2019)
LeCun, Y., Bottou, L., Orr, G.B., Müller, K.-R.: Efficient backprop. In: Orr, G.B., Müller, K.-R. (eds.) Neural Networks: Tricks of the Trade. LNCS, vol. 1524, pp. 9–50. Springer, Heidelberg (1998). https://doi.org/10.1007/3-540-49430-8_2
Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: IEEE Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)
Papandreou, G., Chen, L., Murphy, K., Yuille, A.: Weakly-and semi-supervised learning of a deep convolutional network for semantic image segmentation. In: International Conference on Computer Vision, pp. 1742–1750 (2015)
Reinhard, E., Ashikhmin, M., Gooch, B., Shirley, P.: Color transfer between images. IEEE Comp. Gra. App. 21(5), 34–41 (2001)
Robbins, H., Monro, S.: A stochastic approximation method. Anna. Math. Stat. 22, 400–407 (1951)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015, Part III. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Souly, N., Spampinato, C., Shah, M.: Semi supervised semantic segmentation using generative adversarial network. In: International Conference on Computer Vision, pp. 5688–5696 (2017)
Tardy, M., Mateus, D.: Looking for abnormalities in mammograms with self-and weakly supervised reconstruction. IEEE Trans. Med. Imaging PP, 1 (2021)
Zhou, Y., et al.: Collaborative learning of semi-supervised segmentation and classification for medical images. In: IEEE Computer Vision and Pattern Recognition, pp. 2088–2079 (2019)
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Gaikwad, A.V., Awate, S.P. (2021). Deep MCEM for Weakly-Supervised Learning to Jointly Segment and Recognize Objects Using Very Few Expert Segmentations. In: Feragen, A., Sommer, S., Schnabel, J., Nielsen, M. (eds) Information Processing in Medical Imaging. IPMI 2021. Lecture Notes in Computer Science(), vol 12729. Springer, Cham. https://doi.org/10.1007/978-3-030-78191-0_48
Download citation
DOI: https://doi.org/10.1007/978-3-030-78191-0_48
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-78190-3
Online ISBN: 978-3-030-78191-0
eBook Packages: Computer ScienceComputer Science (R0)