Skip to main content

Boundary-RL: Reinforcement Learning for Weakly-Supervised Prostate Segmentation in TRUS Images

  • Conference paper
  • First Online:
Machine Learning in Medical Imaging (MLMI 2023)

Abstract

We propose Boundary-RL, a novel weakly supervised segmentation method that utilises only patch-level labels for training. We envision segmentation as a boundary detection problem, rather than a pixel-level classification as in previous works. This outlook on segmentation may allow for boundary delineation under challenging scenarios such as where noise artefacts may be present within the region-of-interest (ROI) boundaries, where traditional pixel-level classification-based weakly supervised methods may not be able to effectively segment the ROI. Particularly of interest, ultrasound images, where intensity values represent acoustic impedance differences between boundaries, may also benefit from the boundary delineation approach. Our method uses reinforcement learning to train a controller function to localise boundaries of ROIs using a reward derived from a pre-trained boundary-presence classifier. The classifier indicates when an object boundary is encountered within a patch, serving as weak supervision, as the controller modifies the patch location in a sequential Markov decision process. The classifier itself is trained using only binary patch-level labels of object presence, the only labels used during training of the entire boundary delineation framework. The use of a controller ensures that sliding window over the entire image is not necessary and reduces possible false-positives or -negatives by minimising number of patches passed to the boundary-presence classifier. We evaluate our approach for a clinically relevant task of prostate gland segmentation on trans-rectal ultrasound images. We show improved performance compared to other tested weakly supervised methods, using the same labels e.g., multiple instance learning.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Chalcroft, L.F., et al.: Development and evaluation of intraoperative ultrasound segmentation with negative image frames and multiple observer labels. In: Noble, J.A., Aylward, S., Grimwood, A., Min, Z., Lee, S.-L., Hu, Y. (eds.) ASMUS 2021. LNCS, vol. 12967, pp. 25–34. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87583-1_3

    Chapter  Google Scholar 

  2. Chen, M.Y., Woodruff, M.A., Dasgupta, P., Rukin, N.J.: Variability in accuracy of prostate cancer segmentation among radiologists, urologists, and scientists. Cancer Med. 9(19), 7172–7182 (2020)

    Article  Google Scholar 

  3. Cubuk, E.D., Zoph, B., Mane, D., Vasudevan, V., Le, Q.V.: Autoaugment: learning augmentation policies from data. arXiv preprint arXiv:1805.09501 (2018)

  4. Czolbe, S., Arnavaz, K., Krause, O., Feragen, A.: Is segmentation uncertainty useful? In: Feragen, A., Sommer, S., Schnabel, J., Nielsen, M. (eds.) IPMI 2021. LNCS, vol. 12729, pp. 715–726. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-78191-0_55

    Chapter  Google Scholar 

  5. Dertkigil, S., Appenzeller, S., Lotufo, R., Rittner, L.: A systematic review of automated segmentation methods and public datasets for the lung and its lobes and findings on computed tomography images. Yearbook Med. Inf. 31(01), 277–295 (2022)

    Article  Google Scholar 

  6. Han, C., Lin, J., Mai, J., Wang, Y., Zhang, Q., et al.: Multi-layer pseudo-supervision for histopathology tissue semantic segmentation using patch-level classification labels. MedIA 80, 102487 (2022)

    Google Scholar 

  7. Hulsmans, F.J.J., Castelijns, J.A., Reeders, J.W., Tytgat, G.N.: Review of artifacts associated with transrectal ultrasound: understanding, recognition, and prevention of misinterpretation. J. Clin. Ultrasound 23(8), 483–494 (1995)

    Article  Google Scholar 

  8. Jia, Z., Huang, X., Eric, I., Chang, C., Xu, Y.: Constrained deep weak supervision for histopathology image segmentation. IEEE TMI 36(11), 2376–2388 (2017)

    Google Scholar 

  9. Karimi, D., Zeng, Q., Mathur, P., Avinash, A., et al.: Accurate and robust deep learning-based segmentation of the prostate clinical target volume in ultrasound images. MedIA 57, 186–196 (2019)

    Google Scholar 

  10. Kervadec, H., Dolz, J., Tang, M., Granger, E., Boykov, Y., et al.: Constrained-cnn losses for weakly supervised segmentation. MedIA 54, 88–89 (2019)

    Google Scholar 

  11. Kots, M., Chukanov, V.: U-net adaptation for multiple instance learning. In: Journal of Physics: Conference Series, vol. 1236. IOP Publishing (2019)

    Google Scholar 

  12. Lei, Y., Tian, S., He, X., Wang, T., et al.: Ultrasound prostate segmentation based on multidirectional deeply supervised v-net. Med. Phys. 46(7), 3194–3206 (2019)

    Article  Google Scholar 

  13. Li, K., Qian, Z., Han, Y., Eric, I., Chang, C., et al.: Weakly supervised histopathology image segmentation with self-attention. MedIA 86, 102791 (2023)

    Google Scholar 

  14. Liu, D., Wang, L., Du, Y., Cong, M., Li, Y.: 3-d prostate MR and TRUS images detection and segmentation for puncture biopsy. IEEE Trans. Instrument. Meas. 71, 1–13 (2022)

    Article  Google Scholar 

  15. Pathak, D., Krahenbuhl, P., Darrell, T.: Constrained convolutional neural networks for weakly supervised segmentation. In: ICCV (2015)

    Google Scholar 

  16. Pathak, D., Shelhamer, E., Long, J., Darrell, T.: Fully convolutional multi-class multiple instance learning. arXiv preprint arXiv:1412.7144 (2014)

  17. Pinheiro, P.O., Collobert, R.: From image-level to pixel-level labeling with convolutional networks. In: CVPR (2015)

    Google Scholar 

  18. Ramesh, K., Kumar, G.K., Swapna, K., Datta, D., Rajest, S.S.: A review of medical image segmentation algorithms. EAI PHAT 7(27), e6 (2021)

    Google Scholar 

  19. Ren, W., Huang, K., Tao, D., Tan, T.: Weakly supervised large scale object localization with multiple instance learning and bag splitting. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 405–416 (2015)

    Article  Google Scholar 

  20. Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28

    Chapter  Google Scholar 

  21. Saeed, S.U., et al.: Adaptable image quality assessment using meta-reinforcement learning of task amenability. In: Noble, J.A., Aylward, S., Grimwood, A., Min, Z., Lee, S.-L., Hu, Y. (eds.) ASMUS 2021. LNCS, vol. 12967, pp. 191–201. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87583-1_19

    Chapter  Google Scholar 

  22. Saeed, S.U., Fu, Y., Stavrinides, V., Baum, Z.M., Yang, Q., et al.: Image quality assessment for machine learning tasks using meta-reinforcement learning. MedIA 78, 102427 (2022)

    Google Scholar 

  23. Saeed, S.U., Yan, W., Fu, Y., Giganti, F., et al.: Image quality assessment by overlapping task-specific and task-agnostic measures: application to prostate multiparametric MR images for cancer segmentation. In: Machine Learning for Biomedical Imaging (IPMI 2021), vol. 1 (2022)

    Google Scholar 

  24. Savjani, R.R., Lauria, M., Bose, S., Deng, J., et al.: Automated tumor segmentation in radiotherapy. In: Seminars in Radiation Oncology, vol. 32. Elsevier (2022)

    Google Scholar 

  25. Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 (2017)

  26. Sharma, N., Aggarwal, L.M.: Automated medical image segmentation techniques. J. Med. Phys. 35(1), 3 (2010)

    Article  Google Scholar 

  27. Shi, X., Xing, F., Xie, Y., Zhang, Z., Cui, L., et al.: Loss-based attention for deep multiple instance learning. In: AAAI 2020, vol. 34 (2020)

    Google Scholar 

  28. Tan, M., Le, Q.: Efficientnet: rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning. PMLR (2019)

    Google Scholar 

  29. Wang, X., Chang, Z., Zhang, Q., Li, C., Miao, F., et al.: Prostate ultrasound image segmentation based on dsu-net. Biomedicines 11(3), 646 (2023)

    Article  Google Scholar 

  30. Wang, Y., Dou, H., Hu, X., Zhu, L., Yang, X., et al.: Deep attentive features for prostate segmentation in 3D transrectal ultrasound. IEEE TMI 38(12), 2768–2778 (2019)

    Google Scholar 

  31. Xu, G., Song, Z., Sun, Z., Ku, C., et al.: Camel: a weakly supervised learning framework for histopathology image segmentation. In: IEEE/CVF ICCV (2019)

    Google Scholar 

  32. Xu, R.S., Michailovich, O., Salama, M.: Information tracking approach to segmentation of ultrasound imagery of the prostate. IEEE Trans. Ultrasonics Ferroelectr. Freq. Control 57(8), 1748–1761 (2010)

    Article  Google Scholar 

  33. Xu, Y., Zhu, J.Y., Eric, I., Chang, C., Lai, M., et al.: Weakly supervised histopathology cancer image segmentation and classification. MedIA 18(3), 591–604 (2014)

    Google Scholar 

  34. Yoon, J., Arik, S., Pfister, T.: Data valuation using reinforcement learning. In: International Conference on Machine Learning. PMLR (2020)

    Google Scholar 

  35. Zeng, Y., Tsui, P.H., Wu, W., Zhou, Z., Wu, S.: Fetal ultrasound image segmentation for automatic head circumference biometry using deeply supervised attention-gated v-net. J. Dig. Imaging 34, 134–148 (2021)

    Article  Google Scholar 

  36. Zhang, M., Zhou, Y., Zhao, J., Man, Y., Liu, B., et al.: A survey of semi-and weakly supervised semantic segmentation of images. AIRE 53, 4259–4288 (2020)

    Google Scholar 

  37. Zhang, X., Wang, Q., Zhang, J., Zhong, Z.: Adversarial autoaugment. arXiv preprint arXiv:1912.11188 (2019)

Download references

Acknowledgements

This work was supported by the EPSRC grant [EP/T029404/1], Wellcome/EPSRC Centre for Interventional and Surgical Sciences [203145Z/16/Z], and the International Alliance for Cancer Early Detection, an alliance between Cancer Research UK [C28070/A30912; 73666/A31378], Canary Center at Stanford University, the University of Cambridge, OHSU Knight Cancer Institute, University College London and the University of Manchester.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Weixi Yi .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Yi, W. et al. (2024). Boundary-RL: Reinforcement Learning for Weakly-Supervised Prostate Segmentation in TRUS Images. In: Cao, X., Xu, X., Rekik, I., Cui, Z., Ouyang, X. (eds) Machine Learning in Medical Imaging. MLMI 2023. Lecture Notes in Computer Science, vol 14348. Springer, Cham. https://doi.org/10.1007/978-3-031-45673-2_28

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-45673-2_28

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-45672-5

  • Online ISBN: 978-3-031-45673-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics