Abstract
We propose Boundary-RL, a novel weakly supervised segmentation method that utilises only patch-level labels for training. We envision segmentation as a boundary detection problem, rather than a pixel-level classification as in previous works. This outlook on segmentation may allow for boundary delineation under challenging scenarios such as where noise artefacts may be present within the region-of-interest (ROI) boundaries, where traditional pixel-level classification-based weakly supervised methods may not be able to effectively segment the ROI. Particularly of interest, ultrasound images, where intensity values represent acoustic impedance differences between boundaries, may also benefit from the boundary delineation approach. Our method uses reinforcement learning to train a controller function to localise boundaries of ROIs using a reward derived from a pre-trained boundary-presence classifier. The classifier indicates when an object boundary is encountered within a patch, serving as weak supervision, as the controller modifies the patch location in a sequential Markov decision process. The classifier itself is trained using only binary patch-level labels of object presence, the only labels used during training of the entire boundary delineation framework. The use of a controller ensures that sliding window over the entire image is not necessary and reduces possible false-positives or -negatives by minimising number of patches passed to the boundary-presence classifier. We evaluate our approach for a clinically relevant task of prostate gland segmentation on trans-rectal ultrasound images. We show improved performance compared to other tested weakly supervised methods, using the same labels e.g., multiple instance learning.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Chalcroft, L.F., et al.: Development and evaluation of intraoperative ultrasound segmentation with negative image frames and multiple observer labels. In: Noble, J.A., Aylward, S., Grimwood, A., Min, Z., Lee, S.-L., Hu, Y. (eds.) ASMUS 2021. LNCS, vol. 12967, pp. 25–34. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87583-1_3
Chen, M.Y., Woodruff, M.A., Dasgupta, P., Rukin, N.J.: Variability in accuracy of prostate cancer segmentation among radiologists, urologists, and scientists. Cancer Med. 9(19), 7172–7182 (2020)
Cubuk, E.D., Zoph, B., Mane, D., Vasudevan, V., Le, Q.V.: Autoaugment: learning augmentation policies from data. arXiv preprint arXiv:1805.09501 (2018)
Czolbe, S., Arnavaz, K., Krause, O., Feragen, A.: Is segmentation uncertainty useful? In: Feragen, A., Sommer, S., Schnabel, J., Nielsen, M. (eds.) IPMI 2021. LNCS, vol. 12729, pp. 715–726. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-78191-0_55
Dertkigil, S., Appenzeller, S., Lotufo, R., Rittner, L.: A systematic review of automated segmentation methods and public datasets for the lung and its lobes and findings on computed tomography images. Yearbook Med. Inf. 31(01), 277–295 (2022)
Han, C., Lin, J., Mai, J., Wang, Y., Zhang, Q., et al.: Multi-layer pseudo-supervision for histopathology tissue semantic segmentation using patch-level classification labels. MedIA 80, 102487 (2022)
Hulsmans, F.J.J., Castelijns, J.A., Reeders, J.W., Tytgat, G.N.: Review of artifacts associated with transrectal ultrasound: understanding, recognition, and prevention of misinterpretation. J. Clin. Ultrasound 23(8), 483–494 (1995)
Jia, Z., Huang, X., Eric, I., Chang, C., Xu, Y.: Constrained deep weak supervision for histopathology image segmentation. IEEE TMI 36(11), 2376–2388 (2017)
Karimi, D., Zeng, Q., Mathur, P., Avinash, A., et al.: Accurate and robust deep learning-based segmentation of the prostate clinical target volume in ultrasound images. MedIA 57, 186–196 (2019)
Kervadec, H., Dolz, J., Tang, M., Granger, E., Boykov, Y., et al.: Constrained-cnn losses for weakly supervised segmentation. MedIA 54, 88–89 (2019)
Kots, M., Chukanov, V.: U-net adaptation for multiple instance learning. In: Journal of Physics: Conference Series, vol. 1236. IOP Publishing (2019)
Lei, Y., Tian, S., He, X., Wang, T., et al.: Ultrasound prostate segmentation based on multidirectional deeply supervised v-net. Med. Phys. 46(7), 3194–3206 (2019)
Li, K., Qian, Z., Han, Y., Eric, I., Chang, C., et al.: Weakly supervised histopathology image segmentation with self-attention. MedIA 86, 102791 (2023)
Liu, D., Wang, L., Du, Y., Cong, M., Li, Y.: 3-d prostate MR and TRUS images detection and segmentation for puncture biopsy. IEEE Trans. Instrument. Meas. 71, 1–13 (2022)
Pathak, D., Krahenbuhl, P., Darrell, T.: Constrained convolutional neural networks for weakly supervised segmentation. In: ICCV (2015)
Pathak, D., Shelhamer, E., Long, J., Darrell, T.: Fully convolutional multi-class multiple instance learning. arXiv preprint arXiv:1412.7144 (2014)
Pinheiro, P.O., Collobert, R.: From image-level to pixel-level labeling with convolutional networks. In: CVPR (2015)
Ramesh, K., Kumar, G.K., Swapna, K., Datta, D., Rajest, S.S.: A review of medical image segmentation algorithms. EAI PHAT 7(27), e6 (2021)
Ren, W., Huang, K., Tao, D., Tan, T.: Weakly supervised large scale object localization with multiple instance learning and bag splitting. IEEE Trans. Pattern Anal. Mach. Intell. 38(2), 405–416 (2015)
Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Saeed, S.U., et al.: Adaptable image quality assessment using meta-reinforcement learning of task amenability. In: Noble, J.A., Aylward, S., Grimwood, A., Min, Z., Lee, S.-L., Hu, Y. (eds.) ASMUS 2021. LNCS, vol. 12967, pp. 191–201. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87583-1_19
Saeed, S.U., Fu, Y., Stavrinides, V., Baum, Z.M., Yang, Q., et al.: Image quality assessment for machine learning tasks using meta-reinforcement learning. MedIA 78, 102427 (2022)
Saeed, S.U., Yan, W., Fu, Y., Giganti, F., et al.: Image quality assessment by overlapping task-specific and task-agnostic measures: application to prostate multiparametric MR images for cancer segmentation. In: Machine Learning for Biomedical Imaging (IPMI 2021), vol. 1 (2022)
Savjani, R.R., Lauria, M., Bose, S., Deng, J., et al.: Automated tumor segmentation in radiotherapy. In: Seminars in Radiation Oncology, vol. 32. Elsevier (2022)
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy optimization algorithms. arXiv preprint arXiv:1707.06347 (2017)
Sharma, N., Aggarwal, L.M.: Automated medical image segmentation techniques. J. Med. Phys. 35(1), 3 (2010)
Shi, X., Xing, F., Xie, Y., Zhang, Z., Cui, L., et al.: Loss-based attention for deep multiple instance learning. In: AAAI 2020, vol. 34 (2020)
Tan, M., Le, Q.: Efficientnet: rethinking model scaling for convolutional neural networks. In: International Conference on Machine Learning. PMLR (2019)
Wang, X., Chang, Z., Zhang, Q., Li, C., Miao, F., et al.: Prostate ultrasound image segmentation based on dsu-net. Biomedicines 11(3), 646 (2023)
Wang, Y., Dou, H., Hu, X., Zhu, L., Yang, X., et al.: Deep attentive features for prostate segmentation in 3D transrectal ultrasound. IEEE TMI 38(12), 2768–2778 (2019)
Xu, G., Song, Z., Sun, Z., Ku, C., et al.: Camel: a weakly supervised learning framework for histopathology image segmentation. In: IEEE/CVF ICCV (2019)
Xu, R.S., Michailovich, O., Salama, M.: Information tracking approach to segmentation of ultrasound imagery of the prostate. IEEE Trans. Ultrasonics Ferroelectr. Freq. Control 57(8), 1748–1761 (2010)
Xu, Y., Zhu, J.Y., Eric, I., Chang, C., Lai, M., et al.: Weakly supervised histopathology cancer image segmentation and classification. MedIA 18(3), 591–604 (2014)
Yoon, J., Arik, S., Pfister, T.: Data valuation using reinforcement learning. In: International Conference on Machine Learning. PMLR (2020)
Zeng, Y., Tsui, P.H., Wu, W., Zhou, Z., Wu, S.: Fetal ultrasound image segmentation for automatic head circumference biometry using deeply supervised attention-gated v-net. J. Dig. Imaging 34, 134–148 (2021)
Zhang, M., Zhou, Y., Zhao, J., Man, Y., Liu, B., et al.: A survey of semi-and weakly supervised semantic segmentation of images. AIRE 53, 4259–4288 (2020)
Zhang, X., Wang, Q., Zhang, J., Zhong, Z.: Adversarial autoaugment. arXiv preprint arXiv:1912.11188 (2019)
Acknowledgements
This work was supported by the EPSRC grant [EP/T029404/1], Wellcome/EPSRC Centre for Interventional and Surgical Sciences [203145Z/16/Z], and the International Alliance for Cancer Early Detection, an alliance between Cancer Research UK [C28070/A30912; 73666/A31378], Canary Center at Stanford University, the University of Cambridge, OHSU Knight Cancer Institute, University College London and the University of Manchester.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Yi, W. et al. (2024). Boundary-RL: Reinforcement Learning for Weakly-Supervised Prostate Segmentation in TRUS Images. In: Cao, X., Xu, X., Rekik, I., Cui, Z., Ouyang, X. (eds) Machine Learning in Medical Imaging. MLMI 2023. Lecture Notes in Computer Science, vol 14348. Springer, Cham. https://doi.org/10.1007/978-3-031-45673-2_28
Download citation
DOI: https://doi.org/10.1007/978-3-031-45673-2_28
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-45672-5
Online ISBN: 978-3-031-45673-2
eBook Packages: Computer ScienceComputer Science (R0)