Abstract
We present a multistage method for deep semantic segmentation of bone structures based on a landmark-based shape regression and subsequent local segmentation of relevant areas. Our solution covers the entire pipeline from 2D-based pre-segmentation, a method for fast deep 3D shape regression and subsequent patch-based 3D semantic segmentation for final segmentation. Since we perform landmark regression using a statistical shape model, our method is able to fit an arbitrary number of landmarks without increase in model complexity. The algorithm is evaluated on the OAI-ZIB dataset, for which we use the binary masks to generate sets of corresponding landmarks and build a deep statistical shape model. By employing our proposed deep shape fitting, our method achieves the performance of existing high-precision approaches in terms of segmentation accuracy while at the same time drastically reducing computational complexity and improving runtime by a large margin.
J. Schock, M. Kopaczka, S. Nebelung and D. Merhof—Both authors contributed equally to this work.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Code available at https://github.com/justusschock/OAI-Knee-Segmentation.
References
Ambellan, F., Tack, A., Ehlke, M., Zachow, S.: Automated segmentation of knee bone and cartilage combining statistical shape knowledge and convolutional neural networks: data from the osteoarthritis initiative. Med. Image Anal. 52 (2018). https://doi.org/10.1016/j.media.2018.11.009
Bonaretti, S., Gold, G.E., Beaupre, G.S.: pyKNEEr: an image analysis workflow for open and reproducible research on femoral knee cartilage. PLoS ONE 15(1), 1–19 (2020). https://doi.org/10.1371/journal.pone.0226501
Eschweiler, D., Klose, T., Müller-Fouarge, F.N., Kopaczka, M., Stegmaier, J.: Towards annotation-free segmentation of fluorescently labeled cell membranes in confocal microscopy images. In: Burgos, N., Gooya, A., Svoboda, D. (eds.) SASHIMI 2019. LNCS, vol. 11827, pp. 81–89. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32778-1_9
Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1125–1134 (2017)
Karimi, D., Samei, G., Kesch, C., Nir, G., Salcudean, S.E.: Prostate segmentation in MRI using a convolutional neural network architecture and training strategy based on statistical shape models. Int. J. Comput. Assist. Radiol. Surg. 13(8), 1211–1219 (2018)
Kingma, D.P., Welling, M.: Auto-encoding variational bayes. arXiv preprint arXiv:1312.6114 (2013)
Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)
Milletari, F., Rothberg, A., Jia, J., Sofka, M.: Integrating statistical prior knowledge into convolutional neural networks. In: Descoteaux, M., Maier-Hein, L., Franz, A., Jannin, P., Collins, D.L., Duchesne, S. (eds.) MICCAI 2017. LNCS, vol. 10433, pp. 161–168. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-66182-7_19
Mitchell, S.C., Bosch, J.G., Lelieveldt, B.P., Van der Geest, R.J., Reiber, J.H., Sonka, M.: 3-D active appearance models: segmentation of cardiac MR and ultrasound images. IEEE Trans. Med. Imaging 21(9), 1167–1178 (2002)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Tan, C., Yan, Z., Zhang, S., Li, K., Metaxas, D.N.: Collaborative multi-agent learning for MR knee articular cartilage segmentation. In: Shen, D., et al. (eds.) MICCAI 2019. LNCS, vol. 11765, pp. 282–290. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-32245-8_32
Vercauteren, T., Pennec, X., Perchant, A., Ayache, N.: Diffeomorphic demons: efficient non-parametric image registration. NeuroImage 45(1), S61–S72 (2009)
Acknowledgements
We thank The OAI initiative for providing Access to their image data and Felix Ambellan from ZIB Berlin for making the segmentations publically available. The OAI is a public-private partnership comprised of five contracts (N01-AR-2-2258; N01-AR-2-2259; N01-AR-2-2260; N01-AR-2-2261; N01-AR-2-2262) funded by the National Institutes of Health, a branch of the Department of Health and Human Services, and conducted by the OAI Study Investigators. Private funding partners include Merck Research Laboratories; Novartis Pharmaceuticals Corporation, GlaxoSmithKline; and Pfizer, Inc. Private sector funding for the OAI is managed by the Foundation for the National Institutes of Health. This manuscript was prepared using an OAI public use data set and does not necessarily reflect the opinions or views of the OAI investigators, the NIH, or the private funding partners.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Schock, J. et al. (2020). A Method for Semantic Knee Bone and Cartilage Segmentation with Deep 3D Shape Fitting Using Data from the Osteoarthritis Initiative. In: Reuter, M., Wachinger, C., Lombaert, H., Paniagua, B., Goksel, O., Rekik, I. (eds) Shape in Medical Imaging. ShapeMI 2020. Lecture Notes in Computer Science(), vol 12474. Springer, Cham. https://doi.org/10.1007/978-3-030-61056-2_7
Download citation
DOI: https://doi.org/10.1007/978-3-030-61056-2_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-61055-5
Online ISBN: 978-3-030-61056-2
eBook Packages: Computer ScienceComputer Science (R0)
