Abstract
Automatic anatomical segmentation and landmark localization in medical images are important tasks during craniofacial analysis. While deep neural networks have been recently applied to segment cranial bones and identify cranial landmarks from computed tomography (CT) or magnetic resonance (MR) images, existing methods often provide suboptimal and sometimes unrealistic results because they do not incorporate contextual image information. Additionally, most state-of-the-art deep learning methods for cranial bone segmentation and landmark detection rely on multi-stage data processing pipelines, which are inefficient and prone to errors. In this paper, we propose a novel context encoding-constrained neural network for single-stage cranial bone labeling and landmark localization. Specifically, we design and incorporate a novel context encoding module into a U-Net-like architecture. We explicitly enforce the network to capture context-related features for representation learning so pixel-wise predictions are not isolated from the image context. In addition, we introduce a new auxiliary task to model the relative spatial configuration of different anatomical landmarks, which serves as an additional regularization that further refines network predictions. The proposed method is end-to-end trainable for single-stage cranial bone labeling and landmark localization. The method was evaluated on a highly diverse pediatric 3D CT image dataset with 274 subjects. Our experiments demonstrate superior performance of our method compared to state-of-the-art approaches.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Wood, B.C., et al.: What’s in a name? Accurately diagnosing metopic craniosynostosis using a computational approach. Plast. Reconstr. Surg. 137, 205–213 (2016)
Mendoza, C.S., Safdar, N., Okada, K., Myers, E., Rogers, G.F., Linguraru, M.G.: Personalized assessment of craniosynostosis via statistical shape modeling. Med. Image Anal. 18, 635–646 (2014)
Porras, A.R., et al.: Quantification of head shape from three-dimensional photography for presurgical and postsurgical evaluation of craniosynostosis. Plast. Reconstr. Surg. 144, 1051e–1060e (2019)
Rodriguez-Florez, N., et al.: Statistical shape modelling to aid surgical planning: associations between surgical parameters and head shapes following spring-assisted cranioplasty. Int. J. Comput. Assis. Radiol. Surg. 12, 1739–1749 (2017)
Lamecker, H., et al.: Surgical treatment of craniosynostosis based on a statistical 3D-shape model: first clinical application. Int. J. Comput. Assis. Radiol. Surg. 1, 253 (2006)
Porras, A.R., et al.: Locally affine diffeomorphic surface registration and its application to surgical planning of fronto-orbital advancement. IEEE Trans. Med. Imaging 37, 1690–1700 (2018)
Liu, L., et al.: Interactive separation of segmented bones in CT volumes using graph cut. In: Metaxas, D., Axel, L., Fichtinger, G., Székely, G. (eds.) MICCAI 2008. LNCS, vol. 5241, pp. 296–304. Springer, Heidelberg (2008). https://doi.org/10.1007/978-3-540-85988-8_36
Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 3431–3440 (2014)
Lian, C., Liu, M., Zhang, J., Shen, D.: Hierarchical fully convolutional network for joint atrophy localization and alzheimer’s disease diagnosis using structural MRI. IEEE Trans. Pattern Anal. Mach. Intell. 42, 880–893 (2020)
Lian, C., et al.: Multi-channel multi-scale fully convolutional network for 3D perivascular spaces segmentation in 7T MR images. Med. Image Anal. 46, 106–117 (2018)
Egger, J., Pfarrkirchner, B., Gsaxner, C., Lindner, L., Schmalstieg, D., Wallner, J.: Fully convolutional mandible segmentation on a valid ground-truth dataset. In: The 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 656–660 (2018)
Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
González Sánchez, J.C., Magnusson, M., Sandborg, M., Carlsson Tedgren, Å., Malusek, A.: Segmentation of bones in medical dual-energy computed tomography volumes using the 3D U-Net. Physica Medica 69, 241–247 (2020)
Liu, Q., et al.: SkullEngine: a multi-stage CNN framework for collaborative CBCT image segmentation and landmark detection. In: Lian, C., Cao, X., Rekik, I., Xu, X., Yan, P. (eds.) MLMI 2021. LNCS, vol. 12966, pp. 606–614. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-87589-3_62
Zhang, H., et al.: Context encoding for semantic segmentation. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7751–7160 (2018)
Oliva, A., Torralba, A.: The role of context in object recognition. Trends Cogn. Sci. 11, 520–527 (2007)
Tu, Z., Bai, X.: Auto-context and its application to high-level vision tasks and 3D brain image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 32, 1744–1757 (2009)
Zhang, J., et al.: Joint Craniomaxillofacial Bone Segmentation ans Landmark Digitization by Context-Guided Fully Convolutional Networks. Springer, Cham (2017)
Torosdagli, N., Liberton, D.K., Verma, P., Sincan, M., Lee, J.S., Bagci, U.: Deep geodesic learning for segmentation and anatomical landmarking. IEEE Trans. Med. Imaging 38, 919–931 (2018)
Lian, C., et al.: Multi-task dynamic transformer network for concurrent bone segmentation and large-scale landmark localization with dental CBCT. In: Martel, A.L., et al. (eds.) MICCAI 2020. LNCS, vol. 12264, pp. 807–816. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-59719-1_78
Payer, C., Štern, D., Bischof, H., Urschler, M.: Regressing heatmaps for multiple landmark localization using CNNs. In: Ourselin, S., Joskowicz, L., Sabuncu, M.R., Unal, G., Wells, W. (eds.) MICCAI 2016. LNCS, vol. 9901, pp. 230–238. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46723-8_27
Dangi, S., et al.: Robust head CT image registration pipeline for craniosynostosis skull correction surgery. In: Healthcare Technology Letters, pp. 174–178. Institution of Engineering and Technology (2017)
Sgouros, S., Goldin, J.H., Hockley, A.D., Wake, M., Natarajan, K.: Intracranial volume change in childhood. J. Neurosurg. 94, 610–616 (1999)
Acknowledgments
The research reported in this publication was supported by the National Institute Of Dental & Craniofacial Research of the National Institutes of Health under Award Number R00DE027993. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
1 Electronic supplementary material
Below is the link to the electronic supplementary material.
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Liu, J., Xing, F., Shaikh, A., Linguraru, M.G., Porras, A.R. (2022). Learning with Context Encoding for Single-Stage Cranial Bone Labeling and Landmark Localization. In: Wang, L., Dou, Q., Fletcher, P.T., Speidel, S., Li, S. (eds) Medical Image Computing and Computer Assisted Intervention – MICCAI 2022. MICCAI 2022. Lecture Notes in Computer Science, vol 13438. Springer, Cham. https://doi.org/10.1007/978-3-031-16452-1_28
Download citation
DOI: https://doi.org/10.1007/978-3-031-16452-1_28
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-16451-4
Online ISBN: 978-3-031-16452-1
eBook Packages: Computer ScienceComputer Science (R0)
