Abstract
Efficient and accurate vertebra labeling in medical images is important for longitudinal assessment, pathological diagnosis, and clinical treatment of the spinal diseases. In practice, the abnormal conditions in the images increase the difficulties to accurately identify the vertebrae locations. Such conditions include uncommon spinal curvature, bright imaging artifacts caused by metal implants, and limited field of the imaging view, etc. In this chapter, we propose an automatic vertebra localization and labeling method with high accuracy and efficiency for medical images. First, we introduce a deep image-to-image network (DI2IN) which generates the probability maps for vertebral centroids. The DI2IN adopts multiple prevailing techniques, including feature concatenation and deep supervision, to boost its performance. Second, a message-passing scheme is used to evolve the probability maps from DI2IN within multiple iterations, according to the spatial relationship of vertebrae. Finally, the locations of vertebra are refined and constrained with a learned sparse representation. We evaluate the proposed method on two categories of public databases, 3D CT volumes, and 2D X-ray scans, under various pathologies. The experimental results show that our method outperforms other state-of-the-art methods in terms of localization accuracy. In order to further boost the performance, we add 1000 extra 3D CT volumes with expert annotation when training the DI2IN for CT images. The results justify that large databases can improve the generalization capability and the performance of the deep neural networks. To the best of our knowledge, it is the first time that more than 1000 3D CT volumes are utilized for the anatomical landmark detection and the overall identification rate reaches 90% in spine labeling.
Dong Yang and Tao Xiong contributed equally to this work.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Badrinarayanan V, Kendall A, Cipolla R (2015) Segnet: a deep convolutional encoder-decoder architecture for image segmentation. arXiv:1511.00561
Benameur S, Mignotte M, Parent S, Labelle H, Skalli W, de Guise J (2003) 3d/2d registration and segmentation of scoliotic vertebrae using statistical models. Comput Med Imaging Graph 27(5):321–337
Boisvert J, Cheriet F, Pennec X, Labelle H, Ayache N (2008) Geometric variability of the scoliotic spine using statistics on articulated shape models. IEEE Trans Med Imaging 27(4):557–568
Chen H, Shen C, Qin J, Ni D, Shi L, Cheng JC, Heng PA (2015) Automatic localization and identification of vertebrae in spine ct via a joint learning model with deep neural networks. In: International conference on medical image computing and computer-assisted intervention, pp 515–522. Springer International Publishing
Chu X, Ouyang W, Li H, Wang X (2016) Structured feature learning for pose estimation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4715–4723
Demner-Fushman D, Kohli MD, Rosenman MB, Shooshan SE, Rodriguez L, Antani S, Thoma GR, McDonald CJ (2015) Preparing a collection of radiology examinations for distribution and retrieval. J Am Med Inform Assoc 23(2):304–310
Genant HK, Wu CY, van Kuijk C, Nevitt MC (1993) Vertebral fracture assessment using a semiquantitative technique. J Bone Miner Res 8(9):1137–1148
Glocker B, Feulner J, Criminisi A, Haynor D, Konukoglu E (2012) Automatic localization and identification of vertebrae in arbitrary field-of-view ct scans. In: Medical image computing and computer-assisted intervention-MICCAI, pp 590–598
Glocker B, Zikic D, Konukoglu E, Haynor DR, Criminisi A (2013) Vertebrae localization in pathological spine ct via dense classification from sparse annotations. In: International conference on medical image computing and computer-assisted intervention, pp 262–270. Springer
Jaeger S, Candemir S, Antani S, Wáng YXJ, Lu PX, Thoma G (2014) Two public chest x-ray datasets for computer-aided screening of pulmonary diseases. Quant Imaging Med Surg 4(6):475–477
Klinder T, Ostermann J, Ehm M, Franz A, Kneser R, Lorenz C (2009) Automated model-based vertebra detection, identification, and segmentation in ct images. Med Image Anal 13(3):471–482
Komodakis N, Paragios N, Tziritas G (2007) Mrf optimization via dual decomposition: message-passing revisited. In: 2007 IEEE 11th international conference on computer vision, ICCV 2007, pp 1–8. IEEE
Long J, Shelhamer E, Darrell T (2015) Fully convolutional networks for semantic segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3431–3440
Milletari F, Navab N, Ahmadi SA (2016) V-net: fully convolutional neural networks for volumetric medical image segmentation. In: 2016 Fourth International Conference on 3D Vision (3DV), pp 565–571. IEEE
Nowozin S, Lampert CH et al (2011) Structured learning and prediction in computer vision. Foundations and Trends® in Computer Graphics and Vision 6(3–4):185–365
Payer C, Stern D, Bischof H, Urschler M (2016) Regressing heatmaps for multiple landmark localization using cnns. In: MICCAI 2, pp 230–238
Roberts M, Cootes T, Adams J (2005) Vertebral shape: automatic measurement with dynamically sequenced active appearance models. In: Medical image computing and computer-assisted intervention-MICCAI 2005, pp 733–740
Ronneberger O, Fischer P, Brox T (2015) U-net: convolutional networks for biomedical image segmentation. In: International conference on medical image computing and computer-assisted intervention, pp. 234–241. Springer
Ross S, Munoz D, Hebert M, Bagnell JA (2011) Learning message-passing inference machines for structured prediction. In: 2011 IEEE conference on computer vision and pattern recognition (CVPR), pp 2737–2744. IEEE
Rubinstein R, Bruckstein AM, Elad M (2010) Dictionaries for sparse representation modeling. Proc IEEE 98(6):1045–1057
Schmidt S, Kappes J, Bergtholdt M, Pekar V, Dries S, Bystrov D, Schnörr C (2007) Spine detection and labeling using a parts-based graphical model. In: Information processing in medical imaging, pp 122–133. Springer
Schwarzenbach O, Berlemann U, Jost B, Visarius H, Arm E, Langlotz F, Nolte LP, Ozdoba C (1997) Accuracy of computer-assisted pedicle screw placement: an in vivo computed tomography analysis. Spine 22(4):452–458
Shiraishi J, Katsuragawa S, Ikezoe J, Matsumoto T, Kobayashi T, Komatsu KI., Matsui M, Fujita H, Kodera Y, Doi K (2000) Development of a digital image database for chest radiographs with and without a lung nodule: receiver operating characteristic analysis of radiologists’ detection of pulmonary nodules. Am J Roentgenol 174(1):71–74
Sun H, Zhen X, Bailey C, Rasoulinejad P, Yin Y, Li S (2017) Direct estimation of spinal cobb angles by structured multi-output regression. In: International conference on information processing in medical imaging, pp 529–540. Springer
Suzani A, Seitel A, Liu Y, Fels S, Rohling RN, Abolmaesumi P (2015) Fast automatic vertebrae detection and localization in pathological ct scans-a deep learning approach. In: International conference on medical image computing and computer-assisted intervention, pp 678–686. Springer
Tomazevic D, Likar B, Slivnik T, Pernus F (2003) 3-d/2-d registration of ct and mr to x-ray images. IEEE Trans Med Imaging 22(11):1407–1416
Wainwright MJ, Jordan MI et al (2008) Graphical models, exponential families, and variational inference. Foundations and Trends® in Machine Learning 1(1–2):1–305
Wright J, Ma Y, Mairal J, Sapiro G, Huang TS, Yan S (2010) Sparse representation for computer vision and pattern recognition. Proc IEEE 98(6):1031–1044
Xie S, Tu Z (2015) Holistically-nested edge detection. In: Proceedings of the IEEE international conference on computer vision, pp 1395–1403
Yang D, Xiong T, Xu D, Huang Q, Liu D, Zhou SK, Xu Z, Park J, Chen M, Tran TD et al (2017) Automatic vertebra labeling in large-scale 3d ct using deep image-to-image network with message passing and sparsity regularization. In: International conference on information processing in medical imaging, pp 633–644. Springer
Yang W, Ouyang W, Li H, Wang X (2016) End-to-end learning of deformable mixture of parts and deep convolutional neural networks for human pose estimation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 3073–3082
Yao J, Burns JE, Forsberg D, Seitel A, Rasoulian A, Abolmaesumi P, Hammernik K, Urschler M, Ibragimov B, Korez R et al (2016) A multi-center milestone study of clinical vertebral ct segmentation. Comput Med Imaging Graph 49:16–28
Acknowledgements
We thank Dr. David Liu, Dr. Kevin Zhou, and Dr. Mingqing Chen who provided insight and expertise that greatly assisted this research. We acknowledge the data which has been provided by the Department of Radiology at University of Washington.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Yang, D., Xiong, T., Xu, D. (2019). Automatic Vertebra Labeling in Large-Scale Medical Images Using Deep Image-to-Image Network with Message Passing and Sparsity Regularization. In: Lu, L., Wang, X., Carneiro, G., Yang, L. (eds) Deep Learning and Convolutional Neural Networks for Medical Imaging and Clinical Informatics. Advances in Computer Vision and Pattern Recognition. Springer, Cham. https://doi.org/10.1007/978-3-030-13969-8_9
Download citation
DOI: https://doi.org/10.1007/978-3-030-13969-8_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-13968-1
Online ISBN: 978-3-030-13969-8
eBook Packages: Computer ScienceComputer Science (R0)