Elsevier

Neurocomputing

Volume 500, 21 August 2022, Pages 750-760
Neurocomputing

Anatomical prior based vertebra modelling for reappearance of human spines

https://doi.org/10.1016/j.neucom.2022.05.033Get rights and content

Abstract

Scoliosis is a disease caused by deformation of the spine, and precise imaging of human spine tissues is crucial and instructive for the subsequent treatment. Currently, radiographs are clinically used to monitor disease progress. However, radiation exposure is a safety concern and is undesirable for regular monitoring during rehabilitation intervention. In this study, we developed a novel technique to generate the 3D structure of human spine from a tracked freehand ultrasound scanning. First of all, we designed an approach for detection and location of every vertebra in the spine by firstly detecting the vertebral landmarks from the ultrasound B-scan sequence, then computing the size, location and posture of every vertebra and finally using a method of interpolation to form the whole spine according to the prior knowledge of vertebral anatomical structure. Deep learning based object detection methods were used to find the landmarks of the vertebrae, and those landmarks were then clustered to model every vertebra (i.e. computing its location, posture and real size). Due to the situations where the vertebra is not clearly seen, we therefore used the technique of interpolation to estimate the location of those missing vertebrae with respect to the anatomical prior. In addition, we reconnected all vertebral models and reappeared the whole spine in 3D space. With that, the Cobb angle could be easily measured. The phantom and in vivo experiments were also conducted to further verify the precision of the proposed approaches. The results have demonstrated its feasibility in the clinical practice of adolescent idiopathic scoliosis patients (R2 = 0.9780,p < 0.001), which is beneficial for the regular quantitative monitoring of disease progression.

Introduction

At present, the main method for clinical examination of scoliosis is to take the X-ray image in the coronal or sagittal planes [1]. However, radiation exposure is a safety concern and is undesirable for regular monitoring during rehabilitation intervention [2], [3]. In addition, it is difficult to observe the 3D structure of scoliosis by radiographs. As an alternative, magnetic resonance imaging (MRI) could be used for scoliosis imaging [4]. However, it is more expensive and time-consuming. Hence, it is unsuitable for those clinical cases that require frequent examinations during rehabilitation treatment.

Compared with radiography and MRI, ultrasound (US) imaging has the advantages of no radiation, low cost, and easy availability. Therefore, US imaging for spinal imaging is gaining popularity. Two-dimensional (2D) US imaging of the spine is a common method of preoperative guidance and interventional treatment [5], [6], [7], [8], [9]. However, 2D US imaging cannot guide the scoliosis examination due to the limited visual field. Therefore, 3D US for spine imaging has become a hot spot in diagnosis of scoliosis. Studies have confirmed that 3D US has great potential in diagnosis of scoliosis. Researchers assessed the curvature of the spine by identifying landmarks of vertebrae, ie. transverse process or spinous process. However, the conventional 3D US techniques for spine imaging would lead to some challenges. The common weakness of the above 3D US-based methods was that the reconstruction of volume data is time-consuming and results in poor visual effect due to the low resolution of US images, limiting its clinical applications. In view of the shortcomings of existing 3D US-based spine imaging methods, we propose a novel imaging method to generate the 3D structure of human spine from a tracked freehand US scanning in this paper. Specifically, we designed an approach for detection and location of every vertebra in spine by firstly detecting the vertebral landmarks from the raw B-scan sequence, then computing the location and posture of every vertebra and finally using a method of vertebra modelling to form the whole spine according to the prior knowledge of vertebral anatomical structure. The proposed method can minimize some of the defects in currently used US imaging methods for the spine, making the US-based imaging method an alternative to radiographs in the monitoring of scoliosis in clinical practice.

The overall contribution of this study is that we propose a novel approach to reappearing the 3D bone structure of human spine using only conventional US imaging data. In particular, to the best of our knowledge, it is the first time to investigate how to geometrically model the human vertebrae in 3D space within the domain of medical US means. Another key contribution is that we are the first to reconstruct and visualize the human spine using virtual computer models of vertebra, providing much improved visual effect for subsequent diagnosis and treatment decisions in comparison with conventional 3D US imaging. Without the need to reconstruct 3D volume from the raw US B-scans, the computation time could also be greatly reduced.

This paper is organized as follows. In section 3, we introduced the overall freehand US system and the detailed methods for landmarks positioning and spine reappearance. Phantom experiments and in vivo experiments were then conducted to demonstrate the performance of the proposed imaging method in section 4. Finally, section 5 discusses the results of the proposed method. Section 6 concludes the findings of this work.

Section snippets

3D US imaging for spine

Since the field of view of 2D US was limited by the probe, earlier researchers took advantage of 3D US to record volume data and evaluate the Cobb angle of the patients with scoliosis. Chen et al. [10] used the TomoScan Focus Phased Array Ultrasound system (Olympus NDT Inc., Canada) with a US transducer to acquire US data in 3D space. Laminae were regarded as the landmarks to assess the scoliosis. The result showed that there was a potential to measure the Cobb angle by using laminae as

Methods

In this section, the proposed 3D imaging framework for human spines is introduced in detail. First of all, a series of B-scan images as well as their positional data were firstly acquired by our freehand US system. Then, we proposed a novel approach to modelling the vertebrae using the collected US image data. In modelling a vertebra, a deep learning based object detection method was firstly used to recognize its vertebral landmarks, e.g., spinous processes and transverse processes. In

Experiments

We conducted phantom experiments and in vivo experiments following the principles of the Declaration of Helsinki. To support the experiments, we trained the deep learning model on high performance computer with GPU (NVIDIA TITAN RTX, memory 24 GB). There were 1162 body US images provided by the First Affiliated Hospital of Sun Yat-sen University and 300 phantom US images used for training to detect the vertebral landmarks. In the phantom experiments, we used a PVC spine phantom with a length of

Discussion

In this paper, we designed a novel approach to reappearing the 3D spine with vertebral models. First of all, object detection and clustering were applied to predict the spatial location of vertebral landmarks. Linear interpolation was then used to estimate the location of the missing landmarks. We modeled the vertebrae (e.i. calculated the position, the attitude and the size of vertebrae) based on the location of the vertebral landmarks to form the whole 3D spine. Clustering experiments have

Conclusions

In this paper, we designed a novel technique to generate the 3-D structure of human spine from a tracked freehand US scanning. In particular, we first detected vertebral landmarks using tiny-YOLOv3 from the B-scans sequence, and then the K-means cluster was used to determine the spatial location of landmarks. Linear interpolation was adopted to estimate the missing or redundant vertebral landmarks. Finally, we modeled the vertebrae based on the spatial position of the vertebral landmarks to

CRediT authorship contribution statement

Qinghua Huang: Investigation, Conceptualization, Supervision, Methodology, Writing – review & editing, Project administration, Funding acquisition. Hao Luo: Methodology, Software, Validation, Writing – original draft. Cui Yang: Conceptualization, Writing – review & editing, Supervision. Jianyi Li: Validation, Writing – review & editing. Qifeng Deng: Software, Validation, Data curation. Peng Liu: . Maoqing Fu: . Le Li: Conceptualization, Writing – review & editing, Funding acquisition. Xuelong

Declaration of Competing Interest

The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.

Acknowledgements

This work was supported in part by National Natural Science Foundation of China (Grant nos. 62071382, 32071316, 82030047, 31771330 and 81902176), National Key R & D Program of China (Grant no. 2017YFC0110602), the Innovation Capability Support Program of Shaanxi (Program No. 2021TD-57), the Key Research and Development Project of Shaanxi Province (Grant no. 2022SF-117) , Natural Science Foundation of Guangdong Province (Grant no. 2022A1515011604) and Science and Technology Program of Guangzhou

Qinghua Huang received the B.E. and M.E. degrees in automatic control and pattern recognition from the University of Science and Technology of China, Hefei, China, in 1999 and 2002, respectively, and the Ph.D. degree in biomedical engineering from The Hong Kong Polytechnic University, Hong Kong, in 2007. In 2008, he joined the School of Electronic and Information Engineering, South China University of Technology, Guangzhou, China, as an Associate Professor, and was promoted to a Full Professor

References (33)

  • A.S. Ahmed et al.

    ultrasound diagnosis and therapeutic intervention in the spine

    J. Spine Surg.

    (2018)
  • A. Alaqeel et al.

    the utility of ultrasound for surgical spinal decompression

    Med. Ultrason.

    (2015)
  • M. Chi et al.

    ultrasound for lumbar spinal procedures

    Phys. Med. Reh. Clin. N.

    (2018)
  • A. Rasoulian et al.

    Ultrasound-guided spinal injections: a feasibility study of a guidance system

    Int. J. Comput. Ass. Rad.

    (2015)
  • W. Chen et al.

    Using ultrasound imaging to identify landmarks in vertebra models to assess spinal deformity

  • C.W.J. Cheung, S.Y. Law, Y. Zheng. Development of 3-D ultrasound system for assessment of adolescent idiopathic...
  • Cited by (48)

    • Fully automatic spinal scanning and measurement based on multi-source vision information

      2024, Measurement: Journal of the International Measurement Confederation
    View all citing articles on Scopus

    Qinghua Huang received the B.E. and M.E. degrees in automatic control and pattern recognition from the University of Science and Technology of China, Hefei, China, in 1999 and 2002, respectively, and the Ph.D. degree in biomedical engineering from The Hong Kong Polytechnic University, Hong Kong, in 2007. In 2008, he joined the School of Electronic and Information Engineering, South China University of Technology, Guangzhou, China, as an Associate Professor, and was promoted to a Full Professor in 2013. He is currently a Distinguished Professor with Northwestern Polytechnical University, Xi’an, China. His research interests include ultrasonic imaging, medical image/data analysis, and artificial intelligence based applications.

    Hao Luo received the B. E. degree from School of Mechanical Engineering, Northwestern Poly-technical University, Xi’an, China, in 2019. He is currently pursuing the master’s degree with the same school. His research interest is intelligent robotic system for medical ultrasound.

    Cui Yang received the B.S. and Ph.D. degrees in communication and information system from the South China University of Technology, Guangzhou, China, in 2005 and 2010, respectively. She is currently an Associate Professor with the School of Electronic and Information, South China University of Technology. Her current research interests include signal processing, ultrasound imaging, and robotic ultrasound.

    Jianyi Li received the Ph.D. degree in human anatomy from Southern Medical University, Guangzhou, China, in 2007. He is currently a Professor with Academy of Orthopedics of Guangdong Province, The Third Affiliated Hospital, Southern Medical University, the Professor of Guangdong Provincial Key Laboratory of Medical Biomechanics, Guangdong Engineering Research Center for Translation of Medical 3D Printing Application, Department of Anatomy, School of Basic Medical Sciences, Southern Medical University, and the Professor of the Department of orthopedics, The Seventh Affiliated Hospital, Southern Medical University. His research interests include digital medicine, 3D printing, and orthopedic biomechanics.

    Qifeng Deng received the B.E. degree from the College of Electrical and Information, Jinan University, Zhuhai, China, in 2017 and the master’s degree with the School of Electronic and Information Engineering, South China University of Technology, Guangzhou, China, in 2020. His research interest is medical image processing.

    Peng Liu received the B.M degree in Clinical Medicine from Zhongshan Medical University in 1994, M.M and M.D degrees in Physical Medicine and Rehabilitation from Sun Yat-sen University, Guangzhou, China, in 2002 and 2011, respectively. She worked as Attending doctor in the First affiliated hospital, Sun Yat-sen University from 2003, and then associated chief physician from 2008. She is currently a chief physician with the First affiliated hospital, Sun Yat-sen University, Guangzhou, China. Her research interests include neurorehabilitation, AIS, and clinical research method in physical medicine and rehabilitation.

    Maoqing Fu received the Ph.D. degree in human anatomy from Southern Medical University, Guangzhou, China, in 2017. He is currently a full-time Spine Surgeon with The Seventh Affifiliated Hospital, Southern Medical University. His research interests include minimally invasive spinal technology, treatment of chronic spinal pain, digital medicine, and orthopedic biomechanics.

    Le Li received the B.E. and M.E. degrees in Biomedical Engineering from the Xi'an Jiaotong University, Xi'an, China, in 2000 and 2003, respectively, and the Ph.D. degree in biomedical engineering from The Hong Kong Polytechnic University, Hong Kong, in 2007. Then he worked as a postdoctoral fellow in the same university. He joined the First Affiliated hospital, Sun Yat-sen University, Guangzhou, China, as an Associate Professor, and was promoted to a Full Professor in 2018. He is currently a Professor with Northwestern Polytechnical University, Xi’an, China. His research interests include ultrasonic imaging, biosignal processing, and rehabilitation engineering and its applications.

    Xuelong Li is a Full Professor with the School of Artificial Intelligence, Optics and Electronics (iOPEN), Northwestern Polytechnical University, Xi’an, China.

    View full text