Anatomical prior based vertebra modelling for reappearance of human spines
Introduction
At present, the main method for clinical examination of scoliosis is to take the X-ray image in the coronal or sagittal planes [1]. However, radiation exposure is a safety concern and is undesirable for regular monitoring during rehabilitation intervention [2], [3]. In addition, it is difficult to observe the 3D structure of scoliosis by radiographs. As an alternative, magnetic resonance imaging (MRI) could be used for scoliosis imaging [4]. However, it is more expensive and time-consuming. Hence, it is unsuitable for those clinical cases that require frequent examinations during rehabilitation treatment.
Compared with radiography and MRI, ultrasound (US) imaging has the advantages of no radiation, low cost, and easy availability. Therefore, US imaging for spinal imaging is gaining popularity. Two-dimensional (2D) US imaging of the spine is a common method of preoperative guidance and interventional treatment [5], [6], [7], [8], [9]. However, 2D US imaging cannot guide the scoliosis examination due to the limited visual field. Therefore, 3D US for spine imaging has become a hot spot in diagnosis of scoliosis. Studies have confirmed that 3D US has great potential in diagnosis of scoliosis. Researchers assessed the curvature of the spine by identifying landmarks of vertebrae, ie. transverse process or spinous process. However, the conventional 3D US techniques for spine imaging would lead to some challenges. The common weakness of the above 3D US-based methods was that the reconstruction of volume data is time-consuming and results in poor visual effect due to the low resolution of US images, limiting its clinical applications. In view of the shortcomings of existing 3D US-based spine imaging methods, we propose a novel imaging method to generate the 3D structure of human spine from a tracked freehand US scanning in this paper. Specifically, we designed an approach for detection and location of every vertebra in spine by firstly detecting the vertebral landmarks from the raw B-scan sequence, then computing the location and posture of every vertebra and finally using a method of vertebra modelling to form the whole spine according to the prior knowledge of vertebral anatomical structure. The proposed method can minimize some of the defects in currently used US imaging methods for the spine, making the US-based imaging method an alternative to radiographs in the monitoring of scoliosis in clinical practice.
The overall contribution of this study is that we propose a novel approach to reappearing the 3D bone structure of human spine using only conventional US imaging data. In particular, to the best of our knowledge, it is the first time to investigate how to geometrically model the human vertebrae in 3D space within the domain of medical US means. Another key contribution is that we are the first to reconstruct and visualize the human spine using virtual computer models of vertebra, providing much improved visual effect for subsequent diagnosis and treatment decisions in comparison with conventional 3D US imaging. Without the need to reconstruct 3D volume from the raw US B-scans, the computation time could also be greatly reduced.
This paper is organized as follows. In section 3, we introduced the overall freehand US system and the detailed methods for landmarks positioning and spine reappearance. Phantom experiments and in vivo experiments were then conducted to demonstrate the performance of the proposed imaging method in section 4. Finally, section 5 discusses the results of the proposed method. Section 6 concludes the findings of this work.
Section snippets
3D US imaging for spine
Since the field of view of 2D US was limited by the probe, earlier researchers took advantage of 3D US to record volume data and evaluate the Cobb angle of the patients with scoliosis. Chen et al. [10] used the TomoScan Focus Phased Array Ultrasound system (Olympus NDT Inc., Canada) with a US transducer to acquire US data in 3D space. Laminae were regarded as the landmarks to assess the scoliosis. The result showed that there was a potential to measure the Cobb angle by using laminae as
Methods
In this section, the proposed 3D imaging framework for human spines is introduced in detail. First of all, a series of B-scan images as well as their positional data were firstly acquired by our freehand US system. Then, we proposed a novel approach to modelling the vertebrae using the collected US image data. In modelling a vertebra, a deep learning based object detection method was firstly used to recognize its vertebral landmarks, e.g., spinous processes and transverse processes. In
Experiments
We conducted phantom experiments and in vivo experiments following the principles of the Declaration of Helsinki. To support the experiments, we trained the deep learning model on high performance computer with GPU (NVIDIA TITAN RTX, memory 24 GB). There were 1162 body US images provided by the First Affiliated Hospital of Sun Yat-sen University and 300 phantom US images used for training to detect the vertebral landmarks. In the phantom experiments, we used a PVC spine phantom with a length of
Discussion
In this paper, we designed a novel approach to reappearing the 3D spine with vertebral models. First of all, object detection and clustering were applied to predict the spatial location of vertebral landmarks. Linear interpolation was then used to estimate the location of the missing landmarks. We modeled the vertebrae (e.i. calculated the position, the attitude and the size of vertebrae) based on the location of the vertebral landmarks to form the whole 3D spine. Clustering experiments have
Conclusions
In this paper, we designed a novel technique to generate the 3-D structure of human spine from a tracked freehand US scanning. In particular, we first detected vertebral landmarks using tiny-YOLOv3 from the B-scans sequence, and then the K-means cluster was used to determine the spatial location of landmarks. Linear interpolation was adopted to estimate the missing or redundant vertebral landmarks. Finally, we modeled the vertebrae based on the spatial position of the vertebral landmarks to
CRediT authorship contribution statement
Qinghua Huang: Investigation, Conceptualization, Supervision, Methodology, Writing – review & editing, Project administration, Funding acquisition. Hao Luo: Methodology, Software, Validation, Writing – original draft. Cui Yang: Conceptualization, Writing – review & editing, Supervision. Jianyi Li: Validation, Writing – review & editing. Qifeng Deng: Software, Validation, Data curation. Peng Liu: . Maoqing Fu: . Le Li: Conceptualization, Writing – review & editing, Funding acquisition. Xuelong
Declaration of Competing Interest
The authors declare that they have no known competing financial interests or personal relationships that could have appeared to influence the work reported in this paper.
Acknowledgements
This work was supported in part by National Natural Science Foundation of China (Grant nos. 62071382, 32071316, 82030047, 31771330 and 81902176), National Key R & D Program of China (Grant no. 2017YFC0110602), the Innovation Capability Support Program of Shaanxi (Program No. 2021TD-57), the Key Research and Development Project of Shaanxi Province (Grant no. 2022SF-117) , Natural Science Foundation of Guangdong Province (Grant no. 2022A1515011604) and Science and Technology Program of Guangzhou
Qinghua Huang received the B.E. and M.E. degrees in automatic control and pattern recognition from the University of Science and Technology of China, Hefei, China, in 1999 and 2002, respectively, and the Ph.D. degree in biomedical engineering from The Hong Kong Polytechnic University, Hong Kong, in 2007. In 2008, he joined the School of Electronic and Information Engineering, South China University of Technology, Guangzhou, China, as an Associate Professor, and was promoted to a Full Professor
References (33)
- et al.
Ultrasound image de-speckling by a hybrid deep network with transferred filtering and structural prior
Neurocomputing.
(2020) - et al.
Development of a portable 3D ultrasound imaging system for musculoskeletal tissues
Ultrasonics
(2005) - et al.
Validity study of vertebral rotation measurement using 3-D ultrasound in adolescents idiopathic scoliosis
Ultrasound Med. Biol.
(2016) - et al.
Freehand three-dimensional ultrasound system for assessment of scoliosis
J. Orthopaedic Transl.
(2015) - et al.
Automated vertebral landmarks and spinal curvature estimation using non-directional part affinity fields
Neurocomputing
(2021) - et al.
An improved tiny-yolov3 pedestrian detection algorithm
Optik
(2019) - et al.
imaging in the diagnosis and monitoring of children with idiopathic scoliosis
Open Orthop. J.
(2017) - et al.
Reducing the lifetime risk of cancer from spinal radiographs among people with adolescent idiopathic scoliosis
Spine
(1996) - et al.
Adolescent idiopathic scoliosis: review and current concepts
Am. Family Physician
(2001) - et al.
Reweighted minimization model for MR image reconstruction with split Bregman method
Sci. China Inf. Sci.
(2012)
ultrasound diagnosis and therapeutic intervention in the spine
J. Spine Surg.
the utility of ultrasound for surgical spinal decompression
Med. Ultrason.
ultrasound for lumbar spinal procedures
Phys. Med. Reh. Clin. N.
Ultrasound-guided spinal injections: a feasibility study of a guidance system
Int. J. Comput. Ass. Rad.
Using ultrasound imaging to identify landmarks in vertebra models to assess spinal deformity
Cited by (48)
Automatic Assessment of Ultrasound Curvature Angle for Scoliosis Detection Using 3-D Ultrasound Volume Projection Imaging
2024, Ultrasound in Medicine and BiologyDeep learning-powered biomedical photoacoustic imaging
2024, NeurocomputingLM-Net: A light-weight and multi-scale network for medical image segmentation
2024, Computers in Biology and MedicineFully automatic spinal scanning and measurement based on multi-source vision information
2024, Measurement: Journal of the International Measurement ConfederationFully automated thyroid ultrasound screening utilizing multi-modality image and anatomical prior
2024, Biomedical Signal Processing and Control
Qinghua Huang received the B.E. and M.E. degrees in automatic control and pattern recognition from the University of Science and Technology of China, Hefei, China, in 1999 and 2002, respectively, and the Ph.D. degree in biomedical engineering from The Hong Kong Polytechnic University, Hong Kong, in 2007. In 2008, he joined the School of Electronic and Information Engineering, South China University of Technology, Guangzhou, China, as an Associate Professor, and was promoted to a Full Professor in 2013. He is currently a Distinguished Professor with Northwestern Polytechnical University, Xi’an, China. His research interests include ultrasonic imaging, medical image/data analysis, and artificial intelligence based applications.
Hao Luo received the B. E. degree from School of Mechanical Engineering, Northwestern Poly-technical University, Xi’an, China, in 2019. He is currently pursuing the master’s degree with the same school. His research interest is intelligent robotic system for medical ultrasound.
Cui Yang received the B.S. and Ph.D. degrees in communication and information system from the South China University of Technology, Guangzhou, China, in 2005 and 2010, respectively. She is currently an Associate Professor with the School of Electronic and Information, South China University of Technology. Her current research interests include signal processing, ultrasound imaging, and robotic ultrasound.
Jianyi Li received the Ph.D. degree in human anatomy from Southern Medical University, Guangzhou, China, in 2007. He is currently a Professor with Academy of Orthopedics of Guangdong Province, The Third Affiliated Hospital, Southern Medical University, the Professor of Guangdong Provincial Key Laboratory of Medical Biomechanics, Guangdong Engineering Research Center for Translation of Medical 3D Printing Application, Department of Anatomy, School of Basic Medical Sciences, Southern Medical University, and the Professor of the Department of orthopedics, The Seventh Affiliated Hospital, Southern Medical University. His research interests include digital medicine, 3D printing, and orthopedic biomechanics.
Qifeng Deng received the B.E. degree from the College of Electrical and Information, Jinan University, Zhuhai, China, in 2017 and the master’s degree with the School of Electronic and Information Engineering, South China University of Technology, Guangzhou, China, in 2020. His research interest is medical image processing.
Peng Liu received the B.M degree in Clinical Medicine from Zhongshan Medical University in 1994, M.M and M.D degrees in Physical Medicine and Rehabilitation from Sun Yat-sen University, Guangzhou, China, in 2002 and 2011, respectively. She worked as Attending doctor in the First affiliated hospital, Sun Yat-sen University from 2003, and then associated chief physician from 2008. She is currently a chief physician with the First affiliated hospital, Sun Yat-sen University, Guangzhou, China. Her research interests include neurorehabilitation, AIS, and clinical research method in physical medicine and rehabilitation.
Maoqing Fu received the Ph.D. degree in human anatomy from Southern Medical University, Guangzhou, China, in 2017. He is currently a full-time Spine Surgeon with The Seventh Affifiliated Hospital, Southern Medical University. His research interests include minimally invasive spinal technology, treatment of chronic spinal pain, digital medicine, and orthopedic biomechanics.
Le Li received the B.E. and M.E. degrees in Biomedical Engineering from the Xi'an Jiaotong University, Xi'an, China, in 2000 and 2003, respectively, and the Ph.D. degree in biomedical engineering from The Hong Kong Polytechnic University, Hong Kong, in 2007. Then he worked as a postdoctoral fellow in the same university. He joined the First Affiliated hospital, Sun Yat-sen University, Guangzhou, China, as an Associate Professor, and was promoted to a Full Professor in 2018. He is currently a Professor with Northwestern Polytechnical University, Xi’an, China. His research interests include ultrasonic imaging, biosignal processing, and rehabilitation engineering and its applications.
Xuelong Li is a Full Professor with the School of Artificial Intelligence, Optics and Electronics (iOPEN), Northwestern Polytechnical University, Xi’an, China.