Skeletal bone age prediction based on a deep residual network with spatial transformer

doi:10.1016/j.cmpb.2020.105754

Computer Methods and Programs in Biomedicine

Volume 197, December 2020, 105754

https://doi.org/10.1016/j.cmpb.2020.105754 Get rights and content

Highlights

•
Clinicians predict bone age by manually reading X-ray hand bone images.
•
SVM automatic bone age assessment method has bone age prediction capability.
•
ST-ResNet network model is based on ResNet and spatial transformer.
•
ST-ResNet network model demonstrates better bone age prediction accuracy.
•
Deep learning can facilitate bone age detection more effectively.

Abstract

Objective

Bone age prediction can be performed by medical experts manually assessment of X-ray images of the hand bone. In practice, the workload is huge, resource consumption is large, measurement takes a long time, and it is easily influenced by human factors. As such, manual estimation of bone age takes a long time and the results fluctuate greatly depending on the proficiency of the radiologist.

Methods

The left-hand X-ray image data was identified and pre-processed. X-ray image analysis method using on deep neural network was used to automatically extract the key features of the left-hand joint bone age, and evaluation performance of the model was implemented.

Results

In this paper, the deep learning method can be used to obtain the X-ray bone image features, and the convolutional neural network is used to automatically assess the age of bone. The feature region extraction method based on deep learning can extract feature information with superior performance compared to the traditional image analysis technique. Based on the residual network (ResNet) model in the deep learning algorithm, the average absolute error of the age of bones detected by the bone age assessment model is 0.455 better than traditional methods and only end-to-end deep learning methods. When the learning rate is greater than 0.0005, the MAE of Inception Resnet v2 model is higher than most models. Accuracy of bone age prediction is as high as 97.6%.

Conclusion

In comparison with the traditional machine learning feature extraction technique, the convolutional neural network based on feature extraction has better performance in the bone age regression model, and further improves the accuracy of image-based age of bone assessment.

Introduction

In the medical field, human growth and development is mainly measured by the following types of 'ages', namely the chronological age and biological age. Among them, age of chronology is relatively simple, and is determined by the date of birth. The biological age mainly reflects the development of human beings, which is mainly determined by the age of teeth and bone age. Among them, the tooth age is the earliest used biological age indicator [1]. During the Roman Empire, eruption of second molar was used as the standard of service. Before the 19^th century, the age of the teeth was widely accepted by medical scientists. However, in 1846, the British doctor by the name of Petro proposed that the eruption of specific teeth as the criterion for biological age is extremely imprecise [2]. Therefore, in 1886, Angererr [[22]] first proposed that the biological age of adolescents could be determined by hand bones. Compared with the dental age, the bone age shows the growth and development of whole-body bone of test subject, is more suitable as a criterion for determining the biological age.

Skeletal bone age prediction has been commonly used in clinical medicine, preventive medicine, biology, sports science, forensic anthropology and other fields. In the pediatric clinical, through the analysis of skeletal bone age, combined with physical examination and laboratory tests, it is possible to detect the causes of growth and disease in children in a timely manner, and timely take effective interventions to obtain a good prognosis [3].

With continuous innovation and computer technology, the application of computer technology has penetrated into various fields and has had a tremendous impact on people's lives. In medicine, incorporation of computer vision technique in image recognition and image understanding, bone age assessment technology has also been rapidly developed and improved.

Although the method based on deep learning has made great achievements in bone age prediction [[29]], it still faces considerable challenges. Among them, previous work focused on improving the prediction accuracy, and then in the real scene, various reasons may lead to poor quality of X-ray images, but the existing work has ignored this point. In addition, compared with ordinary natural images, because medical image acquisition is more costly and marking requires professional radiologists, so the dataset dedicated to bone age prediction and with high-quality labels is very limited. As such, this poses a challenge to the training of neural networks [4].

Compared with traditional simple learning [[30], [31], [32], [33]], the difference between deep learning is that the former uses a multi-layer network structure to learn the characteristics of the data autonomously, while the latter mostly needs to manually extract feature information. The features extracted manually are often not accurate enough or is unable to represent the essence of things well, and it is difficult to improve the learning effect. Neural network is a successful application of deep learning in the field of images, and so we use deep learning framework to predict bone age.

Section snippets

Image scaling

Images with inconsistent resolution are training samples that cannot be used as a classification model, so you first need to scale the image to a uniform size, where the original image in the dataset is uniformly scaled to 300 × 300 pixels resolution. This is because after the image is scaled, the feature information of the image is not greatly lost, and the feature extraction of the scaled image is not greatly affected, and the huge computational amount of the feature extraction process can be

Image scaling

First, the image pyramid is used to complete the scaling of the image. The image pyramid is composed of a plurality of sample images of the same image arranged in a pyramid form from bottom to top in descending order of resolution. The two common types of image pyramids are divided into Gaussian pyramids and Laplacian pyramids. This section uses the Gaussian pyramid to downsample the image, i.e. to create (i+1)-th layer from the i-th layer of the pyramid. To obtain (i+1)-th sampled image, the

Comparison between deep learning and traditional methods

Bone age studies range from early percentile methods, counting methods, GP mapping methods to recent CHN methods, TW3 methods, etc. [[12],[13]]. Currently, GP mapping is the most widely used, and radiologists usually treat patients' left wrist X. The results are affected by the level and ability of the reader and the consistency is poor. Compared with the G-P map method commonly used in hospitals, automatic image analysis has always been the goal of computer vision and radiology research.

Early

Conclusion

Skeletal bone age prediction is a common technique method usually based on determination of bone development characteristics to obtain a numerical assessment of human development. It is widely used in the prediction of teenagers' physical development, the discovery and prevention of diseases, sports selection, etc. It has important social significance. At present, the work in our country is mainly carried out by medical experts to read the X-ray hand bone images manually, but the workload is

Declaration of Competing Interest

The authors declare that there is no conflict of interests.

References (33)

A Schmeling et al.
Age estimation
Forensic Sci. Int.
(2007)
C Spampinato et al.
Deep learning for automated skeletal bone age assessment in X-Ray images
Med. Image Anal.
(2017)
A Gertych et al.
Bone age assessment of children using a digital hand atlas[J]
Comput. Med. Imaging Graph.
(2007)
A Zhang et al.
Automatic bone age asssment for young children from ne wborn to 7-year-old using carpal bones
Comput. Med. Imaging Graph. Off. J. Comput. Med. Imaging Soc.
(2007)
Xu Chen et al.
Automatic feature extraction in X-ray image based on deep learning approach for determination of bone age
Future Generation Computer Systems
(2020)
Yu Lu et al.
Prediction of fetal weight at varying gestational age in the absence of ultrasound examination using ensemble learning
Artificial Intelligence in Medicine
(2020)
Ming Zhao et al.
A novel U-Net approach to segment the cardiac chamber in magnetic resonance images with ghost artifacts
Computer Methods and Programs in Biomedicine
(2020)
N Lynnerup et al.
Assessment of age at death by microscopy: unbiased quantification of secondary osteons in femoral cross sections
Forensic Sci. Int.
(2006)
S Fishman L
Radiographic evaluation of skeletal maturation. A clinically oriented method based on hand-wrist films
Angle Orthod .
(1982)
A Krizhevsky et al.
Image net classification with deep convolutional neural networks
Curran. Assoc. Inc.
(2012)

P Miller F et al.

Talamancan Montane Forests[J]

Alphascript Publ.

(2011)

L Vincent et al.

Watersheds in digital spaces: an efficient algorithm based on immersion simulations

IEEE Trans. Pattern Anal. Mach. Intell.

(1991)

K He et al.

Deep residual learning for image recognition

(2016)

Henrik H. Thodberg

An Automated method for determination of bone age

J. Clin. Endocrinol. Metab.

(2009)

W Hsieh C et al.

Bone age estimation based on phalanx information with fuzzy constrain of carpals

Med. Biol. Eng. Comput.

(2007)

G King D et al.

Reproducibility of bone ages when performed by radiology registrars: an audit of Tanner and Whitehouse II versus Greulich and Pyle methods

Br. J. Radiol.

(1994)

Cited by (22)

Artificial intelligence and the future of life sciences
2021, Drug Discovery Today
Citation Excerpt :
They explored a wide variety of state-of-the-art DL models and found that Transformers can achieve a better result against all other learning models and state-of-the-art methods.44 Skeletal bone age prediction based on a deep residual network with spatial transformer has also been performed.45 Transformers are being applied to Electronic Health Records (EHRs)46 to improve the accuracy of predicting future diagnoses to evaluate disease embedding, attention, and interpretability, and, thus, disease prediction.
Over the past few decades, the number of health and ‘omics-related data’ generated and stored has grown exponentially. Patient information can be collected in real time and explored using various artificial intelligence (AI) tools in clinical trials; mobile devices can also be used to improve aspects of both the diagnosis and treatment of diseases. In addition, AI can be used in the development of new drugs or for drug repurposing, in faster diagnosis and more efficient treatment for various diseases, as well as to identify data-driven hypotheses for scientists. In this review, we discuss how AI is starting to revolutionize the life sciences sector.
Development of an age estimation method for bones based on machine learning using post-mortem computed tomography images of bones
2021, Forensic Imaging
Citation Excerpt :
Several recent approaches have involved the automatic extraction of age-related features from different bones for the age estimation of subadults. Automatic feature extraction followed by machine learning (ML) for age estimation has been applied to X-ray images of the hand [34–40] and pelvis [41] and to MRI images of the hand [42] and knee [43]. Such approaches increase not only the objectivity of the estimation methods, but also their rapidity and accuracy.
Age estimation from bones plays a major role in the identification of skeletal remains. We present a novel age estimation method developed through the application of machine learning (ML) to post-mortem computed tomography (PMCT) images of bones.
This study used PMCT images of the vertebral body, ischial tuberosity, iliac crest, and femur, which were transformed into homologous models. Two-dimensional discrete wavelet transform (2D-DWT) was conducted to extract high-frequency components. Dimensionality reductions of the prepared data arrays were conducted with principal component analysis and partial least squares regression (PLS). The known ages and scores of the principal components were supplied to ridge regression, least absolute shrinkage and selection operator regression, and support vector regression with a linear kernel or a radial basis function kernel. A 10-fold double-looped cross-validation was conducted and estimation accuracies were verified with the mean absolute errors and correlation coefficients (r) between the actual and estimated ages.
Preprocessing with 2D-DWT and PLS obtained good results. Of the ML methods examined, support vector regression with radial basis function kernel achieved the highest accuracy, with an optimum mean absolute error and r of 7.92 (male vertebral body) and 0.837 (female ischial tuberosity), respectively. The method developed in this study could be used as a rapid, accurate, and objective tool for identifying both skeletal remains and non-skeletonized cadavers.
DFP-ResUNet:Convolutional Neural Network with a Dilated Convolutional Feature Pyramid for Multimodal Brain Tumor Segmentation
2021, Computer Methods and Programs in Biomedicine
Citation Excerpt :
Han et al. predicted the age of a skeleton using a deep residual network and a spatial transformer. Through a deep residual network based on feature extraction, the accuracy of image-based bone age assessment has been further improved [14]. To acquire the global feature information, Zhao et al. proposed PSPNet, which develops the function of global context information through context aggregation based on different regions [15].
Manual brain tumor segmentation by radiologists is time consuming and subjective. Therefore, fully automatic segmentation of different brain tumor subregions is essential to the treatment of patients. In this paper, we propose a neural network for automatically segmenting the enhancing tumor (ET), whole tumor (WT), and tumor core (TC) brain tumor subregions.
The network is based on a U-Net with encoding and decoding structure, a residual module, and a spatial dilated feature pyramid (DFP) module, namely, DFP-ResUNet. First, we propose using a spatial DFP module composed of multiple parallel dilated convolution layers to extract the multiscale image features. This spatial DFP structure improves the ability of the neural network to extract and utilize the multiscale image features. Then, we use the residual module to deepen the network structure. Further, we propose using a multiclass Dice loss function to suppress the impact of class imbalance on brain tumor segmentation. We carried out a large number of ablation experiments to verify the feasibility and superiority of our approach using the Multimodal Brain Tumor Segmentation (BraTS) challenge dataset.
The mean Dice score of different subregions was ET 0.8431, WT 0.897 and TC 0.9068 using the proposed method on the BraTS 2018 challenge validation set and 0.7985, 0.90281, 0.8453 on the BraTS 2019 challenge, respectively. Further, we got a high Sensitivity and Specificity and low Hausdorff distance.
Through the analysis of the experimental results, it can be seen that the proposed approach DFP-ResUNet has a great potential in segmenting different subregions of brain tumors and can be applied in clinical practice.
Advances of noninvasive imaging in bone age assessment
2024, Chinese Journal of Applied Clinical Pediatrics
Predicting Pedestrian Behavior at Zebra Crossings using Bottom-up Pose Estimation and Deep Learning
2024, International Journal of Intelligent Systems and Applications in Engineering
AlexNet-based deep convolutional neural network optimized with group teaching optimization algorithm (GTOA) for paediatric bone age assessment from hand X-ray images
2024, Imaging Science Journal

View all citing articles on Scopus

View full text

Skeletal bone age prediction based on a deep residual network with spatial transformer

Highlights

Abstract

Objective

Methods

Results

Conclusion

Introduction

Section snippets

Image scaling

Image scaling

Comparison between deep learning and traditional methods

Conclusion

Declaration of Competing Interest

Forensic Sci. Int.

Med. Image Anal.

Comput. Med. Imaging Graph.

Comput. Med. Imaging Graph. Off. J. Comput. Med. Imaging Soc.

Future Generation Computer Systems

Artificial Intelligence in Medicine

Computer Methods and Programs in Biomedicine

Assessment of age at death by microscopy: unbiased quantification of secondary osteons in femoral cross sections

Forensic Sci. Int.

Radiographic evaluation of skeletal maturation. A clinically oriented method based on hand-wrist films

Angle Orthod .

Image net classification with deep convolutional neural networks

Curran. Assoc. Inc.

Talamancan Montane Forests[J]

Alphascript Publ.

Watersheds in digital spaces: an efficient algorithm based on immersion simulations

IEEE Trans. Pattern Anal. Mach. Intell.

Deep residual learning for image recognition

An Automated method for determination of bone age

J. Clin. Endocrinol. Metab.

Bone age estimation based on phalanx information with fuzzy constrain of carpals

Med. Biol. Eng. Comput.

Reproducibility of bone ages when performed by radiology registrars: an audit of Tanner and Whitehouse II versus Greulich and Pyle methods

Br. J. Radiol.