Yunyan Yan
ABSTRACT
Bone age assessment (BAA) is vital in pediatric clinical diagnosis. Existing deep learning methods predict bone age based on Regions of Interest (RoIs) detection or segmentation of hand radiograph, which requires expensive annotations. Limitations of radiographic technique on imaging and cost hinder their clinical application as well. Compared to X-ray images, ultrasonic images are rather clean, cheap and flexible, but the deep learning research on ultrasonic BAA is still a white space. For this purpose, we propose a weakly supervised interpretable framework entitled USB-Net, utilizing ultrasonic pelvis images and only image-level age annotations. USB-Net consists of automatic anatomical RoI detection stage and age assessment stage. In the detection stage, USB-Net locates the discriminative anatomical RoIs of pelvis through attention heatmap without any extra RoI supervision. In the assessment stage, the cropped anatomical RoI patch is fed as fine-grained input to estimate age. In addition, we provide the first ultrasonic BAA dataset composed of 1644 ultrasonic hip joint images with image-level labels of age and gender. The experimental results verify that our model keeps consistent attention with human knowledge and achieves 16.24 days mean absolute error (MAE) on USBAA dataset.
Supplemental Material
- Byeonguk Bae, Jaewon Lee, Seo Taek Kong, Jinkyeong Sung, and Kyu-Hwan Jung. 2020. Manifold Ordinal-Mixup for Ordered Classes in TW3-Based Bone Age Assessment. In International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 662--670.Google Scholar
Digital Library
- ADMV Castriota-Scanderbeg and V De Micheli. 1995. Ultrasound of femoral head cartilage: a new method of assessing bone age. Skeletal radiology, Vol. 24, 3 (1995), 197--200.Google Scholar
- Jintai Chen, Bohan Yu, Biwen Lei, Ruiwei Feng, Danny Z Chen, and Jian Wu. 2020. Doctor Imitator: A Graph-Based Bone Age Assessment Framework Using Hand Radiographs. In International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 764--774.Google Scholar
- Maria Escobar, Cristina González, Felipe Torres, Laura Daza, Gustavo Triana, and Pablo Arbeláez. 2019. Hand pose estimation for pediatric bone age assessment. In International conference on medical image computing and computer-assisted intervention. Springer, 531--539.Google ScholarDigital Library
- Jianlong Fu, Heliang Zheng, and Tao Mei. 2017. Look closer to see better: Recurrent attention convolutional neural network for fine-grained image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 4438--4446.Google ScholarCross Ref
- Ping Gong, Zihao Yin, Yizhou Wang, and Yizhou Yu. 2020. Towards Robust Bone Age Assessment: Rethinking Label Noise and Ambiguity. In International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 621--630.Google Scholar
- William Walter Greulich and Sarah Idell Pyle. 1959. Radiographic atlas of skeletal development of the hand and wrist .Stanford university press.Google Scholar
- Kaiming He, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition. 770--778.Google ScholarCross Ref
- Jie Hu, Li Shen, and Gang Sun. 2018. Squeeze-and-Excitation Networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) .Google Scholar
- Gao Huang, Zhuang Liu, Laurens Van Der Maaten, and Kilian Q Weinberger. 2017. Densely connected convolutional networks. In Proceedings of the IEEE conference on computer vision and pattern recognition. 4700--4708.Google ScholarCross Ref
- Vladimir I Iglovikov, Alexander Rakhlin, Alexandr A Kalinin, and Alexey A Shvets. 2018. Paediatric bone age assessment using deep convolutional neural networks. In Deep learning in medical image analysis and multimodal learning for clinical decision support. Springer, 300--308.Google Scholar
- Khalid M Khan, Bradley S Miller, Eric Hoggard, Arif Somani, and Kyriakie Sarafoglou. 2009. Application of ultrasound for bone age estimation in clinical practice. The Journal of pediatrics, Vol. 154, 2 (2009), 243--247.Google ScholarCross Ref
- David B Larson, Matthew C Chen, Matthew P Lungren, Safwan S Halabi, Nicholas V Stence, and Curtis P Langlotz. 2018. Performance of a deep-learning neural network model in assessing skeletal maturity on pediatric hand radiographs. Radiology, Vol. 287, 1 (2018), 313--322.Google ScholarCross Ref
- Chuanbin Liu, Hongtao Xie, Yunyan Yan, Zhendong Mao, and Yongdong Zhang. 2020 b. Learning Rich Attention for Pediatric Bone Age Assessment. In International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 232--242.Google Scholar
- Chuanbin Liu, Hongtao Xie, Sicheng Zhang, Zhendong Mao, Jun Sun, and Yongdong Zhang. 2020 c. Misshapen Pelvis Landmark Detection With Local-Global Feature Learning for Diagnosing Developmental Dysplasia of the Hip. IEEE Transactions on Medical Imaging, Vol. 39, 12 (2020), 3944--3954. https://doi.org/10.1109/TMI.2020.3008382Google ScholarCross Ref
- Chuanbin Liu, Hongtao Xie, and Yongdong Zhang. 2020 a. Self-supervised Attention Mechanism for Pediatric Bone Age Assessment with Efficient Weak Annotation. IEEE Transactions on Medical Imaging (2020).Google Scholar
- Simukayi Mutasa, Peter D Chang, Carrie Ruzal-Shapiro, and Rama Ayyala. 2018. MABAL: a novel deep-learning architecture for machine-assisted bone age labeling. Journal of digital imaging, Vol. 31, 4 (2018), 513--519.Google ScholarCross Ref
- Xuhua Ren, Tingting Li, Xiujun Yang, Shuai Wang, Sahar Ahmad, Lei Xiang, Shaun Richard Stone, Lihong Li, Yiqiang Zhan, Dinggang Shen, et al. 2018. Regression convolutional neural network for automated pediatric bone age assessment from hand radiograph. IEEE journal of biomedical and health informatics, Vol. 23, 5 (2018), 2030--2038.Google Scholar
- Sung Joon Son, Youngmin Song, Namgi Kim, Younghae Do, Nojun Kwak, Mu Sook Lee, and Byoung-Dai Lee. 2019. TW3-based fully automated bone age assessment system using deep neural networks. IEEE Access, Vol. 7 (2019), 33346--33358.Google ScholarCross Ref
- Concetto Spampinato, Simone Palazzo, Daniela Giordano, Marco Aldinucci, and Rosalia Leonardi. 2017. Deep learning for automated skeletal bone age assessment in X-ray images. Medical image analysis, Vol. 36 (2017), 41--51.Google Scholar
- Christian Szegedy, Vincent Vanhoucke, Sergey Ioffe, Jon Shlens, and Zbigniew Wojna. 2016. Rethinking the inception architecture for computer vision. In Proceedings of the IEEE conference on computer vision and pattern recognition. 2818--2826.Google ScholarCross Ref
- James Mourilyan Tanner, RH Whitehouse, N Cameron, WA Marshall, MJR Healy, H Goldstein, et al. 2001. Assessment of skeletal maturity and prediction of adult height (TW2 method) .WB Saunders London.Google Scholar
- Dong Wang, Kexin Zhang, Jia Ding, and Liwei Wang. 2020. Improve bone age assessment by learning from anatomical local regions. In International Conference on Medical Image Computing and Computer-Assisted Intervention. Springer, 631--640.Google ScholarDigital Library
- Sanghyun Woo, Jongchan Park, Joon-Young Lee, and In So Kweon. 2018. CBAM: Convolutional Block Attention Module. In Proceedings of the European Conference on Computer Vision (ECCV) .Google ScholarDigital Library
- Eric Wu, Bin Kong, Xin Wang, Junjie Bai, Yi Lu, Feng Gao, Shaoting Zhang, Kunlin Cao, Qi Song, Siwei Lyu, et al. 2019. Residual attention based network for hand bone age assessment. In 2019 IEEE 16th International Symposium on Biomedical Imaging (ISBI 2019). IEEE, 1158--1161.Google ScholarCross Ref
- Hongtao Xie, Dongbao Yang, Nannan Sun, Zhineng Chen, and Yongdong Zhang. 2019. Automated pulmonary nodule detection in CT images using deep convolutional neural networks. Pattern Recognition, Vol. 85 (2019), 109--119. https://doi.org/10.1016/j.patcog.2018.07.031Google ScholarCross Ref
- Jingyuan Xu, Hongtao Xie, Chuanbin Liu, Fang Yang, Sicheng Zhang, Xun Chen, and Yongdong Zhang. 2021. Hip Landmark Detection With Dependency Mining in Ultrasound Image. IEEE Transactions on Medical Imaging, Vol. 40, 12 (2021), 3762--3774. https://doi.org/10.1109/TMI.2021.3097355Google ScholarCross Ref
- Heliang Zheng, Jianlong Fu, Tao Mei, and Jiebo Luo. 2017. Learning multi-attention convolutional neural network for fine-grained image recognition. In Proceedings of the IEEE international conference on computer vision. 5209--5217.Google ScholarCross Ref
Index Terms
- Weakly Supervised Pediatric Bone Age Assessment Using Ultrasonic Images via Automatic Anatomical RoI Detection
Recommendations
Improve Bone Age Assessment by Learning from Anatomical Local Regions
Medical Image Computing and Computer Assisted Intervention – MICCAI 2020AbstractSkeletal bone age assessment (BAA), as an essential imaging examination, aims at evaluating the biological and structural maturation of human bones. In the clinical practice, Tanner and Whitehouse (TW2) method is a widely-used method for ...
Weakly Supervised Localisation for Fetal Ultrasound Images
Deep Learning in Medical Image Analysis and Multimodal Learning for Clinical Decision SupportAbstractThis paper addresses the task of detecting and localising fetal anatomical regions in 2D ultrasound images, where only image-level labels are present at training, i.e. without any localisation or segmentation information. We examine the use of ...
Feature description with SIFT, SURF, BRIEF, BRISK, or FREAK? A general question answered for bone age assessment
Solving problems in medical image processing is either generic (being applicable to many problems) or specific (optimized for a certain task). For example, bone age assessment (BAA) on hand radiographs is a frequent but cumbersome task for radiologists. ...
Comments