Skip to main content
SpringerLink
Log in

A multi-scale framework based on jigsaw patches and focused label smoothing for bone age assessment

  • Original article
  • Published:
The Visual Computer Aims and scope Submit manuscript

Abstract

The clinical assessment of bone age is critical. Because it allows the study of the endocrine, genetic, and growth disorders in children's growth. However, clinical assessment of bone age is time-consuming and labor-intensive and is susceptible to observer error. Fully automated bone age assessment (BAA) systems can be a good solution to this problem, assisting or even replacing specialists in bone age assessment. Most of the existing fully automated BAA systems use more discriminative areas (carpal bones, finger bones, etc.) from the whole image as local information input. However, extracting multiple local regions is still complex and time-consuming. Selective extraction of regions is not objective enough and can lose some useful global information. In this paper, we propose a fully automated end-to-end BAA system that requires no additional annotation and no extraction of regions of interest. Specifically, we use a puzzle generator to generate skeletal puzzles containing information at different scales. By inputting smaller-scale skeletal puzzles, the network is forced to mining local fine-grained information first, then input larger-scale skeletal puzzles to obtain coarse-grained information, and finally learn the intact picture to obtain global information. BAA task is suboptimal as a general classification or regression task using single-valued labels due to the high similarity between hand images of similar ages. We propose a new label processing method called focused label smoothing for the BAA task and combine it with expectation regression to obtain a more robust age estimate. We perform adequate experiments on the public dataset from the 2017 Pediatric Bone Age Challenge organized by the Radiological Society of North America and we achieve great experimental performance compared to the method without manual annotation and competitive results with the method using additional manual annotation.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Carty, H.: Assessment of skeletal maturity and prediction of adult height (TW3 method) (2002)

  2. Albanese, A., Stanhope, R.: Investigation of delayed puberty. Clin. Endocrinol. 43, 105–110 (1995)

    Article  Google Scholar 

  3. Poznanski, A.K., Hernandez, R.J., Guire, K.E., Bereza, U.L., Garn, S.M.: Carpal length in children: a useful measurement in the diagnosis of rheumatoid arthritis and some congenital malformation syndromes. Radiology 129(3), 661–668 (1978)

    Article  Google Scholar 

  4. De Sanctis, V., Di Maio, S., Soliman, A.T., Raiola, G., Elalaily, R., Millimaggi, G.: Hand X-ray in pediatric endocrinology: skeletal age as- sessment and beyond. Indian J. Endocrinol. Metab. 18, 63–71 (2014)

    Article  Google Scholar 

  5. Garn, S.M.: Radiographic atlas of skeletal development of the hand and wrist. Am. J. Hum. Genet. 11, 282 (1959)

    Google Scholar 

  6. Tanner, J.M.: Assessment of Skeletal Maturity and Prediction of Adult Height (TW2 Method). Academic Press, Cambridge (1983)

    Google Scholar 

  7. Goldstein, H., Tanner, J.M., Healy, M., Cameron, N.: Assessment of Skeletal Maturity and Prediction of Adult Height (TW3 Method), 3rd edn. Harcour, New York (2001)

    Google Scholar 

  8. Bull, R., Edwards, P., Kemp, P., Fry, S., Hughes, I.: Bone age assessment: a large scale comparison of the Greulich and Pyle, and tanner and whitehouse (TW2) methods. Arch. Dis. Child. 81(2), 172–173 (1999)

    Article  Google Scholar 

  9. Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press, Cambridge (2016)

    MATH  Google Scholar 

  10. Ehteshami Bejnordi, B., et al.: Diagnostic assessment of deep learning algorithms for detection of lymph node metastases in women with breast cancer. JAMA 318, 2199–2210 (2017)

    Article  Google Scholar 

  11. Zhang, J., Guo, X., Wang, B., et al.: Automatic detection of invasive ductal carcinoma based on the fusion of multi-scale residual convolutional neural network and SVM. IEEE Access 9, 40308–40317 (2021)

    Article  Google Scholar 

  12. Litjens, G., Kooi, T., Bejnordi, B.E., Setio, A.A.A., Ciompi, F., Ghafoorian, M., Van Der Laak, J.A., Van Ginneken, B., Sánchez, C.I.: A survey on deep learning in medical image analysis. Med. Image Anal. 42, 60–88 (2017)

    Article  Google Scholar 

  13. Halabi, S.S., Prevedello, L.M., Kalpathy-Cramer, J., Mamonov, A.B., Bilbily, A., Cicero, M., Pan, I., Pereira, L.A., Sousa, R.T., Abdala, N., et al.: The rsna pediatric bone age machine learning challenge. Radiology 290(2), 498–503 (2019)

    Article  Google Scholar 

  14. Cicero, M., Bilbily, A.: Machine learning and the future of radiology: how we won the 2017 RSNA ML challenge (2017)

  15. Pietka, E., Pospiech-Kurkowska, S., Gertych, A., Cao, F.: Integration of computer assisted bone age assessment with clinical PACS. Comput. Med. Imaging Graph. 27(2–3), 217–228 (2003)

    Article  Google Scholar 

  16. Gertych, A., Zhang, A., Sayre, J., Pospiech-Kurkowska, S., Huang, H.: Bone age assessment of children using a digital hand atlas. Comput. Med. Imaging Graph. 31(4–5), 322–331 (2007)

    Article  Google Scholar 

  17. Thodberg, H.H., Kreiborg, S., Juul, A., Pedersen, K.D.: The bonexpert method for automated determination of skeletal maturity. IEEE Trans. Med. Imaging 28(1), 52–66 (2008)

    Article  Google Scholar 

  18. Giordano, D., Spampinato, C., Scarciofalo, G., Leonardi, R.: An automatic system for skeletal bone age measurement by robust processing of carpal andepiphysial/metaphysial bones. IEEE Trans. Instrum. Meas. 59(10), 2539–2553 (2010)

    Article  Google Scholar 

  19. Pietka, E., Gertych, A., Pospiech, S., Cao, F., Huang, H., Gilsanz, V.: Computer-assisted bone age assessment: image preprocessing and epiphyseal/metaphyseal roi extraction. IEEE Trans. Med. Imaging 20(8), 715–729 (2001)

    Article  Google Scholar 

  20. Hsieh, C., Jong, T., Tiu, C.: Bone age estimation based on phalanx information with fuzzy constrain of carpals. Med. Biol. Eng. Comput. 45(3), 283–295 (2007)

    Article  Google Scholar 

  21. Spampinato, C., Palazzo, S., Giordano, D., Aldinucci, M., Leonardi, R.: Deep learning for automated skeletal bone age assessment in X-ray images. Med. Image Anal. 36, 41–51 (2017)

    Article  Google Scholar 

  22. Tong, C., Liang, B., Li, J., Zheng, Z.: A deep automated skeletal bone age assessment model with heterogeneous features learning. J. Med. Syst. 42(12), 249 (2018)

    Article  Google Scholar 

  23. Ren, X., Li, T., Yang, X., et al.: Regression convolutional neural network for automated pediatric bone age assessment from hand radiograph. IEEE J. Biomed. Health Inform. 23, 2030–2038 (2019)

    Article  Google Scholar 

  24. Son, S.J., Song, Y., Kim, N., Do, Y., Kwak, N., Lee, M.S., Lee, B.D.: TW3-based fully automated bone age assessment system using deep neural networks. IEEE Access 7, 33346–33358 (2019)

    Article  Google Scholar 

  25. Escobar, M., González, C., Torres, F., Daza, L., Triana, G., Arbeláez, P.: Hand pose estimation for pediatric bone age assessment. In: International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 531–539. Springer (2019)

  26. Chen, C., Chen, Z., Jin, X., Li, L., Speier, W., Arnold, C.W.: Attentionguided discriminative region localization for bone age assessment. arXiv preprint. https://arxiv.org/abs/2006.00202 (2020)

  27. Frangi, A.F., Niessen, W.J., Vincken, K.L., Viergever, M.A.: Multi-scale vessel enhancement filtering. In: Proceedings of International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 130–137 (1998)

  28. Chen, X., Zhang, C., Liu, Y.: Bone age assessment with X-ray images based on contourlet motivated deep convolutional networks. In: International Workshop on Multimedia Signal Processing, pp. 1–6 (2018)

  29. Iglovikov, V., Rakhlin, A., Kalinin, A., et al.: Pediatric bone age assessment using deep convolutional neural networks (2017)

  30. Du, R., Chang, D., Bhunia, A.K., Xie, J., Song, Y.Z., Ma, Z., Guo, J.: Fine-grained visual classification via progressive multi-granularity training of jigsaw patches. arXiv preprint. https://arxiv.org/abs/2003.03836 (2020)

  31. Müller, R., Kornblith, S., Hinton, G.E.: When does label smoothing help? In: Advances in Neural Information Processing Systems, pp. 4694–4703 (2019)

  32. Geng, X., Yin, C., Zhou, Z.-H.: Facial age estimation by learning from label distributions. IEEE Trans. Pattern Anal. Mach. Intell. 35(10), 2401–2412 (2013)

    Article  Google Scholar 

  33. Huo, Z., Yang, X., Xing, C., Zhou, Y., Hou, P., Lv, J., Geng, X.: Deep age distribution learning for apparent age estimation. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 17–24 (2016)

  34. Selvaraju, R.R., Cogswell, M., Das, A., Vedantam, R., Parikh, D., Batra, D.: Grad-cam: visual explanations from deep networks via gradient-based localization. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 618–626 (2017)

Download references

Funding

This research has been partially supported by the National Natural Science Foundation of China (Grant No. 61672202), the State Key Program of NSFC-Shenzhen Joint Foundation (Grant No. U1613217), and the Fundamental Research Funds for the Central Universities of China (Grant Nos. PA2019GDPK0076 and PA2020GDSK0060).

Author information

Authors and Affiliations

  1. Key Laboratory of Knowledge Engineering with Big Data, Ministry of Education, Hefei University of Technology, Hefei, 230601, China

    Xiaohua Wang, Ming Xu & Min Hu

  2. School of Computer and Information, Anhui Province Key Laboratory of Affective Computing and Advanced Intelligent Machine, Hefei University of Technology, Hefei, 230601, China

    Xiaohua Wang, Ming Xu, Min Hu & Fuji Ren

  3. Graduate School of Advanced Technology and Science, University of Tokushima, Tokushima, 7708502, Japan

    Fuji Ren

Authors
  1. Xiaohua Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ming Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Min Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Fuji Ren
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Ming Xu or Min Hu.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Wang, X., Xu, M., Hu, M. et al. A multi-scale framework based on jigsaw patches and focused label smoothing for bone age assessment. Vis Comput 39, 1015–1025 (2023). https://doi.org/10.1007/s00371-021-02381-2

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00371-021-02381-2

Keywords

Search

Navigation