Skip to main content
SpringerLink
Log in

Automatic vertebral fracture and three-column injury diagnosis with fracture visualization by a multi-scale attention-guided network

  • Original Article
  • Published:
Medical & Biological Engineering & Computing Aims and scope Submit manuscript

Abstract

Deep learning methods have the potential to improve the efficiency of diagnosis for vertebral fractures with computed tomography (CT) images. Most existing intelligent vertebral fracture diagnosis methods only provide dichotomized results at a patient level. However, a fine-grained and more nuanced outcome is clinically needed. This study proposed a novel network, a multi-scale attention-guided network (MAGNet), to diagnose vertebral fractures and three-column injuries with fracture visualization at a vertebra level. By imposing attention constraints through a disease attention map (DAM), a fusion of multi-scale spatial attention maps, the MAGNet can get task highly relevant features and localize fractures. A total of 989 vertebrae were studied here. After four-fold cross-validation, the area under the ROC curve (AUC) of our model for vertebral fracture dichotomized diagnosis and three-column injury diagnosis was 0.884 ± 0.015 and 0.920 ± 0.104, respectively. The overall performance of our model outperformed classical classification models, attention models, visual explanation methods, and attention-guided methods based on class activation mapping. Our work can promote the clinical application of deep learning to diagnose vertebral fractures and provide a way to visualize and improve the diagnosis results with attention constraints.

Graphical Abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Data availability

The data underlying this article cannot be shared publicly due to the privacy of the patients in the study. The data will be shared on reasonable request to the corresponding author.

References

  1. Whitney E, Alastra AJ (2022) Vertebral fracture. In: StatPearls [Internet]. Treasure Island (FL): Stat Pearls Publishing

  2. Baum T, Bauer JS, Klinder T, Dobritz M, Rummeny EJ, Noël PB, Lorenz C (2014) Automatic detection of osteoporotic vertebral fractures in routine thoracic and abdominal MDCT. Eur Radiol 24(4):872–880. https://doi.org/10.1007/s00330-013-3089-2

    Article  PubMed  Google Scholar 

  3. Bar A, Wolf L, Bergman Amitai O, Toledano E, Elnekave E (2017) Compression fractures detection on CT. In: Medical imaging 2017: computer-aided diagnosis. SPIE 10134:1036–1043. https://doi.org/10.1117/12.2249635

  4. Tomita N, Cheung YY, Hassanpour S (2018) Deep neural networks for automatic detection of osteoporotic vertebral fractures on CT scans. Comput Biol Med 98:8–15. https://doi.org/10.1016/j.compbiomed.2018.05.011

    Article  PubMed  Google Scholar 

  5. Iyer S, Sowmya A, Blair A, White C, Dawes L, Moses D (2020) A novel approach to vertebral compression fracture detection using imitation learning and patch based convolutional neural network. In: 2020 IEEE 17th International Symposium on Biomedical Imaging (ISBI). IEEE, Iowa, USA, pp 726–730. https://doi.org/10.1109/ISBI45749.2020.9098714

  6. Burns JE, Yao J, Summers RM (2017) Vertebral body compression fractures and bone density: automated detection and classification on CT images. Radiology 284(3):788–797. https://doi.org/10.1148/radiol.2017162100

    Article  PubMed  Google Scholar 

  7. Suri A, Jones BC, Ng G, Anabaraonye N, Beyrer P, Domi A, Choi G, Tang S, Terry A, Leichner T, Fathali I, Bastin N, Chesnais H, Taratuta E, Kneeland BJ, Chamith SR (2021) Vertebral deformity measurements at MRI, CT, and radiography using deep learning. Radiology 4(1):e210015. https://doi.org/10.1148/ryai.2021210015

    Article  PubMed  PubMed Central  Google Scholar 

  8. Kalmet PH, Sanduleanu S, Primakov S, Wu G, Jochems A, Refaee T, Ibrahim A, Hulst LV, Lambin P, Poeze M (2020) Deep learning in fracture detection: a narrative review. Acta Orthop 91(2):215–220. https://doi.org/10.1080/17453674.2019.1711323

    Article  PubMed  PubMed Central  Google Scholar 

  9. Raghavendra U, Bhat NS, Gudigar A, Acharya UR (2018) Automated system for the detection of thoracolumbar fractures using a CNN architecture. Futur Gener Comput Syst 85:184–189. https://doi.org/10.1016/j.future.2018.03.023

    Article  Google Scholar 

  10. Sha G, Wu J, Yu B (2020) Detection of spinal fracture lesions based on improved faster-RCNN. In: 2020 IEEE International Conference on Artificial Intelligence and Information Systems (ICAIIS). IEEE, Dalian, China, pp 29–32. https://doi.org/10.1109/ICAIIS49377.2020.9194863

  11. Ferguson RL, Allen B Jr (1984) A mechanistic classification of thoracolumbar spine fractures. Clin Orthop Relat Res 189:77–88

    Article  Google Scholar 

  12. Denis F (1983) The three column spine and its significance in the classification of acute thoracolumbar spinal injuries. Spine 8(8):817–31. https://doi.org/10.1097/00007632-198311000-00003

    Article  CAS  PubMed  Google Scholar 

  13. Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A (2016) Learning deep features for discriminative localization. In: Proc IEEE Conf Comput Vis Pattern Recognit. IEEE, pp 2921–2929. https://doi.org/10.48550/arXiv.1512.04150

  14. Selvaraju RR, Cogswell M, Das A, Vedantam R, Parikh D, Batra D (2017) Grad-cam: Visual explanations from deep networks via gradient-based localization. In: Proc IEEE Int Conf Comput Vision. pp 618–626. https://doi.org/10.48550/arXiv.1802.10171

  15. Chattopadhyay A, Sarkar A, Howlader P, Balasubramanian VN (2018) Grad-CAM++: generalized gradient-based visual explanations for deep convolutional networks. In: IEEE Winter Conference on Applications of Computer Vision (WACV). IEEE, Lake Tahoe, NV, USA, pp 839–847. https://doi.org/10.1109/WACV.2018.00097

  16. Li K, Wu Z, Peng K-C, Ernst J, Fu Y (2018) Tell me where to look: guided attention inference network. In: Proc IEEE Conf Comput Vis Pattern Recognit.  pp 9215-9223. https://doi.org/10.48550/arXiv.1802.10171

  17. Wang S, Ouyang X, Liu T, Wang Q, Shen D (2022) Follow my eye: using gaze to supervise computer-aided diagnosis. IEEE Trans Med Imaging 41(7):1688–1698. https://doi.org/10.1109/TMI.2022.3146973

    Article  PubMed  Google Scholar 

  18. Ouyang X, Karanam S, Wu Z, Chen T, Huo J, Zhou XS, Wang Q, Cheng JZ (2020) Learning hierarchical attention for weakly-supervised chest X-ray abnormality localization and diagnosis. IEEE Trans Med Imaging 40(10):2698–2710. https://doi.org/10.1109/TMI.2020.3042773

    Article  Google Scholar 

  19. Yushkevich PA, Gerig G (2017) ITK-SNAP: an intractive medical image segmentation tool to meet the need for expert-guided segmentation of complex medical images. IEEE Pulse 8(4):54–57. https://doi.org/10.1109/MPUL.2017.2701493

    Article  PubMed  Google Scholar 

  20. Urschler M, Bischof H, Štern D, Payer C (2020) Coarse to fine vertebrae localization and segmentation with spatial configuration-net and U-Net. In: Proceedings of the 15th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications. pp 124–133. https://doi.org/10.5220/0008975201240133

  21. Payer C, Štern D, Bischof H, Urschler M (2019) Integrating spatial configuration into heatmap regression based CNNs for landmark localization. Med Image Anal 54:207–219. https://doi.org/10.1016/j.media.2019.03.007

    Article  PubMed  Google Scholar 

  22. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proc IEEE Conf Comput Vis Pattern Recognit. pp 770–778. https://doi.org/10.48550/arXiv.1512.03385

  23. Woo S, Park J, Lee JY, Kweon IS (2018) Cbam: Convolutional block attention module. In: Proc Eur Conf Comput Vision (ECCV). pp  3–19. https://doi.org/10.48550/arXiv.1807.06521

  24. Kingma D, Ba J (2014) Adam: a method for stochastic optimization. arXiv:1412.6980. https://doi.org/10.48550/arXiv.1412.6980

  25. Hossin M, Sulaiman MN (2015) A review on evaluation metrics for data classification evaluations. Int J Data Mining Knowl Manag Proc 5(2):01–11

    Article  Google Scholar 

  26. Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:14091556. https://doi.org/10.48550/arXiv.1409.1556

  27. Szegedy C, Liu W, Jia Y, Sermanet P, Reed S, Anguelov D, Erhan D, Vanhoucke V, Rabinovich A (2015) Going deeper with convolutions. In: Proc IEEE Conf Comput Vis Pattern Recognit. pp 1–9. https://doi.org/10.48550/arXiv.1409.4842

  28. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In: Proc IEEE Conf Comput Vis Pattern Recognit. pp 2818–2826. https://doi.org/10.48550/arXiv.1512.00567

  29. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proc IEEE Conf Comput Vis Pattern Recognit. pp 4700–4708. https://doi.org/10.48550/arXiv.1608.06993

  30. Xie S, Girshick R, Dollár P, Tu Z, He K (2017) Aggregated residual transformations for deep neural networks. In: Proc IEEE Conf Comput Vis Pattern Recognit.pp 1492–1500. https://doi.org/10.48550/arXiv.1611.05431

  31. Hu J, Shen L, Sun G (2018) Squeeze-and-excitation networks. In: Proc IEEE Conf Comput Vis Pattern Recognit. pp 7132–7141. https://doi.org/10.48550/arXiv.1709.01507

  32. Park J, Woo S, Lee JY, Kweon IS (2018) Bam: bottleneck attention module. arXiv preprint arXiv:1807.06514. https://doi.org/10.48550/arXiv.1807.06514

  33. Pereira S, Pinto A, Amorim J, Ribeiro A, Alves V, Silva CA (2019) Adaptive feature recombination and recalibration for semantic segmentation with fully convolutional networks. IEEE Trans Med Imaging 38(12):2914–2925. https://doi.org/10.1109/TMI.2019.2918096

    Article  PubMed  Google Scholar 

  34. Zhang B, Qi S, Wu Y, Pan X, Yao Y, Qian W, Guan Y (2022). Multi-scale segmentation squeeze-and-excitation UNet with conditional random field for segmenting lung tumor from CT images. Comput Methods Prog Biomed 106946.https://doi.org/10.1016/j.cmpb.2022.106946

  35. Zhang Y, Kang B, Hooi B, Yan S, Feng J (2021) Deep long-tailed learning: a survey. arXiv preprint arXiv:2110.04596. https://doi.org/10.48550/arXiv.2110.04596

  36. Tarekegn AN, Giacobini M, Michalak K (2021) A review of methods for imbalanced multi-label classification. Pattern Recognit 118:107965. https://doi.org/10.1016/j.patcog.2021.107965

    Article  Google Scholar 

Download references

Funding

This work was supported by the National Key Research and Development Program under grant 2016YFC01004608, National Natural Science Foundation of China under grant U1732119, Shanghai Jiao Tong University Medical Engineering Cross Research Funds under grant YG2021ZD05, and National Key R&D Program of China under grant 2021YFF0703702.

Author information

Authors and Affiliations

  1. School of Biomedical Engineering, Shanghai Jiao Tong University, Shanghai, 200240, China

    Shunan Zhang, Ziqi Zhao, Lu Qiu, Duan Liang, Jun Zhao & Jianqi Sun

  2. Renji Hospital, Shanghai, 200127, China

    Kun Wang

  3. Shanghai Sixth People’s Hospital, Shanghai, 200233, China

    Jun Xu

Authors
  1. Shunan Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ziqi Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Lu Qiu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Duan Liang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Kun Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Jun Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Jun Zhao
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Jianqi Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

All authors contributed to the study conception and design. SZ, JX, and KW built the dataset. SZ wrote the main manuscript text. SZ, ZZ, LQ, DL, JZ, and JS reviewed and revised the manuscript. All authors read and approved the final manuscript.

Corresponding authors

Correspondence to Jun Xu, Jun Zhao or Jianqi Sun.

Ethics declarations

Ethics approval and consent to participate

This study was approved by the Ethics Committee of Shanghai Sixth People’s Hospital, and the informed consent was waived.

Competing interests

The authors declare no competing interests.

Additional information

Publisher's note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Below is the link to the electronic supplementary material.

Supplementary file1 (PDF 56 KB)

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, S., Zhao, Z., Qiu, L. et al. Automatic vertebral fracture and three-column injury diagnosis with fracture visualization by a multi-scale attention-guided network. Med Biol Eng Comput 61, 1661–1674 (2023). https://doi.org/10.1007/s11517-023-02805-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11517-023-02805-2

Keywords

Search

Navigation