Skip to main content
SpringerLink
Log in

Multi-instance learning based on spatial continuous category representation for case-level meningioma grading in MRI images

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

Meningiomas have the highest incidence rate of all primary intracranial and central nervous system tumors. Accurate preoperative grading of meningiomas is extraordinarily meaningful for treatment strategy and patient prognosis. Magnetic resonance imaging (MRI) is the most common method for meningioma grading. Existing methods are typically two-stage models and require image-level classifications or region of interest (ROI) annotations for assistant diagnosis, thereby adding massive manual annotations and time costs. Meanwhile, most of these methods use only a single MRI slice, which may lose the overall meningioma information and are inconsistent with the actual clinical diagnosis process. To address the above problems, a multi-instance learning (MIL) method based on spatial continuous category representation is proposed for case-level meningioma grading. It considers the MRI case and corresponding slices as a bag and instances, respectively, and requires only a case-level label to diagnose a patient. To make the most of the serialization characteristics of MRI images, this method selects continuous instance-feature sequences under each category that are most suspected to contain tumors and further integrates these sequences into bag-level features for classification. In addition, an end-to-end meningioma grading architecture is designed to support the proposed MIL method, which directly takes the original MRI images of the patient as input and outputs the meningioma grade prediction. To train and evaluate the proposed method, datasets with different slice thicknesses are constructed. The experimental results demonstrate that our method performs satisfactorily compared with other related methods for meningioma grading.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Algorithm 1
Fig. 4

Similar content being viewed by others

Data Availability

The medical data cannot be made publicly available due to them containing information that could compromise research participant privacy.

References

  1. Goldbrunner R, Stavrinou P, Jenkinson MD, Sahm F, Mawrin C, Weber DC, Preusser M, Minniti G, Lund-Johansen M, Lefranc F et al (2021) Eano guideline on the diagnosis and management of meningiomas. Neuro-oncology 23(11):1821–1834

    Article  Google Scholar 

  2. Viaene AN, Zhang B, Martinez-Lage M, Xiang C, Tosi U, Thawani JP, Gungor B, Zhu Y, Roccograndi L, Zhang L et al (2019) Transcriptome signatures associated with meningioma progression. Acta Neuropathol Commun 7(1):1–13

    Article  Google Scholar 

  3. Champeaux C, Dunn L (2016) World health organization grade ii meningioma: a 10-year retrospective study for recurrence and prognostic factor assessment. World Neurosurg 89:180–186

    Article  Google Scholar 

  4. Zhu H, Fang Q, He H, Hu J, Jiang D, Xu K (2019) Automatic prediction of meningioma grade image based on data amplification and improved convolutional neural network. Comput Math Methods Med 2019:9

    Article  MATH  Google Scholar 

  5. Hwang KL, Hwang WL, Bussière MR, Shih HA (2017) The role of radiotherapy in the management of high-grade meningiomas. Chinese clinical oncology 6(Suppl 1):5–5

    Article  Google Scholar 

  6. Zhang H, Mo J, Jiang H, Li Z, Hu W, Zhang C, Wang Y, Wang X, Liu C, Zhao B et al (2021) Deep learning model for the automated detection and histopathological prediction of meningioma. Neuroinformatics 19(3):393–402

    Article  Google Scholar 

  7. Hamerla G, Meyer H-J, Schob S, Ginat DT, Altman A, Lim T, Gihr GA, Horvath-Rizea D, Hoffmann K-T, Surov A (2019) Comparison of machine learning classifiers for differentiation of grade 1 from higher gradings in meningioma: a multicenter radiomics study. Magn Reson Imaging 63:244–249

    Article  Google Scholar 

  8. Coroller TP, Bi WL, Huynh E, Abedalthagafi M, Aizer AA, Greenwald NF, Parmar C, Narayan V, Wu WW, Miranda de Moura S et al (2017) Radiographic prediction of meningioma grade by semantic and radiomic features. PloS ONE 12(11):0187908

    Article  Google Scholar 

  9. Laukamp KR, Shakirin G, Baeßler B, Thiele F, Zopfs D, Hokamp NG, Timmer M, Kabbasch C, Perkuhn M, Borggrefe J (2019) Accuracy of radiomics-based feature analysis on multiparametric magnetic resonance images for noninvasive meningioma grading. World Neurosurg 132:366–390

    Article  Google Scholar 

  10. Ke C, Chen H, Lv X, Li H, Zhang Y, Chen M, Hu D, Ruan G, Zhang Y, Zhang Y et al (2020) Differentiation between benign and nonbenign meningiomas by using texture analysis from multiparametric mri. J Magn Reson Imaging 51(6):1810–1820

    Article  Google Scholar 

  11. Prabhu LAJ, Jayachandran A (2018) Mixture model segmentation system for parasagittal meningioma brain tumor classification based on hybrid feature vector. J Med Syst 42(12):1–6

    Google Scholar 

  12. Zhu Y, Man C, Gong L, Dong D, Yu X, Wang S, Fang M, Wang S, Fang X, Chen X et al (2019) A deep learning radiomics model for preoperative grading in meningioma. Eur J Radiol 116:128–134

    Article  Google Scholar 

  13. Chen C, Cheng Y, Xu J, Zhang T, Shu X, Huang W, Hua Y, Zhang Y, Teng Y, Zhang L et al (2021) Automatic meningioma segmentation and grading prediction: a hybrid deep-learning method. J Personal Med 11(8):786

    Article  Google Scholar 

  14. Laukamp KR, Thiele F, Shakirin G, Zopfs D, Faymonville A, Timmer M, Maintz D, Perkuhn M, Borggrefe J (2019) Fully automated detection and segmentation of meningiomas using deep learning on routine multiparametric mri. European radiology 29(1):124–132

    Article  Google Scholar 

  15. Wodzinski M, Banzato T, Atzori M, Andrearczyk V, Cid YD, Muller H (2020) Training deep neural networks for small and highly heterogeneous mri datasets for cancer grading. In: 2020 42nd annual international conference of the IEEE engineering in medicine & biology society (EMBC), IEEE, pp 1758–1761

  16. Wang X, Deng X, Fu Q, Zhou Q, Feng J, Ma H, Liu W, Zheng C (2020) A weakly-supervised framework for covid-19 classification and lesion localization from chest ct. IEEE Trans Med Imaging 39(8):2615–2625

    Article  Google Scholar 

  17. Han Z, Wei B, Hong Y, Li T, Cong J, Zhu X, Wei H, Zhang W (2020) Accurate screening of covid-19 using attention-based deep 3d multiple instance learning. IEEE Trans Med Imaging 39 (8):2584–2594

    Article  Google Scholar 

  18. Lu Y, Liu L, Luan S, Xiong J, Geng D, Yin B (2019) The diagnostic value of texture analysis in predicting who grades of meningiomas based on adc maps: an attempt using decision tree and decision forest. Eur Radiol 29(3):1318–1328

    Article  Google Scholar 

  19. Yan P-F, Yan L, Hu T-T, Xiao D-D, Zhang Z, Zhao H-Y, Feng J (2017) The potential value of preoperative mri texture and shape analysis in grading meningiomas: a preliminary investigation. Translational oncology 10(4):570–577

    Article  Google Scholar 

  20. Pi Y, Li Q, Qi X, Deng D, Yi Z (2022) Automated assessment of bi-rads categories for ultrasound images using multi-scale neural networks with an order-constrained loss function. Appl Intell :1–14

  21. Chen C, Wang Y, Niu J, Liu X, Li Q, Gong X (2021) Domain knowledge powered deep learning for breast cancer diagnosis based on contrast-enhanced ultrasound videos. IEEE Trans Med Imaging 40 (9):2439–2451

    Article  Google Scholar 

  22. Ozdemir O, Russell RL, Berlin AA (2019) A 3d probabilistic deep learning system for detection and diagnosis of lung cancer using low-dose ct scans. IEEE Trans Med Imaging 39(5):1419–1429

    Article  Google Scholar 

  23. Gu Y, Lu X, Yang L, Zhang B, Yu D, Zhao Y, Gao L, Wu L, Zhou T (2018) Automatic lung nodule detection using a 3d deep convolutional neural network combined with a multi-scale prediction strategy in chest cts. Comput Biol Med 103:220–231

    Article  Google Scholar 

  24. Li X, Jia M, Islam MT, Yu L, Xing L (2020) Self-supervised feature learning via exploiting multi-modal data for retinal disease diagnosis. IEEE Trans Med Imaging 39(12):4023–4033

    Article  Google Scholar 

  25. Wang J, Ju R, Chen Y, Zhang L, Hu J, Wu Y, Dong W, Zhong J, Yi Z (2018) Automated retinopathy of prematurity screening using deep neural networks. EBioMedicine 35:361–368

    Article  Google Scholar 

  26. Li S, Liu Y, Sui X, Chen C, Tjio G, Ting DSW, Goh RSM (2019) Multi-instance multi-scale cnn for medical image classification. In: Medical image computing and computer assisted intervention – MICCAI 2019, Springer, pp 531–539

  27. He K, Zhao W, Xie X, Ji W, Liu M, Tang Z, Shi Y, Shi F, Gao Y, Liu J, Zhang J, Shen D (2021) Synergistic learning of lung lobe segmentation and hierarchical multi-instance classification for automated severity assessment of covid-19 in ct images, vol 113

  28. Dietterich TG, Lathrop RH, Lozano-Pérez T (1997) Solving the multiple instance problem with axis-parallel rectangles. Artif Intell 89(1-2):31–71

    Article  MATH  Google Scholar 

  29. Sadafi A, Makhro A, Bogdanova A, Navab N, Peng T, Albarqouni S, Marr C (2020) Attention based multiple instance learning for classification of blood cell disorders. In: International conference on medical image computing and computer-assisted intervention, Springer, pp 246–256

  30. Zhu W, Lou Q, Vang YS, Xie X (2017) Deep multi-instance networks with sparse label assignment for whole mammogram classification. In: International conference on medical image computing and computer-assisted intervention, Springer, pp 603–611

  31. Seibold C, Kleesiek J, Schlemmer H-P, Stiefelhagen R (2020) Self-guided multiple instance learning for weakly supervised thoracic diseaseclassification and localizationin chest radiographs. In: Proceedings of the asian conference on computer vision (ACCV)

  32. Feng Y, Zhang L, Mo J (2020) Deep manifold preserving autoencoder for classifying breast cancer histopathological images. IEEE/ACM Trans Comput Bio Bioinforma 17(1):91–101

    Article  Google Scholar 

  33. Feng J, Zhou Z-H (2017) Deep miml network. In: Proceedings of the AAAI conference on artificial intelligence, vol 31

  34. Hu T, Zhang L, Xie L, Yi Z (2021) A multi-instance networks with multiple views for classification of mammograms. Neurocomputing 443:320–328

    Article  Google Scholar 

  35. Ilse M, Tomczak J, Welling M (2018) Attention-based deep multiple instance learning. In: International conference on machine learning, PMLR, pp 2127–2136

  36. Zhu W, Sun L, Huang J, Han L, Zhang D (2021) Dual attention multi-instance deep learning for alzheimer’s disease diagnosis with structural mri. IEEE Trans Med Imaging 40(9):2354–2366

    Article  Google Scholar 

  37. Hu J, Chen Y, Zhong J, Ju R, Yi Z (2019) Automated analysis for retinopathy of prematurity by deep neural networks. IEEE Trans Med Imaging 38(1):269–279

    Article  Google Scholar 

  38. Wang L, Zhang L, Zhu M, Qi X, Yi Z (2020) Automatic diagnosis for thyroid nodules in ultrasound images by deep neural networks. Med Image Anal 61:101665

    Article  Google Scholar 

  39. Isensee F, Schell M, Pflueger I, Brugnara G, Bonekamp D, Neuberger U, Wick A, Schlemmer H-P, Heiland S, Wick W et al (2019) Automated brain extraction of multisequence mri using artificial neural networks. Human Brain Map 40(17):4952–4964

    Article  Google Scholar 

  40. Otsu N (1979) A threshold selection method from gray-level histograms. IEEE Trans Syst Man Cybernet 9(1):62–66

    Article  MathSciNet  Google Scholar 

  41. Narendra PM, Fitch RC (1981) Real-time adaptive contrast enhancement. IEEE Trans Pattern Anal Mach Intell 6:655–661

    Article  Google Scholar 

  42. Louis DN, Perry A, Reifenberger G, Von Deimling A, Figarella-Branger D, Cavenee WK, Ohgaki H, Wiestler OD, Kleihues P, Ellison DW (2016) The 2016 world health organization classification of tumors of the central nervous system: a summary. Acta neuropathologica 131(6):803–820

    Article  Google Scholar 

  43. Deng J, Dong W, Socher R, Li L-J, Li K, Fei-Fei L (2009) Imagenet: a large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition, IEEE, pp 248–255

  44. Shu X, Zhang L, Wang Z, Lv Q, Yi Z (2020) Deep neural networks with region-based pooling structures for mammographic image classification. IEEE Trans Med Imaging 39(6):2246–2255

    Article  Google Scholar 

  45. He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778

  46. Huang G, Liu Z, Van Der Maaten L, Weinberger KQ (2017) Densely connected convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 4700–4708

  47. Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2818–2826

  48. Hara K, Kataoka H, Satoh Y (2018) Can spatiotemporal 3d cnns retrace the history of 2d cnns and imagenet?. In: Proceedings of the IEEE conference on computer vision and pattern recognition (CVPR), pp 6546–6555

  49. Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A (2016) Learning deep features for discriminative localization. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2921–2929

Download references

Acknowledgements

This work was supported by the National Natural Science Fund for Distinguished Young Scholar under Grant No.62025601.

Author information

Authors and Affiliations

  1. Machine Intelligence Laboratory, College of Computer Science, Sichuan University, No.24 South Section 1, Yihuan Road, Chengdu, 610065, Sichuan, China

    Jiayi Li, Lei Zhang & Xin Shu

  2. Neurosurgery Department, West China Hospital, Sichuan University, No. 37 GuoXue Alley, Chengdu, 610041, Sichuan, China

    Yuen Teng & Jianguo Xu

Authors
  1. Jiayi Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Lei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Xin Shu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Yuen Teng
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Jianguo Xu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Lei Zhang.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, J., Zhang, L., Shu, X. et al. Multi-instance learning based on spatial continuous category representation for case-level meningioma grading in MRI images. Appl Intell 53, 16015–16028 (2023). https://doi.org/10.1007/s10489-022-04114-x

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-022-04114-x

Keywords

Search

Navigation