Skip to main content
SpringerLink
Log in

Transparent Ensembles for Covid-19 Prognosis

  • Conference paper
  • First Online:
Machine Learning and Knowledge Extraction (CD-MAKE 2021)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 12844))

Abstract

A natural method aiming at explaining the answers of a black-box model is by means of propositional rules. Nevertheless, rule extraction from ensembles of Machine Learning models was rarely achieved. Moreover, experiments in this context have rarely been evaluated by cross-validation trials. Based on stratified tenfold cross-validation, we performed experiments with several ensemble models on Covid-19 prognostic data. Specifically, we compared the characteristics of the propositional rules generated from: Random Forests; Shallow Trees trained by Gradient Boosting; Decision Stumps trained by several variants of Boosting; and ensembles of transparent neural networks trained by Bagging. The Discretized Interpretable Multi Layer Perceptron (DIMLP) allowed us to generate rules from all the used ensembles by transforming Decision Trees into DIMLPs. Our rule extraction technique simply determines whether an axis-parallel hyperplane is discriminative or not, with a greedy algorithm that progressively removes rule antecedents. Rules extracted from Decision Stumps trained by modest Adaboost were the simplest with the highest fidelity. Our best average predictive accuracy result was equal to 96.5%. Finally, we described a particular ruleset extracted from an ensemble of Decision Stumps and it turned out that the rule antecedents seem to be plausible with respect to several recent works related to the Covid-19 virus.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Andrews, R., Diederich, J., Tickle, A.B.: Survey and critique of techniques for extracting rules from trained artificial neural networks. Knowl.-Based Syst. 8(6), 373–389 (1995)

    Article  Google Scholar 

  2. Bologna, G.: A study on rule extraction from several combined neural networks. Int. J. Neural Syst. 11(03), 247–255 (2001)

    Article  Google Scholar 

  3. Bologna, G.: A model for single and multiple knowledge based networks. Artif. Intell. Med. 28(2), 141–163 (2003)

    Article  Google Scholar 

  4. Bologna, G.: Is it worth generating rules from neural network ensembles? J. Appl. Log. 2(3), 325–348 (2004)

    Article  MathSciNet  Google Scholar 

  5. Breiman, L.: Bagging predictors. Mach. Learn. 24(2), 123–140 (1996)

    Google Scholar 

  6. Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)

    Google Scholar 

  7. Chen, J., Wu, C., Wang, X., Yu, J., Sun, Z.: The impact of covid-19 on blood glucose: a systematic review and meta-analysis. Front. Endocrinol. 11 (2020)

    Google Scholar 

  8. Cohen, W.W.: Fast effective rule induction. In: Proceedings of the Twelfth International Conference on Machine Learning, pp. 115–123 (1995)

    Google Scholar 

  9. Freund, Y., Schapire, R.E.: A desicion-theoretic generalization of on-line learning and an application to boosting. In: Vitainyi, P. (eds.) Computational Learning Theory. EuroCOLT 1995. Lecture Notes in Computer Science (Lecture Notes in Artificial Intelligence), vol. 904. Springer, Berlin, Heidelberg (1995). https://doi.org/10.1007/3-540-59119-2_166

  10. Friedman, J., Hastie, T., Tibshirani, R., et al.: Additive logistic regression: a statistical view of boosting (with discussion and a rejoinder by the authors). Ann. Stat. 28(2), 337–407 (2000)

    Article  Google Scholar 

  11. Hara, A., Hayashi, Y.: Ensemble neural network rule extraction using re-rx algorithm. In: The 2012 International Joint Conference on Neural Networks (IJCNN), pp. 1–6. IEEE (2012)

    Google Scholar 

  12. Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer Science & Business Media (2009)

    Google Scholar 

  13. Holzinger, A., Biemann, C., Pattichis, C.S., Kell, D.B.: What do we need to build explainable AI systems for the medical domain? arXiv:1712.09923 (2017)

  14. Holzinger, A., Malle, B., Saranti, A., Pfeifer, B.: Towards multi-modal causability with graph neural networks enabling information fusion for explainable AI. Inf. Fusion 71, 28–37 (2021)

    Article  Google Scholar 

  15. Hu, R., Han, C., Pei, S., Yin, M., Chen, X.: Procalcitonin levels in covid-19 patients. Int. J. Antimicrob. Agents 56(2), 106051 (2020)

    Google Scholar 

  16. Jiang, S.Q., Huang, Q.F., Xie, W.M., Lv, C., Quan, X.Q.: The association between severe covid-19 and low platelet count: evidence from 31 observational studies involving 7613 participants. Br. J. Haematol. 190(1), e29–e33 (2020)

    Article  Google Scholar 

  17. Johansson, U.: Obtaining accurate and comprehensible data mining models: an evolutionary approach. Linköping University, Department of Computer and Information Science (2007)

    Google Scholar 

  18. Mashayekhi, M., Gras, R.: Rule extraction from decision trees ensembles: new algorithms based on heuristic search and sparse group lasso methods. Int. J. Inf. Technol. Dec. Making 1–21 (2017)

    Google Scholar 

  19. Pedregosa, F., et al.: Scikit-learn: machine learning in python. J. Mach. Learn. Res. 12, 2825–2830 (2011)

    Google Scholar 

  20. Quinlan, J.R.: C4.5: Programs for machine learning. morgan kaufmann publishers, inc., 1993. Mach. Learn. 16(3), 235–240 (1994)

    Google Scholar 

  21. Schapire, R.E.: A brief introduction to boosting. Ijcai. 99, 1401–1406 (1999)

    Google Scholar 

  22. Setiono, R., Baesens, B., Mues, C.: Recursive neural network rule extraction for data with mixed attributes. IEEE Trans. Neural Netw. 19(2), 299–307 (2008)

    Google Scholar 

  23. Van Assche, A., Blockeel, H.: Seeing the forest through the trees: learning a comprehensible model from an ensemble. In: Kok, J.N., Koronacki, J., Mantaras, R.L., Matwin, S., Mladenic, D., Skowron, A. (eds.) Machine Learning: ECML 2007. ECML 2007. Lecture Notes in Computer Science, vol. 4701. Springer, Berlin, Heidelberg (2007). https://doi.org/10.1007/978-3-540-74958-5_39

  24. Vezhnevets, A., Vezhnevets, V.: Modest adaboost-teaching adaboost to generalize better. In: Proceedings of the International Conference on Computer Graphics in Europe and Asia, vol. 12, pp. 987–997. Computer Graphics in Russia (2005)

    Google Scholar 

  25. Wang, Y., et al.: The peak levels of highly sensitive troponin i predicts in-hospital mortality in covid-19 patients with cardiac injury: a retrospective study. Eur. Heart J. Acute Cardiovasc. Care 10(1), 6–15 (2021)

    Google Scholar 

  26. Yan, L., et al.: An interpretable mortality prediction model for covid-19 patients. Nat. Mach. Intell. 2(5), 283–288 (2020)

    Google Scholar 

  27. Zhou, Z.H., Jiang, Y., Chen, S.F.: Extracting symbolic rules from trained neural network ensembles. Artif. Intell. Commun. 16(1), 3–16 (2003)

    MATH  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Applied Sciences and Arts of Western Switzerland, Rue de la Prairie 4, 1202, Geneva, Switzerland

    Guido Bologna

Authors
  1. Guido Bologna
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Guido Bologna .

Editor information

Editors and Affiliations

  1. Institute for Medical Informatics, Statistics and Documentation and Institute for Information Systems and Computer Media, Medical University Graz and Graz University of Technology, Graz, Austria

    Andreas Holzinger

  2. St. Pölten University of Applied Sciences, St. Pölten, Austria

    Peter Kieseberg

  3. Institute of Software Technology and Interactive Systems, Technische Universität Wien, Vienna, Austria

    A Min Tjoa

  4. SBA Research, Vienna, Austria

    Edgar Weippl

Rights and permissions

Reprints and permissions

Copyright information

© 2021 IFIP International Federation for Information Processing

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Bologna, G. (2021). Transparent Ensembles for Covid-19 Prognosis. In: Holzinger, A., Kieseberg, P., Tjoa, A.M., Weippl, E. (eds) Machine Learning and Knowledge Extraction. CD-MAKE 2021. Lecture Notes in Computer Science(), vol 12844. Springer, Cham. https://doi.org/10.1007/978-3-030-84060-0_22

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-84060-0_22

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-84059-4

  • Online ISBN: 978-3-030-84060-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation