Skip to main content

Advertisement

Log in

A machine learning approach for mortality prediction only using non-invasive parameters

  • Review Article
  • Published:
Medical & Biological Engineering & Computing Aims and scope Submit manuscript

Abstract

At present, the traditional scoring methods generally utilize laboratory measurements to predict mortality. It results in difficulties of early mortality prediction in the rural areas lack of professional laboratorians and medical laboratory equipment. To improve the efficiency, accuracy, and applicability of mortality prediction in the remote areas, a novel mortality prediction method based on machine learning algorithms is proposed, which only uses non-invasive parameters readily available from ordinary monitors and manual measurement. A new feature selection method based on the Bayes error rate is developed to select valuable features. Based on non-invasive parameters, four machine learning models were trained for early mortality prediction. The subjects contained in this study suffered from general critical diseases including but not limited to cancer, bone fracture, and diarrhea. Comparison tests among five traditional scoring methods and these four machine learning models with and without laboratory measurement variables are performed. Only using the non-invasive parameters, the LightGBM algorithms have an excellent performance with the largest accuracy of 0.797 and AUC of 0.879. There is no apparent difference between the mortality prediction performance with and without laboratory measurement variables for the four machine learning methods. After reducing the number of feature variables to no more than 50, the machine learning models still outperform the traditional scoring systems, with AUC higher than 0.83. The machine learning approaches only using non-invasive parameters achieved an excellent mortality prediction performance and can equal those using extra laboratory measurements, which can be applied in rural areas and remote battlefield for mortality risk evaluation.

Graphical abstract

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  1. Le GJ, Lemeshow S, Saulnier F (1993) A new simplified acute physiology score (SAPS II) based on a European/North American multicenter study. J Am Med Assoc

  2. Arts DGT, de Keizer NF, Vroom MB, de Jonge E (2005) Reliability and accuracy of sequential organ failure assessment (SOFA) scoring. Crit Care Med 33:1988. https://doi.org/10.1097/01.CCM.0000178178.02574.AB

    Article  CAS  PubMed  Google Scholar 

  3. Pollack MM, Patel KM, Ruttimann UE (1997) The pediatric risk of mortality III— acute physiology score (PRISM III-APS): a method of assessing physiologic instability for pediatric intensive care unit patients. J Pediatr 131:575–581. https://doi.org/10.1016/S0022-3476(97)70065-9

    Article  CAS  PubMed  Google Scholar 

  4. Strand K, Flaatten H (2008) Severity scoring in the ICU: a review: severity scoring in the ICU. Acta Anaesthesiol Scand 52:467–478. https://doi.org/10.1111/j.1399-6576.2008.01586.x

    Article  CAS  PubMed  Google Scholar 

  5. Moreno R, Matos R (2001) New issues in severity scoring: interfacing the ICU and evaluating it. Curr Opin Crit Care 7:469–474. https://doi.org/10.1097/00075198-200112000-00018

    Article  CAS  PubMed  Google Scholar 

  6. Mayaud L, Lai PS, Clifford GD et al (2013) Dynamic data during hypotensive episode improves mortality predictions among patients with sepsis and hypotension*. Crit Care Med 41:954–962. https://doi.org/10.1097/CCM.0b013e3182772adb

    Article  PubMed  PubMed Central  Google Scholar 

  7. Davoodi R, Moradi MH (2018) Mortality prediction in intensive care units (ICUs) using a deep rule-based fuzzy classifier. J Biomed Inform 79:48–59. https://doi.org/10.1016/j.jbi.2018.02.008

    Article  PubMed  Google Scholar 

  8. Kim KA, Choi JY, Yoo TK (2013) Mortality prediction of rats in acute hemorrhagic shock using machine learning techniques. Med Biol Eng Comput 51:1059–1067. https://doi.org/10.1007/s11517-013-1091-0

    Article  PubMed  Google Scholar 

  9. Ramon J, Fierens D, Güiza F et al (2007) Mining data from intensive care patients. Adv Eng Inform 21:243–256. https://doi.org/10.1016/j.aei.2006.12.002

    Article  Google Scholar 

  10. Lee CH, Arzeno NM, Ho JC et al (2012) An imputation-enhanced algorithm for ICU mortality prediction. Comput Cardiol:253–256

  11. Weissman GE, Hubbard RA, Ungar LH, Harhay MO, Greene CS, Himes BE, Halpern SD (2018) Inclusion of unstructured clinical text improves early prediction of death or prolonged ICU stay*. Crit Care Med 46:1125–1132. https://doi.org/10.1097/CCM.0000000000003148

    Article  PubMed  PubMed Central  Google Scholar 

  12. Cooper JN, Minneci PC, Deans KJ (2018) Postoperative neonatal mortality prediction using superlearning. J Surg Res 221:311–319. https://doi.org/10.1016/j.jss.2017.09.002

    Article  PubMed  Google Scholar 

  13. Awad A, Bader-El-Den M, McNicholas J, Briggs J (2017) Early hospital mortality prediction of intensive care unit patients using an ensemble learning approach. Int J Med Inform 108:185–195. https://doi.org/10.1016/j.ijmedinf.2017.10.002

    Article  PubMed  Google Scholar 

  14. Senzhang Wang, Zhoujun Li, Wenhan Chao, Qinghua Cao (2012) Applying adaptive over-sampling technique based on data density and cost-sensitive SVM to imbalanced learning. In: The 2012 International Joint Conference on Neural Networks (IJCNN). IEEE, Brisbane, pp 1–8

  15. Li D, Djulovic A, Xu J (2013) A study of kNN using ICU multivariate time series data. Proc Int Conf Data Min Eds R Stahlbock GM Weiss 211–217

  16. Rajkomar A, Oren E, Chen K et al (2018) Scalable and accurate deep learning with electronic health records. Npj Digit Med 1. https://doi.org/10.1038/s41746-018-0029-1

  17. Bernstein A, Provost F, Hill S (2005) Toward intelligent assistance for a data mining process: an ontology-based approach for cost-sensitive classification. IEEE Trans Knowl Data Eng 17:503–518. https://doi.org/10.1109/TKDE.2005.67

    Article  Google Scholar 

  18. Johnson AEW, Pollard TJ, Shen L, Lehman LW, Feng M, Ghassemi M, Moody B, Szolovits P, Celi LA, Mark RG (2016) MIMIC-III, a freely accessible critical care database. Sci Data 3:160035. https://doi.org/10.1038/sdata.2016.35

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  19. Penny W, Frost D (1996) Neural networks in clinical medicine. Med Decis Mak 16:386–398. https://doi.org/10.1177/0272989X9601600409

    Article  CAS  Google Scholar 

  20. Wu CL, Chau KW, Fan C (2010) Prediction of rainfall time series using modular artificial neural networks coupled with data-preprocessing techniques. J Hydrol 389:146–167. https://doi.org/10.1016/j.jhydrol.2010.05.040

    Article  Google Scholar 

  21. Allen D (1993) Automatic one-hot re-encoding for FPLs. In: Selected Papers from the Second International Workshop on Field-Programmable Logic and Applications, Field-Programmable Gate Arrays: Architectures and Tools for Rapid Prototyping. Springer-Verlag, London, pp 71–77

  22. Cools F, Askie LM, Offringa M, Asselin JM, Calvert SA, Courtney SE, Dani C, Durand DJ, Gerstmann DR, Henderson-Smart DJ, Marlow N, Peacock JL, Pillow JJ, Soll RF, Thome UH, Truffert P, Schreiber MD, van Reempts P, Vendettuoli V, Vento G, PreVILIG collaboration (2010) Elective high-frequency oscillatory versus conventional ventilation in preterm infants: a systematic review and meta-analysis of individual patients’ data. Lancet 375:2082–2091. https://doi.org/10.1016/S0140-6736(10)60278-4

    Article  PubMed  Google Scholar 

  23. Mei Z (2007) Standard deviation of anthropometric Z-scores as a data quality assessment tool using the 2006 WHO growth standards: a cross country analysis. Bull World Health Organ 85:441–448. https://doi.org/10.2471/BLT.06.034421

    Article  PubMed  PubMed Central  Google Scholar 

  24. Spolaor N, Cherman EA, Monard MC, Lee HD (2013) ReliefF for multi-label feature selection. In: 2013 Brazilian Conference on Intelligent Systems. IEEE, Fortaleza, pp 6–11

    Chapter  Google Scholar 

  25. Gu Q, Li Z, Han J Generalized Fisher score for feature selection. 8

  26. Park H, Kwon S, Kwon H-C Complete Gini-index text (GIT) feature-selection algorithm for text classification. 7

  27. Alonso-Atienza F, Morgado E, Fernandez-Martinez L et al (2014) Detection of life-threatening arrhythmias using feature selection and support vector machines. IEEE Trans Biomed Eng 61:832–840. https://doi.org/10.1109/TBME.2013.2290800

    Article  PubMed  Google Scholar 

  28. Cho H-W, Baek SH, Youn E, Jeong MK, Taylor A (2009) A two-stage classification procedure for near-infrared spectra based on multi-scale vertical energy wavelet thresholding and SVM-based gradient-recursive feature elimination. J Oper Res Soc 60:1107–1115. https://doi.org/10.1057/jors.2008.179

    Article  Google Scholar 

  29. Ke G, Meng Q, Finley T, et al (2017) LightGBM: a highly efficient gradient boosting decision tree. In: Guyon I, Luxburg UV, Bengio S, et al (eds) Advances in Neural Information Processing Systems 30. Curran Associates, Inc., pp 3146–3154

  30. Chen T, Guestrin C (2016) XGBoost: a scalable tree boosting system. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining - KDD ’16. ACM Press, San Francisco, pp 785–794

    Chapter  Google Scholar 

  31. Svetnik V, Liaw A, Tong C, Culberson JC, Sheridan RP, Feuston BP (2003) Random Forest: a classification and regression tool for compound classification and QSAR modeling. J Chem Inf Comput Sci 43:1947–1958. https://doi.org/10.1021/ci034160g

    Article  CAS  PubMed  Google Scholar 

  32. Dodek PM, Wiggs BR (1998) Logistic regression model to predict outcome after in-hospital cardiac arrest: validation, accuracy, sensitivity and specificity. Resuscitation 36:201–208. https://doi.org/10.1016/S0300-9572(98)00012-4

    Article  CAS  PubMed  Google Scholar 

  33. Moon A, Cosgrove JF, Lea D, Fairs A, Cressey DM (2011) An eight year audit before and after the introduction of modified early warning score (MEWS) charts, of patients admitted to a tertiary referral intensive care unit after CPR. Resuscitation 82:150–154. https://doi.org/10.1016/j.resuscitation.2010.09.480

    Article  CAS  PubMed  Google Scholar 

  34. Carney C (2011) A new classification system for grading the severity of onychomycosis: onychomycosis severity index. Arch Dermatol 147:1277–1282. https://doi.org/10.1001/archdermatol.2011.267

    Article  PubMed  Google Scholar 

  35. Jiao Y, Du P (2016) Performance measures in evaluating machine learning based bioinformatics predictors for classifications. Quant Biol 4:320–330. https://doi.org/10.1007/s40484-016-0081-2

    Article  Google Scholar 

  36. Figuera C, Irusta U, Morgado E, Aramendi E, Ayala U, Wik L, Kramer-Johansen J, Eftestøl T, Alonso-Atienza F (2016) Machine learning techniques for the detection of shockable rhythms in automated external defibrillators. PLoS One 11:e0159654. https://doi.org/10.1371/journal.pone.0159654

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  37. Hastie T, Friedman J, Tibshirani R (2001) The elements of statistical learning. In: The elements of statistical learning. Springer New York, New York, pp 193–224

    Chapter  Google Scholar 

  38. Alves-Filho J, Freitas A, Spiller F, Souto FO, Paula-Neto H, Silva JS, Cunha FQ (2007) Fourth international symposium on intensive care and emergency medicine for Latin America. Crit Care 11:P3. https://doi.org/10.1186/cc5790

    Article  PubMed Central  Google Scholar 

  39. Junger A, Böttger S, Engel J et al (2002) Automatic calculation of a modified APACHE II score using a patient data management system (PDMS). Int J Med Inform 65:145–157. https://doi.org/10.1016/S1386-5056(02)00014-X

    Article  PubMed  Google Scholar 

  40. Sun Y, Kamel MS, Wong AKC, Wang Y (2007) Cost-sensitive boosting for classification of imbalanced data. Pattern Recogn 40:3358–3378. https://doi.org/10.1016/j.patcog.2007.04.009

    Article  Google Scholar 

  41. Zadrozny B, Langford J, Abe N (2003) Cost-sensitive learning by cost-proportionate example weighting. In: Third IEEE International Conference on Data Mining. IEEE Comput. Soc, Melbourne, pp 435–442

  42. Sicari R, Pasanisi E, Venneri L, Landi P, Cortigiani L, Picano E, Echo Persantine International Cooperative (EPIC) Study Group, Echo Dobutamine International Cooperative (EDIC) Study Group (2003) Stress echo results predict mortality: a large-scale multicenter prospective international study. J Am Coll Cardiol 41:589–595. https://doi.org/10.1016/S0735-1097(02)02863-2

    Article  PubMed  Google Scholar 

Download references

Funding

This study was supported by National Key R&D Program of China (Grant Number: 2017YFC0806402, 2017YFC0806404, 2017YFC0806406) and Science and Technology Program of Tianjin, China (Grant Number: 18ZXJMTG00060). The work was also funded in part by logistics scientific research foundation program at the Military Medical Innovation Project (Grant Number: 16CXZ034).

Author information

Authors and Affiliations

Authors

Corresponding authors

Correspondence to Guang Zhang or Feng Chen.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

1.1 Machine learning algorithms

As mentioned previously, four commonly used machine learning algorithms: LightGBM, XGBoost, Rand Forest and Logistic regression model, which have excellent classification performance were employed in this present study.

1.1.1 Random forest [1]

Random forest is an integrated classifier, which is composed of a set of decision tree classifiers. The optimal prediction result is determined by the decision tree classifier vote, as indicated by Equation (1).

$$ \hat{y}=\frac{1}{m}\sum \limits_{j=1}^m\sum \limits_{i=1}^n{W}_j\left({x}_i,x^{\prime}\right){y}_i=\sum \limits_{i=1}^n\left(\frac{1}{m}\sum \limits_{j=1}^m{W}_j\left({x}_i,x^{\prime}\right)\right){y}_i $$
(1)

The predictions \( \hat{y} \) for new points x′ are achieved by seeking the neighboring point from a training set \( {\left\{\left({x}_i,{y}_i\right)\right\}}_{i=1}^n \). A forest averages the predictions of a set of m trees with individual weight functions Wj. Wj(xj, x′) is the non-negative weight of the i-th training point relative to the new point x′ in the same tree.

1.1.2 XGBOOST [2]

Boosting is a very effective integrated learning algorithm, which can transform a weak classifier into a strong classifier. Gradient boosting, which is an improved version of boosting, makes the loss function to fall along its gradient direction during the iteration process, thus improving algorithm robustness.

Based on the gradient boosting loss function, XGBoost performs a second-order expansion to achieve a faster search for the optimal solution.

The t-th loss function of XGBOOST is defined as Equation (2):

$$ {\mu}^{(t)}\approx \sum \limits_{i=1}^n\left[l\left({y}_i,{\hat{y}}^{\left(t-1\right)}\right)+{g}_i{f}_t\left({X}_i\right)+\frac{1}{2}{h}_i{f}_t^2\left({X}_i\right)\right]+\Omega \left({f}_t\right) $$
(2)

where \( {g}_i={\partial}_{\hat{y}}l\left({y}_i,{\hat{y}}^{\left(t-1\right)}\right) \), \( {h}_i={\partial}_{{\hat{y}}^{\left(t-1\right)}}^2l\left({y}_i,{\hat{y}}^{\left(t-1\right)}\right) \) and Ω(ft) are regular terms.

1.1.3 LightGBM [3]

The lightGBM algorithm is a gradient lifting framework that uses a histogram-based algorithm. Instead of finding the split points on the sorted feature values, the histogram-based algorithm buckets continuous feature values into discrete bins and uses these bins to construct feature histograms during training.

Fig. 7
figure 7

Leaf-wise growth strategy

The white point indicates the leaf with the largest split gain, and the black point is other leaf.

1.1.4 Logistic regression model (LR) [4]

The logistic regression model is a classical classification algorithm commonly used to resolve the binary classification problem. In LR, the sigmoid function presents a relationship between the predicted target variable hθ(x) and input features x, as formulated by Equation (6).

$$ {h}_{\theta }(x)={\left(1+\exp \left({\theta}_0+{\sum}_{i=1}^N{\theta}_i{x}_i\right)\right)}^{-1} $$
(6)

Where θ is a feature weight vector, which is optimized by the stochastic gradient decent algorithm.

1.2 Traditional scoring systems

Five traditional scoring systems were used to compare with machine learning models and these systems and models have been developed and tested on the same set of data. The traditional scoring systems are briefly introduced as follows:

SAPSII [5] assesses the severity of illness in patients who are older than 15 years in the ICU. The evaluation process is based on 17 parameters, e.g., age, HR, SBP, T, and GCS. The measurements should be finished within 24 h of admission to ICU, with integer measurement values between 0 and 163.

SOFA [6] is suitable for tracking the state of patients treated in the ICU. The SOFA score is based on six different scoring systems, which include the respiratory, cardiovascular, liver, coagulation, kidney, and nervous scoring system.

APS [7], which is based on the results of blood tests and vital signs, is widely used for assessment of the condition of patients in the emergency critical care unit (EICU). The higher the APS score is, the higher the mortality risk is.

Modified early warning score (MEWS) [8], which is suitable for ICU patients over 14 years of age is calculated based on five non-invasive parameters including heart rate, systolic blood pressure, breathing, body temperature, and consciousness. The scoring range of each parameter is from 0 to 3. The final score is the sum of each one of the parameter scores. The higher score value indicates the more deteriorated health condition.

The Oxford acute severity of illness score (OASIS)[9], which was developed for ICU patients using a hybrid genetic algorithm and particle swarm optimization approach, was designed to have an extremely low burden for data collection and quality control, using only 14 features without laboratory measurements, diagnosis, or comorbidity information.

Table 6. The statistical result for non-invasive parameters
Table 7. The OPT_subset and the MIN_subset of XGBoost
Table 8 The OPT_subset and the MIN_subset of LightGBM
Table 9 The OPT_subset and the MIN_subset of Random Forest
Table 10 The OPT_subset and the MIN_subset of Logistic Regression

1.3 Parameters weight

Where (parameter)_XX_XX is the derivative variables of parameter, XX_twe_XX, XX_tf_XX, XX_ts_XX, XX_fe_XX is the derivative variables of the time series data of 0~12 h, 12~24 h, 24~36h, 36~48h after admitting to hospital, respectively and XX_XX_(eigenvalue) is the derivative variables about the eigenvalue.

1.4 Further details on feature extraction

References

1. Svetnik V, Liaw A, Tong C, et al (2003) Random Forest: A Classification and Regression Tool for Compound Classification and QSAR Modeling. J Chem Inf Comput Sci 43:1947–1958. https://doi.org/10.1021/ci034160g

2. Chen T, Guestrin C (2016) XGBoost: A Scalable Tree Boosting System. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining - KDD ’16. ACM Press, San Francisco, California, USA, pp 785–794

3. Ke G, Meng Q, Finley T, et al (2017) LightGBM: A Highly Efficient Gradient Boosting Decision Tree. In: Guyon I, Luxburg UV, Bengio S, et al (eds) Advances in Neural Information Processing Systems 30. Curran Associates, Inc., pp 3146–3154

4. Dodek PM, Wiggs BR (1998) Logistic regression model to predict outcome after in-hospital cardiac arrest: validation, accuracy, sensitivity and specificity. Resuscitation 36:201–208. https://doi.org/10.1016/S0300-9572(98)00012-4

5. Le GJ, Lemeshow S, Saulnier F (1993) A new Simplified Acute Physiology Score (SAPS II) based on a European/North American multicenter study. J Am Med Assoc

6. Arts DGT, de Keizer NF, Vroom MB, de Jonge E (2005) Reliability and accuracy of Sequential Organ Failure Assessment (SOFA) scoring. Crit Care Med 33:1988. https://doi.org/10.1097/01.CCM.0000178178.02574.AB

7. Pollack MM, Patel KM, Ruttimann UE (1997) The pediatric risk of mortality III— Acute physiology score (PRISM III-APS): A method of assessing physiologic instability for pediatric intensive care unit patients. J Pediatr 131:575–581. https://doi.org/10.1016/S0022-3476(97)70065-9

8. Moon A, Cosgrove JF, Lea D, et al (2011) An eight year audit before and after the introduction of modified early warning score (MEWS) charts, of patients admitted to a tertiary referral intensive care unit after CPR. Resuscitation 82:150–154. https://doi.org/10.1016/j.resuscitation.2010.09.480

9. Carney C (2011) A New Classification System for Grading the Severity of Onychomycosis: Onychomycosis Severity Index. Arch Dermatol 147:1277. https://doi.org/10.1001/archdermatol.2011.267

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Zhang, G., Xu, J., Yu, M. et al. A machine learning approach for mortality prediction only using non-invasive parameters. Med Biol Eng Comput 58, 2195–2238 (2020). https://doi.org/10.1007/s11517-020-02174-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11517-020-02174-0

Keywords

Navigation