Life Model: A novel representation of life-long temporal sequences in health predictive analytics
Introduction
Predictive analytics in healthcare can prevent patients from having emergency health conditions, reduce the cost of healthcare in the long term, and save lives. The USA budget for healthcare in 2017 is just over a trillion dollars [1]. A 2012 study [2] showed that of acute hospital patients experience discharge delay, which causes delays for other patients, raises the costs, and increases complications. In July 2017, a cohort Canadian study [3] showed that dying risk for patients experiencing emergency surgery delay is 4.9% compared with 3.2% for those without delay. Researchers in prediction analytics have developed some tools to predict hospital readmission rates [4], mortality risks in the hospitals and particularly in the intensive care units (ICUs), and assign severity scores to patients [5], [6]. The next step in this trend is disease diagnosis and anomaly prediction, by which the hospital information system can automatically identify a patient’s diagnosis code and even forecast a disease.
Medical records are time-stamped and form a sequence. Two popular sequence classification methods are either Markovian models or recurrent neural networks. The problem with Markovian models, such as hidden Markov model is that they assume each state is only dependent on the previous state. In long-term health data prediction, this may not be true. Certain life-style and diagnosis in the past may affect a patient’s current diagnosis—for instance, history of certain drug consumption or surgery. Thus, first order Markovian chains do not seem suitable for this type of classification as they ignore long-term correlations. One solution might be using higher order Markovian chains [7]. However, they are known to be complex and computationally expensive as the order increases (e.g., using orders higher than two). Therefore, recurrent neural networks can be an alternative. RNNs are proved to be Turing complete [8] thus seem to be able to handle this task given enough resources. However, the regular RNN cells are shown to be inefficient in remembering long dependencies. Long-term short memories (LTSM) [9] cells instead perform better at remembering past history.
In order to predict from medical history, it is necessary to model life-long sequences of an individual’s health and activity records in order to predict the future health anomalies or diseases. The problem with time-series modeling and activity recognition for long periods of time is the length of the data and the presence of missing values. The first step is usually discretizing the continuous data in order to create fewer time steps for easier processing. In addition to discretization errors [10] in temporal data abstraction, discrete value sequences obtained from historical medical data may require missing value imputation first. Moreover, each data interval (short-term vs long-term) generates a similar sequence length as any other interval in the history which grow linearly as long as the medical history is present. For example, if a person’s medical history for a day is summarized to 1000 time steps, a 6-months history of the person would result in sequence length of 183,000 steps. Consequently, sequence length of patients’ records vary significantly from each other.
The following proposed solution for modeling temporal pattern sequences results in a concise sequence of an individual’s data records, in order to train a deep network efficiently with enough data from a variety of patients samples. Life Model () provides an -bit sequence to represent the historical or the future records as novel intensity temporal sequence (ITS) tensors. algorithms and properties enable ITS to train long-short-term memory (LSTM) RNNs efficiently, in order to predict anomalies and disease diagnosis despite having missing values.
Another objective of this research is to address the challenges in the development of a system that can provide predictive analytics for health monitoring. The proposed -powered predictive health analytics and real-time monitoring schema (PHARMS) promises to provide a solution to improve predictive health analytics via IoT edge devices and wearables. It enables real-time, minimally-invasive, intelligent activity monitoring, and predictive analysis based on various deep learning techniques. It is also the testbed for evaluating the in a cloud environment, using real-world and simulated data.
Different scenarios show how smart health using real-time monitoring and predictive analysis can improve healthcare synergistically. Fig. 1 shows how a remote patient monitoring system can use the -enabled PHARMS to detect and predict anomalies to recover from an emergency condition (here, it predicts a ‘fall’). Fig. 2 shows another example of how a remote dialysis assessment system can benefit from PHARMS to help renal patients avoid early/late visits to hospitals. Combined with real-time monitoring and IoT, severe accidents, such as falls, heart attacks, and seizures, could be prevented with health anomaly prediction. Warning users of complications of a drug, or providing early predictions of a disease, are among the many applications of PHARMS.
The next section covers some necessary background information. In the subsequent sections, the research question, challenges, related works, and the proposed solution are presented.
Section snippets
Background
In disease prediction and health monitoring we are interested in temporal sequence data. Patient records are high dimensional data often recorded during patients visits only. Time-series modeling techniques are not suitable for high dimensional data with irregular intervals. Therefore, other modeling techniques, such as temporal abstractions, are more suitable for medical temporal sequences. The notation and modeling of this abstractions will be further discussed.
Deep neural networks can
Related works
Among researchers in health prediction frameworks, Forkan et al. [15] proposed a cloud-based middleware for ambient assisted living (AAL) called CoCaML. They tested their concept with some performance tests (response time and arrival rate). Later, in another work [16], they proposed a context-aware approach for long-term behavioral change detection and abnormality prediction in AAL, in which they assumed a linear trend model and used the Holt’s linear trend method along with the hidden Markov
Motivation and main contributions
The motivation of this paper is to address some of the challenges in temporal sequences modeling to improve real-time prediction analysis in health monitoring. Some of these challenges are as follows:
- •
Modeling long temporal sequences of sparse health data for prediction Modeling long-term sparse temporal data and training a machine learning model to properly benefit from critical dependencies, and distinguish that information from irrelevant noise, is an open problem. For example, the hourly
Life model (LM)
We are modeling the process of predicting health anomalies and disease diagnosis from past activity and health records. Medical records of a patient, including any past diagnoses, along with a health profile, such as age, gender, and race, constitutes the prior information denoted as . The objective is to predict the probability distribution of anomalies , given the past activities , regarding the patient’s profile :
Not all learning algorithms can estimate this model. In a
Experimental results
Several experiments have been conducted and case studies are provided as a proof of concept. This section covers an introduction about the datasets and then several applications and experimental results are provided.
Discussion
LM opens the door to many predictive analytics areas, healthcare in particular, by addressing the challenge of mapping long-term periods to concise representations. For example, in healthcare, LM can be used for disease diagnosis, anomaly prediction, cancer and post-surgery monitoring, and in-home care. It can also be applied in lifestyle planning (e.g., career planning, investment, fitness, weight management, and parenting) and in management (e.g., team and risk management, project planning,
Conclusion and future works
The proposed solution, , provides a concise sequence to represent the history or future, using the novel ITS tensors. LM algorithms and properties enable ITS tensors to train LSTM networks efficiently in order to predict anomalies and diagnosis from long historical records, even in the absence of some values.
LM is used to predict mortality of 10,000 patients from MIMIC III dataset based on their diagnosis and procedures codes. The results show improvement in the model trained by LM-mapped
Acknowledgment
The authors want to thank Microsoft Research for providing two years of Azure for Research Award for using Microsoft cloud services.
Alireza Manashty is a data scientist, and a researcher in big health data and cloud. He is a computer science Ph.D. candidate at the University of New Brunswick, Canada. He received his M.Sc. in Artificial Intelligence and B.Sc. in Software Engineering in 2012 and 2010, respectively. Ali is also a Microsoft MVP in Azure for his community work and blog. He has published several conference and journal papers and a book chapter in cloud computing. His areas of research expertise include machine
References (48)
- et al.
Turing computability with neural nets
Appl. Math. Lett.
(1991) - et al.
CoCaMAAL: A cloud-oriented context-aware middleware in ambient assisted living
Future Gener. Comput. Syst.
(2014) - . US Government, Healthcare Budget US 2017. URL...
- et al.
Delay in discharge and its impact on unnecessary hospital bed occupancy
BMC Health Serv Res
(2012) - et al.
Association of delay of urgent or emergency surgery with mortality and use of health care resources: A propensity scorematched observational cohort study
CMAJ
(2017) - . KenSci, KenSci: Predictive Risk Management Platform for Healthcare Powered by Machine Learning. URL...
- et al.
Mortality Prediction and Acuity Assessment in Critical Care
(2014) - J.E. Zimmerman, A.A. Kramer, D.S. Mcnair, F.M. Malila, Acute Physiology and Chronic Health Evaluation (APACHE) IV:...
Further applications of higher-order Markov chains and developments in regime-switching models
(2012)- K. Greff, R.K. Srivastava, J. Koutnik, B.R. Steunebrink, J. Schmidhuber, LSTM: A Search Space Odyssey, 2016....
Human activity recognition from wireless sensor network data : Benchmark and software
Act. Recognit. Pervasive Intell. Environ.
On the computational power of neural nets
Comput. Complexity
Deep Learning
The vanishing gradient problem during learning recurrent neural nets and problem solutions
Int. J. Uncertain. Fuzziness Knowl.-Based Syst.
A context-aware approach for long-term behavioural change detection and abnormality prediction in ambient assisted living
Pattern Recognit.
Deep patient: An unsupervised representation to predict the future of patients from the electronic health records
Sci. Rep.
Directly modeling missing data in sequences with RNNs: Improved classification of clinical time series
Mach. Learn. Healthc.
An efficient pattern mining approach for event detection in multivariate temporal data
Knowl. Inf. Syst.
The effect of threshold values and weighting factors on the association between entropy measures and mortality after myocardial infarction in the Cardiac Arrhythmia suppression trial (CAST)
Entropy
Cited by (8)
Prediction of superheated steam temperature for thermal power plants using a novel integrated method based on the hybrid model and attention mechanism
2022, Applied Thermal EngineeringCitation Excerpt :In addition, the components of LSTM, how they interact, and the evaluation methods of learnable parameters are extensively described [55,56]. Due to its high applicability and popularity, this neural structure as a state-of-the-art model is commonly used in various fields, like building energy usage [57], carbon emission levels [58], wind speed forecasting [58] and health predictive analytics [59], etc. Attention-based RNN is mainly utilized to optimize sequence-to-sequence models such as natural language processing [45] and process modeling [63].
Revising recurrent neural networks from a granular perspective
2019, Applied Soft Computing JournalCitation Excerpt :Recurrent neural networks (RNNs) are able to approximate dynamical systems with arbitrary accuracy, offering a mapping between any input and output sequence [1]. They found several applications in different fields; just to mention: for predictive analytics in healthcare [2], for target recognition [3], to predict turbine engine vibration [4], for language modelling [5], for modelling and analysis of squeeze casting process [6], for the energy efficiency problem [7], for the software reliability prediction [8]. In RNNs each hidden state is a function of all previous hidden states and then they can be regarded as inherently deep in time.
Behavior Intention of Chronic Illness Patients in Malaysia to Use IoT-based Healthcare Services
2023, International Journal of Advanced Computer Science and ApplicationsA Systematic Literature Review of Enabling IoT in Healthcare: Motivations, Challenges, and Recommendations
2022, Electronics (Switzerland)Survey of Testing Methods and Testbed Development Concerning Internet of Things
2022, Wireless Personal CommunicationsDeepMDP: A Novel Deep-Learning-Based Missing Data Prediction Protocol for IoT
2021, IEEE Internet of Things Journal
Alireza Manashty is a data scientist, and a researcher in big health data and cloud. He is a computer science Ph.D. candidate at the University of New Brunswick, Canada. He received his M.Sc. in Artificial Intelligence and B.Sc. in Software Engineering in 2012 and 2010, respectively. Ali is also a Microsoft MVP in Azure for his community work and blog. He has published several conference and journal papers and a book chapter in cloud computing. His areas of research expertise include machine learning, deep learning, computer vision, smart health monitoring and cloud computing.
Janet Light, Ph.D., former WIE chair in NB, is a Professor in the Department of Computer Science, University of New Brunswick, Saint John, Canada. She received her Bachelor degree in Electronics and Communications Engineer- ing in 1983, Masters in Electrical and Electronics Engineering in 1990 and Ph.D. in Computer Science in 2002 in India. Her research interests are wireless and mo- bile computing, ubiquitous computing, sensor networks, network traffic study, and security. For the past 12–17 years, her research focus has been on applied health.