Elsevier

Expert Systems with Applications

Volume 135, 30 November 2019, Pages 237-248
Expert Systems with Applications

Probabilistic grammar-based neuroevolution for physiological signal classification of ventricular tachycardia

https://doi.org/10.1016/j.eswa.2019.06.012Get rights and content

Highlights

  • A deep neuroevolution system is presented.

  • Our system is based on Probabilistic Model Building Genetic Programming approach.

  • Deep neural networks discovered by the system can diagnose heart rhythm problems.

  • Performance of the proposed network is evaluated on a physiological dataset.

  • Our network can outperform several machine learning algorithms in diagnosis.

Abstract

Ventricular tachycardia is a rapid heart rhythm that begins in the lower chambers of the heart. When it happens continuously, this may result in life-threatening cardiac arrest. In this paper, we apply deep learning techniques to tackle the problem of the physiological signal classification of ventricular tachycardia, since deep learning techniques can attain outstanding performance in many medical applications. Nevertheless, human engineers are required to manually design deep neural networks to handle different tasks. This can be challenging because of many possible deep neural network structures. Therefore, a method, called ADAG-DNE, is presented to automatically design deep neural network structures using deep neuroevolution. Our approach defines a set of structures using probabilistic grammar and searches for best network structures using Probabilistic Model Building Genetic Programming. ADAG-DNE takes advantages of the probabilistic dependencies found among the structures of networks. When applying ADAG-DNE to the classification problem, our discovered model achieves better accuracy than AlexNet, ResNet, and seven non-neural network classifiers. It also uses about 2% of parameters of AlexNet, which means the inference can be made quickly. To summarize, our method evolves a deep neural network, which can be implemented in expert systems. The deep neural network achieves high accuracy. Moreover, it is simpler than existing deep neural networks. Thus, computational efficiency and diagnosis accuracy of the expert system can be improved.

Introduction

Cardiovascular disease is the leading cause of death among people. According to the report released by the American Heart Association (Benjamin et al., 2018), cardiac arrest as an underlying cause of death in 2015 was 17668; any-mention mortality in 2015 was 366807. If patients suffer from endstage renal disease (a type of kidney disease), arrhythmias (i.e. heart rhythm disorder) and sudden cardiac death accounts for nearly 40% of the deaths (Benjamin et al., 2018). In this paper, we are focusing on the diagnosis of ventricular tachycardia in Intensive Care Unit (ICU). Ventricular tachycardia is a heart arrhythmia initiated by abnormal electrical signals in the lower chambers of the heart (Berbari, Scherlag, Hope, Lazzara, 1978, Bradfield, Boyle, Shivkumar, 2017, Uther, Dennett, Duffy, Freedman, Tan, 1979). An ICU in a hospital is a facility dedicated to providing life support and monitoring in patients who are critically ill, for instance, life-threatening illness, injuries, and multiple organ failures. ICU care is important to other medical services, including surgery (World Health Organization, 2003), or care for patients with complications of diseases (Baker, 2009). Higher quality of ICU will also increase citizen confidence in the health care system (Riviello, Letchford, Achieng, & Newton, 2011). In clinical management of ICU, real-time physiological measurement systems help clinicians to continually monitor the physiological status of patients. For example, pulse oximeter provides the oxygen saturation values and shows the plethysmographic waveform of the pulse signal over time (Shamir, Eidelman, Floman, Kaplan, & Pizov, 1999). A report on 2016 Get With The Guidelines programs shows that the location of adult in-hospital cardiac arrest was 53.7% in the ICU, operating room, or emergency department (Benjamin et al., 2018). Among 16.1% of these incidents, the initial recorded cardiac rhythm was ventricular fibrillation, ventricular tachycardia, or shockable (Benjamin et al., 2018). Therefore, there is a monitoring system in an ICU to detect if ventricular tachycardia happens. When ventricular tachycardia happens continuously, this may result in life-threatening cardiac arrest. When a life-threatening situation is detected, an alarm will be raised. Sometimes, an alarm may not be clinically significant. We call this a false alarm. False alarms in the ICU decreases the quality of care due to the noise disruptions and desensitization to warnings (Donchin & Seagull, 2002). This can lengthen the stay for recovery and increase the load on the health care system (Chambrin, 2001). It is also reported that less than 10% of alarms are associated with therapeutic modification (Chambrin, Ravaux, Calvelo-Aros, Jaborska, Chopin, Boniface, 1999, Lawless, 1994). Can the accuracy of the system be improved?

Recently, Deep Neural Network (DNN) is a powerful machine learning technique and attains outstanding performance in many applications. It has been applied in many medical applications, such as annotation of mitosis in breast cancer histology images (Cireşan, Giusti, Gambardella, & Schmidhuber, 2013), identification of skin lesions (Esteva et al., 2017), and detection of standard scan plane during fetal abnormality screening (Baumgartner et al., 2016). There has been much interest in applying DNN in other medical applications. Even a slight improvement in the performance implies an early diagnosis of diseases which can be critical to the treatment of patients. We believe that the physiological status of patients can also be diagnosed using DNN.

However, manually configuring of a DNN is complicated. Because the large search space, which is called the design space, of DNN covers many aspects, such as the topology of the network and the learning parameters. Properly designing DNN can be complicated to human engineers, medical experts, or other people who are knowledgeable in other application areas. Secondly, an expert may have data collected from a group of heterogeneous sensors. Understanding the nature of the signals from these sensors and performing feature engineering can be time-consuming. In the early stage of the study, it may be better to use automatic learning approaches on these signals and prioritize the signal analysis afterwards. This can increase the productivity of human engineers and medical experts. In this paper, we explore a physiological application of Deep Neuroevolution (DNE) to the automatically design of Convolutional Neural Network (CNN), which has not been done in the past.

The main contribution of this paper is that we propose a new procedure to evolve DNN structures from a set of convolutional layers, which are called modules, using Probabilistic Model Building Genetic Programming (PMBGP) approach. Theoretically, it is demonstrated that simpler and better DNNs can be evolved by evolutionary computation. Moreover, we also showed that structural dependencies within components in DNN can be encoded in grammar and then used by DNE. This enhances the search space modeling of DNNs during DNE. Finally, it is demonstrated that DNE is applicable to, not only image classification, but also physiological signal classification. A set of DNN structures are specified by a set of rules in Probabilistic Context-Sensitive Grammar (PCSG) in PMBGP. Each rule tells how a network structure is formed, how modules are connected, and which combinations of modules are preferred. In each iteration, the rules are updated iteratively based on the feedback from the fitness evaluation to guide evolution. There are several advantages to this approach. First of all, it explicitly models the preference for combinations among different modules via probabilistic dependencies in the set of DNNs. The probabilistic dependencies are captured by a set of Bayesian networks associated with every rule. The task to train a DNN is computationally expensive. As PMBGP approach can automatically learn the rules to compose the network modules, it can reduce the number of times of training DNNs with inappropriate structures. Secondly, since the DNN is represented in grammar, it is now possible to discover new forms of regularities and extract new traits to better optimize the DNN structures. The system can learn and decide what components to be inserted using context, i.e. information about the location of a new component concerning other existing components in a network structure. The context information can be nicely incorporated in PCSG while this is also the first study to apply PCSG and PMBGP on DNE. Experts in DNN can study the patterns and learn from the evolved design.

Section snippets

Neuroevolution

The development of neuroevolution began in the late 1980s (Montana & Davis, 1989). There are three aspects of Artificial Neural Network (ANN) and DNN which can be evolved: 1) evolution of connection weights; 2) evolution of architectures; 3) evolution of learning algorithms. The evolution of these configurations has been proposed in neuroevolution of ANN and reviewed in Yao (1999). These configurations can be controlled through hyperparameters and meta-heuristics. Evolution of connection

Deep neural network structure search space

A DNN model contains a DNN structure of several DNN modules and a set of weights. A DNN topology is a specification describing a set of DNN structures of interest. This specification is described using a grammar and a translation program. A translation program converts a parse tree, which can also be called an individual, derived from the grammar to a DNN structure.

In this paper, structures of DNN are built from eight network modules. An input module casts the input data to a specific dimension

Deep neural network topology

In this section, the grammar, the meaning of context, and the translation program are explained.

Adaptive grammar-based deep neuroevolution

ADAG-DNE system integrates the components developed in the previous sections into one system. As shown in Fig. 3, it involves five steps.

  • 1.

    Deriving DNN structures encoded in parse trees according to the grammar;

  • 2.

    Translating the parse trees into Python code;

  • 3.

    Evaluating the performance of the network using data;

  • 4.

    Collecting samples from the parse trees of the set of good networks (by their ranking);

  • 5.

    Updating Bayesian networks in the grammar.

Steps 1–5 are repeated until it reaches the maximum number of

Data set

PhysioNet is a research resource for complex physiological signals (Goldberger et al., 2000). It provides data sets and software for the physiological signal processing. In this paper, a bedside monitor data set collected from four hospitals in the USA and Europe was downloaded. The physiological measures from electrocardiography leads and pulse oximetry were used in the experiment. There are 310 records. Each record lasts 20 s long and contains 5000 features (time points) in total. Our goal is

Evaluation: comparison with other classifiers

The ADAG-DNE using Depth and Term context variables1 was adopted using a population of size 30 for 50 generations for 10 runs. The records in the dataset are divided into three parts while maintaining the class distribution to be similar. 30% of records are reserved for testing. Training and

Evaluation: comparison among different variants

In this section, we analyze the performance of all combinations of context variables in the ADAG-DNE system. Each combination of context variables was executed for 10 runs. The system evolved a population of size 30 for 50 generations for every run. Training accuracy of a network structure is used as its fitness value. The depth-based, rule-based and non-terminal-based context variables are abbreviated by D, R, and T respectively. For example, the variant ADAG-DNE/D means that only depth-based

Findings and implications

Cardiovascular disease is the leading cause of death among people. Since the problem of detection of abnormal heart rhythm is prevalent, ADAG-DNE provides a viable solution to design suitable CNNs for ECG signal classification. From our results, we found that ADAG-DNE is slightly better than AlexNet-A, AlexNet-B, and ResNet18 regarding accuracy, but the number of parameters used in ADAG-DNE is only 0.05% of that in ResNet18. We also showed that the performance of the learnt model is also better

Conflict of interest

No conflict of interest.

CRediT authorship contribution statement

Pak-Kan Wong: Conceptualization, Methodology, Software, Data curation, Visualization, Software, Writing - original draft, Writing - review & editing. Kwong-Sak Leung: Funding acquisition, Supervision, Writing - review & editing. Man-Leung Wong: Conceptualization, Supervision, Project administration, Resources.

Acknowledgments

This research is supported by the Lingnan University Direct Grant DR16A7.

References (102)

  • Y. Shan et al.

    Program evolution with explicit learning

    Proceedings of the 2003 congress on evolutionary computation

    (2003)
  • K.O. Stanley

    Compositional pattern producing networks: A novel abstraction of development

    Genetic Programming and Evolvable Machines

    (2007)
  • J. Uther et al.

    Detection of delayed activation potentials of potentials of low amplitude in the vectorcardiogram of patients with recurrent ventricular tachycardia

    Australian and New Zealand journal of medicine

    (1979)
  • X. Yao

    Evolving artificial neural networks

    Proceedings of the IEEE

    (1999)
  • H.A. Abbass et al.

    AntTAG: A new method to compose computer programs using colonies of ants

    Proceedings of the 2002 congress on evolutionary computation

    (2002)
  • U.R. Acharya et al.

    Automated characterization of arrhythmias using nonlinear features from tachycardia ECG beats

    Proceedings of 2016 IEEE international conference on systems, man, and cybernetics

    (2016)
  • F. Alonso-Atienza et al.

    Detection of life-threatening arrhythmias using feature selection and support vector machines

    IEEE Transactions on Biomedical Engineering

    (2014)
  • A. Amann et al.

    Reliability of old and new ventricular fibrillation detection algorithms for automated external defibrillators

    Biomedical Engineering Online

    (2005)
  • M. Arif et al.

    Detection and localization of myocardial infarction using k-nearest neighbor classifier

    Journal of Medical Systems

    (2012)
  • Assunção, F., Lourenço, N., Machado, P., & Ribeiro, B. (2018). DENSER: Deep evolutionary network structured...
  • T. Baker

    Critical care in low-income countries

    Tropical Medicine & International Health

    (2009)
  • K. Balasundaram et al.

    A classification scheme for ventricular arrhythmias using wavelets analysis

    Medical & Biological Engineering & Computing

    (2013)
  • A. Baldominos et al.

    Evolutionary convolutional neural networks: An application to handwriting recognition

    Neurocomputing

    (2017)
  • A. Baldominos et al.

    Evolutionary design of convolutional neural networks for human activity recognition in sensor-rich environments

    Sensors

    (2018)
  • C.F. Baumgartner et al.

    Real-time standard scan plane detection and localisation in fetal ultrasound using fully convolutional neural networks

    International conference on medical image computing and computer-assisted intervention

    (2016)
  • Y. Bengio et al.

    Curriculum learning

    Proceedings of the 26th annual international conference on machine learning

    (2009)
  • E.J. Benjamin et al.

    Heart disease and stroke statistics—2018 update: A report from the American Heart Association

    Circulation

    (2018)
  • J.S. Bradfield et al.

    Ventricular arrhythmias

  • L. Breiman

    Random forests

    Machine Learning

    (2001)
  • D.S. Broomhead et al.

    Radial basis functions, multi-variable functional interpolation and adaptive networks

    (1988)
  • M.-C. Chambrin

    Alarms in the intensive care unit: How can the number of false alarms be reduced?

    Critical Care

    (2001)
  • M.-C. Chambrin et al.

    Multicentric study of monitoring alarms in the adult intensive care unit (ICU): a descriptive analysis

    Intensive Care Medicine

    (1999)
  • F. Chollet

    Keras

    (2015)
  • D.C. Cireşan et al.

    Mitosis detection in breast cancer histology images with deep neural networks

    International conference on medical image computing and computer-assisted intervention

    (2013)
  • J. Dai et al.

    R-FCN: Object detection via region-based fully convolutional networks

  • D. Dasgupta et al.

    Designing application-specific neural networks using the structured genetic algorithm

    Proceedings of the international workshop on combinations of genetic algorithms and neural networks.

    (1992)
  • Y. Donchin et al.

    The hostile environment of the intensive care unit

    Current Opinion in Critical Care

    (2002)
  • R. Dzwonczyk et al.

    The median frequency of the ECG during ventricular fibrillation: Its use in an algorithm for estimating the duration of cardiac arrest

    IEEE Transactions on Biomedical Engineering

    (1990)
  • J.L. Elman

    Finding structure in time

    Cognitive Science

    (1990)
  • A. Esteva et al.

    Dermatologist-level classification of skin cancer with deep neural networks

    Nature

    (2017)
  • Y. Freund et al.

    Experiments with a new boosting algorithm

    Proceedings of the 13th international conference on machine learning

    (1996)
  • D. Gao et al.

    Bayesian ANN classifier for ECG arrhythmia diagnostic system: A comparison study

    Proceedings of 2005 IEEE international joint conference on neural networks, 2005.

    (2005)
  • A.L. Goldberger et al.

    Physiobank, physiotoolkit, and physionet

    Circulation

    (2000)
  • F. Gomez et al.

    Incremental evolution of complex general behavior

    Adaptive Behavior

    (1997)
  • F. Gruau et al.

    Adding learning to the cellular development of neural networks: Evolution and the baldwin effect

    Evolutionary Computation

    (1993)
  • N. Hansen et al.

    Completely derandomized self-adaptation in evolution strategies

    Evolutionary Computation

    (2001)
  • Y. Hasegawa

    Programming with annotated grammar estimation

  • Y. Hasegawa et al.

    A Bayesian network approach to program generation

    IEEE Transactions on Evolutionary Computation

    (2008)
  • Y. Hasegawa et al.

    Latent variable model for estimation of distribution algorithm based on a probabilistic context-free grammar

    IEEE Transactions on Evolutionary Computation

    (2009)
  • M. Hauschild et al.

    An introduction and survey of estimation of distribution algorithms

    Swarm and Evolutionary Computation

    (2011)
  • View full text