Deep learning for predicting respiratory rate from biosignals
Introduction
Bio-signals are activity-based signals generated by physiological processes of subsystems of living organisms [1]. These signals have a variety of sources which are bio-acoustic signals, bio-impedance signals, electrical conductance, bio-magnetic signals, bio-mechanical signals, and bio-electrical signals [2]. Some examples of biomedical signals are electroneurogram (ENG), electromyogram (EMG), vibromyogram (VMG), otoacoustic emission (OAE) [2], mechanomyogram (MMG), photo-plethysmogram (PPG) [3], electrocardiogram (ECG), phonocardiogram (PCG), electroencephalogram (EEG), electrogastrogram (EGG), galvanic skin response (GSR) [4], vibro-arthogram (VAG) [2,4], electro-oculogram (EOG) [2,3] and electroretinogram (ERG) [5]. Electromyography measures the bio-electrical activity of the muscles [2,3]. EMG signal can be acquired by invasive or non-invasive methods. Surface EMG (sEMG) is a non-invasive approach for measuring muscles activity and have many applications like prosthetic hand control, gesture recognition, human-computer interface (HCI), respiratory monitoring, fall detection, and gait analysis. Photoplethysmography (PPG) is used to detect volumetric changes in blood using optical techniques [6]. Electrocardiogram (ECG) is a very commonly used biomedical signal which measures the contractile cardiac activities [2]. ECG and PPG are mostly used in clinical applications such as estimating respiratory rate [7] and blood pressure [6], measuring cardiac cycles and for remote monitoring.
Deep learning methods are increasingly becoming prominent for biomedical applications such as medical image analysis [8], bio-signals [[9], [10], [11], [12]], health monitoring [13] and clinical decision support systems [11,14]. Recurrent neural networks (RNNs) are prominent deep learning methods that are suited for time series and sequential data [15]. Long short term memory (LSTM) network [16] is an extension of simple RNNs which have been prominent in modeling temporal sequences. Convolutional neural network (CNNs) are widely used in signal processing and computer vision tasks such as EEG based brain-computer interface [17], blood oxygenation level dependent signals [18], face recognition [19,20], hand gesture recognition [21] and image segmentation [22,23].
Health monitoring of individuals’ has become easier with the introduction of wearable biosensors [4]. Human body systems such as the cardiovascular system, pulmonary system, neural system, and neuro-muscular systems are often monitored. There has been a lot of progress in the use of deep learning models for bio-signal processing. Saadatnejad et al. [24] demonstrated that integrating wavelet transform with LSTM models can accurately predict R-R interval in ECG. ECG is composed of many types of waveforms, and R type waveform is one of the most important features in ECG signals. The interval between two R waves in ECG is called the R-R interval. Myocardial infarction has been classified from ECG signals using CNN [25] and hybrid LSTM-CNN models [26,27]. Abnormal beats in ECG have been predicted with LSTM [28], CNN [29], and convolutional LSTM [30]. Moreover, CNN has been used with optimization algorithm [31] to classify different types of heart disease, where peaks in R-R have been detected. Different CNN models, such as densely connected network (DenseNet), residual network (ResNet) and Xception has been used for identifying time-frequency features of ECG signals [32]. EEG bio-signals detect brain activity via electric impulses of the brain. CNN is employed in motor imagery EEG where the features are learned from local receptive fields [17]. High accuracy is obtained in the work by using CNN.
It is vital to monitor respiratory systems parameters [33] since it discloses the respiratory pattern and fraction of inspired oxygen. The lung function test is the most common method to get parameters related to respiratory dynamics. Some key instruments used for measuring parameters related to respiration are pulse oximeter, spirometer, plethysmogram, photoplethysmogram (PPG), EMG, and ECG [34]. Respiratory rate is essential in deducing breathing patterns (normal, abnormal pattern) for clinical decision [14]. The respiratory pattern varies for different activities and different subjects [35]. Predicting respiratory rate is important for situations like labor pain [36], and anxiety. The precision of respiratory rate is dependent on the method of measurements [34]. Special cameras have also been used for monitoring respiration [37,38] as it protects privacy issues [39] unlike contact-based monitoring. Deep learning methods such as CNN's have been applied to spectrogram sequences from a thermal camera to detect respiratory rate [37]. Bidirectional-LSTM model has been applied to support the radiotherapy treatment for thoracic-abdominal tumors by analyzing respiratory motions [40]. LSTM has been combined with CNN to determine respiratory actions during sleep [41].
Estimating precise respiratory rate is challenging especially using wearable sensors [42,43]. The respiratory pattern discloses information about underlying diseases [9]. Monitoring respiratory rate and predicting respiratory values correctly will reduce alert fatigue in the clinical decision support system [44]. Accurate and precise prediction of respiratory rate using wearable bio-sensors such as EMG can help in continuous monitoring of respiratory activity. It is also cost-effective and can provide clinical support effectively. Pulse-oximeters used for measuring respiratory rate are less precise and informative to provide the best clinical support since the region for measurement is not closer to respiratory muscles [34]. The prediction and deep feature extraction of bio-impulses in the human body are motivated and inspired by evolving deep learning techniques. The bio-signals from a particular human body system are thus accompanied by interference from other human body systems. It is challenging to vividly differentiate between the signals and true parameters of an organ especially using non-invasive methods [45]. Estimating breathing rate with low error using diaphragmatic sEMG is challenging. Deep learning can identify interference's especially in breathing pattern recognition [46] and can be thus used for sEMG based applications.
In this paper, we provide a framework for predicting respiratory rate and one step ahead respiratory impedance using deep learning methods such as long short term memory (LSTM) networks, convolutional neural networks (CNN), CNN-LSTM and LSTM with attention networks. Our goal is to predict respiratory rate using data from three different bio-sensors and determine the reliability of sensors in respiratory monitoring applications. Most respiratory rate estimations have an error of more than 1 s for a larger window. In this work, we show the efficacy of deep learning models over two separate windows and three separate data involving 3 different types of bio-signals. We evaluate the aforementioned deep learning models to predict respiratory rate using three different sources of time-series bio-signal data and disclose the best prediction model for similar applications. It will also inform on the accuracy of the bio-sensors in measuring respiratory parameters. We use ECG-PPG from the Capnobase dataset [47], and medical information mart for intensive care (MIMIC II) dataset [48] to predict the respiratory rate. We then apply the models to the diaphragm sEMG dataset [49] for respiratory rate prediction. Seven deep learning models namely, LSTM, CNN, convolutional LSTM, bidirectional LSTM, CNN-LSTM, LSTM with attention and bidirectional LSTM with attention are used with datasets consisting of bio signals of different breathing patterns. Low root mean square error (RMSE) and mean absolute error (MAE) is recorded for all deep learning models. The effectiveness of the algorithm is tested for a 32 s window and 64 s window.
The rest of the paper is organized as follows. In Section 2, we present a background and literature review of related methods for prediction with bio-sensors, especially in the medical field. The motivation of the work is presented in section 3. Section 4 presents the proposed methodology, followed by experiments and results in Section 5. Section 6 provides a discussion and Section 6 concludes the paper with directions for future work.
Section snippets
Related work
The medical field has constantly adapted to deep learning methods. Machine learning is proven effective in detecting and predicting coronavirus disease (COVID) [[50], [51], [52], [53], [54], [55], [56], [57], [58]]. Recurrent neural networks and deep learning models have evolved in the past decades. Some of the traditional architectures of simple RNNs are Jordan RNN [59], Elman RNN [60], and echo state networks [61]. Simple RNNs faced challenges due to the vanishing gradient problem during
Respiratory patterns
Breathing is sequential and it has patterns associated to it. Two common patterns related to pulmonary impedance are restrictive breathing pattern and obstructive breathing pattern. The common difference is that in obstructive breathing, the subjects have difficulty exhaling all the air from the lungs while in restrictive breathing, the patients have difficulty in fully expanding the lungs. Obstructive breathing patterns are found in subjects with chronic obstructive pulmonary disease (COPD),
Data
In our approach, we use three independent data sets which are publicly available. First, we use the Capnobase dataset [47] which contains the partial pressure of inhaled and exhaled carbon dioxide (CO2) obtained via capnogram. The Capnobase dataset also has PPG and reference ECG signal from 42 subjects integrated with Capnogram features. Second is the BIDMC dataset, where the signals and numeric data are extracted from MIMIC II waveform database using impedance respiratory signal [48]. The
Results
In Table 3, we report the MAE and RMSE for predicted respiratory flow (impedance prediction). After the conversion of the respiratory signal by peak detection, we estimate the MAE for respiratory rate for both 32s and 64s window, shown in Table 4. Our choice of datasets, Capnobase and MIMIC-II has been used in many works (86; 89?; 134). Table 5 presents a comparison between our models and previous works.
Capnobase dataset focuses on the prediction of carbon dioxide exhalation rate whereas MIMIC
Discussion
The results mentioned in the previous section are analyzed. We visualise the model performances, by looking into the respiratory signal tracking for the test subjects in each of the datasets (Fig. 5, Fig. 6, Fig. 7) and analyse it based on the respiratory patterns and its characteristics given in Fig. 1, Fig. 2 sEMG data also consider respiratory flow signals and has patterns similar to a spirometer signal.
Comparing the breath signals of subjects from three datasets with reference to breathing
Conclusion and future work
We evaluated deep learning models for respiratory signals from three independent sources of data for respiratory rate prediction. One step ahead prediction is used for surface EMG, PPG, and ECG data. Two sets of windows were used to evaluate performance and we notice that the 64-s window gives a better estimate of respiratory rates in breaths per minute. The results show that LSTM and its variants have the best performance. Deep learning models used in predicting respiratory flow and estimating
Data and code
We provide Python-based open source code and data for further research.1
Declaration of competing interest
No conflict of interest.
References (135)
- et al.
Advances in biomedical signal and image processing – a systematic review
Inf. Med. Unlocked
(2017) Derivation of respiration rate from ambulatory ECG and PPG using ensemble empirical mode decomposition: comparison and fusion
Comput. Biol. Med.
(2017)- et al.
A survey on deep learning in medical image analysis
Med. Image Anal.
(2017) - et al.
Interpreting deep learning models for epileptic seizure detection on EEG signals
Artif. Intell. Med.
(2021) - et al.
Deep learning methods for screening patients' S-ICD implantation eligibility
Artif. Intell. Med.
(2021) Deep learning in neural networks: an overview
Neural Network.
(2015)- et al.
Classification of ECG arrhythmia using recurrent neural networks
Procedia Comput. Sci.
(2018) - et al.
The effectiveness of breathing patterns to control maternal anxiety during the first period of labor: a randomized controlled clinical trial
Compl. Ther. Clin. Pract.
(2017) - et al.
A new COVID-19 detection method from human genome sequences using CpG island features and KNN classifier, Engineering Science and Technology
Int. J.
(2021) Serial order: a parallel distributed processing approach
Adv. Psychol.
(1997)
Finding structure in time
Cognit. Sci.
A review of hidden Markov models and recurrent neural networks for event detection and localization in biomedical signals
Inf. Fusion
Forecasting Indonesia exports using a hybrid model ARIMA-LSTM
Data fusion for estimating respiratory rate from a single-lead ECG
Biomed. Signal Process Control
Introduction to Biomedical Signals
Biomedical Sensors and Data Acquisition
Clinical Significance of Pattern Electroretinogram and Retinal Nerve Layer Thickness in the Diagnosis of Glaucoma
Photoplethysmography and its application in clinical physiological measurement
Physiol. Meas.
Breathing rate estimation from the electrocardiogram and photoplethysmogram: a review
IEEE Rev. Biomed. Eng.
Deep learning for time series classification: a review
Data Min. Knowl. Discov.
Deep learning on 1-D biosignals: a taxonomy-based survey
Yearb. Med. Inf.
Long short-term memory
Neural Comput.
A simplified CNN classification method for MI-EEG via the electrode pairs signals
Front. Hum. Neurosci.
A multichannel 2D convolutional neural network model for task-evoked fMRI data classification
Comput. Intell. Neurosci.
Design of a CNN face recognition system dedicated to blinds
Back, Face recognition: a convolutional neural-network approach
IEEE Trans. Neural Network.
Hand gesture recognition using compact CNN via surface electromyography signals
Sensors
Fully convolutional neural networks for volumetric medical image segmentation
Image segmentation using convolutional neural network
Int. J. Sci. Technol. Res.
LSTM-based ECG classification for continuous monitoring on personal wearable devices
IEEE J. Biomed. Health Inf.
Performance of a convolutional neural network derived from an ECG database in recognizing myocardial infarction
Sci. Rep.
Classification of ECG Signals Based on LSTM and CNN
CNN-LSTM based model for ECG arrhythmias and myocardial infarction classification
Adv. Sci. Technol. Eng. Syst. J.
A study on arrhythmia via ECG signal classification using the convolutional neural network
Front. Comput. Neurosci.
Non-contact heartbeat detection by heartbeat signal reconstruction based on spectrogram analysis with convolutional LSTM
IEEE Access
Intellectual heartbeats classification model for diagnosis of heart disease from ECG signal using hybrid convolutional neural network with
GOA
Ensemble Deep Learning Models for ECG-Based Biometrics, 2020 Cybernetics Informatics
The importance of respiratory rate monitoring
Br. J. Nurs.
Current clinical methods of measurement of respiratory rate give imprecise values
ERJ Open Res.
Tidal breathing patterns derived from structured light plethysmography in COPD patients compared with healthy subjects, Medical Devices (Auckland
N. Z. For.
Deepbreath: deep learning of breathing patterns for automatic stress recognition using low-cost thermal imaging in unconstrained settings
Contactless monitoring of breathing patterns and respiratory rate at the pit of the neck: a single camera approach
J. Sensor. 2018
Non-contact respiration monitoring and body movements detection for sleep using thermal imaging
Sensors
A feasibility of respiration prediction based on deep Bi-LSTM for real-time tumor tracking
IEEE Access
Smartwatch based respiratory rate estimation during sleep using CNN/LSTM neural network
A robust fusion model for estimating respiratory rate from photoplethysmography and electrocardiography
IEEE (Inst. Electr. Electron. Eng.) Trans. Biomed. Eng.
Probabilistic Estimation of Respiratory Rate from Wearable Sensors
A data-driven clinical decision support system for the diagnosis of sleep apneas
Cited by (29)
Software defined radio frequency sensing framework for Internet of Medical Things
2024, Information FusionPhysics-informed neural entangled-ladder network for inhalation impedance of the respiratory system
2023, Computer Methods and Programs in BiomedicineOpen-source software for respiratory rate estimation using single-lead electrocardiograms
2024, Scientific Reports