PCG classification through spectrogram using transfer learning
Graphical abstract
Introduction
Electro-cardiogram (ECG) and Phono-cardiogram (PCG) are the two most commonly employed modalities for the diagnosis of cardiovascular diseases (CVDs), the major cause of mortality in the world [1]. ECG and PCG record, respectively, the electrical activity and sounds produced as a result of beating of the human heart. ECG is generally collected using 12 electrodes for clinical use and PCG is recorded using digital stethoscope. In general, PCG is preferred over ECG due to its ease of use. Moreover, it also provides additional information as during one cycle of heart beat, two sounds are generated corresponding to one major peak as shown in Fig. 1. However, being an acoustic signal, the spectral contents of heart sound are overlapped by multiple sources like noise, additional heart sounds, respiration sounds from lungs and so on. PCG signal can be modeled by the relation given in Eq. (1). In the above equation, is the noise-free cyclo-stationary PCG signal, is the noise due to external sources like body motion etc., while is the contribution due to abnormal heart related sounds like murmur, extra heart sounds (S3, S4) and so on. is the resultant composite PCG signal. Fig. 2 illustrates the effects of noise and spectral variation. It can be seen that the noise can mask heart sounds, the spectral contents of heart sound (S2) vary from position ‘A’ to position ‘B’. Furthermore, Fig. 3 shows different types of PCG signals i.e. normal heart sound, abnormal sound, murmur etc. It also shows the variation in the length of different sampled signals and the impact of noise on these signals. Moreover, heart rate variation is also evident due to presence of different number of beats per second in different sounds.
Thanks to the ease of access to computational resources, automated analysis of PCG signals has emerged as a popular research problem in the signal processing community. In one of the earlier attempts to analyze PCG signals, Ricke et al. [3] segmented the PCG signals utilizing ECG signals as reference and employing Hidden Markov Model (HMM). During modeling, signals were pre-processed using band-pass filtering and calculation of average Shannon energy envelop. Subsequently, Mel-spaced filters are used to generate regression coefficients which result in spectral features in the frequency range between 10–430 Hz. Finally, HMM is applied to the model which subsequently segments the PCG sounds. While Ricke et al. [3] used ECG signal for segmentation, studies are generally focused on segmentation as well as classification of PCG signals using the local information from the signal itself. In one such work, Oliveira et al. [4] segmented the PCG signals using entropy and envelogram. In a relatively recent study, Parasad et al. [5] segmented the signal into locations of S1 and S2 using zero frequency filtering (ZFF).
Generally, segmentation is carried out for classification of PCG signals for disease diagnosis. Touhira et al. [6], for instance, modeled heart signals using HMM to classify signal as normal or abnormal. In a similar study, Abbas et al. [7] carried out binary classification of the signals into normal and murmurs, based on thresholding. Ari and Goutam [8] in a similar work differentiated CVD signals from normal signals using an artificial neural network (ANN). The authors employed filtering and normalization before extracting features that were fed to an ANN. The approach resulted in a high classification rate of 99.3%. In another study, Grzegorczyk et al. [9] also used an ANN for signal classification. Their study, however, was limited to categorizing signals as normal and abnormal. Likewise, Bayesian networks have been explored for classification of PCG signals in studies like [10], [11].
Another popular choice for classification of PCG signals is support vector machine (SVM). Among studies employing SVM classifier, Tange et al. [12] combined SVM with multi-domain features. Features are extracted from time, frequency and time–frequency domains and an accuracy of 88% is reported. Similarly, Singh et al. [13] investigate multiple classifiers (kNN, SVM and Ensembles) for classification of PCG signals. In one of the relatively recent studies, Bourouhou et al. [14] employ kNN and SVM for multi-class classification.
In the recent years, there has been a paradigm shift from conventional machine learning to deep learning-based methods. While traditional machine learning techniques rely on domain knowledge to extract (hand-crafted) features, deep learning rests on data-driven feature learning. Among various deep learning techniques, convolutional neural networks (CNN) have been most commonly employed. Among notable studies, Baccouche et al. [15] exploit a combination of convolutional and recurrent neural networks to carry out binary classification of signals as normal or abnormal. The authors argue that the heart sound data, being cyclo-stationary in nature, can be modeled as a sequence. The technique first extracts feature sequences using convolutional layers followed by sequence modeling and classification using LSTM. Among other studies, He et al. [16] and Chowdhary et al. [17], also employ deep neural networks for signal analysis. He [16] employ AdaBoost and CNN for classification on segmented signals while Chowdhary et al. [17] propose a relatively more sophisticated system where Mel-Spectrogram is used for learning.
Employing off-the-shelf, pre-trained deep neural networks and adapting them to the problem at hand (transfer learning), is another approach that has gained significant popularity among researches in signal processing as well as machine learning domains. While transfer learning is quite common in computer vision tasks like object recognition, it is relatively less explored for signal processing-based problems. Recent trends however, indicate that this approach is attracting research attention of the signal processing community [18], [19]. AlexNet [20] and WaveNet [21], for instance, have been employed for the task of heart rate classification.
An overview of prominent studies on analysis of PCG signals is presented in Table 1. An analysis of these techniques reveals that signal processing, machine learning and deep learning are the three main approaches investigated for classification of PCG signals. Among these, deep learning has emerged as an attractive solution in the recent years and, has also reported state-of-the-art performance. Table 1 also shows that both public and private datasets have been used in different studies. Commonly employed public datasets include PASCAL-2011 [26], MHSML-2014 [27], PhysioNet-2016 [28] and Yaseen-2018 [18]. Signals are typically divided into two broad categories i.e. normal and abnormal. Additionally, the abnormal signals are further categorized as a function of pathology and other artifacts. The pre-processing stage, in general, comprises of filtering and decimation for signal processing-based approaches. However, for machine learning-based methods, sophisticated techniques like cross-wavelet transform (CWT) have been used. Classification can be carried out using heuristics like locations of S1, S2 etc., or supervised methods like SVM or DNN. Classification (binary or multi-class) has been reported at either signal or cycle level in different studies.
This study presents a hybrid approach that leverages both signal processing and the recent deep learning-based methods for classification of PCG signals. The key processing steps of the proposed technique include filtering, decimation, signal segmentation, spectrogram generation, pre-classification and voting-based final classification. Analysis is carried out at both cycle and signal levels and binary as well as multi-class classification is considered. The key highlights of this study are outlined in the following.
- •
This study introduces and validates the postulate that multi-class PCG signal classification can be carried out from 2–3 s of data.
- •
It is shown that an overlap of retains all major peaks in the signal and is utilized to classify the signals which contain rare events like extra systole; ( is the sampling frequency).
- •
The study shows that spectrum limited to 800 Hz is required for PCG signal classification.
- •
A hybrid classifier composed of a CNN and SVM is used for cycle classification.
- •
It is shown that the training time can be significantly reduced by using pre-trained off-the-shelf models.
- •
The hybrid classier is complemented with a voting based system for final classification.
The subsequent contents of this paper are organized as follows. Section 2 introduces the dataset employed in our study. Details of the proposed technique are presented in Section 3 while Section 4 outlines the experimental protocol and summarizes the obtained results. A discussion on the reported results is next presented and at the end we conclude this paper in Section 6 with a recall of the findings.
Section snippets
Dataset
We have used the publicly available PASCAL dataset [26] which was compiled and labeled primarily for a challenge on localization and classification of heart sounds. PASCAL dataset is composed of two parts, dataset ‘A’ and dataset ‘B’. Samples in dataset ‘A’ are collected using iStethoscope Pro, an iPhone application while dataset ‘B’ is sampled by DigiScope, Littmann Model, 3100, a digital stethoscope. The sampling frequencies of the signals in the two datasets are 44.1 kHz and 4 kHz,
Methods
We now present the details of the proposed method for classification of PCG signals. As mentioned earlier, we employ a hybrid technique that relies on both signal processing and deep learning methods. The key processing steps of the technique are outlined in Fig. 4 and include pre-processing, classification using AlexNet and SVM and finally a majority voting-based decision. The details of these processing steps are presented in the following.
Experiments and results
A comprehensive experimental study is carried out to validate the presented technique. The experimental protocol is designed taking into account the two levels of analysis i.e. cycle level and then signal level using a voting method. Cycle classification refers to the process in which spectrogram representations of various time resolutions are classified. Since the dataset is not balanced, we employ multiple experimental protocol as outlined in the following.
- •
Protocol I: In Protocol I, cycles
Discussion
We now present a discussion on different aspects of our technique along with a performance comparison with recent studies on this problem. A summary of this comparison is presented in Table 9, Table 10 where it can be observed that usage of machine learning-based methods has been dominant in the recent studies [19], [32], [33], [34], [35], [36], [37], [38], [39], [40], [41]. In addition to conventional machine learning classifiers like kNN [42], [43], SVM [42], [44], [45], [46] and ensemble
Conclusion
This study introduced a hybrid network for PCG signal classification. Classification is based on two levels of analysis. First, cycles which represent overlapped time segments are converted to spectrograms. These spectrograms are fed to a pre-trained convolutional neural network which maps the input to a feature vector. These deep features are next fed to an SVM which classifies the cycles. Once the cycles are classified, a voting-based scheme classifies the signals. The technique reported high
Declaration
I, Shahid Ismail, on behalf of myself and co-authors testify that our work titled ‘PCG Classification using Spectrogram via Transfer Learning’ is our own work. The presented research material is not in consideration for publication in part or as a whole elsewhere.
CRediT authorship contribution statement
Shahid Ismail: Major contribution towards research on PCG signal classification. Basit Ismail: Provided support in implementation and various technical aspects of this research. Imran Siddiqi: Supervision, Algorithmic development, Paper writing. Usman Akram: Supervision, Contributed to the technical as well as non-technical aspects of the paper.
Declaration of Competing Interest
No author associated with this paper has disclosed any potential or pertinent conflicts which may be perceived to have impending conflict with this work. For full disclosure statements refer to https://doi.org/10.1016/j.bspc.2022.104075.
Acknowledgments
The authors would like to thank Bahria University, Islamabad, Pakistan, for providing us with the opportunity to carry out the reported research. All authors approved the final version of the manuscript” to acknowledgment.
Funding
The reported research is not a part of any funded project.
References (50)
- et al.
In search of an optimization technique for artificial neural network to classify abnormal heart sounds
Appl. Soft Comput.
(2009) - et al.
Multi-level basis selection of wavelet packet decomposition tree for heart sound classification
Comput. Biol. Med.
(2013) - et al.
Classification of heart sound signals using a novel deep WaveNet model
Comput. Methods Programs Biomed.
(2020) - et al.
Cross-wavelet assisted convolution neural network (AlexNet) approach for phonocardiogram signals classification
Biomed. Signal Process. Control
(2021) - et al.
Brain tumor classification using deep CNN features via transfer learning
Comput. Biol. Med.
(2019) - et al.
Computer-aided diagnosis of cataract using deep transfer learning
Biomed. Signal Process. Control
(2019) - et al.
Leukemia diagnosis in blood slides using transfer learning in CNNs and SVM for classification
Eng. Appl. Artif. Intell.
(2018) Heart sound classification with signal instant energy and stacked autoencoder network
Biomed. Signal Process. Control
(2021)- et al.
Diagnosis of heart diseases by a secure internet of health things system based on autoencoder deep neural network
Comput. Commun.
(2020) - et al.
Novel PCG analysis method for discriminating between abnormal and normal heart sounds
Irbm
(2020)
Analysis of heart sound anomalies using ensemble learning
Biomed. Signal Process. Control
Robust heart sound segmentation based on spectral change detection and genetic algorithms
Biomed. Signal Process. Control
Cadiovascular disease, the number 1 killer
Mathematical Cardiac Electrophysiology, Vol. 13
Automatic segmentation of heart sound signals using hidden Markov models
Exploring embedding matrices and the entropy gradient for the segmentation of heart sounds in real noisy environments
Detection of S1 and S2 locations in phonocardiogram signals using zero frequency filter
Mitral regurgitation PCG-signal classification based on adaptive db-wavelet
PCG classification using a neural network approach
Heart sounds classification using feature extraction of phonocardiography signal
Int. J. Comput. Appl.
PCG classification using multidomain features and SVM classifier
BioMed Res. Int.
Heart abnormality classification using PCG and ECG recordings
Computación Y Sistemas
Heart Sound Signals Segmentation and Multiclass Classification
Ensemble deep learning models for heart disease classification: a case study from Mexico
Information
Cited by (19)
A novel attention-based cross-modal transfer learning framework for predicting cardiovascular disease
2024, Computers in Biology and MedicineFeature selection algorithms highlight the importance of the systolic segment for normal/murmur PCG beat classification
2023, Biomedical Signal Processing and ControlCNN-based classification of phonocardiograms using fractal techniques
2023, Biomedical Signal Processing and ControlRotor fault characterization study by considering normalization analysis, feature extraction, and a multi-class classifier
2024, Engineering Research ExpressArtificial intelligence for heart sound classification: A review
2024, Expert Systems