Analysis of unlabeled lung sound samples using semi-supervised convolutional neural networks

Abstract

Lung sounds convey valuable information relevant to human respiratory health. Therefore, it is important to classify lung sounds for early diagnoses of respiratory disorders. In recent years, computerized lung sound analysis with machine learning algorithms has attracted researchers, especially the state-of-the-art convolutional neural network (CNN). However, most of these algorithms require a large number of labeled respiratory sound samples, which is time- and cost-consuming. Based on a four-layers CNN, this study proposes graph semi-supervised CNNs (GS-CNNs), which can classify respiratory sounds into normal, crackle and wheeze ones with only a small labeled sample size and a large unlabeled sample size. The graph of respiratory sounds (Graph-RS) with labeled and unlabeled respiratory sound samples as vertexes is first constructed, which can indicate not only the reasonable metric information but also the relationship of all the samples. Then, GS-CNNs are developed by adding the information extracted from Graph-RS to the loss function of the original CNN. The added information enables the GS-CNNs to regulate the structure of the original CNN, thus enhancing classification accuracy. The GS-CNNs are evaluated by experiments with the samples collected by electronic stethoscope. Results demonstrate that the proposed GS-CNNs outperform the original CNN, and that the more information from Graph-RS is used, the better recognition effect will be achieved.

Introduction

Lung disease ranks third among fatality causes worldwide, therefore it is important to monitor and determine the pathological conditions from human lungs. Since Lung sounds contain the information relevant to pulmonary pathology, physicians study lung sound characteristics to diagnose respiratory disorders. Auscultation is perhaps the oldest and most common method, which remains a manual task by using a stethoscope. It has many advantages, such as non-invasive, safe, and so on. However, the diagnosing results are subject to the doctors’ ability. If not done by a well-trained physician, this may lead to wrong diagnosis. Therefore, much attention has been paid to the development of computerized lung sound analysis systems. It uses an electronic stethoscope to record lung sounds, and uses the machine learning algorithms to classify the recording lung sounds, which help to overcome the limitations of traditional auscultation. The 10th International Respiratory Sounds Association (ILSA) Conference [1] defined the classification criteria for respiratory sounds. Generally speaking, respiratory sounds are divided into normal,wheeze, crackle, snore and so on.

Machine learning methods such as support vector machine (SVM), k-nearest neighbors (KNN), Gaussian mixed model (GMM) and artificial neural network (ANN) are applied in respiratory sound classification [2]. At present, ANN is one of the most feasible supervised machine learning methods. Kandaswamy et al. combined wavelet transform and ANN, and the method got 94.02% for testing accuracy in classification of respiratory sounds [3]. In [4], Mel Frequency Cepstral Coefficients (MFCC) features along with ANN are utilized, achieving accuracy of 75% for the crackle, 100% for the wheeze and 80% for the normal.

Convolutional neural network (CNN), which is a variant of the ANN, has become state-of-the-art solution for handling high dimensionality structured inputs. CNN also has been applied to classification of respiratory sounds successfully [5], [6], [7], [8]. In [5], CNN using spectrograms for classification of lung sounds showed to perform superior to SVM using MFCC features. Compared with other methods, CNN can directly use the original spectrograms as input, and carry out feature extraction and lung sound classification simultaneously. It not only avoids the sidedness of artificial design features, but also makes full use of the characteristics of spectrogram to avoid the loss of detail information. Therefore, CNN is the preferred method in this paper for classifying respiratory sounds.

CNN is supervised learning method, which requires sufficient labeled data. However, it is difficult to collect labeled respiratory sound samples. There are three main reasons. (1) Labeling lung sounds should be done by well skilled doctors, so it is expensive job both in time and money. (2) Abnormal lung sounds are intermittent. Therefore, the adventitious abnormal sound appears for only a short time within a long recording, which lead to the lack of abnormal samples [9]. (3) For rare lung disease, the corresponding respiratory sound has quite few amount of labeled data. On the other side, collecting unlabeled data by electronic stethoscope is simple. If the information contained in unlabeled samples is used, the shortcoming of CNN when facing limited labeled respiratory sound samples can be overcome. Therefore, the semi-supervised CNN for classifying respiratory sounds is studied in this paper.

One way to implement semi-supervised CNN is to do unsupervised and supervised learning successively. Firstly, the unsupervised learning methods are used to extract features from unlabeled data, and get the features set $\Phi =\{f_1,f_2,\cdots ,f_n\}$. For example, denoising autoencoders can learn robust high-level features from unlabeled image data proved to be successful in image classification [10]. Secondly, the values $f_1,f_2,\cdots ,f_n$ are calculated from $m$ labeled samples, and the set of feature vectors $\Omega =\{{\mathit{\alpha }}_1,{\mathit{\alpha }}_2,\cdots ,{\mathit{\alpha }}_m\}$ is got, where ${\mathit{\alpha }}_j,j=1,2,\cdots m$ are n-dimensional column vectors. Finally, supervised learning is carried out using $\Omega$ as training set. This method extracts features by unsupervised learning, which overcomes the influence of disaster dimension and reduces the demand of CNN for sample size. This method was already applied for classifying respiratory sounds [11]. A denoising autoencoder was used to extract features from lung sounds, which requires a small proportion (5%) of dataset to be labeled, helping to achieve ROC curves with AUCs of 0.86 for wheeze and 0.74 for crackle.

Pseudo-label is another approach to make use of unlabeled data [12]. The pseudo-labels of unlabeled data pick up the class which has the maximum predicted probability and are used as true labels. Consequently, network can be trained in a supervised fashion. If the accuracy of pseudo-labels is high, this method solves the problem of lacking labeled lung sound samples succinctly.

The most promising method for semi-supervised neural network so far is to regulate the structure by adding the information of unlabeled data to loss function. Inspired by auto-encoder, network in [13] used consistency regulation. For a sample $\mathbf{x}$, two transformations $T_1,T_2$ are applied to it without changing its category characteristic, and let ${\tilde{\mathbf{x}}}_1=T_1\left(x\right),{\tilde{\mathbf{x}}}_2=T_2\left(x\right)$. For a network $g$, the consistency regulation requires consistency of the outputs $g\left({\tilde{\mathbf{x}}}_1\right),g\left({\tilde{\mathbf{x}}}_2\right)$. Namely, their distance $\parallel g\left({\tilde{\mathbf{x}}}_1\right)-g\left({\tilde{\mathbf{x}}}_2\right)\parallel$ should be minimized. This method avoiding the demand of high accuracy of labeling unlabeled data compared to pseudo-labels, but it limits the features used for classification because of the transformations $T_1,T_2$. Mixed with other techniques like data augmentation, models with these regulations achieved excellent performance [14], [15].

However, these semi-supervised networks focus on the classification performance on test set, with little improvement on feature metric, which is important in lung sound analysis. A proper metric of feature space should satisfy clustering property, namely samples from the same class should be in the same cluster whether labeled or unlabeled. This suggests that metric can be improved with cluster regulation. In this paper, graph-based semi-supervised machine learning methods is used to CNN for cluster regulation. A graph is made up of vertices, nodes, or points which are connected by edges, arcs, or lines. Graphs can be used to model many types of relations and processes in physical, biological [16], computer science [17], [18], lung sounds [19], and so on. The graph of respiratory sound samples can show the relationship between labeled and unlabeled samples, and then the information implicated in the graph can be used to perform cluster regulation on CNN to learn better feature metric. To our knowledge, graph-based semi-supervised CNN for classifying respiratory sounds has not been addressed before.

In this paper, a novel graph-based semi-supervised learning method is proposed to improve both classification performance and featrue metric. The graph works in dynamic and differentiable manner, thus can be integrated into CNN for end to end training. Firstly, a graph of respiratory sound samples is constructed. The adjacent and distributive information of the respiratory sound samples is shown in the graph. By adding an additional loss function obtained from the graph to the original CNN loss function, the structure of the original CNN can be regulated. The graph-based semi-supervised CNN makes the best use of the information conveyed in respiratory sounds without any restrictions for unlabeled samples, which is crucial for respiratory sounds analysis.

This remainder of the paper proceeds as follows. Section 2 illustrates MFCC features of the respiratory sounds, and shows the rationality of choosing them as network input. In Section 3, the frame of the Graph semi-supervised CNN is provided, and the details of the graph of respiratory sounds are presented. And then a graph-based semi-supervised CNN for classifying respiratory sounds and a method for training it are proposed. Section 4 reports the experimental results and analysis the effect of different graph components and feature selections. Finally, Section 5 gives a conclusion.