Zhenyu Yang
Yantao Li
Gang Zhou
Abstract
Deep learning has achieved significant success on intelligent medical treatments, such as automatic diagnosis and analysis of medical data. To train an automatic diagnosis system with high accuracy and strong robustness in healthcare, sufficient training data are required when using deep learning-based methods. However, given that the data collected by sensors that are embedded in medical or mobile devices are inadequate, it is challenging to train an effective and efficient classification model with state-of-the-art performance. Inspired by generative adversarial networks (GANs), we propose TS-GAN, a Time-series GAN architecture based on long short-term memory (LSTM) networks for sensor-based health data augmentation, thereby improving the performance of deep learning-based classification models. TS-GAN aims to learn a generative model that creates time-series data with the same space and time dependence as the real data. Specifically, we design an LSTM-based generator for creating realistic data and an LSTM-based discriminator for determining how similar the generated data are to real data. In particular, we design a sequential-squeeze-and-excitation module in the LSTM-based discriminator to better understand space dependence of real data, and apply the gradient penalty originated from Wasserstein GANs in the training process to stabilize the optimization. We conduct comparative experiments to evaluate the performance of TS-GAN with TimeGAN, C-RNN-GAN and Conditional Wasserstein GANs through discriminator loss, maximum mean discrepancy, visualization methods and classification accuracy on health datasets of ECG_200, NonInvasiveFatalECG_Thorax1, and mHealth, respectively. The experimental results show that TS-GAN exceeds other state-of-the-art time-series GANs in almost all the evaluation metrics, and the classifier trained on synthetic datasets generated by TS-GAN achieves the highest classification accuracy of 97.50% on ECG_200, 94.12% on NonInvasiveFatalECG_Thorax1, and 98.12% on mHealth, respectively.
- [1] . 2021. Differentially private medical texts generation using generative neural networks. ACM Trans. Comput. Healthcare 3, 1, Article
5 (Oct. 2021), 27 pages.DOI: Google Scholar
Digital Library
- [2] . 2018. SIMADL: Simulated activities of daily living dataset. Data 3, 2 (2018), 11.Google ScholarCross Ref
- [3] . 2019. Abnormal behaviour detection for dementia sufferers via transfer learning and recursive auto-encoders. In Proceedings of the IEEE International Conference on Pervasive Computing and Communications Workshops (PerCom Workshops’19). IEEE, 529–534.Google Scholar
- [4] . 2017. Wasserstein generative adversarial networks. In Proceedings of the International Conference on Machine Learning. PMLR, 214–223.Google Scholar
- [5] . 2015. Design, implementation and validation of a novel open framework for agile development of mobile health applications. Biomed. Eng. Online 14, 2 (2015), 1–20. Retrieved from http://archive.ics.uci.edu/ml/datasets/mhealth+dataset.Google Scholar
- [6] . 2018. Gan augmentation: Augmenting training data using generative adversarial networks. Retrieved from https://arXiv:1810.10863.Google Scholar
- [7] . 2019. A comparative study of LSTM and phased LSTM for gait prediction. Int. J. Artificial. Intelli. App. 10, 4 (2019), 57–66.Google Scholar
- [8] . 2020. Modals: Modality-agnostic automated data augmentation in the latent space. In Proceedings of the International Conference on Learning Representations.Google Scholar
- [9] . 2018. Autoaugment: Learning augmentation policies from data. Retrieved from https://arXiv:1805.09501.Google Scholar
- [10] . 2020. Randaugment: Practical automated data augmentation with a reduced search space. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops. 702–703.Google ScholarCross Ref
- [11] . 2019. The UCR time-series archive. IEEE/CAA J. Autom. Sinica 6, 6 (2019), 1293–1305. Retrieved from https://www.cs.ucr.edu/eamonn/time_series_data_2018/.Google ScholarCross Ref
- [12] . 2021. CNN-based continuous authentication on smartphones with auto augmentation search. In Information and Communications Security, , , , and (Eds.). Springer International Publishing, Cham, 169–186. Google ScholarDigital Library
- [13] . 2015. Rotation-invariant convolutional neural networks for galaxy morphology prediction. Monthly Notices Roy. Astron. Soc. 450, 2 (2015), 1441–1459.Google ScholarCross Ref
- [14] . 2020. A review on deep learning methods for ECG arrhythmia classification. Expert Syst. Appl.: X 7 (2020), 100033.Google ScholarCross Ref
- [15] . 2020. Deep 1D-convnet for accurate Parkinson disease detection and severity prediction from gait. Expert Syst. Appl. 143 (2020), 113075.Google ScholarDigital Library
- [16] . 2020. Do GANs always have Nash equilibria? In Proceedings of the 37th International Conference on Machine Learning (Proceedings of Machine Learning Research), and (Eds.), Vol. 119. PMLR, 3029–3039.Google Scholar
- [17] . 2019. Mitigating drift in time-series data with noise augmentation. In Proceedings of the International Conference on Computational Science and Computational Intelligence (CSCI’19). IEEE, 227–230.Google Scholar
- [18] . 2018. Artificial intelligence powers digital medicine. NPJ Dig. Med. 1, 1 (2018), 1–4.Google Scholar
- [19] . 2021. Adaptive weighting scheme for automatic time-series data augmentation. Retrieved from https://arXiv:2102.08310.Google Scholar
- [20] . 2016. Nips 2016 tutorial: Generative adversarial networks. Retrieved from https://arXiv:1701.00160.Google Scholar
- [21] . 2020. Generative adversarial networks. Commun. ACM 63, 11 (2020), 139–144.Google ScholarDigital Library
- [22] . 2005. Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neural Netw. 18, 5-6 (2005), 602–610.Google ScholarDigital Library
- [23] . 2017. Improved training of wasserstein gans. Retrieved from https://arXiv:1704.00028.Google Scholar
- [24] . 2019. Mode collapse and regularity of optimal transportation maps. Retrieved from https://arXiv:1902.02934.Google Scholar
- [25] . 2018. Squeeze-and-excitation networks. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 7132–7141.Google ScholarCross Ref
- [26] . 2020. Augmenting DL with adversarial training for robust prediction of epilepsy seizures. ACM Trans. Comput. Healthcare 1, 3, Article
18 (June 2020), 18 pages.DOI: Google ScholarDigital Library - [27] . 2013. Vocal tract length perturbation (VTLP) improves speech recognition. In Proceedings of the International Conference on Machine Learning Workshop on Deep Learning for Audio, Speech and Language (ICML’13), Vol. 117. 21.Google Scholar
- [28] Madhura Joshi, Ankit Pal, and Malaikannan Sankarasubbu. 2022. Federated learning for healthcare domain - pipeline, applications and challenges. ACM Trans. Comput. Healthcare 3, 4 (2022), 36 pages. Google ScholarDigital Library
- [29] . 2020. Gait classification using LSTM networks for tagging system. In Proceedings of the IEEE 15th International Conference of System of Systems Engineering (SoSE’20). IEEE, 295–300.Google Scholar
- [30] Sayeri Lala, Maha Shady, Anastasiya Belyaeva, and Molei Liu. 2018. Evaluation of mode collapse in generative adversarial networks. In Proceedings of the IEEE High Performance Extreme Computing Conference (HPEC’18). IEEE, 1–9.Google Scholar
- [31] . 2017. Mmd gan: Towards deeper understanding of moment matching network. Retrieved from https://arXiv:1705.08584.Google Scholar
- [32] . 2018. Using data augmentation in continuous authentication on smartphones. IEEE Internet Things J. 6, 1 (2018), 628–640.Google Scholar
- [33] . 2022. Adaptive deep feature fusion for continuous authentication with data augmentation. IEEE Trans. Mobile Comput. (2022), 1–16.
DOI: Google ScholarCross Ref - [34] Yuan-Pin Lin, Yi-Hsuan Yang, and Tzyy-Ping Jung. 2014. Fusion of electroencephalographic dynamics and musical contents for estimating emotional responses in music listening. Front Neurosci. 8 (2014), 14 pages. Google ScholarCross Ref
- [35] . 2021. Deep learning in ECG diagnosis: A review. Knowl.-Based Syst. 227 (2021), 107187.Google ScholarDigital Library
- [36] . 2015. Combining eye movements and EEG to enhance emotion recognition. In Proceedings of the 24th International Joint Conference on Artificial Intelligence.Google ScholarDigital Library
- [37] . 2018. EEG data augmentation for emotion recognition using a conditional Wasserstein GAN. In Proceedings of the 40th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC’18). IEEE, 2535–2538.Google ScholarCross Ref
- [38] . 2016. C-RNN-GAN: Continuous recurrent neural networks with adversarial training. Retrieved from https://arXiv:1611.09904.Google Scholar
- [39] . 2021. Dynamic data augmentation with gating networks. Retrieved from https://arXiv:2111.03253.Google Scholar
- [40] . 2021. Generating synthetic ECGs using GANs for anonymizing healthcare data. Electronics 10, 4 (2021), 389.Google Scholar
- [41] . 2017. Deep convolutional neural networks and data augmentation for environmental sound classification. IEEE Signal Process. Lett. 24, 3 (2017), 279–283.
DOI: Google ScholarCross Ref - [42] Arash Shaban-Nejad, Martin Michalowski, and David L. Buckeridge. 2018. Health intelligence: How artificial intelligence transforms population and personalized health. npj Digital Med 1 (2018), 2 pages. Google ScholarCross Ref
- [43] . 2019. Singan: Learning a generative model from a single natural image. In Proceedings of the IEEE/CVF International Conference on Computer Vision. 4570–4580.Google ScholarCross Ref
- [44] . 2017. Data augmentation of wearable sensor data for Parkinson’s disease monitoring using convolutional neural networks. In Proceedings of the 19th ACM International Conference on Multimodal Interaction. 216–220.Google ScholarDigital Library
- [45] . 2008. Visualizing data using t-SNE. J. Mach. Learn. Res. 9, 86 (2008), 2579–2605.Google Scholar
- [46] . 2017. Atrial fibrillation detection using stationary wavelet transform and deep learning. In Proceedings of the Computing in Cardiology (CinC’17). IEEE, 1–4.Google Scholar
- [47] Jinsung Yoon, Daniel Jarrett, and Mihaela Van der Schaar. 2019. Time-series generative adversarial networks. In Proceedings of Advances in Neural Information Processing Systems (NeurIPS’19), Vol. 32, 1–11.Google Scholar
- [48] . 2017. Patient-specific ECG classification based on recurrent neural networks and clustering technique. In Proceedings of the 13th IASTED International Conference on Biomedical Engineering (BioMed’17). IEEE, 63–67.Google Scholar
- [49] . 2021. CNN-LSTM based traffic prediction using spatial-temporal features. In Journal of Physics: Conference Series, Vol. 2037. IOP Publishing, 012065.Google Scholar
- [50] . 2017. Unpaired image-to-image translation using cycle-consistent adversarial networks. In Proceedings of the IEEE International Conference on Computer Vision. 2223–2232.Google ScholarCross Ref
Index Terms
- TS-GAN: Time-series GAN for Sensor-based Health Data Augmentation
Recommendations
EnvGAN: a GAN-based augmentation to improve environmental sound classification
AbstractSeveral deep learning algorithms have emerged for the automatic classification of environmental sounds. However, the non-availability of adequate labeled data for training limits the performance of these algorithms. Data augmentation is an ...
Medical Image Synthetic Data Augmentation Using GAN
CSAE '20: Proceedings of the 4th International Conference on Computer Science and Application EngineeringMost medical image datasets are limited and unbalanced. The classification method based on deep neural networks is prone to under-fitting or over-fitting problems on such data sets, which affects the final classification performance. This paper proposes ...
Data augmentation using fast converging CIELAB-GAN for efficient deep learning dataset generation
The commercial deep learning applications require large training datasets with many samples from different classes. The generative adversarial networks (GAN) are able to create new data samples for training these machine learning models. However, the low ...
Comments