Abstract
Multivariate Time Series Classification (MTSC) is believed to be a crucial task towards dynamic process recognition and has been widely studied. Recent years, end-to-end MTSC with Convolutional Neural Network (CNN) has gained increasing attention thanks to its ability to integrates local features. However, it remains a significant challenge for CNN to handle global information and long-range dependencies of time series. In this paper, we present a simple and feasible architecture for MTSC to address these problems. Our model benefits from self-attention, which can help CNN directly capture the relationships of time series between two random time steps or variables. Experimental results of the proposed model work on thirty five complex MTSC tasks show its effectiveness and universality that has to outperform existing state-of-the-art (SOTA) model overall. Besides, our model is computationally efficient, and the parsing speed is six hours faster than the current model.
Supported in part by NSFC under Grant No. U1836107 and Shenzhen Science and Technology Program under Grant No. JCYJ20180507183823045.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bagnall, A., Lines, J., Bostrom, A., Large, J., Keogh, E.: The great time series classification bake off: a review and experimental evaluation of recent algorithmic advances. Data Min. Knowl. Discov. 31(3), 606–660 (2017)
Ronao, C.A., Cho, S.B.: Human activity recognition with smartphone sensors using deep learning neural networks. Expert Syst. Appl. 59, 235–244 (2016)
Ye, L., Keogh, E.: Time series shapelets: a novel technique that allows accurate, interpretable and fast classification. Data Min. Knowl. Discov. 22(1–2), 149–182 (2011)
Schäfer, P.: The boss is concerned with time series classification in the presence of noise. Data Min. Knowl. Discov. 29(6), 1505–1530 (2015)
Failure prediction of concrete piston for concrete pump vehicles. https://www.datafountain.cn/competitions/336
Zheng, Y., Liu, Q., Chen, E., Ge, Y., Zhao, J.L.: Time series classification using multi-channels deep convolutional neural networks. In: International Conference on Web-Age Information Management, pp. 298–310. Springer (2014)
Lipton, Z.C., Kale, D.C., Elkan, C., Wetzel, R.: Learning to diagnose with LSTM recurrent neural networks. arXiv preprint arXiv:1511.03677 (2015)
Wang, K., He, J., Zhang, L.: Attention-based convolutional neural network for weakly labeled human activities recognition with wearable sensors. IEEE Sens. J. 19, 7598–7604 (2019)
Bai, S., Kolter, J.Z., Koltun, V.: An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv:1803.01271 (2018)
Yang, J., Nguyen, M.N., San, P.P., Li, X.L., Krishnaswamy, S.: Deep convolutional neural networks on multichannel time series for human activity recognition. In: Twenty-Fourth International Joint Conference on Artificial Intelligence (2015)
Lee, S.M., Yoon, S.M., Cho, H.: Human activity recognition from accelerometer data using convolutional neural network. In: 2017 IEEE International Conference on Big Data and Smart Computing (BigComp), pp. 131–134. IEEE (2017)
Devineau, G., Moutarde, F., Xi, W., Yang, J.: Deep learning for hand gesture recognition on skeletal data. In: 2018 13th IEEE International Conference on Automatic Face and Gesture Recognition (FG 2018), pp. 106–113. IEEE (2018)
Karim, F., Majumdar, S., Darabi, H., Harford, S.: Multivariate lstm-fcns for time series classification. Neural Netw. 116, 237–245 (2019)
Fawaz, H.I., Forestier, G., Weber, J., Idoumghar, L., Muller, P.A.: Deep learning for time series classification: a review. Data Min. Knowl. Discov. 33(4), 917–963 (2019)
Vaswani, A., Shazeer, N., Parmar, N., Uszkoreit, J., Jones, L., Gomez, A.N., Kaiser, Ł., Polosukhin, I.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008 (2017)
Tan, Z., Wang, M., Xie, J., Chen, Y., Shi, X.: Deep semantic role labeling with self-attention. In: Thirty-Second AAAI Conference on Artificial Intelligence (2018)
Verga, P., Strubell, E., McCallum, A.: Simultaneously self-attending to all mentions for full-abstract biological relation extraction. arXiv preprint arXiv:1802.10569 (2018)
Multivariate time series dataset archive for LSTM-FCNs. https://github.com/titu1994/MLSTM-FCN/releases
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Lin, H., Ye, Y., Leung, KC., Zhang, B. (2020). A Multivariate Time Series Classification Method Based on Self-attention. In: Pan, JS., Lin, JW., Liang, Y., Chu, SC. (eds) Genetic and Evolutionary Computing. ICGEC 2019. Advances in Intelligent Systems and Computing, vol 1107. Springer, Singapore. https://doi.org/10.1007/978-981-15-3308-2_54
Download citation
DOI: https://doi.org/10.1007/978-981-15-3308-2_54
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-15-3307-5
Online ISBN: 978-981-15-3308-2
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)