Abstract
This study proposes a novel module in the convolutional neural networks (CNN) framework named permutation layer. With the new layer, we are particularly targeting time-series tasks where 2-dimensional CNN kernel loses its ability to capture the spatially co-related features. Multivariate time-series analysis consists of stacked input channels without considering the order of the channels resulting in an unsorted “2D-image”. 2D convolution kernels are not efficient at capturing features from these distorted as the time-series lacks spatial information between the sensor channels. To overcome this weakness, we propose learnable permutation layers as an extension of vanilla convolution layers which allow to interchange different sensor channels such that sensor channels with similar information content are brought together to enable a more effective 2D convolution operation. We test the approach on a benchmark time-series classification task and report the superior performance and applicability of the proposed method.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
LeCun, Y., et al.: Handwritten digit recognition with a back-propagation network. In: Advances in Neural Information Processing Systems, pp. 396–404 (1990)
Bai, S., Kolter, J.Z., Koltun, V.: An empirical evaluation of generic convolutional and recurrent networks for sequence modeling. arXiv preprint arXiv:1803.01271 (2018)
Da Li, X., He, W., Li, S.: Internet of things in industries: a survey. IEEE Trans. Industr. Inf. 10(4), 2233–2243 (2014)
Chadha, G.S., Krishnamoorthy, M., Schwung, A.: Time series based fault detection in industrial processes using convolutional neural networks. In: 45th Annual Conference of the IEEE Industrial Electronics Society, IECON 2019, vol. 1, pp. 173–178 (2019)
Jiang, G., et al.: Multiscale convolutional neural networks for fault diagnosis of wind turbine gearbox. IEEE Trans. Industr. Electron. 66(4), 3196–3207 (2019)
Ince, T., et al.: Real-time motor fault detection by 1-D convolutional neural networks. IEEE Trans. Industr. Electron. 63(11), 7067–7075 (2016). https://doi.org/10.1109/TIE.2016.2582729. ISSN: 0278-0046
Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press, Cambridge (2016)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in Neural Information Processing Systems, pp. 1097–1105 (2012)
Redmon, J., et al.: You only look once: unified, real-time object detection. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)
Marshall, A.W., Olkin, I., Arnold, B.C.: Doubly stochastic matrices. Inequalities: Theory of Majorization and Its Applications. SSS, pp. 29–77. Springer, New York (2010). https://doi.org/10.1007/978-0-387-68276-1_2
Mena, G., et al.: Learning latent permutations with Gumbel-Sinkhorn networks. In: International Conference on Learning Representations (2018). https://openreview.net/forum?id=Byt3oJ-0W
Cruz, R.S., et al.: Visual permutation learning. IEEE Trans. Pattern Anal. Mach. Intell. 41(12), 3100–3114 (2019). https://doi.org/10.1109/tpami.2018.2873701
Helmbold, D.P., Warmuth, M.K.: Learning permutations with exponential weights. J. Mach. Learn. Res. 10, 1705–1736 (2009). ISSN: 1532-4435. http://dl.acm.org/citation.cfm?id=1577069.1755841
Yang, J., et al.: Deep convolutional neural networks on multichannel time series for human activity recognition. In: IJCAI, vol. 15, pp. 3995–4001 (2015)
Liu, C.-L., Hsaio, W.-H., Tu, Y.-C.: Time series classification with multivariate convolutional neural network. IEEE Trans. Industr. Electron. 66(6), 4788–4797 (2019). https://doi.org/10.1109/TIE.2018.2864702. ISSN: 0278-0046
Trask, A., et al.: Neural arithmetic logic units. In: Advances in Neural Information Processing Systems, pp. 8035–8044 (2018)
Boyd, S.P., Vandenberghe, L.: Convex Optimization. Cambridge University Press, Cambridge (2004)
Downs, J.J., Vogel, E.F.: A plant-wide industrial process control problem. Comput. Chem. Eng. 17(3), 245–255 (1993)
Park, P., et al.: Fault detection and diagnosis using combined autoencoder and long short-term memory network. Sensors 19(21), 4612 (2019)
Rieth, C.A., et al.: Additional Tennessee Eastman Process Simulation Data for Anomaly Detection Evaluation (2017). https://doi.org/10.7910/dvn/6c3jr1
Ioffe, S., Szegedy, C.: Batch normalization: accelerating deep network training by reducing internal covariate shift. In: Proceedings of the 32nd International Conference on International Conference on Machine Learning, ICML 2015, vol. 37, pp. 448–456. JMLR.org (2015)
Glorot, X., Bordes, A., Bengio, Y.: Deep sparse rectifier neural networks. In: AISTATS, vol. 15, p. 275 (2011)
Srivastava, N., et al.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv e-prints (2014). arXiv:1412.6980
Iqbal, R., et al.: Fault detection and isolation in industrial processes using deep learning approaches. IEEE Trans. Industr. Inf. 15(5), 3077–3084 (2019). https://doi.org/10.1109/TII.2019.2902274
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Chadha, G.S., Kim, J., Schwung, A., Ding, S.X. (2020). Permutation Learning in Convolutional Neural Networks for Time-Series Analysis. In: Farkaš, I., Masulli, P., Wermter, S. (eds) Artificial Neural Networks and Machine Learning – ICANN 2020. ICANN 2020. Lecture Notes in Computer Science(), vol 12396. Springer, Cham. https://doi.org/10.1007/978-3-030-61609-0_18
Download citation
DOI: https://doi.org/10.1007/978-3-030-61609-0_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-61608-3
Online ISBN: 978-3-030-61609-0
eBook Packages: Computer ScienceComputer Science (R0)