Abstract
Capturing, storing, and analyzing high-dimensional time series data are important challenges that need to be effectively tackled nowadays, as the extremely large amounts of such data are being generated every second. In this paper, we introduce the recurrent neural networks equipped with attention modules that quantify the importance of features, hence can be employed to select only an informative subset of all available features. Additionally, our models are trained in an end-to-end fashion, hence are directly applicable to infer over the unseen data. Our experiments included datasets from various domains and showed that the proposed technique is data-driven, easily applicable to new use cases, and competitive to other dimensionality reduction algorithms.
This work was co-financed by the Silesian University of Technology grant for maintaining and developing research potential, and by the Polish National Centre for Research and Development under Grant POIR.01.01.01-00-0853/19. JN was supported by the Silesian University of Technology funds (02/080/BKM20/0012).
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Battiti, R.: Using mutual information for selecting features in supervised neural net learning. IEEE Trans. Neural Netw. 5(4), 537–550 (1994)
Boonyakitanont, P., Lek-uthai, A., Chomtho, K., Songsiri, J.: A review of feature extraction and performance evaluation in epileptic seizure detection using EEG. Biomed. Signal Process. Control 57, 101702 (2020)
Dau, H.A., et al.: The UCR time series archive. arXiv preprint arXiv:1810.07758 (2019)
Gui, N., Ge, D., Hu, Z.: AFS: an attention-based mechanism for supervised feature selection. In: Proceedings of AAAI CAI, vol. 33, pp. 3705–3713 (2019)
Hornik, K.: Approximation capabilities of multilayer feedforward networks. Neural Netw. 4(2), 251–257 (1991)
Li, X., Wang, Y., Ruiz, R.: A survey on sparse learning models for feature selection. IEEE Trans. Cybern., 1–19 (2020). https://doi.org/10.1109/TCYB.2020.2982445
Masoudi-Sobhanzadeh, Y., Motieghader, H., Masoudi-Nejad, A.: FeatureSelect: a software for feature selection based on machine learning approaches. BMC Bioinform. 20(1), 170 (2019). https://doi.org/10.1186/s12859-019-2754-0
Nalepa, J., Myller, M., Kawulok, M.: Transfer learning for segmenting dimensionally reduced hyperspectral images. IEEE Geosci. Remote Sens. Lett. 17(7), 1228–1232 (2020)
Ribalta Lorenzo, P., Tulczyjew, L., Marcinkiewicz, M., Nalepa, J.: Hyperspectral band selection using attention-based convolutional neural networks. IEEE Access 8, 42384–42403 (2020)
Zhi, R., Liu, M., Zhang, D.: A comprehensive survey on automatic facial action unit analysis. Visual Comput. 36(5), 1067–1093 (2019). https://doi.org/10.1007/s00371-019-01707-5
Zhu, X., Wang, Y., Li, Y., Tan, Y., Wang, G., Song, Q.: A new unsupervised feature selection algorithm using similarity-based feature clustering. Comput. Intell. 35(1), 2–22 (2019)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Myller, M., Kawulok, M., Nalepa, J. (2021). Selecting Features from Time Series Using Attention-Based Recurrent Neural Networks. In: Torsello, A., Rossi, L., Pelillo, M., Biggio, B., Robles-Kelly, A. (eds) Structural, Syntactic, and Statistical Pattern Recognition. S+SSPR 2021. Lecture Notes in Computer Science(), vol 12644. Springer, Cham. https://doi.org/10.1007/978-3-030-73973-7_9
Download citation
DOI: https://doi.org/10.1007/978-3-030-73973-7_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-73972-0
Online ISBN: 978-3-030-73973-7
eBook Packages: Computer ScienceComputer Science (R0)
