Abstract
Neural networks are powerful at discovering the hidden relation, such as classifying facial expressions to emotions. The performance of the neural network is typically limited by the number of informative features. In this paper, a novel feature augmentation is proposed for generating new informative features in an unsupervised manner. Current data augmentation focuses on synthesizing new samples according to data distribution. Instead, our approach, Feature Space Expansion (FSE), enriches data feature by providing their distribution information, which brings benefit based on model performance and convergence speed. To the best of our knowledge, FSE is the first feature augmentation method, which is developed based on feature distribution. We evaluate FSE performance on face emotion dataset and music effect dataset. We provide diverse comparisons with different alternative baselines. The experimental results indicate FSE provides significant improvement in model’s prediction accuracy when the number of features in original dataset is relatively small, and less remarkable improvement when the number of features in original dataset is large. In addition, training on FSE augmented training set can have at least ten times faster convergence speed than training on original training set.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Dhall, A., Goecke, R., Lucey, S., Gedeon, T.: Static facial expressions in tough conditions: data, evaluation protocol and benchmark. In: 1st IEEE International Workshop on Benchmarking Facial Image Analysis Technologies BeFIT, ICCV2011, November 2011
Nejad, A.F., Gedeon, T.D.: Bidirectional neural networks and class prototypes. In: IEEE International Conference on Neural Networks. Proceedings, vol. 3, pp. 1322–1327. IEEE, November 1995
Zhang, T., Zheng, W., Cui, Z., Zong, Y., Li, Y.: Spatial-Temporal recurrent neural network for emotion recognition. IEEE Trans. Cybern. 49(3), 839–847 (2018)
Liu, Y., Sun, C., Lin, L., Wang, X.: Learning Natural Language Inference using Bidirectional LSTM model and Inner-Attention. arXiv preprint arXiv:1605.09090 (2016)
Hansen, L., Salamon, P.: Neural network ensembles. IEEE Trans. Pattern Anal. Mach. Intell. 12(10), 993–1001 (1990)
Shanker, M., Hu, M.Y., Hung, M.S.: Effect of data standardization on neural network training. Omega 24(4), 385–397 (1996)
Wöllmer, M., Metallinou, A., Eyben, F., Schuller, B., Narayanan, S.: Context-sensitive multimodal emotion recognition from speech and facial expression using bidirectional LSTM modeling. In: Proceedings INTERSPEECH 2010, Makuhari, Japan, pp. 2362–2365 (2010)
Lin, C.T., Lee, C.S.G.: Neural-network-based fuzzy logic control and decision system. IEEE Trans. Comput. 40(12), 1320–1336 (1991)
Ng, H.W., Nguyen, V.D., Vonikakis, V., Winkler, S.: Deep learning for emotion recognition on small datasets using transfer learning. In: Proceedings of the 2015 ACM on International Conference on Multimodal Interaction, pp. 443–449, November 2015
Dai, Z., et al.: CNN descriptor improvement based on L2-normalization and feature pooling for patch classification. In: 2018 IEEE International Conference on Robotics and Biomimetics (ROBIO), pp. 144–149. IEEE, December 2018
Maulik, U., Saha, I.: Automatic fuzzy clustering using modified differential evolution for image classification. IEEE Trans. Geosci. Remote Sens. 48(9), 3503–3510 (2010)
Cai, W., Chen, S., Zhang, D.: Fast and robust fuzzy c-means clustering algorithms incorporating local information for image segmentation. Pattern Recogn. 40(3), 825–838 (2007)
Shorten, C., Khoshgoftaar, T.M.: A survey on image data augmentation for deep learning. J. Big Data 6(1), 1–48 (2019)
Perez, L., Wang, J.: The effectiveness of data augmentation in image classification using deep learning. arXiv preprint arXiv:1712.04621 (2017)
Schmidhuber, J.: Deep learning in neural networks: an overview. Neural Networks 61, 85–117 (2015)
Xu, Y., Noy, A., Lin, M., Qian, Q., Li, H., Jin, R.: WeMix: How to Better Utilize Data Augmentation. arXiv preprint arXiv:2010.01267 (2020)
Rahman, J.S., Gedeon, T., Caldwell, S., Jones, R.: Brain melody informatics: analysing effects of music on brainwave patterns. In: 2020 International Joint Conference on Neural Networks (IJCNN), pp. 1–8. IEEE, July 2020
Alashwal, H., El Halaby, M., Crouse, J.J., Abdalla, A., Moustafa, A.A.: The application of unsupervised clustering methods to Alzheimer’s disease. Front. Comput. Neurosci. 13, 31 (2019). https://doi.org/10.3389/fncom.2019.00031
Reynolds, D.A.: Gaussian mixture models. Encycl. Biometrics 741, 659–663 (2009)
Auer, P., Burgsteiner, H., Maass, W.: A learning rule for very simple universal approximators consisting of a single layer of perceptrons. Neural Networks 21(5), 786–795 (2008)
Zell, A.: Simulation neuronaler netze, vol. 1, no. 5.3. Addison-Wesley, Bonn (1994)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Liu, Y., Yang, Y., Hossain, M.Z. (2021). FSE: a Powerful Feature Augmentation Technique for Classification Task. In: Mantoro, T., Lee, M., Ayu, M.A., Wong, K.W., Hidayanto, A.N. (eds) Neural Information Processing. ICONIP 2021. Lecture Notes in Computer Science(), vol 13109. Springer, Cham. https://doi.org/10.1007/978-3-030-92270-2_55
Download citation
DOI: https://doi.org/10.1007/978-3-030-92270-2_55
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-92269-6
Online ISBN: 978-3-030-92270-2
eBook Packages: Computer ScienceComputer Science (R0)