Abstract
Feature extraction is the first step in building real-life classification engines—it aims at elaborating features to characterize objects that are to be labeled by a trained model. Time-consuming feature extraction requires domain expertise to effectively design features. Deep neural networks (DNNs) appeared as a remedy in this context—their shallow layers perform representation learning, being an automated discovery of various-level features that robustly represent objects. However, the representations that are being learnt are still extremely difficult to interpret, and DNNs are prone to memorizing small datasets. In this paper, we introduce evolvable deep features (EDFs)—a DNN is used to extract automatic features that undergo genetic feature selection. Such evolved features are fed into a supervised learner. The experiments, backed up with statistical tests, performed on multi- and binary-class sets showed that our approach automatically learns object representations, greatly reduces the number of features without deteriorating the performance of trained models, and can even boost their classification performance.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Available at: https://keras.io/ (last access: November 7, 2017).
- 2.
Available at: https://www.tensorflow.org/ (last access: November 7, 2017).
References
Sheikhpour, R., Sarram, M.A., Gharaghani, S., Chahooki, M.A.Z.: A survey on semi-supervised feature selection methods. Pattern Recogn. 64, 141–158 (2017)
Chandrashekar, G., Sahin, F.: A survey on feature selection methods. Comput. Electr. Eng. 40(1), 16–28 (2014)
Dy, J.G., Brodley, C.E.: Feature selection for unsupervised learning. J. Mach. Learn. Res. 5, 845–889 (2004)
Wang, X., Zhang, X., Zeng, Z., Wu, Q., Zhang, J.: Unsupervised spectral feature selection with l 1-norm graph. Neurocomputing 200, 47–54 (2016)
Shi, H., Li, Y., Han, Y., Hu, Q.: Cluster structure preserving unsupervised feature selection for multi-view tasks. Neurocomputing 175, 686–697 (2016)
Nakariyakul, S., Casasent, D.P.: An improvement on floating search algorithms for feature subset selection. Pattern Recogn. 42(9), 1932–1940 (2009)
Chakravarty, K., Das, D., Sinha, A., Konar, A.: Feature selection by differential evolution algorithm - a case study in personnel identification. In: Proceedings of IEEE CEC, pp. 892–899 (2013)
Alba, E., Garcia-Nieto, J., Jourdan, L., Talbi, E.G.: Gene selection in cancer classification using PSO/SVM and GA/SVM hybrid algorithms. In: Proceedings of IEEE CEC, pp. 284–290 (2007)
Lee, J., Kim, D.W.: Memetic feature selection algorithm for multi-label classification. Inf. Sci. 293, 80–96 (2015)
Goodfellow, I., Bengio, Y., Courville, A.: Deep Learning. MIT Press, Cambridge (2016)
Zhou, B., Lapedriza, A., Xiao, J., Torralba, A., Oliva, A.: Learning deep features for scene recognition using places database. In: Proceedings of NIPS, pp. 487–495 (2014)
Poria, S., Cambria, E., Gelbukh, A.: Deep convolutional neural network textual features and multiple kernel learning for utterance-level multimodal sentiment analysis. In: Proceedings of EMNLP, pp. 2539–2544 (2015)
Sun, Y., Chen, Y., Wang, X., Tang, X.: Deep learning face representation by joint identification-verification. In: Proceedings of NIPS, pp. 1988–1996 (2014)
Nezhad, M.Z., Zhu, D., Li, X., Yang, K., Levy, P.: Safs: a deep feature selection approach for precision medicine. In: Proceedings of IEEE BIBM, pp. 501–506. IEEE (2016)
Li, Y., Chen, C.Y., Wasserman, W.W.: Deep feature selection: theory and application to identify enhancers and promoters. J. Comput. Biol. 23(5), 322–336 (2016)
Zou, Q., Ni, L., Zhang, T., Wang, Q.: Deep learning based feature selection for remote sensing scene classification. IEEE Geosci. Remote Sens. Lett. 12(11), 2321–2325 (2015)
Kowaliw, T., Banzhaf, W., Doursat, R.: Networks of transform-based evolvable features for object recognition. In: Proceedings of GECCO, USA, pp. 1077–1084. ACM (2013)
Acknowledgments
JN was supported by the Polish National Centre for Research and Development under the Innomed grant POIR.01.02.00-00-0030/15. JN and MK were supported by the National Science Centre, Poland, under Research Grant No. DEC-2017/25/B/ST6/00474.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Nalepa, J., Mrukwa, G., Kawulok, M. (2018). Evolvable Deep Features. In: Sim, K., Kaufmann, P. (eds) Applications of Evolutionary Computation. EvoApplications 2018. Lecture Notes in Computer Science(), vol 10784. Springer, Cham. https://doi.org/10.1007/978-3-319-77538-8_34
Download citation
DOI: https://doi.org/10.1007/978-3-319-77538-8_34
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-77537-1
Online ISBN: 978-3-319-77538-8
eBook Packages: Computer ScienceComputer Science (R0)