Abstract
The advent of high dimensionality problems has brought new challenges for machine learning researchers, who are now interested not only in the accuracy but also in the scalability of algorithms. In this context, machine learning can take advantage of feature selection methods to deal with large-scale databases. Feature selection is able to reduce the temporal and spatial complexity of learning, turning an impracticable algorithm into a practical one. In this work, the influence of feature selection on the scalability of four of the most well-known training algorithms for feedforward artificial neural networks (ANNs) is studied. Six different measures are considered to evaluate scalability, allowing to establish a final score to compare the algorithms. Results show that including a feature selection step, ANNs algorithms perform much better in terms of scalability.
This work was supported by Spanish Ministerio de Ciencia e Innovacioń under project TIN 2009-02402, partially supported by the European Union ERDF.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Guyon, I., Gunn, S., Nikravesh, M., Zadeh, L.: Feature Extraction. Foundations and Applications. Springer, Heidelberg (2006)
Bolon-Canedo, V., Sanchez-Maroño, N., Alonso-Betanzos, A.: On the Effectiveness of Discretization on Gene Selection of Microarray Data. In: Proceedings of the International Joint Conference on Neural Networks, pp. 3167–3174 (2010)
Bolon-Canedo, V., Sanchez-Maroño, N., Alonso-Betanzos, A.: Feature Selection and Classification in Multiple Class Datasets: An Application to KDD Cup 99 Dataset. Journal of Expert Systems with Applications (38), 5947–5957 (2011)
Dong, J.: Speed and accuracy: large-scale machine learning algorithms and their applications. Concordia University Montreal, PQ (2003)
Sonnenburg, S., Ratsch, G., Rieck, K.: Large scale learning with string kernels. Journal of Large-Scale Kernel Machines, 73–104 (2007)
Bottou, L., Bousquet, O.: The tradeoffs of large scale learning. Journal of Advances in Neural Information Processing Systems 20, 161–168 (2008)
Catlett, J.: Megainduction: machine learning on very large databases. Ph.D. dissertation, School of Computer Science, University of Technology, Sydney, Australia (1991)
Provost, F., Kolluri, V.: A survey of methods for scaling up inductive algorithms. Journal of Data Mining and Knowledge Discovery 3(2), 131–169 (1999)
Peteiro-Barral, D., Guijarro-Berdinas, B., Pérez-Sánchez, B., Fontenla-Romero, O.: On the Scalability of Machine Learning Algorithms for Artificial Neural Networks. Journal of IEEE Transactions on Neural Networks (under review)
Sonnemburg, S., Franc, V., Yom-Tov, E., Sebag, M.: PASCAL Large Scale Learning Challenge. Journal of Machine Learning Research (2009)
Yu, L., Liu, H.: Efficient feature selection via analysis of relevance and redundancy. Journal of Machine Learning Research 5, 1205–1224 (2004)
Hall, M.A.: Correlation-based Feature Selection for Machine Learning. PhD thesis, University of Waikato, Hamilton, New Zealand (1999)
Zhao, Z., Liu, H.: Searching for Interacting Features. In: Proceedings of the International Joint Conference on Artificial Intelligence, pp. 1156–1167 (1991)
Dash, M., Liu, H.: Consistency-based Search in Feature Selection. Journal of Artificial Intelligence 151(1-2), 155–176 (2003)
Bishop, C.M.: Pattern recognition and machine learning. Springer, New York (2006)
Møller, M.F.: A scaled conjugate gradient algorithm for fast supervised learning. Journal of Neural Networks 6(4), 525–533 (1993)
More, J.: The Levenberg-Marquardt algorithm: implementation and theory. Journal of Numerical Analysis, 105-116 (1978)
Weiss, S.M., Kulikowski, C.A.: Computer systems that learn: classification and prediction methods from statistics, neural nets, machine learning, and expert systems. Morgan Kaufmann, San Francisco (1991)
Hecht-Nielsen, R.: Neurocomputing. Addison-Wesley, Menlo Park (1990)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Bolón-Canedo, V., Peteiro-Barral, D., Alonso-Betanzos, A., Guijarro-Berdiñas, B., Sánchez-Maroño, N. (2011). Scalability Analysis of ANN Training Algorithms with Feature Selection. In: Lozano, J.A., Gámez, J.A., Moreno, J.A. (eds) Advances in Artificial Intelligence. CAEPIA 2011. Lecture Notes in Computer Science(), vol 7023. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-25274-7_9
Download citation
DOI: https://doi.org/10.1007/978-3-642-25274-7_9
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-25273-0
Online ISBN: 978-3-642-25274-7
eBook Packages: Computer ScienceComputer Science (R0)