Abstract
An active research area in Machine Learning is the construction of multiple classifier systems to increase learning accuracy of simple classifiers. In this paper we present a method to improve even more the accuracy: ML-CIDIM. This method has been developed by using a multiple classifier system which basic classifier is CIDIM, an algorithm that induces small and accurate decision trees. CIDIM makes a random division of the training set into two subsets and uses them to build an internal bound condition. ML-CIDIM induces some multiple classifier systems based on CIDIM and places them in different layers, trying to improve the accuracy of the previous layer with the following one. In this way, the accuracy obtained thanks to a unique multiple classifier system based on CIDIM can be improved. In reference to the accuracy of the classifier system built with ML-CIDIM, we can say that it competes well against bagging and boosting at statistically significant confidence levels.
This work has been partially supported by the MOISES project, number TIC2002-04019-C03-02, of the MCyT, Spain.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsPreview
Unable to display preview. Download preview PDF.
References
Breiman, L., Friedman, J.H., Olshen, R.A., Stone, C.J.: Classification and Regression Trees. Wadsworth (1984)
Quinlan, J.R.: Induction of decision trees. Machine Learning 1, 81–106 (1986)
Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Francisco (1993)
Utgoff, P.E., Berkman, N.C., Clouse, J.A.: Decision tree induction based on efficient tree restructuring. Machine Learning 29, 5–44 (1997)
Domingos, P., Hulten, G.: Mining high-speed data streams. In: Proc. of the 6th Intern. Conf. on Knowledge Discovery and Data Mining, pp. 71–80. ACM Press, New York (2000)
Breiman, L.: Bagging predictors. Machine Learning 24, 123–140 (1996)
Freund, Y., Schapire, R.E.: Experiments with a new boosting algorithm. In: Proc. of 13th Intern. Conf. on Machine Learning, pp. 146–148. Morgan Kaufmann, San Francisco (1996)
Wolpert, D.: Stacked generalization. Neural Networks 5, 241–260 (1992)
Gama, J., Brazdil, P.: Cascade generalization. Machine Learning 41, 315–343 (2000)
Ferri, C., Flach, P., Hernández-Orallo, J.: Delegating classifiers. In: Proc. of the 21st International Conference on Machine Learning, Omnipress,
Ramos-Jiménez, G., Morales-Bueno, R., Villalba-Soria, A.: CIDIM. Control of induction by sample division methods. In: Proc. of the Intern. Conf. on Artificial Intelligence, pp. 1083–1087 (2000)
Ruiz-Gómez, J., Ramos-Jiménez, G., Villalba-Soria, A.: Modelling based on rule induction learning. In: Computers and Computacional Engineering in Control, pp. 158–163. World Scientific and Engineering Society Press, Greece (1999)
Jerez-Aragonés, J.M., Gómez-Ruiz, J.A., Ramos-Jiménez, G., Muñoz Pérez, J., Alba-Conejo, E.: A combined neural network and decision trees model for prognosis of breast cancer relapse. Artificial Intelligence in Medicine 27, 45–63 (2003)
Schapire, R.E.: The strength of weak learnability. Machine Learning 5, 197–227 (1990)
Freund, Y.: Boosting a weak learning algorithm by majority. Information and Computation 121, 256–285 (1995)
Freund, Y., Schapire, R.E.: The strength of weak learnability. Journal of Computer and System Sciences 55, 119–139 (1997)
Aslam, J.A., Decatur, S.E.: General bounds on statistical query learning and PAC learning with noise via hypothesis boosting. Information and Computation 141, 85–118 (1998)
Bauer, E., Kohavi, R.: An empirical comparison of voting classification algorithms: Bagging, boosting and variants. Machine Learning 36, 105–139 (1999)
Kearns, M.J., Vazirani, U.V.: On the boosting ability of top-down decision tree learning algorithms. Journal of Computer and System Sciences 58, 109–128 (1999)
Breiman, L.: Arcing classifiers. Technical report, Berkeley: Statistics Department. University of California (1996)
Stone, P.: Layered Learning in Multi-Agent Systems. PhD thesis, School of Computer Science, Carnegie Mellon University, Pittsburgh (1998)
Utgoff, P.E., Stracuzzi, D.J.: Many-layered learning. Neural Computation 14, 2497–2529 (2002)
Stone, P., Veloso, M.: A Layered Approach to Learning Client Behaviors in the RoboCup Soccer Server. Applied Artificial Intelligence 12, 165–188 (1998)
Takahashi, Y., Asada, M.: Multi-layered learning systems for vision-based behavior acquisition of a real mobile robot. In: Proc. of SICE Annual Conference, pp. 2937–2942 (2003)
Whiteson, S., Stone, P.: Concurrent layered learning. In: Second International Joint Conference on Autonomous Agents and Multiagent Systems (2003)
Blake, C., Merz, C.J.: UCI repository of machine learning databases. University of California, Department of Information and Computer Science (2000)
Lin, T.Y., Cercone, N. (eds.): Rough Sets and Data Mining: Analysis of Imprecise Data. Kluwer Academic Publishers, Norwell (1996)
Witten, I.H., Frank, E.: Data Mining: Practical machine learning tools with Java implementations. Morgan Kaufmann, San Francisco (2000)
Predki, B., Slowinski, R., Stefanowski, J., Susmaga, R., Wilk, S.z.: ROSE - Software Implementation of the Rough Set Theory. In: Polkowski, L., Skowron, A. (eds.) RSCTC 1998. LNCS (LNAI), vol. 1424, pp. 605–608. Springer, Heidelberg (1998)
Predki, B., Wilk, S.: Rough Set Based Data Exploration Using ROSE System. In: Raś, Z.W., Skowron, A. (eds.) ISMIS 1999. LNCS, vol. 1609, pp. 172–180. Springer, Heidelberg (1999)
Herrera, F., Hervás, C., Otero, J., Sánchez, L.: Un estudio empírico preliminar sobre los tests estadísticos más habituales en el aprendizaje automático. In: Tendencias de la Minería de Datos en España (2004)
R Development Core Team: R: A language and environment for statistical computing. R Foundation for Statistical Computing, Vienna, Austria (2004), http://www.R-project.org 3-900051-07-0
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2005 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Ramos-Jiménez, G., del Campo-Ávila, J., Morales-Bueno, R. (2005). ML-CIDIM: Multiple Layers of Multiple Classifier Systems Based on CIDIM. In: Ślęzak, D., Yao, J., Peters, J.F., Ziarko, W., Hu, X. (eds) Rough Sets, Fuzzy Sets, Data Mining, and Granular Computing. RSFDGrC 2005. Lecture Notes in Computer Science(), vol 3642. Springer, Berlin, Heidelberg. https://doi.org/10.1007/11548706_15
Download citation
DOI: https://doi.org/10.1007/11548706_15
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-28660-8
Online ISBN: 978-3-540-31824-8
eBook Packages: Computer ScienceComputer Science (R0)