Abstract
This paper presents an implementation of bagging techniques over the heuristic algorithm for induction of classification rules called SA Tabu Miner (Simulated Annealing and Tabu Search data miner). The goal was to achieve better predictive accuracy of the derived classification rules. Bagging (Bootstrap aggregating) is an ensemble method that has attracted a lot of attention, both experimentally, since it behaves well on noisy datasets, and theoretically, because of its simplicity. In this paper we present the experimental results of various bagging versions of the SA Tabu Miner algorithm. The SA Tabu Miner algorithm is inspired by both research on heuristic optimization algorithms and rule induction data mining concepts and principles. Several bootstrap methodologies were applied to SA Tabu Miner, including reducing repetition of instances, forcing repetition of instances not to exceed two, using different percentages of the original basic training set. Various experimental approaches and parameters yielded different results on the compared datasets.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Chorbev, I., Mihajlov, D., Jolevski, I.: Web Based Medical Expert System with a Self Training Heuristic Rule Induction Algorithm. In: Proc. of The First International Conference on Advances in Databases, Knowledge, and Data Applications, DBKDA 2009, Cancun, Mexico, March 2009, pp. 143–148 (2009)
Dietterich, T.G.: Machine Learning Research: Four Current Directions. AI Magazine 18(4), 97–136 (1997)
Dietterich, T.G.: Ensemble Methods in Machine Learning, Oregon State University, Corvallis, Oregon, USA, tgd@cs.orst.edu. WWW home page, http://www.cs.orst.edu/tgd
Gentle, J.E., Härdle, W., Mori, Y.: Handbook of Computational Statistics, ch. 16. Springer, Heidelberg, http://fedc.wiwi.hu.berlin.de/xplore/ebooks/html/csa
Breiman, L.: Bagging predictors. Machine Learning 24(2), 123–140 (1996)
Stefanowski, J.: Bagging and Introduction of Decision Rules. In: Klopotek, M., et al (eds.) Intelligent information systems (2002)
Freund, Y., Schapire, R.: Experiments with a new boosting algorithm. In: Proc. of the Thirteenth Int. Conf. on Machine Learning, pp. 148–156 (1996)
Pino-MejÃas, R., et al.: Bagging Classification Models with Reduced Bootstrap. Structural, Syntactic, and Statistical Pattern Recognition, 966–973 (2004), http://www.springerlink.com/content/6r0b2payc24fj93e/
Grandvalet, Y.: Bagging equalizes influence. Machine Learning 55(3), 251–270 (2004)
Friedman, J.: Stochastic gradient boosting. Computational Statistics and Data Analysis 38(4), 367–378 (2002)
Parmanto, B., Munro, P., Doyle, H.: Improving Committee Diagnosis with Resampling Techniques. In: Touretzky, D., Mozer, M., Hasselmo, M. (eds.) Advances in Neural Information Processing Systems, vol. 8, pp. 882–888 (1996)
Quinlan, J.R.: Bagging, boosting and C4.5. In: Proceedings of the 13th National Conference on Artifitial Intelligence, pp. 725–730 (1996)
Bühlmann, P., Yu, B.: Explaining Bagging’. Technical Report 92, Seminar für Statistik, ETH, Zürich (2000)
Buja, A., Stuetzle, W.: The Effect of Bagging on Variance, Bias and Mean Squared Error. Technical report, AT&T Labs-Research (2000)
Friedman, J.H., Hall, P.: On Bagging and Non-linear Estimation. Technical report, Stanford University, Stanford, CA (2000)
Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Francisco (1993)
Xia, F.: Bagging (2006), http://faculty.washington.edu/fxia/courses/LING572/pagging.ppt
Buja, A., Stuetzle, W.: Observation of Bagging. Statistica Sinica 16, 323–351 (2006)
Hall, P., Samworth, R.J.: Properties of Bagged Nearest-neighbor Classifiers. J. Roy. Statist. Soc., Ser. B 67, 363–379 (2005)
Weiss, S.M.: Small Sample Error Rate Estimation for k-neares Neighbor Classifiers. IEEE Transaction of pattern analysis and Machine Intelligent 13(3), 285–289 (1991)
Brown, G., Wyatt, J., Harris, R., Yao, X.: Diversity Creation Methods: A Survey and Categorisation. Information Fusion 6(1), 5–20 (2005)
Hansen, L., Salamon, P.: Neural network ensembles. IEEE Transactions on Pattern Analysis and Machine Intelligence 12, 993–1001 (1990)
Krogh, A., Vedelsby, J.: Neural network ensembles, cross validation, and active learning. In: Tesauro, G., Touretzky, D., Leen, T. (eds.) Advances in Neural Information Processing Systems, vol. 7, pp. 231–238. MIT Press, Cambridge (1995)
Hashem, S.: Optimal linear combinations of neural networks. Neural Networks 10(4), 599–614 (1997)
Opitz, D., Shavlik, J.: Actively searching for an effective neural-network ensemble. Connection Science 8(3/4), 337–353 (1996a)
Opitz, D., Shavlik, J.: Generating accurate and diverse members of a neural-network ensemble. In: Touretsky, D., Mozer, M., Hasselmo, M. (eds.) Advances in Neural Information Processing Systems, vol. 8, pp. 535–541. MIT Press, Cambridge (1996b)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2010 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Chorbev, I., Andovska, M. (2010). Applying Bagging Techniques to the SA Tabu Miner Rule Induction Algorithm. In: Davcev, D., Gómez, J.M. (eds) ICT Innovations 2009. ICT Innovations 2009. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-10781-8_9
Download citation
DOI: https://doi.org/10.1007/978-3-642-10781-8_9
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-642-10780-1
Online ISBN: 978-3-642-10781-8
eBook Packages: EngineeringEngineering (R0)