Summary
A previous work explores a Multi-Objective Subset Selection algorithm, denominated the Pareto Front Elite, to induce classifiers. These classifiers are composed by a set of rules selected following Pareto dominance concepts and forming unordered classifiers. These rules are previously created by an association rule algorithm. The performance of the classifiers induced were compared with other well known rule induction algorithms using the area under the ROC curve. The area under the ROC curve (AUC) is considered a relevant criterion to deal with imbalanced data, misclassification costs and noisy data. The results show that the Pareto Front Elite algorithm is comparable to the best known techniques. In this paper we explore multi-objective meta-heuristic approach to create rules and to build the Pareto Front using the sensitivity and specificity criteria, the chosen Metaheuristic is a Greedy Randomized Adaptive Search Procedure (GRASP) with path-relinking. We perform an experimental study to compare the two algorithms: one based on a complete set of rules, and the other based on Metaheuristic Approach. In this study we analyze the classification results, through the AUC criterion, and the Pareto Front coverage produced by each algorithm.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Agrawal, R., Srikant, R.: Fast algorithms for mining association rules. In: Bocca, J.B., Jarke, M., Zaniolo, C. (eds.) Proc. 20th Int. Conf. Very Large Data Bases, VLDB, pp. 487–499, 12–15. Morgan Kaufmann, San Francisco (1994)
Batista, G., Milare, C., Prati, R.C., Monard, M.: A comparison of methods for rule subset selection applied to associative classification. Inteligencia Artificial. Revista Iberoamericana de IA (32), 29–35 (2006)
Bleuler, S., Laumanns, M., Thiele, L., Zitzler, E.: PISA – A platform and programming language independent interface for search algorithms. In: Fonseca, C.M., Fleming, P.J., Zitzler, E., Deb, K., Thiele, L. (eds.) EMO 2003, vol. 2632, pp. 494–508. Springer, Heidelberg (2003)
Clark, P., Niblett, T.: The CN2 induction algorithm. Machine Learning 3, 261–283 (1989)
Clark, P., Niblett, T.: Rule induction with cn2: Some recent improvements. In: ECML: European Conference on Machine Learning. Springer, Berlim (1991)
Cohen, W.W.: Fast effective rule induction. In: ICML, pp. 115–123 (1995)
Cohen, W.W., Singer, Y.: A simple, fast, and effective rule learner. In: Proceedings of the 6th National Conference on Artificial Intelligence (AAAI 1999); Proceedings of the 11th Conference on Innovative Applications of Artificial Intelligence, July 18–22, pp. 335–342. AAAI/MIT Press (1999)
Conover, W.J.: Practical nonparametric statistics. Wiley, Chichester (1971)
de la Iglesia, B., Philpott, M.S., Bagnall, A.J., Rayward-Smith, V.J.: Data mining rules using multi-objective evolutionary algorithms. In: Congress on Evolutionary Computation, pp. 1552–1559. IEEE Computer Society, Los Alamitos (2003)
de la Iglesia, B., Reynolds, A., Rayward-Smith, V.J.: Developments on a multi-objective metaheuristic (MOMH) algorithm for finding interesting sets of classification rules. In: Coello Coello, C.A., Hernández Aguirre, A., Zitzler, E. (eds.) EMO 2005. LNCS, vol. 3410, pp. 826–840. Springer, Heidelberg (2005)
Newman, C.B.D.J., Merz, C.: UCI repository of machine learning databases (1998)
Domingos, P.: Unifying instance-based and rule-based induction. Machine Learning 24(2), 141–168 (1996)
Ehrgott, M.: Approximation algorithms for combinatorial multicriteria optimization problems. International Transactions in Operational Research 7, 5–31 (2000)
Fawcett, T.: Using rule sets to maximize ROC performance. In: Cercone, N., Lin, T.Y., Wu, X. (eds.) ICDM, pp. 131–138. IEEE Computer Society, Los Alamitos (2001)
Feo, T.A., Resende, M.G.C.: Greedy randomized adaptive search procedures. Journal of Global Optimization 6, 109–133 (1995)
Ferri, C., Flach, P., Hernandez-Orallo, J.: Learning decision trees using the area under the ROC curve. pp. 139–146 (July 2002)
Festa, P., Resende, M.G.C.: GRASP: An annotated bibliography. Technical report, AT& T Labs Research, Florham Park, NJ (January 2001)
Glover, F.: Tabu search and adaptive memory programming - advances, applications and challenges. In: Barr, R.S., Helgason, R.V., Kennington, J.L. (eds.) Interfaces in Computer Science and Operations Research, pp. 1–75. kluwer, Dordrecht (1996)
Hansen, M.P., Jaszkiewicz, A.: Evaluating the quality of approximations to the non-dominated set. Technical Report IMM-REP-1998-7, Technical University of Denmark (March 1998)
Ishibuchi, H.: Multiobjective association rule mining. In: PPSN Workshop on Multiobjective Problem Solving from Nature, pp. 39–48. Reykjavik, Iceland (2006)
Ishibuchi, H., Nojima, Y.: Accuracy-complexity tradeoff analysis by multiobjective rule selection. In: ICDM, pp. 39–48. IEEE Computer Society, Los Alamitos (2005)
Ishida, C.Y., de Carvalho, A.B., Pozo, A.T.R., Goldbarg, E.F.G., Goldbarg, M.C.: Exploring multi-objective pso and grasp-pr for rule induction. In: Eighth European Conference on Evolutionary Computation in Combinatorial Optimisation. Springer, Heidelberg (to appear, 2008)
Ishida, C.Y., Pozo, A.T.R.: Optimization of the auc criterion for rule subset selection. In: Intelligent Systems Design and Applications, 7th. International Conference on Intelligent Systems Design and Application. IEEE Computer Society, New York (2007)
Ishida, C.Y., Pozo, A.T.R.: Pareto front elite. In: XXVII Congresso SBC 2007. ENIA VI Encontro Nacional de Inteligencia Artificial (2007)
Jin, Y.: Multi-Objective Machine Learning. Springer, Berlin (2006)
Jovanoski, V., Lavrač, N.: Classification rule learning with APRIORI-C. In: Brazdil, P.B., Jorge, A.M. (eds.) EPIA 2001. LNCS (LNAI), vol. 2258, pp. 44–51. Springer, Heidelberg (2001)
Knowles, J., Thiele, L., Zitzler, E.: A tutorial on the performance assessment of stochastic multiobjective optimizers. 214, Computer Engineering and Networks Laboratory (TIK), Swiss Federal Institute of Technology (ETH) Zurich (July 2005)
Knowles, J., Thiele, L., Zitzler, E.: A Tutorial on the Performance Assessment of Stochastic Multiobjective Optimizers. 214, Computer Engineering and Networks Laboratory (TIK), ETH Zurich, Switzerland, revised version (Feburary 2006)
Laguna, M., Marti, R.: Grasp and path relinking for 2-layer straight line crossing minimization. INFORMS J. on Computing 11(1), 44–52 (1999)
Lavrač, N., Flach, P., Zupan, B.: Rule evaluation measures: A unifying view. In: Džeroski, S., Flach, P.A. (eds.) ILP 1999. LNCS (LNAI), vol. 1634, pp. 174–185. Springer, Heidelberg (1999)
Li, W., Han, J., Pei, J.: CMAR: Accurate and efficient classification based on multiple class-association rules. In: Cercone, N., Lin, T.Y., Wu, X. (eds.) ICDM, pp. 369–376. IEEE Computer Society, Los Alamitos (2001)
Liu, B., Hsu, W., Ma, Y.: Integrating classification and association rule mining. In: Knowledge Discovery and Data Mining, pp. 80–86 (1998)
Prati, R.C., Flach, P.A.: ROCCER: An algorithm for rule learning based on ROC analysis. In: Kaelbling, L.P., Saffiotti, A. (eds.) IJCAI, pp. 823–828. Professional Book Center (2005)
Provost, F., Domingos, P.: Tree induction for probability based ranking. Machine Learning 52(3), 199–215 (2003)
Provost, F.J., Fawcett, T.: Analysis and visualization of classifier performance: Comparison under imprecise class and cost distributions. In: KDD, pp. 43–48 (1997)
Quinlan, J.: C4. 5: Programs for Machine Learning. Morgan Kaufmann, San Francisco (1993)
Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Francisco (1992)
Rakotomamonjy, A.: Optimizing area under roc curve with SVMs. In: Hernández-Orallo, J., Ferri, C., Lachiche, N., Flach, P.A. (eds.) ROCAI, pp. 71–80 (2004)
Resende, M., Ribeiro, C.: Greedy randomized adaptive search procedures. In: Glover, F., Kochenberger, G. (eds.) Handbook of Metaheuristics, pp. 219–249. Kluwer Academic Publishers, Dordrecht (2002)
Sebag, A., Lucas: ROC-based evolutionary learning: Application to medical data mining. In: International Conference on Artificial Evolution, Evolution Artificielle. LNCS, vol. 6 (2003)
Shapiro, S.S., Wilk, M.B.: An analysis of variance test for normality (complete samples). Biometrika 52(3-4), 591–611 (1965)
Yin, X., Han, J.: Cpar: Classification based on predictive association rules. In: Proceedings SIM International Conference on Data Mining (SDM 2003), pp. 331–335 (2003)
Zitzler, E., Thiele, L.: Multiobjective Evolutionary Algorithms: A Comparative Case Study and the Strength Pareto Approach. IEEE Transactions on Evolutionary Computation 3(4), 257–271 (1999)
Zitzler, E., Thiele, L., Laumanns, M., Fonseca, C.M., da Fonseca, V.G.: Performance assessment of multiobjective optimizers: an analysis and review. IEEE-EC 7, 117–132 (2003)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer-Verlag Berlin Heidelberg
About this chapter
Cite this chapter
Ishida, C.Y., Pozo, A., Goldbarg, E., Goldbarg, M. (2009). Multiobjective Optimization and Rule Learning: Subselection Algorithm or Meta-heuristic Algorithm?. In: Nedjah, N., de Macedo Mourelle, L., Kacprzyk, J. (eds) Innovative Applications in Data Mining. Studies in Computational Intelligence, vol 169. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-88045-5_3
Download citation
DOI: https://doi.org/10.1007/978-3-540-88045-5_3
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-88044-8
Online ISBN: 978-3-540-88045-5
eBook Packages: EngineeringEngineering (R0)