Skip to main content
SpringerLink
Log in

A multi-objective GRASP for partial classification

  • Focus
  • Published:
Soft Computing Aims and scope Submit manuscript

Abstract

Metaheuristic algorithms have been used successfully in a number of data mining contexts and specifically in the production of classification rules. Classification rules describe a class of interest or a subset of this class, and as such may also be used as an aid in prediction. The production and selection of classification rules for a particular class of the database is often referred to as partial classification. Since partial classification rules are often evaluated according to a number of conflicting objectives, the generation of such rules is a task that is well suited to a multi-objective (MO) metaheuristic approach. In this paper we discuss how to adapt well known MO algorithms for the task of partial classification. Additionally, we introduce a new MO algorithm for this task based on a greedy randomized adaptive search procedure (GRASP). GRASP has been applied to a number of problems in combinatorial optimization, but it has very seldom been used in a MO setting, and generally only through repeated optimization of single objective problems, using either linear combinations of the objectives or additional constraints. The approach presented takes advantage of some specific characteristics of the data mining problem being solved, allowing for the very effective construction of a set of solutions that form the starting point for the local search phase of the GRASP. The resulting algorithm is guided solely by the concepts of dominance and Pareto-optimality. We present experimental results for our partial classification GRASP and other MO metaheuristics. These show that such algorithms are generally very well suited to this data mining task and furthermore, the GRASP brings additional efficiency to the search for partial classification rules.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  • Abello J, Pardalos PM, Resende MGC (1999) On Maximum clique problems in very large graphs. In: Abello J, Vitter J (eds) External memory algorithms and visualization, vol 50. American Mathematical Society, New York, pp 119–130

    Google Scholar 

  • Aiex RM, Binato S, Resende MGC (2003) Parallel GRASP with path-relinking for job shop scheduling. Parallel Comput 29: 393–430

    Article  MathSciNet  Google Scholar 

  • Ali K, Manganaris S, Srikant R (1997) Partial classification using association rules. In: Proceedings of the third international conference on knowledge discovery and data mining, pp 115–118

  • Andreatta AA, Ribeiro CC (2002) Heuristics for the phylogeny problem. J Heuristics 8: 429–447

    Article  MATH  Google Scholar 

  • Argüello MF, Bard JF (1997) A GRASP for aircraft routing in response to groundings and delays. J Comb Optim 5: 211–228

    Article  Google Scholar 

  • Bayardo RJ Jr, Agrawal R (1999) Mining the most interesting rules. In: Proceedings of the 5th international conference on knowledge discovery and data mining (KDD ’99), pp 145–154

  • Binato S, Hery WJ, Loewenstern DM, Resende MGC (2002) A greedy randomized adaptive search procedure for job shop scheduling. In: Ribeiro CC, Hansen P (eds) Essays and surveys in metaheuristics. Kluwer, Dordrecht, pp 58–79

    Google Scholar 

  • Bishop CM (1995) Neural networks for pattern recognition. Oxford University Press, New York

    Google Scholar 

  • Breiman L, Friedman JH, Olshen RA, Stone C (1984) Classification and regression trees. Wadsworth, Belmont

    MATH  Google Scholar 

  • Cano JR, Cordón O, Herrera F, Sánchez L (2002) A greedy randomized adaptive search procedure applied to the clustering problem as an initialization process using K-Means as a local search procedure. J Intell Fuzzy Syst 12(3–4): 235–242

    MATH  Google Scholar 

  • Cristianini N, Shawe-Taylor J (2000) An introduction to support vector machines and other kernel-based learning methods. Cambridge University Press, London

    Google Scholar 

  • de la Iglesia B, Philpott MS, Bagnall AJ, Rayward-Smith VJ (2003) Data mining rules using multi-objective evolutionary algorithms. In: Proceedings of the 2003 IEEE congress on evolutionary computation, pp 1552–1559

  • de la Iglesia B, Reynolds A, Rayward-Smith VJ (2005) Developments on a multi-objective metaheuristic (MOMH) algorithm for finding interesting sets of classification rules. In: Evolutionary multi-criterion optimization: third international conference, EMO 2005, pp 826–840

  • de la Iglesia B, Richards G, Philpott MS, Rayward-Smith VJ (2006) The application and effectiveness of a multi-objective metaheuristic algorithm for partial classification. Eur J Oper Res 169(3): 898–917

    Article  MATH  Google Scholar 

  • Deb K (2001) Multi-objective optimization using evolutionary algorithms. Wiley, New York

    MATH  Google Scholar 

  • Deb K, Pratap A, Agarwal S, Meyarivan T (2002) A fast and elitist multiobjective genetic algorithm: NSGA-II. IEEE Trans Evol Comput 6(2): 182–197

    Article  Google Scholar 

  • Feo TA, Resende MGC (1995) Greedy randomized adaptive search procedures. J Glob Optim 6: 109–133

    Article  MATH  MathSciNet  Google Scholar 

  • Festa P, Resende MGC (2004) An annotated bibliography of GRASP. Technical Report TD-5WYSEW, AT and T Labs

  • Fieldsend JE, Everson RM, Singh S (2003) Using unconstrained elite archives for multi-objective optimization. IEEE Trans Evol Comput 7(3): 305–323

    Article  Google Scholar 

  • Ghosh A, Nath B (2004) Multi-objective rule mining using genetic algorithms. Inf Sci 163: 123–133

    Article  MathSciNet  Google Scholar 

  • Han J, Kamber M (2006) Data mining: concepts and techniques, 2nd edn. Morgan Kaufmann, Menlo Park

    Google Scholar 

  • Hettich S, Bay DD (1999) The UCI KDD archive. http://kdd.ics.uci.edu

  • Ishibuchi H, Murata T (1998) A multi-objective genetic local search algorithm and its application to flowshop scheduling. IEEE Trans Syst Man Cybern Part C Appl Rev 28(3): 392–403

    Article  Google Scholar 

  • Ishibuchi H, Murata T (1999) Local search procedures in a multi-objective genetic local search algorithm for scheduling problems. In: Proceedings of the IEEE international conference on systems, man, and cybernetics, vol 1, pp 665–670

  • Jaszkiewicz A (2002) Genetic local search for multi-objective combinatorial optimization. Eur J Oper Res 137(1): 50–71

    Article  MATH  MathSciNet  Google Scholar 

  • Kaufman L, Rousseuw PJ (1990) Finding groups in data: an introduction to cluster analysis, Wiley series in probability and mathematical statistics. Wiley, New York

    Google Scholar 

  • Khabzaoui M, Dhaenens C, Talbi E-G (2005) Parallel Genetic algorithms for multi-objective rule mining. In: Proceedings of the 6th metaheuristics international conference (MIC 2005), pp 571–576

  • Knowles JD, Corne D (1999) The pareto archived evolution strategy: a new baseline algorithm for pareto multiobjective optimisation. In: Angeline PJ, Michalewicz Z, Schoenauer M, Yao X, Zalzala A (eds) Proceedings of the 1999 congress on evolutionary computation (CEC ’99), vol 1, pp 98–105

  • Knowles JD, Corne D (2000) Approximating the nondominated front using the pareto archived evolution strategy. Evol Comput 8(2): 149–172

    Article  Google Scholar 

  • Lourenço HR, Paixpo JP, Portugal R (2001) Multiobjective metaheuristics for the bus-driver scheduling problem. Transp Sci 35(3): 331–343

    Article  MATH  Google Scholar 

  • Murphey RA, Pardalos PM, Pasiliao E (2000) Multicriteria optimization for frequency assignment. In: Rajasekaran S, Pardalos P, Hsu DF (eds) Mobile networks and computing. DIMACS series in discrete mathematics and theoretical computer science, vol 52. American Mathematical Society, New York, pp 203–219

    Google Scholar 

  • Newman DJ, Hettich S, Blake CL, Merz CJ (1998) UCI repository of machine learning databases. http://www.ics.uci.edu/~mlearn/MLRepository.html

  • Okabe T, Jin Y, Sendhoff B (2003) A critical survey of performance indices for multi-objective optimisation. In: Abbass H, Verma B (eds) Proceedings of the 2003 congress on evolutionary computation (CEC 2003), vol 2, pp 878–885

  • Oltean M, Grosan C, Abraham A, Köppen M (2005) Multiobjective optimization using adaptive pareto archived evolution strategy. In: Proceedings of the 5th international conference on intelligent systems design and applications (ISDA ’05), pp 558–563

  • Paquete L, Chiarandini M, Stützle T (2004) Pareto local optimum sets in the biobjective traveling salesman problem: an experimental study. In: Gandibleux X, Sevaux M, Sörensen K, T’kindt V (eds) Metaheuristics for multiobjective optimisation. Lecture notes in economics and mathematical systems, vol 535. Springer, Heidelberg, pp 177–199

  • Pardalos PM, Chaovalitwongse W, Kim D (2003) GRASP with a new local search scheme for vehicle routing problems with time windows. J Comb Optim 7: 179–207

    Article  MATH  MathSciNet  Google Scholar 

  • Pasiliao EL (1998) A greedy randomized adaptive search procedure for the multi-criteria radio link frequency assignment problem. Technical report, Department of Industrial and Systems Engineering, University of Florida, Gainesville, FL, USA

  • Pitsoulis L, Resende MGC (2001) Greedy randomized adaptive search procedures. In: Pitsoulis LS, Resende MGC (eds) Handbook of applied optimization. Oxford University Press, New York, pp 168–181

    Google Scholar 

  • Quinlan JR (1993) C4.5: Programs for machine learning. Morgan Kaufmann, Menlo Park

    Google Scholar 

  • Reynolds A, de la Iglesia B (2006) Rule induction using multi-objective metaheuristics: encouraging rule diversity. In: Proceedings of the 2006 IEEE world congress on computational intelligence, pp 6375–6382

  • Reynolds AP, Richards G, de la Iglesia B, Rayward-Smith VJ (2006) Clustering rules: a comparison of partitioning and hierarchical clustering algorithms. J Math Modell Algorithms 5(4): 475–504

    Article  MATH  Google Scholar 

  • Riddle P, Segal R, Etzioni O (1994) Representation design and brute-force induction in a Boeing manufacturing domain. Appl Artif Intell 8: 125–147

    Article  Google Scholar 

  • Vianna DS, Arroyo JEC (2004) A GRASP algorithm for the multi-objective knapsack problem. In: Proceedings of the XXIV international conference of the Chilean Computer Science Society (SCCC’04), pp 69–75

  • Zitzler E, Laumanns M, Thiele L (2001) SPEA2: improving the strength pareto evolutionary algorithm. Technical Report 103, Computer Engineering and Networks Laboratory (TIK), Department of Electrical Engineering, Swiss Federal Institute of Technology (ETH)

  • Zitzler E, Laumanns M, Thiele L (2002) SPEA2: improving the strength pareto evolutionary algorithm for multiobjective optimization. In: Gainnakoglou K, Tsahalis D, Periaux J, Papailiou K, Fogarty T (eds) Evolutionary methods for design, optimisation and control, pp 95–100

  • Zitzler E, Thiele L (1999) Multiobjective optimization using evolutionary algorithms: a comparative case study and the strength pareto approach. IEEE Trans Evol Comput 3(4): 257–271

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Computing Sciences, University of East Anglia, Norwich, England

    Alan P. Reynolds & Beatriz de la Iglesia

Authors
  1. Alan P. Reynolds
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Beatriz de la Iglesia
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alan P. Reynolds.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Reynolds, A.P., de la Iglesia, B. A multi-objective GRASP for partial classification. Soft Comput 13, 227–243 (2009). https://doi.org/10.1007/s00500-008-0320-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00500-008-0320-1

Keywords

Search

Navigation