Skip to main content
SpringerLink
Log in

New Algorithms for Generation Decision Trees—Ant-Miner and Its Modifications

  • Chapter
Foundations of Computational, IntelligenceVolume 6

Part of the book series: Studies in Computational Intelligence ((SCI,volume 206))

Abstract

In our approach we want to ensure the good performance of Ant- Miner by applying the well-known (from the ACO algorithm) two pheromone updating rules: local and global, and the main pseudo-random proportional rule, which provides appropriate mechanisms for search space: exploitation and exploration. Now we can utilize an improved expression of this classification rule discovery system as an Ant-Colony-Miner. Further modifications are connected with the simplicity of the heuristic function used in the standard Ant-Miner. We propose to employing a new heuristic function based on quantitative, not qualitative parameters used during the classification process. The main transition rule will be changed dynamically as a result of the simple frequency analysis of the number of cases from the point of view characteristic partitions. This simplified heuristic function will be compensated by the pheromone update in different degrees, which helps ants to collaborate and is a good stimulant on ants’ behavior during the rule construction. The comparative study will be conducted using 5 data sets from the UCI Machine Learning repository.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Corne, D., et al.: New Ideas in Optimization. Mc Graw-Hill, Cambridge (1999)

    Google Scholar 

  2. Bauer, A., Bullnheimer, B., Hartl, R.F., Strauss, C.: An Ant Colony Optimization approach for the single machine total tardiness problem. In: Proceedings of the 1999 Congress on Evolutionary Computation, pp. 1445–1450. IEEE Press, Piscataway (1999)

    Chapter  Google Scholar 

  3. Boffey, B.: Multiobjective routing problems. Top 3(2), 167–220 (1995)

    Article  MATH  MathSciNet  Google Scholar 

  4. Bonabeau, E., Dorigo, M., Theraulaz, G.: Swarm Intelligence. In: From Natural to Artificial Systems. Oxford University Press, Oxford (1999)

    Google Scholar 

  5. Bonabeau, E., Henaux, F., Guérin, S., Snyers, D., Kuntz, P., Théraulaz, G.: Routing in telecommunication networks with ”Smart” ant–like agents telecommunication applications. Springer, Heidelberg (1998)

    Google Scholar 

  6. Breiman, L., Friedman, J.H., Olshen, R.A., Stone, C.J.: Classification and Regression Trees, Belmont C.A., Wadsworth (1984)

    Google Scholar 

  7. Bullnheimer, B., Hartl, R.F., Strauss, C.: An improved Ant System algorithm for the Vehicle Routing Problem. Technical Report POM–10/97, Institute of Management Science, University of Vienna (1997)

    Google Scholar 

  8. Bullnheimer, B., Hartl, R.F., Strauss, C., Bullnheimer, B., Hartl, R.F., Strauss, C.: A new rankbased version of the Ant System: A computational study. Technical Report POM–03/97, Institute of Management Science, University of Vienna (1997)

    Google Scholar 

  9. Bullnheimer, B., Hartl, R.F., Strauss, C.: Applying the Ant System to the Vehicle Routing Problem. In: Martello, S., Osman, I.H., Voß, S., Martello, S., Roucairoll, C. (eds.) MetaHeuristics: Advances and Trends in Local Search Paradigms for Optimization, pp. 109–120. Kluwer Academics, Dordrecht (1998)

    Google Scholar 

  10. Bullnheimer, B., Strauss, C., Bullnheimer, B., Hartl, R.F., Strauss, C.: Instituts für Betriebwirtschaftslehre, Universität Wien (1996)

    Google Scholar 

  11. Chan, A., Freitas, A.A.: A new ant colony algorithm for multi-label alssification with applications in bioinformatics. In: Proceedings of Genetic and Evolutionary Computation Conf (GECCO 2006), San Francisco, pp. 27–34 (2006)

    Google Scholar 

  12. Chen, C., Chen, Y., He, J.: Neural network ensemble based ant colony classification rule mining. In: Proceedings of First Int. Conf. Innovative Computing, Information and Control (ICICIC 2006), pp. 427–430 (2006)

    Google Scholar 

  13. Chen, Z.: Data Mining and uncertain reasoning. An integrated approach. John Wiley and Sons, Chichester (2001)

    Google Scholar 

  14. Clark, P., Boswell, R.: Rule induction with CN2: some recent improvements. In: Kodratoff, Y. (ed.) EWSL 1991. LNCS (LNAI), vol. 482, pp. 151–163. Springer, Heidelberg (1991)

    Chapter  Google Scholar 

  15. Clark, P., Niblett, T.: The CN2 rule Induction algorithm. Machine Learning 3(4), 261–283 (1989)

    Google Scholar 

  16. Colorni, A., Dorigo, M., Maniezzo, V.: Distributed optimization by ant colonies. In: Vavala, F., Bourgine, P. (eds.) Proceedings First Europ. Conference on Artificial Life, pp. 134–142. MIT Press, Cambridge (1991)

    Google Scholar 

  17. Colorni, A., Dorigo, M., Maniezzo, V., Trubian, M.: Ant system for job–shop scheduling. Belgian Journal of Operations Research, Statistics and Computer Science (JORBEL) 34, 39–53 (1994)

    MATH  Google Scholar 

  18. Costa, D., Hertz, A.: Ants can colour graphs. Journal of the Operational Research Society 48, 295–305 (1997)

    Article  MATH  Google Scholar 

  19. Den Besten, M., Stützle, T., Dorigo, M.: Scheduling single machines by ants. Technical Report 99–16, IRIDIA, Université Libre de Bruxelles, Belgium (1999)

    Google Scholar 

  20. Deneubourg, J.–. L., Goss, S., Franks, N.R., Pasteels, J.M.: The Blind Leading the Blind: Modelling Chemically Mediated Army Ant Raid Patterns. Insect Behaviour 2, 719–725 (1989)

    Article  Google Scholar 

  21. DiCaro, G., Dorigo, M.: AntNet: A mobile agents approach to adaptive routing. Technical report, IRIDIA, Université Libre de Bruxelles (1998)

    Google Scholar 

  22. DiCaro, G., Dorigo, M.: AntNet: Distributed stigmergetic control for communications networks. Journal of Artificial Intelligence Research (JAIR) 9, 317–365 (1998)

    Google Scholar 

  23. DiCaro, G., Dorigo, M.: Extending AntNet for best–effort Quality–of–Service routing. In: ANTS 1998 – From Ant Colonies to Artificial Ants: First International Workshop on Ant Colony Optimization, October 15–16 (1998) (Unpublished presentation)

    Google Scholar 

  24. DiCaro, G., Dorigo, M.: Two ant colony algorithms for best–effort routing in datagram networks. In: Proceedings of the Tenth IASTED International Conference on Parallel and Distributed Computing and Systems (PDCS 1998), pp. 541–546. IASTED/ACTA Press (1998)

    Google Scholar 

  25. Dorigo, M.: Optimization, Learning and Natural Algorithms (in Italian). PhD thesis, Dipartimento di Elettronica, Politecnico di Milano, IT (1992)

    Google Scholar 

  26. Dorigo, M., DiCaro, G.: The ant colony optimization meta–heuristic. In: Corne, D., Dorigo, M., Glover, F. (eds.) New Ideas in Optimization. McGraw–Hill, London (1999)

    Google Scholar 

  27. Dorigo, M., DiCaro, G., Gambardella, L.: Ant algorithms for distributed discrete optimization. Artif. Life 5(2), 137–172 (1999)

    Article  Google Scholar 

  28. Dorigo, M., Gambardella, L.: A Study of Some Properties of Ant–Q. In: Proceedings of Fourth International Conference on Parallel Problem Solving from Nature, PPSNIV, pp. 656–665. Springer, Berlin (1996)

    Chapter  Google Scholar 

  29. Dorigo, M., Gambardella, L.: Ant Colonies for the Traveling Salesman Problem. Biosystems 43, 73–81 (1997)

    Article  Google Scholar 

  30. Dorigo, M., Gambardella, L.: Ant Colony System: A Cooperative Learning Approach to the Traveling Salesman Problem. IEEE Trans. Evol. Comp. 1, 53–66 (1997)

    Article  Google Scholar 

  31. Dorigo, M., Maniezzo, V., Colorni, A.: Positive feedback as a search strategy. Technical Report 91–016, Politechnico di Milano, Italy (1991)

    Google Scholar 

  32. Dorigo, M., Maniezzo, V., Colorni, A.: The Ant System: Optimization by a Colony of Cooperating Agents. IEEE Trans. Syst. Man. Cybern. B26, 29–41 (1996)

    Google Scholar 

  33. Dorigo, M., Stützle, T.: Ant Colony Optimization. MIT Press, Cambridge (2004)

    MATH  Google Scholar 

  34. Freitas, A.A., Johnson, C.G.: Research cluster in swarm intelligence. Technical Report EPSRC Research Proposal GR/S63274/01 — Case for Support, Computing Laboratory, Computing Laboratory, Laboratory of Kent, Kent (2003)

    Google Scholar 

  35. Galea, M.: Applying swarm intelligence to rule induction. MS thesis, University of Edingbourgh (2002)

    Google Scholar 

  36. Galea, M., Shen, Q.: Simultaneous ant colony optimization algorithms for learning linguistic fuzzy rules. In: Agraham, A., Grosan, C., Ramos, V. (eds.) Swarm Intelligence in Data Mining. Springer, Berlin (2006)

    Google Scholar 

  37. Gambardella, L.M., Dorigo, M.: AntQ.Ant–Q. A Reinforcement Learning Approach to the Traveling Salesman Problem. In: Proceedings of Twelfth International Conference on Machine Learning, pp. 252–260. Morgan Kaufman, Palo Alto (1995)

    Google Scholar 

  38. Gambardella, L.M., Dorigo, M.: Solving symmetric and asymmetric TSPs by ant colonies. In: Proceedings of the IEEE Conference on Evolutionary Computation, ICEC 1996, pp. 622–627. IEEE Press, Los Alamitos (1996)

    Chapter  Google Scholar 

  39. Gambardella, L.M., Dorigo, M.: HAS–SOP: Hybrid Ant System for the Sequential Ordering Problem. Technical Report 11, IDSIA Lugano (1997)

    Google Scholar 

  40. Gambardella, L.M., Taillard, E., Agazzi, G.: MACS–VRPTW: A Multiple Ant Colony System for Vehicle Routing Problems with Time Windows. Technical Report 06–99, IDSIA, Lugano, Switzerland (1999)

    Google Scholar 

  41. Gambardella, L.M., Taillard, E.D., Dorigo, M.: Ant colonies for the QAP. Technical Report 4–97, IDSIA, Lugano, Switzerland (1997)

    Google Scholar 

  42. Gambardella, L.M., Taillard, E.D., Dorigo, M.: Ant colonies for the QAP. Journal of the Operational Research Society (JORS) 50(2), 167–176 (1999)

    MATH  Google Scholar 

  43. Glover, F., Laguna, M.: Tabu Search. Kluwer Academic Publishers, Dordrecht (1997)

    MATH  Google Scholar 

  44. Goss, S., Beckers, R., Denebourg, J.L., Aron, S., et al.: How Trail Laying and Trail Following Can Solve Foraging Problems for Ant Colonies. In: Hughes, R.N. (ed.) Behavioural Mechanisms for Food Selection, vol. G20. Springer, Berlin (1990)

    Google Scholar 

  45. Grasse, P.-P.: La Reconstruction du Nid et les Coordinations Inter–Individuelles chez Bellicositermes Natalensis et Cubitermes sp. La Theorie de La Stigmerie. Insects Soc. 6, 41–80 (1959)

    Article  Google Scholar 

  46. Grasse, P.-P.: Termitologia, vol. II, Paris, Masson (1984)

    Google Scholar 

  47. Heusse, M., Guérin, S., Snyers, D., Kuntz, P.: Adaptive agent–driven routing and load balancing in communication networks. Technical Report RR–98001–IASC, Départment Intelligence Artificielle et Sciences Cognitives, ENST Bretagne, ENST Bretagne (1998)

    Google Scholar 

  48. Smaldon, J., Freitas, A.A.: A new version of the Ant-Miner algorithm discovering unordered rule sets. In: Proceedings of Genetic and Evolutionary Computation Conf (GECCO 2006), San Francisco, pp. 43–50 (2006)

    Google Scholar 

  49. Kohavi, R., Sahami, M.: Error-based and entropy-based discretization of continuous features. In: Proc. 2nd Intern. Conference Knowledge Discovery and Data Mining, pp. 114–119 (1996)

    Google Scholar 

  50. Leguizamón, G., Michalewicz, Z.: A new version of Ant System for subset problems. In: Proceedings of the 1999 Congress on Evolutionary Computation, pp. 1459–1464. IEEE Press, Piscataway (1999)

    Chapter  Google Scholar 

  51. Liang, Y.–C., Smith, A.E.: An Ant System approach to redundancy allocation. In: Proceedings of the 1999 Congress on Evolutionary Computation, pp. 1478–1484. IEEE Press, Piscataway (1999)

    Chapter  Google Scholar 

  52. Liu, B., Abbas, H.A., Mc Kay, B.: Classification rule discovery with ant colony optimization. IEEE Computational Intelligence Bulletin 1(3), 31–35 (2004)

    Google Scholar 

  53. Ramalhinho Lourenço, H., Serra, D.: Adaptive approach heuristics for the generalized assignment problem. Technical Report EWP Series No. 304, Department of Economics and Management, Universitat Pompeu Fabra, Barcelona (1998)

    Google Scholar 

  54. Maniezzo, V.: Exact and approximate nondeterministic tree–search procedures for the quadratic assignment problem. Technical Report CSR 98–1, C. L. In: Scienze dellInformazione, Universita di Bologna, sede di Cesena, Italy (1998)

    Google Scholar 

  55. Maniezzo, V., Carbonaro, A.: An ANTS heuristic for the frequency assignment problem. Technical Report CSR 98–4, Scienze dell Informazione, Universita di Bologna, Sede di Cesena, Italy (1998)

    Google Scholar 

  56. Maniezzo, V., Colorni, A.: The Ant System applied to the Quadratic Assignment Problem. IEEE Trans. Knowledge and Data Engineering (1999)

    Google Scholar 

  57. Maniezzo, V., Colorni, A.: An ANTS heuristic for the frequency assignment problem. Future Generation Computer Systems 16, 927–935 (2000)

    Article  Google Scholar 

  58. Maniezzo, V., Colorni, A., Dorigo, M.: The Ant System applied to the Quadratic Assignment Problem. Technical Report 94–28, IRIDIA, Université Libre de Bruxelles, Belgium (1994)

    Google Scholar 

  59. Martens, D., De Backer, M., Haesen, R., Baesens, B., Holvoet, T.: Ants constructing rule-based classifiers. In: Agraham, A., Grosan, C., Ramos, V. (eds.) Swarm Intelligence in Data Mining. Springer, Berlin

    Google Scholar 

  60. Michalski, R., Mozetic, J., Hong, J., Lavrac, N.: The multi-purpose incremental learning system AQ15 and its testing application to three medical domains. In: AAAI 1986, vol. 2, pp. 1041–1045 (1987)

    Google Scholar 

  61. Michel, R., Middendorf, M.: An island model based Ant System with lookahead for the Shortest Supersequence Problem. In: Eiben, A.E., Back, T., Schoenauer, M., Schwefel, H.–P. (eds.) Proceedings of PPSN–V, Fifth International Conference on Parallel Problem Solving from Nature, pp. 692–701. Springer, Heidelberg (1998)

    Chapter  Google Scholar 

  62. Michel, R., Middendorf, M.: An ACO algorithm for the Shortest Common Supersequence Problem. In: Corne, D., Dorigo, M., Glover, F. (eds.) New Methods in Optimisation. McGraw-Hill, New York (1999)

    Google Scholar 

  63. Oakes, M.P.: Ant colony optimization for stylometry: the federalist papers. In: Proceedings of Recent Advances in Soft Computing (RASC 2004), pp. 86–91 (2004)

    Google Scholar 

  64. Osman, I., Laporte, G.: Metaheuristics: A bibliography. Annals of Operations Research 63, 513–623

    Google Scholar 

  65. Parpinelli, R.S., Lopes, H.S., Freitas, A.A.: An ant colony algorithm for classification rule discovery. In: Abbas, H., Sarker, R., Newton, C. (eds.) Data Mining: a Heuristic Approach. Idea Group Publishing, London (2002)

    Google Scholar 

  66. Parpinelli, R.S., Lopes, H.S., Freitas, A.A.: Data mining with an ant colony optimization algorithm. IEEE Transactions on Evolutionary Computation, Special issue on Ant Colony Algorithms 6(4), 321–332 (2004)

    Google Scholar 

  67. Quinlan, J.R.: Introduction of decision trees. Machine Learning 1, 81–106 (1986)

    Google Scholar 

  68. Quinlan, J.R.: Generating production rules from decision trees. In: Proc. of the Tenth International Joint Conference on Artificial Intelligence, pp. 304–307. Morgan Kaufmann, San Francisco (1987)

    Google Scholar 

  69. Quinlan, J.R.: C4.5: Programs for Machine Learning. Morgan Kaufmann, San Francisco (1993)

    Google Scholar 

  70. Reeves, C.: Modern Heuristic Techniques for Combinatorial Problems. In: Advanced Topics in Computer Science. McGrawHill, London (1995)

    Google Scholar 

  71. Schoonderwoerd, R., Holland, O., Bruten, J.: Ant–like agents for load balancing in telecommunications networks. In: Proceedings of the First International Conference on Autonomous Agents, pp. 209–216. ACM Press, New York (1997)

    Chapter  Google Scholar 

  72. Schoonderwoerd, R., Holland, O., Bruten, J., Rothkrantz, L.: Ant–based load balancing in telecommunications networks. Adaptive Behavior 5(2), 169–207 (1996)

    Article  Google Scholar 

  73. Stützle, T.: An ant approach to the Flow Shop Problem. Technical Report AIDA–97–07, FG Intellektik, FB Informatik, TH Darmstadt (September 1997)

    Google Scholar 

  74. Stützle, T., Hoos: Improvements on the Ant System: Introducing MAX–MIN Ant System. In: Improvements on the Ant System: Introducing MAX–MIN Ant System Algorithms, pp. 245–249. Springer, Heidelberg (1997)

    Google Scholar 

  75. Stützle, T., Hoos: The MAX–MIN Ant System and Local Search for the Traveling Salesman Problem. In: Baeck, T., Michalewicz, Z., Yao, X. (eds.) Proceedings of IEEE–ICEC–EPS 1997, IEEE International Conference on Evolutionary Computation and Evolutionary Programming Conference, pp. 309–314. IEEE Press, Los Alamitos (1997)

    Chapter  Google Scholar 

  76. Stützle, T., Hoos: MAX–MIN Ant System and Local Search for Combinatorial Optimisation Problems. In: Proceedings of the Second International conference on Metaheuristics MIC 1997, Kluwer Academic, Dordrecht (1998)

    Google Scholar 

  77. Subramanian, D., Druschel, P., Chen, J.: Ants and Reinforcement Learning: A case study in routing in dynamic networks. In: Proceedings of IJCAI 1997, International Joint Conference on Artificial Intelligence. Morgan Kaufmann, San Francisco (1997)

    Google Scholar 

  78. van der Put, R.: Routing in the faxfactory using mobile agents. Technical Report R&D–SV–98–276, KPN Research (1998)

    Google Scholar 

  79. Navarro Varela, G., Sinclair, M.C.: Ant Colony Optimisation for virtual–wavelength–path routing and wavelength allocation. In: Proceedings of the 1999 Congress on Evolutionary Computation, pp. 1809–1816. IEEE Press, Piscataway (1999)

    Chapter  Google Scholar 

  80. Wang, Z., Feng, B.: Classification rule mining with an improved ant colony algorithm. In: Webb, G.I., Yu, X. (eds.) AI 2004. LNCS (LNAI), vol. 3339, pp. 357–367. Springer, Heidelberg (2004)

    Google Scholar 

  81. White, T., Pagurek, B., Oppacher, F.: Connection management using adaptive mobile agents. In: Arabnia, H.R. (ed.) Proceedings of the International Conference on Parallel and Distributed Processing Techniques and Applications (PDPTA 1998), pp. 802–809. CSREA Press,

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute of Computer Science, University of Silesia, Bȩdzińska 39, 41–200, Sosnowiec, Poland

    Urszula Boryczka & Jan Kozak

Authors
  1. Urszula Boryczka
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jan Kozak
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Machine Intelligence Research Labs, (MIR Labs), Scientific Network for Innovation and Research Excellence, Auburn, P.O. Box 2259, 98071-2259, Washington, USA

    Ajith Abraham

  2. College of Business Administration, Quantitative and Information System Department, Kuwait University, P.O. Box 5486, 13055, Safat, Kuwait

    Aboul-Ella Hassanien

  3. Department of Computer Science, University of São Paulo, Caixa Postal 668, 13560-970, Sao Carlos, SP, Brazil

    André Ponce de Leon F. de Carvalho

  4. Dept. Computer Science, Technical University Ostrava, Tr. 17. Listopadu 15, 708 33, Ostrava, Czech Republic

    Václav Snášel

Rights and permissions

Reprints and permissions

Copyright information

© 2009 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Boryczka, U., Kozak, J. (2009). New Algorithms for Generation Decision Trees—Ant-Miner and Its Modifications. In: Abraham, A., Hassanien, AE., de Leon F. de Carvalho, A.P., Snášel, V. (eds) Foundations of Computational, IntelligenceVolume 6. Studies in Computational Intelligence, vol 206. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-01091-0_11

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-01091-0_11

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-01090-3

  • Online ISBN: 978-3-642-01091-0

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation