Skip to main content
SpringerLink
Log in

Adaptive artificial datasets through learning classifier systems for classification tasks

  • Special Issue
  • Published:
Evolutionary Intelligence Aims and scope Submit manuscript

Abstract

In producing an artificial dataset, humans usually play a major role in creating and controlling the problem domain. In particular, humans set up and tune the problem’s difficulty. If humans can set up the difficulty levels appropriately, then learning systems can solve classification tasks successfully. This paper introduces an autonomous classification problem generation approach. The problem’s difficulty is adapted based on the classification agent’s performance within the defined attributes. An automated problem generator has been created to evolve simulated datasets whilst the classification agent, in this case a learning classifier system (LCS), attempts to learn the evolving datasets. The idea here is to tune the problem’s difficulty autonomously such that the problem’s characteristics may be determined effectively. Furthermore, this framework can empirically test the learning bounds of the classification agent whilst lowering human involvement. Initially, tabu search was integrated in the problem generator to discover the best combination of domain features in order to adjust the problem’s difficulty. In order to overcome stagnation in local optimum, a Pittsburgh-style LCSs, A-PLUS, was adapted for the first time to the problem generator. In this way, the effect of the problem’s characteristics, e.g. noise, which alter the classification agent’s performance, becomes human readable. Experiments confirm that the problem generator was able to tune the problem’s difficulty either to make the problem ‘harder’ or ‘easier’ so that it can either ‘increase’ or ‘decrease’ the classification agent’s performance.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

References

  1. Azuaje F (2003) A computational evolutionary approach to evolving game strategy and cooperation. IEEE Trans Syst Man Cybern 33(3):498–503

    Article  Google Scholar 

  2. Bacardit J (2004) Pittsburgh Genetics-based machine learning in the data mining era: representations, generalization, and run-time. PhD thesis, University of Ramon Llull

  3. Bacardit J, Butz M (2007) Data mining in learning classifier systems: comparing XCS with GAssist. In: Kovacs T, Llorà X, Takadama K, Lanzi PL, Stolzmann W, Wilson SW (eds) Learning Classifier Systems, Lecture Notes in Artificial Intelligence (LNAI), vol 4399. Springer, Berlin, pp 282–290

  4. Bernado E, Garrell J (2003) Accuracy-based learning classifier systems: models, analysis and applications to classification tasks. Evol Comput 11(3):209–238

    Article  Google Scholar 

  5. Bernadó-Mansilla NME, Orriols-Puig A (2008) Preliminary approach on synthetic data sets generation based on class separability measure. In: The 8th international conference on pattern recognition (ICPR 2008). IEEE Xplore, pp 1–4

  6. Bishop CM (2006) Pattern recognition and machine learning. Springer, Berlin

    MATH  Google Scholar 

  7. Butz M (2005) Kernel-based, ellipsoidal conditions in the real-valued XCS classifier system. In: Genetic evolutionary computational conference (GECCO 2005). ACM

  8. Butz M (2006) Rule-based evolutionary online learning systems: a principal approach to LCS analysis and design. Springer, Berlin

    Google Scholar 

  9. Catalin S, Ruxandra S, Preuss M, Dumitrescu D (2008) Coevolution for classification. Technical report, ISSN 1433-3325, Technical University of Dortmund, Department of Computer Science/LS 2, 44221 Dortmund, Germany

  10. Congdon C (1995) A comparison of genetic algorithms and other machine learning systems of a complex classification task from common disease research. PhD thesis, University of Michigan, Michigan, USA

  11. Farahmand AM et al (2010) Interaction of culture-based learning and cooperative co-evolution and its application to automatic behavior-based system design. IEEE Trans Evol Comput 14(1):23–57

    Article  Google Scholar 

  12. Gendreau M (2000) A tutorial on the tabu search. Department of Computer Science, de Montreal University, Canada

    Google Scholar 

  13. Harrison ML, Foster JA (2004) Co-evolving faults to improve the fault tolerance of sorting networks. In: EuroGP, pp 57–66

  14. Himmelstein DS, Greene CS, Moore JH (2011) Evolving hard problems: generating human genetics datasets with a complex etiology. BioData Min 4:21

    Article  Google Scholar 

  15. Ho TK, Basu M (2002) Complexity measure of supervised classification problems. IEEE Trans Pattern Anal Mach Intell 24(2002(3):289–300

    Google Scholar 

  16. Holland JH (1975) Adaptation in natural and artificial systems: an introductory analysis with applications to biology, control, artificial intelligence. University of Michigan Press, Ann Arbor, IL, pp 313–329

    Google Scholar 

  17. Huang C-Y, Sun C-T (2004) Parameter adaptation within co-adaptive learning classifier systems. In: GECCO ’04, pp 774–784

  18. Iqbal M, Browne WN, Zhang M (2012) XCSR with computed continuous action. In: Australisian AI 2012. Springer, Berlin, pp 350–361

  19. Jiadong Yang PJ, Hua Xu (2012) Effective search for Pittsburgh learning classifier systems via estimation of distribution algorithms. Inf Sci 198:100–117

    Article  Google Scholar 

  20. Jong J, Viezzer M, Kain S (2004) Coevolution for classification. Technical report White Paper SSP041001, Thales Research and Technology (UK) Limited, Worton Drive, Worton Grange, Reading, Berkshire, RG2 0SB, UK

  21. Kharbat F, Bull L, Odeh M (2005) Revisiting genetic selection in the XCS learning classifier system. In: The 2005 IEEE congress on evolutionary computation, 2005, vol 3, pp 2061–2068

  22. Kovacs T (2000) Strength or accuracy? Fitness calculation in learning classifier system. In: Lanzi PL, Stolzmann W, Wilson SW (eds) Learning Classifier Systems, Lecture Notes in Computer Science (LNCS), vol 1813. Springer, Berlin, pp 143–160

  23. Kovacs T (2004) Rule fitness and pathology in learning classifier systems. Springer, London

    Google Scholar 

  24. Kushida J, Taniguchi N, Hoshino Y, Kamei K (2007) A coevolutionary system for development of strategies in poker game. In: Second international conference on innovative computing, information and control (ICICIC ’07). IEEE, p 401

  25. Lanzi PL, Riolo RL (2000) A roadmap to the last decade of learning classifier system research (from 1989 to 1999). In: Lanzi PL, Stolzmann W, Wilson SW (eds) Learning Classifier Systems , Lecture Notes in Artificial Intelligence (LNAI), vol 1813. Springer, Berlin, pp 33–61

  26. Llorà X, i Guiu JMG (2002) Coevolving Different knowledge representations with fine-grained parallel learning classifier systems. In: GECCO ’02, pp 934–941

  27. Luke S (2010) Essentials of metaheuristics: a set of undergraduate lecture notes. Department of Computer Science, George Mason University, Fairfax VA

    Google Scholar 

  28. Macià N, Orriols-Puig A, Bernadó-Mansilla E (2008) Genetic-based synthetic data sets for the analysis of classifiers behavior. In: The 8th international conference on hybrid intelligent systems. IEEE Xplore, pp 507–512

  29. Macià N, Orriols-Puig A, Bernadó-Mansilla E (2009) Beyond homemade artificial data sets. Hybrid artif intell syst 5572:605–612

    Google Scholar 

  30. Mani A, Michael K (2009) CoXCS: a coevolutionary learning classifier based on feature space partitioning. In: AI 2009: advances in artificial intelligence, volume 5866 of Lecture Notes in Computer Science (LNCS). Springer, Berlin, pp 360–369

  31. Marzukhi S, Browne WN, Zhang M (2012) Two-cornered learning classifier systems for pattern generation and classification. In: The 12th genetic and evolutionary computation conference (GECCO 2012). ACM, pp 895–902

  32. Michalewicz Z, Fogel D (2000) How to solve it: modern heuristics. Springer, Berlin

    Book  MATH  Google Scholar 

  33. Orriols-Puig A, Casillas J, Bernadó-Mansilla E (2008) Genetic-based machine learning systems are competitive for pattern recognition. Evol Intell 1:209–232

    Article  Google Scholar 

  34. Potter MA, Jong KAD (1994) A cooperative coevolutionary approach to function optimization. In: PPSN, pp 249–257

  35. Potter MA, Jong KAD (2000) Cooperative coevolution: an architecture for evolving ccoadapted subcomponents. Evol Comput 8(1):1–29

    Article  Google Scholar 

  36. Potter MA, Jong KAD, Grefenstette JJ (1995) A coevolutionary approach to learning sequential decision rules. In: ICGA, pp 366–372

  37. Schürmann J (1996) Pattern classification: a unified view of statistical and neural approaches. Wiley, Hoboken, NJ

    Google Scholar 

  38. Scott PD, Wilkins E (1999) Evaluating data mining procedures: techniques for generating artificial data sets. Inf Softw Technol 41(9):579–587

    Article  Google Scholar 

  39. Sigaud O, Wilson SW (2007) Learning classifier systems: a survey. Soft Comput 11(11):1065–1078

    Article  MATH  Google Scholar 

  40. Smith S (1980) A learning system based on genetic algorithms. PhD thesis, University of Pittsburgh, Pittsburgh, PA, USA

  41. Sood NP, Williams AG, De Jong KA (2005) Evaluating the XCS learning classifier system in competitive simultaneous learning environments. In: Proceedings of the 2005 workshops on genetic and evolutionary computation, GECCO ’05, pp 112–118

  42. Stacey A (2004) An investigation of techniques for improving the performance of the Pittsburgh LCS. Technical report UWELCSG04-005, Department o Computer Science, University of Bath, United Kingdom

  43. Troc M, Unold O (2010) Self-adaptation of parameters in a learning classifier system ensemble machine. Appl Math Comput Sci 20(1):157–174

    Google Scholar 

  44. Unold O (2010) Self-adaptive learning classifier system. J Circuits Syst Comput 19(1):275–296

    Article  Google Scholar 

  45. Urbanowicz R, Moore J (2009) Review article learning classifier systems: a complete introduction, review, and roadmap. J Artif Evol Appl 2009:1–25

    Article  Google Scholar 

  46. Weigand R, Liles W, Jong KD (2002) Analyzing cooperative coevolution with evolutionary game theory. In: Congress on evolutionary computation 2002 (CEC’02). IEEE, pp 1600–1605

  47. Wen Y, Xu H (2011) A cooperative coevolution-based pittsburgh learning classifier system embedded with memtic feature selection. In: IEEE congress on evolutionary computation, pp 2415–2422

  48. Wiegand RP, Liles WC, Jong KAD (2002) Modeling variation in cooperative coevolution using evolutionary game theory. In: FOGA, pp 203–220

  49. Wilson S (1995) Classifier fitness based on accuracy. Evol Comput 3(2):149–175

    Article  Google Scholar 

  50. Wilson S (2000) Get real! XCS with continuous-valued inputs. In: Learning classifier systems (IWLCS 1999), vol 1813 of Lecture Notes in Computer Science (LNCS). IEEE, Springer, Berlin, pp 209–219

  51. Wilson S (2009) Coevolution of pattern generators and recognizers. In: GECCO ’09. ACM, New York

Download references

Author information

Authors and Affiliations

  1. Evolutionary Computation Research Group, School of Engineering and Computer Science, Victoria University of Wellington, PO Box 600, Wellington, 6140, New Zealand

    Syahaneim Marzukhi, Will N. Browne & Mengjie Zhang

Authors
  1. Syahaneim Marzukhi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Will N. Browne
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mengjie Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Syahaneim Marzukhi.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Marzukhi, S., Browne, W.N. & Zhang, M. Adaptive artificial datasets through learning classifier systems for classification tasks. Evol. Intel. 6, 93–107 (2013). https://doi.org/10.1007/s12065-013-0094-y

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12065-013-0094-y

Keywords

Search

Navigation