Skip to main content
SpringerLink
Log in

Rule reduction by selection strategy in XCS with adaptive action map

  • Special Issue
  • Published:
Evolutionary Intelligence Aims and scope Submit manuscript

Abstract

The XCS classifier system is a rule-based evolutionary machine learning system. XCS evolves classifiers in order to learn generalized solutions. The XCS with adaptive action mapping (XCSAM) is inherited from XCS, which evolves a best action map where it evolves classifiers that advocate the best action in every state. Accordingly, XCSAM can potentially evolve solutions that are more compact than XCS, which in contrast focuses on a complete action map. Previous experimental results however have shown that, in some problems, XCSAM may produce solutions with more classifiers than XCS. In this paper, we initially show that the original fitness-based selection strategy of XCS produces non effective classifiers which are not likely to be in the best action map (i.e., they are inaccurate ones or do not have best actions) in XCSAM. Then, we introduce a new selection strategy for XCSAM that promotes the evolution of classifiers advocating the best action map and thus produces more compact solutions. The new strategy selects classifiers based both on their fitness (like XCS) and on the parameter optimality of action of XCSAM. The result is a pressure towards classifiers that are accurate and advocate the best actions. We present analyses showing that the new selection strategy successfully enables XCSAM to focus on classifiers having best actions. Our experimental results show that XCSAM with the new selection strategy (called XCSAM-SS) can evolve smaller solutions than XCS (and the original XCSAM) both in single-step and multi-step problems. As a consequence, XCSAM can also learn with smaller iterations than XCS in the single-step problem. Our conclusion is that, as the best action map potentially has a compact solution, XCSAM evolves a much compact solution than XCS by adding an adequate selection strategy.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Similar content being viewed by others

Notes

  1. XCSAM uses the selection array for single-step problems because in these problems it sometimes overestimates the system prediction for the not-best actions. This happens because it is easy for XCSAM to find a pure best action map when there are only two possible rewards. But sometimes mutation changes the action of a classifier from best to not-best action and the offspring temporarily inherits the high prediction of the parent. This has been observed in single-step problems but not in multi-step problems. We hypothesis this problem does not occur in multi-step problems because XCSAM does not find a pure best action map [25] but finds an adaptive action map where some not-best actions remain to keep generalized classifiers; and the not-best actions in it prevent occasional mutations from changing the max system prediction.

References

  1. Bernadó-mansilla E, Garrell-Guiu JM (2003) Accuracy-based learning classifier systems: models, analysis and applications to classification tasks. Evol Comput 11(3):209–238

    Article  Google Scholar 

  2. Bull L, Bernadó-Mansilla E, Holmes JH (eds) (2008) Learning classifier systems in data mining, volume 125 of Studies in computational intelligence. Springer, Berlin

    Google Scholar 

  3. Bull L, Lanzi PL (2009) Introduction to the special issue on learning classifier systems. Nat Comput 8(1):1–2

    Article  MathSciNet  Google Scholar 

  4. Butz MV, Kovacs T, Lanzi PL, Wilson SW (2004) Toward a theory of generalization and learning in XCS. IEEE Trans Evol Comput 8(1):28–46

    Article  Google Scholar 

  5. Butz MV (2006) Rule-based evolutionary online learning systems. Springer, Berlin

    MATH  Google Scholar 

  6. Butz MV, Goldberg DE, Lanzi PL (2005) Gradient descent methods in learning classifier systems: improving XCS performance in multistep problems. IEEE Trans Evol Comput 9(5):452–473

    Article  Google Scholar 

  7. Butz MV, Goldberg DE, Tharakunnel K (2003) Analysis and improvement of fitness exploitation in XCS: bounding models, tournament selection, and bilateral accuracy. Evol Comput 11(3):239–277

    Article  Google Scholar 

  8. Butz MV, Lanzi PL, Wilson SW (2008) Function approximation with XCS: hyperellipsoidal conditions, recursive least squares, and compaction. IEEE Trans Evol Comput 12(3):355–376

    Article  Google Scholar 

  9. Butz MV, Wilson SW (2002) An algorithmic description of XCS. Soft Comput 6(3–4):144–153

    Article  MATH  Google Scholar 

  10. Dixon PW, Corne DW, Oates MJ (2004) Encouraging compact rulesets from XCS for enhanced data mining. In: Bull L (ed) Applications of learning classifier systems, studies in fuzziness and soft computing, vol 150. Springer, Berlin, pp 92–109

    Chapter  Google Scholar 

  11. Dixon PW, Corne D, Oates MJ (2002) A preliminary investigation of modified XCS as a generic data mining tool. In:  Lanzi PL, Stolzmann W, Wilson SW (eds) Revised papers from the 4th international workshop on advances in learning classifier systems, IWLCS ’01, Springer, London, UK

    Google Scholar 

  12. Dixon PW, Corne DW, Oates MJ (2003) A ruleset reduction algorithm for the XCS learning classifier system. In: Learning classifier systems. Springer, Berlin, pp 20–29

  13. Ebadi T, Zhang M, Browne W (2012) XCS-based versus UCS-based feature pattern classification system. In: Proceedings of the fourteenth international conference on genetic and evolutionary computation conference, ACM, pp 839–846

  14. Goldberg DE (1989) Genetic algorithms in search, pptimization, and machine learning. Addison Wesley, Reading

    Google Scholar 

  15. Katagami D, Yamada S (2000) Interactive classifier system for real robot learning. In: Proceedings of 9th IEEE international workshop on robot and human interactive communication, 2000. RO-MAN 2000. IEEE, pp 258–263

  16. Kharbat F, Bull L, Odeh M (2007) Mining breast cancer data with XCS. In: Proceedings of the genetic and evolutionary computation conference (GECCO2007), pp 2066–2073

  17. Kovacs T (1996) Evolving optimal populations with XCS classifier systems. Technical report CSR-96-17 and CSRP-96-17, School of Computer Science, University of Birmingham, Birmingham, UK

  18. Kovacs T (2000) Strength or accuracy? Fitness calculation in learning classifier systems. In: Lanzi PL, Stolzmann W, Wilson SW (eds) Learning classifier systems, volume 1813 of Lecture Notes in Computer Science. Springer, Berlin, pp 143–160

    Google Scholar 

  19. Lanzi PL (1999) An analysis of generalization in the XCS classifier system. Evol Comput 7(2):125–149

    Article  Google Scholar 

  20. Loiacono D, Orriols-Puig A, Urbanowicz R (2012) Special issue on advances in learning classifier systems. Evol Intel 5(2):57–58

    Article  Google Scholar 

  21. Nakata M, Kovacs T, Takadama K (2014) A modified XCS classifier system for sequence labeling. In: Proceedings of the 2014 conference on genetic and evolutionary computation, GECCO 2014. ACM, pp 565–572

  22. Nakata M, Lanzi PL, Takadama K (2012) Enhancing learning capabilities by XCS with best action mapping. Parallel problem solving from nature (PPSN XII), pp 256–265

  23. Nakata M, Lanzi PL, Takadama K (2012) XCS with adaptive action mapping. The ninth international conference on simulated evolution and learning (SEAL 2012), pp 256–265

  24. Nakata M, Lanzi PL, Takadama K (2013) Selection strategy for XCS with adaptive action mapping. In: Proceedings of the 15th annual conference on genetic and evolutionary computation, GECCO ’13, New York, NY, USA. ACM, pp 1085–1092

  25. Nakata M, Lanzi PL, Takadama K (2014) Evolving best action maps in accuracy-based reinforcement learning classifier system. Evol Comput (submitted)

  26. Orriols-Puig A, Bernadó-Mansilla E (2005) Class imbalance problem in UCS classifier system: fitness adaptation. In: CEC2005, vol 1, pp 604–611. IEEE

  27. Orriols-Puig A, Bernadó-Mansilla E (2009) Evolutionary rule-based systems for imbalanced data sets. Soft Comput 13(3):213–225

    Article  Google Scholar 

  28. Shafi K, Abbass HA, Zhu Weiping (2007) Real time signature extraction from a supervised classifier system. In: IEEE congress on evolutionary Computation. CEC 2007, pp 2509–2516

  29. Shafi K, Urbanowicz R, Iqbal M (2013) Special issue on advances in learning classifier systems. Evol Intel 6(2):55–56

    Article  Google Scholar 

  30. Sutton RS, Barto AG (1998) Reinforcement learning—an introduction. MIT Press, Cambridge

    Google Scholar 

  31. Wilson SW (1995) Classifier fitness based on accuracy. Evol Comput 3(2):149–175

    Article  Google Scholar 

  32. Wilson SW (2002) Compact rulesets from XCSI. In:  Lanzi PL, Stolzmann W, Wilson SW (eds) Advances in learning classifier systems. Springer, Berlin, pp 197–208

    Chapter  Google Scholar 

  33. Zang Z, Li D, Wang J (2013) Knowledge extraction and rule set compaction in XCS for non-Markov multi-step problems. Evol Intel 6(1):41–53

    Article  Google Scholar 

Download references

Acknowledgments

This work was supported by the JSPS Institutional Program.

Author information

Authors and Affiliations

  1. Graduate School of Informatics, The University of Electro-Communications, Chofu, 182-0021, Japan

    Masaya Nakata & Keiki Takadama

  2. Japan Society for the Promotion of Science, Chiyoda-ku, 102-0083, Japan

    Masaya Nakata

  3. Department of Electronics and Information, Politecnico di Milano, 20133, Milan, Italy

    Pier Luca Lanzi

Authors
  1. Masaya Nakata
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Pier Luca Lanzi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Keiki Takadama
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Masaya Nakata.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nakata, M., Lanzi, P.L. & Takadama, K. Rule reduction by selection strategy in XCS with adaptive action map. Evol. Intel. 8, 71–87 (2015). https://doi.org/10.1007/s12065-015-0130-1

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12065-015-0130-1

Keywords

Search

Navigation