Abstract
Learning Classifier Systems (LCSs) are a group of rule-based evolutionary computation techniques, which have been frequently applied to data mining tasks. The LCSs’ rules are designed to be human-readable to enable the underlying knowledge to be investigated. However, the models for the majority of domains with high feature interaction contain a large number of rules that cooperatively represent the knowledge. However, the interaction between many rules is too complex to be comprehended by humans. Thus, it is hypothesized that translating the models’ underlying patterns into human-discernable visualizations will advance the understanding of the learned patterns and LCSs themselves. Interrogatable artificial Boolean domains with varying numbers of attributes are considered as benchmarks. Three new visualization techniques, termed as Feature Importance Map, Action-based Feature Importance Map, and Action-based Feature’s Average value Map, successfully produce interpretable results for all the complex domains tested. This includes both tracing the training progress and analyzing the trained models from LCSs. The visualization techniques’ ability to handle complex optimal solutions is observed for the 14-bits Majority-On problem, where the patterns from 6435 different cooperating rules were translated into human-discernable graphs.
Similar content being viewed by others
References
Aenugu S, Spector L (2019) Lexicase selection in learning classifier systems. In: Genetic and evolutionary computation conference, pp 356–364
Arulkumaran K, Cully A, Togelius J (2019) Alphastar: an evolutionary computation perspective. In: Proceedings of the genetic and evolutionary computation conference companion, pp 314–315
Bull L (2004) Applications of learning classifier systems. Springer, Berlin, pp 1–12
Butz MV, Goldberg DE, Lanzi PL (2005) Computational complexity of the XCS classifier system. In: Foundations of learning classifier systems, pp 91–125
Butz MV, Goldberg DE, Lanzi PL (2005) Gradient descent methods in learning classifier systems: improving XCS performance in multistep problems, pp 452–473
Butz MV, Lanzi PL (2009) Sequential problems that test generalization in learning classifier systems. Evol Intell 2:141–148
Butz MV, Lanzi PL, Wilson SW (2008) Function approximation with XCS: hyperellipsoidal conditions, recursive least squares, and compaction. IEEE Trans Evol Comput 12:355–376
Dixon PW, Corne DW, Oates MJ (2002) A ruleset reduction algorithm for the XCS learning classifier system. In: International workshop on learning classifier systems, pp 20–29
Fu C, Davis L (2002) A modified classifier system compaction algorithm. In: 4th Annual conference on genetic and evolutionary computation, pp 920–925
Iqbal M, Browne WN, Zhang M (2013) Evolving optimum populations with XCS classifier systems. Soft Comput 17:503–518
Iqbal M, Browne WN, Zhang M (2013) Learning complex, overlapping and niche imbalance Boolean problems using XCS-based classifier systems, evolutionary. Intelligence 6:73–91
Karlsen MR, Moschoyiannis S (2018) Evolution of control with learning classifier systems. Appl Netw Sci 3:30–66
Kukenys I, Browne WN, Zhang M (2011) Transparent, online image pattern classification using a learning classifier system. In: European conference on the applications of evolutionary computation, pp 183–193
Lanzi PL (2000) Learning classifier systems: from foundations to applications. Springer, Berlin
Lanzi PL (2002) Learning classifier systems from a reinforcement learning perspective. Soft Comput 6:162–170
Lanzi PL (2008) Learning classifier systems: then and now. Evol Intell 1:63–82
Liu Y, Browne WN, Xue B (2018) Adapting bagging and boosting to learning classifier systems. In: International conference on the applications of evolutionary computation, pp 405–420
Liu Y, Browne WN, Xue B (2019) Absumption to complement subsumption in learning classifier systems. In: Proceedings of the genetic and evolutionary computation conference, pp 410–418
Liu Y, Browne WN, Xue B (2020a) Absumption and subsumption based learning classifier systems. In: Under processing of genetic and evolutionary computation conference
Liu Y, Browne WN, Xue B (2020b) Rule compaction algorithms for learning classifier systems. In: Transactions on evolutionary learning and optimization
Muhammad I, Browne WN, Zhang M (2017) Extending XCS with cyclic graphs for scalability on complex boolean problems. Evol Comput 25:173–204
Nakata M, Kovacs T, Takadama K (2014) A modified XCS classifier system for sequence labeling. In: 2014 Annual conference on genetic and evolutionary computation, vol 14, pp 565–572
Nakata M, Kovacs T, Takadama K (2015) XCS-SL: a rule-based genetic learning system for sequence labeling. Evol Intell 8:133–148
Naqvi SS, Browne WN (2016) Adapting learning classifier systems to symbolic regression. In: IEEE congress on evolutionary computation (CEC) 2016, pp 2209–2216
Stein A, Maier R, Rosenbauer L, Hahner J (2020) XCS classifier system with experience replay, arXiv preprint arXiv:2002.05628
Sutton CD (2005) Classification and regression trees, bagging, and boosting. Handb Stat 24:303–329
Takadama K, Yamazaki D, Nakata M, Sato H (2019) Complex-valued-based learning classifier system for POMDP environments, pp 1852–1859
Tan J, Moore J, Urbanowicz R (2013) Rapid rule compaction strategies for global knowledge discovery in a supervised learning classifier system. Artif Life Conf Proc 13:110–117
Urbanowicz RJ, Browne WN (2017) Introduction to learning classifier systems. Springer, Berlin
Urbanowicz RJ, Moore JH (2010) The application of michigan-style learning classifiersystems to address genetic heterogeneity and epistasisin association studies. In: Proccedings of the 12th annual conference on genetic and evolutionary computation, vol 12, pp 195–202
Urbanowicz RJ, Moore JH (2015) ExSTraCS 2.0: description and evaluation of a scalable learning classifier system. Evol Intell 8:89–116
Urbanowicz RJ, Granizo-Mackenzie A, Moore JH (2012) An analysis pipeline with statistical and visualization-guided knowledge discovery for michigan-style learning classifier systems
Urbanowicz RJ, Granizo-Mackenzie A, Moore JH (2012) Instance-linked attribute tracking and feedback for michigan-style supervised learning classifier systems. In: Proceedings of the 14th annual conference on genetic and evolutionary computation, vol. 14, pp 927–934
Urbanowicz RJ, Andrew AS, Karagas MR, Moore JH (2013) Role of genetic heterogeneity and epistasis in bladder cancer susceptibility and outcome: a learning classifier system approach. J Am Med Inform Assoc 20:603–612
Wilson SW (1999) State of XCS classifier system research. In: International workshop on learning classifier systems, pp 63–81
Wyatt D, Bull L, Parmee IC (2004) Building compact rulesets for describing continuous-valued problem spaces using a learning classifier system. In: Adaptive computing in design and manufacture VI, pp 235–246
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Liu, Y., Browne, W.N. & Xue, B. Visualizations for rule-based machine learning. Nat Comput 21, 243–264 (2022). https://doi.org/10.1007/s11047-020-09840-0
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11047-020-09840-0