Abstract
On-line learning mechanisms are frequently employed to implement self-adaptivity in modern systems. With more widespread use in technical systems that interact with their physical environment, e.g. cyber-physical systems, the fulfillment of safety requirements is increasingly gaining attention. We focus on the learning classifier system XCS with its human-interpretable rules and propose an approach to integrate safety guarantees into its rule base. We leverage the interpretability of XCS’ rules to internalize the safety-critical knowledge, as opposed to related work, which relies on an external safety monitor. The experimental evaluation shows that such manually injected knowledge not only gives safety guarantees but aids the learning mechanism of XCS. Especially in complex environments where XCS is struggling to find the optimal solution, the use of hand-crafted forbidden classifiers leads to a performance that is up to 41.7 % better than with an external safety monitor.
This work was partially supported by the German Research Foundation (DFG) within the Collaborative Research Centre On-The-Fly Computing (GZ: SFB 901/3) under the project number 160364472.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
That is \({\beta = 0.2}\), \({\alpha = 0.1}\), \({\nu =5}\), \({\mu =0.04}\), \({\delta =0.1}\), \({p_I=10}\), \({\epsilon _I=0}\), \({f_I=0.01}\), \(P_{\#}=0.5\), \({\epsilon _0=10}\), \({\chi =0.8}\), \({\theta _{GA}=25}\), \({\theta _{sub}=20}\), \({\theta _{del}=20}\), \({\gamma =0.71}\), \({\textit{DoGaSubsumption}}={} \textit{True}\), \(\textit{DoActionSetSubsumption}=\textit{False}\).
References
Alshiekh, M., Bloem, R., Ehlers, R., Könighofer, B., Niekum, S., Topcu, U.: Safe reinforcement learning via shielding. In: Proceedings of the AAAI Conference on Artificial Intelligence, vol. 32 (2018)
Amodei, D., Olah, C., Steinhardt, J., Christiano, P.F., Schulman, J., Mané, D.: Concrete problems in AI safety. CoRR abs/1606.06565 (2016)
Bellman, K., et al.: Self-aware cyber-physical systems. ACM Trans. Cyber-Phys. Syst. 4(4), 1–26 (2020)
Butz, M.V., Wilson, S.W.: An algorithmic description of XCS. Soft Comput. 6(3–4), 144–153 (2002)
García, J., Fernández, F.: A comprehensive survey on safe reinforcement learning. J. Mach. Learn. Res. 16(42), 1437–1480 (2015)
Iqbal, M., Browne, W.N., Zhang, M.: Reusing building blocks of extracted knowledge to solve complex, large-scale boolean problems. IEEE Trans. Evol. Comput. 18(4), 465–480 (2014)
Lanzi, P.L.: A Study of the Generalization Capabilities of XCS. In: Bäck, T. (ed.) Proceedings of the 7th International Conference on Genetic Algorithms, East Lansing, MI, USA, July 19–23, 1997, pp. 418–425. Morgan Kaufmann (1997)
Lanzi, P.L.: An analysis of generalization in the XCS classifier system. Evol. Comput. 7(2), 125–149 (1999)
Lewis, P.R., Platzner, M., Rinner, B., Tørresen, J., Yao, X. (eds.): Self-aware Computing Systems. Springer International Publishing (2016)
Müller-Schloer, C., Schmeck, H., Ungerer, T. (eds.): Organic Computing — A Paradigm Shift for Complex Systems. Springer Basel (2011)
Prothmann, H., Tomforde, S., Branke, J., Hähner, J., Müller-Schloer, C., Schmeck, H.: Organic traffic control. In: Organic Computing - A Paradigm Shift for Complex Systems, pp. 431–446. Springer Basel (2011)
Stein, A., Tomforde, S.: Reflective learning classifier systems for self-adaptive and self-organising agents. In: 2021 IEEE International Conference on Autonomic Computing and Self-Organizing Systems Companion (ACSOS-C), pp. 139–145 (2021)
Stone, C., Bull, L.: For real! xcs with continuous-valued inputs. Evol. Comput. 11(3), 299–336 (2003)
Tomforde, S., Brameshuber, A., Hahner, J., Müller-Schloer, C.: Restricted on-line learning in real-world systems. In: 2011 IEEE Congress of Evolutionary Computation, CEC 2011, pp. 1628–1635 (2011)
Wilson, S.W.: Classifier fitness based on accuracy. Evol. Comput. 3(2), 149–175 (1995)
Wilson, S.W.: Generalization in the XCS classifier system. In: Koza, J., et al. (ed.) Genetic Programming 1998: Proceedings of the Third Annual Conference, pp. 665–674. Morgan Kaufmann, San Franciso (1998)
Zeppenfeld, J., Herkersdorf, A.: Applying autonomic principles for workload management in multi-core systems on chip. In: Proceedings of the 8th International Conference on Autonomic Computing, pp. 3–10. ACM, New York (2011)
Zhang, R.F., Urbanowicz, R.J.: A scikit-learn compatible learning classifier system. In: Proceedings of the 2020 Genetic and Evolutionary Computation Conference Companion. ACM, New York (2020)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Hansmeier, T., Platzner, M. (2022). Integrating Safety Guarantees into the Learning Classifier System XCS. In: Jiménez Laredo, J.L., Hidalgo, J.I., Babaagba, K.O. (eds) Applications of Evolutionary Computation. EvoApplications 2022. Lecture Notes in Computer Science, vol 13224. Springer, Cham. https://doi.org/10.1007/978-3-031-02462-7_25
Download citation
DOI: https://doi.org/10.1007/978-3-031-02462-7_25
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-02461-0
Online ISBN: 978-3-031-02462-7
eBook Packages: Computer ScienceComputer Science (R0)