Abstract
Learning classifier tables (LCTs) are classifier based and lightweight hardware reinforcement learning building blocks which inherit the concepts of learning classifier systems. LCTs are used as a per-core low level controllers to learn and optimize potentially conflicting objectives e.g. achieving a performance target under a power budget. A supervisor is used at the system level which translate system and application requirements into objectives for the LCTs. The classifier population in the LCTs has to be evolved in run-time to adapt to the changes in the mode, performance targets, constraints or workload being executed. Towards this goal, we present GAE-LCT, a genetic algorithm (GA) based classifier evolution for hardware learning classifier tables. The GA uses accuracy to evolve classifiers in run-time. We introduce extensions to the LCT to enable accuracy based genetic algorithm. The GA runs as a software process on one of the cores and interacts with the hardware LCT via interrupts. We evaluate our work using DVFS on an FPGA using Leon3 cores. We demonstrate GAE-LCT’s ability to generate accurate classifiers in run-time from scratch. GAE-LCT achieves 5% lower difference to IPS reference and 51.5% lower power budget overshoot compared to Q-table while requiring 75% less memory. The hybrid GAE-LCT also requires 12 times less software overhead compared to a full software implementation.
We thank our project partners in the IPF project and acknowledge the financial support from the DFG under Grant HE4584/7-1.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bernauer, A., Zeppenfeld, J., Bringmann, O., Herkersdorf, A., Rosenstiel, W.: Combining software and hardware LCS for lightweight on-chip learning. In: Hinchey, M., et al. (eds.) BICC/DIPES -2010. IAICT, vol. 329, pp. 278–289. Springer, Heidelberg (2010). https://doi.org/10.1007/978-3-642-15234-4_27
Bitirgen, R., Ipek, E., Martinez, J.F.: Coordinated management of multiple interacting resources in chip multiprocessors: a machine learning approach. In: 2008 41st IEEE/ACM International Symposium on Microarchitecture (2008)
Bolchini, C., Ferrandi, P., Lanzi, P.L., Salice, F.: Evolving classifiers on field programmable gate arrays: migrating XCS to FPGAs. J. Syst. Archit. 52, 516–533 (2006)
Bouajila, A., et al.: Autonomic system on chip platform. In: Müller-Schloer, C., Schmeck, H., Ungerer, T. (eds.) Organic Computing–A Paradigm Shift for Complex System, vol. 1, pp. 413–425. Springer, Basel (2011). https://doi.org/10.1007/978-3-0348-0130-0_27
Butz, M.V., Kovacs, T., Lanzi, P.L., Wilson, S.W.: How XCS evolves accurate classifiers. In: Proceedings of the Third Genetic and Evolutionary Computation Conference (GECCO-2001) (2001)
Chen, Z., Marculescu, D.: Distributed reinforcement learning for power limited many-core system performance optimization. In: 2015 Design, Automation & Test in Europe Conference & Exhibition (DATE) (2015)
Dhiman, G., Rosing, T.S.: Dynamic power management using machine learning. In: Proceedings of the 2006 IEEE/ACM International Conference on Computer-Aided Design (2006)
Donyanavard, B., et al.: SOSA: self-optimizing learning with self-adaptive control for hierarchical system-on-chip management. In: Proceedings of the 52nd Annual IEEE/ACM International Symposium on Microarchitecture (2019)
Gaisler, J., Catovic, E., Isomaki, M., Glembo, K., Habinc, S.: Grlib IP core user’s manual. Gaisler research (2007)
Gupta, U., Mandal, S.K., Mao, M., Chakrabarti, C., Ogras, U.Y.: A deep Q-learning approach for dynamic management of heterogeneous processors. IEEE Comput. Archit. Lett. 18, 14–17 (2019)
Guthaus, M.R., Ringenberg, J.S., Ernst, D., Austin, T.M., Mudge, T., Brown, R.B.: MiBench: a free, commercially representative embedded benchmark suite. In: Proceedings of the Fourth Annual IEEE International Workshop on Workload Characterization (2001)
Hansmeier, T.: Self-aware operation of heterogeneous compute nodes using the learning classifier system XCS. In: Proceedings of the 11th International Symposium on Highly Efficient Accelerators and Reconfigurable Technologies (2021)
Hansmeier, T., Platzner, M.: An experimental comparison of explore/exploit strategies for the learning classifier system XCS. In: Proceedings of the Genetic and Evolutionary Computation Conference Companion (2021)
Kovacs, T.: Strength or accuracy? Fitness calculation in learning classifier systems. In: Lanzi, P.L., Stolzmann, W., Wilson, S.W. (eds.) IWLCS 1999. LNCS (LNAI), vol. 1813, pp. 143–160. Springer, Heidelberg (2000). https://doi.org/10.1007/3-540-45027-0_7
Liu, W., Tan, Y., Qiu, Q.: Enhanced Q-learning algorithm for dynamic power management with performance constraint. In: 2010 Design, Automation & Test in Europe Conference & Exhibition (DATE) (2010)
Maurer, F., Donyanavard, B., Rahmani, A.M., Dutt, N., Herkersdorf, A.: Emergent control of MPSOC operation by a hierarchical supervisor/reinforcement learning approach. In: 2020 Design, Automation & Test in Europe Conference & Exhibition (DATE) (2020)
Pagani, S., Manoj, P.S., Jantsch, A., Henkel, J.: Machine learning for power, energy, and thermal management on multicore processors: a survey. IEEE Trans. Comput.-Aided Design Integr. Circ. Syst. 39, 101–116 (2018)
Rajat, R., Meng, Y., Kuppannagari, S., Srivastava, A., Prasanna, V., Kannan, R.: QTAccel: a generic FPGA based design for q-table based reinforcement learning accelerators. In: Proceedings of the 2020 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays (2020)
Shen, H., Lu, J., Qiu, Q.: Learning based DVFS for simultaneous temperature, performance and energy management. In: Thirteenth International Symposium on Quality Electronic Design (ISQED) (2012)
Wang, Y.C., Usher, J.M.: Application of reinforcement learning for agent-based production scheduling. Eng. Appl. Artif. Intell. 18, 73–82 (2005)
Zeppenfeld, J., et al.: Applying ASOC to multi-core applications for workload management. In: Müller-Schloer, C., Schmeck, H., Ungerer, T. (eds.) Organic Computing–A Paradigm Shift for Complex Systems. Autonomic Systems, vol. 1, pp. 461–472. Springer, Basel (2011). https://doi.org/10.1007/978-3-0348-0130-0_30
Zeppenfeld, J., Bouajila, A., Stechele, W., Herkersdorf, A.: Learning classifier tables for autonomic systems on chip. INFORMATIK 2008. Beherrschbare Systeme-dank Informatik. Band 2 (2008)
Zhang, Q., Lin, M., Yang, L.T., Chen, Z., Li, P.: Energy-efficient scheduling for real-time systems based on deep Q-learning model. IEEE Trans. Sustain. Comput. 4, 132–141 (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Surhonne, A., Doan, N.A.V., Maurer, F., Wild, T., Herkersdorf, A. (2022). GAE-LCT: A Run-Time GA-Based Classifier Evolution Method for Hardware LCT Controlled SoC Performance-Power Optimization. In: Schulz, M., Trinitis, C., Papadopoulou, N., Pionteck, T. (eds) Architecture of Computing Systems. ARCS 2022. Lecture Notes in Computer Science, vol 13642. Springer, Cham. https://doi.org/10.1007/978-3-031-21867-5_18
Download citation
DOI: https://doi.org/10.1007/978-3-031-21867-5_18
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-21866-8
Online ISBN: 978-3-031-21867-5
eBook Packages: Computer ScienceComputer Science (R0)