Skip to main content
SpringerLink
Log in

CoLeCTs: Cooperative Learning Classifier Tables for Resource Management in MPSoCs

  • Conference paper
  • First Online:
Architecture of Computing Systems (ARCS 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13949))

Included in the following conference series:

Abstract

The increasing complexity and unpredictability of emerging applications makes it challenging for multi-processor system-on-chips to satisfy their performance requirements while keeping power consumption within bounds. In order to tackle this problem, the research community has focused on developing dynamic resource managers that aim to optimize runtime parameters, such as clock frequency, voltage and task mapping. There is a large diversity in the approaches proposed in this context, but a class of resource managers that has gained traction recently is that of reinforcement learning-based controllers. In this paper we propose CoLeCTs, a resource manager that enhances the state-of-the-art resource manager SOSA by employing a joint reward assignment function and enabling collaborative information exchange among multiple learning agents. In this manner we tackle the suboptimal determination of local performance targets for heterogeneous applications and allow cooperative decision making for the learning agents. We evaluate and quantify the benefits of our approach via trace-based simulations.

We acknowledge the financial support from the DFG Grant HE4584/7-2.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 54.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 69.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Askarizade Haghighi, M., Maeen, M., Haghparast, M.: An energy-efficient dynamic resource management approach based on clustering and meta-heuristic algorithms in cloud computing IaaS platforms. Wirel. Pers. Commun. 104(4), 1367–1391 (2019)

    Article  Google Scholar 

  2. Beckmann, N., Sanchez, D.: Maximizing cache performance under uncertainty. In: 2017 IEEE International Symposium on High Performance Computer Architecture (HPCA), pp. 109–120. IEEE (2017)

    Google Scholar 

  3. Cobham Gaisler AB: GRLIB IP Library User’s Manual (2022). https://www.gaisler.com/products/grlib/grlib.pdf, version 2022.2

  4. Costero, L., Iranfar, A., Zapater, M., Igual, F.D., Olcoz, K., Atienza, D.: MAMUT: multi-agent reinforcement learning for efficient real-time multi-user video transcoding. In: 2019 Design, Automation & Test in Europe Conference & Exhibition (DATE), pp. 558–563. IEEE (2019)

    Google Scholar 

  5. Costero, L., Iranfar, A., Zapater, M., Igual, F.D., Olcoz, K., Atienza, D.: Resource management for power-constrained HEVC transcoding using reinforcement learning. IEEE Trans. Parallel Distrib. Syst. 31(12), 2834–2850 (2020)

    Article  Google Scholar 

  6. Cucker, F., Smale, S.: Emergent behavior in flocks. IEEE Trans. Autom. Control 52(5), 852–862 (2007)

    Article  MathSciNet  MATH  Google Scholar 

  7. Deng, Q., Meisner, D., Bhattacharjee, A., Wenisch, T.F., Bianchini, R.: CoScale: coordinating CPU and memory system DVFS in server systems. In: 2012 45th Annual IEEE/ACM International Symposium on Microarchitecture, pp. 143–154. IEEE (2012)

    Google Scholar 

  8. Donyanavard, B., et al.: SOSA: self-optimizing learning with self-adaptive control for hierarchical system-on-chip management. In: Proceedings of the 52nd Annual IEEE/ACM International Symposium on Microarchitecture, pp. 685–698 (2019)

    Google Scholar 

  9. Donyanavard, B., Mück, T., Sarma, S., Dutt, N.: SPARTA: runtime task allocation for energy efficient heterogeneous manycores. In: 2016 International Conference on Hardware/Software Codesign and System Synthesis (CODES+ ISSS), pp. 1–10. IEEE (2016)

    Google Scholar 

  10. Dutt, N., Kurdahi, F.J., Ernst, R., Herkersdorf, A.: Conquering MPSoC complexity with principles of a self-aware information processing factory. In: Proceedings of the Eleventh IEEE/ACM/IFIP International Conference on Hardware/Software Codesign and System Synthesis, pp. 1–4 (2016)

    Google Scholar 

  11. Gupta, U., et al.: Adaptive performance prediction for integrated GPUs. In: 2016 IEEE/ACM International Conference on Computer-Aided Design (ICCAD), pp. 1–8. ACM (2016)

    Google Scholar 

  12. Kim, W., Gupta, M.S., Wei, G.Y., Brooks, D.: System level analysis of fast, per-core DVFS using on-chip switching regulators. In: 2008 IEEE 14th International Symposium on High Performance Computer Architecture, pp. 123–134. IEEE (2008)

    Google Scholar 

  13. Kim, Y.G., Wu, C.J.: Autoscale: energy efficiency optimization for stochastic edge inference using reinforcement learning. In: 2020 53rd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO), pp. 1082–1096. IEEE (2020)

    Google Scholar 

  14. LLVM: LLVM “test-suite” repository (2022). https://github.com/llvm/llvm-test-suite/tree/main/SingleSource/Benchmarks/Stanford

  15. Ma, Y., Zhou, J., Chantem, T., Dick, R.P., Wang, S., Hu, X.S.: Online resource management for improving reliability of real-time systems on “Big-Little’’ type MPSoCs. IEEE Trans. Comput.-Aided Des. Integrated Circuits Syst. 39(1), 88–100 (2018)

    Article  Google Scholar 

  16. Maity, B., Donyanavard, B., Dutt, N.: Self-aware memory management for emerging energy-efficient architectures. In: 2020 11th International Green and Sustainable Computing Workshops (IGSC), pp. 1–8. IEEE (2020)

    Google Scholar 

  17. Maurer, F., Donyanavard, B., Rahmani, A.M., Dutt, N., Herkersdorf, A.: Emergent control of MPSoC operation by a hierarchical supervisor/reinforcement learning approach. In: 2020 Design, Automation & Test in Europe Conference & Exhibition (DATE), pp. 1562–1567. IEEE (2020)

    Google Scholar 

  18. del Mestre Martins, A.L., da Silva, A.H.L., Rahmani, A.M., Dutt, N., Moraes, F.G.: Hierarchical adaptive multi-objective resource management for many-core systems. J. Syst. Architect. 97, 416–427 (2019)

    Google Scholar 

  19. Mishra, N., Imes, C., Lafferty, J.D., Hoffmann, H.: CALOREE: learning control for predictable latency and low energy. ACM SIGPLAN Not. 53(2), 184–198 (2018)

    Article  Google Scholar 

  20. Moazzemi, K., Maity, B., Yi, S., Rahmani, A.M., Dutt, N.: HESSLE-FREE: heterogeneous systems leveraging fuzzy control for runtime resource management. ACM Trans. Embed. Comput. Syst. (TECS) 18(5s), 1–19 (2019)

    Article  Google Scholar 

  21. Mück, T., Donyanavard, B., Moazzemi, K., Rahmani, A.M., Jantsch, A., Dutt, N.: Design methodology for responsive and robust MIMO control of heterogeneous multicores. IEEE Trans. Multi-Scale Comput. Syst. 4(4), 944–951 (2018)

    Article  Google Scholar 

  22. Pal, S., Amarnath, A., Feng, S., O’Boyle, M., Dreslinski, R., Dubach, C.: SparseAdapt: runtime control for sparse linear algebra on a reconfigurable accelerator. In: 54th Annual IEEE/ACM International Symposium on Microarchitecture, MICRO-54, pp. 1005–1021 (2021)

    Google Scholar 

  23. Panait, L., Luke, S.: Cooperative multi-agent learning: the state of the art. Auton. Agent Multi-Agent Syst. 11(3), 387–434 (2005)

    Article  Google Scholar 

  24. Pothukuchi, R.P., Ansari, A., Voulgaris, P., Torrellas, J.: Using multiple input, multiple output formal control to maximize resource efficiency in architectures. In: 2016 ACM/IEEE 43rd Annual International Symposium on Computer Architecture (ISCA), pp. 658–670. IEEE (2016)

    Google Scholar 

  25. Pothukuchi, R.P., Pothukuchi, S.Y., Voulgaris, P., Torrellas, J.: Yukta: multilayer resource controllers to maximize efficiency. In: 2018 ACM/IEEE 45th Annual International Symposium on Computer Architecture (ISCA), pp. 505–518. IEEE (2018)

    Google Scholar 

  26. Rahmani, A.M., et al.: SPECTR: formal supervisory control and coordination for many-core systems resource management. In: Proceedings of the Twenty-Third International Conference on Architectural Support for Programming Languages and Operating Systems, pp. 169–183 (2018)

    Google Scholar 

  27. Rahmani, A.M., Haghbayan, M.H., Miele, A., Liljeberg, P., Jantsch, A., Tenhunen, H.: Reliability-aware runtime power management for many-core systems in the dark silicon era. IEEE Trans. Very Large Scale Integration (VLSI) Syst. 25(2), 427–440 (2016)

    Article  Google Scholar 

  28. Ramadge, P.J., Wonham, W.M.: The control of discrete event systems. Proc. IEEE 77(1), 81–98 (1989)

    Article  Google Scholar 

  29. da Rosa, T.R., Larréa, V., Calazans, N., Moraes, F.G.: Power consumption reduction in MPSoCs through DFS. In: 2012 25th Symposium on Integrated Circuits and Systems Design (SBCCI), pp. 1–6. IEEE (2012)

    Google Scholar 

  30. Sadighi, A., et al.: Design methodologies for enabling self-awareness in autonomous systems. In: 2018 Design, Automation & Test in Europe Conference & Exhibition (DATE), pp. 1532–1537. IEEE (2018)

    Google Scholar 

  31. Tan, M.: Multi-agent reinforcement learning: independent vs. cooperative agents. In: Proceedings of the Tenth International Conference on Machine Learning, pp. 330–337 (1993)

    Google Scholar 

  32. Xun, L., Tran-Thanh, L., Al-Hashimi, B.M., Merrett, G.V.: Optimising resource management for embedded machine learning. In: 2020 Design, Automation & Test in Europe Conference & Exhibition (DATE), pp. 1556–1561. IEEE (2020)

    Google Scholar 

  33. Zeppenfeld, J., Herkersdorf, A.: Applying autonomic principles for workload management in multi-core systems on chip. In: Proceedings of the 8th ACM International Conference on Autonomic Computing, pp. 3–10 (2011)

    Google Scholar 

Download references

Acknowledgement

We thank our IPF project partners at TU Braunschweig and UC Irvine, Rolf Ernst, Fadi Kurdahi, Nikil Dutt and their teams, as well as our colleagues at TUM for their valuable feedback and suggestions during our discussions.

Author information

Authors and Affiliations

  1. Chair of Integrated Systems, Technical University of Munich, Munich, Germany

    Klajd Zyla, Florian Maurer, Thomas Wild & Andreas Herkersdorf

Authors
  1. Klajd Zyla
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Florian Maurer
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Thomas Wild
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Andreas Herkersdorf
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Klajd Zyla .

Editor information

Editors and Affiliations

  1. National Technical University of Athens, Athens, Greece

    Georgios Goumas

  2. Kiel University, Kiel, Germany

    Sven Tomforde

  3. Gottfried Wilhelm Leibniz Universität Hannover, Hannover, Germany

    Jürgen Brehm

  4. Friedrich-Alexander-Universität Erlangen-Nürnberg (FAU), Erlangen, Germany

    Stefan Wildermann

  5. Otto-von-Guericke University Magdeburg, Magdeburg, Germany

    Thilo Pionteck

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zyla, K., Maurer, F., Wild, T., Herkersdorf, A. (2023). CoLeCTs: Cooperative Learning Classifier Tables for Resource Management in MPSoCs. In: Goumas, G., Tomforde, S., Brehm, J., Wildermann, S., Pionteck, T. (eds) Architecture of Computing Systems. ARCS 2023. Lecture Notes in Computer Science, vol 13949. Springer, Cham. https://doi.org/10.1007/978-3-031-42785-5_15

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-42785-5_15

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-42784-8

  • Online ISBN: 978-3-031-42785-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation