Skip to main content
SpringerLink
Log in

Rapid Development of OS Support with PMCSched for Scheduling on Asymmetric Multicore Systems

  • Conference paper
  • First Online:
Euro-Par 2022: Parallel Processing Workshops (Euro-Par 2022)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 13835))

Included in the following conference series:

Abstract

Asymmetric multicore processors (AMPs) couple high-performance big cores and power-efficient small ones, all exposing a shared instruction set architecture to software, but with different microarchitectural features. The energy efficiency benefits of AMPs together with the general-purpose nature of the various cores, have led hardware manufactures to build commercial AMP-based products, first for the mobile and embedded domains, and more recently for the desktop market segment, as with the Intel Alder Lake processor family. This indicates that AMPs may become a solid and more energy efficient replacement to symmetric multicores in a wide range of application domains.

Previous research has demonstrated that the system software can substantially improve scheduling –critical to get the most out of heterogeneous cores– by leveraging hardware facilities that are directly managed by the OS, such as performance monitoring counters, or the recently introduced Intel Thread Director technology. Unfortunately, the OS-level support enabling to access scheduling-relevant hardware support may take a long time to be adopted in operating systems, or may come in forms that make its utilization challenging from specific levels of the system software stack, especially in production systems. To fill this gap, we propose the PMCSched framework, which enables the creation of custom OS support on Linux to aid in the design of novel scheduling and resource-management policies for multicores implemented at different layers of the system software, but without requiring to patch the kernel. To demonstrate the potential of our framework, we implement a set of OS-level schedulers for AMPs, that make use of custom OS extensions to access scheduling-relevant hardware facilities in an x86 AMP processor.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 54.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 69.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. AMD: AMD64 Technology Platform QoS Extensions. https://developer.amd.com/wp-content/resources/56375.pdf

  2. Barbalace, A., Lyerly, R., Jelesnianski, C., Carno, A., Chuang, H.R., Legout, V., Ravindran, B.: Breaking the boundaries in heterogeneous-ISA datacenters. In: ACM SIGPLAN Notices, vol. 52, pp. 645–659. ACM (2017)

    Google Scholar 

  3. Blagodurov, S., et al.: A case for NUMA-aware contention management on multicore systems. In: Proceedings of USENIX ATC 2011. USA (2011)

    Google Scholar 

  4. Calandrino, J.M., et al.: LITMUS-RT : a Testbed for empirically comparing real-time multiprocessor schedulers. In: 2006 27th IEEE International Real-Time Systems Symposium (RTSS’06), pp. 111–126 (2006)

    Google Scholar 

  5. Chronaki, K., et al.: Criticality-aware dynamic task scheduling for heterogeneous architectures. In: Proceedings of the 29th ACM on International Conference on Supercomputing, pp. 329–338. ICS 2015 (2015)

    Google Scholar 

  6. Chronaki, K., et al.: On the maturity of parallel applications for asymmetric multi-core processors. J. Par. Distrib. Comput. 127, 105–115 (2019)

    Article  Google Scholar 

  7. Costero, L., et al.: Energy efficiency optimization of task-parallel codes on asymmetric architectures. In: Proceedings of HPCS 2017, pp. 402–409 (July 2017)

    Google Scholar 

  8. Dongarra, J.: Report on the sunway taihulight system. Tech Report University of Tennessee: UT-EECS-16-742 (2016)

    Google Scholar 

  9. Feliu, J., et al.: Perf &fair: a progress-aware scheduler to enhance performance and fairness in SMT multicores. IEEE Trans. Comput. 66(5), 905–911 (2017)

    Article  MathSciNet  MATH  Google Scholar 

  10. Garcia-Garcia, A., et al.: Contention-aware fair scheduling for asymmetric single-ISA multicore systems. IEEE Trans. Comput. 67(12) (2018)

    Google Scholar 

  11. Garcia-Garcia, A., et al.: LFOC: A lightweight fairness-oriented cache clustering policy for commodity multicores. In: Proceedings of ICPP 2019, pp. 14:1–14:10 (2019)

    Google Scholar 

  12. Haque, M.E., et al.: Exploiting heterogeneity for tail latency and energy efficiency. In: 50th Annual IEEE/ACM International Symposium on Microarchitecture, pp. 625–638 (2017)

    Google Scholar 

  13. Harris, T., Maas, M., Marathe, V.J.: Callisto: Co-scheduling parallel runtime systems. In: Proceedings of 9th European Conference on Computing Systems, EuroSys 2014 (2014)

    Google Scholar 

  14. Hennessy, J.L., Patterson, D.A.: A new golden age for computer architecture. Commun. ACM 62(2), 48–60 (2019)

    Article  Google Scholar 

  15. Intel: Intel® 64 and IA-32 Architectures Software Developer’s Manual, Vol. 3: System Programming Guide (2021)

    Google Scholar 

  16. Intel: Optimizing software for x86 hybrid archiecture. Intel White Paper (Oct 2021)

    Google Scholar 

  17. Intel: User space software for Intel(R) Resource Director Technology. https://github.com/intel/intel-cmt-cat (2022)

  18. Koufaty, D., Reddy, D., Hahn, S.: Bias Scheduling in Heterogeneous Multi-core Architectures. In: Eurosys 10, pp. 125–138 (2010)

    Google Scholar 

  19. Kumar, R., et al.: Single-ISA Heterogeneous Multi-Core Architectures for Multithreaded Workload Performance. In: 31st Annual International Symposium on Computer Architecture (ISCA 2004), pp. 64–75 (2004)

    Google Scholar 

  20. Lepers, B., et al.: Provable multicore schedulers with Ipanema: Application to work conservation. In: Proceedings of Eurosys 2020 (2020)

    Google Scholar 

  21. Li, T., et al.: Operating system support for overlapping-ISA heterogeneous multi-core architectures. In: Proceedings of HPCA 2010, pp. 1–12 (2010)

    Google Scholar 

  22. Linux: Using the linux kernel tracepoints. https://www.kernel.org/doc/html/latest/trace/tracepoints.html

  23. Lozi, J.P., et al.: The linux scheduler: A decade of wasted cores. In: Proceedings of the 11th ACM European Conference on Computer Systems (Eurosys 2016) (2016)

    Google Scholar 

  24. Lyerly, R., et al.: An OpenMP Runtime for Transparent Work Sharing Across Cache-Incoherent Heterogeneous Nodes. ACM Trans. Comput. Syst. (dec 2021)

    Google Scholar 

  25. Lyerly, R., et al.: An Openmp runtime for transparent work sharing across cache-incoherent heterogeneous nodes. ACM Trans. Comput. Syst. (2021)

    Google Scholar 

  26. Mvondo, D., et al.: Towards user-programmable schedulers in the operating system kernel. In: Proceedings of the 11th Workshop on Systems for Post-Moore Architectures, SPMA 2022 (2022)

    Google Scholar 

  27. PMCTrack: Github repository. https://github.com/jcsaezal/pmctrack (2015)

  28. Pricopi, M., et al.: Power-performance modeling on asymmetric multi-cores. In: Proceedings of CASES 2013., pp. 15:1–15:10 (2013)

    Google Scholar 

  29. Rostedt, S.: "ftrace: Where modifying a running kernel all started". https://kernel-recipes.org/en/2019/talks/ftrace-where-modifying-a-running-kernel-all-started/

  30. Saez, J.C., Castro, F., Prieto-Matias, M.: Enabling performance portability of data-parallel openmp applications on asymmetric multicore processors. In: 49th International Conference on Parallel Processing. ICPP 2020 (2020)

    Google Scholar 

  31. Saez, J.C., Prieto-Matias, M.: Evaluation of the Intel Thread Director technology on an Alder Lake processor. In: 13th ACM SIGOPS Asia-Pacific Workshop on Systems (APSys 2022) (2022)

    Google Scholar 

  32. Saez, S.C., et al.: An OS-oriented performance monitoring tool for multicore systems. In: Hunold, S., et al. (eds.) Euro-Par 2015. LNCS, vol. 9523, pp. 697–709. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-27308-2_56

    Chapter  Google Scholar 

  33. Saez, J.C., et al.: PMCTrack: delivering performance monitoring counter support to the OS scheduler. Comput. J. 60(1), 60–85 (2017)

    Article  MathSciNet  Google Scholar 

  34. Saez, J.C., et al.: Towards completely fair scheduling on asymmetric single-ISA multicore processors. J. Parallel Distrib. Comput. 102 (2017)

    Google Scholar 

  35. Salami, B., et al.: Online energy-efficient fair scheduling for heterogeneous multi-cores considering shared resource contention. J. Supercomput. 78(6) (2022)

    Google Scholar 

  36. Servat, H., et al.: On the instrumentation of OpenMP and OmpSs tasking constructs. In: Euro-Par 2012: Parallel Processing Workshops, pp. 414–428 (2013)

    Google Scholar 

  37. Torng, C., Wang, M., Batten, C.: Asymmetry-aware work-stealing runtimes. In: Proceedings of ISCA 2016, pp. 40–52 (2016)

    Google Scholar 

  38. Weaver, V.M.: Linux perf event features and overhead. FastPath Workshop (2013)

    Google Scholar 

  39. Xu, V.M., et al.: Lush: Lightweight framework for user-level scheduling in heterogeneous multicores. In: 2021 IEEE 14th International Symposium on Embedded Multicore/Many-core Systems-on-Chip (MCSoC), pp. 396–404 (2021)

    Google Scholar 

Download references

Acknowledgements

Work supported by the EU (FEDER), the Spanish MINECO and CM, under grants RTI2018-093684-B-I00 and S2018/TCS-4423.

Author information

Authors and Affiliations

  1. Facultad de Informática, Universidad Complutense de Madrid, Madrid, Spain

    Carlos Bilbao, Juan Carlos Saez & Manuel Prieto-Matias

Authors
  1. Carlos Bilbao
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Juan Carlos Saez
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Manuel Prieto-Matias
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Carlos Bilbao .

Editor information

Editors and Affiliations

  1. University of Glasgow, Glasgow, UK

    Jeremy Singer

  2. University of Glasgow, Glasgow, UK

    Yehia Elkhatib

  3. University of Santiago de Compostela, Santiago de Compostela, La Coruña, Spain

    Dora Blanco Heras

  4. Louisiana State University, Baton Rouge, LA, USA

    Patrick Diehl

  5. University of Edinburgh, Edinburgh, UK

    Nick Brown

  6. Universidade de Lisboa, Lisbon, Portugal

    Aleksandar Ilic

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Bilbao, C., Saez, J.C., Prieto-Matias, M. (2023). Rapid Development of OS Support with PMCSched for Scheduling on Asymmetric Multicore Systems. In: Singer, J., Elkhatib, Y., Blanco Heras, D., Diehl, P., Brown, N., Ilic, A. (eds) Euro-Par 2022: Parallel Processing Workshops. Euro-Par 2022. Lecture Notes in Computer Science, vol 13835. Springer, Cham. https://doi.org/10.1007/978-3-031-31209-0_14

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-31209-0_14

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-31208-3

  • Online ISBN: 978-3-031-31209-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation