Abstract
NVM provides large memory capacity, long-term data durability, and high memory bandwidth for multi-thread applications on cloud servers. Nowadays, cloud servers often employ NUMA architecture, where the thread scheduling mechanism plays a vital role in overall system performance because of the NUMA property. However, with the increase in server resources’ diversity, i.e., hybrid memory systems using DRAM and NVM on NUMA nodes, the exploration space for thread scheduling is expanding rapidly. Unfortunately, the existing thread schedulers, including rule-based algorithms and scheduling domain methods, cannot provide ideal scheduling solutions in such complicated cases. And, those thread schedulers neglect customized heterogeneous memory structures, thus degrading overall system performance. Fortunately, reinforcement learning can choose actions with maximum rewards values in a specific environment, leading the scheduler towards an optimal solution. In this paper, we propose a thread scheduling approach, i.e., Smart Scheduler, by leveraging a reinforcement learning method. Smart Scheduler takes OS event information as input, extends LinUCB to explore the scheduling space, and guides thread-level scheduling. We evaluate Smart Scheduler on the off-the-shelf server equipped with NVM. The experimental results show that the proposed Smart Scheduler can converge faster (usually within 20 actions) than rule-based algorithms and scheduling domain methods and reduce program execution time by up to 59.9%. It also outperforms rule-based algorithms and scheduling domain methods by 4.1% and 19.1% in quality of service latency.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Explore related subjects
Discover the latest articles and news from researchers in related subjects, suggested using machine learning.Change history
09 February 2023
A Correction to this paper has been published: https://doi.org/10.1007/s42514-023-00139-x
References
Agrawal, S., Goyal, N.: Thompson sampling for contextual bandits with linear payoffs (2012)
Arulraj, J., Pavlo, A.: How to build a non-volatile memory database management system. Proceedings of the 2017 ACM International Conference on Management of Data. SIGMOD ’17, pp. 1753–1758. Association for Computing Machinery, New York, NY, USA (2017). https://doi.org/10.1145/3035918.3054780
Azimi, R., Tam, D.K., Soares, L., Stumm, M.: Enhancing operating system support for multicore processors by using hardware performance monitoring. SIGOPS Oper. Syst. Rev. 43(2), 56–65 (2009). https://doi.org/10.1145/1531793.1531803
Auer, P.: Using confidence bounds for exploitation-exploration trade-offs. J. Mach. Learn. Res. 3(Nov), 397–422 (2002)
Ban, A.N.: Spearman correlation. (2019)
Bera, R., Kanellopoulos, K., Nori, A.V., Shahroodi, T., Subramoney, S., Mutlu, O.: Pythia: A customizable hardware prefetching framework using online reinforcement learning. 2021 54rd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO) (2021). IEEE
Blagodurov, S., Fedorova, A., Zhuravlev, S., Kamali, A.: A case for numa-aware contention management on multicore systems. 2010 19th International Conference on Parallel Architectures and Compilation Techniques (PACT), pp. 557–558 (2010)
Bonett, D., Wright, T.: Sample size requirements for estimating pearson, kendall and spearman correlations. Psychometrika 65(1), 23–28 (2000)
Chen, S., Jin, A., Delimitrou, C., Martınez, J.F.: ReTail: Opting for Learning Simplicity to Enable QoS-Aware Power Management in the Cloud. The 28th IEEE International Symposium on High-Performance Computer Architecture (HPCA-28), (2022)
Chen, Y., Peng, I.B., Peng, Z., Liu, X., Ren, B.: Atmem: Adaptive data placement in graph applications on heterogeneous memories. Proceedings of the 18th ACM/IEEE International Symposium on Code Generation and Optimization. CGO 2020, pp. 293–304. Association for Computing Machinery, New York, NY, USA (2020). https://doi.org/10.1145/3368826.3377922
Intel: Intel Optane Technology. Website. https://www.intel.com/content/www/us/en/architecture-and-technology/intel-optane-technology.html (2022)
Intel: Scaling MySQL with Intel Optane Persistent Memory (2019). https://www.intel.com/content/www/us/en/architecture-and-technology/scaling-mysql-with-optane-persistent-memory.html
Ipek, E., Mutlu, O., Martínez, J.F., Caruana, R.: Self-optimizing memory controllers: A reinforcement learning approach. 2008 International Symposium on Computer Architecture, pp. 39–50 (2008). https://doi.org/10.1109/ISCA.2008.21
Intel: https://github.com/opcm/pcm. On-line Resources (2018)
Intel-UPI (2017). https://en.wikipedia.org/wiki/Intel_Ultra_Path_Interconnect
Kiefer, T., Schlegel, B., Lehner, W.: Experimental evaluation of numa effects on database management systems. Markl, V., Saake, G., Sattler, K.-U., Hackenbroich, G., Mitschang, B., Härder, T., Köppen, V. (eds.) Datenbanksysteme Für Business, Technologie und Web (BTW) 2025, pp. 185–204. Gesellschaft für Informatik e.V., Bonn (2013)
Kernel, L.: https://www.kernel.org/doc/Documentation/filesystems/dax.txt. On-line Resources
Levy, S., Yao, R., Wu, Y., Dang, Y., Huang, P., Mu, Z., Zhao, P., Ramani, T., Govindaraju, N., Li, X., et al.: Predictive and adaptive failure mitigation to avert production cloud \(\{\)VM\(\}\) interruptions. 14th \(\{\)USENIX\(\}\) Symposium on Operating Systems Design and Implementation (\(\{\)OSDI\(\}\) 20), pp. 1155–1170 (2020)
Lepers, B., Quema, V., Fedorova, A.: Thread and memory placement on NUMA systems: Asymmetry matters. 2015 USENIX Annual Technical Conference (USENIX ATC 15), pp. 277–289. USENIX Association, Santa Clara, CA (2015). https://www.usenix.org/conference/atc15/technical-session/presentation/lepers
Li, T., Baumberger, D., Koufaty, D.A., Hahn, S.: Efficient operating system scheduling for performance-asymmetric multi-core architectures. SC ’07: Proceedings of the 2007 ACM/IEEE Conference on Supercomputing, pp. 1–11 (2007). https://doi.org/10.1145/1362622.1362694
Lin, T.-R., Penney, D., Pedram, M., Chen, L.: A deep reinforcement learning framework for architectural exploration: A routerless noc case study. 2020 IEEE International Symposium on High Performance Computer Architecture (HPCA), pp. 99–110 (2020). https://doi.org/10.1109/HPCA47549.2020.00018
Liu, L., Yang, S., Peng, L., Li, X.: Hierarchical hybrid memory management in os for tiered memory systems. IEEE Trans. Parallel Distributed Syst. 30(10), 2223–2236 (2019)
Liu, L.: QoS-Aware Machine Learning-based Multiple Resources Scheduling for Microservices in Cloud Environment. arxiv, (2019). https://arxiv.org/abs/1911.13208
Liu, L., Li, Y., Ding, C., Yang, H., Wu, C.: Rethinking memory management in modern operating system: horizontal, vertical or random? IEEE Trans. Comput. 65(6), 1921–1935 (2016)
Li, L., Chu, W., Langford, J., Schapire, R.E.: A contextual-bandit approach to personalized news article recommendation. Proceedings of the 19th International Conference on World Wide Web. WWW ’10, pp. 661–670. Association for Computing Machinery, New York, NY, USA (2010). https://doi.org/10.1145/1772690.1772758
Lv, F., Liu, L., Cui, H., Wang, L., Liu, Y., Feng, X., Yew, P.C.: WiseThrottling: a new asynchronous task scheduler for mitigating I/O bottleneck in large-scale datacenter servers. J. Supercomput. 71(8), 3054–3093 (2015)
Lv, F., Cui, H., Wang, L., Liu, L., Wu, C., Feng, X., Yew, P.C.: Dynamic I/O-aware scheduling for batch-mode applications on chip multiprocessor systems of cluster platforms. J. Comput. Sci. Technol. 29(1), 21–37 (2014)
McCurdy, C., Vetter, J.: Memphis: Finding and fixing numa-related performance problems on multi-core platforms. 2010 IEEE International Symposium on Performance Analysis of Systems Software (ISPASS), pp. 87–96 (2010). https://doi.org/10.1109/ISPASS.2010.5452060
Nishtala, R., Petrucci, V., Carpenter, P., Sjalander, M.: Twig: Multi-agent task management for colocated latency-critical cloud services. 2020 IEEE International Symposium on High Performance Computer Architecture (HPCA), pp. 167–179 (2020). https://doi.org/10.1109/HPCA47549.2020.00023
Mnih, V., Kavukcuoglu, K., Silver, D., Rusu, A.A., Veness, J., Bellemare, M.G., Graves, A., Riedmiller, M.A., Fidjeland, A., Ostrovski, G., Petersen, S., Beattie, C., Sadik, A., Antonoglou, I., King, H., Kumaran, D., Wierstra, D., Legg, S., Hassabis, D.: Human-level control through deep reinforcement learning. Nat. 518(7540), 529–533 (2015). https://doi.org/10.1038/nature14236
Piggin, N.: Scheduling Domains. On-line Resources (2002). https://lwn.net/Articles/80911/
PMDK: https://github.com/pmem/pmemkv. On-line Resources (2019)
Siddha, S., Pallipadi, V., Mallick, A.: Chip multi processing aware linux kernel scheduler. In: Linux Symposium, vol. 193 (2005). Citeseer
Scargall, S.: pmemkv: A Persistent In-Memory Key-Value Store, pp. 141–153. Apress, Berkeley, CA (2020). https://doi.org/10.1007/978-1-4842-4932-1_9
Van Riel, R., Chegu, V.: Automatic numa balancing. Red Hat Summit (2014)
Vermorel, J., Mohri, M.: Multi-armed bandit algorithms and empirical evaluation. Gama, J., Camacho, R., Brazdil, P.B., Jorge, A.M., Torgo, L. (eds.) Machine Learning: ECML 2005, pp. 437–448. Springer, Berlin, Heidelberg (2005)
Virouleau, P., Broquedis, F., Gautier, T., Rastello, F.: Using data dependencies to improve task-based scheduling strategies on numa architectures. Dutot, P.-F., Trystram, D. (eds.) Euro-Par 2016: Parallel Processing, pp. 531–544. Springer, Cham (2016)
Wang, Y., Jiang, D., Xiong, J.: Numa-aware thread migration for high performance nvmm file systems
Wang, Z., Liu, X., Yang, J., Michailidis, T., Swanson, S., Zhao, J.: Characterizing and modeling non-volatile memory systems. 2020 53rd Annual IEEE/ACM International Symposium on Microarchitecture (MICRO), pp. 496–508 (2020). IEEE
Yang, J., Kim, J., Hoseinzadeh, M., Izraelevitz, J., Swanson, S.: An Empirical Guide to the Behavior and Use of Scalable Persistent Memory (2019)
Yang, S., Li, X., Dou, X., Gong, X., Liu, H., Chen, L., Liu, L.: Monitoring memory behaviors and mitigating numa drawbacks on tiered nvm systems
Yu, S., Park, S., Baek, W.: Design and implementation of bandwidth-aware memory placement and migration policies for heterogeneous memory systems. Proceedings of the International Conference on Supercomputing, pp. 1–10 (2017)
Acknowledgements
We thank the reviewers for their valuable comments. This work is supported by the Key Research and Development Program of Guang Dong (No. 2021B0101310002), NSFC under grant No. 62072432. Yuetao Chen is a student member in Sys-Inventor Lab led by Lei Liu.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Chen, Y., Qiu, K., Chen, L. et al. Smart scheduler: an adaptive NVM-aware thread scheduling approach on NUMA systems. CCF Trans. HPC 4, 394–406 (2022). https://doi.org/10.1007/s42514-022-00110-2
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s42514-022-00110-2