Abstract
In this paper, we investigate the applicability of reinforcement learning as a possible approach to select code variants. Our approach is based on the observation that code variants are usually convertible between one another by code transformations. Actor-critic proximal policy optimization is identified as a suitable reinforcement learning algorithm. To study its applicability, a software framework is implemented and used to perform experiments on three different hardware platforms using a class of explicit solution methods for systems of ordinary differential equations as an example application.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Adams, A., et al.: Learning to optimize halide with tree search and random programs. ACM Trans. Graph. 38(4), 1–12 (2019). https://doi.org/10.1145/3306346.3322967
Ansel, J., et al.: OpenTuner: an extensible framework for program autotuning. In: Proceedings of the 23rd International Conference on Parallel Architectures and Compilation (PACT 2014). ACM (2014). https://doi.org/10.1145/2628071.2628092
Beckingsale, D., Pearce, O., Laguna, I., Gamblin, T.: Apollo: reusable models for fast, dynamic tuning of input-dependent code. In: 2017 IEEE International Parallel and Distributed Processing Symposium (IPDPS), pp. 307–316 (2017). https://doi.org/10.1109/IPDPS.2017.38
Bilmes, J., Asanovic, K., Chin, C.W., Demmel, J.: Optimizing matrix multiply using PHiPAC: a portable, high-performance, ANSI C coding methodology. In: Proceedings of the 11th International Conference on Supercomputing (ICS 1997), pp. 340–347. ACM, July 1997. https://doi.org/10.1145/263580.263662
Haj-Ali, A., Ahmed, N.K., Willke, T., Shao, Y.S., Asanovic, K., Stoica, I.: NeuroVectorizer: end-to-end vectorization with deep reinforcement learning. In: Proceedings of the 18th ACM/IEEE International Symposium on Code Generation and Optimization. ACM, February 2020. https://doi.org/10.1145/3368826.3377928
Heo, K., Lee, W., Pashakhanloo, P., Naik, M.: Effective program debloating via reinforcement learning. In: Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security. ACM, October 2018. https://doi.org/10.1145/3243734.3243838
van der Houwen, P., Sommeijer, B.: Parallel iteration of high-order Runge-Kutta methods with stepsize control. J. Comput. Appl. Math. 29(1), 111–127 (1990). https://doi.org/10.1016/0377-0427(90)90200-J
Konduit: DL4J: Deep learning for Java (2022). https://deeplearning4j.konduit.ai/
Mammadli, R., Jannesari, A., Wolf, F.: Static neural compiler optimization via deep reinforcement learning. In: 2020 IEEE/ACM 6th Workshop on the LLVM Compiler Infrastructure in HPC (LLVM-HPC) and Workshop on Hierarchical Parallelism for Exascale Computing (HiPar), pp. 1–11 (2020). https://doi.org/10.1109/LLVMHPCHiPar51896.2020.00006
Mirhoseini, A., et al.: Device placement optimization with reinforcement learning. In: Proceedings of the 34th International Conference on Machine Learning (ICML 2017), vol. 70, pp. 2430–2439. PMLR (2017)
Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518(7540), 529–533 (2015). https://doi.org/10.1038/nature14236
Muralidharan, S., Roy, A., Hall, M., Garland, M., Rai, P.: Architecture-adaptive code variant tuning. SIGOPS Oper. Syst. Rev. 50(2), 325–338 (2016). https://doi.org/10.1145/2954680.2872411
Pfaffe, P., Grosser, T., Tillmann, M.: Efficient hierarchical online-autotuning: a case study on polyhedral accelerator mapping. In: Proceedings of the ACM International Conference on Supercomputing (ICS 2019), pp. 354–366. ACM (2019). https://doi.org/10.1145/3330345.3330377
Pfaffe, P., Tillmann, M., Walter, S., Tichy, W.F.: Online-autotuning in the presence of algorithmic choice. In: 2017 IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW 2017), pp. 1379–1388 (2017). https://doi.org/10.1109/IPDPSW.2017.28
Ragan-Kelley, J., Barnes, C., Adams, A., Paris, S., Durand, F., Amarasinghe, S.: Halide: a language and compiler for optimizing parallelism, locality, and recomputation in image processing pipelines. In: Proceedings of the 34th ACM SIGPLAN Conference on Programming Language Design and Implementation (PLDI 2013), pp. 519–530. ACM (2013). https://doi.org/10.1145/2491956.2462176
Rasch, A., Gorlatch, S.: ATF: a generic directive-based auto-tuning framework. Concurr. Comput. Pract. Exp. 31(5) (2019). https://doi.org/10.1002/cpe.4423
Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy optimization algorithms, July 2017. http://arxiv.org/abs/1707.06347
Seiferth, J., Korch, M., Rauber, T.: Offsite autotuning approach. In: Sadayappan, P., Chamberlain, B.L., Juckeland, G., Ltaief, H. (eds.) ISC High Performance 2020. LNCS, vol. 12151, pp. 370–390. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-50743-5_19
Sutton, R.S., Barto, A.G.: Reinforcement Learning, 2nd edn. The MIT Press, Cambridge (2018)
Tiwari, A., Hollingsworth, J.K.: Online adaptive code generation and tuning. In: Proceedings of the 2011 IEEE International Parallel Distributed Processing Symposium (IPDPS 2011), pp. 879–892. IEEE, May 2011. https://doi.org/10.1109/IPDPS.2011.86
Vigueras, G., Carro, M., Tamarit, S., Mariño, J.: Towards automatic learning of heuristics for mechanical transformations of procedural code. In: Electronic Proceedings in Theoretical Computer Science, vol. 237, pp. 52–67. Open Publishing Association, January 2017. https://doi.org/10.4204/eptcs.237.4
Wang, T., Jain, N., Boehme, D., Beckingsale, D., Mueller, F., Gamblin, T.: CodeSeer: input-dependent code variants selection via machine learning. In: Proceedings of the 34th ACM International Conference on Supercomputing. ACM, June 2020. https://doi.org/10.1145/3392717.3392741
Whaley, R.C., Petitet, A., Dongarra, J.: Automated empirical optimizations of software and the ATLAS project. Parallel Comput. 27(1), 3–35 (2001). https://doi.org/10.1016/S0167-8191(00)00087-9
Acknowledgments
This work has been supported by the German Research Foundation (DFG) under grant KO 2252/3-2.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Hacker, O., Korch, M., Seiferth, J. (2022). A Motivating Case Study on Code Variant Selection by Reinforcement Learning. In: Varbanescu, AL., Bhatele, A., Luszczek, P., Marc, B. (eds) High Performance Computing. ISC High Performance 2022. Lecture Notes in Computer Science, vol 13289. Springer, Cham. https://doi.org/10.1007/978-3-031-07312-0_15
Download citation
DOI: https://doi.org/10.1007/978-3-031-07312-0_15
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-07311-3
Online ISBN: 978-3-031-07312-0
eBook Packages: Computer ScienceComputer Science (R0)