Abstract
Python has become the de facto programming language in machine learning and scientific computing, but high performance implementations are challenging to create especially for embedded systems with limited resources. We address the challenge of compiling and optimizing Python source code for a low-level target by introducing Rust as an intermediate source code step. We show that pre-existing Python implementations that depend on optimized libraries, such as NumPy, can be transpiled to Rust semi-automatically, with potential for further automation. We use two representative test cases, Black–Scholes for financial options pricing and robot trajectory optimization. The results show up to \(12\times \) speedup and \(1.5\times \) less memory use on PC, and the same performance but \(4\times \) less memory use on an ARM processor on PYNQ SoC FPGA. We also present a comprehensive list of factors for the process, to show the potential for fully automated transpilation. Our findings are generally applicable and can improve the performance of many Python applications while keeping their easy programmability.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Notes
- 1.
Rust. https://www.rust-lang.org/.
- 2.
Cython. https://github.com/cython/cython.
- 3.
MicroPython. https://github.com/micropython/micropython.
- 4.
C2Rust. https://github.com/immunant/c2rust.
- 5.
- 6.
valgrind. https://valgrind.org/.
- 7.
The LLDB Debugger. https://lldb.llvm.org/.
- 8.
Aliasing in Rust. https://doc.rust-lang.org/nomicon/aliasing.html.
- 9.
Rust: No stdlib. https://doc.rust-lang.org/1.7.0/book/no-stdlib.html.
- 10.
cargo the Rust package manager. https://doc.rust-lang.org/cargo/.
- 11.
- 12.
Snapdragon 835 HDK. https://developer.qualcomm.com/hardware/snapdragon-835-hdk.
- 13.
rustup the Rust toolchain installer. https://github.com/rust-lang/rustup.
- 14.
MonkeyType. https://github.com/Instagram/MonkeyType.
- 15.
Visitor Pattern. https://en.wikipedia.org/wiki/Visitor_pattern.
- 16.
- 17.
Black–Scholes based on [19]. https://github.com/hegza/black-scholes-py.
- 18.
Transpiled Black–Scholes. https://github.com/hegza/black-scholes-transpiled-rs.
- 19.
ndarray. https://github.com/rust-ndarray/ndarray.
- 20.
OpenBLAS. https://www.openblas.net/.
- 21.
criterion.rs. https://github.com/bheisler/criterion.rs.
- 22.
- 23.
matrixmultiply. https://github.com/bluss/matrixmultiply.
- 24.
tracemalloc. https://docs.python.org/3/library/tracemalloc.html.
- 25.
RustPython. https://github.com/RustPython/RustPython.
- 26.
- 27.
cargo-asm. https://github.com/gnzlbg/cargo-asm.
- 28.
FlameGraph. https://github.com/brendangregg/FlameGraph.
- 29.
References
Abadi, M., et al.: Tensorflow: a system for large-scale machine learning. In: 12th \(\{\)USENIX\(\}\) Symposium on Operating Systems Design and Implementation (\(\{\)OSDI\(\}\) 16), pp. 265–283 (2016)
Behnel, S., et al.: Cython: C-Extensions for Python (2008)
Bradbury, J., et al.: JAX: Composable Transformations of Python+NumPy programs (2018). http://github.com/google/jax
Chen, T., et al.: \(\{\)TVM\(\}\): An automated end-to-end optimizing compiler for deep learning. In: 13th \(\{\)USENIX\(\}\) Symposium on Operating Systems Design and Implementation (\(\{\)OSDI\(\}\) 18), pp. 578–594 (2018)
Frostig, R., Johnson, M.J., Leary, C.: Compiling machine learning programs via high-level tracing. In: Systems for Machine Learning (2018)
Instagram: (2020). https://github.com/Instagram/MonkeyType
JetBrains: (2020). https://www.jetbrains.com/idea/
Konchunas, J.: Python to rust transpiler (2020). http://www.github.com/konchunas/pyrs
Kristensen, M.R., Lund, S.A., Blum, T., Skovhede, K., Vinter, B.: Bohrium: unmodified numpy code on cpu, gpu, and cluster. In: 4th Workshop on Python for High Performance and Scientific Computing (PyHPC’13) (2013)
Kristensen, M.R., Lund, S.A., Blum, T., Skovhede, K., Vinter, B.: Bohrium: a virtual machine approach to portable parallelism. In: 2014 IEEE International Parallel & Distributed Processing Symposium Workshops, pp. 312–321. IEEE (2014)
Leary, C., Wang, T.: Xla: Tensorflow, compiled. TensorFlow Dev Summit (2017)
Maclaurin, D., Duvenaud, D., Adams, R.P.: Autograd: Effortless gradients in numpy. In: ICML 2015 AutoML Workshop, vol. 238 (2015)
Oliphant, T., Peterson, P., Eric, J.: Scipy.org (2001). https://www.scipy.org/
Ousterhout, K., Rasti, R., Ratnasamy, S., Shenker, S., Chun, B.G.: Making sense of performance in data analytics frameworks. In: 12th \(\{\)USENIX\(\}\) Symposium on Networked Systems Design and Implementation (\(\{\)NSDI\(\}\) 15), pp. 293–307 (2015)
Palkar, S., et al.: Weld: rethinking the interface between data-intensive applications. arXiv preprint arXiv:1709.06416 (2017)
Palkar, S., et al.: Evaluating end-to-end optimization for data analytics applications in weld. Proc. VLDB Endow. 11(9), 1002–1015 (2018)
Palkar, S., et al.: Weld: A common runtime for high performance data analytics. In: Conference on Innovative Data Systems Research (CIDR) (2017)
Palkar, S., Zaharia, M.: Optimizing data-intensive computations in existing libraries with split annotations. In: Proceedings of the 27th ACM Symposium on Operating Systems Principles, pp. 291–305 (2019)
Schlegel, A.: Black-scholes formula and python implementation (2018). https://aaronschlegel.me/black-scholes-formula-python.html
Singh, A.K., Ahonen, A., Ghabcheloo, R., Muller, A.: Inducing multi-convexity in path constrained trajectory optimization for mobile manipulators. arXiv preprint arXiv:1904.09780 (2019)
van der Walt, S., Colbert, S.C., Varoquaux, G.: The numpy array: a structure for efficient numerical computation. Comput. Sci. Eng. 13(2), 22–30 (2011)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Lunnikivi, H., Jylkkä, K., Hämäläinen, T. (2020). Transpiling Python to Rust for Optimized Performance. In: Orailoglu, A., Jung, M., Reichenbach, M. (eds) Embedded Computer Systems: Architectures, Modeling, and Simulation. SAMOS 2020. Lecture Notes in Computer Science(), vol 12471. Springer, Cham. https://doi.org/10.1007/978-3-030-60939-9_9
Download citation
DOI: https://doi.org/10.1007/978-3-030-60939-9_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-60938-2
Online ISBN: 978-3-030-60939-9
eBook Packages: Computer ScienceComputer Science (R0)