Skip to main content
SpringerLink
Log in

Transpiling Python to Rust for Optimized Performance

  • Conference paper
  • First Online:
Book cover Embedded Computer Systems: Architectures, Modeling, and Simulation (SAMOS 2020)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 12471))

Included in the following conference series:

Abstract

Python has become the de facto programming language in machine learning and scientific computing, but high performance implementations are challenging to create especially for embedded systems with limited resources. We address the challenge of compiling and optimizing Python source code for a low-level target by introducing Rust as an intermediate source code step. We show that pre-existing Python implementations that depend on optimized libraries, such as NumPy, can be transpiled to Rust semi-automatically, with potential for further automation. We use two representative test cases, Black–Scholes for financial options pricing and robot trajectory optimization. The results show up to \(12\times \) speedup and \(1.5\times \) less memory use on PC, and the same performance but \(4\times \) less memory use on an ARM processor on PYNQ SoC FPGA. We also present a comprehensive list of factors for the process, to show the potential for fully automated transpilation. Our findings are generally applicable and can improve the performance of many Python applications while keeping their easy programmability.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Rust. https://www.rust-lang.org/.

  2. 2.

    Cython. https://github.com/cython/cython.

  3. 3.

    MicroPython. https://github.com/micropython/micropython.

  4. 4.

    C2Rust. https://github.com/immunant/c2rust.

  5. 5.

    perf. https://perf.wiki.kernel.org/.

  6. 6.

    valgrind. https://valgrind.org/.

  7. 7.

    The LLDB Debugger. https://lldb.llvm.org/.

  8. 8.

    Aliasing in Rust. https://doc.rust-lang.org/nomicon/aliasing.html.

  9. 9.

    Rust: No stdlib. https://doc.rust-lang.org/1.7.0/book/no-stdlib.html.

  10. 10.

    cargo the Rust package manager. https://doc.rust-lang.org/cargo/.

  11. 11.

    PYNQ Z1.https://reference.digilentinc.com/reference/programmable-logic/pynq-z1/start.

  12. 12.

    Snapdragon 835 HDK. https://developer.qualcomm.com/hardware/snapdragon-835-hdk.

  13. 13.

    rustup the Rust toolchain installer. https://github.com/rust-lang/rustup.

  14. 14.

    MonkeyType. https://github.com/Instagram/MonkeyType.

  15. 15.

    Visitor Pattern. https://en.wikipedia.org/wiki/Visitor_pattern.

  16. 16.

    cross. https://github.com/rust-embedded/cross.

  17. 17.

    Black–Scholes based on [19]. https://github.com/hegza/black-scholes-py.

  18. 18.

    Transpiled Black–Scholes. https://github.com/hegza/black-scholes-transpiled-rs.

  19. 19.

    ndarray. https://github.com/rust-ndarray/ndarray.

  20. 20.

    OpenBLAS. https://www.openblas.net/.

  21. 21.

    criterion.rs. https://github.com/bheisler/criterion.rs.

  22. 22.

    https://en.wikipedia.org/wiki/Put_option.

  23. 23.

    matrixmultiply. https://github.com/bluss/matrixmultiply.

  24. 24.

    tracemalloc. https://docs.python.org/3/library/tracemalloc.html.

  25. 25.

    RustPython. https://github.com/RustPython/RustPython.

  26. 26.

    syn. https://docs.rs/syn/1.0.16/syn/.

  27. 27.

    cargo-asm. https://github.com/gnzlbg/cargo-asm.

  28. 28.

    FlameGraph. https://github.com/brendangregg/FlameGraph.

  29. 29.

    rayon. https://github.com/rayon-rs/rayon.

References

  1. Abadi, M., et al.: Tensorflow: a system for large-scale machine learning. In: 12th \(\{\)USENIX\(\}\) Symposium on Operating Systems Design and Implementation (\(\{\)OSDI\(\}\) 16), pp. 265–283 (2016)

    Google Scholar 

  2. Behnel, S., et al.: Cython: C-Extensions for Python (2008)

    Google Scholar 

  3. Bradbury, J., et al.: JAX: Composable Transformations of Python+NumPy programs (2018). http://github.com/google/jax

  4. Chen, T., et al.: \(\{\)TVM\(\}\): An automated end-to-end optimizing compiler for deep learning. In: 13th \(\{\)USENIX\(\}\) Symposium on Operating Systems Design and Implementation (\(\{\)OSDI\(\}\) 18), pp. 578–594 (2018)

    Google Scholar 

  5. Frostig, R., Johnson, M.J., Leary, C.: Compiling machine learning programs via high-level tracing. In: Systems for Machine Learning (2018)

    Google Scholar 

  6. Instagram: (2020). https://github.com/Instagram/MonkeyType

  7. JetBrains: (2020). https://www.jetbrains.com/idea/

  8. Konchunas, J.: Python to rust transpiler (2020). http://www.github.com/konchunas/pyrs

  9. Kristensen, M.R., Lund, S.A., Blum, T., Skovhede, K., Vinter, B.: Bohrium: unmodified numpy code on cpu, gpu, and cluster. In: 4th Workshop on Python for High Performance and Scientific Computing (PyHPC’13) (2013)

    Google Scholar 

  10. Kristensen, M.R., Lund, S.A., Blum, T., Skovhede, K., Vinter, B.: Bohrium: a virtual machine approach to portable parallelism. In: 2014 IEEE International Parallel & Distributed Processing Symposium Workshops, pp. 312–321. IEEE (2014)

    Google Scholar 

  11. Leary, C., Wang, T.: Xla: Tensorflow, compiled. TensorFlow Dev Summit (2017)

    Google Scholar 

  12. Maclaurin, D., Duvenaud, D., Adams, R.P.: Autograd: Effortless gradients in numpy. In: ICML 2015 AutoML Workshop, vol. 238 (2015)

    Google Scholar 

  13. Oliphant, T., Peterson, P., Eric, J.: Scipy.org (2001). https://www.scipy.org/

  14. Ousterhout, K., Rasti, R., Ratnasamy, S., Shenker, S., Chun, B.G.: Making sense of performance in data analytics frameworks. In: 12th \(\{\)USENIX\(\}\) Symposium on Networked Systems Design and Implementation (\(\{\)NSDI\(\}\) 15), pp. 293–307 (2015)

    Google Scholar 

  15. Palkar, S., et al.: Weld: rethinking the interface between data-intensive applications. arXiv preprint arXiv:1709.06416 (2017)

  16. Palkar, S., et al.: Evaluating end-to-end optimization for data analytics applications in weld. Proc. VLDB Endow. 11(9), 1002–1015 (2018)

    Article  Google Scholar 

  17. Palkar, S., et al.: Weld: A common runtime for high performance data analytics. In: Conference on Innovative Data Systems Research (CIDR) (2017)

    Google Scholar 

  18. Palkar, S., Zaharia, M.: Optimizing data-intensive computations in existing libraries with split annotations. In: Proceedings of the 27th ACM Symposium on Operating Systems Principles, pp. 291–305 (2019)

    Google Scholar 

  19. Schlegel, A.: Black-scholes formula and python implementation (2018). https://aaronschlegel.me/black-scholes-formula-python.html

  20. Singh, A.K., Ahonen, A., Ghabcheloo, R., Muller, A.: Inducing multi-convexity in path constrained trajectory optimization for mobile manipulators. arXiv preprint arXiv:1904.09780 (2019)

  21. van der Walt, S., Colbert, S.C., Varoquaux, G.: The numpy array: a structure for efficient numerical computation. Comput. Sci. Eng. 13(2), 22–30 (2011)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Tampere University, Tampere, Finland

    Henri Lunnikivi, Kai Jylkkä & Timo Hämäläinen

Authors
  1. Henri Lunnikivi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kai Jylkkä
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Timo Hämäläinen
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Henri Lunnikivi .

Editor information

Editors and Affiliations

  1. Department of Computer Science and Engineering, University of California San Diego, La Jolla, CA, USA

    Alex Orailoglu

  2. Department of Electrical and Computer Engineering, Fraunhofer IESE, Kaiserslautern, Germany

    Matthias Jung

  3. Department of Computer Science, Friedrich-Alexander University, Erlangen, Germany

    Marc Reichenbach

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Lunnikivi, H., Jylkkä, K., Hämäläinen, T. (2020). Transpiling Python to Rust for Optimized Performance. In: Orailoglu, A., Jung, M., Reichenbach, M. (eds) Embedded Computer Systems: Architectures, Modeling, and Simulation. SAMOS 2020. Lecture Notes in Computer Science(), vol 12471. Springer, Cham. https://doi.org/10.1007/978-3-030-60939-9_9

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-60939-9_9

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-60938-2

  • Online ISBN: 978-3-030-60939-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation