Points for Free: Embedding Pointful Array Programming in Python
Pages 1 - 12
Abstract
Multidimensional array operations are ubiquitous in machine learning. The dominant ecosystem in this field is centred around Python and NumPy, where programs are expressed with elaborate and error-prone calls in the point-free array programming model. Such code is difficult to statically analyse and maintain. Various other array programming paradigms offer to solve these problems, in particular the pointful style of Dex. However, only limited approaches – based on Einstein summation – have been embedded in Python. We introduce Ein, a pointful array DSL embedded in Python. We also describe a novel connection between pointful and point-free array programming. Thanks to this connection, Ein generates performant and type-safe calls to NumPy with potential for further optimisations. Ein reconciles the readability of comprehension-style definitions with the capabilities of existing array frameworks.
References
[1]
Martín Abadi, Paul Barham, Jianmin Chen, Zhifeng Chen, Andy Davis, Jeffrey Dean, Matthieu Devin, Sanjay Ghemawat, Geoffrey Irving, and Michael Isard. 2016. TensorFlow: a system for large-scale machine learning. In 12th USENIX symposium on operating systems design and implementation (OSDI 16). 265–283.
[2]
S Kamal Abdali and B David Saunders. 1985. Transitive closure and related semiring properties via eliminants. Theoretical Computer Science, 40 (1985), 257–274.
[3]
Robert Atkey, Sam Lindley, and Jeremy Yallop. 2009. Unembedding domain-specific languages. In Proceedings of the 2nd ACM SIGPLAN symposium on Haskell. 37–48. https://doi.org/10.1145/1596638.1596644
[4]
Manuel MT Chakravarty, Gabriele Keller, Sean Lee, Trevor L McDonell, and Vinod Grover. 2011. Accelerating Haskell array codes with multicore GPUs. In Proceedings of the sixth workshop on Declarative aspects of multicore programming. 3–14. https://doi.org/10.1145/1926354.1926358
[5]
David Chiang, Alexander M Rush, and Boaz Barak. 2022. Named Tensor Notation. Transactions on Machine Learning Research.
[6]
Stephen Dolan. 2013. Fun with semirings: a functional pearl on the abuse of linear algebra. In Proceedings of the 18th ACM SIGPLAN international conference on Functional programming. 101–110.
[7]
Roy Frostig, Matthew James Johnson, and Chris Leary. 2018. Compiling machine learning programs via high-level tracing. Systems for Machine Learning, 4, 9 (2018).
[8]
Jeremy Gibbons. 2016. Aplicative programming with Naperian functors. In Proceedings of the 1st International Workshop on Type-Driven Development. 13–14. https://doi.org/10.1145/2976022.2976023
[9]
Jeremy Gibbons and Nicolas Wu. 2014. Folding domain-specific languages: deep and shallow embeddings (functional pearl). In Proceedings of the 19th ACM SIGPLAN international conference on Functional programming. 339–347. https://doi.org/10.1145/2628136.2628138
[10]
The Futhark Hackers. 2024. Futhark Benchmarks. https://github.com/diku-dk/futhark-benchmarks
[11]
Charles R Harris, K Jarrod Millman, Stéfan J Van Der Walt, Ralf Gommers, Pauli Virtanen, David Cournapeau, Eric Wieser, Julian Taylor, Sebastian Berg, and Nathaniel J Smith. 2020. Array programming with NumPy. Nature, 585, 7825 (2020), 357–362. https://doi.org/10.1038/s41586-020-2649-2
[12]
Troels Henriksen, Niels GW Serup, Martin Elsman, Fritz Henglein, and Cosmin E Oancea. 2017. Futhark: purely functional GPU-programming with nested parallelism and in-place array updates. In Proceedings of the 38th ACM SIGPLAN Conference on Programming Language Design and Implementation. 556–571. https://doi.org/10.1145/3062341.3062354
[13]
Paul Hudak. 1996. Building domain-specific embedded languages. ACM Computing Surveys (CSUR), 28, 4es (1996), 196–es.
[14]
Kenneth E Iverson. 1962. A programming language. In Proceedings of the May 1-3, 1962, spring joint computer conference. 345–351. https://doi.org/10.1145/1460833.1460872
[15]
Ł ukasz Lachowski. 2018. On the complexity of the standard translation of lambda calculus into combinatory logic. Reports on Mathematical Logic, 19–42.
[16]
Dougal Maclaurin, Alexey Radul, Matthew J Johnson, and Dimitrios Vytiniotis. 2019. Dex: array programming with typed indices. In Program Transformations for ML Workshop at NeurIPS 2019.
[17]
Conor McBride and Ross Paterson. 2008. Applicative programming with effects. Journal of functional programming, 18, 1 (2008), 1–13. https://doi.org/10.1017/S0956796807006326
[18]
Adam Paszke, Sam Gross, Francisco Massa, Adam Lerer, James Bradbury, Gregory Chanan, Trevor Killeen, Zeming Lin, Natalia Gimelshein, and Luca Antiga. 2019. PyTorch: An imperative style, high-performance deep learning library. Advances in neural information processing systems, 32 (2019).
[19]
Adam Paszke, Daniel D Johnson, David Duvenaud, Dimitrios Vytiniotis, Alexey Radul, Matthew J Johnson, Jonathan Ragan-Kelley, and Dougal Maclaurin. 2021. Getting to the Point: index sets and parallelism-preserving autodiff for pointful array programming. Proceedings of the ACM on Programming Languages, 5, ICFP (2021), 1–29. https://doi.org/10.1145/3473593
[20]
Alex Rogozhnikov. 2021. Einops: Clear and reliable tensor manipulations with einstein-like notation. In International Conference on Learning Representations.
[21]
Sven-Bodo Scholz. 2003. Single Assignment C: efficient support for high-level array operations in a functional setting. Journal of functional programming, 13, 6 (2003), 1005–1059. https://doi.org/10.1017/S0956796802004458
[22]
Sven-Bodo Scholz and Artjoms Šinkarovs. 2019. Tensor comprehensions in SaC. In Proceedings of the 31st Symposium on Implementation and Application of Functional Languages. 1–13.
[23]
Amir Shaikhha, Andrew Fitzgibbon, Dimitrios Vytiniotis, and Simon Peyton Jones. 2019. Efficient differentiable programming in a functional array-processing language. Proceedings of the ACM on Programming Languages, 3, ICFP (2019), 1–30. https://doi.org/10.1145/3341701
[24]
Bo Joel Svensson and Josef Svenningsson. 2014. Defunctionalizing push arrays. In Proceedings of the 3rd ACM SIGPLAN Workshop on Functional High-performance Computing. 43–52.
[25]
Nicolas Vasilache, Oleksandr Zinenko, Theodoros Theodoridis, Priya Goyal, Zachary DeVito, William S Moses, Sven Verdoolaege, Andrew Adams, and Albert Cohen. 2018. Tensor Comprehensions: Framework-agnostic high-performance machine learning abstractions. arXiv preprint arXiv:1802.04730.
Index Terms
- Points for Free: Embedding Pointful Array Programming in Python
Recommendations
Programming with Python
Python is a powerful, easy-to-use programming language based on other traditional languages. In this article, the author discusses why his company, NovaSys Health, chose Python as its primary programming language. The author notes that Python is ...
Comments
Information & Contributors
Information
Published In

Copyright © 2024 Copyright is held by the owner/author(s). Publication rights licensed to ACM.
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 20 June 2024
Check for updates
Author Tags
Qualifiers
- Research-article
Conference
ARRAY '24
Sponsor:
ARRAY '24: 10th ACM SIGPLAN International Workshop on Libraries, Languages and Compilers for Array Programming
June 25, 2024
Copenhagen, Denmark
Acceptance Rates
Overall Acceptance Rate 17 of 25 submissions, 68%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 171Total Downloads
- Downloads (Last 12 months)171
- Downloads (Last 6 weeks)12
Reflects downloads up to 16 Feb 2025
Other Metrics
Citations
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in