Abstract
Sparse computation, where sparse formats are used to compress nonzero values of big data, are commonly used in real world applications. However, the compiler-based data dependence analysis of sparse computations needed for automatic parallelization is difficult due to usage of indirect memory accesses through index arrays, e.g. col in val[col[j]], in these computations. One use of such data dependence analysis is to find opportunities for array privatization, which is an approach to increase available parallelism in a loop by providing each parallel thread its own copy of arrays where in each iteration the array reads are dominated by array writes in the same iteration. In this paper, we expand opportunities for compile-time array privatization in sparse computations by using newly formulated index array properties and a novel concept we call content-based privatization. Furthermore, we discuss existing opportunities to use our approach for detecting private arrays in existing library implementations of sparse computations.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Bell, N., Garland, M.: Implementing sparse matrix-vector multiplication on throughput-oriented processors. In: SC 2009: Proceedings of the Conference on High Performance Computing Networking, Storage and Analysis, pp. 1–11. ACM, New York (2009)
Byun, J.H., Lin, R., Yelick, K.A., Demmel, J.: Autotuning sparse matrix-vector multiplication for multicore. Technical report, UCB/EECS-2012-215, November 2012
Cheshmi, K., Kamil, S., Strout, M.M., Dehnavi, M.M.: Sympiler: Transforming sparse matrix codes by decoupling symbolic analysis. In: Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2017, pp. 13:1–13:13. ACM, New York (2017). https://doi.org/10.1145/3126908.3126936, http://doi.acm.org/10.1145/3126908.3126936
Davis, T., Hager, W., Duff, I.: Suitesparse (2014). http://faculty.cse.tamu.edu/davis/suitesparse.html
Lin, Y., Padua, D.: Compiler analysis of irregular memory accesses. In: Proceedings of the 21st Conference on Programming Language Design and Implementation, PLDI 2000, pp. 157–168 (2000). https://doi.org/10.1145/349299.349322
Lin, Y., Padua, D.: Compiler analysis of irregular memory accesses. In: Proceedings of the ACM SIGPLAN Conference on Programming Language Design and Implementation, vol. 35, pp. 157–168. ACM, New York, May 2000
McAuley, J., Leskovec, J.: Hidden factors and hidden topics: understanding rating dimensions with review text. In: Proceedings of the 7th ACM Conference on Recommender Systems, RecSys 2013, pp. 165–172. ACM, New York (2013). https://doi.org/10.1145/2507157.2507163, http://doi.acm.org/10.1145/2507157.2507163
Mohammadi, M.S., Cheshmi, K., Dehnavi, M.M., Venkat, A., Yuki, T., Strout, M.M.: Extending Index-array properties for data dependence analysis. In: Hall, M., Sundar, H. (eds.) LCPC 2018. LNCS, vol. 11882, pp. 78–93. Springer, Cham (2019). https://doi.org/10.1007/978-3-030-34627-0_7
Mohammadi, M.S., et al.: Sparse computation data dependence simplification for efficient compiler-generated inspectors. In: Proceedings of the 40th ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI 2019, pp. 594–609. Association for Computing Machinery, New York (2019). https://doi.org/10.1145/3314221.3314646
Oancea, C.E., Rauchwerger, L.: Logical inference techniques for loop parallelization. In: Proceedings of the 33rd ACM SIGPLAN Conference on Programming Language Design and Implementation, PLDI 2012, pp. 509–520. ACM, New York (2012)
Padua, D.A., Wolfe, M.J.: Advanced compiler optimizations for supercomputers. Commun. ACM 29(12), 1184–1201 (1986)
Paek, Y., Hoeflinger, J., Padua, D.: Efficient and precise array access analysis. ACM Trans. Program. Lang. Syst. 24(1), 65–109 (2002)
Park, J., Smelyanskiy, M., Sundaram, N., Dubey, P.: Sparsifying synchronization for high-performance shared-memory sparse triangular solver. In: Kunkel, J.M., Ludwig, T., Meuer, H.W. (eds.) ISC 2014. LNCS, vol. 8488, pp. 124–140. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-07518-1_8
Park, J., et al.: Efficient shared-memory implementation of high-performance conjugate gradient benchmark and its application to unstructured matrices. In: Proceedings of International Conference for High Performance Computing, Networking, Storage and Analysis, SC 2014, pp. 945–955. IEEE Press, Piscataway (2014)
Pugh, B., Wonnacott, D.: Nonlinear array dependence analysis. Technical report CS-TR-3372, Department of Computer Science, University of Maryland, November 1994
Pugh, W., Wonnacott, D.: Constraint-based array dependence analysis. ACM Trans. Program. Lang. Syst. 20(3), 635–678 (1998)
Rauchwerger, L., Padua, D.: The privatizing DOALL test: a run-time technique for DOALL loop identification and array privatization. In: Proceedings of the 8th International Conference on Supercomputing. ICS 1994, pp. 33–43. Association for Computing Machinery, New York (1994). https://doi.org/10.1145/181181.181254
Rauchwerger, L., Padua, D.A.: The LRPD test: speculative run-time parallelization of loops with privatization and reduction parallelization. IEEE Trans. Parallel Distrib. Syst. 10(2), 160–180 (1999). https://doi.org/10.1109/71.752782
Rennich, S.C., Stosic, D., Davis, T.A.: Accelerating sparse Cholesky factorization on GPUs. Parallel Comput. 59, 140–150 (2016)
Rus, S., Hoeflinger, J., Rauchwerger, L.: Hybrid analysis: static & dynamic memory reference analysis. Int. J. Parallel Program. 31(4), 251–283 (2003)
Rus, S.V.: Hybrid analysis of memory references and its application to automatic parallelization. Ph.D. thesis, Texas A&M (2006)
Tu, P., Padua, D.: Automatic array privatization. In: Banerjee, U., Gelernter, D., Nicolau, A., Padua, D. (eds.) LCPC 1993. LNCS, vol. 768, pp. 500–521. Springer, Heidelberg (1994). https://doi.org/10.1007/3-540-57659-2_29
Wang, E., et al.: Intel math kernel library. In: Wang, E., et al. (eds.) High-Performance Computing on the Intel® Xeon Phi™, pp. 167–188. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-06486-4_7
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 Springer Nature Switzerland AG
About this paper
Cite this paper
Mohammadi, M.S., Hall, M., Strout, M.M. (2022). Expanding Opportunities for Array Privatization in Sparse Computations. In: Chapman, B., Moreira, J. (eds) Languages and Compilers for Parallel Computing. LCPC 2020. Lecture Notes in Computer Science(), vol 13149. Springer, Cham. https://doi.org/10.1007/978-3-030-95953-1_3
Download citation
DOI: https://doi.org/10.1007/978-3-030-95953-1_3
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-95952-4
Online ISBN: 978-3-030-95953-1
eBook Packages: Computer ScienceComputer Science (R0)