PINstruct – Efficient Memory Access to Data Structures

Abstract

Modular programming requires structuring a program’s data into classes, records and arrays. Access to these data structures is highly dependent on the nature of the algorithm, the programming language, it’s compiler, the Application Binary Interface (ABI) defined by the OS and finally the requirements of the underlying hardware. The programming language however may not allow the compiler to reorder data structures. This is specially true for libraries, with MPI being a good example. Since the communication library’s routines are comparatively seldom called, the cache may be ”cold”, i.e. the MPI library’s memory accesses into data structures may be slow, too diverse and pollute the cache. In this paper we introduce a tool to analyse the memory access pattern of structures the Open MPI implementation. This tool shows the order in which data structures such as datatype and communicator information are accessed. It shows that previous work to re-order data structures for better padding has proven worthwhile, however further restructuring is necessary.