Shared Memory vs Message Passing on SCI: A Case Study Using Split-C

Abstract

Non-coherent remote memory access with a simple load-store interface is one of the characteristic features of the Scalable Coherent Interface (SCI). It allows SCI to bridge the gap between efficient (and expensive) hardware implementations of true coherent shared memory and low-performance software implementations. It also allows SCI to be used as a substrate for efficient implementations of higher-level communication mechanisms. In this chapter, we evaluate the tradeoffs for using SCI to implement user-level message-passing and a global address-space scheme with shared-memory segments. We perform this evaluation in the context of Split-C, an explicitly parallel language which provides a global address-space abstraction as well as bulk transfer operations.

For the message-passing implementation, we use SCI to implement Active Messages and use Active Messages as the communication layer for the Split-C runtime system. For the shared memory implementation, we use SCI operations directly as the communication layer for Split-C. The direct implementation has the potential to provide a performance advantage by reducing copying and synchronization. However, our SCI hardware poses problems that make an efficient direct implementation very challenging. Our studies show that neither paradigm is optimal in our current setting and suggest a combination of both paradigms for increased efficiency.