Hiding message delivery latency using Direct-to-Cache-Transfer techniques in message passing environments

Abstract

Communication overhead is the key obstacle to reaching hardware performance limits. The majority is associated with software overhead, a significant portion of which is attributed to message copying. To reduce this copying overhead, we have devised techniques that do not require to copy a received message in order for it to be bound to its final destination. Rather, a late-binding mechanism, which involves address translation and a dedicated cache, facilitates fast access to received messages by the consuming process/thread.

We have introduced two policies namely Direct to Cache Transfer (DTCT) and lazy DTCT that determine whether a message after it is bound needs to be transferred into the data cache. We have studied the proposed methods in simulation and have shown their effectiveness in reducing access times to message payloads by the consuming process.

Introduction

Parallel architectures promise to provide increased computing power and scalability for computation-intensive applications. Low-latency and high-bandwidth communication, together with the sharing of data among processing elements are critical to obtaining high performance in these environments. The raw bandwidth of networks has increased significantly in the past few years and networking hardware supporting bandwidth in the order of gigabits per second has become available. However, the communication latency of traditional networking architectures and protocols has not decreased at the same pace as the raw interconnect capacity nor the increase in processor performance [1], [2].

As well as the above-mentioned observations concerning hardware, software makes the situation worse. Applications issue a system call when they need to send (receive) a message to (from) a network in a cluster system. The layered nature of legacy networking software and the use of expensive system calls and extra memory-to-memory copies profoundly affect the communication subsystem performance as seen in the running applications.

In traditional software messaging layers, there are usually four message copying operations from the send buffer to the receive buffer, as shown in Fig. 1. These occur during data transfer from the sender to a system buffer, and from the system to the network interface (NI) buffer. Crossing the network, at the receive side, the arrived message is copied from the NI to the system buffer, and also from the system to the receive buffer when the receive call is posted. Below, we elaborate further on the send and receive mechanisms.

Different methods can be employed to move a message from a source location to the network. These techniques involve Direct Memory Access (DMA) transfer or regular copy operations. Due to their better performance, contemporary processors leverage DMA techniques to transfer data between a processor’s memory system and a Network Interface Card (NIC). As such, this in turn affects the cache subsystem that experiences increased number of cache misses, which further affect the working set of a running application. Given the increased gap between memory and processor performance, any additional cache misses further affect the performance of the system.

I/O operations directly involve memory subsystems through accessing destination or source buffers, or indirectly by manipulating message descriptors. The cache subsystem, which is affected by the execution of communication protocols, experiences misses that further affect the working set of the running application. In turn, the gap between processor and memory performance makes the cache misses expensive.

To overcome the aforementioned problems, various techniques have been proposed to enhance some aspects of communication performance. These include improving network performance, providing more efficient communication software layers, designing dedicated message processors, providing bulk data transfers, off-loading or on-loading communication protocol processing, and integrating NIC and the processor on the same chip or using a cache to hold messages.

This paper is organized as follows. Section 2 discusses the background and the related work; Section 3 presents a description of the architectural extensions introduced to support efficient processing of sending and receiving of messages in message passing environments. The implementation of network cache is presented in Section 4; Section 5 describes the Direct-to-Cache-Data transfer polices. Then, Sections 6 Methodology and simulation environment, 7 Implementation discuss the simulation environment and our assumptions, Section 8 discusses the obtained results and the conclusions are presented in Section 9. Finally, we wrap up the paper with future work in Section 10.