D³-Machine: A decoupled data-driven multithreaded architecture with variable resolution support

Abstract

This paper presents the Decoupled Data-Driven machine (D³-machine), a multithreaded architecture with data-driven synchronization. The D³-machine is an efficient and cost-effective design that combines the advantages of the data-driven synchronization with those of Instruction Level Parallelism (ILP). Two major design ideas are utilized by the proposed model: asynchronous execution of synchronization and computation operations and multithreaded graphs with variable resolution. The guiding principle in the generation of the threads is to fully exploit the ILP capabilities of the target processor. The entire dynamic Data-Flow (DF) graph is mapped by a one-to-one function onto the virtual space of the machine. Thus, the traditional DF graph operations (synchronization) of token matching and token formatting/routing are reduced into memory access operations. This allows us to utilize the dynamic DF principles, that exploit ultimate parallelism, for thread scheduling at a hardware minimal cost. With a combination of deterministic and stochastic simulation experiments is shown that the D³-machine has the necessary attributes for efficient parallel processing; it can tolerate long latencies, exploit parallelism, and also benefit from locality. Furthermore, by decoupling the synchronization portion of a thread from the computation, the D³-machine effectively neutralizes the overhead associated with dynamic DF scheduling.

Introduction

Multithreading has emerged as one of the most promising and exciting approaches for the exploitation of parallelism. It utilizes techniques developed in several independent research directions such as Data-Flow (DF), RISC, compilation for Instruction Level parallelism (ILP) and dynamic resource management [1]. Multithreading provides hardware support for more than one concurrent program counter and has the ability to switch with some efficiency among them [2]. A thread of control is very similar to the notion of process from multiprogramming. The main difference is that a thread in a multithreaded machine is visible at the architecture level [3].

The Decoupled Data-Driven machine D³-machine is a multithreaded architecture that employs data-driven sequencing based on the Decoupled Data-Driven (D³) model [4]. A thread can be scheduled for execution only after all its input data have been produced and arrived to the level of the memory hierarchy that is closer to the execution unit. When a thread is scheduled for execution in the D³-Machine it will proceed without interruption. The D³ model of execution decouples (separates) each thread (actor¹) into two parts: the graph (or synchronization) portion and the computation portion. The computation portion of each actor is a collection of conventional instructions (load/store, add, etc). The graph portion contains information about the executability of the actor and its consumers. Thus, a D³ graph can be viewed as a partially ordered conventional program with a data-dependency graph superimposed on it. The graph synchronization is handled by the Data-Flow Graph Engine (DFGE) whereas computation instruction is executed by the Computation Engine (CE). The two engines execute in a decoupled i.e., asynchronous mode. This helps to reduce the length of the processors' critical path.

The abstract model of execution of the D³-Machine has as its starting point the dynamic DF graphs that are based on the U-Interpret model [5]. The basic unit of computation, however, is the thread (macro-actor in the DF terminology) and not the instruction. We have developed extensions to the basic model, such as hierarchical matching and variable resolution support, that make it more suitable for Multithreaded execution. The entire dynamic DF graph is mapped on the logical space (virtual-memory) of the machine. Therefore, for each instantiation of a thread, we know at execution time were its data will reside by using the standard virtual memory translation techniques. For each thread there is a synchronization point, in the logical space, that keeps track of the number of inputs that have been produced. Therefore, all the DF synchronization operation can be implemented by references/updates to the logical space. These references are mapped into the physical space using conventional virtual space mapping techniques.

The computation elements of each PE of the D³-machine a high performance microprocessor support some form of ILP. However, in order to achieve efficient parallel processing it is not sufficient to just connect a large number of von Neumann processors; the latency and synchronization issues [6] must also be resolved. The D³-machine tolerates long latency and synchronization costs by employing data-driven thread scheduling. Furthermore, it exploits locality by varying the length of the threads to match the machine and program characteristics for best performance.

Several deterministic and stochastic simulation experiments have been conducted in order to assess the D³-machine's ability to tolerate latency, and exploit parallelism and locality. The results, presented in this paper, indicated that the D³-machine can indeed tolerate long latencies. For example increasing the communication latency from 1 to 5 cycles results to an increase in execution time of no more than 25%; going from 5 to 15 cycles increases the execution time by 37%. Furthermore, it does exploit locality: the simulation results have indicated that increasing thread length (TL) does indeed reduce execution time. Finally the experiments have shown that the D³-machine for the most part neutralizes the overhead associated with the data-driven synchronization: a fivefold increase in the processing time per thread in the DFGE resulted in an increase of the overall execution time of no more than 15%.

In summary, the main features of the D³-machine include its ability to:

• Tolerate long latencies and synchronization delays by using the dynamic DF principles of execution for thread scheduling and synchronization.

• Neutralize the overhead associated with dynamic DF scheduling by decoupling the synchronization portion of a thread from the computation portion.

• Keep the computation portion of the processor architecture close to the sequential machines. This enables the use of state-of-the-art microprocessors and compilation technology in the CE design.

• Employs threads that fully exploit the internal architecture through the use of variable resolution threads.

The overall goal of this paper is to demonstrate the potential of the D³-machine through simulation experiments. In Section 2, the variable-resolution graphs are introduced. The basic framework of the D³ model is presented in Section 3. Section 4 presents the decoupled architecture and addresses various implementation issues. Analytical and simulated performance analysis is presented in Section 5. Concluding remarks are presented in Section 6.