A configurable multiplex data transfer model for asynchronous and heterogeneous FPGA accelerators on single DMA device

Abstract

To reduce DMA utilization for multiple algorithm IPs on FPGA, a channel configurable and multiplex DMA device (CMDMA) is proposed for asynchronous and heterogeneous algorithm IPs. Firstly, we abstract the entities and data-flow in CMDMA system with a formal description for function definition and work-flow analysis. Then based on the functions and work-flow, we design and implement a prototype of CMDMA, which includes CMDMA software driver (SW) and hardware circuits (HW) of one DMA IP, a configurable input switch (CISwitch), algorithm IPs, and an asynchronous output switch (AOSwitch). The configurable function of CMDMA is implemented by CISwitch through a configuration port in HW-level, and a configurable Round-Robin (CRR) algorithm is proposed to implement channel and input data schedule in SW-level. For output, a channel distinguishable output buffer (ChnDistBuf) is proposed, which is able to deliver channel ID and data size to SW earlier than the end time of an algorithm IP. With a double interrupt coordination method of both ChnDistBuf and algorithm IPs, CMDMA is able to successively store complete output data from different algorithm IPs. With a double interrupt coordination method of both ChnDistBuf and algorithm IPs, CMDMA is able to successively store complete output data from different algorithm IPs. The experiments based on 4 heterogeneous matrix multiplication algorithm IPs on Xilinx Zynq platform show that CMDMA is able to improve about 8%-29% average algorithm acceleration rates on single algorithm IP compared to the exclusive method that one DMA works for one algorithm IP only, and it is able to increase about 10–40 MB/s and 5–15 MB/s of DMA input and output data throughput with multiple algorithm IPs running in parallel. Moreover, the extended LUT and FF resources in CMDMA are 756 and 1219, both of which are about 1% of Zynq platform. Besides, in a double CNN algorithm IPs test on Mnist application, an enhanced function of data broadcasting in CMDMA is able to improve 4 s than the system with 4 exclusive DMA running in parallel, meanwhile reduce 3 DMA utilization and 0.03 W power consumption.

Introduction

The key challenge of data movement in a System-on-a-Chip (SoC) is the data transfer method from a peripheral or accelerated IP to a memory subsystem [1]. While small on-chip data movement can be accomplished using software instructions, larger data transfers are usually implemented with special data transfer resources, e.g. DMA (Direct Memory Access). DMA is an efficient data transfer device in both computer and embedded systems, which is able to load/store large size data without involving CPU systems. Generally, in a FPGA based embedded system, DMA is often integrated as a software IP core that is implemented with logic resources in FPGA, and there is only one pair of data input and output channels in one DMA device. As the resources in FPGA are precious and limited, designers always employed DMA for larger size and higher real-time data transfers in accelerated algorithm IPs or interfaces. Kidav et al. [2] proposed a cycle stealing DMA, which is to load large precomputed delay values from external flash for Array Signal Processor. Hossein et al. [3] use one DMA device for high performance data transfers of PCIe Interface design on FPGA. Rota et al. [4] employ two DMA devices to transmit and receive data in a pair of circularly in and out buffers for a PCIe core on FPGA. Kim et al. [5], use multiple DMA devices independently for movement of video stream data in different stages of object and event tracking algorithms on FPGA. For algorithm IPs with compute-intensive algorithms on FPGA, e.g. CNN, the DMA device is usually exclusively owned by one algorithm IP for large size data transfers from input and output buffers, to decrease algorithm total running time [6], [7], [8], [9], [10]. Above all, the DMA devices are used in an exclusive method that one DMA IP only works for one host IP. However, with the exclusive method, in an embedded system with multiple algorithm IPs, there need be multiple DMA IPs for data transfers in these algorithm IPs. Moreover, it may be uneconomic in the exclusive method as the DMA may be idle and wait a long time when there is no input or output data transfers in the only one host IP.

Multiplex method is a commonly and effectively used method to extend single channel to multiple in various system design [11], [12], [13], [14], [15]. In a multiplexed design, the difficulties mainly lie in the asynchronous scheduling mechanism [16], and it is hard to share transferring data between heterogeneous targets [17]. Generally, for the controlling of multiple data transfers, an arbitrator is integrated inside the multiplexer switch for multiplex channel arbitration [18], and the control algorithm of the arbitrator can be static, e.g. first-come-first-serve (FCFS) [19], and Round-Robin (RR) [20,21], or dynamic with data map tables [22,23]. However, the static arbitrator is deficient in flexibility of channel schedule and in a dynamic arbitrator algorithm, the data map tables inside the multiplexer cost more logic and memory resources, which may be uneconomical for resource limited FPGA platforms.

Another method is to enlarge cache buffer sizes of a DMA device [24], [25], [26], [27], [28]. In this method, to implement multiple channel data transfers, usually a parallel buffer, or buffer with cache algorithm [25], is needed to keep data coherence in DDR and FPGA, and the buffer size is larger, yet the data throughput is lower than actual transfer data, as some additional router information needs to be packaged to transfer data packets, e.g. target node ID for sending direction, timestamp for sending sequence, etc. [27]. More importantly, the storage of multiple temporary data packets outside the multiplexer switch consumes significant memory resources, which may be unnecessary for a multiplex DMA design on FPGA, as the DMA device is able to access DDR and send the data in DDR to an algorithm IP directly.

Unlike above methods, in this paper, a configurable multiplex data transfer model for asynchronous and heterogeneous FPGA accelerators on a single DMA device (CMDMA) is proposed, which is able to extend a single DMA device to multiplex to reduce DMA utilization in a multiple algorithm IP system. CMDMA is designed as a multiplexed data transmission device with a software and hardware co-design method, that the input data are configured by software application and scheduled by CMDMA software driver, while the output is scheduled by hardware with the channel information carried in output data sent by each algorithm IPs. The contributions of this paper are listed as follows.

• A method of multiplex data transfers on a single DMA device is proposed, i.e., CMDMA, which is configurable for multiplex data transfers in asynchronous and heterogeneous algorithm IPs. A special work in CMDMA is that it supports the configuration of data broadcasting to share transferring data to multiple channels at the same time.

• A configurable multiplex data transfer model, i.e. CMmodel, is proposed, which abstracts entities and data-flow in a CMDMA system as vectors of components and vectors of configuration parameters and time events. CMmodel helps to define functions of each element in CMDMA and analyze data-flow between different entities in CMDMA system.

• A configurable input switch (CISwitch) is proposed for multiplexed input data transfers in CMDMA. CISwitch is a multiplexed switch and the data transferring channels in CISwitch is configurable through a configuration port. CISwitch makes the input scheduling method in CMDMA customize by the user in software.

• A configurable Round-Robin algorithm (CRR) is proposed for input data schedule in CMDMA software driver. CRR is a two-dimensional schedule algorithm for multiple algorithm IPs with multiple data queues. For algorithm IPs, the IP number and priority are configurable, and for data queues in each algorithm IPs, the DMA parameters and data transfer channels are configurable based on CISwitch.

• A channel distinguishable buffer (ChnDistBuf) is proposed in CMDMA. The innovation in ChnDistBuf is that it is able to deliver channel ID and output data size to CMDMA software driver earlier than the real end time of an algorithm IP, which makes CMDMA successively store complete output data from different algorithm IPs, meanwhile improve single algorithm IP performance and DMA data throughput.

• A prototype of CMDMA with four multiplex channels is implemented. To test the prototype and measure resource consumption and performance of CMDMA, one system with four heterogeneous matrix multiplication algorithm IPs and one system with double CNN IPs for Mnist application are implemented on Zynq platform.

The rest of this paper is organized as follows. Section II introduces the base DMA core and some of its extensions. Section III presents the function definition and work flow of each elements in CMDMA with a formal description of CMDMA system. Section IV describes the implementation of CMDMA. Section V shows the experimental results, and Section VI concludes this paper.