Automatic cache partitioning method for high-level synthesis

Abstract

Existing algorithms can be automatically translated from software to hardware using High-Level Synthesis (HLS), allowing for quick prototyping or deployment of embedded designs. High-level software is written with a single main memory in mind, whereas hardware designs can take advantage of many parallel memories. The translation and optimization of memory usage, and the generation of resulting architectures, is important for high-performance designs. Tools provide optimizations on memory structures targeting data reuse and partitioning, but generally these are applied separately for a given object in memory. Memory access that cannot be effectively optimized is serialized to the memory, hindering any further parallelization of the surrounding generated hardware.

In this work, we present an automated optimization method for creating custom cache memory architectures for HLS generated designs. Our optimization uses runtime profiling data, and is performed at a localized scope. This method combines data reuse savings and memory partitioning to further increase the potential parallelism and alleviate the serialized memory access, increasing performance. Comparisons are made against architectures without this optimization, and against other HLS caching approaches. Results are presented showing this method requires 72% of the number of execution cycles compared to a single-cache design, and 31% compared to designs with no caches.

Introduction

In recent years, the performance requirements for embedded systems have grown significantly. A standard embedded system using a microprocessor often cannot keep up with the requirements of high-performance applications, including many parallel tasks, increased data throughput, and computation power. Many developers are turning to reconfigurable hardware to speed up their designs. An architecture that is created specifically for the task at hand will always outperform a general purpose architecture.

Field-Programmable Gate Arrays (FPGAs) are becoming more popular in the area of embedded systems, which allow for the creation of special-purpose digital circuits for applications. They can be used in a wide range of applications, and have shown to be beneficial for use in high-performance systems while also reducing power consumption.

Designing a high-performance embedded system on an FPGA can be difficult, whether it is created from scratch, or migrated/hybridized from an existing system. HLS tools allow for the automatic translation of existing software algorithms into a Register-Transfer Level (RTL) hardware description, which can then be synthesized to FPGAs or Application-Specific Integrated Circuits (ASICs). Hardware design time can be significantly shortened by using this tool flow. This gives developers the ability to rapidly prototype new systems, or easily utilize legacy code in a high-level language, with C/C++ being the most common.

In the automatic generation of FPGA hardware from software, the memory architecture is a key component, and a prime focus for optimization. Software is written under the assumption of a single monolithic memory. FPGAs on the other hand, are designed with a distributed network of on-chip memory resources, Block Random-Access Memory (BRAM). HLS tools must be intelligent when translating software into hardware in order to best utilize the distributed memory resources. With most designs, memory is the bottleneck, and the ability for tools to effectively distribute and optimize the memory architecture is critical to achieve high-performance requirements.

HLS tools offer a wide range of optimizations that are possible when translating a design from software to hardware. Typically an extensive knowledge of hardware is not needed, but some is required to understand how the specific tool works and what abilities are available to the end user. There are usually opportunities for the user to get automatic benefits through various optimizations. Tools do a relatively good job of this on their own, however, user directives are available and typically required to perform further optimizations.

In general, maximizing the throughput and speed of the design can be achieved by increasing parallelism. With regard to memory access, throughput can be increased by reducing the access time to resources, and parallelizing multiple operations to allow simultaneous access. Memory operations accessing the same memory port must wait for one another, and their execution becomes serialized. We classify this to be the memory access serialization problem, where further parallelization of a design is hindered by the serial access requirement of a memory resource. Reducing the individual access times or executing multiple instructions in parallel allows for a overall reduction in computation time.

Common HLS optimizations such as loop unrolling and loop pipelining further accentuate the access serialization problem. Loop unrolling is the act of duplicating the body of the loop in exchange for a reduction in the number of loop iterations. By copying the loop, the number of memory operations doubles, and typically the accesses will be incremented versions of the previous iteration (e.g. a[i], a[i+1]). Loop iterations can be parallelized but access to memory must be serialized. Loop pipelining is where multiple iterations of a loop are pipelined such that they overlap in execution. With both of these optimizations potential parallelism is limited by the serial requirement for access to memory.

This work aims to address this problem with respect to HLS designs. We present an automated approach for creating multiple localized and partitioned caches based on memory access patterns of the chosen application. Commercial tools, such as Vivado HLS [1], do not currently support caching architectures for HLS designs, and there is limited research on the topic. This method combines data reuse and partitioning techniques (explained in Section 2), supports both on-chip and off-chip memory, operates on both global and local scopes of an application, and utilizes many analytical techniques.

The remainder of this paper is organized as follows. Section 2 outlines some existing work. Section 3 describes the HLS toolchain used to implement this work. Section 4 presents the method for this work. Section 5 gives results for this method showing a significant speedup in performance, and reduction in execution latency. Finally, a conclusion and future work are provided in Section 6.