Timing optimization via nest-loop pipelining considering code size

Abstract

Embedded systems have strict timing and code size requirements. Software pipelining is one of the most important optimization techniques to improve the execution time of loops by increasing the parallelism among successive loop iterations. However, there is no effective techniques exist for solving the software pipelining problem on nested loops. The existing software pipelining techniques for single loops can only explore the parallelism of the innermost loop, so the final timing performance is inferior. While multi-dimensional (MD) retiming can explore the outer loop parallelism, it introduces large overheads in loop index generation and code size due to loop transformation. In this paper, we show how the computation time and code size of a pipelined nested loop is affected by execution sequence and retiming, assuming there is no loop unfolding. We present the theory of Software PIpelining for NEsted loops (SPINE) to reveal the relationship among the computation time of an iteration, the execution sequence, and the software pipelining degree of a nested loop using retiming concepts. Two algorithms of Software PIpelining for NEsted loops (SPINE) are proposed based on the fundamental understanding of the properties of software pipelining for nested loops: the SPINE-FULL algorithm generates fully parallelized loops with the minimal overheads. The SPINE-ROW-WISE algorithm achieves the maximal parallelism in an iteration with a fixed row-wise execution sequence. Therefore, the overheads due to loop transformation are minimal. Our technique can be directly applied to imperfect nested loops. The experimental results show that the average improvement on the execution time of the pipelined loop generated by SPINE is 71.7% compared with that generated by the standard software pipelining technique. The average code size is reduced by 69.5% compared with that generated by the MD retiming technique.

Introduction

Embedded systems usually have stringent requirements in timing and code size. With the advance of the technology, embedded systems with multiple cores or VLIW-like architectures, such as TI’s TMS320C6x, Philips’ TriMedia, and IA64, etc., become necessary to achieve the required high performance for the applications with growing complexity. To reduce the execution time of loops, software pipelining is widely used to explore the instruction-level parallelism in a loop by parallelizing the execution of successive iterations [8], [15]. However, software pipelining can dramatically expand the code size by adding code sections in prologue and epilogue¹ [17], [21]. Code size is one of the most critical concerns for many embedded processors because the capacity of on-chip memory modules is still very limited due to the chip size, cost and power considerations. The designers try their best to fit the code into the small on-chip memory to avoid slow (external) memory accesses. If the software-pipelined code cannot be fit into on-chip memory, a designer, without proper techniques, may have to give up using software pipelining, resulting in a design with a deteriorated timing performance. This awkward situation still exists in leading industry such as Texas Instruments [7] because there is no effective design tools to consider the code size issue along with timing optimization. Therefore, loop optimization with both timing and code size requirements becomes a great challenge for embedded system design.

Timing and code size issues are even greater concerns for software pipelining on nested loops. Unlike software pipelining of single loops which has been extensively studied and implemented [8], [15], [3], [21], [13], very few work has been done for the software pipelining problem on nested loops. A few existing techniques that could be applied to nested loop optimization either cannot fully explore the parallelism in a nested loop or do not consider the overheads such as loop index and loop bounds computation, and code size expansion due to transformation. To the authors’ knowledge, there is no existent technique that can effectively solve the software pipelining problem for nested loops in embedded systems.

Software pipelining for single loops focuses on one-dimensional problems. When applied to nested loops, it only optimizes the innermost loop [8], [15], [3], [1], [2]. While nested loops usually exhibit dependencies cross loop dimensions. They provide abundant opportunities to increase the parallelism in an iteration. For example, the execution time of the Floyd-Steinberg algorithm generated by the modulo scheduling [15] is 25,000 time units according to our experimental results, while it is only 2950 time units when generated by our Software PIpelining for NEsted loops (SPINE) technique. The improvement on the loop execution time is 88.2%. It indicates that a lot of potential parallelism cannot be explored by software pipelining for single loops. Therefore, the performance improvement that can be obtained by the standards software pipelining techniques is very limited.

Another technique called hyperplane scheduling [6], [14] tries to convert a nested loop into a single loop using loop unrolling and skewing to reduce the execution time. However, this technique makes code generation extremely difficult, and results in large overheads computation time and code size due to loop transformation. Data locality is also disrupted as a result of loop skewing. The best effort existing in industry on nested loop pipelining is to overlap the executions of the prologue and epilogue of the innermost loop, called outer loop pipelining [10], [19]. In this method, the dependencies among the outer loop iterations are still not exploited. Hence, the potential parallelism that can be explored is very limited.

The only existing method that can fully explore the potential parallelism in multi-dimensional problems is multi-dimensional (MD) retiming [12]. MD retiming can achieve full parallelism of an multi-dimensional problem with polynomial time algorithm. That is, all the computations in an MD problem can be executed in parallel. MD retiming techniques can be effectively applied to high-level synthesis. However, MD retiming does not consider some critical issues for loop optimization, such as loop index generation and code size. We found that the regular row-wise execution sequence, which is implemented in most nested loops, can be altered unnecessarily by using MD retiming technique. Loop transformation needs to be performed to compute new loop indexes and loop bounds due to a skewed execution sequence [12], [20]. Therefore, large code size and computation overhead are introduced, and data locality is also disrupted. According to our experimental results, the code size of Floyd-Steinberg algorithm generated by MD retiming is 1646 instructions, while it is only 169 instructions using our SPINE technique. The code size is reduced by 89.7%. The execution time is also reduced by 31.6%. Although MD retiming can benefit high-level synthesis with specialized hardware support, it is not suitable for software pipelining on nested loops.

The following example shows that a skewed execution sequence significantly affects the performance and code size of the generated code. Fig. 1a shows the original code of a nested loop. Fig. 1c shows the pipelined loop generated by the standard MD retiming technique [12]. It uses a diagonal execution sequence to achieve full parallelism. The code size grows dramatically not only because prologue and epilogue sections are produced in both loop levels, but also because extra codes are required to compute the new loop bounds and loop indexes. These extra computations deteriorate the performance and code quality of the final code. Due to the space limitation, we cannot show the whole piece of the program. Fig. 1b shows the software-pipelined code generated by our SPINE algorithm using row-wise execution. The loop is fully parallelized. Assuming that each computation in the loop body can be executed in one time unit, then, the execution time of one iteration of the pipelined loop is just one time unit. Only the innermost loop has prologue and epilogue. The code size can be further reduced by directly applying the code size reduction technique presented in [21].

In our research, multi-dimensional retiming framework is used to model the software pipelining on nested loops. In the SPINE theory, we show how schedule vector and retiming affect the computation time of an iteration for nested loops assuming that loops are not unfolded [11], [5]. Based on the SPINE theory, we show how to achieve maximum parallelism in an iteration while keeping the overheads minimal using a row-wise execution sequence whenever it is possible. Then, more efficient SPINE algorithms are developed to optimize nested loops considering various design requests. We make our contributions as follows:

• We use multi-dimensional retiming concept to effectively model the software pipelining problem on nested loops (Section 2.2).

• We present the theory of software pipelining for nested loops to reveal the relationship among the computation time of a loop iteration, the execution sequence, and the software pipelining degree (Section 3).

• We prove that the minimum computation time of an iteration for a nested loop with given execution sequence can be achieved by using a retiming vector that is orthogonal to the execution sequence (Theorem 3.4).

• We develop two algorithms for Software PIpelining for NEsted loops (SPINE) technique (Section 4):

  • The SPINE-FULL algorithm fully parallelizes a nested loop with the minimal overheads.

  • The SPINE-ROW-WISE algorithm achieves the maximal parallelism in an iteration with a fixed row-wise execution sequence. Therefore, the overheads due to loop transformation are minimal.

• Our technique can be directly applied to loops with codes between loop levels (imperfect nested loops) (Section 4).

We conduct experiments on a set of two-dimensional benchmarks to compare the code quality generated by SPINE with that generated by the standard software pipelining, and MD retiming. Our experimental results show that SPINE out-performs or ties both of the other two techniques on all of our benchmarks. The average improvement on the execution time of the pipelined loop generated by SPINE is 71.7% compared with that generated by the standard software pipelining technique, such as modulo scheduling. The average code size is reduced by 69.5% compared with that generated by MD retiming. Based on the result of this paper, the future research can be extended to other optimization objectives of nested loops, such as low power scheduling, address register allocation, etc.

The rest of the paper is organized as follows: Section 2 gives an overview for the graph representation of nested loops and multi-dimensional retiming model. A brief discussion on the lower bound of the computation time of a nested loop iteration is also provided in this section. Section 3 presents the theory of Software Pipelining for NEsted loops (SPINE). The SPINE algorithms and an illustrative example are presented in Section 4. We also show that our technique can be applied to imperfect nested loops. Section 5 shows our experimental results on a set of 2D benchmarks. Finally, we conclude the paper in Section 6.