Instruction-throughput regulation in computer processors with data-center applications

Abstract

This paper tests a recently-proposed technique for regulating output performance of Discrete Event Dynamic Systems and Stochastic Hybrid Systems. The controller is based on an integrator with a variable gain, adjusted so as to guarantee wide stability margins of the closed-loop system. The gain is adjusted by estimating, in real time, the derivative of the plant function via approximations to its IPA derivative. The technique is robust to computational errors in the loop, and hence these approximations are designed for fast computation rather than precision. The development of the regulation technique has been motivated by applications in computer processors, and extensively tested in the past on a cycle-level, full system simulator. In this paper we describe implementations of the regulator on an Intel machine based on the Haswell processor, and apply it to control the instructions’ throughput of various industry program-benchmarks as well as data-center applications.

Appendix

This section provides a quantitative description of the instruction-flow in the OOO-cache high-level model described at the beginning of Section III.

Denote by I _i , i = 1, 2, … , the instructions arriving at the instruction queue in increasing order. Let u denote the clock rate, or frequency, and let τ := u ⁻¹ be the clock cycle. Denote by a _i (τ) the arrival time of I _i relative to the arrival time of I ₁, namely a ₁(0) := 0, and let ξ _i be the clock counter at which I _i arrives. Then, a _i (τ) = ξ _i τ. Denote by α _i (τ) the time at which execution of I _i starts, and let β _i (τ) denote the time at which execution of I _i ends.

We next describe a way to compute α _i (τ). Consider first the case were I _i is a computational instruction. If all of its required variables are available at its arrival time then α _i (τ) = a _i (τ) + τ. On the other hand, if I _i has to wait for such variables, let k(i) denote the index (counter) of the instruction last to provide such a variable, then α _i (τ) = β _k(i)(τ) + τ. Next, if I _i is a memory instruction, then α _i (τ) is the time it starts a cache access. If the memory queue is not full at time a _i (τ), then α _i (τ) = a _i (τ) + τ. On the other hand, if the memory queue is full at time a _i (τ), let ℓ(i) denote the index of the instruction at the head of the queue, then, α _i (τ) = β _ℓ(i)(τ) + τ.

To compute β _i (τ), consider first the case where I _i is a computational instruction. Let μ _i denote the number of clock cycles it takes to execute I _i . Then, β _i (τ) = α _i (τ) + μ _i τ. On the other hand, if I _i is a memory instruction, let ν _i denote the number of clock cycles it takes to perform a cache attempt. If the cache attempt is successful and the variable is found in cache, then β _i (τ) = α _i (τ) + ν _i τ. If the variable is not in cache, the instruction is directed to the memory queue. Its transfer there involves a small number of clock cycles, m _i , hence it arrives at the queue at time α _i (τ) + ν _i τ + m _i τ. The memory queue is a FIFO queue whose service time represents an external-memory access, which is independent of the core’s clock. Denote by S _i the sojourn time of I _i at the memory queue. Then β _i (τ) = α _i (τ) + ν _i τ + m _i τ + S _i + τ.

Finally, the departure time of I _i from the instruction queue, denoted by d _i (τ), is $d_{i}(\tau )=\max$ $\left \{\beta _{i}(\tau ),d_{i-1}(\tau )\right \}+\tau$. Given a control cycle consisting of N instructions, the throughput is defined as N/d _N (τ). Since u = τ ⁻¹, we can view the throughput as a function of u and denote it by y(u). A more detailed discussion of the model can be found in Wardi et al. (2016).

Concerning the IPA derivative $\frac {\partial y}{\partial u}$, Ref. Wardi et al. (2016) has described a recursive algorithm for its computation in real time, and that algorithm was used in the simulations described in Section 3. It is based on the facts that y = N/d _N (u) and u = 1/τ which imply (after some algebra) that

$$ \frac{\partial y}{\partial u}=\frac{1}{N}\left( \frac{y}{u}\right)^{2} \frac{\partial d_{N}}{\partial\tau}. $$

(16)

Assuming that y is measured in real time, it remains to compute the term $\frac {\partial d_{N}}{\partial \tau }$ in Eq. 16. This can be done by a recursive procedure for real-time computations of the terms $\frac {\partial d_{i}}{\partial \tau }$, i = 1, 2, … , N, as described in detail in Wardi et al. (2016), pp. 24–25.