BOOM-Explorer: RISC-V BOOM Microarchitecture Design Space Exploration

2.1 RISC-V BOOM Core

Figure 2 illustrates the overall pipeline organization of BOOM [12, 13, 14]. BOOM is mainly composed of a front end (FrontEnd), an instruction decoding unit (IDU), an execution unit (EU), and a load-store unit (LSU). FrontEnd fetches instructions from I-cache, predicting branch target addresses, handling return addresses, and packing consecutive instructions as a fetch packet to the fetch buffer. IDU decodes instructions retrieved from the fetch buffer as micro-ops and dispatches, schedules, and issues them according to instruction types. EU integrates various functional units, including dividers, multipliers, accelerator interfaces, and so on. LSU interacts with EU and D-cache, deciding when to fire memory instructions to D-cache. BOOM implements explicit renaming logics, short-forwards branch optimizations, and integrates a two-level branch predictor [30] to improve its overall performance. With high-level hardware description language like Chisel [31], many components and their connections can be parameterized to support various BOOM microarchitectures. For example, the parameterization of the size of the branch target buffer, RAS, branch prediction history tables, and so on, broadens BOOM’s potential to balance the performance and power dissipation in FrontEnd. We can achieve divergent tradeoffs by adjusting these parameters to meet design requirements involving low-power and embedded computation applications.

Fig. 2.

View Figure

A microarchitecture design space of BOOM is constructed according to BOOM’s overall architecture, as listed in Table 1. The design space of each module is composed of various components, the structure of which can affect the performance power tradeoff and deserve to be optimized. As shown in Table 1, different entries of RAS (RasEntry), branch counters (BranchCount), organization of I-cache/D-cache (associativity, block width, and size), translation lookaside buffer (TLB) structures are considered in this article. The column of “Candidate values” in Table 1 denotes supported hardware resources. For example, the reorder buffer is provided to support 32, 64, or 96 entries, and so on. Performance and power are negatively correlated. Assigning more hardware resources often improves performance and brings considerable power dissipation. Hence, a good microarchitecture demands an appropriate compromise across all components’ hardware resources.

Some combinations in Table 1 are illegal. A legal combination should observe the constraints of BOOM design specifications as in Table 2. Otherwise, it would fail to generate reasonable RTL designs. For example, DecodeWidth defines the maximal instructions to be decoded simultaneously. If a BOOM microarchitecture breaks rule 2 in Table 2, then the reorder buffer may not reserve enough entries for each decoded instruction or contain redundant entries that we cannot fully utilize. The last three rules in Table 2 are added to simplify the design space. Their incorporations do not affect the design of a DSE algorithm. After we prune the design space w.r.t. rules in Table 2, the size of the legal microarchitecture design space is approximately \(1.6 \times 10^8\).

2.2 Bayesian Optimization

CAS offers advantages in terms of fast performance estimations compared to the VLSI flow mentioned earlier in Section 1. Thus, it becomes convenient for previous approaches to construct extensive training datasets using CAS [18, 21]. These datasets allow black-box models to learn the relationships between microarchitectures and their corresponding performance or power values. DSE is then conducted by sweeping the design space using these trained black-box models. However, given the benefits of DSE at the RTL level instead of using CAS (see Figure 1), such a new solution should aim to reduce the call for the expensive VLSI flow effectively. We find that the inherent structure of the problem fits Bayesian optimization well. So, we present the corresponding preliminary in this section.

Bayesian optimization [32, 33] is widely applied in such optimization problems when an evaluation flow is expensive, as shown in Equation (1), where we use the minimization as an example. (1) \(\begin{equation} x^* = \mathop {\arg }\mathop {\min }_{x \in \mathcal {X}}f(x), \end{equation}\) where \(\mathcal {X}\) is the solution space (design space), and f represents the evaluation flow, which maps x to a metric value. The optimal solution \(x^*\) attains the minimal value of \(f(x)\). The main idea of Bayesian optimization is only to select the potentially promising microarchitectures for evaluation using the VLSI flow. These solutions are predicted promising based on what we have known in previous searched samples. A distinction between DSE using Bayesian optimization and previous black-box methods is the absence of the need to construct a large dataset for training a black-box model. We instead progressively explore the design space in an online fashion.

Bayesian optimization consists of a surrogate model and an acquisition function. A surrogate model is often constructed from the Gaussian process (GP) [34]. It models f in Equation (1) as a generated probability distribution from the GP. The acquisition function is designed to characterize the relative rankings between different solutions based on the probability distribution. In other words, the acquisition function answers whether \(x_1\) is better than \(x_2\) without considering exactly how much better \(x_1\) performs than \(x_2\). The solution that achieves the acquisition function’s optimum is selected as a potential promising/optimal solution. The potential optimal solutions are then evaluated using the VLSI flow and their corresponding performance and power values are utilized to tune the surrogate model. Better solutions are expected to be explored using the tuned surrogate model with a more accurate modeled probability distribution. Suppose \(\boldsymbol {x}\) denotes a microarchitecture embedding (Section 3), \(\tau\) denotes the current explored best performance/power value. GP characterizes the evaluation flow with the mean \(\mu (\boldsymbol {x})\) and the covariance \(\sigma ^2(\boldsymbol {x})\). The expected improvement (EI), a popular acquisition function, is shown in Equation (2). (2) \(\begin{equation} \begin{aligned}\text{EI}(\boldsymbol {x}) &= \mathbb {E}[\min (f(\boldsymbol {x}) - \tau , 0)] \\ &= \mathbb {E}\left[\min \left(\frac{f(\boldsymbol {x}) - \mu (\boldsymbol {x}))}{\sqrt {\sigma ^2(\boldsymbol {x})}} - \frac{\tau - \mu (\boldsymbol {x})}{\sqrt {\sigma ^2(\boldsymbol {x})}}, 0\right)\right] \cdot \sqrt {\sigma ^2(\boldsymbol {x})} \\ &= \sigma (\boldsymbol {x})(\lambda \Phi (\boldsymbol {x}) + \phi (\boldsymbol {x})), \\ \lambda &= \frac{\tau - \xi - \mu (\boldsymbol {x})}{\sigma (\boldsymbol {x})}, \end{aligned} \end{equation}\) where \(\Phi (\boldsymbol {x})\) and \(\phi (\boldsymbol {x})\) are the cumulative distribution function and the probability density function of GP, and \(\xi\) is a coefficient to improve the numerical ability. Figure 3 visualizes the optimization procedure for maximizing \(f(x)\) with Bayesian optimization. A surrogate model with GP is built in the \(x - f(x)\) space from already-known samples represented by red dots. The red curve denotes the golden values (ground truths) for each input x, and the dashed blue curve denotes GP predictions. The band colored in orange shows the GP prediction uncertainty of each x. If the band is wider, then the uncertainty is larger. The potential optimal solution is selected according to EI. Namely, point \(x = 2\), which achieves the maximal EI, is chosen as a potential optimal solution, as shown in the \(x - \text{EI}\) space.

Fig. 3.

View Figure

4.1 Overview of BOOM-Explorer

Figure 4 shows an overview of BOOM-Explorer. First, the active learning algorithm MicroAL (Section 4.2) is adopted to sample a set of initial microarchitectures from the design space. Domain-specific knowledge is embedded in the initialization based on the first learned lesson. Second, a Gaussian process model with deep kernel learning (DKL-GP) (Section 4.3) is then built on the initial set as a surrogate model. Third, the expectation improvement of Pareto hypervolume (Section 4.4) is applied as the acquisition function. During the DSE procedure, BOOM-Explorer interacts with the VLSI flow to acquire performance and power values for the selected microarchitectures. Due to the customized algorithm flow, such interactions make the DSE at the RTL level feasible. Moreover, we improve the exploration with diversity-guided sampling to handle different sub-regions (Section 4.5). Finally, The outputs of BOOM-Explorer are the predicted Pareto frontier and corresponding Pareto-optimal microarchitectures.

Fig. 4.

View Figure

4.2 Microarchitecture-aware Active Learning Algorithm

The initial microarchitectures sampled from the design space are critical to constructing a surrogate model for later exploration. A naive solution is to sample microarchitecture embeddings randomly [18, 27]. Some advanced sampling techniques with statistic analysis like orthogonal design [21] are also applied in previous works. Nevertheless, the methods mentioned above are less effective, since they require many VLSI flow interactions while the runtime cost of the VLSI flow is much higher. We follow two guidelines to design the initialization algorithm. First, the initial microarchitecture embeddings should uniformly scatter in the entire design space. Different PPA tradeoffs should be captured via uniform scattering. Second, the diversity of these microarchitecture embeddings should represent different characteristics of the design space as much as possible. Simple characteristics can be described with a few samples, while complex ones can be described with more samples. Besides, combining with prior knowledge helps us remove samples unworthy to be evaluated with the VLSI flow.

In MicroAL, we first perform clustering w.r.t. the decode width (DecodeWidth), then we conduct transductive experimental design with samples per each cluster. DecodeWidth in IDU, as referred to in Table 1, decides the maximal numbers of instructions that can be decoded simultaneously, i.e., it determines the width of the pipeline, as shown in Figure 2. It allows more instruction execution parallelism in the pipeline if DecodeWidth is assigned a considerable value in BOOM, and we also allocate appropriate hardware resources to other components accordingly. Although large DecodeWidth leads to remarkable performance improvement, in most cases, power dissipation increases correspondingly due to more transistors integrated. We find that the impact of a PPA tradeoff by configuring DecodeWidth is significant.

Figure 5 visualizes the objective space by clustering w.r.t. DecodeWdith on sampled microarchitecture embeddings. Better microarchitectures will be closer to the original point in Figure 5, indicating higher performance and lower power dissipation. Figure 5 demonstrates that the distributions of clusters are highly correlated with the potential Pareto frontier. The entire design space is discrete and non-smooth. Nonetheless, many microarchitectures with the same DecodeWidth achieve similar performance-power characteristics within their clusters. We embed this domain knowledge in the initialization algorithm. Additionally, within each cluster, we adopt the transductive experimental design(TED) [35] to sample microarchitectures that best reflect the characteristics of their groups.

Fig. 5.

View Figure

The method of TED is a widely used algorithm to construct samples that deserve to be evaluated with an expensive flow. It tends to choose microarchitecture embeddings that can spread across the design space to retain the most information [35]. We can acquire a pool of representative samples with high mutual divergences by iteratively maximizing the trace of the distance matrix constructed on newly sampled and unsampled microarchitecture embeddings. Algorithm 1 shows the backbone of TED, where f represents the distance function used in computing the distance matrix \(\mathrm{K}\). It should be noted that there are no restrictions on the choice of distance functions. The microarchitecture embeddings can be clustered based on DecodeWidth, allowing us to identify clusters that are most closely related to the Pareto frontier. Within each cluster, TED is performed to create representative samples. This process leads to the formulation of MicroAL, as described in detail in Algorithm 2.

In Algorithm 2, first, we cluster the entire design space according to \(\Phi\), which is the distance function with a higher penalty along the dimension of DecodeWidth. One possible alternative can be \(\Phi = (\boldsymbol {x}_i - \boldsymbol {c}_j)^\top \boldsymbol {\Lambda }(\boldsymbol {x}_i - \boldsymbol {c}_j)\), with \(i \in \lbrace 1, \ldots , \vert \mathcal {U} \vert \rbrace\) and \(j \in \lbrace 1, \ldots , k\rbrace\), where \(\boldsymbol {\Lambda }\) is a pre-defined diagonal weight matrix. The procedure runs n rounds to make the design space sufficiently clustered. Second, we apply TED to each cluster to perform sampling, i.e., line 10 in Algorithm 2. Finally, the initial microarchitectures that are worth to be estimated with the VLSI flow are constructed. Each microarchitecture is estimated using the VLSI flow. Our acquisition function is built based on these microarchitectures and their respective performance and power values.

4.3 Gaussian Process with Deep Kernel Learning

It is common to build GP models for the initial microarchitectures due to the non-parametric approximation in terms of reliability in uncertainty estimation and robust performance for many applications [36, 37].

Without loss of generality, let \(\boldsymbol {X} = \lbrace \boldsymbol {x}_1, \boldsymbol {x}_2, \ldots \boldsymbol {x}_n\rbrace\) denote a set of microarchitecture embeddings. The corresponding objective values formulate a matrix: (4) \(\begin{equation} \begin{aligned}\boldsymbol {Y} = \begin{pmatrix} y_{11} & y_{12} & \dots & y_{1m} \\ y_{21} & y_{22} & \dots & y_{2m} \\ \vdots & \vdots & \ddots & \vdots \\ y_{n1} & y_{n2} & \dots & y_{nm} \end{pmatrix}, \end{aligned} \end{equation}\) where m denotes the number of objectives, and \(\boldsymbol {Y}\) is an \(n \times m\) matrix. The objective functions \(\lbrace f_1, f_2, \ldots , f_m\rbrace\), which characterize how we get the objective values, map an \(\boldsymbol {x}\) to \(\boldsymbol {y} = \lbrace y_1, y_2, \ldots , y_m \rbrace\). We place an appropriate GP prior on each f, i.e., \(f_i(\boldsymbol {x}) \sim \mathcal {G}\mathcal {P}(\mu _i, \sigma _i^2)\), where \(\mu _i\) and \(\sigma _i^2\) are the mean value function and the covariance function for the ith objective, respectively. A non-trivial problem is that the objectives in our problem are not orthogonal, since higher performance often comes with higher power dissipation. Characterizing individual objectives using independent GP models could degrade the overall modeling performance. Hence, we introduce multi-task GP models to handle the difficulty [38]. The main idea of multi-task GP is to model the hidden mappings based on the objective identities and the observed objective values for each microarchitecture. Objective identities are scalars utilized to distinguish different objectives. Hence, a GP prior is also placed between each objective function, as shown in Equation (5). (5) \(\begin{equation} \text{cov}(f_i(\boldsymbol {x}), f_j(\boldsymbol {x}^{\prime })) = \boldsymbol {K}_{ij}^f\boldsymbol {K}^x(\boldsymbol {x}, \boldsymbol {x}^{\prime }), \end{equation}\) where \(\text{cov}(f_i(\boldsymbol {x}), f_j(\boldsymbol {x}^{\prime }))\) is a covariance between \(f_i\) and \(f_j\) on two microarchitecture embeddings \(\boldsymbol {x}\) and \(\boldsymbol {x}^{\prime }\), \(\boldsymbol {K}_{ij}^f\) is a positive semi-definite matrix specifying the similarities between objective i and j, and \(\boldsymbol {K}^x\) is a covariance function over \(\boldsymbol {x}\) and \(\boldsymbol {x}^{\prime }\).

Given a newly sampled microarchitecture embedding \(\boldsymbol {x}^*\), the mean value representing the prediction of an objective value is shown in Equation (6). (6) \(\begin{equation} \mu _i(\boldsymbol {x}^*) = \mu _i + \left(\boldsymbol {K}_i^f \otimes \boldsymbol {K}^x(\boldsymbol {x}^*, \boldsymbol {X})\right)^\top \Sigma ^{-1}(\boldsymbol {Y}_i - \boldsymbol {\mu }_i), \end{equation}\) where \(\boldsymbol {K}_i^f\) is the ith column of \(\boldsymbol {K}^f\) defined in Equation (5), \(\boldsymbol {\mu }_i\) denote a vector of ith objective mean values, and \(\boldsymbol {Y}_i\) is the ith column of \(\boldsymbol {Y}\) denoting the ith objective values in the training data set. The uncertainty of the prediction \(\mu _i(\boldsymbol {x}^*)\) in Equation (6) is calculated by Equation (7). (7) \(\begin{equation} \begin{aligned}\sigma _i^2(\boldsymbol {x}^*) &= \left(\boldsymbol {K}_i^f \otimes \boldsymbol {K}^x(\boldsymbol {x}^*, \boldsymbol {x}^*)\right) - \\ &\left(\boldsymbol {K}_i^f \otimes \boldsymbol {K}^x(\boldsymbol {x}^*, \boldsymbol {X})\right)\Sigma ^\top \left(\boldsymbol {K}_i^f \otimes \boldsymbol {K}^x(\boldsymbol {X}, \boldsymbol {x}^*)\right)\!, \end{aligned} \end{equation}\) where \(\Sigma\) in Equations (6) and (7) is obtained from Equation (8). (8) \(\begin{equation} \Sigma = \left(\boldsymbol {K}_i^f \otimes \boldsymbol {K}^x(\boldsymbol {X}, \boldsymbol {X})\right) + \boldsymbol {D} \otimes \boldsymbol {I}. \end{equation}\) The operator \(\otimes\) denotes Kronecker product. In Equation (8), \(\boldsymbol {D}\) is an \(m \times m\) diagonal matrix with \(\sigma _k^2\) as the kth diagonal element, and \(\boldsymbol {I}\) is the identity matrix. Therefore, the posterior distribution is \(f_i(\boldsymbol {x}^* \mid \boldsymbol {X}, \boldsymbol {Y}_i, \boldsymbol {K}_i^f, \boldsymbol {K}^x) \sim \mathcal {G}\mathcal {P}(\mu _i(\boldsymbol {x}^*), \sigma _i^2(\boldsymbol {x}^*))\). The likelihood estimation of the multi-task GP is as shown in Equation (9). (9) \(\begin{equation} \begin{aligned}l &= -\frac{mn}{2}\log 2\pi - \frac{n}{2}\log |\boldsymbol {K}^f| - \frac{m}{2}\log |\boldsymbol {K}^x| \\ &- \frac{1}{2}\text{Tr}[(\boldsymbol {K}^f)^{-1}\Gamma ^\top (\boldsymbol {K}^x)^{-1}\Gamma ] \\ &- \frac{n}{2}\sum \limits _{i = 1}^m\log \sigma _i^2 - \frac{1}{2}\text{Tr}[(\boldsymbol {Y} - \Gamma)\boldsymbol {D}^{-1}(\boldsymbol {Y} - \Gamma)^\top ], \end{aligned} \end{equation}\) where \(\Gamma\) is a matrix of \(\mu _{ij}\) corresponding to \(\boldsymbol {Y}\). The parameters describing \(\boldsymbol {K}^f\) and \(\boldsymbol {K}^x\) are optimized via the maximization of Equation (9).

We use deep kernels [39] stacked by multiple linear perceptrons (MLP) with non-linear transformations to construct the Gaussian kernels, i.e., \(\boldsymbol {K}^f\) and \(\boldsymbol {K}^x\). We term our surrogate model DKL-GP due to the construction of the GP and deep kernel functions. The use of deep kernels can result in improved performance due to their strong modeling ability. Thus far, \(\boldsymbol {K}^f\) and \(\boldsymbol {K}^x\) are described by Equation (10). (10) \(\begin{equation} \boldsymbol {K} \rightarrow (\varphi (g(\boldsymbol {x}, \boldsymbol {\theta })), \varphi (g(\boldsymbol {x}^{\prime }, \boldsymbol {\theta }))), \end{equation}\) where \(\varphi\) denotes non-linear transformation functions, e.g., the ReLU function, and so on, and the MLP g is parameterized by \(\boldsymbol {\theta }\).

4.4 Correlated Multi-objective Optimization

Although DKL-GP, as demonstrated in Section 4.3, can be utilized to characterize the uncertainty of predicted clock cycles and power, designing a suitable acquisition function to find the Pareto frontier remains an unsolved problem. Since clock cycles and power are a pair of negatively correlated metrics, we introduce the expected improvement of Pareto hypervolume (EIPV) [40] to model the tradeoff in the performance-power space.

We denote the points in the performance-power space as \(\boldsymbol {Y} = \lbrace \boldsymbol {y}_1, \boldsymbol {y}_2, \ldots \boldsymbol {y}_n\rbrace\). As Figure 6 shows, a better performance and power tradeoff lies closer to the origin. Pareto hypervolume is the volume of the area bounded by the Pareto frontier and a reference point. The reference point is a self-defined point dominated by all objective values. We use \(\mathcal {P}(\boldsymbol {Y})\) to represent the Pareto frontier, i.e., \(\mathcal {P}(\boldsymbol {Y}) = \lbrace \boldsymbol {y}_i \in \boldsymbol {Y} \mid \boldsymbol {y}_j \nsucceq \boldsymbol {y}_i, \forall \boldsymbol {y}_j \in \boldsymbol {Y} \setminus \lbrace \boldsymbol {y}_i\rbrace \rbrace\). Given a reference point \(\boldsymbol {v}_{\text{ref}}\) in the objective space, which is dominated by \(\mathcal {P}(\boldsymbol {Y})\). The Pareto hypervolume [40] bounded by \(\boldsymbol {v}_{\text{ref}}\) and \(\mathcal {P}(\boldsymbol {Y})\), as the orange region highlighted in Figure 6(a), can be computed by Equation (11). (11) \(\begin{equation} \text{PVol}_{\boldsymbol {v}_{\text{ref}}}(\mathcal {P}(\boldsymbol {Y})) = \int _{\boldsymbol {Y}} {1\!\!1}[\boldsymbol {y} \succeq \boldsymbol {v}_{\text{ref}}]\left[1 - \prod _{\boldsymbol {y}_* \in \mathcal {P}(\boldsymbol {Y})}{1\!\!1}[\boldsymbol {y}_* \nsucceq \boldsymbol {y}]\right] \mathrm{d}\boldsymbol {y}, \end{equation}\) where \({1\!\!1}(\cdot)\) is the indicator function, which outputs one if its argument is true and zero otherwise. The integral characterized by Equation (11) sums up all bounded regions. Intuitively, if a new point \(\boldsymbol {y}_{n + 1}\) is searched out, and \(\boldsymbol {y}_{n + 1}\) is not dominated by any points in \(\boldsymbol {Y}\), then \(\text{PVol}_{\boldsymbol {v}_{\text{ref}}}(\mathcal {P}(\boldsymbol {Y} \cup \lbrace \boldsymbol {y}_{n + 1}\rbrace))\) is increased. The increased part is specified as the improvement of Pareto hypervolume. The larger the increased part, the better the improvement of Pareto hypervolume is. The EIPV is the expectation of the improvement w.r.t. potential solution candidates at the \((n + 1)\)-th optimization steps. Formally, the EIPV is computed as Equation (12). (12) \(\begin{equation} \text{EIPV}(\boldsymbol {x}_{n + 1} \mid \mathcal {D}) = \mathbb {E}_{p(f(\boldsymbol {x}_{n + 1}) \mid \mathcal {D})}[\text{PVol}_{\boldsymbol {v}_{\text{ref}}}(\mathcal {P}(\boldsymbol {Y}) \cup f(\boldsymbol {x}_{n + 1})) - \text{PVol}_{\boldsymbol {v}_{\text{ref}}}(\mathcal {P}(\boldsymbol {Y}))], \end{equation}\) where f is the DKL-GP mentioned in Section 4.3, \(\mathcal {D} =\lbrace \boldsymbol {x}_i, \boldsymbol {y}_i\rbrace _{i = 1}^n\) is the dataset, and \(\boldsymbol {x}_{n + 1}\) is a newly sampled microarchitecture embedding at the step \(n + 1\). Figure 6(b) visualizes the EIPV based on Figure 6(a), where the green region highlights \(\text{EIPV}(\boldsymbol {x}_{n + 1} \mid \mathcal {D})\).

Fig. 6.

View Figure

In the performance-power space, we can simplify Equation (12) to make EIPV better computable by decomposing the space as grid cells. Assume \(\boldsymbol {v}_\text{ref} = \lbrace (a_1^0, a_2^0)\rbrace\). The union of grid cells can be phrased as \(\mathcal {C} = [a_1^1, a_1^0) \times [a_2^1, a_2^0) \times [a_1^2, a_1^1) \times [a_2^2, a_2^1) \times \cdots [a_1^n), a_1^{n - 1} \times [a_2^n, a_2^{n - 1})\). Denote \({C}_\text{nd} = \lbrace C \in \mathcal {C} \mid \boldsymbol {y}^{\prime } \nsucceq \boldsymbol {y}, \forall \boldsymbol {y} \in \mathcal {C}, \boldsymbol {y}^{\prime } \in \mathcal {P}(\boldsymbol {Y}) \rbrace\) as non-dominated cells. Hence, the simplified version of EIPV computation is derived, as shown in Equation (13). (13) \(\begin{equation} \text{EIPV}(\boldsymbol {x}_{n + 1} \mid \mathcal {D}) = \sum \limits _{C \in {C}_{\text{nd}}} \int _C \text{PVol}_{\boldsymbol {v}_C}(\boldsymbol {y})p(\boldsymbol {y} \mid \mathcal {D})\mathrm{d}\boldsymbol {y}. \end{equation}\)

The relative relations between microarchitecture embeddings \(\boldsymbol {x}_1\) and \(\boldsymbol {x}_2\), i.e., whether \(\boldsymbol {x}_1\) is better than \(\boldsymbol {x}_2\), can be precisely described by EIPV. The microarchitecture embedding \(\boldsymbol {x}^*\), which achieves the maximal EIPV, explicitly demonstrates that \(f(\boldsymbol {x}^*)\) is the potential Pareto-optimal solution, as shown in Equation (14). (14) \(\begin{equation} \boldsymbol {x}^* = \mathop {\arg \max }\limits _{\boldsymbol {x} \in \mathcal {D}}\text{EIPV}(\boldsymbol {x} \mid \mathcal {D}). \end{equation}\) We sample \(\boldsymbol {x}^*\) each time and evaluate its performance and power values \(\boldsymbol {y}^*\) via the VLSI flow. If a generous time budget is available, then we can sweep the design space \(\mathcal {D}\) using DKL-GP to obtain \(\boldsymbol {x}^*\). The advantage of using DKL-GP is that it has low runtime, making the estimation highly efficient. The pair \((\boldsymbol {x}^*, \boldsymbol {y}^*)\) is added to \(\mathcal {D}\). And we utilize the aggregated \(\mathcal {D}\) to tune DKL-GP, hoping to sample the following \(\boldsymbol {x}^*\) with better \(\boldsymbol {y}^*\) that can dominate already explored points in the design space.

4.5 Diversity-guided Parallel Exploration

Despite the benefits from MicroAL and the negatively correlated multi-objective Bayesian optimization flow, we also propose diversity-guided parallel exploration to further improve the algorithm’s efficiency. With the technique, we can sample and evaluate more microarchitecture embeddings at the same time. And we also improve the overall DSE performance effectively.

Although exploring with EIPV is mostly effective, a limitation is also viewed. We notice that searching via EIPV can lead to local optimum, i.e., it cannot recover the complete Pareto frontier due to potential “outliers.” Outliers have good properties in performance and power tradeoff but may not have high EIPV. The reason behind this is that the Pareto frontier tends to group in various regions due to components impacting the performance and power tradeoff in various degrees. Some objective values with relatively higher EIPV can hide outliers when we do not have a large optimization budget. In other words, exploring with EIPV misses such outliers when insufficient optimization is applied.

Two methodologies can be integrated to explore those outliers with higher Pareto hypervolume. First, improve the number of samples while maintaining an unchanged optimization budget with batch optimization. Applying parallel VLSI estimations in the batch optimization instead of original sequential optimization is necessary. And the parallelism depends on the number of available EDA tools licenses. Second, embedding domain knowledge or heuristics in the exploration is a good supplement to the exploration with original acquisition design (discussed in Section 4.4). Consequently, we propose a parallel-based exploration by combining two pathways.

In the exploration, we select multiple microarchitectures simultaneously to conduct parallel estimations using the VLSI flow. Intuitively, a straightforward way to select microarchitectures can be achieved according to the ranking of EIPV w.r.t. each design, as introduced in Section 4.4. Conversely, we provide some prior knowledge in the selection. DecodeWidth (mentioned in Section 4.2) contributes more to the performance and power tradeoff than other components, since modifying DecodeWidth can lead to distinct clusters shown in Figure 5. Except for DecodeWidth, another component IntPhyRegister, determining the number of integer physical registers, can also distinctly affect performance and power if a benchmark contains considerable integer-related instructions. Hence, the Pareto frontier is scattered in multiple sub-regions, as shown in Figure 7. In Figure 7, the DecodeWidth equals 1, and other components differ. We can observe that the Pareto frontier spread across multiple sub-regions, highlighted in red ribbons in Figure 7. An algorithm should inspect those sub-regions and sample microarchitecture embeddings across them, circumventing trapping into local optimum. Therefore, we propose the diversity-guided parallel exploration.

Fig. 7.

View Figure

MicroAL leverages DecodeWidth as a critical factor to capture the main characteristics of the design space. Similarly, we also notice that IntPhyRegister can significantly affect the tradeoff. Figure 8 visualizes such impacts. Dhrystone and whetstone are integer instructions-intensive and floating-point instructions-intensive benchmarks, respectively. When the microarchitecture selects IntPhyRegister from 48 to 112, the performance is improved by \(8.52\%\) and dissipates \(6.86\%\) more power. Since whetstone is mainly composed of floating-point instructions, increasing IntPhyRegister introduces no change to the performance but worsens power dissipation more. BOOM implements the unified integer physical register files design, i.e., the registers hold both committed and speculative values/states.

Fig. 8.

View Figure

More integer physical registers help to resolve more integer-related instructions conflicts, e.g., data dependencies, and provide more support for instruction parallelism. Most commonly used benchmarks contain many integer-related instructions, e.g., dhrystone, mm, median, and so on. Thus, the impact on the tradeoff between performance and power can highly correlate with IntPhyRegister on these benchmarks. The findings motivate us to propose the backbone for diversity guidance.

We leverage IntPhyRegister to partition the clustered design space by MicroAL, as demonstrated in Algorithm 3. In Algorithm 3, the clusters are obtained from MicroAL (line 2), and partitions within each cluster are formulated according to IntPhyRegister (line 5). Different sub-regions are constructed based on Algorithm 3. We sample microarchitecture embeddings according to the highest EIPV from each sub-region, respectively, as Equation (15) shows, (15) \(\begin{equation} \boldsymbol {x}^* = \left\lbrace \mathop {\arg \max } _{\boldsymbol {x}^{\prime } \in \mathrm{P}_j}\text{EIPV}(\boldsymbol {x}^{\prime } \mid \mathrm{P}_j) \mid j = 1, 2, \ldots , N\right\rbrace \!, \end{equation}\) where \(P_j\) denotes the partition j, and N is the total number of partitions. The sampled batch \(\boldsymbol {x}^*\) is evaluated with the VLSI flow in parallel.

Algorithm 4 details the whole framework. Line 4 performs the partition. Combined with Table 1 and MicroAL, the partition operation incurs 25 different sub-regions for the design space. We establish and train the surrogate model, DKL-GP, for each optimization round (line 6) and select unvisited partitions to aggregate the explored dataset (line 7). We maintain a visited buffer to record the access times to a specific partition in line 7. The framework extends the explored dataset gradually (line 13). Finally, the predicted Pareto frontier and corresponding microarchitecture embeddings are obtained. It is worth noting that Algorithm 4 requires at least b available EDA tools licenses to implement.

Other solutions could also be leveraged in Algorithm 3 to further improve the performance. First, some partitions can be directly pruned by the expertise. Architects have predetermined performance or power design goals. The selection of partitions can be made based on these goals. For example, we prefer to use a wider microarchitecture to pursue higher performance. So, we can only focus on partitions with decode width attaining the maximal value while neglecting other partitions, i.e., we only focus on “important” partitions. Second, the exploration-exploitation heuristic can be utilized, similar to the action selection in reinforcement learning [41]. We decide whether to exploit the current best-known partition that effectively achieves higher Pareto hypervolume or explore new partitions. The decision could be based on epsilon-greedy heuristics, upper confidence bound, and so on. We leave these discussed solutions in our future work.

5.1 Experiments Settings & Evaluation Metrics

We utilize Chipyard [42] to generate various BOOM RTL designs. And we use 7-nm ASAP7 PDK [29] for the VLSI flow. Cadence Genus 18.12-e012_1 is used to synthesize every sampled RTL design with 1 GHz timing constraints, and Synopsys VCS M-2017.03 is used to simulate the design with different benchmarks. PrimeTime PX R-2020.09-SP1 is leveraged to get power values for all benchmarks. All experiments are conducted in 80 cores of Intel(R) Xeon(R) CPU E7-4830 v2 @ 2.20 GHz with 1 TB main memory.

In the settings of BOOM-Explorer, DKL-GP is stacked with three hidden layers, each of which has 1,000, 500, and 50 hidden neurons, respectively, and it adopts ReLU as the non-linear transformation for deep kernels. The Adam optimizer [43] is used, with an initial learning rate equal to 0.001. BOOM-Explorer performs Bayesian exploration with 9 iterations. All experiments together with baselines are repeated 10 times, and we report corresponding average results.

5.2 Benchmarks, Baselines & Evaluation Metrics

To assess each microarchitecture, we employ a set of benchmarks selected from bare models [15]. These benchmarks include median, mt-vvadd, whetstone, mm, and so on. The benchmarks have covered all kinds of RV64G instructions, e.g., integer-related, floating-point numbers-related, memory-related instructions, and so on, and each benchmark focuses on testing specific instruction features. For example, mm focuses on multi-threaded memory reads and writes operations. Since a BOOM design needs to handle various applications rather than specific instruction categories, we average the clock cycles and power values from these benchmarks to denote the design’s performance and power. After warming up the design by executing the benchmark, we proceed to measure the performance and power for a specific BOOM configuration. This process allows us to obtain accurate performance and power values for the given microarchitecture, taking into account any initial variability or transient effects that may occur during the warm-up phase.

Several representative baselines are compared with the diversity-guided BOOM-Explorer. The ANN-based method [18] (shorted as ASPLOS’06) stacks ANN as the performance model for a multiprocessor to conduct DSE. The regression-based method [27] (termed HPCA’07) leverages regression models with non-linear transformations to explore the performance-power Pareto frontier for POWER microprocessors. The AdaBoost-RT-based method [21] (abbreviated as DAC’16) utilizes the orthogonal design sampling [49] and active learning-based AdaBoost regression tree models [28] to explore an Alpha21264-like microprocessor [8, 11]. The arts mentioned above proved effective in exploring microarchitecture parameters in their works, respectively. Therefore, it is requisite to compare these methodologies with our proposed methodology. The high-level synthesis (HLS) predictive model-based method [45] (named DAC’19) exploring the HLS design is also chosen as our baseline. Although the starting point is different, their method proved robust and transferable. We also compare diversity-guided BOOM-Explorer with traditional machine learning models, including support vector regression (SVR) and tree-based approaches such as random forest, XGBoost [50], and tree-structured Parzen estimator approach (abbreviated as NIPS’11) [44]. In addition, we compare previous state-of-the-art Bayesian optimization approaches [46, 47] with our proposed methodology, since both methods adopt the same optimization framework but are distinct in the design of initialization, surrogate model, and acquisition function. We name the two state-of-the-art approaches as NIPS’19 and NIPS’22, respectively. For fair comparisons, the experimental settings of the baselines are the same as those mentioned in their papers. In traditional machine learning algorithms (SVR, random forest, and XGBoost), since they are not fit for Bayesian optimization due to unavailable predictive uncertainties, we leverage simulated annealing [51] to explore the design space. We also compare the proposed methodology with the sequential optimization version of BOOM-Explorer [48]. All algorithms explore the same design space, as defined in Table 1. In our future work, we will delve into more intricate design spaces that encompass branch prediction algorithms, prefetching, and additional objectives, such as area metrics. However, one caveat is that there is a resemblance between microarchitecture and area values, akin to power values, where a larger area usually corresponds to higher power dissipation. Meeting higher performance requirements often entails employing more hardware resources, which, in turn, leads to a larger area.

To compare the diversity-guided BOOM-Explorer with all baselines, we utilize multiple metrics. These metrics include the Pareto hypervolume, the average distance to the reference set (ADRS), and the overall running time (ORT). Pareto hypervolume and ADRS are two widely used metrics in estimating the performance of DSE among multiple objectives. As mentioned in Section 4.4, the Pareto hypervolume measures the volume of the space enclosed by all solutions on the explored Pareto frontier and a user-defined reference point, where the computation of Pareto hypervolume is defined in Equation (11). The ADRS computes the average distance between the predicted Pareto frontier and the real Pareto frontier, providing insights into the quality and proximity of the solutions to the golden results. Equation (16) lists the computation of ADRS. (16) \(\begin{equation} \text{ADRS}(\Gamma , \Omega) = \frac{1}{\vert \Gamma \vert } \sum \limits _{\gamma \in \Gamma } \mathop {\min }\limits _{\omega \in \Omega }f(\gamma , \omega), \end{equation}\) where f is the Euclidean distance function, \(\Gamma\) is the real Pareto frontier, and \(\Omega\) is the predicted Pareto frontier. The ORT measures the total running time of algorithms, including initialization, exploring, and the VLSI runtime cost. The higher the Pareto hypervolume, the lower the ADRS and ORT, the better the DSE algorithm is.

5.3 Evaluation Results

Figure 9 shows the predicted Pareto frontier obtained by the baselines and diversity-guided BOOM-Explorer. The results show that the Pareto frontier generated by BOOM-Explorer and its diversity-guided version is much closer to the actual Pareto frontier in general. Moreover, the diversity-guided BOOM-Explorer improves the results visually.

Fig. 9.

View Figure

The normalized Pareto hypervolume, ADRS, and ORT results are listed in Table 3. Three summaries can be drawn from Table 3. First, BOOM-Explorer achieves an average of \(18.75\%\) higher Pareto hypervolume, \(35.47\%\) less ADRS, and \(65.38\%\) less ORT compared to all baselines. Specifically, BOOM-Explorer outperforms ASPLOS’06, HPCA’07, DAC’16, and DAC’19 by \(70.13\%\), \(66.55\%\), \(28.65\%\), and \(64.54\%\) in ADRS, respectively. Meanwhile, it accelerates the exploration by more than \(88.19\%\) compared with DAC’16. The improvement achieved by our method over previous approaches stems from a customized algorithm design explicitly tailored to the problem at hand. Our method asks for fewer estimations using the VLSI flow while effectively modeling the design space with as few samples as possible. Second, MicroAL contributes \(11.00\%\) and \(20.53\%\) to exalt the Pareto hypervolume and ADRS. The results are obtained from the comparison between BOOM-Explorer and BOOM-Explorer w/o MicroAL. It also illustrates that without MicroAL, the performance of BOOM-Explorer would be close to DAC’16. If more time budget is allowed, then we expect better results received from BOOM-Explorer. Third, incorporating diversity guidance as a key enhancement, we achieve a significant improvement in the ADRS, surpassing BOOM-Explorer by \(20.09\%\) with comparable ORT. The improvement on ADRS is larger than the Pareto hypervolume, demonstrating that “outliers” can be explored. As discussed in Section 4.5, these outliers pertain to similar or smaller EIPV but closer to the real Pareto frontier. It is worth noting that we partition the design space w.r.t. IntPhyRegister, as discussed in Section 4.5. Hence, the proposed method has a better effect when we target to optimize for integer-intensive applications. Additionally, previous state-of-the-art approaches (NIPS’19 and NIPS’22) receive a relatively good Pareto hypervolume and smaller ADRS more efficiently than other baselines. Like random forest and XGBoost, the NIPS’11 baseline [44] adopts tree-based structures as the surrogate model but uses Bayesian optimization. The NIPS’19 [46] and NIPS’22 baselines [47] leverage information-theoretic acquisition function designs. Although, these baselines perform well in general DSE problems, they achieve mediocre results compared to diversity-guided BOOM-Explorer largely due to the missing customization in the algorithm design such as tightly coupled with expert knowledge.

5.4 Comparison of Pareto-optimal BOOM Microarchitectures

We compare Pareto-optimal microarchitectures explored by proposed algorithms and human implementations [12, 13, 14] on more benchmarks to study how each BOOM microarchitecture balance the performance and power. The Pareto-optimal microarchitectures have similar parameter settings, as listed in Table 4.

Pareto-optimal designs found by BOOM-Explorer and diversity-guided BOOM-Explorer have the same decode width as the two-wide BOOM. However, the Pareto-optimal design reduces hardware components on the branch predictor (i.e., RasEntry, BranchCount, etc.), entries of the reorder buffer, and so on, but enlarges instructions issue width, load queue (LDQ), store queue (STQ), and so on. Moreover, it has different cache organizations, e.g., different associate sets. Because LSU introduced in Section 2.1 tends to become a bottleneck of the microarchitecture, the Pareto-optimal design increases hardware resources for LDQ and STQ, increasing associate sets and miss status handling register (MSHR) entries for D-cache to overcome more data conflicts. Furthermore, the Pareto-optimal design found by diversity-guided BOOM-Explorer reduces the resources of return address stack (RAS) and branch target buffer (BTB), since they affect the tradeoff less for the same branch prediction algorithm [30]. It also reduces the instruction issue slot but increases the I-cache associative sets. Both Pareto-optimal designs achieve a better tradeoff on power and performance by reducing redundant hardware resources while increasing necessary components on critical paths [52].

We evaluate these microarchitectures with more benchmarks. Table 4 shows the average clock cycles and power values for all these benchmarks. These benchmarks are chosen from different application scenarios, e.g., add-int, add-fp, and so on, are from ISA basic instructions, iir, firdim, and so on, are from DSP-oriented algorithms [53], compress, duff, and so on, are from real-time computing applications [54], and so on. Figure 10 shows the comparison of performance and power values, respectively. For all of these benchmarks, BOOM-Explorer’s design runs approximately \(2.11\%\) faster and, at the same time, dissipates \(3.45\%\) less power than the two-wide BOOM. The solution from diversity-guided BOOM-Explorer achieves \(2.18\%\) faster and \(3.99\%\) better on power dissipation than human implementations.

Fig. 10.

View Figure

5.5 Effectiveness of MicroAL

To assess the effectiveness of MicroAL, we conduct an ablation study on the effectiveness of MicroAL. We investigate the integration of MicroAL to some baselines, which allows us to modify the initialization algorithm that is not tightly coupled with the exploration procedure. Specifically, we conduct MicroAL with SVR, random forest, XGBoost, ASPLOS’06 [18], HPCA’07 [27], NIPS’11 [44], NIPS19 [46], and NIPS’22 [47].

Figure 11 lists the comparative results on the Pareto hypervolume and ADRS when we integrate MicroAL to baselines. Two summaries are drawn from Figure 11. First, integrating MicroAL into baselines can improve an average of \(8.06\%\) Pareto hypervolume, and \(12.15\%\) ADRS. For example, when we integrate MicroAL into SVR, the Pareto hypervolume can be increased by approximately \(18.08\%\) while the ADRS is reduced by a large margin. Second, MicroAL can have negative effects on some benchmarks like NIPS’19 [46]. Although integrating MicroAL to NIPS’19 [46] results in a similar Pareto hypervolume, it degrades the ADRS by \(25.16\%\). We argue that the reason behind the phenomenon is that we leverage single-task GP models to individually model the performance and power. In contrast, the samples generated by MicroAL are highly different. Therefore, the built surrogate models are rather “weak.” However, objective-related features can be utilized to capture the distinctions between MicroAL’s samples. So, a good co-design between the initialization and the surrogate model is significant.

Fig. 11.

View Figure