1 Introduction

Energy consumption is now one of the primary concerns for computer science because it tends to increase over the years, raising several technical, economical, and ecological issues. Considering economical consequences, not only more electricity is consumed over the years but also the price of electricity is raising [15]. Thus, energy consumption of computing devices cannot be ignored any longer and energy savings must be achieved.

Although CPU and memory are often identified as the major consumers, the cooling machinery accounts for a non-negligible part of the overall energy consumption [3]. Except for some uncommon configurations, fans are still often in charge of cooling within a computer; it is common for a rack server to be made of several fans, each potentially consuming as much as 10 W at full speed. As the fan power consumption can account for a large part of the total energy consumption, they are a good target for energy optimization.

In general, the temperature of the main server components drives fan speed: fans accelerate when the temperature increases in order to maintain the system temperature below an arbitrary threshold, often set to a conservative value. Such fan control is called thermal-directed. Thermal-directed fan control focuses only on temperature management, trying to avoid hardware failures due to overheating, and ignores energy consumption. Thus, thermal-directed fan control typically results in fans unnecessarily rotating at high speeds consuming too much energy, leaving room for optimization.

On the other hand, slowing fans down increases the temperature and, apart from the increasing risk of hardware failures, it increases the power leakage of several components including the CPU. Thus, efficiently managing fan speed consists in determining the optimal fan setting, which simultaneously minimizes the power leakage and fan power consumption, i.e. cooling enough but not too much. The issue is illustrated in Fig. 1, where the optimal fan setting is the one leading to minimal power consumption from both CPU power leakage and fan power consumption. The fan controller introduced in the following paper, DFaCE, is able to perform such a subtle fan control: it optimizes the power consumption of a server, using fan speeds that simultaneously minimize fan power consumption and power leakage.

Fig. 1
figure 1

Illustrating the balance between fan power consumption and power leakage

Full size image

Compared to existing dynamic fan control strategies, DFaCE has the following advantages:

  • Learns optimal fan settings, without provoking any performance degradation, while real applications are executed

  • Immediately applies optimum fan setting once the hardware characterization process ends

  • Simultaneously controls several fans

A prototype system was implemented and evaluated over a set of benchmarks. It achieves power savings of up to 46 % power savings in fan power consumption and power leakage compared to the commonly used thermal-directed fan controllers. The gains translate into up to 3.3 % in the total system consumption.

The paper is organized as follows. The actual fan control performed by DFaCE is described in Sect. 2. The power savings and the efficiency of DFaCE were measured over a set of benchmark programs; Sect. 3 contains the experimental results. Finally, DFaCE and previous proposals are compared in Sect. 4, before concluding in Sect. 5.

2 Optimal state discovery

2.1 Overview

Considering all the possible fan settings, the objective is to determine the one leading to minimal energy consumption. There is however no unique fan speed suitable to any situation because the heat generated by the processor is not constant but depends on its usage. Thus, DFaCE aims at finding the optimal fan setting, i.e. the one leading to minimal energy consumption, for the current workload while ensuring extreme temperatures cannot be reached to avoid hardware failures.

The impact of a fan on the temperature is in fact not absolutely undefined as some simple rules govern the relationship between fan speed, power consumption, and temperature. First, fan power consumption monotonically increases with its speed: a fan cannot run faster with less energy. Second, for a fixed workload, the temperature decreases as the fan speed increases. Finally, power leakage decreases as temperature decreases. Provided the three rules are respected, the space of the possible solutions is convex and contains only one minimum. The optimal fan setting is found where the total system power consumption is minimal, as illustrated in Fig. 1. As the space of possible fan settings is convex, hill-climbing optimization can be used to determine the fan setting with minimal energy consumption.

To achieve its objective, DFaCE uses a hill-climbing optimization during a training phase to determine the optimal fan setting depending on the processor load, as defined in Sect. 2.3. Because the temperature evolves slowly when a new fan setting is applied, the training phase requires stable workloads but can be performed while actual computations are achieved on the computer. Such long and stable workloads are typically found in HPC for which DFaCE is particularly well suited. In a second step, once the optimal fan setting is determined for a given load level, it is applied as soon as the same load level is observed, minimizing energy consumption.

2.2 Hill-climbing

Hill-climbing is a simple optimization technique for determining the maximum, and conversely minimum, in a convex space. Hill-climbing optimizers first choose an arbitrary solution then evaluate the surrounding ones. As soon as the optimizer detects a better solution, it uses the new best solution as the reference point for restarting the evolution. If none of the surrounding solutions are better than the original reference solution, the evaluation restarts on closer points. Hill-climbing optimization allows a fast convergence towards the optimal solution in a convex space.

Many fan speed settings are available on modern computers. As an example, our experimental platform allows 256 speed steps for every connected fan. If several fans are available on the system, the number of different settings quickly becomes huge. The experimental platform has three fans, leading to nearly 17 million different possible settings. It is then critical to traverse the space of possible setting using an efficient method such as hill-climbing.

DFaCE traverses the space of possible fan settings using a hill-climbing optimization. It first evaluates the power consumption induced by a reference fan setting. Then, it evaluates all the fan settings close to the reference setting. The settings close to the reference setting are called the neighborhood, each setting in it being a neighbor of the reference point. The concept of neighborhood is more precisely defined in the following paragraph. If one of the neighbor settings leads to a lower power consumption than the reference setting, it becomes the new reference setting for the evolution. In case no surrounding setting is better than the reference setting, the evolution starts over on a closer neighbor, as illustrated in Fig. 2. The evolution finally ends when the set of surrounding solutions cannot be restricted anymore. Figure 3 describes DFaCE’s methodology. As a result, DFaCE evaluates the power consumption induced by several settings until converging on the optimal fan setting, ensuring minimal power consumption.

Fig. 2
figure 2

The optimizer evaluates several solutions while always progressing toward the optimum

Full size image
Fig. 3
figure 3

Overview of the learning strategy

Full size image

Let n be the number of fans controlled by DFaCE, fs i be a specific speed of the fan i with 1≤i≤n and ref be a fan setting such that ref=(fs 1,…,fs n ). Every neighbor of the setting ref is defined as (fs 1+α 1×step,…,fs n +α n ×step) with α i ∈[−1,0,1], 1≤i≤n and step being a distance between ref and its neighbors. Every time DFaCE finds a better setting, it evaluates the power consumption of every neighbor, excluding the case where ∀i,α i =0, which defines the reference setting itself. The value step is divided by 2 when closer settings have to be evaluated.

Power leakage depends on the temperature. Thus, to measure the power leakage induced by a fan setting, DFaCE has to wait for the temperature to stabilize, which lasts approximately for ten minutes during the experiments: DFaCE considers the temperature as stable when is changed by less than half a degree in the last three minutes. As for the all the temperature measurements in the rest of the paper, CPU temperature is measured using the Linux lm-sensors program. Once the temperature is stable, DFaCE estimates the current power leakage and fan power consumption to deduce the overall power consumption of the fan setting.

In order to estimate power leakage and fan consumption during the evolution phase, both powers are measured offline. Fan power consumption is measured while increasing the fan speed for each available fan to estimate fan consumption depending on their speed. Power measurements are performed at the system scale on an idle computer. Then, fan power consumption is directly deduced by its speed. Power leakage is also measured offline by stopping all fans and running a computationally demanding task on all the cores while measuring power consumption using the energy probe embedded in the CPU as the temperature increases. Then, DFaCE builds a table from the measured power consumptions to associate a power leakage to each observed temperature. Both power leakage and fan consumption are then added when a given temperature is reached during the evolution to estimate the power consumption of the currently evaluated fan setting. The estimated power consumption drives the evolution and hill climbing ultimately determines the setting leading to minimal estimated power.

2.3 System load

A fan setting can only be optimal for a given amount of heat generated by the CPU. Thus, DFaCE learns a different fan setting depending on the amount of heat generated by the CPU. It is hard to directly measure the amount of heat generated by a CPU while it is cooled but the activity of a processor is strongly correlated to the heat it generates and usually translates into simple metrics. For instance, the average CPU load, i.e. the ratio of active CPU cycles during the last second, is a good metric for estimating the heat generated by a processor. Obviously, the CPU load metric does not reflect the exact stress applied on hardware but is sufficient to roughly distinguish various heat generation levels. Moreover, it can be combined or replaced by more precise information to increase precision. It also assumes that, when frequency scaling is applied, it is performed based on the CPU load level as it is currently the case on Linux. Thus, DFaCE uses the device activity to estimate the amount of heat it generates and learns different fan settings depending on it.

Hardware devices may have many different activity levels. Thus, instead of CPU load values, DFaCE considers ranges of CPU loads. The best fan setting is then determined for a given load range. Considering load ranges instead of exact load values leads to a slightly sub-optimal result as the amount of heat generated is not constant for all the possible activities in the range, but, when small ranges are used, the fan setting remains comparably efficient. In the current implementation, DFaCE considers load ranges of 10 %, meaning that it learns 10 different settings to cover all the possible cases.

As power consumption can only be estimated once temperature stabilized, DFaCE can only measure the impact of a given fan setting on power consumption once the system temperature is stable, which can take several minutes. Meanwhile, the system load must be stable. If the CPU load changes before temperature stabilizes, the measurement is paused. The next time the computer runs at the same load level, DFaCE resumes the evolution from the setting being evaluated when the evolution was paused. Thus, in the presence of variable system loads, the learning process could be longer, although useful computations are performed meanwhile. Notice that nothing prohibits the use of artificial stable computations to speed up the evolution process or to deal with environments where the load level fluctuates quickly. As it can resume the evolution from where it was paused, DFaCE limits the effects of variable workloads on temperature stabilization.

The result of the evolution phase is then an optimal fan setting for different workload level ranges. Once DFaCE determined the optimal fan setting for one workload level, it applies it immediately once the workload level is observed.

2.4 Temperature stability

Unexpected external events impact the system temperature and therefore which fan setting is optimal. Two major events can lead to such a situation. First, the external temperature of the computer might vary, due to local hotspots appearing in the cluster or to a weakness of the air cooling system. Second, one of the fans under control might fail and slow down, or even stop. In both cases, the system temperature is different from the one achieved during the evolution phase. As the temperature is not the expected one, power consumption is sub-optimal: either the system is colder than what was expected and the fans should be slowed down, or the temperature is higher than what the optimizer learned and the fan speed may have to be increased. Considering the consequences on energy consumption, such external events have to be considered.

In DFaCE, the fan settings learned are in fact not permanently fixed. Instead, when a previously learned fan setting is applied, DFaCE waits for the temperature to stabilize and compares it to the temperature observed when learning the fan setting during the evolution phase. If the difference is greater than 3 °C, the fan setting is not considered as optimal anymore and a new learning phase is started, using the previous optimal setting as the starting point for hill-climbing. The ability to restart the evolution ensures system reaction to unexpected events such as a fan failure or a local hotspot.

Temperature instability is also used to the advantage of DFaCE. When a previously learned fan setting is applied, it takes several minutes before the temperature settles. As long as the temperature is below the one observed during the evolution phase, all fans are shut down to decrease power consumption. Once temperature reaches the expected value, DFaCE starts the fans and sets them to the optimal speed learned during the evolution, minimizing the power consumption and maintaining temperature stability. Keeping the fans shut down as long as possible allows additional power savings until the temperature stabilizes.

2.5 Critical temperature

Although DFaCE estimates the generated heat based on system activity, the processor temperature cannot be totally ignored: any cooling system has to ensure no hardware failure occurs due to overheating. Most of the components have critical temperatures above which the hardware lifetime is greatly shortened. It is then essential for any cooling system to prevent the hardware from reaching its critical temperatures. DFaCE makes no exception as it is designed to react quickly when a setting provokes overheating. Notice that CPU lifetime can also be impacted by DFaCE if it increases the operating temperature [14]. During the evolution phase, a fan setting is immediately rejected if it leads to a temperature just below the critical threshold. Additionally, if a previously learned setting leads to near-overheating situations, DFaCE considers the setting as invalid and starts a new evolution using higher fan speeds. Thus, DFaCE never sets the fans to speeds that cause overheating.

2.6 Implementation

DFaCE implements all the presented mechanisms. It determines the fan setting leading to minimal energy consumption thanks to hill climbing optimization for a particular load level. Once learned, the setting is applied as soon as the same load level is observed. The optimization process is restarted from the previously determined fan setting if the temperature settles to a different value than what DFaCE measured during the evolution phase. The mechanisms are combined as described in Fig. 4. Additionally, the fan speed is increased at any time if a critical temperature is detected to avoid hardware failures. All the mechanisms implemented in DFaCE allow an automatic selection of fan speeds reducing the system energy consumption without any prior knowledge on the existing hardware nor any approximation on physics. Moreover, DFaCE also ensures no hardware failure occurs because of overheating.

Fig. 4
figure 4

Overview of the general system’s algorithm

Full size image

3 Experiments

3.1 Experimental setting

DFaCE was implemented and evaluated on an experimental computer. The experimental platform is made of a desktop computer with three different controllable fans: a Scythe Mugen 3 CPU fan and two Alpenföhn Wing Boost 120 chassis fans. One of the chassis fans is located in front of the box, while the other one extracts air at the rear, near the CPU. The CPU is an Intel Core i5 2380P quad-core processor with one thread per core. The running operating system is Linux 3.4, using an experimental driver for the Nuvoton NCT 6775 fan controller chip [12] embedded on the ASUS P8Z77V PRO motherboard. The C-states C0 to C3 are active. The ondemand DVFS governor is set. The computer was used to run every DFaCE phase, from characterization to optimization. The experimental results are presented below.

3.2 Evolution

In the first step of our experiments, we artificially loaded the computer using specially-crafted micro benchmarks performing CPU intensive operations such as square root operations and random number generations, using an increasing number of processes. It generates various load levels that last long enough to let DFaCE determine the ideal fan setting for every generated load level.

Hill-climbing allows optimizers to quickly converge towards the optimal solution in a convex space. Although the optimal setting discovery may be long, near optimal solutions are evaluated during a large portion of the evolution. DFaCE is no exception and benefits from such behavior: a lot of evaluated settings are in fact close to the optimal fan setting, as illustrated in Figs. 5 and 6. One can notice on the figures that power consumption is close to minimal long before the optimization ends, especially in Fig. 6. Thus, even if DFaCE needs a long time before determining the optimal fan setting because of the slow temperature stabilization, it can achieve near-optimal fan control even long before the learning process finishes.

Fig. 5
figure 5

Fan power consumption plus power leakage, and CPU temperature converge towards the optimal solution with a 25 % load level

Full size image
Fig. 6
figure 6

Fan power consumption plus power leakage, and CPU temperature converge towards the optimal solution with an half-loaded CPU

Full size image

3.3 Optimal temperature

The usual method for managing fans consists in defining a temperature threshold and adapting a fan speed to remain as close as possible to the threshold: when the temperature increases, the fans speed is also increased, when the temperature is below the threshold, the fans are stopped. The temperature threshold has to be low enough for the power leakage to remain limited and, at the same time, high enough for fans to operate at a low speed for any temperature. Usually, such thermal-directed fan controllers use a default temperature threshold, arbitrarily chosen by motherboard designers or at the operating system level with little consideration for power consumption. Thus, even if thermal-directed fan controllers perform well, arbitrary temperature thresholds often limit their power efficiency.

During its evolution phase, DFaCE determines the optimal fan setting for several load levels, each one leading to a different CPU temperature. Interestingly, the stable temperature to reach in order to optimize power consumption depends on the system load, as seen in Table 1. Thus, thermal-directed fan controllers cannot reach optimal power consumption because they consider a single temperature threshold for every system load. On the other hand, DFaCE is able to consider several fan settings depending on the workload level and is then superior to classical thermal-directed controllers.

Table 1 Optimal CPU temperature for different workload levels
Full size table

3.4 Power savings

Once the optimizer discovers the best fan setting for all possible load levels, it immediately applies it as soon as a new load level is observed. Load level variations are detected thanks to periodic checks that do not induce any measurable CPU overhead. Due to time constraints, DFaCE only learned the optimal fan settings for a subset of the possible load levels using artificial loads in our experiments. However such partial knowledge is still sufficient to evaluate the power savings achieved at the learned load levels.

An arbitrary sequence of programs, built from the NAS-OMP 3.0 benchmarks, was run using a different numbers of threads for each program using DFaCE and thermal-directed fan control. The experimental procedure executes a realistic scenario made of existing programs, while the variable number of threads provokes variable load levels, trying to fool the optimizer. The load level seen by DFaCE roughly corresponds to a fourth of the number of threads as the tasks are compute intensive and as there are four cores on the processor. The programs and corresponding number of threads used for each program are presented in Table 2. The power consumed by the computer was then measured using a Yokogawa WT-210 power meter plugged to the computer. The optimal fan setting discovered by DFaCE was compared to the default thermal-directed fan controller targeting 50 °C, the factory setting, or 60 °C, arbitrarily determined as a relevant value. Such thermal-directed temperature is provided by the experimental motherboard, an ASUS P8Z77V PRO, and consists, as any thermal-directed controllers, in increasing fan speeds when the temperature exceed the threshold in order to maintain the system as much as possible below the threshold temperature.

Table 2 Power savings achieved by DFaCE compared to thermal-directed cooling with a target temperature of 50 °C or 60 °C
Full size table

During our experiments, we measured the power saved by DFaCE at the overall system scale. In order to determine how significant are the power savings compared to existing mechanisms, we express the power savings in Table 2 as a percentage of the maximal fan power consumption and power leakage induced by the default thermal-directed cooling targeting 50 °C or 60 °C. The maximal fan power consumption is the one induced by the maximal fan speed and the considered power leakage is the one observed at the temperature targeted by the cooling system, i.e. 50 °C or 60 °C. As we consider maximal power consumption, the savings expressed as percentage of the fan consumption and power leakage are in fact a pessimistic lower bound: DFaCE has often saved a greater percentage of fan power and power leakage than what is presented in the table. Additionally, we also express the power savings relatively to the overall system consumption.

The presented numbers show that DFaCE is able to significantly improve the power consumption of the fans and power leakage. However, because the cooling subsystem does not account for a major part of the overall system power consumption, the gains seem negligible at the system scale. It is reasonable to expect major gains in computers where the fans account for a larger part of the total power consumption, such as the ones used by Wang et al. [16].

4 Related work

Some previous work builds theoretical models to estimate the temperature induced by a given load or the best air cooling setting for a given temperature. Rao and Vrudhula use such a theoretical model to determine the best CPU frequency according to the chip temperature [11]. Heo et al. propose a simple model to deduce the leakage from the temperature with the goal of migrating tasks [5]. Liu et al. also build a theoretical model to predict power leakage [9]. The three models could be used to design a fan controller but are unable to determine an optimal fan speed. DFaCE is not based on theoretical models but rather directly observe the effects of a fan setting on temperature.

Several other researchers propose using theoretical models with the goal of optimizing fan settings. Shin et al. theoretically model the effects of the temperature to simultaneously set the optimal fan speed and processor frequency [13]. Their system could lead to performance degradation as DVFS is considered an option for cooling the CPU down. DFaCE only considers fans and therefore cannot degrade the programs performance. Similarly, Wang et al. propose a Fan Controller (FC) based on a theoretical model that sets the best fan speed for several fans as soon as the CPU load changes [16]. FC, unlike DFaCE, does not consider power leakage: FC minimizes the fan power consumption while maintaining the system temperature at an arbitrary temperature threshold. The presented systems consider theoretical analysis and models to determine an efficient fan setting. Therefore, the solutions suffer from bias induced by the approximations needed to model the complex temperature-related physics.

Theoretical models are built for a specific system or context and external events, such as fan failures or local hot-spots, cause them to be temporarily inaccurate as the models’ parameters may change without being reevaluated. DFaCE is not based on theoretical representations of the problem and does not have to approximate the problem because the effects of fan settings are evaluated directly on the computer itself.

Dynamic systems react according to what they observe on the system they run on; they do not suffer from the flaws of theoretical models. One such dynamic system, Thermal-Aware Power Optimization for servers (TAPO-server), was proposed by Huang et al. [6]. TAPO-server regularly switches the fan speed to determine if the fan speed has to be increased or decreased in order to reach the minimal power consumption. The authors present convincing results, but TAPO-server does not take into consideration the quick variations of the heat generated by the device. It also restarts the learning process at every major system load change. Moreover, it is unable to handle more than one single fan; whereas, DFaCE is dedicated to multiple fan control, allowing it to efficiently optimize the cooling power consumption. DFaCE also works in two distinct sequences: once the best setting is learned for a given load, it is immediately applied as soon as the load is observed again. Such knowledge capitalization and the ability to reuse the optimal fan settings learned is a key advantage over existing dynamic systems, which are currently unable to react as quickly as DFaCE.

Additionally, several mechanisms were described in the patent literature although they are often similar to the system presented before. For instance, many patents [1, 2, 7, 8] perform simple fan control close to what is achieved by thermal-directed fan control. The work described in [4, 10] is close to TACO-server and suffers from the same flaws. To the best of our knowledge, they represent the most complex attempts for energy-efficient fan control in the patent literature.

5 Conclusion

DFaCE is the first runtime fan controller to learn energy-optimal fan settings on computers where several fans are available, while the programs can perform useful computations. Although a long learning time is required, the server does not necessarily have to execute a specific workload during the learning process and close to optimal settings are applied long before the end of the evolution phase. Once the best setting is automatically determined, DFaCE immediately applies it, significantly reducing fan power consumption and power leakage. Moreover, DFaCE is a dynamic system so it adapts to unexpected external events such as fan failures. In such critical situations, DFaCE ensures no hardware overheating occurs and even learns the new optimal fan setting.

Different extensions can be envisioned for DFaCE such as evaluating it on real cluster nodes, where power savings should be more significant. Evaluating the potential of hill-climbing parallelization and integrating theoretical temperature models to speed up the learning process are also interesting extensions that could and enable DFaCE to perform better.