Characterizing GPU Overclocking Faults

Abstract

Graphics Processing Units (GPUs) are powerful parallel processors that are becoming common on computers. They are used in many high-performance tasks such as crypto-mining and neural-network training. It is common to overclock a GPU to gain performance, however this practice may introduce calculation faults. In our work, we lay the foundations to exploiting these faults, by characterizing their formation and structure. We find that temperature is a contributing factor to the fault rate, but is not the sole cause. We also find that faults are a byte-wide phenomenon: individual bit-flips are rare. Surprisingly, we find that the vast majority of byte faults are in fact byte-flips: all 8 bits are simultaneously negated. Finally, we find strong evidence that faults are triggered by memory-remnant reads at an alignment of a 32 byte memory transaction size.

A Appendix

1.1 A.1 CUDA basics

CUDA Kernels are declared using the $\mathtt{\_}{} \mathtt{\_}{} \mathtt{global}{} \mathtt{\_}{} \mathtt{\_}$ declaration specifier and can be invoked using the syntax in Algorithm 2. Kernels are executed in blocks where each block consists of multiple threads. The parameters numBlocks and threadsPerBlock specify the execution configuration syntax. Each thread that executes the kernel is given unique thread/block IDs that are accessible within the kernel through built-in variables. All threads of a block reside on the same processor core and must share the memory resources of that core. Therefore, the number of threads per block is limited (up 1024 on current GPUs). Instructions are issued and executed in groups of 32 threads, called warps.

Thread blocks are required to execute independently: It must be possible to execute them in any order, in parallel or in series. Threads within a block can cooperate by sharing data through some shared memory and by synchronizing their execution to coordinate memory accesses. Synchronization points can be declared using intrinsic functions, e.g., $\mathtt{\_}{} \mathtt{\_}{} \mathtt{syncthreads()}$.

1.2 A.2 Future Work

Future work involves leveraging the characterization of faults presented in this paper towards the development of efficient tailored exploitation algorithms and methods. Examples include:

Breaking Cryptographic Calculations Implemented on GPUs. One can speculate that using the byte-flip phenomenon may be incorporated with the work done by Sabbagh et al. [35]. As their work relies on exploiting an instrumented-AES, our characterization might enable the attack to target non-instrumented kernels, as well as reducing the number of messages required to break the encryption. Also it seems that byte-flips may be used to improve attacks on public-key calculations done in a GPU.

Faulty Instructions. During our tests we observed that as the faults rate increased, occasionally the graphics card stopped responding (API calls failed), crashed, or acted extremely slow. We also received kernel crashes with error codes such as: “An illegal instruction was encountered” and “Invalid program counter”. This suggests that the GPU is not only vulnerable to data corruption, but also to instruction corruption [14, 26, 38,39,40], since code-registers (apart from data-registers) are also vulnerable to the faults caused by overclocking.

The knowledge in this paper may allow an attacker to develop code which triggers precise and predictable faults - effectively allowing it to hide malicious instruction in a legitimate code. To design this, the attacker could create a more “prone-to-errors” region of the code (e.g., by performing many loops in a specific alignment). The attacker also knows that it is likely the fault value will be a byte-flip. By studying of GPU opcodes and their inverse, the attacker can then craft his own command in the misread CUDA code. Similar technique can be used to leverage the faults to modification of the Program Counter register.

Other GPUs. Our tests were conducted on an Nvidia GPU, similar work can be carried out to characterize the faults on other GPUs.