On the Influence of Optimizers in Deep Learning-Based Side-Channel Analysis

Abstract

The deep learning-based side-channel analysis represents a powerful and easy to deploy option for profiling side-channel attacks. A detailed tuning phase is often required to reach a good performance where one first needs to select relevant hyperparameters and then tune them. A common selection for the tuning phase are hyperparameters connected with the neural network architecture, while those influencing the training process are less explored.

In this work, we concentrate on the optimizer hyperparameter, and we show that this hyperparameter has a significant role in the attack performance. Our results show that common choices of optimizers (Adam and RMSprop) indeed work well, but they easily overfit, which means that we must use short training phases, small profiling models, and explicit regularization. On the other hand, SGD type of optimizers works well on average (slower convergence and less overfit), but only if momentum is used. Finally, our results show that Adagrad represents a strong option to use in scenarios with longer training phases or larger profiling models.

Appendices

A Machine Learning Classifiers

We consider two neural network types that are standard techniques in the profiling SCA: multilayer perceptron and convolutional neural networks.

Multilayer Perceptron. The multilayer perceptron (MLP) is a feed-forward neural network that maps sets of inputs onto sets of appropriate outputs. MLP consists of multiple layers of nodes in a directed graph, where each layer is fully connected to the next one (thus, layers are called fully-connected or dense layers). An MLP consists of three or more layers (since input and output represent two layers) of nonlinearly-activating nodes [5].

Convolutional Neural Networks. Convolutional neural networks(CNNs) are feed-forward neural networks commonly consisting of three types of layers: convolutional layers, pooling layers, and fully-connected layers. The convolution layer computes neurons’ output connected to local regions in the input, each computing a dot product between their weights and a small region connected to the input volume. Pooling decrease the number of extracted features by performing a down-sampling operation along the spatial dimensions. The fully-connected layer (the same as in multilayer perceptron) computes either the hidden activations or the class scores.

B Neural Networks Hyperparameter Ranges

In Table 1, we show the hyperparameter ranges we explore for the MLP architectures, while in Table 2, we show the hyperparameter ranges for CNNs.

Table 1. Hyperparameter search space for multilayer perceptron.

Table 2. Hyperparameters search space for convolutional neural network (l = convolution layer index).

In our experiments, training, validation, and test side-channel traces are normalized using z-score normalization:

$$\begin{aligned} t_{i} = \frac{t_{i} - \mu _{T_{train}}}{\sigma _{T_{train}}}, \end{aligned}$$

(1)

where $T_{train}$ is the set of training traces and $t_{i}$ is a single trace from training, validation, or test sets. Note that all traces are normalized according to the training set. For all configured neural networks, there are no batch normalization layers.