TransNet: Shift Invariant Transformer Network for Side Channel Analysis

Abstract

Deep learning (DL) has revolutionized Side Channel Analysis (SCA) in recent years. One of the major advantages of DL in the context of SCA is that it can automatically handle masking and desynchronization countermeasures, even while they are applied simultaneously for a cryptographic implementation. However, the success of the attack strongly depends on the DL model used for the attack. Traditionally, Convolutional Neural Networks (CNNs) have been utilized in this regard. This work proposes to use Transformer Network (TN) for attacking implementations secured with masking and desynchronization. Our choice is motivated by the fact that TN is good at capturing the dependencies among distant points of interest in a power trace. Furthermore, we show that TN can be made shift-invariant which is an important property required to handle desynchronized traces. Experimental validation on several public datasets establishes that our proposed TN-based model, called TransNet, outperforms the present state-of-the-art on several occasions. Specifically, TransNet outperforms the other methods by a wide margin when the traces are highly desynchronized. Additionally, TransNet shows good attack performance against implementations with desynchronized traces even when it is trained on synchronized traces.

Appendices

A Proof of Lemma 1

The attention probabilities in the self-attention layer of $\text {TN}_{\text {1L}}$ is calculated following Eqs. (8) and (9). If we set $\textrm{W}_Q$, $\textrm{W}_K$, $\{\textbf{r}_i\}_{i\ne l}$ all to zero of appropriate dimensions, $\textbf{r}_l=c\sqrt{d_k}\textbf{1}$ and $\textbf{t}=\textbf{1}$ where $\textbf{1}$ is a vector whose only first element is 1 and rest are zero, and c is a real constant in Eq. (8) and Eq. (9), we have $p_{ij}$ equals to $\frac{e^c}{e^c+n-1}$ if $j=i+l$ and $\frac{1}{e^c+n-1}$ otherwise for $0\le i<n-l$. Setting $c> \text {ln}\left( \frac{1-\epsilon }{\epsilon }\right) +\text {ln}(n-1)$, we get $p_{i,i+l}>1-\epsilon$ for all $0\le i < n-l$ and $0<\epsilon <1$. Similarly, it is straight forward to show that $p_{ij}=1/n$ for any $n-l\le i< n$ and $0\le j < n$ for the same value of the parameters.

B Proof of Proposition 1

From the Eqs (11), we have $\textbf{U}_i=\textbf{Y}_i+\textbf{X}_i$, $\textbf{U}''_i=\text {FFN}(\textbf{U}_i)+\textbf{U}_i$, for $i=0, \cdots , n-1$ where $\textbf{Y}_0, \textbf{Y}_1, \cdots , \textbf{Y}_{n-1} = {RelPositionalSelfAttention}(\textbf{X}_0, \textbf{X}_1, \cdots , \textbf{X}_{n-1})$. And the output of $\text {TN}_{\text {1L}}$ is given by $\text {TN}_{1\text {L}}(\textbf{X}_0,\cdots ,\textbf{X}_{n-1}) = \frac{1}{n} \sum _{i=0}^{n-1} \textbf{U}''_i$.

From Eq. (4) and (5), we get $\textbf{Y}_j = \textrm{W}_O\left( \sum _{k=0}^{n-1} P_{jk}\textrm{W}_V\textbf{X}_k \right)$. Thus, we can write $\textbf{Y}_{m_1}$ (where $m_1$ is defined in Assumption 1) as

$$\begin{aligned} \textbf{Y}_{m_1}&= \textrm{W}_O\left( \sum _{k=0}^{n-1} P_{m_1k}\textrm{W}_V\textbf{X}_k \right) = \textrm{W}_O\textrm{W}_V\textbf{X}_{m_1+l}, \text { and thus} \end{aligned}$$

(a2)

$$\begin{aligned} \textbf{U}_{m_1}&= \textrm{W}_O\textrm{W}_V\textbf{X}_{m_1+l} + \textbf{X}_{m_1} \end{aligned}$$

(a3)

Equation (a2) follows since $i=m_1$ satisfies $P_{i,i+l}=1$ in Assumption 3. Similarly we can write $\textbf{Y}_{i}$ for $0\le i< n-l, i\ne m_1$ as

$$\begin{aligned} \textbf{Y}_{i}&= \textrm{W}_O\left( \sum _{k=0}^{n-1} P_{ik}\textrm{W}_V\textbf{X}_k \right) = \textrm{W}_O\textrm{W}_V\textbf{X}_{i+l}, \text { and thus} \end{aligned}$$

(a4)

$$\begin{aligned} \textbf{U}_i&= \textrm{W}_O\textrm{W}_V\textbf{X}_{i+l} + \textbf{X}_i \end{aligned}$$

(a5)

For $n-l \le i <n$, we can write

$$\begin{aligned} \textbf{Y}_{i}&= \frac{1}{n}\textrm{W}_O\textrm{W}_V\sum _{k=0}^{n-1}\textbf{X}_{k}\quad \text { and, } \textbf{U}_{i} = \frac{1}{n}\textrm{W}_O\textrm{W}_V\sum _{k=0}^{n-1}\textbf{X}_{k} + \textbf{X}_i \end{aligned}$$

since, by Assumption 3, $P_{ij} = 1/n$ for $j=0, \cdots , n-1$ and $n-l\le i<n$. Now we compute $\textbf{U}''_i$ for $i=0, \cdots , n-1$.

$$\begin{aligned} \textbf{U}''_i = {FFN}(\textbf{U}_i) + \textbf{U}_i \end{aligned}$$

(a6)

Note that among all the $\{\textbf{U}''_i\}_{0\le i<n}$, only $\textbf{U}''_{m_1}$ and $\{\textbf{U}''_i\}_{n-l\le i <n}$ involve both the terms $\textbf{X}_{m_1}$ and $\textbf{X}_{m_1+l}$, thus can be dependent on the sensitive variable Z (from Assumption 1). Rest of the $\textbf{U}''_i$s are independent of Z (from Assumption 2). The output of $\text {TN}_{\text {1L}}$ can be written as

$$\begin{aligned} \text {TN}_{\text {1L}}(\textbf{X}_0, \cdots , \textbf{X}_{n-1})&= \frac{1}{n} \sum _{i=0}^n\textbf{U}''_i = \frac{1}{n}\textbf{U}''_{m_1} + \frac{1}{n}\sum _{0\le i<n-l, i\ne m_1} \textbf{U}''_i + \frac{1}{n}\sum _{n-l\le i<n }\textbf{U}''_i \end{aligned}$$

(a7)

The expectation of the output conditioned on Z can be given by

$$\begin{aligned} \mathbb {E}[\text {TN}_{\text {1L}}(\textbf{X}_0, \cdots , \textbf{X}_{n-1})|Z]&=\frac{1}{n}\mathbb {E}[\textbf{U}''_{m_1}|Z] +\frac{1}{n}\sum _{n-l\le i<n} \mathbb {E}[\textbf{U}''_i|Z]+\frac{1}{n}\sum _{0\le i<n-l, i\ne m_1} \mathbb {E}[\textbf{U}''_i] \end{aligned}$$

(a8)

The second step follows because the random variables $\{\textbf{U}_i\}_{0\le i <n-l,i\ne m_1}$ are independent of Z. To complete the proof, we compute

$$\begin{aligned}&\mathbb {E}\left[ \text {TN}_{\text {1L}}(T^s(\textbf{X}_{-n+1+m_2}, \cdots , \textbf{X}_{n-1+m_1}))|Z\right] \\&= \mathbb {E}\left[ \text {TN}_{\text {1L}}(\textbf{X}_{-s}, \cdots , \textbf{X}_{n-1-s})|Z\right] \\&= \frac{1}{n}\mathbb {E}[\textbf{U}''_{m_1}|Z] + \frac{1}{n} \sum _{n-l-s\le i<n-s} \mathbb {E}[\textbf{U}''_i|Z] + \frac{1}{n}\sum _{-s\le i<n-l-s, i\ne m_1}\mathbb {E}\left[ \textbf{U}''_i\right] \end{aligned}$$

(a9)

From Assumption 2, we get

$$\begin{aligned} \frac{1}{n}\sum _{n-l\le i<n}\mathbb {E}\left[ \textbf{U}''_{i}|Z\right]&= \frac{1}{n} \sum _{n-l-s\le i<n-s} \mathbb {E}\left[ \textbf{U}''_i|Z\right] , \\ \text {and } \frac{1}{n}\sum _{0\le i<n-l, i\ne {m_1}} \mathbb {E}\left[ \textbf{U}''_i\right]&= \frac{1}{n}\sum _{-s\le i<n-l-s, i\ne m_1}\mathbb {E}[\textbf{U}''_i] \end{aligned}$$

Thus, comparing the right hand side of Eq. (a8) and Eq. (a9) we have

$$\begin{aligned}&\mathbb {E}[\text {TN}_{\text {1L}}(\textbf{X}_0, \cdots , \textbf{X}_{n-1})|Z] = \mathbb {E}\left[ \text {TN}_{\text {1L}}(T^s(\textbf{X}_{-n+1+m_2}, \cdots , \textbf{X}_{n-1+m_1}))|Z\right] \end{aligned}$$

which completes the proof.

C Comparison with CNN Using Global Pooling Model

The state-of-the-art CNN models use a flattening layer after all the convolutional model to convert the two-dimensional feature representation into a one-dimensional feature representation. However, the use of a flattening layer reduces the shift-invariance of the CNN models resulting in their poor performance on highly desynchronized traces (ref. Fig. 4d). This section compares TransNet to a CNN model that uses global pooling instead of the flattening layer. For this purpose, we have used the same model as EffCNN (desync400) except for the flattening layer replaced by the global pooling layer. We refer to the resulting model as EffCNN+GlobalPooling. The results of EffCNN+GlobalPooling on highly desynchronized ASCAD_desync0 dataset is compared with that of TransNet in Fig. 9. The results suggest that TransNet performs significantly better than EffCNN+GlobalPooling.

Fig. 9.

Comparison of TransNet with EffCNN+GlobalPooling on the ASCAD_desync400 datasets.

Fig. 10.

Results of EffCNN. The models have been trained with profiling desync 0.

D Sensitivity of EffCNN to Profiling Desynchronization

As the experiments of TransNet in Sect. 6.6, we verify the robustness of EffCNN training to the amount of profiling desync. To verify that, we trained the EffCNN models using only synchronized traces and tested them on desynchronized traces. The results are shown in Fig. 10. From the figure, it can be seen that as the amount of desynchronization in the attack traces increases, the performance of the models gets worse rapidly, suggesting the superiority of TransNet over EffCNN when the profiling desync is significantly less than the attack desync.