Poisonous Label Attack: Black-Box Data Poisoning Attack with Enhanced Conditional DCGAN

Abstract

Data poisoning is identified as a security threat for machine learning models. This paper explores the poisoning attack against the convolutional neural network under black-box conditions. The proposed attack is “black-box,” which means the attacker has no knowledge about the targeted model’s structure and parameters when attacking the model, and it uses “poisonous-labels” images, fake images with crafted wrong labels, as poisons. We present a method for generating “poisonous-label” images that use Enhanced Conditional DCGAN (EC-DCGAN) to synthesizes fake images and uses asymmetric poisoning vectors to mislabel them. We evaluate our method by generating “poisonous-label” images from MNIST and FashionMNIST datasets and using them to manipulate image classifiers. Our experiments demonstrate that, similarly to white box data poisoning attacks, the poisonous label attack can also dramatically increase the classification error.

Appendices

Appendix A: Proof of Proposition 1

Preliminaries Before the poisoning samples are injected into the training dataset $D_{train}^*$, let $P^* (y_r\vert x_r )$ donate the conditional probability of the sample $x_r$ is correctly labeled as $y_r$, $P_{\varphi }^* (y_r\vert x_r,\omega )$ donate the predicted conditional probability of the sample $x_r$ is classified as $y_r$ by the network $\varphi$ parameterized by $\omega$. After the poisoning samples are injected into the training dataset $(D_{train}^*$ becomes$D_{train})$, let $P(y_r\vert x_r )$ donate the conditional probability of the sample $x_r$ is correctly labeled as $y_r$. $P_{\varphi }(y_r\vert x_r,{\hat{\omega }} )$donate the predicted conditional probability of the sample $x_r$ is classified as $y_r$ by the network $\varphi$ parameterized by ${\hat{\omega }}$.

Conjecture 1

After injecting poisoning samples that satisfy $\exists y_p\ne y_r$ into a training dataset, the conditional probability of the sample $x_r$ is correctly labeled as $y_r$ will decrease, we have

$$\begin{aligned} P^{*}\left( y_{r} | x_{r}\right) -\mathrm {P}\left( y_{r} | x_{r}\right) >a \end{aligned}$$

(12)

where $1>a>0$, and is constant.

Conjecture 1 is true in most situations, e.g., if we randomly label the samples in a test dataset and add them into a training dataset, the probability of getting a wrong labeled sample will be higher. Constant a in 1 can be formulated as follow:

$$\begin{aligned} \mathrm {a}=\frac{\alpha \beta R}{p\left( x_{r}\right) +\alpha \beta R} \end{aligned}$$

(13)

where $p(x_r)$ is the probability of the sample $x_r$ appears in the training dataset, $\alpha$ is the coefficient which is determined by the global coherence across fake images $x_p$, $\beta$ is the coefficient which is determined by the poisoning matrix H and R is the percentage of poisoning samples in the training dataset. From the perspective of probability theory, the coefficients can be formulated as follow:

$$\begin{aligned}&\alpha =p\left( x_{r} | x_{p}\right) \end{aligned}$$

(14)

$$\begin{aligned}&\beta =1-\mathrm {p}\left( y_{r} | y_{p}\right) \end{aligned}$$

(15)

$$\begin{aligned}&\mathrm {R}=\mathrm {p}\left( x_{p}\right) \end{aligned}$$

(16)

The Learning Ability of the network should also be considered when proving the feasibility of poisonous label attacks. In the following Lemma 1, we formulate the learning error of the network $\varphi$.

Lemma 1

If the network $\varphi$ is a PAC-learning algorithm,the training dataset and the test dataset have the same probability distribution. After training T epoch, the learning error $K^*$ of network $\varphi$ on training dataset $D_{train}^*$ is

$$\begin{aligned} \lim _{T \rightarrow \infty } K^{*}=\left| P^{*}\left( y_{r} | x_{r}\right) -P_{\varphi }^{*}\left( y_{r} | x_{r}, \omega \right) \right| =0 \end{aligned}$$

(17)

the learning error K of network $\varphi$ on training dataset $D_{train}$ is

$$\begin{aligned} \lim _{T \rightarrow \infty } K=\left| \mathrm {P}\left( y_{r} | x_{r}\right) -P_{\varphi }\left( y_{r} | x_{r}, {\widehat{\omega }}\right) \right| =0 \end{aligned}$$

(18)

Proof

It is a corollary of Probably Approximately Correct framework [42]. $\square$

Claim 1 reveals that if a network is good enough, after running infinite time, it can learn the conditional probability distribution perfectly, these two assumptions approximately hold when the network is a deep neural network.

Proposition 2

If the network $\varphi$ is strongly learnable and the training dataset and the test dataset have the same probability distribution, after injecting poisoning samples that satisfy $\exists y_p\ne y_r$ into training dataset and training T epoch, we have

$$\begin{aligned} \lim _{T \rightarrow \infty } P_{\varphi }^*(y_r\vert x_r,\omega )-P_{\varphi }(y_r\vert x_r,{\hat{\omega }})>a \end{aligned}$$

(19)

Proof

Based on the definition of learning error $K^*$, K, we have

$$\begin{aligned} P_{\varphi }^{*}\left( y_{r} | x_{r}, \omega \right) -P_{\varphi }\left( y_{r} | x_{r}, {\widehat{\omega }}\right) =P^{*}\left( y_{r} | x_{r}\right) -\mathrm {P}\left( y_{r} | x_{r}\right) \pm K \pm K^{*} \end{aligned}$$

(20)

Hence, Inequation (1) follows from

$$\begin{aligned} P_{\varphi }^{*}\left( y_{r} | x_{r}, \omega \right) -P_{\varphi }\left( y_{r} | x_{r}, {\widehat{\omega }}\right) >a-K-K^{*} \end{aligned}$$

(21)

Based on Claim 1, when $T\rightarrow \infty$, Inequation (5) follows from

$$\begin{aligned} \lim _{T \rightarrow \infty } P_{\varphi }^{*}\left( y_{r} | x_{r}, \omega \right) -P_{\varphi }\left( y_{r} | x_{r}, {\widehat{\omega }}\right) >a \end{aligned}$$

(22)

$\square$

Proposition 1 indicate that the test accuracy will decrease a after inject poisoning samples into a dataset. However, our experiments show that Prop 1. rarely hold when using random noise to label the fake images that generated by conditional GAN (cGAN) as poisoning samples. We think two reasons may lead to this result: (1) When the fake images don’t exhibit global coherence with the images in a training dataset, injecting them into training dataset will change the probability distribution of training dataset which lead to the result that the assumption test dataset and training dataset have the same distribution doesn’t hold true. (2) The deep neural network is robust to the random label noise [11]. So, in Sects. 4.1 and 4.2, we introduce two key methods: (1) Enhanced Conditional DCGAN to synthesize fake images exhibiting global coherence. (2) The asymmetric poisoning vector generates poisonous labels, which is more harmful to deep neural networks than random label noise (symmetric poisoning vector).

Fig. 7

Test accuracy w.r.t the optimization goals. a Test accuracy w.r.t the loss function goal. b Test accuracy w.r.t the probability goal. A positive correlation is found between probability goal and test accuracy

Fig. 8

a Test accuracy curves of symmetric poisonous label attack using different image synthesis models as a generator. b Test accuracy curves of asymmetric poisonous label attack using different image synthesis models as a generator

Fig. 9

Test accuracy curves of symmetric poisonous label attack and asymmetric poisonous label using EC-DCGAN as a generator. Test accuracy declined sharply in asymmetric poisonous label attack

Appendix B: Experiment Results About the Parameters of Poisonous Label Attack

1.1 Experiment Evaluation of Changing Optimization Goal

The results of the correlational analysis of optimization goals and test accuracy are shown in Fig. 7. As shown in Fig. 7a, no significant correlation was found between Loss function and test accuracy. The test accuracy is not likely to decrease when the Loss value increase. However, a positive correlation was found between test accuracy and probability $P_{\varphi } (y_r \vert x_r,{\hat{\omega }})$. Figure 7b reveals that there has been a marked increase for test accuracy when probability $P_{\varphi } (y_r \vert x_r,{\hat{\omega }})$ increase. Compare with two results. It can be seen that: compared with loss value $L(D_r,{\hat{\omega }})$, probability $P_{\varphi } (y_r \vert x_r,{\hat{\omega }})$ is more suitable for optimization goal.

1.2 Importance of Image Synthesis Model

To examined the impact of image synthesis models, we used cGAN(${\overline{\alpha }}=0.362$), cDCGAN(${\overline{\alpha }}=0.509$), EC-DCGAN(${\overline{\alpha }}=0.926$), ACGAN(${\overline{\alpha }}=0.923$) to generate fake images in poisonous label attack. The target class is 0. Figure 8a shows the test accuracy of the target class as the symmetric poisonous label attack goes on. The data in Fig. 8a shows that the test accuracy sharply decreases when using EC-DCGAN and ACGAN as a generator in the symmetric poisonous label attack. Figure 8b shows the test accuracy of the target class as asymmetric poisonous label attacks go on. What can be seen in this figure is the test accuracy declines faster when using EC-DCGAN as a generator in asymmetric poisonous label attack. Overall, these results indicate that the global coherence of fake images directly influences the effect of poisonous label attack, and EC-DCGAN is most suitable for poisonous label attack.

1.3 Comparison of Two Kinds of Poisoning Vector

To examined the impact of poisoning vectors, we used EC-DCGAN as a generator and used symmetric poisoning vector ($\beta =0.9,h=1$), asymmetric poisoning vector ($\beta =1,h=0.913$) to generate poisonous label. The target class is also 0. Figure 9 shows the test accuracy of the target class as the attack goes on. What stands out in the figure is the accuracy degradation of asymmetric poisoning vector (red line) is much faster than the symmetric poisoning vector (blue line). The result indicates that the asymmetric poisoning vector is better than the symmetric poisoning vector. Besides, this result also demonstrates that poisonous labels generated by asymmetric poisoning vector are more threatening for deep neural networks compared with random noise labels.

Appendix C: Architecture of EC-DCGAN and the L1 Loss of Training EC-DCGAN

See Table 6 and Fig. 10.

Table 6 EC-DCGAN ARCHITECTURE

Fig. 10

The L1 Loss of training EC-DCGAN. The L1 Loss decrease at the beginning of the training