An Exploration of Poisoning Attacks on Data-Based Decision Making

Abstract

Many real-world problems involve building a predictive model about an adversary and then determining a decision accordingly, including two-stage (predict then optimize) and decision focused (joint predict and optimize) approaches. The involvement of a predictive model learned from adversary’s behavior data poses a critical threat that an adversary can influence the learning process, which will ultimately deteriorate the end-goal decision quality. In this paper, we study the problem of poisoning attacks in this data-based decision making setting. That is, the adversary can alter the training data by injecting some perturbation into the data to a certain limit that can substantially change the final decision outcome in the end towards the adversary goal. To our knowledge, this is the first work that studies poisoning attacks in such data-based decision making scenarios. In particular, we provide the following main contributions. We introduce a new meta-gradient based poisoning attack for various types of predict and optimize frameworks. We compare to a technique shown effective in computer vision. We find that the complexity of the problem makes attacking decision focused model difficult. We show that an attack crafted against a two-stage model is effectively transferable to a decision-focused model.

8 Appendix

1.1 8.1 Experiment Domain - Bipartite Matching

Bipartite matching is a well established problem in graph learning. In form, it is essentially identical to the synthetic data setting previously discussed:

$$\begin{aligned} \min f(x, \theta ) = \frac{1}{2}x^T Q x - \theta ^T x \; \text { s.t. } \; ||x|| \le D, Ax \le b \end{aligned}$$

(15)

In this case, however, the constraints enforced are that x must be doubly stochastic. Intuitively, x is a square matrix with continuous values. Each value $x_{ij}$ represents the probability that node i on one side of the graph will be matched with node j on the other side. For the learning component of the problem, the goal is to predict the graph’s edges from the nodes’ features. This means that $\epsilon$ represents per-node features, while $\theta$ is the graph’s adjacency matrix (relaxed to be continuous).

For these experiments, we utilize the Cora dataset [25] which consists of scientific papers. The features here are binary values indicating the presence of keywords in the paper, while the edges in the graph are citations. In total, there are 1433 features and 2708 nodes. Inspired by a recent paper [30], we split the dataset into 27 bipartite subgraphs, with 50 nodes on each side in each subgraph. This is accomplished using Metis [13] to partition the graph.

1.2 8.2 Supplementary Experiment Results

In Fig. 9, we display the results of using Metapoison [11] to solve attacks against a simple joint learner, and transferring the found attack to the two-stage and decision focused learners. Both domains display the same trends as observed in our synthetic domain - namely, that the attack is only nominally effective against the simple joint model, and not at all effective when transferred to the other two models. Once again, this suggests that techniques from domains such as computer vision may not be most appropriate for attacking data-based decision making models.

Figure 10 shows the results when attacking two-stage and decision focused models for bipartite matching. The trends are once again similar to the other domains: attacks trained against a two-stage learner can effectively transfer to a decision focused learner. Furthermore, as in portfolio optimization, we observe that the decision focused learner appears more susceptible to direct attack (Fig. 10b) than is the two-stage learner (Fig. 10a). Once again, this is likely due to the decision focused learner outperforming the two-stage counterpart in the absence of attack. Unattacked, the two-stage learner achieves utility values between 2.37 and 2.90 while the decision focused learner obtains utilities between 2.65 and 4.59. Particularly when attacking the decision focused learner (Fig. 10b) we can observe the recurring trend of increased attack budgets often leading to worse attacks and higher utility for the learner, demonstrating the difficulties of finding good attack optima via (meta)gradient descent.

Fig. 10.

Attacking a bipartite matching model