Adversarial classification using signaling games with an application to phishing detection

Abstract

In adversarial classification, the interaction between classifiers and adversaries can be modeled as a game between two players. It is natural to model this interaction as a dynamic game of incomplete information, since the classifier does not know the exact intentions of the different types of adversaries (senders). For these games, equilibrium strategies can be approximated and used as input for classification models. In this paper we show how to model such interactions between players, as well as give directions on how to approximate their mixed strategies. We propose perceptron-like machine learning approximations as well as novel Adversary-Aware Online Support Vector Machines. Results in a real-world adversarial environment show that our approach is competitive with benchmark online learning algorithms, and provides important insights into the complex relations among players.

Appendices

Appendix 1: Text mining for feature extraction

This appendix contains methods from text mining which have been used to extract useful features from the emails we analyzed in our application as presented in Sect. 6.

First, documents are tokenized in order to extract every word used in the message. Both a stopword removal process, and stemming of the words in the messages are realized. Second, various feature extraction methodologies are executed: Keyword extraction, SVD, and LDA. Third, the extracted features are combined in order to extract the features that fully characterize a given phishing message (see Fig. 13). Finally, feature selection methodologies are required in order to minimize the complexity of the classification task.

Fig. 13

Feature extraction flow diagram process for phishing messages

Let,

• $\varOmega$ be the set of features determined by keyword extraction algorithm.

• $\varUpsilon$ be the set of features determined by SVD.

• $\varGamma$ be the set of features determined by LDA

• $\varXi$ be the set of basic structural features.

• Then the final set of features $\mathcal {F}$ is given by,

  $$\begin{aligned} \mathcal {F} = \varXi \cup ((\varGamma \cap \varUpsilon ) \cup (\varOmega \cap \varUpsilon ) ) = \varXi \cup ( (\varGamma \cup \varOmega ) \cap \varUpsilon ) \end{aligned}$$

  (42)

As shown in Eq. (42), the final set of features $\mathcal {F}$ is a combination of structural basic features $\varXi$, independent of the other content-based feature sets $\varGamma , \varOmega$, and $\varUpsilon$. However, these feature sets are not independent of each other. They are represented by binary features, indicating whether or not a word is present in a given message.

1.1 Keyword extraction

Based on a TF-IDF representation for the set of messages, the following keyword extraction technique was applied in order to define a set of relevant features: Using clustering, the entire collection of phishing emails is grouped into segments, and a geometric mean of the weights of each word within the messages contained in a given segment is computed. Then, these words are ranked and selected for each set of messages within their respective clusters. This procedure is based on work described by Velasquez et al. (2005).

1.2 Singular value decomposition

Using the TF-IDF matrix (Salton et al. 1975), its singular value decomposition (SVD) reduces the dimension of the $Term \times Document$-space. SVD uses a new representation of the feature space, in which the underlying semantic relationship between terms and documents is revealed.

As described by Papadimitriou et al. (1998), SVD preserves the relative distances in the vector space model matrix (e.g., TF-IDF), while projecting it into a lower-dimensional semantic space model. This is similar to what principal component analysis achieves by projecting features into their principal components, maintaining the information needed for an appropriate representation of the dataset.

1.3 Probabilistic topic models

A topic model can be viewed as a probabilistic model that relates documents and words through variables which represent the main topics inferred from the text itself. In this context, a document can be looked at as a mixture of topics, represented by probability distributions which generate the words in a document given these topics. The inferring process of the latent variables, or topics, is the key component of this model, whose main objective is to learn the distribution of the underlying topics from text in a given corpus of text documents.

A leading topic model is the LDA (Bíró et al. 2009; Blei et al. 2003). LDA is a Bayesian model in which latent topics of documents are inferred from estimated probability distributions over a training data set. In LDA every topic is modeled as a probability distribution over the set of words represented by the vocabulary ($w\in \mathcal {V}$), and every document as a probability distribution over a set of topics ($\mathcal {T}$). These distributions are sampled from multinomial Dirichlet distributions.

1.4 Structural features

As presented in Basne et al. (2008), Bergholz et al. (2010) and Fette et al. (2007), the extraction of basic structural features is needed for a minimum representation of phishing messages given a certain application. In the case of phishing emails, these features are associated with structural properties of email messages, such as their links, embedded codes, and the output of traditional Spam filters. It is important to note that all these features are extracted directly from email messages.

Appendix 2: List of symbols

Throughout the paper, we used the following notation:

Notation	Explanation
A	Total number of features
N	Total number of instances or objects
$\mathbf {x}$	Set of features $\mathbf {x} = \{x_{1},\dots ,x_{A}\}$
$x_{a}$	ath feature of an object $\mathbf {x}, a\in \{1,\dots ,A\}$
y	Dependent variable
$(\mathbf {x}^{i},y_{i})$	Instance or object $i, i\in \{1,\dots ,N\}$
$\mathcal {F}_{x_{a}}$	Set of possible values for feature $x_{a}$
$\mathcal {F}_{\mathbf {x}}$	All possible values for all features $x_{a}\in \mathbf {x}$
$\mathcal {F}_{y}$	Set of target values
$\mathcal {T}$	Set of instances $\mathcal {T} = \{\mathbf {x}^{i},y_{i}\}_{i = 1}^{N}$
$\mathcal {T}_{r}$	Training dataset, subset of $\mathcal {T}$
$\mathcal {T}_{e}$	Testing dataset, subset of $\mathcal {T}$
$\mathcal {N}$	Agent Nature
$\mathcal {A}$	Agent Adversary
$\mathcal {C}$	Agent Classifier
R	Regular adversary type (non-malicious)
M	Malicious adversary type
k	Number of information sets. Total number of type of messages that can be drawn by nature
$\{I_{i}\}_{i=1}^{k}$	Information sets for types of messages
$T_{\mathcal {A}}$	Adversary types, $t\in T_{\mathcal {A}} = \{R,M\}\times \{1,\dots ,k\}$, also denoted as $T_{\mathcal {A}} = \{t_{R,j}\}_{j=1}^{k}\times \{t_{M,j}\}_{j=1}^{k}$
$T_{M}$	$\{t_{M,j}\}_{j=1}^{k}$
$T_{R}$	$\{t_{R,j}\}_{j=1}^{k}$
p(t)	Probability distribution for each type $t\in T_{\mathcal {A}}$
$\{\mathbf {x}^{1},\dots ,\mathbf {x}^{k}\}$	Set of all possible types of messages
$\varDelta _{k}$	Simplex of dimension $k-1$
$\mathcal {C}: \mathbb {R}^{A}\rightarrow \varDelta _{2}$	Classification function
$\mathcal {\phi }: \mathbb {R}^{A}\rightarrow \varDelta _{k}$	Mimetic function that changes $\mathbf {x}$ to $\mathbf {x}'$, interpreted as the probability of sending a given message i when preferred message is $l, \forall i, l \in \{\mathbf {x}^{1},\dots ,\mathbf {x}^{k}\}$
$\mu (t|\mathbf {x})$	Classifier’s belief that message is type t when observed instance is $\mathbf {x}$
$\sigma ^{*}_{\mathcal {A}}$	Optimal strategy for the Adversary
$\sigma ^{*}_{\mathcal {C}}$	Optimal strategy for the Classifier
$d: \mathbb {R}^{A}\rightarrow \{+1,-1\}$	Decision function
$h_{\tau }\in \{+1,-1\}$	Hypothesis classification for message $\tau$
$\{c_{i}\}_{i=1}^{k}$	Centroids associated to all types of messages
$\delta : \mathbb {R}^{A}\times \mathbb {R}^{A}\rightarrow \mathbb {R}$	Distance function between messages’ centroids
I(a)	Information set for action a
$\pi$	Strategy profile
$\pi _{a}$	Probability that an action a is chosen if information set I(a) is reached
$\lambda$	Parameter that controls the A-QRE algorithm
$\eta _{\epsilon }, \eta _{A}$	Factors which amplifies the distance function between messages’ centroids
$U_{\mathcal {A}}: T_{\mathcal {A}}\times \mathbb {R}^A \times \{+1,-1\}\rightarrow \mathbb {R}$	Utility function for the Adversary
$u_{\mathcal {A}}$	Baseline utility for the Adversary
$U_{\mathcal {C}}:T_{\mathcal {A}}\times \mathbb {R}^A\times \{+1,-1\}\rightarrow \mathbb {R}$	Utility function for the Classifier
$u_{\mathcal {C}}$	Baseline utility for the Classifier
$\theta _{1}$	Cases where messages are correctly classified
$\theta _{2}$	Cases where messages are incorrectly classified
$\mathcal {E}rr: \{+1,1\}\times \{+1,-1\}\rightarrow \mathbb {R}$	Error function
$\nu$	Classification threshold
$\epsilon _{M}$	Classification gain when classifying correctly a malicious message
$\epsilon _{R}$	Classification gain when classifying correctly a regular message
$\gamma _{M}$	Classification cost when classifying incorrectly a malicious message
$\gamma _{R}$	Classification cost when classifying incorrectly a regular message