A New Adaptive Learning Algorithm and Its Application to Online Malware Detection

Abstract

Nowadays, the number of new malware samples discovered every day is in millions, which undermines the effectiveness of the traditional signature-based approach towards malware detection. To address this problem, machine learning methods have become an attractive and almost imperative solution. In most of the previous work, the application of machine learning to this problem is batch learning. Due to its fixed setting during the learning phase, batch learning often results in low detection accuracy when encountered zero-day samples with obfuscated appearance or unseen behavior. Therefore, in this paper, we propose the FTRL-DP online algorithm to address the problem of malware detection under concept drift when the behavior of malware changes over time. The experimental results show that online learning outperforms batch learning in all settings, either with or without retrainings.

A Proof of Theorem 1

Proof

To remind the optimization objective of FTRL-DP:

$$w_{t+1} = \mathop {\mathrm {\arg \!\min }}_w g_{1:t}^\top w + \lambda _1\Vert w\Vert _1 + \frac{1}{2}\lambda _2\Vert w\Vert _2^2 + \frac{1}{2}\lambda _p \sum _{s=1}^{t}\gamma ^{t-s}\Vert w-w_s\Vert _2^2$$

Omitting the constant term $\frac{1}{2}\lambda _p \sum _{s=1}^{t}\gamma ^{t-s}\Vert w_s\Vert _2^2$, we have:

$$\begin{aligned} w_{t+1} = \mathop {\mathrm {\arg \!\min }}_w z_t^\top w + \lambda _1\Vert w\Vert _1 + \frac{1}{2}\big (\lambda _2 + \lambda _p r_t\big )\Vert w\Vert _2^2 \end{aligned}$$

(9)

In Eq. 9, $z_t = g_{1:t}^\top - \lambda _p\sum _{s=1}^{t}\gamma ^{t-s}w_s^\top$ and $r_t = \frac{1-\gamma ^t}{1-\gamma }$. Each component of w contribute independently to the objective function of 9 hence can be solve separately:

$$\begin{aligned} w_{t+1,i} = \mathop {\mathrm {\arg \!\min }}_{w_i} z_{t,i} w_i + \lambda _1\Vert w_i\Vert _1 + \frac{1}{2}\big (\lambda _2 + \lambda _p r_t\big )\Vert w_i\Vert _2^2 \end{aligned}$$

(10)

Note that $w_i$ in 10 refers to the $i^{\text {th}}$ component of w. Let $f(w_i) = z_{t,i} w_i + \lambda _1\Vert w_i\Vert _1 + \frac{1}{2}\big (\lambda _2 + \lambda _p r_t\big )\Vert w_i\Vert _2^2$. There are two cases:

• If $\Vert z_{t,i}\Vert _1 \le \lambda _1$, we have:

  $$f(w_i) \ge -\Vert z_{t,i} w_i\Vert _1 + \lambda _1\Vert w_i\Vert _1 + \frac{1}{2}\big (\lambda _2 + \lambda _p r_t\big )\Vert w_i\Vert _2^2$$
  $$f(w_i) \ge -\lambda _1\Vert w_i\Vert _1 + \lambda _1\Vert w_i\Vert _1 + \frac{1}{2}\big (\lambda _2 + \lambda _p r_t\big )\Vert w_i\Vert _2^2=\frac{1}{2}\big (\lambda _2 + \lambda _p r_t\big )\Vert w_i\Vert _2^2 \ge 0$$

  $f(w_i)$ achieves the minimum at $w_i = 0$

• If $\Vert z_{t,i}\Vert _1 \ge \lambda _1$, $z_{t,i}$ and $w_i$ must have opposite signs at the minimum of $f(w_i)$ as otherwise $w_i$ can always have sign flipped to further reduce $f_i(w_i)$. Therefore, it is equivalent to solving:

  $$w_{t+1,i} = \mathop {\mathrm {\arg \!\min }}_{w_i} z_{t,i} w_i - \mathrm {sign}(z_{t,i})\lambda _1\Vert w_i\Vert _1 + \frac{1}{2}\big (\lambda _2 + \lambda _p r_t\big )\Vert w_i\Vert _2^2$$

  which achieves minimum at zero gradient or $w_i = -\frac{z_{t,i} - \mathrm {sign}(z_{t,i})\lambda _1}{\lambda _2 + \lambda _p r_t}$ This concludes the proof.