A mathematical exploitation of simulated uniform scanning botnet propagation dynamics for early stage detection and management

Abstract

The contribution of this paper is two-fold. Firstly, we propose a botnet detection approach that is sufficiently timely to enable a containment of the botnet outbreak in a supervised network. Secondly, we show that mathematical models of botnet propagation dynamics are a viable means of achieving that level of defense from bot infections in a supervised network. Our approach is built on the idea of processing network traffic such as to localize a weakly connected subgraph within a graph that models network communications between hosts, and thus consider that subgraph as representative of a suspected botnet. We devise applied statistics to infer the propagation dynamics that would characterize the suspected botnet if this latter were indeed a botnet. The inferred dynamics are materialized into a model graph. A subgraph isomorphism search determines whether or not there is an approximate match between the model graph and any subgraph of the weakly connected subgraph. An approximate match between the two leads to a timely identification of infected hosts. We have implemented this research in the Matlab and Perl programming languages, and have validated it in practice in the Emulab network testbed. In the paper, we describe our approach in detail, and discuss experiments along with experimental data that are indicative of the effectiveness of our approach.

Appendix

1.1 Overview of the Variant tool

The Variant tool is a workable and functional prototype implementation of the research discussed in this paper. The Variant tool produces consistent results each time it is run. Our motivation behind choosing the name Variant for this tool lies in the fact that botnet dynamics targeted by this research vary among network infections. Even the same botnet launched in the same network several times may exhibit different dynamics at each one of those individual launches. That phenomenon is due to the stochastic nature of a botnet infection process that belongs to the category of malware considered in this research. As we wrote earlier in this paper, the Variant tool takes in input Pcap files generated by Tcpdump or any other derivative network sniffing tool.

In the case of a positive botnet interception, the output produced by the Variant tool consists of a textual description of the final form of the weakly connected subgraph formed by the botnet infection process within the data graph until the moment of intervention. Typical forms of that specific output are shown in the bottom half of Fig. 5 and in the bottom half of Fig. 10. In the case of absent formation of weakly connected subgraphs within the data graph, or in the case of lack of isomorphism between the model graph and weakly connected subgraphs within the data graph, the Variant tool simply notifies that no botnets were detected. The Variant tool can function in total independence from any human intervention, and thus can conduct automatic early state botnet detection and containment tasks.

Fig. 5

Command line interface of the Variant tool along with a typical botnet search session

Nevertheless, we decided to equip the Variant tool with an interactive interface in order to enable a network security analyst to explore the internals of each processing session. That interactive interface also helped us debug the overall approach as it progressed throughout a processing session. The Variant tool provides a command line interface to the network security analyst. The available commands and their possible respective options are shown in the top half of Fig. 5. The Variant tool is designed to process simultaneously several Pcap files created by network sniffers deployed at various points in a supervised network. The netpackets command sorts entries from those Pcap files and places them in a unified file, which is then processed by other commands.

The logit model that we developed in this research can be solved via the logit command as shown in Figs. 6, 7, and  8. The botnet virulence can be estimated via the virulence command, which accepts three options, namely -i, -v, and -b. Those options allow for estimating the infection rate only, the vulnerability rate only, and both infection rate and vulnerability rate, respectively. A typical session is shown in Fig. 9. The various plots of statistical data that we used in this research to characterize the effectiveness of various components of our approach can be drawn via the plot command. That command is not required for conducting real-world botnet containment tasks. Nevertheless, we developed the functionality behind the command in question as we use the Variant tool also as a research tool.

Fig. 6

Estimation of the intercept terms and coefficient terms of the logit model

Fig. 7

Probability distribution of the optimal number of pools

Fig. 8

Reporting the optimal number of pools

Fig. 9

A session of botnet virulence estimation

The propagation command is used to display data on the botnet propagation process in a supervised network. It is accompanied by two options, namely -a and -d. The -a option displays propagation data based on reports generated by bots and gathered from the collector program during an experiment in the Emulab network testbed. That option is only needed when the Variant tool is employed as a research tool. The -d option displays propagation data that characterize botnet propagation dynamics as inferred by the Variant tool. A typical session of displaying propagation data is shown in Fig. 10. A detailed insight into the inferred dynamics can be obtained via the dynamics command. As we wrote earlier in this paper, our overall approach is equipped with an incremental auto-corrective feature.

Fig. 10

Botnet propagation characteristics as reported by bots during experimental network infections (top), and as detected from the Variant tool (bottom)

The Variant tool keeps intermediate findings as a table that it stores in a file named IntermediateResults.dat. Each entry in that table represents a specific phase or step within the inference process. The table in question is taken into account in full from the dynamics command, and thus the data displayed by that command are generated on the basis that the findings represented by entries in that table hold. In other words, the data displayed by the dynamics command correspond to the degree of progress made by the inference process, which is denoted by the last entry in the table in question. If we need to display inferred dynamics at the beginning of the inference process, we would have to remove all possible entries from the table in question.

Similarly, if we need to display inferred dynamics at the end of the inference process, we would have to have all table entries in place at the moment of running the dynamics command. The -d option allows for obtaining the infection time of each infected host as labeled by a bot number. A typical session is shown on top of Fig. 11. The -o option allows for viewing the estimated number of offsprings for each infected host along with an estimation of the botnet infectious time and an estimated period of time during which the infected host can cause other infections in the enterprise network. The -o option also allows for viewing the average number of infection attempts that are predicted to originate from each infected host as labeled by a bot number.

Fig. 11

A portion of botnet dynamics inferred statistically by the Variant tool

A typical use of the -o option is shown in the middle of Fig. 11. The variability time windows for each infected host can be displayed via the -w option, as shown on the bottom of Fig. 11. The option formed by - and a bot number allows for viewing the estimated probability distribution of the number of offsprings of the infected host labeled with that specific bot number. A typical session involving the use of that option is shown on top of Fig. 12. The -f option displays possible offsprings of each infected host in terms of bot numbers, as shown on the bottom of Fig. 12. Mutually exclusive concurrent edges, i.e. concurrent parent-child relationships between infected hosts, can be viewed via the -c option, as shown in Fig. 13.

Fig. 12

Probability distribution of the number of offsprings of an infected host (top), and possible offsprings of each infected host in terms of bot numbers (bottom)

Fig. 13

Mutually exclusive concurrent edges in the model graph

As we wrote earlier in this paper, botnet dynamics are inferred by the Matlab code within the Variant tool. The inferred dynamics are written in a file named ModelGraph. dat. The Perl code within the Variant tool reads the content of that file, and hence uses it to generate the model graph. The content in question can be viewed via the -m option, as shown in Fig. 14. Each record of the file ModelGraph.dat is formatted as follows. The first field indicates the infection time of a host labeled by a bot number that is represented by the fifth field. The third field indicates a bot number that corresponds to the host that initiated the infection. The presence of an X instead of a bot number denotes that the host that initiated the infection is not located within the perimeter of the enterprise network.

Fig. 14

File content that serves as basis for building the model graph

The sixth and the seventh fields indicate the boundaries of the variability time window that applies to the infected host. The last field indicates the estimated number of offsprings that the infected host is predicted to have after exhaustion of the vulnerable host population. The subgraph isomorphism search is conducted via the -s option, as shown on the bottom of Fig. 5. In conclusion, the network security analyst can issue the storedb command to store in a MySQL database all of the data discussed in this chapter. Those data can then be visualized via a web application, which lies outside the scope of this paper.