Elsevier

Applied Soft Computing

Volume 7, Issue 3, June 2007, Pages 631-641
Applied Soft Computing

Hybrid multi-agent framework for detection of stealthy probes

https://doi.org/10.1016/j.asoc.2005.12.002Get rights and content

Abstract

Probing tools are widely used to discover system information. Once the information is known, attackers can launch computer attacks against the vulnerable services running on the system. Even though current computer systems are protected against known attacks by implementing a number of access restriction policies, protection against novel attacks still remains as an elusive goal for the researchers. Attackers defeat current protection and detection mechanisms by exploiting unknown weakness and bugs in system and application software. Stealthy and low profile probes that include only a few carefully crafted packets over an extended period of time are used to delude firewalls and intrusion detection systems (IDS).

Building effective IDSs, unfortunately, has remained an elusive goal owing to the great technical challenges involved and applied AI techniques are increasingly being utilized in attempts to overcome the difficulties. This paper presents computational intelligent agents-based approach to detect computer probes at the originating host. We also investigate and compare the performance of different classifiers used for detecting probes, with respect to the data collected on a real network that includes a variety of simulated probe attacks and the normal activity.

Through a variety of experiments and analysis, it is found that with appropriately chosen network features computer probes can be detected in real time or near real time at the originating host. Using the detection information, an effective response mechanism can be implemented at the boundary controllers.

Introduction

Intrusion detection is a problem of great significance to protecting information systems security, especially in view of the worldwide increasing incidents of cyber attacks on the critical infrastructures. Since the ability of an IDS to classify a large variety of intrusions in real time or in near real time with accurate results is important, we will consider performance measures in critical aspects like training and testing times; scalability; and classification accuracy.

One of the main problems with IDSs is the overhead, which can become unacceptably high. To analyze system logs, the operating system must keep information regarding all the actions performed, which invariably results in huge amounts of data, requiring disk space and CPU resource. Next, the logs must be processed and converted into a manageable format and then compared with the set of recognized misuse and attack patterns to identify possible security violations. Further, the stored patterns need be continually updated, which would normally involve human expertise. Constructing probing signatures is a bit more complex as the attackers use carefully crafted packets over a period of time that appear to be like normal network traffic. Traditional port scan detectors look for packets coming from several IP addresses with combination of different ports with in a defined time to a target machine, which involves computation to analyze packet headers to search for signatures. In this paper, we present a novel approach of detecting probes at the originating machine using computational intelligent agent-based techniques that reduces the computation at the server and efficiently deal with distributed attacks.

Several artificial intelligence techniques have been utilized to automate the intrusion detection process to reduce human intervention; several such techniques include neural networks [1], [2], [3], [4], [5], [6], fuzzy inference systems, evolutionary computation machine learning [7], [8], and so on. Several data mining techniques have been introduced to identify key features or parameters that define intrusions [9], [10], [11], [12]. A summary of intrusion detection techniques is given by several research works [13], [14]. Some works applied neural networks as classifiers to detect low level probes and a summary of different port-scan detection techniques is also available [15], [16].

In this paper, we implement and evaluate the performance of computational intelligent multi-agent system to detect computer probes at the originating host. Intelligent agents were encapsulated with different AI paradigms involving support vector machines (SVM), multi-variate adaptive regression splines (MARS) and linear genetic programming (LGP) for detecting probes. Performance metrics include critical aspects of intrusion detection such as scalability, real time detection and accuracy. The IDS metric of real-time performance capability is not considered directly since it depends on the actual implementation. The data we use in our experiments is collected on a real performing network at New Mexico Technology, USA that includes normal activity and several classes of probing attacks generated using an open source tool Network Mapper (Nmap). We perform experiments to classify the network traffic sessions into “Normal” and “Probe”. With appropriately chosen population size, program size, crossover rate and mutation rate, linear genetic programs outperform other artificial intelligent techniques in terms of detection accuracy. The experimental results of overall classification accuracy- and class-specific accuracies using SVM, MARS and LGP are reported.

A brief introduction to our computational intelligent agents-based architecture is given in Section 2. Data generation and collection is described in Section 3 of this paper. Section 3 also briefly explains the different AI paradigms we used for classifying normal activity and probes. In Section 4, we briefly describe the offline data analysis and feature extraction for real time detection of probes at the originating host. Real-time data collection and feature extraction are described in Section 5. Implementation details of the computationally intelligent multi-agent system are given in Section 6. Experimental results of using SVM, MARS and LGP as classifiers are given in Section 7. The summary and conclusions of our work are given in Section 8.

Section snippets

Computational intelligent agents (CIA)-based architecture

The CIA-based architecture for detecting computer attacks consists of several modules that will be executed by the agents in a distributed manner. Communication among the agents is done utilizing the TCP/IP sockets. Agent modules running on the host computers consist of data collection agents, data analysis agents and response agents. Agents running on the secure devices consist of the agent control modules that include agent regeneration, agent dispatch, maintaining intrusion signatures and

Data mining: a computational intelligence approach

Data mining (also known as knowledge discovery in databases, KDD) has been defined by Frawley as “The nontrivial extraction of implicit, previously unknown and potentially useful information from data”. Data mining techniques use machine learning, statistical and visualization techniques to discover and present knowledge from the raw information in a easily comprehensible form to humans. In the field of intrusion detection data mining programs are used to analyze audit trails and provide

Offline feature extraction and evaluation

A sub set of the DARPA intrusion detection data set is used for offline analysis. In the DARPA intrusion detection evaluation program, an environment was set up to acquire raw TCP/IP dump data for a network by simulating a typical U.S. Air Force LAN. The LAN was operated like a real environment, but being blasted with multiple attacks [25], [26]. For each TCP/IP connection, 41 various quantitative and qualitative features were extracted [9]. The 41 features extracted fall into three

Real-time data collection and feature extraction

Experiments are performed on a real network using two clients and the server that serves the New Mexico Tech Computer Science Department network. The clients had CIA installed on them to identify or detect probes that are targeted to the server we are protecting. Our primary goal in these experiments is to detect probes targeting the server we are trying to protect. Our network parser gives the summary of each connection made from a host to the server and constructs a feature set to input into

CIA system and implementation

Computer probes that are intended to discover information of a computer system can be detected by careful observation of network packets. Probing tools in an effort to identify host information send connection requests to closed ports and non-existing systems. Knowledge of how a network and its hosts are being used will help in distinguishing between normal activity and probes. The primary goal of CIA is to detect probes at the host level. The implemented host agent comprises three components:

Evaluation

Network packets contain information of protocol and service used to establish connection between a client and the server. Network services have an expected number of bytes of data to be passed between a client and the server. If data flow is too little or too much it indicates a suspicion in a connection established that indicates a misuse in service. Using this information normal and probing activities can be separated. In our evaluation, we perform binary classification (normal/probe). The

Conclusions

Computational intelligent agents-based system that is capable of detecting stealthy probes at the host level is being implemented and the results obtained are demonstrated in this paper. A comparison of different computational intelligent techniques is also given. Linear genetic programming technique outperformed SVM and MARS with a 100% detection rate on the test dataset.

The proposed multi-agent framework can operate asynchronously and in parallel, and hence could be useful in monitoring huge

Acknowledgments

Support for this research received from Institute for Complex Additive Systems Analysis (ICASA, a division of New Mexico Tech), Department of Defense and NSF IASP capacity building grant is gratefully acknowledged.

References (28)

  • A.K. Ghosh

    Learning program behavior profiles for intrusion detection

  • J. Cannady

    Applying neural networks for misuse detection

  • J. Ryan et al.

    Intrusion detection with neural networks

  • H. Debar et al.

    A neural network component for an intrusion detection system

  • H. Debar et al.

    An application of a recurrent network to an intrusion detection system

  • S. Mukkamala et al.

    Intrusion detection using neural networks and support vector machines

  • J. Luo, S.M. Bridges, Mining fuzzy association rules and fuzzy frequency episodes for intrusion detection,...
  • M. Cramer

    New methods of intrusion detection using control-loop measurement

  • J. Stolfo, F. Wei, W. Lee, A. Prodromidis, P.K. Chan, Cost-based modeling and evaluation for data mining with...
  • S. Mukkamala et al.

    Identifying key features for intrusion detection using neural networks

  • S. Mukkamala, A.H. Sung. Feature selection for intrusion detection using neural networks and support vector machines,...
  • S. Mukkamala, A.H. Sung, Identifying significant features for network forensic analysis using artificial intelligence...
  • D. Denning

    An intrusion-detection model

    IEEE Trans. Software Eng.

    (1987)
  • S. Kumar, E.H. Spafford, An application of pattern matching in intrusion detection, Technical Report CSD-TR-94-013,...
  • Cited by (13)

    • Multi-agent modeling for solving profit based unit commitment problem

      2013, Applied Soft Computing Journal
      Citation Excerpt :

      In former approach, a system of multi-agents is required for those applications where agents can respond correctly in a changing environment, agents can be added as and when required, agents can be replaced in other system or be upgraded, and for graceful degradation of system when one or more agents fail. A few such applications can be found in [20–23]. In modeling approach, MAS is used to represent a large and complex system which is difficult to model explicitly.

    • Anomaly Detection Using Hybrid Neuro Genetic Model

      2022, Journal of Interconnection Networks
    • Genetic programming with linear representation: A survey

      2009, International Journal on Artificial Intelligence Tools
    • Association rule-mining-based intrusion detection system with entropy-based feature selection: Intrusion detection system

      2019, Handbook of Research on Intelligent Data Processing and Information Security Systems
    • Unknown malware attack by using intelligence intrusion multi detection systems

      2015, International Journal of Applied Engineering Research
    View all citing articles on Scopus
    View full text