Skip to main content

Game Theoretical Model for Adaptive Intrusion Detection System

  • Chapter
  • First Online:
Transactions on Computational Collective Intelligence XV

Part of the book series: Lecture Notes in Computer Science ((TCCI,volume 8670))

  • 545 Accesses

Abstract

We present a self-adaptation mechanism for network intrusion detection system based on the use of game-theoretical formalism. The key innovation of our method is a secure runtime definition and solution of the game and real-time use of game solutions for immediate system reconfiguration. Our approach is suited for realistic environments where we typically lack any ground truth information regarding traffic legitimacy/maliciousness and where the significant portion of system inputs may be shaped by the attacker in order to render the system ineffective. Therefore, we rely on the concept of challenge insertion: we inject a small sample of simulated attacks into the unknown traffic and use the system response to these attacks to define the game structure and utility functions. This approach is also advantageous from the security perspective, as the manipulation of the adaptive process by the attacker is far more difficult.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    These two matrices can be collapsed into a single one in case of the zero sum game, where the gain of one of the player is directly translated into the equivalent loss of the other player.

  2. 2.

    The only situations where this term is actually globally positive are those where an efficient counter-attack mechanism (in tactical/military problems) or intelligence-processing mechanism allows the defender to counter-attack the attacker’s resources or to deduce attacker’s goals, plans or at least intentions. From the other side of the problem, the attacker needs to structure its actions in such a way, that their eventual compromise would not give away disproportionally high volume of information about its goals or resources. This consideration is integrated in the value of the term \(D_a(a_j)\).

  3. 3.

    If \(p = n\), the concept is trivially equivalent to strict dominance. For sake of notation clarity, we arbitrarily select the first \(p\) attacker strategies, with no loss of generality.

  4. 4.

    Challenges are prerecorded sets of network traffic that are manually labeled as legitimate or malicious and can be seen as training samples.

  5. 5.

    The slight delay of application is unlikely to cause a problem, as suggested by our experimental results. The system using the parameters weighted over 5 last intervals performed comparably with the one using only the precise values for the specific interval.

References

  1. Kayacik, H.G., Zincir-Heywood, A.N.: Mimicry attacks demystified: what can attackers do to evade detection? In: Annual Conference on Privacy, Security and Trust, pp. 213–223 (2008)

    Google Scholar 

  2. Rubinstein, B.I.P., Nelson, B., Huang, L., Joseph, A.D., Lau, S., Taft, N., Tygar, J.D.: Evading anomaly detection through variance injection attacks on PCA. In: Lippmann, R., Kirda, E., Trachtenberg, A. (eds.) RAID 2008. LNCS, vol. 5230, pp. 394–395. Springer, Heidelberg (2008)

    Chapter  Google Scholar 

  3. Barreno, M., Nelson, B., Sears, R., Joseph, A.D., Tygar, J.D.: Can machine learning be secure? In: ASIACCS ’06: Proceedings of the 2006 ACM Symposium on Information, Computer and Communications Security, pp. 16–25. ACM, New York (2006)

    Google Scholar 

  4. Rehák, M., Staab, E., Fusenig, V., Pěchouček, M., Grill, M., Stiborek, J., Bartoš, K., Engel, T.: Runtime monitoring and dynamic reconfiguration for intrusion detection systems. In: Kirda, E., Jha, S., Balzarotti, D. (eds.) RAID 2009. LNCS, vol. 5758, pp. 61–80. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  5. Nisan, N., Roughgarden, T., Tardos, E., Vazirani, V.V.: Algorithmic Game Theory. Cambridge University Press, New York (2007)

    Book  MATH  Google Scholar 

  6. Blum, A., Mansour, Y.: Learning, regret minimization and equilibria. In: Nisan, N., Roughgarden, T., Tardos, E., Vazirani, V. (eds.) Algorithmic Game Theory, pp. 79–101. Cambridge University Press, New York (2007)

    Chapter  Google Scholar 

  7. Alpcan, T., Başar, T.: A game theoretic approach to decision and analysis in network intrusion detection. In: Proceedings of the 42nd IEEE Conference on Decision and Control, Maui, HI, pp. 2595–2600, December 2003

    Google Scholar 

  8. Alpcan, T., Başar, T.: An intrusion detection game with limited observations. In: 12th International Symposium on Dynamic Games and Applications, Sophia Antipolis, France, July 2006

    Google Scholar 

  9. Liu, Y., Comaniciu, C., Man, H.: A bayesian game approach for intrusion detection in wireless ad hoc networks. In: GameNets ’06: Proceeding from the 2006 Workshop on Game Theory for Communications and Networks, p. 4. ACM, New York (2006)

    Google Scholar 

  10. Chen, L., Leneutre, J.: A game theoretical framework on intrusion detection in heterogeneous networks. IEEE Trans. Inf. Forensics Secur. 4(2), 165–178 (2009)

    Article  Google Scholar 

  11. Zhu, Q., Basar, T.: Dynamic policy-based IDS configuration. In: Joint 48th IEEE Conference on Decision and Control and 28th Chinese Control Conference, pp. 8600–8605 (2009)

    Google Scholar 

  12. Jain, M., Pita, J., Tambe, M., Ordónez, F., Paruchuri, P., Kraus, S.: Bayesian stackelberg games and their application for security at Los Angeles international airport. SIGecom Exch. 7(2), 1–3 (2008)

    Article  Google Scholar 

  13. Becker, G.S.: Crime and punishment: an economic approach. J. Polit. Econ. 76(2), 169–217 (1968)

    Article  Google Scholar 

  14. Ptacek, T.H., Newsham, T.N.: Insertion, evasion, and denial of service: eluding network intrusion detection. Technical report, Secure Networks Inc., Suite 330, 1201 5th Street S.W., Calgary, Alberta, Canada, T2R–0Y6 (1998)

    Google Scholar 

  15. Porter, R., Nudelman, E., Shoham, Y.: Simple search methods for finding a nash equilibrium. Games Econ. Behav. 63(2), 642–662 (2008)

    Article  MathSciNet  MATH  Google Scholar 

  16. Wagener, G., State, R., Dulaunoy, A., Engel, T.: Self adaptive high interaction honeypots driven by game theory. In: Guerraoui, R., Petit, F. (eds.) SSS 2009. LNCS, vol. 5873, pp. 741–755. Springer, Heidelberg (2009)

    Chapter  Google Scholar 

  17. Rehak, M., Staab, E., Pechoucek, M., Stiborek, J., Grill, M., Bartos, K.: Dynamic information source selection for intrusion detection systems. In: Decker, K.S., Sichman, J.S., Sierra, C., Castelfranchi, C. (eds.) Proceedings of the 8th International Conference on Autonomous Agents and Multiagent Systems (AAMAS ’09), IFAAMAS, pp. 1009–1016, May 2009

    Google Scholar 

  18. Rehák, M., Pechoucek, M., Grill, M., Stiborek, J., Bartoš, K., Celeda, P.: Adaptive multiagent system for network traffic monitoring. IEEE Intell. Syst. 24(3), 16–25 (2009)

    Article  Google Scholar 

Download references

Acknowledgment

This material is based upon work supported by the ITC-A of the US Army under Contract No. W911NF-10-1-0070. Any opinions, findings and conclusions or recommendations expressed in this material are those of the author(s) and do not necessarily reflect the views of the ITC-A of the US Army. Also supported by Czech Ministry of Education grants 6840770038 and AMVIS-AnomalyNET. Also supported by MVČR Grant number VG2VS/242.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Jan Stiborek .

Editor information

Editors and Affiliations

A CAMNEP

A CAMNEP

In order to evaluate the theoretical model in a production environment, we have used presented mechanism as a component of the CAMNEP network intrusion detection system [18], which is used to detect the attacks against computer networks by means of Network Behavior Analysis (NBA) techniques. This system processes NetFlow/IPFIX data provided by routers or other network equipment and uses this information to identify malicious traffic by means of collaborative, multi-algorithm anomaly detection. The system uses the multi-algorithm and multi-stage approach to optimize the error rate, while not compromising the performance of the system. The system contains two principal classes of classifying agents, which are able to evaluate the received traffic:

1.1 A.1 Detection Agents

Detection agents analyze raw network flows by their anomaly detection algorithms, exchange the anomalies between them and use the aggregated anomalies to build and update the long-term anomaly associated with the abstract traffic classes built by each agent. Each detection agent uses its own anomaly detection method, each works with a different traffic model based on a specific combination of aggregate traffic features. All detection agents map the same flows, together with the shared evaluation of these events, the aggregated immediate anomaly of these events determined by their anomaly detection algorithms, into the traffic clusters built using different features/metrics, thus building the aggregate anomaly hypothesis based on different premises. The aggregated anomalies associated with the individual traffic classes are built and maintained using the classic trust modeling techniques (not to be confused with the way trust is used in this work). The detection agents evaluate the anomaly of each network flow on the whole [0,1] interval, and the output of the detection agents is integrated by the aggregation agents.

1.2 A.2 Aggregation Agents

Aggregation agents \(\alpha _{i}\) from the set \(A = \{\alpha _{1},\dots ,\alpha _{g}\}\) represent the various aggregation operators used to build the joint conclusion regarding the normality/anomaly of the flows from the individual opinions provided by the detection agents. Each agent uses a distinct averaging operator (based on order-weighted averaging or simple weighted averaging) to perform the \(R^{g_{det}}\rightarrow R\) transformation from the \(g_{det}\)-dimensional space to a single real value, thus defining one composite system output that integrates the results of several detection agents. The aggregation agents also dynamically determine the threshold values used to transform the continuous aggregated anomaly value in the \([0,1]\) interval into the crisp normal/anomalous assessment for each flow. The value of the threshold is either relative (i.e. leftmost part of the distribution) or absolute, based on the evaluation of the agent’s response to challenges.

The detection and aggregation agents annotate the individual flows \(\varphi \) with a continuous anomaly/normality value in the \([0,1]\) interval, with the value 1 corresponding to perfectly normal events and the value 0 to completely anomalous ones. This continuous anomaly value describes an agent’s opinion regarding the anomaly of the event, and the agents apply adaptive or predefined thresholds to split the \([0,1]\) interval into the normal and anomalous classes. The threshold used by the aggregation agents divides the flows into two classes: normal and anomalous. The anomalous flows are those whose anomaly falls below the threshold, while the normal flows are those whose anomaly is above the threshold. This distinction allows us to introduce the components of the error rate. False Positives (FP) are the legitimate flows classified as anomalous, while the False Negatives (FN) are the malicious flows classified as normal.

Rights and permissions

Reprints and permissions

Copyright information

© 2014 Springer-Verlag Berlin Heidelberg

About this chapter

Cite this chapter

Stiborek, J., Grill, M., Rehak, M., Bartos, K., Jusko, J. (2014). Game Theoretical Model for Adaptive Intrusion Detection System. In: Nguyen, N., Kowalczyk, R., Corchado, J., Bajo, J. (eds) Transactions on Computational Collective Intelligence XV. Lecture Notes in Computer Science(), vol 8670. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-44750-5_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-662-44750-5_7

  • Published:

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-662-44749-9

  • Online ISBN: 978-3-662-44750-5

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics