Abstract
Companies and organizations typically employ different forms of intrusion detection (and prevention) systems on their computer and network resources (e.g., servers, routers) that monitor and flag suspicious and/or abnormal activities. When a possible malicious activity is detected, one or more cyber-alerts are generated with varying levels of significance (e.g., high, medium, or low). Some subset of these alerts may then be assigned to cyber-security analysts on staff for further investigation. Due to the wide range of potential attacks and the high degrees of attack sophistication, identifying what constitutes a true attack is a challenging problem. In this paper, we present a framework that allows us to derive game-theoretic strategies for assigning security alerts to security analysts. Our approach considers a series of sub-games between the attacker and defender with a state maintained between sub-games. Due to the large sizes of the action and state spaces, we present a technique that uses deep neural networks in conjunction with Q-learning to derive near-optimal Nash strategies for both attacker and defender. We assess the effectiveness of these policies by comparing them to optimal policies obtained from brute force value iteration methods, as well as other sensible heuristics (e.g., random and myopic). Our results show that we consistently obtain policies whose utility is comparable to that of the optimal solution, while drastically reducing the run times needed to achieve such policies.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Altner, D., Servi, L.: A two-stage stochastic shift scheduling model for cybersecurity workforce optimization with on call options (2016)
Brown, G.W.: Iterative solution of games by fictitious play. In: Koopmans, T.C. (ed.) Activity Analysis of Production and Allocation. Wiley, New York (1951)
Brown, M., Sinha, A., Schlenker, A., Tambe, M.: One size does not fit all: a game-theoretic approach for dynamically and effectively screening for threats. In: AAAI Conference on Artificial Intelligence (2016)
Dunstatter, N., Guirguis, M., Tahsini, A.: Allocating security analysts to cyber alerts using markov games. In: 2018 National Cyber Summit (NCS) (2018)
Ganesan, R., Jajodia, S., Cam, H.: Optimal scheduling of cybersecurity analysts for minimizing risk. ACM Trans. Intell. Syst. Technol. 8, 52 (2015)
Ganesan, R., Jajodia, S., Shah, A., Cam, H.: Dynamic scheduling of cybersecurity analysts for minimizing risk using reinforcement learning. ACM Trans. Intell. Syst. Technol. 8, 4 (2016)
Jain, M., Kardes, E., Kiekintveld, C., Ordónez, F., Tambe, M.: Security games with arbitrary schedules: a branch and price approach. In: Proceedings of AAAI (2010)
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. CoRR (2014)
Lagoudakis, M.G., Parr, R.: Value function approximation in zero-sum markov games. In: Proceedings of the Eighteenth Conference on Uncertainty in Artificial Intelligence, pp. 283–292. Morgan Kaufmann Publishers Inc., San Francisco (2002)
Lample, G., Chaplot, D.S.: Playing FPS games with deep reinforcement learning. CoRR abs/1609.05521 (2016)
Lin, L.J.: Reinforcement learning for robots using neural networks. Ph.D. thesis, Pittsburgh, PA, USA (1992)
Littman, M.: Value-function reinforcement learning in markov games. Princeton University Press (2000)
Littman, M.: Markov games as a framework for multi-agent reinforcement learning. In: Proceedings of the Eleventh International Conference on Machine Learning, pp. 157–163. Morgan Kaufmann (1994)
Ma, C., Yau, D., Lou, X., Rao, N.: Markov game analysis for attack-defense of power networks under possible misinformation. IEEE Trans. Power Syst. 28, 1676–1686 (2013)
Mnih, V., et al.: Playing atari with deep reinforcement learning. CoRR (2013)
Ponemon Institute: The cost of malware containment (2015)
Robinson, J.: An iterative method of solving a game. Ann. Math. 54(2), 296–301 (1951)
Schlenker, A., et al.: Don’t bury your head in warnings: a game-theoretic approach for intelligent allocation of cyber-security alerts. In: Proceedings of the Twenty-Sixth International Joint Conference on Artificial Intelligence, IJCAI-17, pp. 381–387 (2017)
Schlenker, A., et al.: Towards a game-theoretic framework for intelligent cyber-security alert allocation. In: Proceedings of the 3rd IJCAI Workshop on Algorithmic Game Theory, Melbourne, Australia (2017)
Shah, A., Ganesan, R., Jajodia, S., Cam, H.: A methodology to measure and monitor level of operational effectiveness of a CSOC. Int. J. Inf. Secur. 17(2) (2018)
Shu, X., Tian, K., Ciambrone, A., Yao, D.: Breaking the target: an analysis of target data breach and lessons learned. CoRR (2017)
Sinha, A., Nguyen, T., Kar, D., Brown, M., Tambe, M., Jiang, A.: From physical security to cybersecurity. J. Cybersecur. 1(1), 19–35 (2015)
Williams, J.D.: The Compleat Strategyst: Being a Primer on the Theory of Games of Strategy. Dover, New York (1986)
Xiaolin, C., Xiaobin, T., Yong, Z., Hongsheng, X.: A markov game theory-based risk assessment model for network information system. In: 2008 International Conference on Computer Science and Software Engineering, vol. 3, pp. 1057–1061, December 2008
Yin, Z., et al.: Trusts: scheduling randomized patrols for fare inspection in transit systems using game theory. In: Proceedings of the 24th IAAI, Palo Alto, CA (2012)
Zimmerman, C.: Ten strategies of a world-class cybersecurity operations center. MITRE corporate communications and public affairs (2014)
Acknowledgement
This work was supported in part by NSF grant #1814064.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Dunstatter, N., Tahsini, A., Guirguis, M., Tešić, J. (2019). Solving Cyber Alert Allocation Markov Games with Deep Reinforcement Learning. In: Alpcan, T., Vorobeychik, Y., Baras, J., Dán, G. (eds) Decision and Game Theory for Security. GameSec 2019. Lecture Notes in Computer Science(), vol 11836. Springer, Cham. https://doi.org/10.1007/978-3-030-32430-8_11
Download citation
DOI: https://doi.org/10.1007/978-3-030-32430-8_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-32429-2
Online ISBN: 978-3-030-32430-8
eBook Packages: Computer ScienceComputer Science (R0)