Abstract
Autonomous decision making for cyber-defence in operational situations is desirable but challenging. This is due to the nature of operational technology (because of its cyber-physical nature) as well as the need to account for multiple contexts. Our contribution is the creation of a co-operative decision-making framework to enable autonomous cyber-defence (which we call Co-Decyber). This framework allows us to break up a big multi-contextual action space into smaller decisions that multiple agents can optimize between. We apply this framework to an autonomous vehicle platooning scenario. Results show that Co-Decyber agents are outperforming random reference agents in the cyber-attack scenarios we have tested. We aim to extend this work with more complex attack scenarios, along with training more agents to defend more of the attack surface. We conclude that this framework when mature will contribute to the goal of providing autonomous cyber-defence for operational technology.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Dhir, N., Hoeltgebaum, H., Adams, N., Briers, M., Burke, A., Jones, P.: Prospective artificial intelligence approaches for active cyber defence (2021). https://arxiv.org/pdf/2104.09981.pdf
Vyas, S., Hannay, J., Bolton, A., Burnap, P.P.: Automated cyber defence: a review (2023). arXiv preprint arXiv:2303.04926
Bridges, R.A., et al.: Testing SOAR tools in use. Comput. Secur. 129, 103201 (2023)
Jhawar, R., Mauw, S., Zakiuddin, I.: Automating cyber defence responses using attack-defence trees and game theory. In: European Conference on Cyber Warfare and Security, p. 163. Academic Conferences International Limited (2016)
Kordy, B., Mauw, S., Melissen, M., Schweitzer, P.: Attack–defense trees and two-player binary zero-sum extensive form games are equivalent. In: Alpcan, T., Buttyán, L., Baras, J.S. (eds) Decision and Game Theory for Security. GameSec 2010. Lecture Notes in Computer Science, vol. 6442. Springer, Berlin, Heidelberg (2010). https://doi.org/10.1007/978-3-642-17197-0_17
Eom, T., Hong, J.B., An, S., Park, J.S., Kim, D.S.: A framework for real-time intrusion response in software defined networking using precomputed graphical security models. Secur. Commun. Networks 2020, 1–15 (2020)
Nguyen, T.T., Reddi, V.J.: Deep reinforcement learning for cyber security. IEEE Transactions on Neural Networks and Learning Systems 34, 1–17 (2021)
Object Management Group: About the DDS security specification version 1.1 (2018). https://www.omg.org/spec/DDS-SECURITY/
Chowdhary, A., Huang, D., Sabur, A., Vadnere, N., Kang, M., Montrose, B.: SDN-based moving target defense using multi-agent reinforcement learning. In: Proceedings of the first International Conference on Autonomous Intelligent Cyber defense Agents, p. 15. Paris, France (2021)
Yao, Q., Wang, Y., Xiong, X., Wang, P., Li, Y.: Adversarial decision-making for moving target defense: a multi-agent Markov game and reinforcement learning approach. Entropy 25(4), 605 (2023)
Kordy, B., Piètre-Cambacédès, L., Schweitzer, P.: DAG-based attack and defense modeling: don’t miss the forest for the attack trees. Comput. Sci. Rev. 13, 1–38 (2014)
Soviany, P., Ionescu, R.T., Rota, P., Sebe, N.: Curriculum learning: a survey. Int. J. Comput. Vision 130(6), 1526–1565 (2022)
Jeon, J., Kim, W., Jung, W., Sung, Y.: Maser: Multi-agent reinforcement learning with subgoals generated from experience replay buffer. In International Conference on Machine Learning, pp. 10041–10052. PMLR (2022)
Brockman, G., et al.: Openai gym. arXiv Preprint arXiv:1606.01540 (2016)
Terry, J., et al.: Pettingzoo: gym for multi-agent reinforcement learning. In: Advances in Neural Information Processing Systems, vol. 34, pp. 15032–15043 (2021)
Acknowledgements
This research is funded by Frazer-Nash Consultancy Ltd. on behalf of the Defence Science and Technology Laboratory (Dstl), an executive agency of the UK Ministry of Defence. The research forms part of the Autonomous Resilient Cyber Defence (ARCD) project within the Dstl Cyber Defence Enhancement programme.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2024 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Cheah, M. et al. (2024). CO-DECYBER: Co-operative Decision Making for Cybersecurity Using Deep Multi-agent Reinforcement Learning. In: Katsikas, S., et al. Computer Security. ESORICS 2023 International Workshops. ESORICS 2023. Lecture Notes in Computer Science, vol 14399. Springer, Cham. https://doi.org/10.1007/978-3-031-54129-2_37
Download citation
DOI: https://doi.org/10.1007/978-3-031-54129-2_37
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-54128-5
Online ISBN: 978-3-031-54129-2
eBook Packages: Computer ScienceComputer Science (R0)