skip to main content
10.1145/3576915.3624381acmconferencesArticle/Chapter ViewAbstractPublication PagesccsConference Proceedingsconference-collections
poster

Poster: Generating Experiences for Autonomous Network Defense

Published: 21 November 2023 Publication History

Abstract

Reinforcement Learning (RL) offers a promising path toward developing defenses for the next generation of computer networks. The hope is that RL not only helps to automate network defenses, but in addition, RL finds novel solutions to defend networks that adapt to deal with the increasing complexity of networks and threats. Despite the promise, existing work applying RL to cybersecurity trains cyber defenders on rigid and narrow problem definitions with small computer networks.
Inspired by research that demonstrates that open-ended learning helps agents to adapt rapidly and to generalize to tasks never seen before, we hypothesize that similar approaches can offer a path toward practical RL for network defense. We provide evidence to support this hypothesis. A key aspect to enable generalizable learning is our approach for generating experiences for the learning agent--based on a universe of tasks-in a manner that allows the agent to learn to defend increasingly more complex networks. We show that RL agents can learn to master a reasonably complex network defense task by learning to solve tasks with varying degrees of difficulty. Our preliminary results show that in addition to contributing to the feasibility of mastering complex tasks, this type of experience generation may result in more robust policies.
Overall, our research demonstrates that the collection of experiences that we present to the learning agent is a critical aspect for achieving high performance. We share with the research community our approaches for (i) defining distributions over network defense tasks; (ii) updating distributions as the agent learns; and (iii) maintaining key aspects of tasks invariant to preserve knowledge as tasks vary.
Our experiments are enabled by the second version of our Framework for Advanced Reinforcement Learning for Autonomous Network Defense (FARLAND), which integrates support for action representations, dynamic task selection, and validation of policies in simulation and emulation. Our hope is that by sharing our ideas and results, we foster collaborations and innovation toward the creation of increasingly sophisticated gyms to train network defenders.

References

[1]
Andres Molina-Markham, Ransom K. Winder, and Ahmad Ridley. Network Defense is Not a Game. AI/ML for Cybersecurity: Challenges, Solutions, and Novel Ideas at SDM, 2021.
[2]
OpenAI, Ilge Akkaya, Marcin Andrychowicz, Maciek Chociej, Mateusz Litwin, Bob McGrew, Arthur Petron, Alex Paino, Matthias Plappert, Glenn Powell, Raphael Ribas, Jonas Schneider, Nikolas Tezak, Jerry Tworek, Peter Welinder, Lilian Weng, Qiming Yuan, Wojciech Zaremba, and Lei Zhang. Solving Rubik's Cube with a Robot Hand. 2019. _eprint: 1910.07113.
[3]
Open Ended Learning Team, Adam Stooke, Anuj Mahajan, Catarina Barros, Charlie Deck, Jakob Bauer, Jakub Sygnowski, Maja Trebacz, Max Jaderberg, Michaël Mathieu, Nat McAleese, Nathalie Bradley-Schmieg, Nathaniel Wong, Nicolas Porcel, Roberta Raileanu, Steph Hughes-Fitt, Valentin Dalibard, and Wojciech Marian Czarnecki. Open-Ended Learning Leads to Generally Capable Agents. CoRR, abs/2107.12808, 2021. arXiv: 2107.12808

Cited By

View all
  • (2025)The Path to Autonomous CyberdefenseIEEE Security & Privacy10.1109/MSEC.2024.342764023:1(38-46)Online publication date: Jan-2025
  • (2024)A Survey on Penetration Path Planning in Automated Penetration TestingApplied Sciences10.3390/app1418835514:18(8355)Online publication date: 17-Sep-2024

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
CCS '23: Proceedings of the 2023 ACM SIGSAC Conference on Computer and Communications Security
November 2023
3722 pages
ISBN:9798400700507
DOI:10.1145/3576915
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 21 November 2023

Check for updates

Author Tags

  1. artificial intelligence
  2. autonomy
  3. network security

Qualifiers

  • Poster

Conference

CCS '23
Sponsor:

Acceptance Rates

Overall Acceptance Rate 1,261 of 6,999 submissions, 18%

Upcoming Conference

CCS '25

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)134
  • Downloads (Last 6 weeks)7
Reflects downloads up to 17 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2025)The Path to Autonomous CyberdefenseIEEE Security & Privacy10.1109/MSEC.2024.342764023:1(38-46)Online publication date: Jan-2025
  • (2024)A Survey on Penetration Path Planning in Automated Penetration TestingApplied Sciences10.3390/app1418835514:18(8355)Online publication date: 17-Sep-2024

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media