Skip to main content
SpringerLink
Log in

Defensive deception framework against reconnaissance attacks in the cloud with deep reinforcement learning

  • Research Paper
  • Published:
Science China Information Sciences Aims and scope Submit manuscript

Abstract

Implementing defensive deception in the cloud is promising to proactively counter reconnaissance attack. This technique presents decoys to camouflage cloud assets and distracts attack resource. However, the major challenge is to develop an effective deception strategy to orchestrate digital decoys. To address this issue, we propose a deep reinforcement learning (DRL)-based defensive deception framework. First, we formulate a utility function, which mathematically models underlying threats associated with common vulnerabilities among virtual machines in the cloud. Then, we customize training interfaces and the neural networks for a DRL agent. The reward function reflects the effectiveness of asset concealment and the waste of attack resources, referring to a comprehensive defense goal. Finally, the well-trained DRL agent generates the optimal defense strategy. It specifies a more granular deception strategy than existing proposals. Simulation results show that the proposed framework leads to a 7.87% average advantage in realizing the comprehensive defense goal. Moreover, it can stably improve the concealment degree of cloud assets up to 20.58%, and increase the attack cost up to 40.40%. This study shows that it is promising to improve cloud security with deception defense and artificial intelligence techniques.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Hutchins E M, Cloppert M J, Amin R M, et al. Intelligence-driven computer network defense informed by analysis of adversary campaigns and intrusion kill chains. Leading Issues Inform Warfare Secur Res, 2011, 1: 80

    Google Scholar 

  2. Compastié M, Badonnel R, Festor O, et al. From virtualization security issues to cloud protection opportunities: An in-depth analysis of system virtualization models. Comput Secur, 2020, 97: 101905

    Article  Google Scholar 

  3. Garcia M, Bessani A, Gashi I, et al. OS diversity for intrusion tolerance: myth or reality? In: Proceedings of 2011 IEEE/IFIP 41st International Conference on Dependable Systems and Networks (DSN), 2011. 383–394

  4. Achleitner S, La Porta T F, McDaniel P, et al. Deceiving network reconnaissance using SDN-based virtual topologies. IEEE Trans Netw Serv Manage, 2017, 14: 1098–1112

    Article  Google Scholar 

  5. Carasik-Henmi A, Shinder T W, Amon C, et al. Chapter 4—Introduction to intrusion detection systems. In: The Best Damn Firewall Book Period. Burlington: Syngress, 2003. 111–124

    Google Scholar 

  6. Virvilis N, Vanautgaerden B, Serrano O S. Changing the game: the art of deceiving sophisticated attackers. In: Proceedings of the 6th International Conference On Cyber Conflict (CyCon 2014), 2014. 87–97

  7. Fraunholz D, Anton S D, Lipps C, et al. Demystifying deception technology: a survey. 2018. ArXiv:1804.06196

  8. Han X, Kheir N, Balzarotti D. Deception techniques in computer security. ACM Comput Surv, 2018, 51: 1–36

    Article  Google Scholar 

  9. Lu Z, Wang C, Zhao S, et al. Cyber deception for computer and network security: survey and challenges. 2020. ArXiv:2007.14497

  10. Durkota K, Lisy V, Bosansky B, et al. Optimal network security hardening using attack graph games. In: Proceedings of the 24th International Conference on Artificial Intelligence, 2015. 526–532

  11. Fraunholz D, Schotten H D. Defending web servers with feints, distraction and obfuscation. In: Proceedings of 2018 International Conference on Computing, Networking and Communications (ICNC), 2018. 21–25

  12. Pawlick J, Colbert E, Zhu Q. A game-theoretic taxonomy and survey of defensive deception for cybersecurity and privacy. ACM Comput Surv, 2019, 52: 1–28

    Article  Google Scholar 

  13. Condon A. The complexity of stochastic games. Inf Comput, 1992, 96: 203–224

    Article  MathSciNet  MATH  Google Scholar 

  14. Kaynar K. A taxonomy for attack graph generation and usage in network security. J Inf Secur Appl, 2016, 29: 27–56

    Google Scholar 

  15. Duan Q, Al-Shaer E, Islam M, et al. CONCEAL: a strategy composition for resilient cyber deception-framework, metrics and deployment. In: Proceedings of IEEE Conference on Communications and Network Security (CNS), 2018. 1–9

  16. Qu Y Y, Zhang J W, Li R D, et al. Generative adversarial networks enhanced location privacy in 5G networks. Sci China Inf Sci, 2020, 63: 220303

    Article  MathSciNet  Google Scholar 

  17. Ning Z L, Sun S M, Wang X J, et al. Intelligent resource allocation in mobile blockchain for privacy and security transactions: a deep reinforcement learning based approach. Sci China Inf Sci, 2021, 64: 162303

    Article  MathSciNet  Google Scholar 

  18. Roy S, Sharmin N, Acosta J C, et al. Survey and taxonomy of adversarial reconnaissance techniques. 2021. ArXiv:2105.04749

  19. Wang Y, Guo Y, Guo Z, et al. CLOSURE: a cloud scientific workflow scheduling algorithm based on attack-defense game model. Future Generation Comput Syst, 2020, 111: 460–474

    Article  Google Scholar 

  20. Kandoussi E M, Hanini M, El Mir I, et al. Toward an integrated dynamic defense system for strategic detecting attacks in cloud networks using stochastic game. Telecommun Syst, 2020, 73: 397–417

    Article  Google Scholar 

  21. Zhan J, Fan X, Han J, et al. CIADL: cloud insider attack detector and locator on multi-tenant network isolation: an OpenStack case study. J Ambient Intell Hum Comput, 2020, 11: 3473–3495

    Article  Google Scholar 

  22. Cohen F. A note on the role of deception in information protection. Comput Secur, 1998, 17: 483–506

    Article  Google Scholar 

  23. Spitzner L. The Honeynet Project: trapping the hackers. IEEE Secur Privacy, 2003, 1: 15–23

    Article  Google Scholar 

  24. Petrunic A R. Honeytokens as active defense. In: Proceedings of the 38th International Convention on Information and Communication Technology, Electronics and Microelectronics (MIPRO), 2015. 1313–1317

  25. Araujo F, Hamlen K W, Biedermann S, et al. From patches to honey-patches. In: Proceedings of the 2014 ACM SIGSAC Conference on Computer and Communications Security. New York: ACM, 2014. 942–953

    Google Scholar 

  26. Shu Z, Yan G. Ensuring deception consistency for FTP services hardened against advanced persistent threats. In: Proceedings of the 5th ACM Workshop on Moving Target Defense. New York: ACM, 2018. 69–79

    Google Scholar 

  27. Rrushi J L. NIC displays to thwart malware attacks mounted from within the OS. Comput Secur, 2016, 61: 59–71

    Article  Google Scholar 

  28. Kyriakou A, Sklavos N. Container-based honeypot deployment for the analysis of malicious activity. In: Proceedings of Global Information Infrastructure and Networking Symposium, 2019

  29. Rowe N C, Rrushi J. Introduction to Cyberdeception. Cham: Springer, 2016

    Book  Google Scholar 

  30. Zhu Q. Game theory for cyber deception: a tutorial. In: Proceedings of the 6th Annual Symposium on Hot Topics in the Science of Security, 2019

  31. Almohri H M J, Watson L T, Evans D. Misery digraphs: delaying intrusion attacks in obscure clouds. IEEE Trans Inform Forensic Secur, 2018, 13: 1361–1375

    Article  Google Scholar 

  32. Horák K, Bošanský B, Tomášek P, et al. Optimizing honeypot strategies against dynamic lateral movement using partially observable stochastic games. Comput Secur, 2019, 87: 101579

    Article  Google Scholar 

  33. Crouse M, Prosser B, Fulp E W. Probabilistic performance analysis of moving target and deception reconnaissance defenses. In: Proceedings of the 2nd ACM Workshop on Moving Target Defense. New York: ACM, 2015. 21–29

    Google Scholar 

  34. Mnih V, Kavukcuoglu K, Silver D, et al. Human-level control through deep reinforcement learning. Nature, 2015, 518: 529–533

    Article  Google Scholar 

  35. Sethi K, Kumar R, Prajapati N, et al. Deep reinforcement learning based intrusion detection system for cloud infrastructure. In: Proceedings of International Conference on Communication Systems & NETworkS (COMSNETS), 2020. 1–6

  36. Provos N. Honeyd—a virtual honeypot daemon. In: Proceedings of the 10th DFN-CERT Workshop, Hamburg, 2003. 4

  37. Cabral W, Valli C, Sikos L, et al. Review and analysis of cowrie artefacts and their potential to be used deceptively. In: Proceedings of International Conference on Computational Science and Computational Intelligence (CSCI), 2019. 166–171

  38. Zhang M, Wang L, Jajodia S, et al. Network diversity: a security metric for evaluating the resilience of networks against zero-day attacks. IEEE Trans Inform Forensic Secur, 2016, 11: 1071–1086

    Article  Google Scholar 

  39. Guo M, Bhattacharya P. Diverse virtual replicas for improving intrusion tolerance in cloud. In: Proceedings of the 9th Annual Cyber and Information Security Research Conference. New York: ACM Press, 2014. 41–44

    Google Scholar 

  40. Schulman J, Wolski F, Dhariwal P, et al. Proximal policy optimization algorithms. 2017. ArXiv:1707.06347

  41. Gutierrez M, Kiekintveld C. Online learning methods for controlling dynamic cyber deception strategies. In: Adaptive Autonomous Secure Cyber Systems. Cham: Springer, 2020. 231–251

    Google Scholar 

  42. Schlenker A, Thakoor O, Xu H, et al. Deceiving cyber adversaries: a game theoretic approach. In: Proceedings of International Conference on Autonomous Agents and Multiagent Systems, 2018

  43. Sengupta S, Chowdhary A, Huang D, et al. Moving target defense for the placement of intrusion detection systems in the cloud. In: Proceedings of International Conference on Decision and Game Theory for Security, 2018. 326–345

  44. Daskalakis C, Goldberg P W, Papadimitriou C H. The complexity of computing a Nash equilibrium. Commun ACM, 2009, 52: 89–97

    Article  MATH  Google Scholar 

  45. Lipton R J, Markakis E, Mehta A. Playing large games using simple strategies. In: Proceedings of the 4th ACM Conference on Electronic Commerce, 2003. 36–41

  46. Chen J Y, Zhang Y, Wang X, et al. A survey of attack, defense and related security analysis for deep reinforcement learning. Acta Autom Sin, 2022, 48: 1–19

    Google Scholar 

Download references

Acknowledgements

This work has been partly supported by National Key Research and Development Program of China (Grant Nos. 2021YFB1006200, 2021YFB1006201) and National Natural Science Foundation of China (Grant Nos. 62072467, 62002383).

Author information

Authors and Affiliations

  1. Department of Computer Science, Information Engineering University, Zhengzhou, 450001, China

    Huanruo Li, Yunfei Guo, Shumin Huo, Hongchao Hu & Penghao Sun

  2. Department of Communication Technologies, Academy of Military Science, Beijing, 100091, China

    Penghao Sun

Authors
  1. Huanruo Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yunfei Guo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shumin Huo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Hongchao Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Penghao Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Huanruo Li.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Li, H., Guo, Y., Huo, S. et al. Defensive deception framework against reconnaissance attacks in the cloud with deep reinforcement learning. Sci. China Inf. Sci. 65, 170305 (2022). https://doi.org/10.1007/s11432-021-3462-4

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11432-021-3462-4

Keywords

Search

Navigation