Skip to main content

Deep CounterStrike: Counter Adversarial Deep Reinforcement Learning for Defense Against Metamorphic Ransomware Swarm Attack

  • Conference paper
  • First Online:
Broadband Communications, Networks, and Systems (BROADNETS 2023)

Abstract

Ransomware, create a devastating impact when it infects a system. Fortunately, post the initial breach, such ransomware could be detected using advanced machine learning techniques, and therefore other high-value assets/systems can be protected from any repeat attack by the same ransomware. However, using metamorphism, advanced/ second-generation ransomware can alter its structure after every successful infection. With this ability of metamorphism, such advanced ransomware could continue to evade any defensive mechanism and keep infecting systems in subsequent networks. Currently, there exists neither any proven defensive mechanism nor any useful dataset to train a defensive mechanism against such advanced ransomware. Therefore, we present a deep counter adversarial reinforcement learning-based system that learns how to normalize the metamorphism of such advanced ransomware to design a credible defence against such advanced attacks. To augment training data for this system, we design and develop a deep adversarial reinforcement learning solution, to generate swarms of such advanced ransomware.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Baysa, D., Low, R.M., Stamp, M.: Structural entropy and metamorphic malware. J. Comput. Virol. Hacking Tech. 9(4), 179–192 (2013)

    Article  Google Scholar 

  2. Behera, C.K., Bhaskari, D.L.: Different obfuscation techniques for code protection. Procedia Comput. Sci. 70, 757–763 (2015)

    Article  Google Scholar 

  3. Bilge, L., Dumitraş, T.: Before we knew it: an empirical study of zero-day attacks in the real world. In: ACM Conference on Computer and Communications Security (CCS), pp. 833–844 (2012)

    Google Scholar 

  4. Freedman, D., Pisani, R., Purves, R.: Statistics. Norton & Company (1998)

    Google Scholar 

  5. Gazet, A.: Comparative analysis of various ransomware virii. J. Comput. Virol. 6(1), 77–90 (2010)

    Article  Google Scholar 

  6. Goodfellow, I., et al.: Generative adversarial networks. Commun. ACM 63(11), 139–144 (2020)

    Article  MathSciNet  Google Scholar 

  7. Grosse, K., Papernot, N., Manoharan, P., Backes, M., McDaniel, P.: Adversarial perturbations against deep neural networks for malware classification. arXiv preprint arXiv:1606.04435 (2016)

  8. van Hasselt, H., Guez, A., Silver, D.: Deep reinforcement learning with double q-learning. CoRR abs/1509.06461 (2015)

    Google Scholar 

  9. Kolosnjaji, B., et al.: Adversarial malware binaries: evading deep learning for malware detection in executables. CoRR abs/1803.04173 (2018)

    Google Scholar 

  10. Madry, A., Makelov, A., Schmidt, L., Tsipras, D., Vladu, A.: Towards deep learning models resistant to adversarial attacks. In: International Conference on Learning Representations (ICLR) (2018)

    Google Scholar 

  11. Meng, D., Chen, H.: Magnet: a two-pronged defense against adversarial examples. In: ACM SIGSAC Conference on Computer and Communications Security, pp. 135–147 (2017)

    Google Scholar 

  12. Mnih, V., et al.: Asynchronous methods for deep reinforcement learning. CoRR abs/1602.01783 (2016)

    Google Scholar 

  13. Mnih, V., et al.: Playing Atari with deep reinforcement learning. CoRR abs/1312.5602 (2013)

    Google Scholar 

  14. Mnih, V., et al.: Human-level control through deep reinforcement learning. Nature 518, 529–533 (2015)

    Article  Google Scholar 

  15. Mohurle, S., Patil, M.: A brief study of wannacry threat: ransomware attack. Int. J. Adv. Res. Comput. Sci. 8(5), 1938–1940 (2017)

    Google Scholar 

  16. Nappa, A., Rafique, M.Z., Caballero, J.: The MALICIA dataset: identification and analysis of drive-by download operations. Int. J. Inf. Secur. 14, 15–33 (2015)

    Article  Google Scholar 

  17. Papernot, N., McDaniel, P., Jha, S., Fredrikson, M., Celik, Z.B., Swami, A.: The limitations of deep learning in adversarial settings. In: IEEE European Symposium on Security and Privacy (Euro S &P), pp. 372–387 (2016)

    Google Scholar 

  18. Rathore, H., Bandwala, T., Sahay, S.K., Sewak, M.: Adversarial robustness of image based Android malware detection models. In: Krishnan, R., Rao, H.R., Sahay, S.K., Samtani, S., Zhao, Z. (eds.) SKM 2021. CCIS, vol. 1549, pp. 3–22. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-97532-6_1

    Chapter  Google Scholar 

  19. Rathore, H., Nikam, P., Sahay, S.K., Sewak, M.: Identification of adversarial Android intents using reinforcement learning. In: International Joint Conference on Neural Networks (IJCNN), pp. 1–8. IEEE (2021)

    Google Scholar 

  20. Rathore, H., Samavedhi, A., Sahay, S.K., Sewak, M.: Robust malware detection models: learning from adversarial attacks and defenses. Forensic Sci. Int.: Digit. Invest. 37, 301183 (2021)

    Google Scholar 

  21. Rathore, H., Samavedhi, A., Sahay, S.K., Sewak, M.: Towards adversarially superior malware detection models: an adversary aware proactive approach using adversarial attacks and defenses. Inf. Syst. Front. 25, 567–587 (2022)

    Google Scholar 

  22. Rathore, H., Sasan, A., Sahay, S.K., Sewak, M.: Defending malware detection models against evasion based adversarial attacks. Pattern Recogn. Lett. 164, 119–125 (2022)

    Article  Google Scholar 

  23. Rathore, H., Sharma, S.C., Sahay, S.K., Sewak, M.: Are malware detection classifiers adversarially vulnerable to actor-critic based evasion attacks? EAI Endorsed Trans. Scalable Inf. Syst. 10(1), e6 (2023)

    Google Scholar 

  24. Ren, K., Zheng, T., Qin, Z., Liu, X.: Adversarial attacks and defenses in deep learning. Engineering 6(3), 346–360 (2020)

    Article  Google Scholar 

  25. Schulman, J., Levine, S., Abbeel, P., Jordan, M., Moritz, P.: Trust region policy optimization. In: International Conference on Machine Learning (ICML), pp. 1889–1897. PMLR (2015)

    Google Scholar 

  26. Schulman, J., Wolski, F., Dhariwal, P., Radford, A., Klimov, O.: Proximal policy optimization algorithms. CoRR abs/1707.06347 (2017)

    Google Scholar 

  27. Sewak, M., Sahay, S.K., Rathore, H.: Value-approximation based deep reinforcement learning techniques: an overview. In: IEEE 5th International Conference on Computing Communication and Automation (ICCCA), pp. 379–384 (2020)

    Google Scholar 

  28. Sewak, M.: Deep Q Network (DQN), double DQN, and dueling DQN. In: Sewak, M. (ed.) Deep Reinforcement Learning, pp. 95–108. Springer, Singapore (2019). https://doi.org/10.1007/978-981-13-8285-7_8

    Chapter  MATH  Google Scholar 

  29. Sewak, M., Karim, M.R., Pujari, P.: Practical Convolutional Neural Networks: Implement Advanced Deep Learning Models Using Python. Packt Publishing (2018)

    Google Scholar 

  30. Sewak, M., Sahay, S.K., Rathore, H.: Policy-approximation based deep reinforcement learning techniques: an overview. In: Joshi, A., Mahmud, M., Ragel, R.G., Thakur, N.V. (eds.) Information and Communication Technology for Competitive Strategies (ICTCS 2020). LNNS, vol. 191, pp. 493–507. Springer, Singapore (2022). https://doi.org/10.1007/978-981-16-0739-4_47

    Chapter  Google Scholar 

  31. Sewak, M., Sahay, S.K., Rathore, H.: Comparison of deep learning and the classical machine learning algorithm for the malware detection. In: 19th IEEE/ACIS SNPD 2018, pp. 293–296. IEEE (2018)

    Google Scholar 

  32. Sewak, M., Sahay, S.K., Rathore, H.: Assessment of the relative importance of different hyper-parameters of LSTM for an IDS. In: IEEE Region 10 Conference (TENCON), pp. 414–419. IEEE (2020)

    Google Scholar 

  33. Sewak, M., Sahay, S.K., Rathore, H.: An overview of deep learning architecture of deep neural networks and autoencoders. J. Comput. Theor. Nanosci. 17(1), 182–188 (2020)

    Article  Google Scholar 

  34. Sewak, M., Sahay, S.K., Rathore, H.: Adversarialuscator: an adversarial-DRL based obfuscator and metamorphic malware swarm generator. In: International Joint Conference on Neural Networks (IJCNN), pp. 1–9. IEEE (2021)

    Google Scholar 

  35. Sewak, M., Sahay, S.K., Rathore, H.: DRLDO: a novel DRL based de-obfuscation system for defence against metamorphic malware. Def. Sci. J. 71(1), 55–65 (2021)

    Article  Google Scholar 

  36. Sewak, M., Sahay, S.K., Rathore, H.: DRo: a data-scarce mechanism to revolutionize the performance of DL-based Security Systems. In: IEEE 46th Conference on Local Computer Networks (LCN), pp. 581–588. IEEE (2021)

    Google Scholar 

  37. Sewak, M., Sahay, S.K., Rathore, H.: Deep reinforcement learning for cybersecurity threat detection and protection: a review. In: Krishnan, R., Rao, H.R., Sahay, S.K., Samtani, S., Zhao, Z. (eds.) SKM 2021. CCIS, vol. 1549, pp. 51–72. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-97532-6_4

    Chapter  Google Scholar 

  38. Sutton, R.S., McAllester, D., Singh, S., Mansour, Y.: Policy gradient methods for reinforcement learning with function approximation. In: International Conference on Neural Information Processing Systems, pp. 1057–1063. MIT Press (1999)

    Google Scholar 

  39. Usama, M., Asim, M., Latif, S., Qadir, J., Ala-Al-Fuqaha: Generative adversarial networks for launching and thwarting adversarial attacks on network intrusion detection systems. In: 15th International Wireless Communications Mobile Computing Conference (IWCMC), pp. 78–83 (2019)

    Google Scholar 

  40. Wang, Z., Schaul, T., Hessel, M., Van Hasselt, H., Lanctot, M., De Freitas, N.: Dueling network architectures for deep reinforcement learning. In: International Conference on International Conference on Machine Learning, pp. 1995–2003 (2016)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Mohit Sewak .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2023 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Sewak, M., Sahay, S.K., Rathore, H. (2023). Deep CounterStrike: Counter Adversarial Deep Reinforcement Learning for Defense Against Metamorphic Ransomware Swarm Attack. In: Wang, W., Wu, J. (eds) Broadband Communications, Networks, and Systems. BROADNETS 2023. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 511. Springer, Cham. https://doi.org/10.1007/978-3-031-40467-2_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-40467-2_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-40466-5

  • Online ISBN: 978-3-031-40467-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics