Sridhar Venkatesan
Sushil Jajodia
ABSTRACT
Modern botnets can persist in networked systems for extended periods of time by operating in a stealthy manner. Despite the progress made in the area of botnet prevention, detection, and mitigation, stealthy botnets continue to pose a significant risk to enterprises. Furthermore, existing enterprise-scale solutions require significant resources to operate effectively, thus they are not practical. In order to address this important problem in a resource-constrained environment, we propose a reinforcement learning based approach to optimally and dynamically deploy a limited number of defensive mechanisms, namely honeypots and network-based detectors, within the target network. The ultimate goal of the proposed approach is to reduce the lifetime of stealthy botnets by maximizing the number of bots identified and taken down through a sequential decision-making process. We provide a proof-of-concept of the proposed approach, and study its performance in a simulated environment. The results show that the proposed approach is promising in protecting against stealthy botnets.
- Tansu Alpcan and Tamer Başar. 2006. An Intrusion Detection Game with Limited Observations. In Proceedings of the 12th International Symposium on Dynamic Games and Applications. Sophia-Antipolis, France.Google Scholar
- Michael Bailey, Evan Cooke, Farnam Jahanian, Yunjing Xu, and Manish Karir. 2009. A Survey of Botnet Technology and Defenses. In Proceedings of the Cybersecurity Applications & Technology Conference for Homeland Security (CATCH 2009). 299--304. Google ScholarDigital Library
- Elaheh Biglar Beigi, Hossein Hadian Jazi, Natalia Stakhanova, and Ali A. Ghorbani. 2014. Towards Effective Feature Selection in Machine Learning-Based Botnet Detection Approaches. In Proceedings of the IEEE Conference on Communications and Network Security (CNS 2014). IEEE, San Francisco, CA, USA, 247--255.Google Scholar
- Richard Bellman. 1957. Dynamic Programming. Princeton University Press, Princeton, NJ, USA.Google Scholar
- Rajesh Ganesan, Sushil Jajodia, Ankit Shah, and Hasan Cam. 2016. Dynamic Scheduling of Cybersecurity Analysts for Minimizing Risk Using Reinforcement Learning. ACM Transactions on Intelligent Systems and Technology, Vol. 8, 1 (2016). Google ScholarDigital Library
- Abhijit Gosavi. 2003. Simulation-Based Optimization: Parametric Optimization Techniques and Reinforcement Learning. Springer. Google ScholarCross Ref
- Guofei Gu, Roberto Perdisci, Junjie Zhang, and Wenke Lee. 2008. BotMiner: Clustering Analysis of Network Traffic for Protocol- and Structure-Independent Botnet Detection. In Proceedings of the 17th USENIX Security Symposium. San Jose, CA, USA, 139--154.Google ScholarDigital Library
- Guofei Gu, Phillip Porras, Vinod Yegneswaran, Martin Fong, and Wenke Lee. 2007. BotHunter: Detecting Malware Infection Through IDS-Driven Dialog Correlation. In Proceedings of the 16th USENIX Security Symposium. Boston, MA, USA, 167--182.Google ScholarDigital Library
- Guofei Gu, Junjie Zhang, and Wenke Lee. 2008. BotSniffer: Detecting Botnet Command and Control Channels in Network Traffic. In Proceedings of the 15th Annual Network and Distributed System Security Symposium (NDSS 2008). San Diego, CA, USA.Google Scholar
- Kaspersky Lab. 2014. The Regin Platform Nation-State Ownage of GSM Networks. Technical Report. Kaspersky Lab.Google Scholar
- Karim Khalil, Zhiyun Qian, Paul Yu, Srikanth Krishnamurthy, and Ananthram Swam. 2016. Optimal Monitor Placement for Detection of Persistent Threats. In Proceedings of the IEEE Global Communications Conference (IEEE GLOBECOM 2016). IEEE, Washington, DC USA. Google ScholarCross Ref
- Marion Marschalek, Paul Kimayong, and Fengmin Gong. 2014. POS Malware Revisited: Look What We Found Inside Your Cashdesk. Technical Report. Cyphort Labs.Google Scholar
- Alberto Montresor and Márk Jelasity. 2009. PeerSim: A Scalable P2P Simulator. In Proceedings of the 9th IEEE International Conference on Peer-to-Peer Computing (P2P 2009). Seattle, WA, USA, 99--100.Google ScholarCross Ref
- Juliana M. Nascimento and Warren B. Powell. 2009. An Optimal Approximate Dynamic Programming Algorithm for the Lagged Asset Acquisition Problem. Mathematics of Operations Research Vol. 34, 1 (February 2009), 210--237. Google ScholarDigital Library
- Warren B. Powell. 2011. Approximate Dynamic Programming: Solving the Curses of Dimensionality (2nd ed.). John Wiley & Sons. Google ScholarCross Ref
- Christian Rossow, Dennis Andriesse, Tillmann Werner, Brett Stone-Gross, Daniel Plohmann, Christian J. Dietrich, and Herbert Bos. 2013. SoK: P2PWNED - Modeling and Evaluating the Resilience of Peer-to-Peer Botnets. In Proceedings of the 2013 IEEE Symposium on Security and Privacy (S&P 2013). IEEE, San Francisco, CA, USA, 97--111.Google ScholarDigital Library
- Stephan Schmidt, Tansu Alpcan, Şahin Albayrak, Tamer Başar, and Achim Mueller. 2007. A Malware Detector Placement Game for Intrusion Detection. In Proceedings of the 2nd International Workshop on Critical Information Infrastructures Security (CRITIS 2007). Springer, Benalmádena, Málaga, Spain, 311--326.Google Scholar
- Seungwon Shin, Lei Xu, Sungmin Hong, and Guofei Gu. 2016. Enhancing Network Security through Software Defined Networking (SDN). In Proceedings of the 25th International Conference on Computer Communication and Networks (ICCCN 2016). IEEE, Waikoloa, HI, USA. Google ScholarCross Ref
- Ali Shiravi, Hadi Shiravi, Mahbod Tavallaee, and Ali A. Ghorbani. 2012. Toward developing a systematic approach to generate benchmark datasets for intrusion detection. Computers & Security Vol. 31, 3 (May 2012), 357--37. Google ScholarDigital Library
- Lance Spitzner. 2002. Honeypots: Tracking Hackers. Addison Wesley, Boston, MA, USA.Google ScholarDigital Library
- Patrick John Sweeney. 2014. Designing Effective And Stealthy Botnets for Cybet Espionage And Interdiction - Finding the Cyber High Ground. Ph.D. Dissertation. Thayer School of Engineering, Darthmouth College.Google Scholar
- Symantec Security Response. 2011. W32.Duqu: The Precursor to the Next Stuxnet. https://www.symantec.com/connect/w32_duqu_precursor_next_stuxnet. (October 2011).Google Scholar
- Trend Micro. 2013. Lateral Movement: How Do Threat Actors Move Deeper Into Your Network? (2013).Google Scholar
- Sridhar Venkatesan, Massimiliano Albanese, George Cybenko, and Sushil Jajodia. 2016. A Moving Target Defense Approach to Disrupting Stealthy Botnets. In Proceedings of the 3rd ACM Workshop on Moving Target Defense (MTD 2016). ACM, Vienna, Austria, 37--46. Google ScholarDigital Library
- Yini Wang, Sheng Wen, Yang Xiang, and Wanlei Zhou. 2014. Modeling the Propagation of Worms in Networks: A Survey. IEEE Communications Surveys & Tutorials Vol. 16, 2 (2014), 942--960. Google ScholarCross Ref
- Michael P. Wellman and Achintya Prakash. 2014. Empirical Game-Theoretic Analysis of an Adaptive Cyber-Defense Scenario (Preliminary Report). In Proceedings of the International Conference on Decision and Game Theory for Security (GameSec 2014) (Lecture Notes in Computer Science), Vol. 8840. Springer, Los Angeles, CA, USA, 43--58.Google Scholar
- Michael West. 2009. Computer and Information Security Handbook. Morgan Kaufmann, Chapter Preventing System Intrusions, 39--51.Google Scholar
- Junjie Zhang, Roberto Perdisci, Wenke Lee, Xiapu Luo, and Unum Sarfraz. 2014. Building a Scalable System for Stealthy P2P-Botnet Detection. IEEE Transactions on Information Forensics and Security, Vol. 9, 1 (January 2014), 27--38. Google ScholarDigital Library
- Junjie Zhang, Roberto Perdisci, Wenke Lee, Unum Sarfraz, and Xiapu Luo. 2011. Detecting stealthy P2P botnets using statistical traffic fingerprints. In Proceedings of the 41st IEEE/IFIP International Conference on Dependable Systems and Networks (DSN 2011). IEEE, Hong Kong, China, 121--132.Google ScholarDigital Library
Index Terms
- Detecting Stealthy Botnets in a Resource-Constrained Environment using Reinforcement Learning
Recommendations
A Moving Target Defense Approach to Disrupting Stealthy Botnets
MTD '16: Proceedings of the 2016 ACM Workshop on Moving Target DefenseBotnets are increasingly being used for exfiltrating sensitive data from mission-critical systems. Research has shown that botnets have become extremely sophisticated and can operate in stealth mode by minimizing their host and network footprint. In ...
Detecting stealthy P2P botnets using statistical traffic fingerprints
DSN '11: Proceedings of the 2011 IEEE/IFIP 41st International Conference on Dependable Systems&NetworksPeer-to-peer (P2P) botnets have recently been adopted by botmasters for their resiliency to take-down efforts. Besides being harder to take down, modern botnets tend to be stealthier in the way they perform malicious activities, making current detection ...
Detecting, validating and characterizing computer infections in the wild
IMC '11: Proceedings of the 2011 ACM SIGCOMM conference on Internet measurement conferenceAlthough network intrusion detection systems (IDSs) have been studied for several years, their operators are still overwhelmed by a large number of false-positive alerts. In this work we study the following problem: from a large archive of intrusion ...
Comments