Abstract
In this paper, we propose and analyze a two-level game theoretical framework to detect advanced and persistent threats across processes. The two-level framework adapted facilitates abstraction of the complexity of process level interactions between defense mechanisms and adversaries from easier to interpret and more flexible system-level interaction. At the process-level, program anomaly detection algorithms have already been proposed to detect anomalous program behavior by comparing monitored activities with the predetermined expected behavior. This had led to significant detection performance initially until advanced adversaries modified the attacks to remain undetected. Therefore, we propose defense mechanisms that anticipate the reaction of advanced evaders and seek to maximize the complexity of undetectable attacks at the expense of additional false alarm rate. Furthermore, in the system-level, we propose defense mechanisms to detect adversarial intervention across processes through the assessment of all process activities together in a cohesive way so that the advanced adversaries need to craft their attacks further to remain undetected also at the system-level. This further increases the cost of complexity for the attacker, and correspondingly degrades the motivation to attack. We provide a game theoretical incentive analysis for both defenders and adversaries, and characterize pure and mixed strategy equilibria. We also analyze the coupling between the two levels of the game.
M. O. Sayin—This research was supported by the U.S. Office of Naval Research (ONR) MURI grant N00014-16-1-2710.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
We have used infimum here since, the set of indefinite length strings, \(\varSigma ^*\), not being compact, there is no immediate guarantee of existence of a minimum.
- 2.
With abuse of notation, we denote \(r^*_i\) as a function of \(\pmb {a}^*_i\) in order to show the dependence explicitly.
- 3.
For notational simplicity, we do not represent \(\alpha _{\omega }\)’s dependence on subset k explicitly.
- 4.
For notational simplicity, we have dropped the subset index k.
References
Başar, T., Olsder, G.: Dynamic Noncooperative Game Theory. Society for Industrial Mathematics (SIAM) Series in Classics, Applied Mathematics (1999)
Barabosch, T., Gerhards-Padilla, E.: Host-based code injection attacks: a popular technique used by malware. In: The 9th International Conference on Malicious and Unwanted Software: The Americas (MALWARE) (2014)
Brangetto, P., Aubyn, M.K.-S.: Economic aspects of national cyber security strategies. Technical report, NATO Cooperative Cyber Defense Centre of Excellence Tallinn, Estonia (2015)
Dritsoula, L., Loiseau, P., Musacchio, J.: A game-theoretic analysis of adversarial classification. IEEE Trans. Inf. Forensics Secur. 12(12), 3094–3109 (2017)
Du, M., Li, F., Zheng, G., Srikumar, V.: DeepLog: anomaly detection and diagnosis from system logs through deep learning. In: Proceedings of the 24th ACM Conference on Computer and Communications Security (2017)
Elisan, C.C.: Malware, Rootkits and Botnets: A Beginner’s Guide. McGraw-Hill, New York (2013)
Feng, H.H., Kolesnikov, O.M., Fogla, P., Lee, W., Gong, W.: Anomaly detection using call stack information. In: Proceedings of the IEEE Security and Privacy (2003)
Forrest, S., Hofmeyr, S.A., Somayaji, A., Longstaff, T.A.: A sense of self for Unix processes. In: Proceedings of IEEE Symposium on Security and Privacy (1996)
Gao, D., Reiter, M.K., Song, D.: On gray-box program tracking for anomaly detection. In: Proceedings of the 13th Conference on USENIX Security Symposium (2004)
Ghafouri, A., Abbas, W., Laszka, A., Vorobeychik, Y., Koutsoukos, X.: Optimal thresholds for anomaly-based intrusion detection in dynamical environments. In: Zhu, Q., Alpcan, T., Panaousis, E., Tambe, M., Casey, W. (eds.) GameSec 2016. LNCS, vol. 9996, pp. 415–434. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-47413-7_24
Ji, Y., et al.: RAIN: refinable attack investigation with on-demand inter-process information flow tracking. In: Proceedings of the 24th ACM Conference on Computer and Communications Security (2017)
Kruegel, C., Kirda, E., Mutz, D., Robertson, W., Vigna, G.: Automating mimicry attacks using static binary analysis. In: Proceedings of the 14th USENIX Security Symposium (2005)
Manzoor, E., Milajerdi, S.M., Akoglu, L.: Fast memory-efficient anomaly detection in streaming heterogeneous graphs. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining (2016)
Parampalli, C., Sekar, R., Johanson, R.: A practical mimicry attack against powerful system-call monitors. In: Proceedings of the 2008 ACM Symposium on Information, Computer and Communications Security (2008)
Shu, X., Yao, D.D., Ryder, B.G.: A formal framework for program anomaly detection. In: Bos, H., Monrose, F., Blanc, G. (eds.) RAID 2015. LNCS, vol. 9404, pp. 270–292. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-26362-5_13
Silberschatz, A., Galvin, P.B., Gagne, G.: Operating System Concepts. Wiley, Hoboken (2013)
Somayaji, A., Forest, S.: Automated response using system-call delays. In: Proceedings of the 9th USENIX Security Symposium (2000)
Wagner, D., Dean, D.: Intrusion detection via static analysis. In: Proceedings of the Symposium on Security and Privacy (2001)
Wagner, D., Soto, P.: Mimicry attacks on host-based intrusion detection systems. In: Proceedings of the 9th ACM Conference on Computer and Communications Security (2002)
Yao, D., Shu, X., Cheng, L., Stolfo, S.J.: Anomaly Detection as a Service: Challenges, Advances, and Opportunities. Synthesis Lectures on Information Security, Privacy, and Thrust #22, Morgan & Claypool Publishers (2017)
Zhu, Q., Başar, T.: Game-theoretic methods for robustness, security, and resiliency of cyberphysical control systems: games-in-games principle for cross-layer resilient control systems. IEEE Control Syst. Mag. 35(1), 46–65 (2015)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Sayin, M.O., Hosseini, H., Poovendran, R., Başar, T. (2018). A Game Theoretical Framework for Inter-process Adversarial Intervention Detection. In: Bushnell, L., Poovendran, R., Başar, T. (eds) Decision and Game Theory for Security. GameSec 2018. Lecture Notes in Computer Science(), vol 11199. Springer, Cham. https://doi.org/10.1007/978-3-030-01554-1_28
Download citation
DOI: https://doi.org/10.1007/978-3-030-01554-1_28
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-01553-4
Online ISBN: 978-3-030-01554-1
eBook Packages: Computer ScienceComputer Science (R0)