Skip to main content
SpringerLink
Log in

Decision model of intrusion response based on markov game in fog computing environment

  • Original Paper
  • Published:
Wireless Networks Aims and scope Submit manuscript

Abstract

For the current problems of complex network state, difficulty in fast response to intrusion and poor adaptation of response decision in fog computing environment, in this paper, we propose an intrusion response decision method based on a combination of deep learning and reinforcement learning based on game theory (Minimax-DQN). First, a Markov game is used as the standard to construct the intrusion response decision model in the fog computing environment. Second, two different sets of random variables are defined considering the possible behavioral choices taken by the attacker and the fog computing intrusion detection systems (FC-IDS), and the continuous state space formed by the game between the attacker and the FC-IDS is processed using a deep Q-network. Finally, the Minimax algorithm is used to solve the optimal value function in a specific state, and the best intrusion response strategy is obtained according to the training output after the training is completed. Three sets of experiments are conducted to compare the results of the Minimax-DQN algorithm, the DQN algorithm and the random strategy. The experimental results data prove that the model and the proposed algorithm can greatly improve the probability of IDS winning in the game process with the attacker, and thus effectively solve the problem of intrusion response decision in fog environment.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Bhatia, J., Italiya, K., Jadeja, K., Kumhar, M., Chauhan, U., Tanwar, S., Bhavsar, M., Sharma, R., Manea, D. L., Verdes, M., & Raboaca, M. S. (2023). An overview of fog data analytics for IoT applications. Sensors, 23, 199.

    Article  Google Scholar 

  2. Ivan, S., Wen, S., Huang, X., et al. (2016). An overview of Fog computing and its security issues. Concurrency & Computation Practice & Experience, 28(10), 2991–3005.

    Article  Google Scholar 

  3. Liu, C., Wang, P., Xiang, F., & Sun, Z. (2019). A review of issues and challenges in fog computing environment. In 2019 IEEE Intl Conf on dependable, autonomic and secure computing, Intl Conf on pervasive intelligence and computing, Intl Conf on cloud and big data computing, Intl Conf on cyber science and technology congress (DASC/PiCom/CBDCom/CyberSciTech). IEEE.

  4. Oma, R., Nakamura, S., Duolikun, D., Enokido, T., & Takizawa, M. (2018). An energy-efficient model for fog computing in the Internet of Things (IoT). Internet of Things, 1–2, 14–26.

    Article  Google Scholar 

  5. Jalasri, M., & Lakshmanan, L. (2023). Managing data security in fog computing in IoT devices using noise framework encryption with power probabilistic clustering algorithm. Cluster Computing, 26, 823–836.

    Article  Google Scholar 

  6. Abu Al-Haija, Q., & Zein-Sabatto, S. (2020). An efficient deep-learning-based detection and classification system for cyber-attacks in IoT communication networks. Electronics, 9, 2152.

    Article  Google Scholar 

  7. Elhaija, W. A., & Al-Haija, Q. A. (2023). A novel dataset and lightweight detection system for broken bars induction motors using optimizable neural networks. Intelligent Systems with Applications, 17, 200167.

    Article  Google Scholar 

  8. Smadi, A. A., Ajao, B. T., Johnson, B. K., Lei, H., Chakhchoukh, Y., & Abu Al-Haija, Q. (2021). A comprehensive survey on cyber-physical smart grid testbed architectures: Requirements and challenges. Electronics, 10, 1043.

    Article  Google Scholar 

  9. Guo, Y., Zhang, H., Li, Z., Li, F., Fang, L., Yin, L., & Cao, J. (2020). Decision-making for intrusion response: Which, where, in what order, and how long. In ICC 2020—2020 IEEE international conference on communications (ICC). IEEE.

  10. Ometov, A., Molua, O. L., Komarov, M., & Nurmi, J. (2022). A survey of security in cloud, edge, and fog computing. Sensors, 22, 927.

    Article  Google Scholar 

  11. Malialis, K., Devlin, S., & Kudenko, D. (2015). Distributed reinforcement learning for adaptive and robust network intrusion response. Connection Science, 27(3), 234–252.

    Article  Google Scholar 

  12. Zidi, S., Mihoub, A., Qaisar, S. M., Krichen, M., & Al-Haija, Q. A. (2023). Theft detection dataset for benchmarking and machine learning based classification in a smart grid environment. Journal of King Saud University - Computer and Information Sciences, 35(1), 13–25.

    Article  Google Scholar 

  13. Lopes, A., & Hutchison, A. (2020). Experimenting with machine learning in automated intrusion response. In Intelligent Distributed Computing XIII (pp. 505–514). Springer International Publishing.

  14. Miehling, E., Rasouli, M., & Teneketzis, D. (2015). Optimal defense policies for partially observable spreading processes on Bayesian attack graphs. In ACM workshop on moving target defense (pp. 67–76).

  15. Stakhanova, N., Basu, S., & Wong, J. (2007). A taxonomy of intrusion response systems. Inderscience Publishers.

    Book  Google Scholar 

  16. Mu, C., Huang, H., Tian, S., & Xiangjun, L. (2008). A survey of intrusion response decision-making techniques of automated intrusion response systems. Journal of Computer Research and Development, 45(8), 1290–1298.

    Google Scholar 

  17. Inayat, Z., Gani, A., Anuar, N. B., Khan, M. K., & Anwar, S. (2016). Intrusion response systems. Journal of Network & Computer Applications, 62(C), 53–74.

    Article  Google Scholar 

  18. Somayaji, A., & Forrest, S. (2000). Automated response using system-call delay. In USENIX security symposium (pp. 185–197).

  19. Schnackengerg, D., Holliday, H., Smith, R., Djahandari, K., & Sterne, D. (2001). Cooperative intrusion traceback and response architecture (CITRA). In DARPA information survivability conference & exposition II, 2001. DISCEX’01. Proceedings (Vol. 1, pp. 56–68). IEEE.

  20. Iafarov, R., Gad, R., & Kappes, M. (2015). Improving attack mitigation with a cost-sensitive and adaptive intrusion response system. In ICN 2015: The fourteenth international conference on networks.

  21. Shameli-Sendi, A., Louafi, H., He, W., & Cheriet, M. (2016). Dynamic optimal countermeasure selection for intrusion response system. IEEE Transactions on Dependable & Secure Computing, PP(99), 1–1.

    Google Scholar 

  22. Ullah, S., Shelly, S., Hassanzadeh, A., Nayak, A., & Hasan, K. (2020). On the effectiveness of intrusion response systems against persistent threats. In 2020 international conference on computing, networking and communications (ICNC).

  23. Yin, Y., Chen, L., Xu, Y., Wan, J., Zhang, H., & Mai, Z. (2019). QoS prediction for service recommendation with deep feature learning in edge computing environment. Mobile Networks and Applications, 25, 391–401.

    Article  Google Scholar 

  24. Sarker, I. H., Colman, A., Han, J., Khan, A. I., Abushark, Y. B., & Salah, K. (2020). BehavDT: A behavioral decision tree learning to build user-centric context-aware predictive model. Mobile Networks and Applications, 25(3), 1151–1161.

    Article  Google Scholar 

  25. Iannucci, S., Barba, O. D., Cardellini, V., & Banicescu, I. (2019). A performance evaluation of deep reinforcement learning for model-based intrusion response. In 2019 IEEE 4th international workshops on foundations and applications of self* systems (FAS*W) (pp. 158–163). IEEE.

  26. Hosu, I. A., & Rebedea, T. (2016). Playing Atari games with deep reinforcement learning and human checkpoint replay.

  27. Iannucci, S., Cardellini, V., Barba, O. D., & Banicescu, I. (2020). A hybrid model-free approach for the near-optimal intrusion response control of non-stationary systems. Future Generation Computer Systems, 109, 111–124.

    Article  Google Scholar 

  28. Littman, M. L. (1994). Markov games as a framework for multi-agent reinforcement learning. Morgan Kauffman Publishers, Inc.

    Book  Google Scholar 

  29. Yan, W. Q. (2021). Computational methods for deep learning—Theoretic, practice and applications Texts in Computer Science (pp. 1–119). Springer. ISBN 978-3-030-61080-7.

    Book  Google Scholar 

Download references

Acknowledgements

This project is supported by Natural Science Foundation of China (Nos. 61572170, 61170254). We hereby express our thanks.

Author information

Authors and Affiliations

  1. Department of Computer Teaching, Hebei University, Baoding, 071002, China

    Xiaoxue Ma

  2. Hebei Key Laboratory of Highly Trusted Information System, Hebei University, Baoding, China

    Yun Li

  3. School of Cyber Security and Computer, Hebei University, Baoding, China

    Yan Gao

Authors
  1. Xiaoxue Ma
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Yun Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Yan Gao
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yan Gao.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ma, X., Li, Y. & Gao, Y. Decision model of intrusion response based on markov game in fog computing environment. Wireless Netw 29, 3383–3392 (2023). https://doi.org/10.1007/s11276-023-03382-w

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11276-023-03382-w

Keywords

Search

Navigation