Skip to main content
Springer Nature Link
Log in

IoT Security Situational Awareness Based on Q-Learning and Bayesian Game

  • Conference paper
  • First Online:
Data Science (ICPCSEE 2021)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1452))

Abstract

IoT security is very crucial to IoT applications, and security situational awareness can assess the overall security status of the IoT. Traditional situational awareness methods only consider the unilateral impact of attack or defense, but lackconsideration of joint actions by both parties. Applying game theory to security situational awareness can measure the impact of the opposition and interdependence of the offensive and defensive parties. This paper proposes an IoT security situational awareness method based on Q-Learning and Bayesian game. Through Q-Learning update, the long-term benefits of action strategies in specific states were obtained, and static Bayesian game methods were used to solve the Bayesian Nash Equilibrium of participants of different types. The proposed method comprehensively considers offensive and defensive actions, obtains optimal defense decisions in multi-state and multi-type situations, and evaluates security situation. Experimental results prove the effectiveness of this method.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution

Subscribe and save

Springer+ Basic
$34.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or eBook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others

References

  1. Shi, L., et al.: Survey of research on network security situation awareness. Comput. Eng. Appl. 55(24), 1–9 (2019)

    Google Scholar 

  2. Rapuzzi, R., Repetto, M.: Building situational awareness for network threats in fog/edge computing: emerging paradigms beyond the security perimeter model. Futur. Gener. Comput. Syst. 85, 235–249 (2018)

    Article  Google Scholar 

  3. Park, M., Oh, H., Lee, K.: Security risk measurement for information leakage in IoT-based smart homes from a situational awareness perspective. Sensors 19(9), 2148 (2019)

    Article  Google Scholar 

  4. Chouhan, P.K., McClean, S., Shackleton, M.: Situation assessment to secure IoT applications. In: 2018 Fifth International Conference on Internet of Things: Systems, Management and Security, pp. 70–77. IEEE (2018)

    Google Scholar 

  5. He, F., et al.: SCPN-based game model for security situational awareness in the intenet of things. In: 2018 IEEE Conference on Communications and Network Security (CNS), pp. 1–5. IEEE (2018)

    Google Scholar 

  6. Kaur, R., Kaur, N., Sood, S.K.: Security in IoT network based on stochastic game net model. Int. J. Netw. Manag. 27(4), e1975 (2017)

    Article  Google Scholar 

  7. Zhang, Y., Tan, X.B., Cui, X.L., Xi, H.S.: Network security situation awareness approach based on Markov game model. J. Softw. 22(3), 495–508 (2011)

    Article  Google Scholar 

  8. Huang, J., Zhang, H., Wang, J.: Markov evolutionary games for network defense strategy selection. IEEE Access 5, 19505–19516 (2017)

    Article  Google Scholar 

  9. Lei, C., Ma, D.H., Zhang, H.Q.: Optimal strategy selection for moving target defense based on Markov game. IEEE Access 5, 156–169 (2017)

    Article  Google Scholar 

  10. Wang, Z., et al.: Optimal network defense strategy selection based on Markov Bayesian game. KSII Trans. Internet Inform. Syst. 13(11), 5631–5652 (2019)

    Google Scholar 

  11. Sutton, R.S., Barto, A.G.: Reinforcement Learning: An Introduction. MIT Press (2018)

    Google Scholar 

  12. Wang, J., Yu, D., Zhang, H., Wang, N.: Active defense strategy selection based on the static Bayesian game. J. Xidian Univ. 43(01), 144–150 (2016)

    Google Scholar 

  13. Hu, J., Wellman, M.P.: Nash Q-learning for general-sum stochastic games. J. Mach. Learn. Res. 4, 1039–1069 (2003)

    MathSciNet  MATH  Google Scholar 

  14. Gibbons, R.: A Primer in Game Theory (1992)

    Google Scholar 

  15. Subba, B., Biswas, S., Karmakar, S.: Intrusion detection in mobile ad-hoc networks: Bayesian game formulation. Eng. Sci. Technol. 19(2), 782–799 (2016)

    Google Scholar 

  16. Wang, Y., et al.: A survey of game theoretic methods for cyber security. In: 2016 IEEE First International Conference on Data Science in Cyberspace (DSC), pp. 631–636. IEEE (2016)

    Google Scholar 

  17. Watkins, C.J.C.H., Dayan, P.: Q-learning. Mach. Learn. 8(3–4), 279–292 (1992)

    MATH  Google Scholar 

  18. Shamshirband, S., et al.: Cooperative game theoretic approach using fuzzy Q-learning for detecting and preventing intrusions in wireless sensor networks. Eng. Appl. Artif. Intell. 32, 228–241 (2014)

    Article  Google Scholar 

Download references

Acknowledgment

This work is supported by the National Key Research and Development Program of China (No. 2017YFB1400700).

Author information

Authors and Affiliations

  1. School of Information, Central University of Finance and Economics, Beijing, 100081, China

    Yang Li, Tianying Liu, Jianming Zhu & Xiuli Wang

  2. Engineering Research Center of State Financial Security, Ministry of Education, Central University of Finance and Economics, Beijing, 102206, China

    Yang Li, Jianming Zhu & Xiuli Wang

Authors
  1. Yang Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tianying Liu
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Jianming Zhu
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiuli Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Yang Li .

Editor information

Editors and Affiliations

  1. North University of China, Taiyuan, China

    Jianchao Zeng

  2. North University of China, Taiyuan, China

    Pinle Qin

  3. Northeast Forestry University, Harbin, China

    Weipeng Jing

  4. Harbin University of Science and Technology, Harbin, China

    Xianhua Song

  5. National Academy of Guo Ding Institute of Data Science, Beijing, China

    Zeguang Lu

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Singapore Pte Ltd.

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Li, Y., Liu, T., Zhu, J., Wang, X. (2021). IoT Security Situational Awareness Based on Q-Learning and Bayesian Game. In: Zeng, J., Qin, P., Jing, W., Song, X., Lu, Z. (eds) Data Science. ICPCSEE 2021. Communications in Computer and Information Science, vol 1452. Springer, Singapore. https://doi.org/10.1007/978-981-16-5943-0_16

Download citation

  • DOI: https://doi.org/10.1007/978-981-16-5943-0_16

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-16-5942-3

  • Online ISBN: 978-981-16-5943-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics