Skip to main content
SpringerLink
Log in

Deep reinforcement learning for optimal denial-of-service attacks scheduling

  • Research Paper
  • Published:
Science China Information Sciences Aims and scope Submit manuscript

Abstract

We consider an optimal denial-of-service (DoS) attack scheduling problem of N independent linear time-invariant processes, where sensors have limited computational capability. Sensors transmit measurements to the remote estimator via a communication channel that is exposed to DoS attackers. However, due to limited energy, an attacker can only attack a subset of sensors at each time step. To maximally degrade the estimation performance, a DoS attacker needs to determine which sensors to attack at each time step. In this context, a deep reinforcement learning (DRL) algorithm, which combines Q-learning with a deep neural network, is introduced to solve the Markov decision process (MDP). The DoS attack scheduling optimization problem is formulated as an MDP that is solved by the DRL algorithm. A numerical example is provided to illustrate the efficiency of the optimal DoS attack scheduling scheme using the DRL algorithm.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Wu G Y, Wang G, Sun J, et al. Optimal partial feedback attacks in cyber-physical power systems. IEEE Trans Autom Control, 2020, 65: 3919–3926

    Article  MathSciNet  MATH  Google Scholar 

  2. Pang Z H, Liu G P, Zhou D H, et al. Data-based predictive control for networked nonlinear systems with network-induced delay and packet dropout. IEEE Trans Ind Electron, 2016, 63: 1249–1257

    Article  Google Scholar 

  3. Lu A Y, Yang G H. Input-to-state stabilizing control for cyber-physical systems with multiple transmission channels under denial of service. IEEE Trans Autom Control, 2018, 63: 1813–1820

    Article  MathSciNet  MATH  Google Scholar 

  4. Wang G, Giannakis G B, Chen J, et al. Distribution system state estimation: an overview of recent developments. Front Inf Technol Electron Eng, 2019, 20: 4–17

    Article  Google Scholar 

  5. Wu C W, Hu Z R, Liu J X, et al. Secure estimation for cyber-physical systems via sliding mode. IEEE Trans Cybern, 2018, 48: 3420–3431

    Article  Google Scholar 

  6. Pang Z H, Liu G P, Zhou D, et al. Two-channel false data injection attacks against output tracking control of networked systems. IEEE Trans Ind Electron, 2016, 63: 3242–3251

    Article  Google Scholar 

  7. Chen J, Wang G, Sun J. Power scheduling for Kalman filtering over lossy wireless sensor networks. IET Control Theory Appl, 2016, 11: 531–540

    Article  MathSciNet  Google Scholar 

  8. Yang G, Zhou X S. Intelligent CPS: features and challenges. Sci China Inf Sci, 2016, 59: 050102

    Article  MathSciNet  Google Scholar 

  9. Gupta B, Brij B. Computer and Cyber Security: Principles, Algorithm, Applications, and Perspectives. Boca Raton: CRC Press, 2018

    Google Scholar 

  10. Vijayan J. Stuxnet renews power grid security concerns. Computerworld, 2010. http://www.computerworld.com/artical/2519574/security0/stuxnet-renews-power-grid-security-concerns.html

  11. Teixeira A, Shames I, Sandberg H, et al. A secure control framework for resource-limited adversaries. Automatica, 2015, 51: 135–148

    Article  MathSciNet  MATH  Google Scholar 

  12. Kwon C, Liu W Y, Hwang I. Security analysis for cyber-physical systems against stealthy deception attacks. In: Proceedings of American Control Conference, Washington, 2013. 3344–3349

  13. Mo Y, Chabukswar R, Sinopoli B. Detecting integrity attacks on SCADA systems. IEEE Trans Control Syst Technol, 2014, 22: 1396–1407

    Article  Google Scholar 

  14. Guo Z Y, Shi D W, Johansson K H, et al. Worst-case stealthy innovation-based linear attack on remote state estimation. Automatica, 2018, 89: 117–124

    Article  MathSciNet  MATH  Google Scholar 

  15. Wu G Y, Sun J, Chen J. Optimal data injection attacks in cyber-physical systems. IEEE Trans Cybern, 2018, 48: 3302–3312

    Article  Google Scholar 

  16. Mo Y, Sinopoli B. Secure control against replay attacks. In: Proceedings of Annual Allerton Conference, Illinois, 2009. 911–918

  17. Teixeira A, Shames I, Sandberg H, et al. Revealing stealthy attacks in control systems. In: Proceedings of Annual Allerton Conference, Illinois, 2012. 1806–1813

  18. Hou F Y, Sun J. Covert attacks against output tracking control of cyber-physical systems. In: Proceedings of the 43rd Annual Conference of the IEEE Industrial Electronics Society, Beijing, 2017. 5743–5748

  19. Cárdenas A, Amin S, Sastry S. Research challenges for the security of control systems. In: Proceedings of Conference Hot Topics Security, California, 2018

  20. Yang Y, Li Y F, Yue D. Event-trigger-based consensus secure control of linear multi-agent systems under DoS attacks over multiple transmission channels. Sci China Inf Sci, 2020, 63: 150208

    Article  MathSciNet  Google Scholar 

  21. Li W T, Wen C K, Chen J C, et al. Location identification of power line outages using PMU measurements with bad data. IEEE Trans Power Syst, 2016, 31: 3624–3635

    Article  Google Scholar 

  22. Amin S, Cárdenas A, Sastry S. Safe and secure networked control systems under denial-of-service attacks. In: Proceedings of International Workshop on Hybrid Systems: Computation and Control, 2009. 31–45

  23. Wang J Z, Shi L. Optimal DoS attacks on remote state estimation with a router. In: Proceedings of IEEE Conference on Decision and Control, Florida, 2018. 6384–6389

  24. Chhabra M, Gupta B, Almomani A. A novel solution to handle DDOS attack in MANET. J Inform Secur, 2013, 4: 165–179

    Article  Google Scholar 

  25. Bhushan K, Gupta B B. Distributed denial of service (DDoS) attack mitigation in software defined network (SDN)-based cloud computing environment. J Ambient Intell Human Comput, 2019, 10: 1985–1997

    Article  Google Scholar 

  26. Bhardwaj A, Goundar S. Comparing single tier and three tier infrastructure designs against DDoS attacks. Int J Cloud Appl Comput, 2017, 7: 59–75

    Google Scholar 

  27. Almomani A, Alauthman M, Albalas F, et al. An online intrusion detection system to cloud computing based on NeuCube algorithms. Intern J Cloud App Comp, 2018, 8: 96–112

    Google Scholar 

  28. Zhang H, Cheng P, Shi L, et al. Optimal DoS attack scheduling in wireless networked control system. IEEE Trans Contr Syst Technol, 2016, 24: 843–852

    Article  Google Scholar 

  29. Zhang H, Qi Y F, Wu J F, et al. DoS attack energy management against remote state estimation. IEEE Trans Control Netw Syst, 2018, 5: 383–394

    Article  MathSciNet  MATH  Google Scholar 

  30. Qin J H, Li M L, Wang J, et al. Optimal denial-of-service attack energy management against state estimation over an SINR-based network. Automatica, 2020, 119: 109090

    Article  MathSciNet  MATH  Google Scholar 

  31. Qin J H, Li M L, Shi L, et al. Optimal denial-of-service attack scheduling with energy constraint over packet-dropping networks. IEEE Trans Autom Control, 2018, 63: 1648–1663

    Article  MathSciNet  MATH  Google Scholar 

  32. Zhang H, Zhu Y Z, Li Z J, et al. DoS attack power allocation against remote state estimation via a block fading channel. In: Proceedings of Chinese Control Conference, Wuhan, 2018. 6441–6446

  33. Yang C, Yang W, Shi H B. DoS attack in centralised sensor network against state estimation. IET Control Theory Appl, 2018, 12: 1244–1253

    Article  MathSciNet  Google Scholar 

  34. Mnih V, Kavukcuoglu K, Silver D, et al. Playing atari with deep reinforcement learning. 2013. ArXiv:1312.5602v1

  35. Mnih V, Kavukcuoglu K, Silver D, et al. Human-level control through deep reinforcement learning. Nature, 2015, 518: 529–533

    Article  Google Scholar 

  36. Yang Q L, Wang G, Sadeghi A, et al. Two-timescale voltage control in distribution grids using deep reinforcement learning. IEEE Trans Smart Grid, 2020, 11: 2313–2323

    Article  Google Scholar 

  37. Leong A, Ramaswamy A, Duevedo D, et al. Deep reinforcement learning for wireless sensor scheduling in cyber-physical systems. Automatica, 2020, 113: 108759

    Article  MathSciNet  MATH  Google Scholar 

  38. Sadeghi A, Wang G, Giannakis G B. Deep reinforcement learning for adaptive caching in hierarchical content delivery networks. IEEE Trans Cogn Commun Netw, 2019, 5: 1024–1033

    Article  Google Scholar 

  39. Lian J, Li C, Xia B. Sampled-data control of switched linear systems with application to an F-18 aircraft. IEEE Trans Ind Electron, 2017, 64: 1332–1340

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by National Natural Science Foundation of China (Grant Nos. U1613225, 61925303, 62088101, 61673023).

Author information

Authors and Affiliations

  1. State Key Laboratory of Intelligent Control and Decision of Complex Systems, Beijing Institute of Technology, Beijing, 100081, China

    Fangyuan Hou, Jian Sun & Qiuling Yang

  2. Key Laboratory of Fieldbus Technology and Automation of Beijing, North China University of Technology, Beijing, 100144, China

    Zhonghua Pang

Authors
  1. Fangyuan Hou
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jian Sun
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Qiuling Yang
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Zhonghua Pang
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jian Sun.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hou, F., Sun, J., Yang, Q. et al. Deep reinforcement learning for optimal denial-of-service attacks scheduling. Sci. China Inf. Sci. 65, 162201 (2022). https://doi.org/10.1007/s11432-020-3027-0

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11432-020-3027-0

Keywords

Search

Navigation