Abstract
False data injection attacks modify the meter measurements to mislead the control center into estimating inaccurate system states and thus affect the reliable operation of smart grids. In this paper, we propose a deep reinforcement learning based vulnerability analysis scheme for smart grids that enables the control center to construct an attack vector from the attacker’s view to identify the vulnerable meters. The control center chooses the attack vector based on power system states, meter measurements, the previous number of analyzed meters, and injected errors without knowing the power system topology. This scheme designs an actor-critic architecture that applies an actor network to output the policy probability distribution to handle the continuous and high-dimensional vulnerability analysis policy and contains a critic network to guide the weights update of the actor network. We also analyze the computational complexity and perform simulations to verify the efficacy of this scheme in terms of the vulnerability detection rate, the number of analyzed meters and the utility.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Liu, X., Li, Z., Liu, X., Li, Z.: Masking transmission line outages via false data injection attacks. IEEE Trans. Inf. Forensics Security 11(7), 1592–1602 (2016)
Zhang, Q., Li, F., Shi, Q., Tomsovic, K., Sun, J., Ren, L.: Profit-oriented false data injection on electricity market: reviews, analyses, and insights. IEEE Trans. Ind. Informat. 17(9), 5876–5886 (2020)
Gu, C., Panida, J., Mehul, M.: Detecting false data injection attacks in ac state estimation. IEEE Trans. Smart Grid 6(5), 2476–2483 (2015)
Chen, Y., Huang, S., Liu, F., Wang, Z., Sun, X.: Evaluation of reinforcement learning-based false data injection attack to automatic voltage control. IEEE Trans. Smart Grid 10(2), 2158–2169 (2018)
Luo, W., Xiao, L.: Reinforcement learning based vulnerability analysis of data injection attack for smart grids. In: 2021 40th Chinese Control Conference (CCC), pp. 6788–6792. IEEE (2021). http://orcid.org/10.23919/CCC52363.2021.9550523
Xie, L., Mo, Y., Sinopoli, B.: Integrity data attacks in power market operations. IEEE Trans. Smart Grid 2(4), 659–666 (2011)
Hao, J., Piechocki, R.J., Kaleshi, D., Chin, W.H., Fan, Z.: Sparse malicious false data injection attacks and defense mechanisms in smart grids. IEEE Trans. Ind. Informat. 11(5), 1–12 (2015)
Kim, T.T., Poor, H.V.: Strategic protection against data injection attacks on power grids. IEEE Trans. Smart Grid 2(2), 326–333 (2011)
Rahman, M.A., Mohsenian-Rad, H.: False data injection attacks with incomplete information against smart power grids. In: 2012 IEEE Global Communications Conference (GLOBECOM), pp. 3153–3158 (2012). http://orcid.org/10.1109/GLOCOM.2012.6503599
Liu, X., Bao, Z., Lu, D., Li, Z.: Modeling of local false data injection attacks with reduced network information. IEEE Trans. Smart Grid 6(4), 1686–1696 (2015)
Deng, R., Liang, H.: False data injection attacks with limited susceptance information and new countermeasures in smart grid. IEEE Trans. Ind. Informat. 15(3), 1619–1628 (2018)
Yan, J., He, H., Zhong, X., Tang, Y.: Q-learning-based vulnerability analysis of smart grid against sequential topology attacks. IEEE Trans. Inf. Forensics Security 12(1), 200–210 (2016)
Yu, Z.H., Chin, W.L.: Blind false data injection attack using PCA approximation method in smart grid. IEEE Trans. Smart Grid 6(3), 1219–1226 (2015)
Liu, X., Zhu, P., Zhang, Y., Chen, K.: A collaborative intrusion detection mechanism against false data injection attack in advanced metering infrastructure. IEEE Trans. Smart Grid 6(5), 2435–2443 (2015)
Liang, G., Zhao, J., Luo, F., Weller, S.R., Dong, Z.Y.: A review of false data injection attacks against modern power systems. IEEE Trans. Smart Grid 8(4), 1630–1638 (2016)
Kingma, D.P., Ba, J.: Adam: A method for stochastic optimization. In: Bengio, Y., LeCun, Y. (eds.) 3rd International Conference on Learning Representations, ICLR 2015, San Diego, CA, USA, 2015, Conference Track Proceedings (2015). arxiv.org/abs/1412.6980
Ou, G., Murphey, Y.L.: Multi-class pattern classification using neural networks. Pattern Recogn. 40(1), 4–18 (2007)
Huang, G.B.: Learning capability and storage capacity of two-hidden-layer feedforward networks. IEEE Trans. Neural Netw. 14(2), 274–281 (2003)
Acknowledgements
This work was supported in part by the Natural Science Foundation of China under Grant 61971366 and Grant U21A20444.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Xu, S., Yu, S., Xiao, L., Lv, Z. (2022). Reinforcement Learning Based Vulnerability Analysis for Smart Grids Against False Data Injection Attacks. In: Wang, L., Segal, M., Chen, J., Qiu, T. (eds) Wireless Algorithms, Systems, and Applications. WASA 2022. Lecture Notes in Computer Science, vol 13471. Springer, Cham. https://doi.org/10.1007/978-3-031-19208-1_36
Download citation
DOI: https://doi.org/10.1007/978-3-031-19208-1_36
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-19207-4
Online ISBN: 978-3-031-19208-1
eBook Packages: Computer ScienceComputer Science (R0)