Abstract
Despite the great progress of deep neural networks (DNNs), they are vulnerable to backdoor attacks. To detect and provide concrete proof for the existence of backdoors, existing techniques generally adopt the reverse engineering approach. However, most of them suffer from high computational complexity and weak scalability. In this paper, we make a key observation that the weights connected to the backdoor target labels in trojaned DNNs tend to have abnormal distributions, including dissimilarity to other labels and anomalously large magnitude. Based on this observation, we propose an efficient and scalable backdoor detection framework guided by static weight analysis. Our approach first detects the outlier existing in weight distributions and identifies suspicious backdoor target/victim label pairs. Then we conduct reverse engineering to recover the triggers, including a newly designed reverse engineering approach for global transformation attacks and one existing approach for local patch attacks. Finally, we analyze the characteristics of the recovered triggers to suppress false positives. Experimental results show that our approach has state-of-the-art performance on MNIST, CIFAR-10, ImageNet, and TrojAI. In particular, it outperforms NC, ABS, and K-Arm by 31%, 8.7%, and 5% on the public detection benchmark TrojAI in terms of detection accuracy while maintaining the highest efficiency.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Datasets details can be found in https://pages.nist.gov/trojai/docs/data.html.
References
acoomans (2013). https://github.com/acoomans/instagram-filters
Bau, D., Zhu, J.Y., Strobelt, H., Lapedriza, A., Zhou, B., Torralba, A.: Understanding the role of individual units in a deep neural network. Proc. Natl. Acad. Sci. 117(48), 30071–30078 (2020)
Bojarski, M., et al.: End to end learning for self-driving cars. CoRR abs/1604.07316 (2016)
Chen, B., et al.: Detecting backdoor attacks on deep neural networks by activation clustering. CoRR abs/1811.03728 (2018)
Chou, E., Tramèr, F., Pellegrino, G.: Sentinet: detecting localized universal attacks against deep learning systems. In: 2020 IEEE Security and Privacy Workshops, SP Workshops, San Francisco, 21 May 2020, pp. 48–54. IEEE (2020)
Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: a large-scale hierarchical image database. In: 2009 IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255. IEEE (2009)
Gao, Y., Xu, C., Wang, D., Chen, S., Ranasinghe, D.C., Nepal, S.: Strip: a defence against trojan attacks on deep neural networks. In: Proceedings of the 35th Annual Computer Security Applications Conference, pp. 113–125 (2019)
Gu, T., Dolan-Gavitt, B., Garg, S.: Badnets: Identifying vulnerabilities in the machine learning model supply chain. CoRR abs/1708.06733 (2017)
Guo, W., Wang, L., Xing, X., Du, M., Song, D.: Tabor: a highly accurate approach to inspecting and restoring trojan backdoors in AI systems. arXiv preprint arXiv:1908.01763 (2019)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Huang, S., Peng, W., Jia, Z., Tu, Z.: One-pixel signature: characterizing CNN models for backdoor detection. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12372, pp. 326–341. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58583-9_20
IARPA: Trojai competition (2020). https://pages.nist.gov/trojai/
Karra, K., Ashcraft, C., Fendley, N.: The trojai software framework: An opensource tool for embedding trojans into deep learning models. arXiv preprint arXiv:2003.07233 (2020)
Kolouri, S., Saha, A., Pirsiavash, H., Hoffmann, H.: Universal litmus patterns: revealing backdoor attacks in CNNs. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 301–310 (2020)
Krizhevsky, A., Hinton, G., et al.: Learning multiple layers of features from tiny images. Technical report (2009)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. Adv. Neural. Inf. Process. Syst. 25, 1097–1105 (2012)
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)
Lin, J., Xu, L., Liu, Y., Zhang, X.: Composite backdoor attack for deep neural network by mixing existing benign features. In: Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security, pp. 113–131 (2020)
Liu, Y., Lee, W.C., Tao, G., Ma, S., Aafer, Y., Zhang, X.: Abs: scanning neural networks for back-doors by artificial brain stimulation. In: Proceedings of the 2019 ACM SIGSAC Conference on Computer and Communications Security, pp. 1265–1282 (2019)
Liu, Y., et al.: Trojaning attack on neural networks. In: 25th Annual Network and Distributed System Security Symposium, NDSS 2018, San Diego, 18–21 February 2018 (2018)
Nguyen, A., Tran, A.: Wanet-imperceptible warping-based backdoor attack. arXiv preprint arXiv:2102.10369 (2021)
Nguyen, A., Yosinski, J., Clune, J.: Deep neural networks are easily fooled: high confidence predictions for unrecognizable images. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 427–436 (2015)
Saha, A., Subramanya, A., Pirsiavash, H.: Hidden trigger backdoor attacks. In: The Thirty-Fourth AAAI Conference on Artificial Intelligence, AAAI 2020, pp. 11957–11965. AAAI Press (2020)
Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.C.: Mobilenetv 2: inverted residuals and linear bottlenecks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4510–4520 (2018)
Saxe, J., Berlin, K.: Deep neural network based malware detection using two dimensional binary program features. In: 2015 10th International Conference on Malicious and Unwanted Software (MALWARE), pp. 11–20. IEEE (2015)
Shen, G., et al.: Backdoor scanning for deep neural networks through k-arm optimization. In: Proceedings of the 38th International Conference on Machine Learning, ICML 2021, Virtual Event, 18–24 July 2021, pp. 9525–9536 (2021)
Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 (2014)
Tang, R., Du, M., Liu, N., Yang, F., Hu, X.: An embarrassingly simple approach for trojan attack in deep neural networks. In: KDD 2020: The 26th ACM SIGKDD Conference on Knowledge Discovery and Data Mining, Virtual Event, CA, 23–27 August 2020, pp. 218–228. ACM (2020)
Torchvision (2020). https://github.com/pytorch/vision/tree/main/torchvision
Tran, B., Li, J., Madry, A.: Spectral signatures in backdoor attacks. In: Advances in Neural Information Processing Systems 31: Annual Conference on Neural Information Processing Systems 2018, NeurIPS 2018, Montréal, 3–8 December 2018, pp. 8011–8021 (2018)
Wang, B., et al.: Neural cleanse: identifying and mitigating backdoor attacks in neural networks. In: 2019 IEEE Symposium on Security and Privacy (SP), pp. 707–723. IEEE (2019)
Wang, R., Zhang, G., Liu, S., Chen, P., Xiong, J., Wang, M.: Practical detection of trojan neural networks: data-limited and data-free cases. In: Computer Vision - ECCV 2020–16th European Conference, Glasgow, UK, 23–28 August 2020, Proceedings, Part XXIII, pp. 222–238 (2020)
Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)
Xu, X., Wang, Q., Li, H., Borisov, N., Gunter, C.A., Li, B.: Detecting AI trojans using meta neural analysis. In: 2021 IEEE Symposium on Security and Privacy (SP), pp. 103–120. IEEE (2021)
Yu, H., et al.: Tensorflow model garden (2020). https://github.com/tensorflow/models
Zhang, Y., Liang, P., Wainwright, M.J.: Convexified convolutional neural networks. In: Precup, D., Teh, Y.W. (eds.) Proceedings of the 34th International Conference on Machine Learning, ICML 2017, Sydney, NSW, Australia, 6–11 August 2017. Proceedings of Machine Learning Research, vol. 70, pp. 4044–4053. PMLR (2017)
Acknowledgement
This work was supported in part by National Natural Science Foundation of China under Grant 62172305 and Key R &D in Hubei Province.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Wang, Q., Li, W., Yang, K., Zhao, Y., Zhao, L., Wang, L. (2023). Efficient DNN Backdoor Detection Guided by Static Weight Analysis. In: Deng, Y., Yung, M. (eds) Information Security and Cryptology. Inscrypt 2022. Lecture Notes in Computer Science, vol 13837. Springer, Cham. https://doi.org/10.1007/978-3-031-26553-2_22
Download citation
DOI: https://doi.org/10.1007/978-3-031-26553-2_22
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-26552-5
Online ISBN: 978-3-031-26553-2
eBook Packages: Computer ScienceComputer Science (R0)