Xudong Pan
ABSTRACT
Deep learning with edge computing arises as a popular paradigm for powering edge devices with intelligence. As the size of deep neural networks (DNN) continually increases, model quantization, which converts the full-precision model into lower-bit representation while mostly preserving the accuracy, becomes a prerequisite for deploying a well-trained DNN on resource-limited edge devices. However, to properly quantize a DNN requires an essential amount of expert knowledge, or otherwise the model accuracy would be devastatingly affected. Alternatively, recent years witness the birth of third-party model supply chains which provide pretrained quantized neural networks (QNN) for free downloading.
In this paper, we systematically analyze the potential threats of trojaned models in third-party QNN supply chains. For the first time, we describe and implement a QUAntization-SpecIfic backdoor attack (QUASI), which manipulates the quantization mechanism to inject a backdoor specific to the quantized model. In other words, the attacker-specified inputs, or triggers, would not cause misbehaviors of the trojaned model in full precision until the backdoor function is automatically completed by a normal quantization operation, producing a trojaned QNN which can be triggered with a near 100% success rate. Our proposed QUASI attack reveals several key vulnerabilities in the existing QNN supply chains: (i) QUASI demonstrates a third-party QNN released online can also be injected with backdoors, while, unlike full-precision models, there is almost no working algorithm for checking the fidelity of a QNN. (ii) More threateningly, the backdoor injected by QUASI remains inactivated in the full-precision model, which inhibits model consumers from attributing undergoing trojan attacks to the malicious model provider. As a practical implication, we alarm it can be highly risky to accept and deploy third-party QNN on edge devices at the current stage, if without future mitigation studies.
- [n.d.]. Android Demo Apps - PyTorch. https://github.com/pytorch/android-demo-app. Accessed: 2021-05-21.Google Scholar
- [n.d.]. Available Models-PaddleLite. https://paddle-lite.readthedocs.io/zh/latest/introduction/support_model_list.html. Accessed: 2021-05-21.Google Scholar
- [n.d.]. Models - Machine Learing - Apple Developer. https://developer.apple.com/machine-learning/models/. Accessed: 2021-05-21.Google Scholar
- [n.d.]. Quantization-PyTorch. https://pytorch.org/docs/stable/quantization.html. Accessed: 2021-05-21.Google Scholar
- [n.d.]. Quantization-Tensorflow. https://www.tensorflow.org/model_optimization/guide/quantization/post_training. Accessed: 2021-05-21.Google Scholar
- [n.d.]. US Government’s TrojAI Program. https://www.iarpa.gov/index.php/research-programs/trojai. Accessed: 2021-02-01.Google Scholar
- Martín Abadi, P. Barham, J. Chen, and et al.2016. TensorFlow: A system for large-scale machine learning. In OSDI.Google Scholar
- E. Bagdasaryan and Vitaly Shmatikov. 2021. Blind Backdoors in Deep Learning Models. USENIX Security Symposium(2021).Google Scholar
- E. Bagdasaryan, Andreas Veit, Yiqing Hua, D. Estrin, and Vitaly Shmatikov. 2020. How To Backdoor Federated Learning. In AISTATS.Google Scholar
- Xiaoyu Cao, J. Jia, and N. Gong. 2021. IPGuard: Protecting Intellectual Property of Deep Neural Networks via Fingerprinting the Classification Boundary. Proceedings of the 2021 ACM Asia Conference on Computer and Communications Security (2021).Google ScholarDigital Library
- Nicholas Carlini and David A. Wagner. 2017. Towards Evaluating the Robustness of Neural Networks. 2017 IEEE Symposium on Security and Privacy (SP) (2017), 39–57.Google Scholar
- Bryant Chen, Wilka Carvalho, Nathalie Baracaldo, Heiko Ludwig, Ben Edwards, Taesung Lee, Ian Molloy, and B. Srivastava. 2019. Detecting Backdoor Attacks on Deep Neural Networks by Activation Clustering. ArXiv abs/1811.03728(2019).Google Scholar
- Huili Chen, Cheng Fu, J. Zhao, and F. Koushanfar. 2019. DeepInspect: A Black-box Trojan Detection and Mitigation Framework for Deep Neural Networks. In IJCAI.Google Scholar
- Jiasi Chen and Xukan Ran. 2019. Deep Learning With Edge Computing: A Review. Proc. IEEE 107(2019), 1655–1674.Google ScholarCross Ref
- X. Chen, Chang Liu, Bo Li, Kimberly Lu, and D. Song. 2017. Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning. ArXiv (2017).Google Scholar
- Matthieu Courbariaux, Yoshua Bengio, and J. David. 2015. BinaryConnect: Training Deep Neural Networks with binary weights during propagations. In NIPS.Google Scholar
- Matthieu Courbariaux, Itay Hubara, Daniel Soudry, Ran El-Yaniv, and Yoshua Bengio. 2016. Binarized Neural Networks: Training Deep Neural Networks with Weights and Activations Constrained to +1 or -1. NeurIPS (2016).Google Scholar
- Bao Gia Doan, Ehsan Abbasnejad, and D. Ranasinghe. 2020. Februus: Input Purification Defense Against Trojan Attacks on Deep Neural Network Systems. ACSAC (2020).Google Scholar
- Min Du, R. Jia, and D. Song. 2020. Robust Anomaly Detection and Backdoor Attack Detection Via Differential Privacy. ICLR (2020).Google Scholar
- Kirsty Duncan, E. Komendantskaya, Rob Stewart, and M. Lones. 2020. Relative Robustness of Quantized Neural Networks Against Adversarial Attacks. 2020 International Joint Conference on Neural Networks (IJCNN) (2020), 1–8.Google Scholar
- Yansong Gao, Chang Xu, Derui Wang, S. Chen, D. Ranasinghe, and S. Nepal. 2019. STRIP: a defence against trojan attacks on deep neural networks. ACSAC (2019).Google Scholar
- Amir Gholami, Sehoon Kim, Zhen Dong, Zhewei Yao, M. Mahoney, and K. Keutzer. 2021. A Survey of Quantization Methods for Efficient Neural Network Inference. ArXiv abs/2103.13630(2021).Google Scholar
- Ian Goodfellow, Yoshua Bengio, and Aaron Courville. 2016. Deep Learning. MIT Press. http://www.deeplearningbook.org.Google ScholarDigital Library
- I. Goodfellow, Jonathon Shlens, and Christian Szegedy. 2015. Explaining and Harnessing Adversarial Examples. CoRR abs/1412.6572(2015).Google Scholar
- R. Gray and D. Neuhoff. 1998. Quantization. IEEE Trans. Inf. Theory 44 (1998), 2325–2383.Google ScholarDigital Library
- Tianyu Gu, K. Liu, Brendan Dolan-Gavitt, and S. Garg. 2019. BadNets: Evaluating Backdooring Attacks on Deep Neural Networks. IEEE Access (2019).Google Scholar
- Wenbo Guo, Lun Wang, Yan Xu, Xinyu Xing, Min Du, and D. Song. 2020. Towards Inspecting and Eliminating Trojan Backdoors in Deep Neural Networks. ICDM (2020).Google Scholar
- Kartik Gupta and Thalaiyasingam Ajanthan. 2020. Improved Gradient based Adversarial Attacks for Quantized Networks. ArXiv abs/2003.13511(2020).Google Scholar
- Sanghyun Hong, Pietro Frigo, Yigitcan Kaya, Cristiano Giuffrida, and T. Dumitras. 2019. Terminal Brain Damage: Exposing the Graceless Degradation in Deep Neural Networks Under Hardware Fault Attacks. In USENIX Security Symposium.Google Scholar
- Benoit Jacob, S. Kligys, Bo Chen, Menglong Zhu, Matthew Tang, Andrew G. Howard, Hartwig Adam, and D. Kalenichenko. 2018. Quantization and Training of Neural Networks for Efficient Integer-Arithmetic-Only Inference. 2018 IEEE/CVF Conference on Computer Vision and Pattern Recognition (2018), 2704–2713.Google Scholar
- Yujie Ji, Xinyang Zhang, Shouling Ji, X. Luo, and Ting Wang. 2018. Model-Reuse Attacks on Deep Learning Systems. Proceedings of the 2018 ACM SIGSAC Conference on Computer and Communications Security (2018).Google ScholarDigital Library
- Elias Boutros Khalil, Amrita Gupta, and B. Dilkina. 2019. Combinatorial Attacks on Binarized Neural Networks. ICLR (2019).Google Scholar
- Yoongu Kim, Ross Daly, Jeremie S. Kim, Chris Fallin, Ji-Hye Lee, Donghyuk Lee, C. Wilkerson, K. Lai, and O. Mutlu. 2014. Flipping bits in memory without accessing them: An experimental study of DRAM disturbance errors. 2014 ACM/IEEE 41st International Symposium on Computer Architecture (ISCA) (2014), 361–372.Google ScholarCross Ref
- Diederik P. Kingma and Jimmy Ba. 2015. Adam: A Method for Stochastic Optimization. CoRR abs/1412.6980(2015).Google Scholar
- Raghuraman Krishnamoorthi. 2018. Quantizing deep convolutional networks for efficient inference: A whitepaper. ArXiv abs/1806.08342(2018).Google Scholar
- A. Krizhevsky. 2009. Learning Multiple Layers of Features from Tiny Images.Google Scholar
- Yann LeCun, Léon Bottou, Yoshua Bengio, 1998. Gradient-based learning applied to document recognition.Google Scholar
- Fengfu Li and Bin Liu. 2016. Ternary Weight Networks. NeurIPS (2016).Google Scholar
- Shaofeng Li, Minhui Xue, B. Zhao, H. Zhu, and Xinpeng Zhang. 2019. Invisible Backdoor Attacks on Deep Neural Networks via Steganography and Regularization. TDSC (2019).Google Scholar
- Ji Lin, Chuang Gan, and Song Han. 2019. Defensive Quantization: When Efficiency Meets Robustness. ArXiv abs/1904.08444(2019).Google Scholar
- Junyu Lin, Lei Xu, Yingqi Liu, and X. Zhang. 2020. Composite Backdoor Attack for Deep Neural Network by Mixing Existing Benign Features. Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications Security (2020).Google ScholarDigital Library
- K. Liu, Brendan Dolan-Gavitt, and S. Garg. 2018. Fine-Pruning: Defending Against Backdooring Attacks on Deep Neural Networks. In RAID.Google Scholar
- Y. Liu, Wen-Chuan Lee, Guanhong Tao, Shiqing Ma, Yousra Aafer, and X. Zhang. 2019. ABS: Scanning Neural Networks for Back-doors by Artificial Brain Stimulation. CCS (2019).Google Scholar
- Yingqi Liu, Shiqing Ma, Yousra Aafer, W. Lee, Juan Zhai, Weihang Wang, and X. Zhang. 2018. Trojaning Attack on Neural Networks. NDSS (2018).Google Scholar
- L. V. D. Maaten and Geoffrey E. Hinton. 2008. Visualizing Data using t-SNE. Journal of Machine Learning Research 9 (2008), 2579–2605.Google ScholarDigital Library
- Alberto G. Matachana, Kenneth T. Co, Luis Muñoz-González, David Martínez, and Emil C. Lupu. 2020. Robustness and Transferability of Universal Attacks on Compressed Models. ArXiv abs/2012.06024(2020).Google Scholar
- A. Nguyen and A. Tran. 2020. Input-Aware Dynamic Backdoor Attack. NeurIPS (2020).Google Scholar
- Adam Paszke, S. Gross, Francisco Massa, and et al.2019. PyTorch: An Imperative Style, High-Performance Deep Learning Library. In NeurIPS.Google Scholar
- Ximing Qiao, Yukun Yang, and Hongbing Li. 2019. Defending Neural Backdoors via Generative Distribution Modeling. In NeurIPS.Google Scholar
- A. S. Rakin, Zhezhi He, and Deliang Fan. 2019. Bit-Flip Attack: Crushing Neural Network With Progressive Bit Search. 2019 IEEE/CVF International Conference on Computer Vision (ICCV) (2019), 1211–1220.Google Scholar
- A. S. Rakin, Zhezhi He, and Deliang Fan. 2020. TBT: Targeted Neural Network Attack With Bit Trojan. 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) (2020), 13195–13204.Google Scholar
- Kaveh Razavi, Ben Gras, E. Bosman, B. Preneel, Cristiano Giuffrida, and H. Bos. 2016. Flip Feng Shui: Hammering a Needle in the Software Stack. In USENIX Security Symposium.Google Scholar
- Cyril Roscian, A. Sarafianos, J. Dutertre, and A. Tria. 2013. Fault Model Analysis of Laser-Induced Faults in SRAM Memory Cells. 2013 Workshop on Fault Diagnosis and Tolerance in Cryptography (2013), 89–98.Google Scholar
- A. Salem, Rui Wen, M. Backes, Shiqing Ma, and Y. Zhang. 2020. Dynamic Backdoor Attacks Against Machine Learning Models. ArXiv (2020).Google Scholar
- K. Simonyan and Andrew Zisserman. 2015. Very Deep Convolutional Networks for Large-Scale Image Recognition. CoRR abs/1409.1556(2015).Google Scholar
- J. Stallkamp, Marc Schlipsing, J. Salmen, and C. Igel. 2012. Man vs. computer: Benchmarking machine learning algorithms for traffic sign recognition. Neural networks : the official journal of the International Neural Network Society 32(2012), 323–32.Google Scholar
- Christian Szegedy, Wojciech Zaremba, Ilya Sutskever, Joan Bruna, D. Erhan, I. Goodfellow, and R. Fergus. 2014. Intriguing properties of neural networks. CoRR abs/1312.6199(2014).Google Scholar
- Brandon Tran, Jerry Li, and A. Madry. 2018. Spectral Signatures in Backdoor Attacks. In NeurIPS.Google Scholar
- Alexander Turner, D. Tsipras, and A. Madry. 2019. Label-Consistent Backdoor Attacks. ArXiv (2019).Google Scholar
- Bolun Wang, Yuanshun Yao, Shawn Shan, Huiying Li, B. Viswanath, H. Zheng, and B. Zhao. 2019. Neural Cleanse: Identifying and Mitigating Backdoor Attacks in Neural Networks. Security & Privacy (2019).Google Scholar
- Jiaxiang Wu, C. Leng, Yuhang Wang, Q. Hu, and Jian Cheng. 2016. Quantized Convolutional Neural Networks for Mobile Devices. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016), 4820–4828.Google Scholar
- Yun Xiang, Zhuang-Zhi Chen, Zuohui Chen, Zebin Fang, Haiyang Hao, Jinyin Chen, Yi Liu, Zhefu Wu, Qi Xuan, and Xiaoniu Yang. 2020. Open DNN Box by Power Side-Channel Attack. IEEE Transactions on Circuits and Systems II: Express Briefs 67 (2020), 2717–2721.Google ScholarCross Ref
- Mengjia Yan, Christopher W. Fletcher, and J. Torrellas. 2020. Cache Telepathy: Leveraging Shared Resource Attacks to Learn DNN Architectures. USENIX Security (2020).Google Scholar
- Jiancheng Yang, R. Shi, and Bingbing Ni. 2021. MedMNIST Classification Decathlon: A Lightweight AutoML Benchmark for Medical Image Analysis. 2021 IEEE 18th International Symposium on Biomedical Imaging (ISBI) (2021), 191–195.Google Scholar
- Fan Yao, A. S. Rakin, and Deliang Fan. 2020. DeepHammer: Depleting the Intelligence of Deep Neural Networks through Targeted Chain of Bit Flips. In USENIX Security Symposium.Google Scholar
- Yuanshun Yao, Huiying Li, H. Zheng, and B. Zhao. 2019. Latent Backdoor Attacks on Deep Neural Networks. CCS (2019).Google Scholar
- Honggang Yu, Haocheng Ma, Kaichen Yang, Yiqiang Zhao, and Yier Jin. 2020. DeepEM: Deep Neural Networks Model Recovery through EM Side-Channel Information Leakage. 2020 IEEE International Symposium on Hardware Oriented Security and Trust (HOST) (2020), 209–218.Google ScholarCross Ref
Index Terms
- Understanding the Threats of Trojaned Quantized Neural Network in Model Supply Chains
Index terms have been assigned to the content through auto-classification.
Recommendations
Composite Backdoor Attack for Deep Neural Network by Mixing Existing Benign Features
CCS '20: Proceedings of the 2020 ACM SIGSAC Conference on Computer and Communications SecurityWith the prevalent use of Deep Neural Networks (DNNs) in many applications, security of these networks is of importance. Pre-trained DNNs may contain backdoors that are injected through poisoned training. These trojaned models perform well when regular ...
Artificial wavelet neuro-fuzzy model based on parallel wavelet network and neural network
From the well-known advantages and valuable features of wavelets when used in neural network, two type of networks (i.e., SWNN and MWNN) have been proposed. These networks are single hidden layer network. Each neuron in the hidden layer is comprised of ...
DIHBA: Dynamic, invisible and high attack success rate boundary backdoor attack with low poison ratio
Highlights- DIT determines decision boundaries to generate backdoors images.
- DIHBA based on DIT has a high attack success rate even with 0% injection.
- DIHBA is effective in attacking different precision networks.
- DIHBA can bypass the strip ...
AbstractMachine learning models are known to be vulnerable to malicious attacks, such as adversarial attacks, data poisoning attacks and backdoor attacks. A model injected by a backdoor attack can work adequately under normal conditions but the malicious ...
Comments