Abstract
Adversarial machine learning is a prominent research area aimed towards exposing and mitigating security vulnerabilities in AI/ML algorithms and their implementations. Data poisoning and neural Trojans enable an attacker to drastically change the behavior and performance of a Convolutional Neural Network (CNN) merely by altering some of the input data during training. Such attacks can be catastrophic in the field, e.g. for self-driving vehicles. In this paper, we propose deploying a CNN as an ecosystem of variants, rather than a singular model. The ecosystem is derived from the original trained model, and though every derived model is structurally different, they are all functionally equivalent to the original and each other. We propose two complementary techniques: stochastic parameter mutation, where the weights θ of the original are shifted by a small, random amount, and a delta-update procedure which functions by XOR’ing all of the parameters with an update file containing the Δ θ values. This technique is effective against transferability of a neural Trojan to the greater ecosystem by amplifying the Trojan’s malicious impact to easily detectable levels; thus, deploying a model as an ecosystem can render the ecosystem more resilient against a neural Trojan attack.
- [1] . 2015. Hacking smart machines with smarter ones: How to extract meaningful data from machine learning classifiers. Int. J. Secur. Netw. 10, 3 (
Sept. 2015), 137–150. Google ScholarDigital Library - [2] . 2019. How To Backdoor Federated Learning.
arxiv:cs.CR/1807.00459 Google Scholar - [3] . 2018. Detecting poisoning attacks on machine learning in IoT environments. In 2018 IEEE International Congress on Internet of Things (ICIOT). 57–64. Google ScholarCross Ref
- [4] . 2003. Randomized instruction set emulation to disrupt binary code injection attacks. In Proceedings of the 10th ACM Conference on Computer and Communications Security (CCS’03). Association for Computing Machinery, New York, NY, USA, 281–289. Google ScholarDigital Library
- [5] . 2018. Analyzing Federated Learning through an Adversarial Lens.
arxiv:cs.LG/1811.12470 Google Scholar - [6] . 2013. Evasion attacks against machine learning at test time. In Machine Learning and Knowledge Discovery in Databases, , , , and (Eds.). Springer Berlin, Berlin,, 387–402. Google ScholarDigital Library
- [7] . 2012. Poisoning attacks against support vector machines. arXiv (
Jun 2012). arXiv:1206.6389 https://arxiv.org/abs/1206.6389v3.Google Scholar - [8] . 2016. End to end learning for self-driving cars. arXiv (
Apr 2016). arXiv:1604.07316 https://arxiv.org/abs/1604.07316v1.Google Scholar - [9] . 2017. Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning.
arxiv:cs.CR/1712.05526 Google Scholar - [10] . 2015. Keras. https://keras.io.Google Scholar
- [11] . 2006. N-variant systems: A secretless framework for security through diversity. In Proceedings of the 15th Conference on USENIX Security Symposium - Volume 15 (USENIX-SS’06). USENIX Association, USA, Article
9 , 1 pages.Google Scholar - [12] . 2008. An encryption scheme using DNA technology. In 2008 3rd International Conference on Bio-Inspired Computing: Theories and Applications. 37–42. Google ScholarCross Ref
- [13] . 2018. Why Do Adversarial Attacks Transfer? Explaining Transferability of Evasion and Poisoning Attacks.
arxiv:cs.LG/1809.02861 Google Scholar - [14] . 2009. How to modify a neural network gradually without changing its input-output functionality. Neural Computation 22 (
10 2009), 1–47. Google ScholarDigital Library - [15] . 2014. Defense on the move: Ant-based cyber defense. IEEE Security Privacy 12, 2 (2014), 36–43. Google ScholarCross Ref
- [16] . 2020. STRIP: A Defence Against Trojan Attacks on Deep Neural Networks.
arxiv:cs.CR/1902.06531 Google Scholar - [17] . 2015. Explaining and Harnessing Adversarial Examples.
arxiv:stat.ML/1412.6572 Google Scholar - [18] . 2013. Speech recognition with deep recurrent neural networks. In 2013 IEEE International Conference on Acoustics, Speech and Signal Processing. 6645–6649. Google ScholarCross Ref
- [19] . 2019. BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain.
arxiv:cs.CR/1708.06733 Google Scholar - [20] . 2015. Learning both weights and connections for efficient neural networks. CoRR abs/1506.02626 (2015). arXiv:1506.02626 http://arxiv.org/abs/1506.02626.Google Scholar
- [21] . 2016. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 770–778. Google ScholarCross Ref
- [22] . 2008. Evaluating biometric encryption key generation using handwritten signatures. In 2008 Bio-inspired, Learning and Intelligent Systems for Security. 17–22. Google ScholarDigital Library
- [23] 2020. DeepSniffer. Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Systems (
Mar 2020). Google ScholarDigital Library - [24] . 2011. Adversarial machine learning. In Proceedings of the 4th ACM Workshop on Security and Artificial Intelligence (AISec’11). Association for Computing Machinery, New York, NY, USA, 43–58. Google ScholarDigital Library
- [25] . 2017. Backdoor attacks against learning systems. In 2017 IEEE Conference on Communications and Network Security (CNS). 1–9.Google ScholarCross Ref
- [26] . 2016. Robust bitstream protection in FPGA-based systems through low-overhead obfuscation. 2016 International Conference on ReConFigurable Computing and FPGAs (ReConFig) (
Nov 2016), 1–8. Google ScholarCross Ref - [27] . 2014. Large-scale video classification with convolutional neural networks. In 2014 IEEE Conference on Computer Vision and Pattern Recognition. 1725–1732. Google ScholarDigital Library
- [28] . 2007. Characterizing software self-healing systems. In Computer Network Security, , , and (Eds.). Springer Berlin, Berlin, 22–33. Google ScholarCross Ref
- [29] . 2012. ImageNet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems 25, , , , and (Eds.). Curran Associates, Inc., 1097–1105. http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf.Google ScholarDigital Library
- [30] . 1998. Gradient-based learning applied to document recognition. Proc. IEEE 86, 11 (
Nov 1998), 2278–2324. https://ieeexplore.ieee.org/document/726791.Google ScholarCross Ref - [31] . 2018. Hu-Fu: Hardware and software collaborative attack framework against neural networks. 482–487. Google ScholarCross Ref
- [32] . 2018. Backdoor Embedding in Convolutional Neural Network Models via Invisible Perturbation.
arxiv:cs.CR/1808.10307 Google Scholar - [33] . 2018. SIN2: Stealth infection on neural network — A low-cost agile neural Trojan attack methodology. In 2018 IEEE International Symposium on Hardware Oriented Security and Trust (HOST). 227–230. Google ScholarCross Ref
- [34] . 2016. Delving into Transferable Adversarial Examples and Black-box Attacks.
arxiv:cs.LG/1611.02770 Google Scholar - [35] . 2018. Trojaning attack on neural networks. In 25th Annual Network and Distributed System Security Symposium, NDSS 2018, San Diego, California, USA, February 18-21, 2018. The Internet Society.Google Scholar
- [36] . 2017. Neural Trojans. arXiv (
Oct 2017).arXiv:1710.00942 https://arxiv.org/abs/1710.00942v1.Google Scholar - [37] . 2018. Architectural diversity: Bio-inspired hardware security for FPGAs. In 2018 IEEE 3rd International Verification and Security Workshop (IVSW). 48–51.Google ScholarCross Ref
- [38] . 2017. Universal adversarial perturbations. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 86–94.Google ScholarCross Ref
- [39] . 2017. Towards Poisoning of Deep Learning Algorithms with Back-gradient Optimization.
arxiv:cs.LG/1708.08689 Google Scholar - [40] . 2020. Tunable FPGA bitstream obfuscation with Boolean satisfiability attack countermeasure. ACM Trans. Des. Autom. Electron. Syst. 25, 2 (
Feb 2020), 1–22. Google ScholarDigital Library - [41] . 1995. Generating accurate and diverse members of a neural-network ensemble. In Proceedings of the 8th International Conference on Neural Information Processing Systems (NIPS’95). MIT Press, Cambridge, MA, USA, 535–541.Google Scholar
- [42] . 2016. Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples.
arxiv:cs.CR/1605.07277 Google Scholar - [43] . 2020. TBT: Targeted neural network attack with bit Trojan. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 13195–13204. Google ScholarCross Ref
- [44] . 2013. Bio-inspired machine learning based wireless sensor network security. In 2013 World Congress on Nature and Biologically Inspired Computing. 140–146. Google ScholarCross Ref
- [45] . 2019. Efficiently stealing your machine learning models. In Proceedings of the 18th ACM Workshop on Privacy in the Electronic Society (WPES’19). Association for Computing Machinery, New York, NY, USA, 198–210. Google ScholarDigital Library
- [46] . 2017. Deep learning is robust to massive label noise. arXiv (
May 2017). arXiv:1705.10694 https://arxiv.org/abs/1705.10694v3.Google Scholar - [47] . 2015. Very Deep Convolutional Networks for Large-Scale Image Recognition.
arxiv:cs.CV/1409.1556 Google Scholar - [48] . 2017. Machine Learning Models that Remember Too Much.
arxiv:cs.CR/1709.07886 Google Scholar - [49] . 2014. Intriguing properties of neural networks. In International Conference on Learning Representations. http://arxiv.org/abs/1312.6199.Google Scholar
- [50] . 2010. A survey of hardware Trojan taxonomy and detection. IEEE Des. Test Comput. 27, 1 (
Feb 2010), 10–25. Google ScholarDigital Library - [51] . 2017. The Space of Transferable Adversarial Examples.
arxiv:stat.ML/1704.03453 Google Scholar - [52] . 2016. Stealing Machine Learning Models via Prediction APIs.
arxiv:cs.CR/1609.02943 Google Scholar - [53] . 2019. Neural cleanse: Identifying and mitigating backdoor attacks in neural networks. 707–723. Google ScholarCross Ref
- [54] . 2020. Practical Detection of Trojan Neural Networks: Data-Limited and Data-Free Cases.
arxiv:cs.LG/2007.15802 Google Scholar - [55] . 2012. Adversarial label flips attack on support vector machines. 242 (
01 2012), 870–875. Google ScholarCross Ref - [56] . 2017. How to steal a machine learning classifier with deep learning. In 2017 IEEE International Symposium on Technologies for Homeland Security (HST). 1–5.Google Scholar
- [57] . 2020. DeepEM: Deep neural networks model recovery through EM side-channel information leakage.Google Scholar
- [58] . 2020. CloudLeak: Large-scale deep learning models stealing through adversarial examples. Google ScholarCross Ref
- [59] . 2018. Detecting RTL Trojans using artificial immune systems and high level behavior classification. 2018 Asian Hardware Oriented Security and Trust Symposium (AsianHOST) (
Dec 2018), 68–73. Google ScholarCross Ref - [60] . 2013. Visualizing and understanding convolutional networks. CoRR abs/1311.2901 (2013). arXiv:1311.2901 http://arxiv.org/abs/1311.2901.Google Scholar
- [61] . 2020. Bridging Mode Connectivity in Loss Landscapes and Adversarial Robustness.
arxiv:cs.LG/2005.00060 Google Scholar
Index Terms
Diverse, Neural Trojan Resilient Ecosystem of Neural Network IP
Recommendations
Defending Against Adversarial Denial-of-Service Data Poisoning Attacks
DYNAMICS '20: Proceedings of the 2020 Workshop on DYnamic and Novel Advances in Machine Learning and Intelligent Cyber SecurityData poisoning is one of the most relevant security threats against machine learning and data-driven technologies. Since many applications rely on untrusted training data, an attacker can easily craft malicious samples and inject them into the training ...
Backdoor Embedding in Convolutional Neural Network Models via Invisible Perturbation
CODASPY '20: Proceedings of the Tenth ACM Conference on Data and Application Security and PrivacyDeep learning models have consistently outperformed traditional machine learning models in various classification tasks, including image classification. As such, they have become increasingly prevalent in many real world applications including those ...
Data Poisoning Attacks Against Federated Learning Systems
Computer Security – ESORICS 2020AbstractFederated learning (FL) is an emerging paradigm for distributed training of large-scale deep neural networks in which participants’ data remains on their own devices with only model updates being shared with a central server. However, the ...
Comments