skip to main content
research-article

Diverse, Neural Trojan Resilient Ecosystem of Neural Network IP

Published:04 August 2022Publication History
Skip Abstract Section

Abstract

Adversarial machine learning is a prominent research area aimed towards exposing and mitigating security vulnerabilities in AI/ML algorithms and their implementations. Data poisoning and neural Trojans enable an attacker to drastically change the behavior and performance of a Convolutional Neural Network (CNN) merely by altering some of the input data during training. Such attacks can be catastrophic in the field, e.g. for self-driving vehicles. In this paper, we propose deploying a CNN as an ecosystem of variants, rather than a singular model. The ecosystem is derived from the original trained model, and though every derived model is structurally different, they are all functionally equivalent to the original and each other. We propose two complementary techniques: stochastic parameter mutation, where the weights θ of the original are shifted by a small, random amount, and a delta-update procedure which functions by XOR’ing all of the parameters with an update file containing the Δ θ values. This technique is effective against transferability of a neural Trojan to the greater ecosystem by amplifying the Trojan’s malicious impact to easily detectable levels; thus, deploying a model as an ecosystem can render the ecosystem more resilient against a neural Trojan attack.

REFERENCES

  1. [1] Ateniese Giuseppe, Mancini Luigi V., Spognardi Angelo, Villani Antonio, Vitali Domenico, and Felici Giovanni. 2015. Hacking smart machines with smarter ones: How to extract meaningful data from machine learning classifiers. Int. J. Secur. Netw. 10, 3 (Sept. 2015), 137150. Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. [2] Bagdasaryan Eugene, Veit Andreas, Hua Yiqing, Estrin Deborah, and Shmatikov Vitaly. 2019. How To Backdoor Federated Learning. arxiv:cs.CR/1807.00459Google ScholarGoogle Scholar
  3. [3] Baracaldo N., Chen B., Ludwig H., Safavi A., and Zhang R.. 2018. Detecting poisoning attacks on machine learning in IoT environments. In 2018 IEEE International Congress on Internet of Things (ICIOT). 5764. Google ScholarGoogle ScholarCross RefCross Ref
  4. [4] Barrantes Elena Gabriela, Ackley David H., Forrest Stephanie, Palmer Trek S., Stefanovic Darko, and Zovi Dino Dai. 2003. Randomized instruction set emulation to disrupt binary code injection attacks. In Proceedings of the 10th ACM Conference on Computer and Communications Security (CCS’03). Association for Computing Machinery, New York, NY, USA, 281289. Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. [5] Bhagoji Arjun Nitin, Chakraborty Supriyo, Mittal Prateek, and Calo Seraphin. 2018. Analyzing Federated Learning through an Adversarial Lens. arxiv:cs.LG/1811.12470Google ScholarGoogle Scholar
  6. [6] Biggio Battista, Corona Igino, Maiorca Davide, Nelson Blaine, Šrndić Nedim, Laskov Pavel, Giacinto Giorgio, and Roli Fabio. 2013. Evasion attacks against machine learning at test time. In Machine Learning and Knowledge Discovery in Databases, Blockeel Hendrik, Kersting Kristian, Nijssen Siegfried, and Železný Filip (Eds.). Springer Berlin, Berlin,, 387402. Google ScholarGoogle ScholarDigital LibraryDigital Library
  7. [7] Biggio Battista, Nelson Blaine, and Laskov Pavel. 2012. Poisoning attacks against support vector machines. arXiv (Jun 2012). arXiv:1206.6389 https://arxiv.org/abs/1206.6389v3.Google ScholarGoogle Scholar
  8. [8] Bojarski Mariusz, Testa Davide Del, Dworakowski Daniel, Firner Bernhard, Flepp Beat, Goyal Prasoon, Jackel Lawrence D., Monfort Mathew, Muller Urs, Zhang Jiakai, Zhang Xin, Zhao Jake, and Zieba Karol. 2016. End to end learning for self-driving cars. arXiv (Apr 2016). arXiv:1604.07316 https://arxiv.org/abs/1604.07316v1.Google ScholarGoogle Scholar
  9. [9] Chen Xinyun, Liu Chang, Li Bo, Lu Kimberly, and Song Dawn. 2017. Targeted Backdoor Attacks on Deep Learning Systems Using Data Poisoning. arxiv:cs.CR/1712.05526Google ScholarGoogle Scholar
  10. [10] Chollet François et al. 2015. Keras. https://keras.io.Google ScholarGoogle Scholar
  11. [11] Cox Benjamin, Evans David, Filipi Adrian, Rowanhill Jonathan, Hu Wei, Davidson Jack, Knight John, Nguyen-Tuong Anh, and Hiser Jason. 2006. N-variant systems: A secretless framework for security through diversity. In Proceedings of the 15th Conference on USENIX Security Symposium - Volume 15 (USENIX-SS’06). USENIX Association, USA, Article 9, 1 pages.Google ScholarGoogle Scholar
  12. [12] Cui G., Qin L., Wang Y., and Zhang X.. 2008. An encryption scheme using DNA technology. In 2008 3rd International Conference on Bio-Inspired Computing: Theories and Applications. 3742. Google ScholarGoogle ScholarCross RefCross Ref
  13. [13] Demontis Ambra, Melis Marco, Pintor Maura, Jagielski Matthew, Biggio Battista, Oprea Alina, Nita-Rotaru Cristina, and Roli Fabio. 2018. Why Do Adversarial Attacks Transfer? Explaining Transferability of Evasion and Poisoning Attacks. arxiv:cs.LG/1809.02861Google ScholarGoogle Scholar
  14. [14] DiMattina Christopher and Zhang Kechen. 2009. How to modify a neural network gradually without changing its input-output functionality. Neural Computation 22 (10 2009), 147. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. [15] Fink G. A., Haack J. N., McKinnon A. D., and Fulp E. W.. 2014. Defense on the move: Ant-based cyber defense. IEEE Security Privacy 12, 2 (2014), 3643. Google ScholarGoogle ScholarCross RefCross Ref
  16. [16] Gao Yansong, Xu Chang, Wang Derui, Chen Shiping, Ranasinghe Damith C., and Nepal Surya. 2020. STRIP: A Defence Against Trojan Attacks on Deep Neural Networks. arxiv:cs.CR/1902.06531Google ScholarGoogle Scholar
  17. [17] Goodfellow Ian J., Shlens Jonathon, and Szegedy Christian. 2015. Explaining and Harnessing Adversarial Examples. arxiv:stat.ML/1412.6572Google ScholarGoogle Scholar
  18. [18] Graves A., Mohamed A., and Hinton G.. 2013. Speech recognition with deep recurrent neural networks. In 2013 IEEE International Conference on Acoustics, Speech and Signal Processing. 66456649. Google ScholarGoogle ScholarCross RefCross Ref
  19. [19] Gu Tianyu, Dolan-Gavitt Brendan, and Garg Siddharth. 2019. BadNets: Identifying Vulnerabilities in the Machine Learning Model Supply Chain. arxiv:cs.CR/1708.06733Google ScholarGoogle Scholar
  20. [20] Han Song, Pool Jeff, Tran John, and Dally William J.. 2015. Learning both weights and connections for efficient neural networks. CoRR abs/1506.02626 (2015). arXiv:1506.02626 http://arxiv.org/abs/1506.02626.Google ScholarGoogle Scholar
  21. [21] He K., Zhang X., Ren S., and Sun J.. 2016. Deep residual learning for image recognition. In 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 770778. Google ScholarGoogle ScholarCross RefCross Ref
  22. [22] Hoque S., Fairhurst M., and Howells G.. 2008. Evaluating biometric encryption key generation using handwritten signatures. In 2008 Bio-inspired, Learning and Intelligent Systems for Security. 1722. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. [23] Hu Xing, Liang Ling, Li Shuangchen, Deng Lei, Zuo Pengfei, Ji Yu, Xie Xinfeng, Ding Yufei, Liu Chang, Sherwood Timothy, al. et2020. DeepSniffer. Proceedings of the Twenty-Fifth International Conference on Architectural Support for Programming Languages and Operating Systems (Mar 2020). Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. [24] Huang Ling, Joseph Anthony D., Nelson Blaine, Rubinstein Benjamin I. P., and Tygar J. D.. 2011. Adversarial machine learning. In Proceedings of the 4th ACM Workshop on Security and Artificial Intelligence (AISec’11). Association for Computing Machinery, New York, NY, USA, 4358. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. [25] Ji Y., Zhang X., and Wang T.. 2017. Backdoor attacks against learning systems. In 2017 IEEE Conference on Communications and Network Security (CNS). 19.Google ScholarGoogle ScholarCross RefCross Ref
  26. [26] Karam Robert, Hoque Tamzidul, Ray Sandip, Tehranipoor Mark, and Bhunia Swarup. 2016. Robust bitstream protection in FPGA-based systems through low-overhead obfuscation. 2016 International Conference on ReConFigurable Computing and FPGAs (ReConFig) (Nov 2016), 18. Google ScholarGoogle ScholarCross RefCross Ref
  27. [27] Karpathy A., Toderici G., Shetty S., Leung T., Sukthankar R., and Fei-Fei L.. 2014. Large-scale video classification with convolutional neural networks. In 2014 IEEE Conference on Computer Vision and Pattern Recognition. 17251732. Google ScholarGoogle ScholarDigital LibraryDigital Library
  28. [28] Keromytis Angelos D.. 2007. Characterizing software self-healing systems. In Computer Network Security, Gorodetsky Vladimir, Kotenko Igor, and Skormin Victor A. (Eds.). Springer Berlin, Berlin, 2233. Google ScholarGoogle ScholarCross RefCross Ref
  29. [29] Krizhevsky Alex, Sutskever Ilya, and Hinton Geoffrey E.. 2012. ImageNet classification with deep convolutional neural networks. In Advances in Neural Information Processing Systems 25, Pereira F., Burges C. J. C., Bottou L., and Weinberger K. Q. (Eds.). Curran Associates, Inc., 10971105. http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf.Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. [30] Lecun Y., Bottou L., Bengio Y., and Haffner P.. 1998. Gradient-based learning applied to document recognition. Proc. IEEE 86, 11 (Nov 1998), 22782324. https://ieeexplore.ieee.org/document/726791.Google ScholarGoogle ScholarCross RefCross Ref
  31. [31] Li Wenshuo, Yu Jincheng, Ning Xuefei, Wang Pengjun, Wei Qi, Wang Yu, and Yang Huazhong. 2018. Hu-Fu: Hardware and software collaborative attack framework against neural networks. 482487. Google ScholarGoogle ScholarCross RefCross Ref
  32. [32] Liao Cong, Zhong Haoti, Squicciarini Anna, Zhu Sencun, and Miller David. 2018. Backdoor Embedding in Convolutional Neural Network Models via Invisible Perturbation. arxiv:cs.CR/1808.10307Google ScholarGoogle Scholar
  33. [33] Liu T., Wen W., and Jin Y.. 2018. SIN2: Stealth infection on neural network — A low-cost agile neural Trojan attack methodology. In 2018 IEEE International Symposium on Hardware Oriented Security and Trust (HOST). 227230. Google ScholarGoogle ScholarCross RefCross Ref
  34. [34] Liu Yanpei, Chen Xinyun, Liu Chang, and Song Dawn. 2016. Delving into Transferable Adversarial Examples and Black-box Attacks. arxiv:cs.LG/1611.02770Google ScholarGoogle Scholar
  35. [35] Liu Yingqi, Ma Shiqing, Aafer Yousra, Lee Wen-Chuan, Zhai Juan, Wang Weihang, and Zhang Xiangyu. 2018. Trojaning attack on neural networks. In 25th Annual Network and Distributed System Security Symposium, NDSS 2018, San Diego, California, USA, February 18-21, 2018. The Internet Society.Google ScholarGoogle Scholar
  36. [36] Liu Yuntao, Xie Yang, and Srivastava Ankur. 2017. Neural Trojans. arXiv (Oct 2017). arXiv:1710.00942 https://arxiv.org/abs/1710.00942v1.Google ScholarGoogle Scholar
  37. [37] Mahmud S., Olney B., and Karam R.. 2018. Architectural diversity: Bio-inspired hardware security for FPGAs. In 2018 IEEE 3rd International Verification and Security Workshop (IVSW). 4851.Google ScholarGoogle ScholarCross RefCross Ref
  38. [38] Moosavi-Dezfooli S., Fawzi A., Fawzi O., and Frossard P.. 2017. Universal adversarial perturbations. In 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). 8694.Google ScholarGoogle ScholarCross RefCross Ref
  39. [39] Muñoz-González Luis, Biggio Battista, Demontis Ambra, Paudice Andrea, Wongrassamee Vasin, Lupu Emil C., and Roli Fabio. 2017. Towards Poisoning of Deep Learning Algorithms with Back-gradient Optimization. arxiv:cs.LG/1708.08689Google ScholarGoogle Scholar
  40. [40] Olney Brooks and Karam Robert. 2020. Tunable FPGA bitstream obfuscation with Boolean satisfiability attack countermeasure. ACM Trans. Des. Autom. Electron. Syst. 25, 2 (Feb 2020), 122. Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. [41] Opitz David W. and Shavlik Jude W.. 1995. Generating accurate and diverse members of a neural-network ensemble. In Proceedings of the 8th International Conference on Neural Information Processing Systems (NIPS’95). MIT Press, Cambridge, MA, USA, 535541.Google ScholarGoogle Scholar
  42. [42] Papernot Nicolas, McDaniel Patrick, and Goodfellow Ian. 2016. Transferability in Machine Learning: from Phenomena to Black-Box Attacks using Adversarial Samples. arxiv:cs.CR/1605.07277Google ScholarGoogle Scholar
  43. [43] Rakin A. S., He Z., and Fan D.. 2020. TBT: Targeted neural network attack with bit Trojan. In 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). 1319513204. Google ScholarGoogle ScholarCross RefCross Ref
  44. [44] Rathore H. and Jha S.. 2013. Bio-inspired machine learning based wireless sensor network security. In 2013 World Congress on Nature and Biologically Inspired Computing. 140146. Google ScholarGoogle ScholarCross RefCross Ref
  45. [45] Reith Robert Nikolai, Schneider Thomas, and Tkachenko Oleksandr. 2019. Efficiently stealing your machine learning models. In Proceedings of the 18th ACM Workshop on Privacy in the Electronic Society (WPES’19). Association for Computing Machinery, New York, NY, USA, 198210. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. [46] Rolnick David, Veit Andreas, Belongie Serge, and Shavit Nir. 2017. Deep learning is robust to massive label noise. arXiv (May 2017). arXiv:1705.10694 https://arxiv.org/abs/1705.10694v3.Google ScholarGoogle Scholar
  47. [47] Simonyan Karen and Zisserman Andrew. 2015. Very Deep Convolutional Networks for Large-Scale Image Recognition. arxiv:cs.CV/1409.1556Google ScholarGoogle Scholar
  48. [48] Song Congzheng, Ristenpart Thomas, and Shmatikov Vitaly. 2017. Machine Learning Models that Remember Too Much. arxiv:cs.CR/1709.07886Google ScholarGoogle Scholar
  49. [49] Szegedy Christian, Zaremba Wojciech, Sutskever Ilya, Bruna Joan, Erhan Dumitru, Goodfellow Ian, and Fergus Rob. 2014. Intriguing properties of neural networks. In International Conference on Learning Representations. http://arxiv.org/abs/1312.6199.Google ScholarGoogle Scholar
  50. [50] Tehranipoor Mohammad and Koushanfar Farinaz. 2010. A survey of hardware Trojan taxonomy and detection. IEEE Des. Test Comput. 27, 1 (Feb 2010), 1025. Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. [51] Tramèr Florian, Papernot Nicolas, Goodfellow Ian, Boneh Dan, and McDaniel Patrick. 2017. The Space of Transferable Adversarial Examples. arxiv:stat.ML/1704.03453Google ScholarGoogle Scholar
  52. [52] Tramèr Florian, Zhang Fan, Juels Ari, Reiter Michael K., and Ristenpart Thomas. 2016. Stealing Machine Learning Models via Prediction APIs. arxiv:cs.CR/1609.02943Google ScholarGoogle Scholar
  53. [53] Wang Bolun, Yao Yuanshun, Shan Shawn, Li Huiying, Viswanath Bimal, Zheng Haitao, and Zhao Ben. 2019. Neural cleanse: Identifying and mitigating backdoor attacks in neural networks. 707723. Google ScholarGoogle ScholarCross RefCross Ref
  54. [54] Wang Ren, Zhang Gaoyuan, Liu Sijia, Chen Pin-Yu, Xiong Jinjun, and Wang Meng. 2020. Practical Detection of Trojan Neural Networks: Data-Limited and Data-Free Cases. arxiv:cs.LG/2007.15802Google ScholarGoogle Scholar
  55. [55] Xiao Han and Eckert Claudia. 2012. Adversarial label flips attack on support vector machines. 242 (01 2012), 870875. Google ScholarGoogle ScholarCross RefCross Ref
  56. [56] Shi Yi, Sagduyu Y., and Grushin A.. 2017. How to steal a machine learning classifier with deep learning. In 2017 IEEE International Symposium on Technologies for Homeland Security (HST). 15.Google ScholarGoogle Scholar
  57. [57] Yu Honggang, Ma Haocheng, Yang Kaichen, Zhao Y., and Jin Yier. 2020. DeepEM: Deep neural networks model recovery through EM side-channel information leakage.Google ScholarGoogle Scholar
  58. [58] Yu Honggang, Yang Kaichen, Zhang Teng, Tsai Yun-Yun, Ho Tsung-Yi, and Jin Yier. 2020. CloudLeak: Large-scale deep learning models stealing through adversarial examples. Google ScholarGoogle ScholarCross RefCross Ref
  59. [59] Zareen Farhath and Karam Robert. 2018. Detecting RTL Trojans using artificial immune systems and high level behavior classification. 2018 Asian Hardware Oriented Security and Trust Symposium (AsianHOST) (Dec 2018), 6873. Google ScholarGoogle ScholarCross RefCross Ref
  60. [60] Zeiler Matthew D. and Fergus Rob. 2013. Visualizing and understanding convolutional networks. CoRR abs/1311.2901 (2013). arXiv:1311.2901 http://arxiv.org/abs/1311.2901.Google ScholarGoogle Scholar
  61. [61] Zhao Pu, Chen Pin-Yu, Das Payel, Ramamurthy Karthikeyan Natesan, and Lin Xue. 2020. Bridging Mode Connectivity in Loss Landscapes and Adversarial Robustness. arxiv:cs.LG/2005.00060Google ScholarGoogle Scholar

Index Terms

  1. Diverse, Neural Trojan Resilient Ecosystem of Neural Network IP

        Recommendations

        Comments

        Login options

        Check if you have access through your login credentials or your institution to get full access on this article.

        Sign in

        Full Access

        • Published in

          cover image ACM Journal on Emerging Technologies in Computing Systems
          ACM Journal on Emerging Technologies in Computing Systems  Volume 18, Issue 3
          July 2022
          428 pages
          ISSN:1550-4832
          EISSN:1550-4840
          DOI:10.1145/3508463
          • Editor:
          • Ramesh Karri
          Issue’s Table of Contents

          Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

          Publisher

          Association for Computing Machinery

          New York, NY, United States

          Publication History

          • Published: 4 August 2022
          • Online AM: 2 February 2022
          • Accepted: 1 June 2021
          • Revised: 1 April 2021
          • Received: 1 November 2020
          Published in jetc Volume 18, Issue 3

          Permissions

          Request permissions about this article.

          Request Permissions

          Check for updates

          Qualifiers

          • research-article
          • Refereed
        • Article Metrics

          • Downloads (Last 12 months)68
          • Downloads (Last 6 weeks)10

          Other Metrics

        PDF Format

        View or Download as a PDF file.

        PDF

        eReader

        View online with eReader.

        eReader

        Full Text

        View this article in Full Text.

        View Full Text

        HTML Format

        View this article in HTML Format .

        View HTML Format