Sayed Erfan Arefin
ABSTRACT
Deep Neural Networks (DNNs) are at the heart of many of today's most innovative technologies. With companies investing lots of resources to design, build and optimize these networks for their custom products, DNNs are now integral to many companies' tightly guarded Intellectual Property. As is the case for every high-value product, one can expect bad actors to increasingly design techniques aimed to uncover the architectural designs of proprietary DNNs. This paper investigates if the power draw patterns of a GPU on which a DNN runs could be leveraged to glean key details of its design architecture. Based on ten of the most well-known Convolutional Neural Network (CNN) architectures, we study this line of attack under varying assumptions about the kind of data available to the attacker. We show the attack to be highly effective, attaining an accuracy in the 80 percentage range for the best performing attack scenario.
Supplemental Material
- [n.d.]. Large Scale Visual Recognition Challenge 2017 (ILSVRC2017).Google Scholar
- [n.d.]. Large Scale Visual Recognition Challenge (ILSVRC). http://www.image-net.org/challenges/LSVRC/. Accessed: 2021-02--24.Google Scholar
- [n.d.]. Pretrained models for Pytorch. https://github.com/Cadene/pretrained-models.pytorch. Accessed: 2021-02--24.Google Scholar
- [n.d.]. Results of ILSVRC2014.Google Scholar
- [n.d.]. Tsfresh. https://tsfresh.readthedocs.io/en/latest/. Accessed: 2021-02--25.Google Scholar
- 2021. Tech PowerUp GPU-Z. https://www.techpowerup.com/gpuz/Google Scholar
- Lejla Batina, Shivam Bhasin, Dirmanto Jap, and Stjepan Picek. 2019. CSI NN:Reverse Engineering of Neural Network Architectures Through Electromagnetic Side Channel. In 28th USENIX Security Symposium (USENIX Security 19). USENIX Association, Santa Clara, CA, 515--532. https://www.usenix.org/conference/usenixsecurity19/presentation/batinaGoogle Scholar
- Dankmar Böhning. 1992. Multinomial logistic regression algorithm. Annals of the Institute of Statistical Mathematics 44, 1 (01 Mar 1992), 197--200. https://doi.org/10.1007/BF00048682Google ScholarCross Ref
- Leo Breiman. 2001. Random Forests. Machine Learning 45, 1 (01 Oct 2001), 5--32. https://doi.org/10.1023/A:1010933404324Google ScholarDigital Library
- J. Deng, W. Dong, R. Socher, L.-J. Li, K. Li, and L. Fei-Fei. 2009. ImageNet: ALarge-Scale Hierarchical Image Database. In CVPR 09.Google Scholar
- Vasisht Duddu, D. Samanta, D. V. Rao, and V. Balas. 2018. Stealing Neural Networks via Timing Side Channels. ArXivabs/1812.11720 (2018).Google Scholar
- Matt Fredrikson, Somesh Jha, and Thomas Ristenpart. 2015. Model Inversion Attacks That Exploit Confidence Information and Basic Countermeasures. In Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security(Denver, Colorado, USA) (CCS '15). Association for Computing Machinery, New York, NY, USA, 1322--1333. https://doi.org/10.1145/2810103.2813677Google ScholarDigital Library
- Kaiming He, X. Zhang, Shaoqing Ren, and Jian Sun. 2016. Deep Residual Learning for Image Recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)(2016), 770--778.Google Scholar
- X. Hu, Ling Liang, L. Deng, Shuangchen Li, Xinfeng Xie, Y. Ji, Yufei Ding, Chang Liu, T. Sherwood, and Yuan Xie. 2020. Neural Network Model Extraction Attacks in Edge Devices by Hearing Architectural Hints. ArXivabs/1903.03916 (2020).Google Scholar
- A. Karpathy, G. Toderici, S. Shetty, T. Leung, R. Sukthankar, and L. Fei-Fei. 2014. Large-Scale Video Classification with Convolutional Neural Networks. In 2014 IEEE Conference on Computer Vision and Pattern Recognition. 1725--1732. https://doi.org/10.1109/CVPR.2014.223Google ScholarDigital Library
- Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. ImageNet Classification with Deep Convolutional Neural Networks. In Advances in Neural Information Processing Systems, F. Pereira, C. J. C. Burges, L. Bottou, and K. Q. Weinberger(Eds.), Vol. 25. Curran Associates, Inc., 1097--1105. https://proceedings.neurips.cc/paper/2012/file/c399862d3b9d6b76c8436e924a68c45b-Paper.pdfGoogle Scholar
- Nicolas Papernot, Patrick McDaniel, Ian Goodfellow, Somesh Jha, Z. BerkayCelik, and Ananthram Swami. 2017. Practical Black-Box Attacks against Machine Learning. In Proceedings of the 2017 ACM on Asia Conference on Computer and Communications Security(Abu Dhabi, United Arab Emirates) (ASIA CCS '17). Association for Computing Machinery, New York, NY, USA, 506--519. https://doi.org/10.1145/3052973.3053009Google ScholarDigital Library
- K. Simonyan and Andrew Zisserman. 2015. Very Deep Convolutional Networks for Large-Scale Image Recognition. CoRRabs/1409.1556 (2015).Google Scholar
- G. K. Venayagamoorthy, V. Moonasar, and K. Sandrasegaran. 1998. Voice recognition using neural networks. In Proceedings of the 1998 South African Symposium on Communications and Signal Processing-COMSIG '98 (Cat. No. 98EX214). 29--32. https://doi.org/10.1109/COMSIG.1998.736916Google ScholarCross Ref
- Yun Xiang, Zhuangzhi Chen, Zuohui Chen, Zebin Fang, Haiyang Hao, Jinyin Chen, Yi Liu, Zhefu Wu, Qi Xuan, and Xiaoniu Yang. 2019. Open DNN Box by Power Side-Channel Attack. arXiv:1907.10406 [cs.CR]Google Scholar
- Y. Xiang, Z. Chen, Z. Chen, Z. Fang, H. Hao, J. Chen, Y. Liu, Z. Wu, Q. Xuan, and X. Yang. 2020. Open DNN Box by Power Side-Channel Attack. IEEE Transactions on Circuits and Systems II: Express Briefs 67, 11 (2020), 2717--2721. https://doi.org/10.1109/TCSII.2020.2973007Google ScholarCross Ref
- X. Zhang, J. Wang, C. Zhu, Y. Lin, J. Xiong, W. Hwu, and D. Chen. 2018. DNN Builder: an Automated Tool for Building High-Performance DNN Hardware Accelerators for FPGAs. In 2018 IEEE/ACM International Conference on Computer-Aided Design (ICCAD). 1--8. https://doi.org/10.1145/3240765.3240801Google ScholarDigital Library
Index Terms
- Deep Neural Exposure: You Can Run, But Not Hide Your Neural Network Architecture!
Recommendations
Deep Elman recurrent neural networks for statistical parametric speech synthesis
Owing to the success of deep learning techniques in automatic speech recognition, deep neural networks (DNNs) have been used as acoustic models for statistical parametric speech synthesis (SPSS). DNNs do not inherently model the temporal structure in ...
Symmetric Power Activation Functions for Deep Neural Networks
LOPAL '18: Proceedings of the International Conference on Learning and Optimization Algorithms: Theory and ApplicationsCommon nonlinear activation functions with large saturation regions, like Sigmoid and Tanh, used for Deep Neural Networks (DNNs) can not guarantee useful and efficient training since they suffer from vanishing gradients problem. Rectified Linear Units ...
On decomposing a deep neural network into modules
ESEC/FSE 2020: Proceedings of the 28th ACM Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software EngineeringDeep learning is being incorporated in many modern software systems. Deep learning approaches train a deep neural network (DNN) model using training examples, and then use the DNN model for prediction. While the structure of a DNN model as layers is ...
Comments