Skip to main content

Advertisement

SpringerLink
Log in

To cloud or not to cloud: an on-line scheduler for dynamic privacy-protection of deep learning workload on edge devices

  • Regular Paper
  • Published:
CCF Transactions on High Performance Computing Aims and scope Submit manuscript

Abstract

Recently deep learning applications are thriving on edge and mobile computing scenarios, due to the concerns of latency constraints, data security and privacy, and other considerations. However, because of the limitation of power delivery, battery lifetime and computation resource, offering real-time neural network inference ability has to resort to the specialized energy-efficient architecture, and sometimes the coordination between the edge devices and the powerful cloud or fog facilities. This work investigates a realistic scenario when an on-line scheduler is needed to meet the requirement of latency even when the edge computing resources and communication speed are dynamically fluctuating, while protecting the privacy of users as well. It also leverages the approximate computing feature of neural networks and actively trade-off excessive neural network propagation paths for latency guarantee even when local resource provision is unstable. Combining neural network approximation and dynamic scheduling, the real-time deep learning system could adapt to different requirements of latency/accuracy and the resource fluctuation of mobile-cloud applications. Experimental results also demonstrate that the proposed scheduler significantly improves the energy efficiency of real-time neural networks on edge devices.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8

Similar content being viewed by others

References

  • Abadi, M., Chu, A., Goodfellow, I., McMahan, H.B., Mironov, I., Talwar, K., Zhang, L.: Deep learning with differential privacy. In: Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security, pp. 308–318 (2016)

  • Barbera, M.V., Kosta, S., Mei, A., Stefa, J.: To offload or not to offload? the bandwidth and energy costs of mobile cloud computing. In: 2013 Proceedings Ieee Infocom, pp. 1285–1293. IEEE (2013)

  • Dosovitskiy, A., Brox, T.: Inverting visual representations with convolutional networks. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 4829–4837 (2016)

  • Du, W., Han, Y.S., Chen, S.: Privacy-preserving multivariate statistical analysis: Linear regression and classification. In: Proceedings of the 2004 SIAM international conference on data mining, pp. 222–233. SIAM (2004)

  • Eshratifar, A.E., Abrishami, M.S., Pedram, M.: JointDNN: an efficient training and inference engine for intelligent mobile cloud computing services. IEEE Transactions on Mobile Computing (2019)

  • Felzenszwalb, P.F., Girshick, R.B., McAllester, D.: Cascade object detection with deformable part models. In: Computer vision and pattern recognition (CVPR), 2010 IEEE conference on, pp. 2241–2248. IEEE (2010)

  • Gilad-Bachrach, R., Dowlin, N., Laine, K., Lauter, K., Naehrig, M., Wernsing, J.: Cryptonets: Applying neural networks to encrypted data with high throughput and accuracy. In: International Conference on Machine Learning, pp. 201–210 (2016)

  • Goldberg, Y.: Neural network methods for natural language processing. Synth. Lect. Hum. Lang. Technol. 10(1), 1–309 (2017)

    Article  Google Scholar 

  • Graepel, T., Lauter, K., Naehrig, M.: ML confidential: Machine learning on encrypted data. In: International Conference on Information Security and Cryptology, pp. 1–21. Springer (2012)

  • Gubbi, J., Buyya, R., Marusic, S., Palaniswami, M.: Internet of Things (IoT): a vision, architectural elements, and future directions. Future Gener. Comput. Syst. 29(7), 1645–1660 (2013)

    Article  Google Scholar 

  • Han, S., Shen, H., Philipose, M., Agarwal, S., Wolman, A., Krishnamurthy, A.: Mcdnn: An approximation-based execution framework for deep stream processing under resource constraints. In: Proceedings of the 14th Annual International Conference on Mobile Systems, Applications, and Services, pp. 123–136 (2016)

  • He, K., Sun, J.: Convolutional neural networks at constrained time cost. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp. 5353–5360 (2015)

  • He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

  • Kang, Y., Hauswald, J., Gao, C., Rovinski, A., Mudge, T., Mars, J., Tang, L.: Neurosurgeon: collaborative intelligence between the cloud and mobile edge. ACM SIGARCH Comput. Archit. News 45(1), 615–629 (2017)

    Article  Google Scholar 

  • Krizhevsky, A., Sutskever, I., Hinton, G.E.: Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp. 1097–1105 (2012)

  • Lane, N.D., Georgiev, P.: Can deep learning revolutionize mobile sensing? In: Proceedings of the 16th International Workshop on Mobile Computing Systems and Applications, pp. 117–122. ACM (2015)

  • Li, M., Lai, L., Suda, N., Chandra, V., Pan, D.Z.: Privynet: A flexible framework for privacy-preserving deep neural network training. CoRR, abs/1709.06161 (2017)

  • Lin, M., Chen, Q., Yan, S.: Network in network. arXiv preprint arXiv:1312.4400(2013)

  • Liu, W., Anguelov, D., Erhan, D., Szegedy, C., Reed, S., Fu, C.-Y., Berg, A.C.: Ssd: Single shot multibox detector. In: Proceedings of European Conference on Computer Vision (ECCV), pp. 21–37 (2016)

  • Ma, X., Guo, F. M., Niu, W., Lin, X., Tang, J., Ma, K., & Wang, Y.: PCONV: The Missing but Desirable Sparsity in DNN Weight Pruning for Real-Time Execution on Mobile Devices. In 2020 AAAI Conference on Artificial Intelligence (AAAI), pp. 5117–5124 (2020)

  • Melis, L., Song, C., De Cristofaro, E., Shmatikov, V.: Exploiting unintended feature leakage in collaborative learning. In: 2019 IEEE Symposium on Security and Privacy (SP), pp. 691–706. IEEE (2019)

  • Mohassel, P., Zhang, Y.: Secureml: A system for scalable privacy-preserving machine learning. In: 2017 IEEE Symposium on Security and Privacy (SP), pp. 19–38. IEEE (2017)

  • Orlandi, C., Piva, A., Barni, M.: Oblivious neural network computing via homomorphic encryption. EURASIP J. Inf. Secur. 2007, 1–11 (2007)

    Article  Google Scholar 

  • Osia, S.A., Shamsabadi, A.S., Sajadmanesh, S., Taheri, A., Katevas, K., Rabiee, H.R., Lane, N.D., Haddadi, H.: A hybrid deep learning architecture for privacy-preserving mobile analytics. IEEE Internet Things J. 7(5), 4505–4518 (2020)

    Article  Google Scholar 

  • Park, E., Ahn, J., & Yoo, S.: Weighted-entropy-based quantization for deep neural networks. In 2017 Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5456–5464 (2017)

  • Qiu, J., Wang, J., Yao, S., Guo, K., Li, B., Zhou, E., Yu, J., Tang, T., Xu, N., Song, S.: Going deeper with embedded fpga platform for convolutional neural network. In: Proceedings of the 2016 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays, pp. 26–35. ACM (2016)

  • Redmon, J., Divvala, S., Girshick, R., Farhadi, A.: You only look once: Unified, real-time object detection. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 779–788 (2016)

  • Sehgal, A., Kehtarnavaz, N.: A convolutional neural network smartphone app for real-time voice activity detection. IEEE Access 6, 9017–9026 (2018)

    Article  Google Scholar 

  • Skala, K., Davidovic, D., Afgan, E., Sovic, I., Sojat, Z.: Scalable distributed computing hierarchy: cloud, fog and dew computing. In: IEEE International Conference on Cloud Computing Technology and Science, vol. 1, pp. 16–24 (2015)

  • Szegedy, C., Liu, W., Jia, Y., Sermanet, P., Reed, S., Anguelov, D., Erhan, D., Vanhoucke, V., Rabinovich, A.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9 (2015)

  • Tang, Y., Wang, Y., Li, H., Li, X.: MV-Net: toward real-time deep learning on mobile GPGPU systems. ACM J. Emerg. Technol. Comput. Syst. (JETC) 15(4), 35 (2019)

    Google Scholar 

  • Teerapittayanon, S., McDanel, B., Kung, H.-T.: Distributed deep neural networks over the cloud, the edge and end devices. In: 2017 IEEE 37th International Conference on Distributed Computing Systems (ICDCS), pp. 328–339. IEEE (2017a)

  • Teerapittayanon, S., Mcdanel, B., Kung, H.T.: Distributed Deep Neural Networks Over the Cloud, the Edge and End Devices. international conference on distributed computing systems, pp. 328–339 (2017b)

  • Vardhana, M., Arunkumar, N., Lasrado, S., Abdulhay, E., Ramirez-Gonzalez, G.: Convolutional neural network for bio-medical image segmentation with hardware acceleration. Cogn. Syst. Res. 50, 10–14 (2018)

    Article  Google Scholar 

  • Wang, X., Wang, J., Wang, X., Chen, X.: Energy and delay tradeoff for application offloading in mobile cloud computing. IEEE Syst. J. 11(2), 858–867 (2017)

    Article  Google Scholar 

  • Wang, J., Zhang, J., Bao, W., Zhu, X., Cao, B., Yu, P.S.: Not just privacy: Improving performance of private deep learning in mobile cloud. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, pp. 2407–2416 (2018)

  • Xia, F., Ding, F., Li, J., Kong, X., Yang, L.T., Ma, J.: Phone2Cloud: Exploiting computation offloading for energy saving on smartphones in mobile cloud computing. Inf. Syst. Front. 16(1), 95–111 (2014)

    Article  Google Scholar 

  • Xiang, L., Ma, H., Zhang, H., Zhang, Y., Ren, J., Zhang, Q.: Interpretable Complex-Valued Neural Networks for Privacy Protection. In International Conference on Learning Representations (2020)

  • Zhang, T., Zhu, Q.: Dynamic differential privacy for ADMM-based distributed classification learning. IEEE Trans. Inf. Forensics Secur. 12(1), 172–187 (2016)

    Article  Google Scholar 

  • Zhang, Q., Yang, L.T., Chen, Z.: Privacy preserving deep computation model on cloud for big data feature learning. IEEE Trans. Comput. 65(5), 1351–1362 (2015)

    Article  MathSciNet  Google Scholar 

  • Zhou, J., Dai, H.N., Wang, H.: Lightweight convolution neural networks for mobile edge computing in transportation cyber physical systems. ACM Trans. Intell. Syst. Technol. (TIST) 10(6), 1–20 (2019)

    Article  Google Scholar 

Download references

Acknowledgements

This work was funded in part by the National Key Research and Development Program of China (Grant number 2018AAA0102705), and in part by the National Natural Science Foundation of China (Grant number 61874124, 61876173).

Author information

Authors and Affiliations

  1. State Key Laboratory of Computer Architecture, Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China

    Yibin Tang, Ying Wang, Huawei Li & Xiaowei Li

  2. University of Chinese Academy of Sciences, Beijing, China

    Yibin Tang, Ying Wang, Huawei Li & Xiaowei Li

  3. Peng Cheng Laboratory, Shenzhen, China

    Huawei Li

  4. Wuhan Digital Engineering Institute, Wuhan, China

    Yibin Tang

Authors
  1. Yibin Tang
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ying Wang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Huawei Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaowei Li
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Huawei Li.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Tang, Y., Wang, Y., Li, H. et al. To cloud or not to cloud: an on-line scheduler for dynamic privacy-protection of deep learning workload on edge devices. CCF Trans. HPC 3, 85–100 (2021). https://doi.org/10.1007/s42514-020-00052-7

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s42514-020-00052-7

Keywords

Search

Navigation