Skip to main content
SpringerLink
Log in
Book cover

International Conference on Pattern Recognition

ICPR 2021: Pattern Recognition. ICPR International Workshops and Challenges pp 523–537Cite as

Evaluation of Edge Platforms for Deep Learning in Computer Vision

  • Conference paper
  • First Online:

  • 2202 Accesses

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 12664))

Abstract

In recent years, companies, such as Intel and Google, have brought onto the market small low-power platforms that can be used to deploy and run inference of Deep Neural Networks at a low cost. These platforms can process data at the edge, such as images from a camera, to avoid transfer of large amount of data across a network. To determine which platform to use for a specific task, practitioners usually compare parameters, such as inference time and power consumption. However, to provide a better incentive on platform selection based on requirements, it is important to also consider the platform price. In this paper, we explore platform/model trade-offs, by providing benchmarks of state-of-the-art platforms within three common computer vision tasks; classification, detection and segmentation. By also considering the price of each platform, we provide a comparison of price versus inference time, to aid quick decision making in regard to platform and model selection. Finally, by analysing the operation allocation of models for each platform, we identify operations that should be optimised, based on platform/model selection.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Abadi, M., et al.: Tensorflow: a system for large-scale machine learning. In: 12th USENIX Symposium on Operating Systems Design and Implementation (OSDI 2016), pp. 265–283 (2016). https://www.usenix.org/system/files/conference/osdi16/osdi16-abadi.pdf

  2. Almeida, M., Laskaridis, S., Leontiadis, I., Venieris, S.I., Lane, N.D.: Embench: quantifying performance variations of deep neural networks across modern commodity devices. In: The 3rd International Workshop on Deep Learning for Mobile Systems and Applications, pp. 1–6 (2019)

    Google Scholar 

  3. Alom, M.Z., et al.: A state-of-the-art survey on deep learning theory and architectures. Electronics 8(3), 292 (2019)

    Article  Google Scholar 

  4. Amazon: Amazon web services (aws) (2020). https://aws.amazon.com/. Accessed 12 July 2020

  5. Bianco, S., Cadene, R., Celona, L., Napoletano, P.: Benchmark analysis of representative deep neural network architectures. IEEE Access 6, 64270–64277 (2018)

    Article  Google Scholar 

  6. Blouw, P., Choo, X., Hunsberger, E., Eliasmith, C.: Benchmarking keyword spotting efficiency on neuromorphic hardware. arXiv preprint arXiv:1812.01739 (2018)

  7. Canziani, A., Paszke, A., Culurciello, E.: An analysis of deep neural network models for practical applications. arXiv preprint arXiv:1605.07678 (2016)

  8. Chen, L.C., Papandreou, G., Schroff, F., Adam, H.: Rethinking atrous convolution for semantic image segmentation. arXiv preprint arXiv:1706.05587 (2017)

  9. Cosmic Shovel, I.: Amazon price tracker, amazon price history charts, price watches, and price drop alerts. https://camelcamelcamel.com/ (March 2019), accessed: 24 September 2020

  10. Deng, J., Dong, W., Socher, R., Li, L.J., Li, K., Fei-Fei, L.: Imagenet: a large-scale hierarchical image database. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 248–255 (2009)

    Google Scholar 

  11. Everingham, M., Van Gool, L., Williams, C.K.I., Winn, J., Zisserman, A.: The pascal visual object classes (VOC) challenge. Int. J. Comput. Vision 88(2), 303–338 (2010)

    Article  Google Scholar 

  12. Garcia-Garcia, A., Orts-Escolano, S., Oprea, S., Villena-Martinez, V., Martinez-Gonzalez, P., Garcia-Rodriguez, J.: A survey on deep learning techniques for image and video semantic segmentation. Appl. Soft Comput. 70, 41–65 (2018)

    Article  Google Scholar 

  13. Guo, Y., Liu, Y., Georgiou, T., Lew, M.S.: A review of semantic segmentation using deep neural networks. Int. J. Multimed. Inform. Retrieval 7(2), 87–93 (2018)

    Article  Google Scholar 

  14. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)

    Google Scholar 

  15. Howard, A.G., et al.: Mobilenets: efficient convolutional neural networks for mobile vision applications. arXiv preprint arXiv:1704.04861 (2017)

  16. Huang, J., et al.: Speed/accuracy trade-offs for modern convolutional object detectors. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition pp. 7310–7311 (2017)

    Google Scholar 

  17. Iandola, F.N., Han, S., Moskewicz, M.W., Ashraf, K., Dally, W.J., Keutzer, K.: Squeezenet: Alexnet-level accuracy with 50x fewer parameters and \(<\)0.5 mb model size. arXiv preprint arXiv:1602.07360 (2016)

  18. Ignatov, A., et al.: Ai benchmark: running deep neural networks on android smartphones. In: Proceedings of the European Conference on Computer Vision, pp. 288–314 (2018)

    Google Scholar 

  19. Intel: Intel neural compute stick 2, September 2020. https://software.intel.com/en-us/neural-compute-stick. Accessed 6 Sept 2020

  20. Intel: Openvino toolkit, December 2020. https://docs.openvinotoolkit.org/2018_R5/index.html. Accessed 3 Sept 2020

  21. Jia, Y., et al.: Caffe: convolutional architecture for fast feature embedding. arXiv preprint arXiv:1408.5093 (2014)

  22. Krishnamoorthi, R.: Quantizing deep convolutional networks for efficient inference: A whitepaper. arXiv preprint arXiv:1806.08342 (2018)

  23. Lin, T.-Y., et al.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10602-1_48

    Chapter  Google Scholar 

  24. Liu, L., et al.: Deep learning for generic object detection: a survey. arXiv preprint arXiv:1809.02165 (2018)

  25. Liu, W., et al.: SSD: single shot MultiBox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 21–37. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_2

    Chapter  Google Scholar 

  26. NVIDIA: Accelerating inference in tf trt user guide, August 2019. https://docs.nvidia.com/deeplearning/frameworks/tf-trt-user-guide/index.html. Accessed 3 Sept 2019

  27. NVIDIA: Jetson tx2 module (2020). https://developer.nvidia.com/embedded/jetson-tx2. Accessed 12 July 2020

  28. Pena, D., Forembski, A., Xu, X., Moloney, D.: Benchmarking of CNNs for low-cost, low-power robotics applications. In: RSS 2017 Workshop: New Frontier for Deep Learning in Robotics, pp. 1–5 (2017)

    Google Scholar 

  29. Russakovsky, O., et al.: ImageNet Large Scale Visual Recognition Challenge. International Journal of Computer Vision (2015)

    Google Scholar 

  30. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.C.: Mobilenetv 2: Inverted residuals and linear bottlenecks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4510–4520 (2018)

    Google Scholar 

  31. Szegedy, C., Ioffe, S., Vanhoucke, V., Alemi, A.A.: Inception-v4, inception-resnet and the impact of residual connections on learning. In: Proceedings of the AAAI Conference on Artificial Intelligence, pp. 4278–4284 (2017)

    Google Scholar 

  32. Trindade, R.G., Lima, J.V.F., Charão, A.S.: Performance evaluation of deep learning frameworks over different architectures. In: International Conference on Vector and Parallel Processing, pp. 92–104 (2018)

    Google Scholar 

  33. Velasco-Montero, D., Fernández-Berni, J., Carmona-Galán, R., Rodríguez-Vázquez, Á.: Optimum selection of DNN model and framework for edge inference. IEEE Access 6, 51680–51692 (2018)

    Article  Google Scholar 

  34. Zhang, X., Wang, Y., Shi, W.: PCAMP: performance comparison of machine learning packages on the edges. In: \(\{\)USENIX\(\}\) Workshop on Hot Topics in Edge Computing (HotEdge 18) (2018)

    Google Scholar 

  35. Zou, Z., Shi, Z., Guo, Y., Ye, J.: Object detection in 20 years: a survey. arXiv preprint arXiv:1905.05055 (2019)

Download references

Acknowledgment

This work was funded by Innovation Fund Denmark under Grant 5189-00222B and 7038-00170B.

Author information

Authors and Affiliations

  1. Department of Architecture, Design and Media Technology, Aalborg University, Aalborg, Denmark

    Christoffer Bøgelund Rasmussen, Aske Rasch Lejbølle, Kamal Nasrollahi & Thomas B. Moeslund

  2. Research Department of Milestone Systems, Copenhagen, Denmark

    Kamal Nasrollahi

Authors
  1. Christoffer Bøgelund Rasmussen
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Aske Rasch Lejbølle
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kamal Nasrollahi
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Thomas B. Moeslund
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Christoffer Bøgelund Rasmussen .

Editor information

Editors and Affiliations

  1. Dipartimento di Ingegneria dell'Informazione, University of Firenze, Florence, Firenze, Italy

    Alberto Del Bimbo

  2. Dipartimento di Ingegneria “Enzo Ferrari”, Università di Modena e Reggio Emilia, Modena, Italy

    Rita Cucchiara

  3. Department of Computer Science, Boston University, Boston, MA, USA

    Stan Sclaroff

  4. Dipartimento di Matematica e Informatica, University of Catania, Catania, Catania, Italy

    Giovanni Maria Farinella

  5. Cloud & AI, JD.COM, Beijing, China

    Tao Mei

  6. Dipartimento di Ingegneria dell’Informazione, University of Firenze, Firenze, Italy

    Marco Bertini

  7. Computational Sciences Department, National Institute of Astrophysics, Optics and Electronics (INAOE), Tonantzintla, Puebla, Mexico

    Hugo Jair Escalante

  8. Dipartimento di Ingegneria “Enzo Ferrari”, Università di Modena e Reggio Emilia, Modena, Italy

    Roberto Vezzani

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Rasmussen, C.B., Lejbølle, A.R., Nasrollahi, K., Moeslund, T.B. (2021). Evaluation of Edge Platforms for Deep Learning in Computer Vision. In: Del Bimbo, A., et al. Pattern Recognition. ICPR International Workshops and Challenges. ICPR 2021. Lecture Notes in Computer Science(), vol 12664. Springer, Cham. https://doi.org/10.1007/978-3-030-68799-1_38

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-68799-1_38

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-68798-4

  • Online ISBN: 978-3-030-68799-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Societies and partnerships

Search

Navigation