skip to main content
10.1145/3555776.3577681acmconferencesArticle/Chapter ViewAbstractPublication PagessacConference Proceedingsconference-collections
research-article

Low-Power Footprint Inference with a Deep Neural Network offloaded to a Service Robot through Edge Computing

Published: 07 June 2023 Publication History

Abstract

Recent advances in the field of Artificial Intelligence (AI) have enabled a vast variety of innovative digital services. Mobile smart devices usually resort to the cloud to run deep neural networks (DNN) due to insufficient computational power or severe power constraints that precludes the use of consumer-grade on-board processors and power-hungry Graphics Processing Units (GPU). However, the use of cloud computing in service robot deployments has shortcomings related with latency, privacy, security and reliability, which often makes it inconvenient or even impractical. A possible solution is the use of specialized edge computing devices with a trade-off between onboard robot computing resources and power footprint. This approach is exploited in this paper for a service robot programmed in ROS, equipped with a camera for image perception, a 2D LiDAR for autonomous navigation, and a system on module Nvidia Jetson AGX Xavier. The viability of running DNN aboard this robot to perform image classification with low-power footprint in a Covid-19 use case scenario is demonstrated.

References

[1]
Jochen Wirtz, Paul G. Patterson, Werner H. Kunz, Thorsten Gruber, Vinh Nhat Lu, Stefanie Paluch, and Antje Martins. Brave new world: service robots in the frontline. Journal of Service Management, 29(5):907--931, 2018.
[2]
International Federation of Robotics. Classification of service robots by application areas. https://www.ifr.org/img/office/Service_Robots_2016_Chapter_1_2.pdf, 2016. Accessed: 2022-10-06.
[3]
Tanya M. Anandan. The consumerization of robots - implications for you, me, and industry. Robotic Industries Association, https://www.robotics.org/content-detail.cfm/Industrial-Robotics-Industry-Insights/The-Consumerization-of-Robots-Implications-for-You-Me-and-Industry/content_id/6377, Jan. 2017. Accessed: 2022-10-06.
[4]
Tiago Carneiro, Raul Da Nóbrega, and Thiago Nepomuceno et al. Performance analysis of google colaboratory as a tool for accelerating deep learning applications. IEEE Access, 6:61677--61685, 2018.
[5]
Peter Mell and Tim Grance. The NIST definition of cloud computing. National Institute of Standards and Technology, 53(6), 2011.
[6]
Weisong Shi, Jie Cao, Quan Zhang, Youhuizi Li, and Lanyu Xu. Edge computing: Vision and challenges. IEEE Internet of Things Journal, 3(5):637--646, 2016.
[7]
Nasir Abbas, Yan Zhang, Amir Taherkordi, and Tor Skeie. Mobile edge computing: A survey. IEEE Internet of Things Journal, 5(1):450--465, 2018.
[8]
Baoling Qin, Qiao Luo, Yunshi Luo, Jianwei Zhang, Jianjie Liu, and Liangyun Cui. Research and application of key technologies of edge computing for industrial robots. In 2020 IEEE 4th Information Technology, Networking, Electronic and Automation Control Conference (ITNEC), volume 1, pages 2157--2164, 2020.
[9]
V. K. Sarker, J. Peña Queralta, T. N. Gia, H. Tenhunen, and T. Westerlund. Offloading SLAM for indoor mobile robots with edge-fog-cloud computing. In 2019 1st International Conference on Advances in Science, Engineering and Robotics Technology (ICASERT), pages 1--6, 2019.
[10]
L. Qingqing, F. Yuhong, J. Peña Queralta, T. N. Gia, H. Tenhunen, Z. Zou, and T. Westerlund. Edge computing for mobile robots: Multi-robot feature-based LiDAR odometry with FPGAs. In 2019 Twelfth International Conference on Mobile Computing and Ubiquitous Network (ICMU), pages 1--2, 2019.
[11]
Peng Huang, Liekang Zeng, Xu Chen, Ke Luo, Zhi Zhou, and Shuai Yu. Edge robotics: Edge-computing-accelerated multirobot simultaneous localization and mapping. IEEE Internet of Things Journal, 9(15):14087--14102, 2022.
[12]
Shenshen Gu, Xinyi Chen, Wei Zeng, and Xin Wang. A deep learning tennis ball collection robot and the implementation on NVIDIA Jetson TX1 board. In IEEE/ASME Int. Conf. on Advanced Intellig. Mechatronics, pages 170--175, 2018.
[13]
Inkyu Sa, Zetao Chen, Marija Popović, Raghav Khanna, Frank Liebisch, Juan Nieto, and Roland Siegwart. weedNet: Dense semantic weed classification using multispectral images and MAV for smart farming. IEEE Robotics and Automation Letters, 3(1):588--595, 2018.
[14]
Joseph Redmon, Santosh Divvala, Ross Girshick, and Ali Farhadi. You Only Look Once: Unified, Real-Time Object Detection. arXiv e-prints, art. arXiv:1506.02640, Jun. 2015. arXiv:1506.02640.
[15]
Oriol Vinyals, Meire Fortunato, and Navdeep Jaitly. Pointer networks. arXiv e-prints, 2017. arXiv:1506.03134.
[16]
NVIDIA Jetson. https://www.nvidia.com/en-us/autonomous-machines/embedded-systems, 2022. Accessed on 2022-10-13.
[17]
Morgan Quigley, Ken Conley, Brian P. Gerkey, Josh Faust, Tully Foote, Jeremy Leibs, Rob Wheeler, and Andrew Y. Ng. ROS: an open-source robot operating system. In ICRA Workshop on Open Source Software, Kobe, Japan, 2009.
[18]
João Machado Santos, Micael S. Couceiro, David Portugal, and Rui P. Rocha. A sensor fusion layer to cope with reduced visibility in SLAM. Journal of Intelligent & Robotic Systems, 80(3):401--422, December 2015. ISSN 1573-0409.
[19]
David Portugal and Rui P. Rocha. Cooperative multi-robot patrol with Bayesian learning. Autonomous Robots, 40(5):929--953, 2016.
[20]
Dieter Fox. KLD-Sampling: Adaptive particle filters. In Proceedings of the 14th International Conference on Neural Information Processing Systems: Natural and Synthetic, NIPS'01, page 713--720, Cambridge, MA, USA, 2001. MIT Press.
[21]
ZED 2 AI stereo camera. https://www.stereolabs.com/zed-2/. Stereolabs. Accessed on 2022-10-13.
[22]
Haotian Liu, Rafael A. Rivera Soto, Fanyi Xiao, and Yong Jae Lee. YolactEdge: Real-time instance segmentation on the edge. arXiv e-prints, art. arXiv:2012.12259, April 2021. arXiv:2012.12259.
[23]
ZED stand. https://www.thingiverse.com/thing:3208903. Thingiverse. Accessed on 2022-10-13.
[24]
NVIDIA Developer. JetPack SDK 4.4 archive. https://developer.nvidia.com/jetpack-sdk-44-archive. Accessed on 2022-10-13.
[25]
Jason Tread. UnixBench benchmark. https://github.com/cloudharmony/unixbench. Accessed on 2022-10-15.
[26]
Chien-Yao Wang, Alexey Bochkovskiy, and Hong-Yuan Mark Liao. YOLOv7: Trainable bag-of-freebies sets new state-of-the-art for real-time object detectors. arXiv e-prints, Jul. 2022. arXiv:2207.02696.
[27]
Alexey Bochkovskiy, Chien-Yao Wang, and H. Liao. YOLOv4: Optimal speed and accuracy of object detection. arXiv e-prints, 2020. arXiv:2004.10934.
[28]
Tsung-Yi Lin, Michael Maire, Serge Belongie, James Hays, Pietro Perona, Deva Ramanan, Piotr Dollár, and C. Lawrence Zitnick. Microsoft COCO: Common objects in context. In David Fleet, Tomas Pajdla, Bernt Schiele, and Tinne Tuytelaars, editors, Computer Vision - ECCV 2014, pages 740--755. Springer, 2014. ISBN 978-3-319-10602-1.
[29]
Stéphane Charette. Different implementations of YOLO. https://github.com/AlexeyAB/darknet, 2022. Accessed on 2022-10-20.
[30]
G. Jignesh Chowdary, Narinder Singh Punn, Sanjay Kumar Sonbhadra, and Sonali Agarwal. Face mask detection using transfer learning of InceptionV3. In Proceedings of 8th International Conference on Big Data Analytics, page 81--90, Dec. 2020.
[31]
Preeti Nagrath, Rachna Jain, Agam Madan, Rohan Arora, Piyush Kataria, and Jude Hemanth. SSDMNV2: A real time DNN-based face mask detection system using single shot multibox detector and MobileNetV2. Sustainable Cities and Society, 66:102692, 2021. ISSN 2210-6707.
[32]
Savyasachi Gupta, Rudraksh Kapil, Goutham Kanahasabai, Shreyas Srinivas Joshi, and Aniruddha Srinivas Joshi. SD-Measure: A social distancing detector. In 12th International Conference on Computational Intelligence and Communication Networks, pages 306--311, 2020.
[33]
Mohammad Al-Sa'd, Serkan Kiranyaz, Iftikhar Ahmad, Christian Sundell, Matti Vakkuri, and Moncef Gabbouj. A social distance estimation and crowd monitoring system for surveillance cameras. Sensors, 22(2), 2022. ISSN 1424-8220.
[34]
Meirui Qian and Jianli Jiang. COVID-19 and social distancing. Journal of Public Health, pages 1--3, 2020.
[35]
MakeML. Mask Dataset. https://makeml.app/datasets/mask, May 2020. Accessed on 2022-10-20.
[36]
Stéphane Charette. Darkhelp: C++ API for the neural network framework Dark-net. https://www.ccoderun.ca/darkhelp/api/index.html, 2022. Accessed on 2022-10-20.

Cited By

View all

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
SAC '23: Proceedings of the 38th ACM/SIGAPP Symposium on Applied Computing
March 2023
1932 pages
ISBN:9781450395175
DOI:10.1145/3555776
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 07 June 2023

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. service robot
  2. edge computing
  3. deep NN
  4. system on module
  5. ROS

Qualifiers

  • Research-article

Funding Sources

  • FCT

Conference

SAC '23
Sponsor:

Acceptance Rates

Overall Acceptance Rate 1,650 of 6,669 submissions, 25%

Upcoming Conference

SAC '25
The 40th ACM/SIGAPP Symposium on Applied Computing
March 31 - April 4, 2025
Catania , Italy

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 81
    Total Downloads
  • Downloads (Last 12 months)40
  • Downloads (Last 6 weeks)0
Reflects downloads up to 16 Feb 2025

Other Metrics

Citations

Cited By

View all

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media