Abstract
Humanoid robots find application in a variety of tasks such as emotional recognition for human-robot interaction (HRI). Despite their capabilities, these robots have a sequential computing system that limits the execution of high computational cost algorithms such as Convolutional Neural Networks (CNNs), which have shown good performance in recognition tasks. This limitation reduces their performance in HRI applications. As an alternative to sequential computing units are Field-programmable gate arrays (FPGAs) and Graphics Processing Units (GPUs), which have a high degree of parallelism, high performance, and low power consumption. In this paper, we propose a visual perception enhancement system for humanoid robots using FPGA or GPU based embedded systems running a CNN, while maintaining autonomy through an external computational system added to the robot structure. Our work has as a case study the humanoid robot NAO, however, the work can be replicated on other robots such as Pepper and Robotis OP3. The development boards used were the Xilinx Ultra96 FPGA, Intel Cyclone V SoC FPGA and Nvidia Jetson TX2 GPU. Nevertheless, our design allows the integration of other heterogeneous architectures with high parallelism and low power consumption. The Tinier-Yolo, Alexnet and Inception-V1 CNNs are executed and real-time results were obtained for the FPGA and GPU cards, while in Alexnet, the expected results were presented in the Jetson TX2.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Fanello, S.R., et al.: Visual recognition for humanoid robots. Robot. Auton. Syst. 91, 151–168 (2017)
Cha, E., Matarić, M., Fong, T.: Nonverbal signaling for non-humanoid robots during human-robot collaboration. In: 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pp. 601–602. IEEE (2016)
Shamsuddin, S.: Initial response of autistic children in human-robot interaction therapy with humanoid robot NAO. In: IEEE 8th International Colloquium on Signal Processing and its Applications, pp. 188–193. IEEE (2012)
Sermanet, P., et al. OverFeat: integrated recognition, localization and detection using convolutional networks. arXiv preprint arXiv:1312.6229 (2013)
Nguyen, H.V., et al.: DASH-N: joint hierarchical domain adaptation and feature learning. IEEE Trans. Image Process. 24(12), 5479–5491 (2017)
Podpora, M., Gardecki, A.: Extending vision understanding capabilities of NAO robot by connecting it to a remote computational resource. In: Progress in Applied Electrical Engineering (PAEE), pp. 1–5. IEEE (2016)
Puheim, M., Bundzel, M., Madarász, L.: Forward control of robotic arm using the information from stereo-vision tracking system. In: IEEE 14th International Symposium on Computational Intelligence and Informatics (CINTI), pp. 57–62. IEEE (2013)
Noda, K., et al.: Multimodal integration learning of robot behavior using deep neural networks. Robot. Auton. Syst. 62(6), 721–736 (2014)
Biddulph, A., Houliston, T., Mendes, A., Chalup, S.K.: Comparing computing platforms for deep learning on a humanoid robot. In: Cheng, L., Leung, A.C.S., Ozawa, S. (eds.) ICONIP 2018. LNCS, vol. 11307, pp. 120–131. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-04239-4_11
Dundar, A., et al.: Embedded streaming deep neural networks accelerator with applications. IEEE Trans. Neural Netw. Learn. Syst. 28(7), 1572–1583 (2016)
Sozzo, D.E.L., Emanuele, O.: The automation of high level synthesis of convolutional neural networks. . In: IEEE International Parallel and Distributed Processing Symposium Workshops (IPDPSW), pp. 217–244. IEEE (2016)
Zhang, C., et al.: Caffeine: toward uniformed representation and acceleration for deep convolutional neural networks. IEEE Trans. Comput. Aided Des. Integr. Circuits Syst. 38(11), 2072–2085 (2018)
Zhang, C., et al.: Optimizing FPGA-based accelerator design for deep convolutional neural networks. In: Proceedings of the 2015 ACM/SIGDA International Symposium on Field-programmable Gate Arrays, pp. 161–170 (2015)
Blott, M., et al.: FINN-R: an end-to-end deep-learning framework for fast exploration of quantized neural networks. ACM Trans. Reconfig. Technol. Syst. (TRETS) 11(3), 1–23 (2018)
Wang, D., Xu, K., Jiang, D.: PipeCNN: an OpenCL-based open-source FPGA accelerator for convolution neural networks. In: 2017 International Conference on Field Programmable Technology (ICFPT), pp. 279–282. IEEE (2017)
Modasshir, M., Li, A.Q., Rekleitis, I.: Deep neural networks: a comparison on different computing platforms. In: 2018 15th Conference on Computer and Robot Vision (CRV), pp. 383–389. IEEE (2018)
Liang, S., et al.: FP-BNN: binarized neural network on FPGA. Neurocomputing 275, 1072–1086 (2018)
Xu, S.: Real-time implementation of YOLO+ JPDA for small scale UAV multiple object tracking. In: international conference on unmanned aircraft systems (ICUAS), pp. 1336–1341. IEEE (2018)
Ma, J., Chen, L., Gao, Z.: Hardware implementation and optimization of tiny-YOLO network. In: Zhai, G., Zhou, J., Yang, X. (eds.) IFTC 2017. CCIS, vol. 815, pp. 224–234. Springer, Singapore (2018). https://doi.org/10.1007/978-981-10-8108-8_21
Pot, E., et al.: Choregraphe: a graphical tool for humanoid robot programming. In: RO-MAN 2009-The 18th IEEE International Symposium on Robot and Human Interactive Communication, pp. 46–51. IEEE (2009)
Mattamala, M., Olave, G., González, C., Hasbún, N., Ruiz-del-Solar, J.: The NAO backpack: an open-hardware add-on for fast software development with the NAO robot. In: Akiyama, H., Obst, O., Sammut, C., Tonidandel, F. (eds.) RoboCup 2017. LNCS (LNAI), vol. 11175, pp. 302–311. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-00308-1_25
Cosmas, K., Kenichi, A.: Utilization of FPGA for onboard inference of landmark localization in CNN-based spacecraft pose estimation. Aerospace 7(11), 159 (2009)
Andri, R.: YodaNN: an ultra-low power convolutional neural network accelerator based on binary weights. In: IEEE Computer Society Annual Symposium on VLSI (ISVLSI), pp. 236–241. IEEE (2016)
Ni, L., et al.: An energy-efficient digital ReRAM-crossbar-based CNN with bitwise parallelism. IEEE Jo. Explor. Solid-State Comput. Devices Circuits 3, 37–46 (2017)
MICHEL, Olivier. Cyberbotics Ltd., Webots\(^{\rm TM}\): professional mobile robot simulation. International Journal of Advanced Robotic Systems, 2004, vol. 1, no 1, p. 5
Franklin, D.: NVIDIA Jetson TX2 Delivers Twice the Intelligence to the Edge (2017). https://devblogs.nvidia.com/jetson-tx2-delivers-twiceintelligence-edge/. Accessed 02 Nov 2019
Acknowledgements
This study were supported by the AE&CC research Group COL0053581, at the Sistemas de Control y Robótica Laboratory, attached to the Instituto Tecnológico Metropolitano. This work is part of the project “Improvement of visual perception in humanoid robots for objects recognition in natural environments using Deep Learning” with ID P17224.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this paper
Cite this paper
Guajo, J., Anzola, C.A., Betancur, D., Castaño-Londoño, L., Marquez-Viloria, D. (2021). Improvement of Visual Perception in Humanoid Robots Using Heterogeneous Architectures for Autonomous Applications. In: Figueroa-García, J.C., Díaz-Gutierrez, Y., Gaona-García, E.E., Orjuela-Cañón, A.D. (eds) Applied Computer Sciences in Engineering. WEA 2021. Communications in Computer and Information Science, vol 1431. Springer, Cham. https://doi.org/10.1007/978-3-030-86702-7_38
Download citation
DOI: https://doi.org/10.1007/978-3-030-86702-7_38
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-86701-0
Online ISBN: 978-3-030-86702-7
eBook Packages: Computer ScienceComputer Science (R0)