Abstract
In standard machine learning implementations, training and inference takes place on servers located remotely from where data is gathered. With the advent of the Internet of Things (IoT), the groundwork is laid to shift half of that computing burden (inferencing) closer to where data is gathered. This paradigm shift to edge computing can significantly decrease the latency and cost of these tasks. Many small, powerful devices have been developed in recent years with the potential to fulfill that goal. In this paper, we analyze two such devices, the NVIDIA Jetson AGX Xavier Developer Kit and the Microsoft Azure Stack Edge Pro (2 GPUs). In addition, the NVIDIA DGX-1 system containerized in a ruggedized case is also taken for running inference model at the Edge. For comparison, the performance of these devices is compared to more common inferencing devices, including a laptop, desktop, and high performance computing (HPC) system. The inferencing model used for testing is the Enhanced Super-Resolution Generative Adversarial Networks (ESRGANs), which was developed using techniques borrowed primarily from other GAN designs, most notably SRGANs and Relativistic average GANs (RaGANs), along with some novel techniques. Metrics chosen for benchmarking were inferencing time, GPU power consumption, and GPU temperature. We found that inferencing using ESRGANs was approximately 10 to 20 times slower on the Jetson edge device, but used approximately 100 to 300 times less power, and was approximately 2 times cooler than any of the other devices tested. The inferencing using ESRGANs performed very similarly on the Azure device as on the more traditional methods. The Azure device performed with slightly slower speeds and equivalent temperatures to the other devices, but with slightly less power consumption.














Similar content being viewed by others
Explore related subjects
Discover the latest articles, news and stories from top researchers in related subjects.Data availability
The DIV2K dataset, which was developed for super-resolution benchmarking by Timofte et al. [11], is publicly available for download.
References
Value to the Nation: Inland Navigation Fast Facts. https://www.iwr.usace.army.mil/Missions/Value-to-the-Nation/Fast-Facts/Inland-Navigation-Fast-Facts/
Xi Y, Zheng J, Jia W, He X, Li H, Ren Z, Lam K-M (2020). See clearly in the distance: Representation learning GAN for low resolution object recognition. IEEE Access 8:53203–53214. https://doi.org/10.1109/ACCESS.2020.2978980
Wang Xintao, Yu Ke, Wu Shixiang, Gu Jinjin, Liu Yihao, Dong Chao, Qiao Yu, Loy Chen (2018). Change: ESRGAN: Enhanced super-resolution generative adversarial networks. In: The European Conference on Computer Vision Workshops (ECCVW 2018)
Goodfellow I, Pouget-Abadie J, Mirza M, Xu B, Warde-Farley D, Ozair S, Courville AC, Bengio Y (2014). Generative adversarial networks. arXiv:1406.2661
Agarwal Shruti, Farid Hany, Gu Yuming, He Mingming, Nagano Koki, Li Hao (2019). Protecting World Leaders Against Deep Fakes. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops
Ledig C, Theis L, Huszár F, Caballero J, Cunningham A, Acosta A, Aitken A, Tejani A, Totz J, Wang Z, Shi W (2017). Photo-realistic single image super-resolution using a generative adversarial network. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 105–114. https://doi.org/10.1109/CVPR.2017.19
Jolicoeur-Martineau A (2018). The relativistic discriminator: a key element missing from standard GAN. CoRR arXiv:1807.00734
Chen J, Ran X (2019). Deep learning with edge computing: A review. Proceed IEEE 107(8):1655–1674. https://doi.org/10.1109/JPROC.2019.2921977
Tegrastats Utility. https://docs.nvidia.com/drive/drive_os_5.1.6.1L/nvvib_docs/index.html#page/DRIVE_OS_Linux_SDK_Development_Guide/Utilities/util_tegrastats.html
NVIDIA System Management Interface. https://developer.nvidia.com/nvidia-system-management-interface
Agustsson E, Timofte R (2017). NTIRE 2017 Challenge on Single Image Super-Resolution: Dataset and Study. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 1122–1131. https://doi.org/10.1109/CVPRW.2017.150
CV Notes Contributors: Common datasets from image Super Resolution. https://cvnote.ddlee.cc/2019/09/22/image-super-resolution-datasets.
Yazdani Aminabadi R, Rajbhandari S, Zhang M, Awan AA, Li C, Li D, Zheng E, Rasley J, Smith S, Ruwase O, He Y (2022). Deepspeed inference: Enabling efficient inference of transformer models at unprecedented scale. Technical Report MSR-TR-2022-21, Microsoft. https://www.microsoft.com/en-us/research/publication/
Zhu S, Duong LHK, Liu W (2022). Tab: unified and optimized ternary, binary and mixed-precision neural network inference on the edge. In: ACM Transactions on Embedded Computing Systems. https://hdl.handle.net/10356/155648
Acknowledgements
We would like to thank the High Performance Modernization Program (HPCMP) for their support to the summer student internship program (HIP) and the Edge Computing team in ERDC Supercomputing Research Center.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of Interest
The authors declare that they have no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Cheng, JR.C., Stanford, C., Glandon, S.R. et al. Macro benchmarking edge devices using enhanced super-resolution generative adversarial networks (ESRGANs). J Supercomput 79, 5360–5373 (2023). https://doi.org/10.1007/s11227-022-04819-3
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11227-022-04819-3