Abstract
Nowadays, image sensors are widely used for industrial or augmented and virtual reality (AR)/(VR) applications. Often, multiple sensors are used and the host platform needs to process each high resolution image frame with low latency. Therefore, the processing units and the communication have a big impact on the power consumption of the image processing system. This paper describes a method to simulate a smart image sensor and the impact of different output resolutions on the image processing system. In-sensor processing is simulated, by sending different image resolutions to the host platform, where a shape matching algorithm was processed. It shows an analysis of the power consumption of the communication from the sensor with region-of-interest (ROI) data to the host and the energy consumption of the whole image processing system with a Mira220 and an Nvidia Jetson Nano. The power consumption of the MIPI interface can be reduced by over 83% for ROIs smaller than 640\(\,\times \,\)480. Furthermore, the system energy consumption can be reduces by 18% at 90 fps and 21% at 60 fps.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Silva, R., Rudek, M., Szejka, A., Canciglieri, J.O.: Machine vision systems for industrial quality control inspections. In: 15th IFIP WG 5.1 International Conference, Turin, Italy (2018). ISBN: 978-3-030-01613-5. https://doi.org/10.1007/978-3-030-01614-2_58
Zhou, W., Fei, M., Zhou, H., Li, K.: A sparse representation based fast detection method for surface defect detection of bottle caps. Neurocomputing 123, 406–414 (2014)
Horputra, P., Phrajonthong, R., Kaewprapha, P.: deep learning-based bottle caps inspection in beverage manufacturing and packaging process. In: 2021 9th International Electrical Engineering Congress (iEECON), pp. 499–502 (2021). https://doi.org/10.1109/iEECON51072.2021.9440326
Manzano, J., Bolaños, T., Muñoz, M., Arenas, V., Orozco, H.: Evaluation of a Machine Vision System Applied to Quality Control in a Liquid Filling, Lid and Labeling Line for Bottles, pp. 140–154, October (2020). ISBN: 978-3-030-62553-5. https://doi.org/10.1007/978-3-030-62554-2_11
Huang, S.-H., Pan, Y.-C.: Automated visual inspection in the semiconductor industry: a survey. Comput. Ind. 66, 1–10 (2015). ISSN: 0166–3615. https://doi.org/10.1016/j.compind.2014.10.006
Tripathi, M., Maktedar, D.: A role of computer vision in fruits and vegetables among various horticulture products of agriculture fields: a survey. Inf. Process. Agric. 7 (2019). https://doi.org/10.1016/j.inpa.2019.07.003
Liu, C., Hall, M., De Nardi, R., Trail, N., Newcombe, R.: Sensors for future VR applications. In: 2017 International Image Sensor Workshop (IISW), pp. 250–253 (2017)
Liu, C., Berkovich, A., Chen, S., Reyserhove, H., Sarwar, S.S., Tsai, T.-H.: Intelligent vision systems - bringing human-machine interface to AR/VR. In: 2019 IEEE International Electron Devices Meeting (IEDM), pp. 10.5.1–10.5.4 (2019). https://doi.org/10.1109/IEDM19573.2019.8993566
Sony Corporation: Sony to Release World’s First Intelligent Vision Sensors with AI Processing Functionality. https://www.sony.com/en/SonyInfo/News/Press/202005/20-037E/, 14 May 2020. Accessed 22 July 2021
Sony Group Corporation: Sony’s Latest Image Sensors and the Technologies that Lies Behind Them. https://www.sony.com/en/SonyInfo/technology/stories/imagesensor7tech/, 15 October 2020. Accessed 22 July 2021]
Sony Semiconductor Solutions Corporation: Imaging and Sensing Technology. https://www.sony-semicon.co.jp/e/technology/imaging-sensing/. 18 February 2021. Accessed 22 July 2021
Pinkham, R., Schmidt, T., Berkovich, A.: Algorithm-aware neural network based image compression for high-speed imaging. In: 2020 IEEE International Conference on Artificial Intelligence and Virtual Reality (AIVR), pp. 196–199, IEEE Computer Society, Los Alamitos, CA, USA, December (2020). https://doi.org/10.1109/AIVR50618.2020.00040
Pinkham, R., Berkovich, A., Zhang, Z.: Near-sensor distributed DNN processing for augmented and virtual reality. IEEE J. Emerg. Sel. Topics Circ. Syst. 4(4), 663–676 (2021). https://doi.org/10.1109/JETCAS.2021.3121259
Howard, A., et al.: Searching for MobileNetV3. In: IEEE/CVF International Conference on Computer Vision, pp. 1314–1324 (2019). https://doi.org/10.1109/ICCV.2019.00140
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 770–778 (2016). https://doi.org/10.1109/CVPR.2016.90
Holly, S., Wendt, A., Lechner, M.: Profiling energy consumption of deep neural networks on NVIDIA Jetson Nano. In: 2020 11th International Green and Sustainable Computing Workshops (IGSC), pp. 1–6, Pullman, WA, USA (2020). https://doi.org/10.1109/IGSC51522.2020.9290876
ams AG: Jetcis evaluation platform for mira sensor family quick start guide. https://ams.com/documents/20143/4421878/JetCis_QG001006_1-00.pdf. 4 February 2021. Accessed 22 September 2021
Nvidia Cooperation: Jetson Nano Development Kit. https://docs.nvidia.com/jetson/l4t/index.html#page/Tegra%20Linux%20Driver%20Package%20Development%20Guide/power_management_nano.html. 3 August 2021. Accessed 14 October 2021
Nvidia Cooperation: Jetson Nano Developer Kit User Guide. https://developer.nvidia.com/embedded/jetson-nano-developer-kit. 15 January 2020. Accessed 24 September 2021
Fiala, G., Loinig, J., Steger, C.: Evaluation of an integer optimized shape matching algorithm. In: 2021 IEEE Sensors Applications Symposium (SAS), pp. 1–6 (2021). https://doi.org/10.1109/SAS51076.2021.9530015
Acknowledgment
This research was supported by ams-OSRAM AG, Premstaetten, Austria. Thank you very much for the support.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Fiala, G., Loining, J., Steger, C. (2023). Impact of Image Sensor Output Data on Power Consumption of the Image Processing System. In: Arai, K. (eds) Intelligent Systems and Applications. IntelliSys 2022. Lecture Notes in Networks and Systems, vol 542. Springer, Cham. https://doi.org/10.1007/978-3-031-16072-1_45
Download citation
DOI: https://doi.org/10.1007/978-3-031-16072-1_45
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-16071-4
Online ISBN: 978-3-031-16072-1
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)