Abstract
While real-time face recognition has become increasingly popular, its use in decentralized systems and on embedded hardware presents numerous challenges. One challenge is the trade-off between accuracy and inference-time on constrained hardware resources. While achieving higher accuracy is desirable, it comes at the cost of longer inference-time. We first conduct a comparative study on the effect of using different face recognition distance functions and introduce a novel inference-time/accuracy plot to facilitate the comparison of different face recognition models. Every application must strike a balance between inference-time and accuracy, depending on its focus. To achieve optimal performance across the spectrum, we propose a combination of multiple models with distinct characteristics. This allows the system to address the weaknesses of individual models and to optimize performance based on the specific needs of the application.
We demonstrate the practicality of our proposed approach by utilizing two face detection models positioned at either end of the inference-time/accuracy spectrum to develop a multimodel face recognition pipeline. By integrating these models on an embedded device, we are able to achieve superior overall accuracy, reliability, and speed; improving the trade-off between inference-time and accuracy by striking an optimal balance between the performance of the two models, with the more accurate model being utilized when necessary and the faster model being employed for generating fast proposals. The proposed pipeline can be used as a guideline for developing real-time face recognition systems on embedded devices.
This work has been carried out within the scope of Digidow, the Christian Doppler Laboratory for Private Digital Authentication in the Physical World and has partially been supported by the LIT Secure and Correct Systems Lab. We gratefully acknowledge financial support by the Austrian Federal Ministry of Labour and Economy, the National Foundation for Research, Technology and Development, the Christian Doppler Research Association, 3 Banken IT GmbH, ekey biometric systems GmbH, Kepler Universitätsklinikum GmbH, NXP Semiconductors Austria GmbH & Co KG, Österreichische Staatsdruckerei GmbH, and the State of Upper Austria.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
A list with recent large-scale data breaches is visualized at https://informationisbeautiful.net/visualizations/worlds-biggest-data-breaches-hacks/.
- 2.
References
Bansal, A., Nanduri, A., Castillo, C.D., Ranjan, R., Chellappa, R.: UMDFaces: an annotated face dataset for training deep networks. In: IEEE International Joint Conference on Biometrics (IJCB), pp. 464–473. IEEE (2017). https://doi.org/10.1109/BTAS.2017.8272731
Cao, Q., Shen, L., Xie, W., Parkhi, O.M., Zisserman, A.: VGGFace2: a dataset for recognising faces across pose and age. In: 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), pp. 67–74. IEEE (2018). https://doi.org/10.1109/FG.2018.00020
Deng, J., Guo, J., Ververas, E., Kotsia, I., Zafeiriou, S.: RetinaFace: single-shot multi-level face localisation in the wild. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 5202–5211 (2020). https://doi.org/10.1109/CVPR42600.2020.00525
Deng, J., Guo, J., Xue, N., Zafeiriou, S.: ArcFace: additive angular margin loss for deep face recognition. In: IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4685–4694 (2019). https://doi.org/10.1109/CVPR.2019.00482
Department of Defense: Human engineering design data digest (2000). https://apps.dtic.mil/sti/pdfs/ADA467401.pdf. Accessed 3 Apr 2023
Europa.eu: Entry/exit system (EES) (2023). https://home-affairs.ec.europa.eu/policies/schengen-borders-and-visa/smart-borders/entry-exit-system_en. Accessed 3 Apr 2023
Farfade, S.S., Saberian, M.J., Li, L.J.: Multi-view face detection using deep convolutional neural networks. In: Proceedings of the 5th ACM on International Conference on Multimedia Retrieval, pp. 643–650 (2015). https://doi.org/10.48550/arXiv.1502.02766
Feng, Y., Yu, S., Peng, H., Li, Y.R., Zhang, J.: Detect faces efficiently: a survey and evaluations. IEEE Trans. Biometrics Behav. Identity Sci. 4(1), 1–18 (2022). https://doi.org/10.1109/TBIOM.2021.3120412
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016). https://doi.org/10.1109/CVPR.2016.90
Heisler, B.: Criterion.rs: statistics-driven benchmarking library for rust (2014). https://github.com/bheisler/criterion.rs
Kumar, A., Kumar, M., Kaur, A.: Face detection in still images under occlusion and non-uniform illumination. Multimedia Tools Appl. 80, 14565–14590 (2021). https://doi.org/10.1007/s11042-020-10457-9
Li, Z., Tang, X., Han, J., Liu, J., He, R.: PyramidBox++: high performance detector for finding tiny face. arXiv:1904.00386 (2019)
Lin, T.Y., Dollár, P., Girshick, R., He, K., Hariharan, B., Belongie, S.: Feature pyramid networks for object detection. In: IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 936–944 (2017). https://doi.org/10.1109/CVPR.2017.106
Linzaer: 1MB lightweight face detection model (2019). https://github.com/Linzaer/Ultra-Light-Fast-Generic-Face-Detector-1MB
Liu, C.: Multiple social credit systems in China. Econ. Soc. Eur. Electr. Newslett. 21(1), 22–32 (2019)
Mayrhofer, R., Roland, M., Höller, T.: Poster: Towards an architecture for private digital authentication in the physical world. In: Network and Distributed System Security Symposium (NDSS Symposium 2020), Posters (2020)
mos.ru: The Face Pay system for fare payment was launched at all metro stations (2023). https://www.mos.ru/news/item/97579073/. Accessed 2 Feb 2023
Parkhi, O.M., Vedaldi, A., Zisserman, A.: Deep face recognition. In: Proceedings of the British Machine Vision Conference (BMVC), pp. 41.1–41.12. BMVA Press (2015). https://doi.org/10.5244/C.29.41
Roland, M., Höller, T., Mayrhofer, R.: Digitale Identitäten in der physischen Welt: Eine Abwägung von Privatsphäreschutz und Praktikabilität. HMD Praxis der Wirtschaftsinformatik 60(2), 283–307 (2023). https://doi.org/10.1365/s40702-023-00949-1
uidai.gov.in: Unique Identification Authority of India (2023). https://uidai.gov.in/en/. Accessed 2 Feb 2023
Viola, P., Jones, M.: Rapid object detection using a boosted cascade of simple features. In: Proceedings of the 2001 IEEE Computer Society Conference on Computer Vision and Pattern Recognition. CVPR 2001, vol. 1, pp. I–I (2001). https://doi.org/10.1109/CVPR.2001.990517
Wolpert, D.H.: The lack of a priori distinctions between learning algorithms. Neural Comput. 8(7), 1341–1390 (1996). https://doi.org/10.1162/neco.1996.8.7.1341
Yang, S., Luo, P., Loy, C.C., Tang, X.: WIDER FACE: a face detection benchmark. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5525–5533 (2016). https://doi.org/10.1109/CVPR.2016.596
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Hofer, P., Roland, M., Schwarz, P., Mayrhofer, R. (2023). Face to Face with Efficiency: Real-Time Face Recognition Pipelines on Embedded Devices. In: Delir Haghighi, P., Khalil, I., Kotsis, G., ER, N.A.S. (eds) Advances in Mobile Computing and Multimedia Intelligence. MoMM 2023. Lecture Notes in Computer Science, vol 14417. Springer, Cham. https://doi.org/10.1007/978-3-031-48348-6_11
Download citation
DOI: https://doi.org/10.1007/978-3-031-48348-6_11
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-48347-9
Online ISBN: 978-3-031-48348-6
eBook Packages: Computer ScienceComputer Science (R0)