Skip to main content
SpringerLink
Log in

A face recognition application for Alzheimer’s patients using ESP32-CAM and Raspberry Pi

  • Research
  • Published:
Journal of Real-Time Image Processing Aims and scope Submit manuscript

Abstract

This paper proposes a real-time face recognition application to aid people living with Alzheimer’s in identifying the people around them. This is achieved by developing a portable system consisting of glasses with an ESP32-CAM and a single-board microcomputer (the Raspberry Pi). The proposed system operates automatically and does not require physical interaction with the user. It utilizes wireless technologies to capture real-time video frames of human faces and transmit them (via Wi-Fi) to the Raspberry Pi, which detects and recognizes the captured human face and sends voice-activated feedback to the user’s ears over Bluetooth to pronounce their name. Several incompatibility challenges are encountered and appropriately handled during the system’s development, integration, and testing processes. A fully functional prototype is developed and tested successfully. When compared to the state-of-the-art, the obtained results have demonstrated superior performance in terms of a training accuracy of 99.46% and a face recognition accuracy of 99.48%. The entire processing time from capturing the human face to generating the voice message is found to be about one second (730 ms on a laptop and 1109 ms on a Raspberry Pi). The developed technology is anticipated to improve the patient’s quality of life and reduce their dependence on others.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10

Similar content being viewed by others

Data availability

An existing image dataset was made available to this study.

References

  1. Lin, Z.H., Li, Y. Z.: Design and Implementation of Classroom Attendance System Based on Video Face Recognition. In: Proc. - 2019 Int. Conf. Intell. Transp. Big Data Smart City, ICITBS 2019, pp 385–388, (2019), doi: https://doi.org/10.1109/ICITBS.2019.00101

  2. Chuo, Y. H., Sheu, R. K., Chen, L. C.: Design and implementation of a cross-camera suspect tracking system. In: 2019 Int. Autom. Control Conf. CACS 2019, pp. 1–6, (2019)https://doi.org/10.1109/CACS47674.2019.9024367

  3. Teixeira, E.H., Mafra, S.B., Rodrigues, J.J.P.C., Da Silveira, W.A.A.N., Diallo, O.: A review and construction of a real-time facial recognition system. Inst. Nac. Telecomun. (2020). https://doi.org/10.5753/sbcup.2020.11225

    Article  Google Scholar 

  4. Almabdy, S., Elrefaei, L.: Deep convolutional neural network-based approaches for face recognition. Appl. Sci. (2019). https://doi.org/10.3390/app9204397

    Article  Google Scholar 

  5. Boufenar, C., Kerboua, A., Batouche, M.: Investigation on deep learning for off-line handwritten Arabic character recognition. Cogn. Syst. Res. 50, 180–195 (2018). https://doi.org/10.1016/j.cogsys.2017.11.002

    Article  Google Scholar 

  6. Ho, H.T., Chellappa, R.: Pose-invariant face recognition using Markov random fields. IEEE Trans. Image Process. 22(4), 1573–1584 (2013). https://doi.org/10.1109/TIP.2012.2233489

    Article  MathSciNet  MATH  Google Scholar 

  7. Phillips, P. J. et al.: Overview of the face recognition grand challenge. In: Proc. - 2005 IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognition, CVPR 2005, vol. I, pp. 947–954, (2005), https://doi.org/10.1109/CVPR.2005.268

  8. Hariri, W., Tabia, H., Farah, N., Benouareth, A., Declercq, D.: 3D face recognition using covariance based descriptors. Pattern Recognit. Lett. 78, 1–7 (2016). https://doi.org/10.1016/j.patrec.2016.03.028

    Article  Google Scholar 

  9. Martikainen, K., Said, K.: A facial recognition application for elderly care: caregiver verification and identification. KTH R. Inst. Technol. Sch. Electr. Eng. Comput. Sci., (2018), http://www.diva-portal.org/smash/record.jsf?pid=diva2%3A1252357&dswid=5861

  10. Bakhshi, Y., Kaur, S., Verma, P.: A study based on various face recognition algorithms. Int. J. Comput. Appl. 129(13), 16–20 (2015). https://doi.org/10.5120/ijca2015907066

    Article  Google Scholar 

  11. Sanchez-Moreno, A.S., Olivares-Mercado, J., Hernandez-Suarez, A., Toscano-Medina, K., Sanchez-Perez, G., Benitez-Garcia, G.: Efficient face recognition system for operating in unconstrained environments. J. Imaging (2021). https://doi.org/10.3390/jimaging7090161

    Article  Google Scholar 

  12. de Sousa-Britto-Neto, L., Maike, V.R.M.L., Koch, F.L., Baranauskas, M.C.C., Rocha, A.D.R., Goldenstein, S.K.: A wearable face recognition system built into a smartwatch and the blind and low vision users. Lect. Notes Bus. Inf. Process. 241(December), 515–528 (2015). https://doi.org/10.1007/978-3-319-29133-8_25

    Article  Google Scholar 

  13. Zhang, Y.: A computational model of quantitatively measuring the Alzheimer’s disease progression in face identification. Electron. Sci. Technol. Appl. 6(1), 29–33 (2019). https://doi.org/10.18686/esta.v6i1.93

    Article  MathSciNet  Google Scholar 

  14. Aljojo, N., et al.: Alzheimer assistant: a mobile application using machine learning. Rev. Română Inform. Autom. 30(4), 7–26 (2020). https://doi.org/10.33436/v30i4y202001

    Article  Google Scholar 

  15. Timeless, “Timeless,” (2019) https://kale-clavichord-7blm.squarespace.com/

  16. Salman, H.M., Rasheed, R.T.: Smart door for handicapped people via face recognition and voice command technique. Eng. Technol. J. 39(1B), 222–230 (2021). https://doi.org/10.30684/etj.v39i1b.1719

    Article  Google Scholar 

  17. Wazwaz, A. A., Herbawi, A. O., Teeti, M. J., Hmeed, S. Y.: Raspberry Pi and computers-based face detection and recognition system. In: 2018 4th Int. Conf. Comput. Technol. Appl. ICCTA 2018, pp. 171–174, (2018)https://doi.org/10.1109/CATA.2018.8398677

  18. Raju, K., Srinivasa-Rao, Y.: Real time implementation of face recognition system on Raspberry Pi. Int. J. Eng. Technol. 7(2), 85–89 (2018). https://doi.org/10.14419/ijet.v7i2.17.11564

    Article  Google Scholar 

  19. Lee, S. J., Jung, S. B., Kwon, J. W., Hong, S. H.: Face detection and recognition using PCA. In: IEEE Reg. 10 Annu. Int. Conf. Proceedings/TENCON, vol. 1, no. December, pp. 84–87, (1999) https://doi.org/10.1109/TENCON.1999.818355

  20. Umm-E-Laila, Khan, M. A., Shaikh, M. K., Bin Mazhar, S. A., Mehboob, K.: Comparative analysis for a real time face recognition system using raspberry Pi. In: 2017 IEEE International Conference on Smart Instrumentation, Measurement and Applications, ICSIMA 2017, vol. 2017-Novem, no. November, pp. 1–4. (2018) https://doi.org/10.1109/ICSIMA.2017.8311984

  21. Wankhede, V., Mule, G., Londhe, R., Tarale, A., Gome, S.: Criminal face recognition using Raspberry Pi. Int. Res. J. Innov. Eng. Technol. 3(12), 1–3 (2019)

    Google Scholar 

  22. Kaur, A., Jadli, A., Sadhu, A., Goyal, S., Mehra, A., Rahul.: Cloud based surveillance using ESP32 CAM. In: Int. Conf. Intell. Technol. Syst. Serv. Internet Everything, ITSS-IoE 2021, no. April 2022, (2021) https://doi.org/10.1109/ITSS-IoE53029.2021.9615334

  23. Dani, P., Adi, P., Wahyu, Y.: Performance evaluation of ESP32 Camera face recognition for various projects. ASCEE, Indones. 02(June), 1 (2021). https://doi.org/10.31763/iota.v2i1.512

    Article  Google Scholar 

  24. Allafi, I., Iqbal, T.: Design and implementation of a low cost web server using ESP32 for real-time photovoltaic system monitoring. In: 2017 IEEE Electr. Power Energy Conf. EPEC 2017, vol. 2017-Octob, no. May 2022, pp. 1–5, (2018) https://doi.org/10.1109/EPEC.2017.8286184

  25. Amato, G., Carrara, F., Falchi, F., Gennaro, C., Vairo, C.: Facial-based intrusion detection system with deep learning in embedded devices. In: ACM Int. Conf. Proceeding Ser., pp. 64–68, (2018) https://doi.org/10.1145/3290589.3290598

  26. Januzaj, Y., Luma, A., Januzaj, Y., Ramaj, V.: Real time access control based on face recognition. November, (2015) https://doi.org/10.15242/iae.iae0615004

  27. Syafeeza, A.R., Mohd-Fitri-Alif, M.K., Nursyifaa-Athirah, Y., Jaafar, A.S., Norihan, A.H., Saleha, M.S.: IoT based facial recognition door access control home security system using raspberry pi. Int. J. Power Electron. Drive Syst. 11(1), 417–424 (2020). https://doi.org/10.11591/ijpeds.v11.i1.pp417-424

    Article  Google Scholar 

  28. Chao, W.-L.: Face recognition. GICE, National Taiwan University, Available online: https://www.orcam.com/en/myeye2/ accessed on 21 May 2022

  29. Yang, S. Luo, P., Loy, C. C., Tang, T.: WIDER FACE: a face detection benchmark. In: Proc. IEEE Comput. Soc. Conf. Comput. Vis. Pattern Recognit., vol. 2016-Decem, pp. 5525–5533, (2016) https://doi.org/10.1109/CVPR.2016.596

  30. Scherhag, U., Rathgeb, C., Merkle, J., Busch, C.: Deep face representations for differential morphing attack detection. IEEE Trans. Inf. Forensics Secur. 15, 3625–3639 (2020). https://doi.org/10.1109/TIFS.2020.2994750

    Article  Google Scholar 

  31. Khan, K., Khan, R.U., Ali, J., Uddin, I., Khan, S., Roh, B.H.: Race classification using deep learning. Comput. Mater. Contin. 68(3), 3483–3498 (2021). https://doi.org/10.32604/cmc.2021.016535

    Article  Google Scholar 

  32. N. I. o. S. a. T. NIST.: Face recognition technology (FERET). https://www.nist.gov/programs-projects/face-recognition-technology-feret. (2021)

  33. Dwivedi, H., Android Instructor.: Comparing MobileNet models in TensorFlow. (2019) https://www.kdnuggets.com/2019/03/comparing-mobilenet-models-tensorflow.html accessed on 8 Mar 2022

  34. Isuyama, V. K., Albertini, B. D. C.: Comparison of convolutional neural network models for mobile devices. Esc. Politecnica Univ. Sao Paulo (USP), Brazil, pp. 73–83, (2021) https://doi.org/10.5753/wperformance.2021.15724.

  35. Wang, W., Li, Y., Zou, T., Wang, X., You, J., Luo, Y.: A novel image classification approach via dense-Mobilenet models. Mob. Inf. Syst. (2020). https://doi.org/10.1155/2020/7602384

    Article  Google Scholar 

  36. Kadhim, T.A., Smaoui Zghal, N., Hariri, W., Ben Aissa, D.: Face recognition in multiple variations using deep learning and convolutional neural networks. In: 9th Int. Conf. Sci. Electron. Technol. Inf. Telecommun. (SETIT’22), 2022., no. 1, (2022)

  37. Khan, S., Rahmani, H., Shah, S.A.A., Bennamoun, M.: A guide to convolutional neural networks for computer vision. Synth. Lect. Comput. Vis. 8(1), 1–207 (2018). https://doi.org/10.2200/s00822ed1v01y201712cov015

    Article  Google Scholar 

  38. Liu, Y., Zhai, G., Zhao, D., Liu, X.: Frame rate and perceptual quality for HD video. In: Springer Int. Publ. Switz., vol. 9315, (2015) https://doi.org/10.1007/978-3-319-24078-7

  39. Kiran, T.T.J.: Computer vision accuracy analysis with deep learning model using TensorFlow. Int. J. Innov. Res. Comput. Sci. Technol. 8(4), 319–325 (2020). https://doi.org/10.2139/ssrn.3673214

    Article  Google Scholar 

  40. Maruseac, M.: Support for 32 bits architecture. (2019) https://github.com/tensorflow/tensorflow/issues/32315

  41. Adi, P. D. P., Kitagawa, A., Akita, J.: Finger robotic control use M5Stack board and MQTT protocol based. In: 7th Int. Conf. Inf. Technol. Comput. Electr. Eng. ICITACEE 2020—Proc., no. October, pp. 1–6, (2020) https://doi.org/10.1109/ICITACEE50144.2020.9239170

  42. Ahmed, H. M., Rasheed, R. T.: A Raspberry Pi real-time identification system on face recognition. In: Proc. 2020 1st Inf. Technol. to Enhanc. E-Learning other Appl. Conf. IT-ELA 2020, pp. 89–93, (2020) https://doi.org/10.1109/IT-ELA50150.2020.9253107

  43. Nikisins, O., Fuksis, R., Kadikis, A., Greitans, M.: Face recognition system on raspberry Pi. In: 2015 5th Int. Work. Comput. Sci. Eng. Inf. Process. Control Eng. WCSE 2015-IPCE, no. April, (2015) https://doi.org/10.18178/wcse.2015.04.054

  44. Suchitra, Suja, P., Tripathi, S.: Real-time emotion recognition from facial images using Raspberry Pi II. In: 3rd Int. Conf. Signal Process. Integr. Networks, SPIN 2016, pp. 666–670, (2016) https://doi.org/10.1109/SPIN.2016.7566780

  45. Lu, J., Fu, X., Zhang, T.: A smart system for face detection with spatial correlation improvement in IoT environment. In: 2017 IEEE SmartWorld Ubiquitous Intell. Comput. Adv. Trust. Comput. Scalable Comput. Commun. Cloud Big Data Comput. Internet People Smart City Innov. SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI 2017, pp. 1–4, (2018) https://doi.org/10.1109/UIC-ATC.2017.8397550

  46. Gsponer, D.: IoT: building a Raspberry Pi security system with facial recognition. Haaga-Helia (2018)

  47. Kak, S. F., Mustafa, F. M.: Smart home management system based on face recognition index in real-time. In: 2019 Int. Conf. Adv. Sci. Eng. ICOASE 2019, pp. 40–45, (2019) https://doi.org/10.1109/ICOASE.2019.8723673

  48. Munir, A., Kashif Ehsan, S., Mohsin Raza, S. M., Mudassir, M.: Face and speech recognition based smart home. In: 2019 Int. Conf. Eng. Emerg. Technol. ICEET 2019, pp. 1–5, (2019) https://doi.org/10.1109/CEET1.2019.8711849

  49. Saputra, R., Surantha, N.: Smart and real-time door lock system for an elderly user based on face recognition. Bull. Electr. Eng. Inform. 10(3), 1345–1355 (2021). https://doi.org/10.11591/eei.v10i3.2955

    Article  Google Scholar 

  50. Orna, G., Benitez, D. S., Perez, N.: A low-cost embedded facial recognition system for door access control using deep learning. In: 2020 IEEE Andescon, Andescon 2020, pp. 0–5, (2020) https://doi.org/10.1109/ANDESCON50619.2020.9271984

  51. Gunawan, T.S., Gani, M.H.H., Rahman, F.D.A., Kartiwi, M.: Development of face recognition on raspberry pi for security enhancement of smart home system. Indones. J. Electr. Eng. Inform. 5(4), 317–325 (2017). https://doi.org/10.11591/ijeei.v5i4.361

    Article  Google Scholar 

  52. Vamsi, T.K., Sai, K.C., Vijayalakshmi, M.: Face recognition based door unlocking system using Raspberry Pi Thulluri. Int. J. Adv. Res. Ideas Innov. Technol. 5(2), 1320–1324 (2019)

    Google Scholar 

  53. Hasban, A. S. et al.: Face recognition for Student Attendance using Raspberry Pi. In: APACE 2019 - 2019 IEEE Asia–Pacific Conf. Appl. Electromagn. Proc., no. November, pp. 1–5, (2019) https://doi.org/10.1109/APACE47377.2019.9020758

  54. Nagpal, G. S., Singh, G., Singh, J., Yadav, N.: Facial detection and recognition using OpenCV on Raspberry Pi Zero. In: Proc. - IEEE 2018 Int. Conf. Adv. Comput. Commun. Control Networking, ICACCCN 2018, pp. 945–950, (2018) https://doi.org/10.1109/ICACCCN.2018.8748389

  55. Singh, S., Ramya, R., Sushma, V., Roshini, S., Pavithra, R.: Facial recognition using machine learning algorithms on Raspberry Pi. In: 4th Int. Conf. Electr. Electron. Commun. Comput. Technol. Optim. Tech. ICEECCOT 2019, pp. 197–202, (2019) https://doi.org/10.1109/ICEECCOT46775.2019.9114716

  56. Nadafa, R.A., Hatturea, S.M., Bonala, V.M., Naikb, S.P.: Home security against human intrusion using Raspberry Pi. Procedia Comput. Sci. 167, 1811–1820 (2020). https://doi.org/10.1016/j.procs.2020.03.200

    Article  Google Scholar 

  57. Rok Novosel, B. M., Ziga Emersic, P. P., Struc V.: Face recognition with Raspberry Pi for IoT environments. In: ERK Portorož, no. September, pp. 477–480, (2017) https://lmi.fe.uni-lj.si/wp-content/uploads/2019/08/novoselface_recognition.pdf

  58. Parthornratt, T., Burapanonte, N., Gunjarueg, W.: People identification and counting system using Raspberry pi. In: 2016 Int. Conf. Electron. Information, Commun., pp. 1–5, (2016)

  59. Preetha, J., Manirathnam, M., Chaitanya, A., Raj, R.P.: Raspberry Pi based face recognition system. Int. J. Eng. Res. Technol. 8(08), 1–4 (2020)

    Google Scholar 

Download references

Acknowledgements

The authors would like to express their gratitude to Springer Nature for their interest in our study and for the meticulous review process. We also wish to extend our gratitude and appreciation to the peer reviewers for their time and efforts in reading our article and offering helpful and constructive feedback.

Funding

There is no external funding.

Author information

Authors and Affiliations

  1. Microwave Electronics Research Laboratory, Faculty of Sciences of Tunis, Tunis El-Manar University, Tunis, Tunisia

    Thair A. Kadhim & Dalenda Ben Aissa

  2. Labged Laboratory, Computer Science Department, Badji Mokhtar Annaba University, Annaba, Algeria

    Walid Hariri

  3. Control and Energy Management Laboratory (CEM Lab), ENIS Sfax, Tunis, Tunisia

    Nadia Smaoui Zghal

Authors
  1. Thair A. Kadhim
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Walid Hariri
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Nadia Smaoui Zghal
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Dalenda Ben Aissa
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

The authors certify that they have participated sufficiently in the work to take public responsibility for the content, including participation in the concept, design, analysis, writing, or revision of the manuscript. Furthermore, each author certifies that this material or similar material has not been and will not be submitted to or published in any other publication before its appearance in the Journal of Real-Time Image Processing. Authorship contributions The specific contributions made by each author are in the three categories below. Category 1 Conception and design of study: TK, WH, NS, DBA; Design a prototype: TK, NS; Devices integration: TK, DBA. Category 2 Drafting the manuscript: TK, NS; revising the manuscript critically for important intellectual content: TK, WH. Category 3 The authors have given their permission for the manuscript to be published in its current form: TK, WH, NS, DBA.

Corresponding author

Correspondence to Thair A. Kadhim.

Ethics declarations

Conflict of interest

There is no conflict of interest.

Research involving human participants and/or animals

This study used existing image datasets with a written permission from the author. The datasets are processed lawfully, fairly and in a transparent manner in accordance with the ethical guidelines at Tunis El-Manar University.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kadhim, T.A., Hariri, W., Smaoui Zghal, N. et al. A face recognition application for Alzheimer’s patients using ESP32-CAM and Raspberry Pi. J Real-Time Image Proc 20, 100 (2023). https://doi.org/10.1007/s11554-023-01357-w

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1007/s11554-023-01357-w

Keywords

Search

Navigation