Skip to main content
SpringerLink
Log in

An Intelligent Human Activity Recognizer for Visually Impaired People Using VGG-SVM Model

  • Conference paper
  • First Online:
Advances in Computational Intelligence (MICAI 2022)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 13613))

Included in the following conference series:

Abstract

With the advancement of artificial intelligence and computer vision, assistive devices for visually impaired people is an active research area for the last decade. People with visual disabilities face challenges in the detection and recognition of various human actions which leads to a lack of confidence and constant dependency while performing daily routine activities. The objective of this research is to recognize and classify different human actions based on the position and posture of their body. In this research, captured video is processed for human pose estimation by extraction of 2D body skeleton using OpenPose method. The extracted body points further to the process for feature extraction using pre-trained VGG-19 CNN which classifies the human actions using an SVM classifier. Furthermore, the proposed method integrates with a voice-enabled feature to deliver instructions on classified human activities. The proposed deep learning method is to train and test using three different dataset (a) Weizmann dataset for gesture based human actions, (b) Kinetics dataset for interaction based human actions and (c) CK+ dataset for behaviour based human actions. The accuracy for classification of different human actions reached to 93.08%, 95.03%, and 93.12% and the F1 score reached 93.59%, 95.19% and 93.66%, respectively. These results indicate the significance of proposed method for the assistance of visually impaired people.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 64.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 84.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Fast facts of common eye disorders, June 2020. https://www.cdc.gov/visionhealth/basics/ced/fastfacts.htm

  2. Vision impairment and blindness (2022). https://www.who.int/news-room/fact-sheets/detail/blindness-and-visual-impairment

  3. Aparna, R., Chitralekha, C., Chaudhari, S.: Comparative study of CNN, VGG16 with LSTM and VGG16 with bidirectional LSTM using kitchen activity dataset. In: 2021 Fifth International Conference on I-SMAC (IoT in Social, Mobile, Analytics and Cloud)(I-SMAC), pp. 836–843. IEEE (2021)

    Google Scholar 

  4. Cao, Z., Simon, T., Wei, S.E., Sheikh, Y.: Realtime multi-person 2D pose estimation using part affinity fields. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 7291–7299 (2017)

    Google Scholar 

  5. Dai, C., Liu, X., Lai, J.: Human action recognition using two-stream attention based LSTM networks. Appl. Soft Comput. 86, 105820 (2020)

    Article  Google Scholar 

  6. Debnath, T., Reza, M., Rahman, A., Beheshti, A., Band, S.S., Alinejad-Rokny, H., et al.: Four-layer convnet to facial emotion recognition with minimal epochs and the significance of data diversity. Sci. Rep. 12(1), 1–18 (2022)

    Article  Google Scholar 

  7. Deep, S., Zheng, X.: Leveraging CNN and transfer learning for vision-based human activity recognition. In: 2019 29th International Telecommunication Networks and Applications Conference (ITNAC), pp. 1–4. IEEE (2019)

    Google Scholar 

  8. Dirgová Luptáková, I., Kubovčík, M., Pospíchal, J.: Wearable sensor-based human activity recognition with transformer model. Sensors 22(5), 1911 (2022)

    Article  Google Scholar 

  9. Dua, N., Singh, S.N., Semwal, V.B.: Multi-input CNN-GRU based human activity recognition using wearable sensors. Computing 103(7), 1461–1478 (2021)

    Article  Google Scholar 

  10. Ehatisham-Ul-Haq, M., Javed, A., Azam, M.A., Malik, H.M., Irtaza, A., Lee, I.H., Mahmood, M.T.: Robust human activity recognition using multimodal feature-level fusion. IEEE Access 7, 60736–60751 (2019)

    Article  Google Scholar 

  11. Gamache, S., Routhier, F., Morales, E., Vandersmissen, M.H., Boucher, N.: Mapping review of accessible pedestrian infrastructures for individuals with physical disabilities. Disabil. Rehabil. Assistive Technol. 14(4), 410–422 (2019)

    Article  Google Scholar 

  12. Gorelick, L., Blank, M., Shechtman, E., Irani, M., Basri, R.: Actions as space-time shapes. Trans. Pattern Anal. Mach. Intell. 29(12), 2247–2253 (2007)

    Article  Google Scholar 

  13. Hao, Z., Zhang, D., Dang, X., Liu, G., Bai, Y.: Wi-CAS: a contactless method for continuous indoor human activity sensing using Wi-Fi devices. Sensors 21(24), 8404 (2021)

    Article  Google Scholar 

  14. Jalal, A., Kamal, S., Kim, D.: A depth video-based human detection and activity recognition using multi-features and embedded hidden markov models for health care monitoring systems (2017)

    Google Scholar 

  15. Kay, W., et al.: The kinetics human action video dataset. arXiv preprint arXiv:1705.06950 (2017)

  16. Khan, M.A., Zhang, Y.D., Khan, S.A., Attique, M., Rehman, A., Seo, S.: A resource conscious human action recognition framework using 26-layered deep convolutional neural network. Multimedia Tools Appl. 80(28), 35827–35849 (2021)

    Article  Google Scholar 

  17. Kim, H., Lee, S., Jung, H.: Human activity recognition by using convolutional neural network. Int. J. Electr. Comput. Eng. 9(6), 5270 (2019)

    Google Scholar 

  18. Lucey, P., Cohn, J.F., Kanade, T., Saragih, J., Ambadar, Z., Matthews, I.: The extended Cohn-Kanade dataset (CK+): A complete dataset for action unit and emotion-specified expression. In: 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition-Workshops, pp. 94–101. IEEE (2010)

    Google Scholar 

  19. Mmereki, W., Jamisola, R.S., Mpoeleng, D., Petso, T.: YOLOv3-based human activity recognition as viewed from a moving high-altitude aerial camera. In: 2021 7th International Conference on Automation, Robotics and Applications (ICARA), pp. 241–246. IEEE (2021)

    Google Scholar 

  20. Muhammad, K., et al.: Human action recognition using attention based LSTM network with dilated CNN features. Future Gener. Comput. Syst. 125, 820–830 (2021)

    Article  Google Scholar 

  21. Mutegeki, R., Han, D.S.: A CNN-LSTM approach to human activity recognition. In: 2020 International Conference on Artificial Intelligence in Information and Communication (ICAIIC), pp. 362–366. IEEE (2020)

    Google Scholar 

  22. Nadeem, A., Jalal, A., Kim, K.: Human actions tracking and recognition based on body parts detection via artificial neural network. In: 2020 3rd International Conference on Advancements in Computational Sciences (ICACS), pp. 1–6. IEEE (2020)

    Google Scholar 

  23. Nadeem, A., Jalal, A., Kim, K.: Automatic human posture estimation for sport activity recognition with robust body parts detection and entropy Markov model. Multimedia Tools Appl. 80(14), 21465–21498 (2021)

    Article  Google Scholar 

  24. Naik, A.J., Gopalakrishna, M.: Deep-violence: individual person violent activity detection in video. Multimedia Tools Appl. 80(12), 18365–18380 (2021)

    Article  Google Scholar 

  25. Paré, S., Bleau, M., Djerourou, I., Malotaux, V., Kupers, R., Ptito, M.: Spatial navigation with horizontally spatialized sounds in early and late blind individuals. PloS ONE 16(2), e0247448 (2021)

    Article  Google Scholar 

  26. Pham, C., et al.: SensCapsNet: deep neural network for non-obtrusive sensing based human activity recognition. IEEE Access 8, 86934–86946 (2020)

    Article  Google Scholar 

  27. Rodríguez-Moreno, I., Martínez-Otzeta, J.M., Sierra, B., Rodriguez, I., Jauregi, E.: Video activity recognition: state-of-the-art. Sensors 19(14), 3160 (2019)

    Article  Google Scholar 

  28. Ronao, C.A., Cho, S.B.: Human activity recognition with smartphone sensors using deep learning neural networks. Expert Syst. Appl. 59, 235–244 (2016)

    Article  Google Scholar 

  29. Sun, H., et al.: Color correction and repair of haze images under hue-saturation-intensity color space and machine learning (2021)

    Google Scholar 

  30. Wan, S., Qi, L., Xu, X., Tong, C., Gu, Z.: Deep learning models for real-time human activity recognition with smartphones. Mob. Netw. Appl. 25(2), 743–755 (2020)

    Article  Google Scholar 

  31. Wu, D., Lv, S., Jiang, M., Song, H.: Using channel pruning-based YOLO v4 deep learning algorithm for the real-time and accurate detection of apple flowers in natural environments. Comput. Electron. Agric. 178, 105742 (2020)

    Article  Google Scholar 

Download references

Acknowledgements

This research with project id 16499 is funded by the National Research Program for Universities (NRPU), Higher Education Commission (HEC), Islamabad, Pakistan.

Author information

Authors and Affiliations

  1. Department of Computer Science, UET Lahore, Lahore, Pakistan

    Rabeeya Saleem, Tauqir Ahmad & Muhammad Aslam

  2. Department of CS, CINVESTAV, Mexico City, Mexico

    A. M. Martinez-Enriquez

Authors
  1. Rabeeya Saleem
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tauqir Ahmad
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Muhammad Aslam
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. A. M. Martinez-Enriquez
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rabeeya Saleem .

Editor information

Editors and Affiliations

  1. Instituto Politécnico Nacional, Mexico, Mexico

    Obdulia Pichardo Lagunas

  2. Centro de Investigación Científica y de Educación Superior de Ensenada, Ensenada, Baja California, Mexico

    Juan Martínez-Miranda

  3. Instituto Politécnico Nacional, Mexico, Mexico

    Bella Martínez Seis

Rights and permissions

Reprints and permissions

Copyright information

© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Saleem, R., Ahmad, T., Aslam, M., Martinez-Enriquez, A.M. (2022). An Intelligent Human Activity Recognizer for Visually Impaired People Using VGG-SVM Model. In: Pichardo Lagunas, O., Martínez-Miranda, J., Martínez Seis, B. (eds) Advances in Computational Intelligence. MICAI 2022. Lecture Notes in Computer Science(), vol 13613. Springer, Cham. https://doi.org/10.1007/978-3-031-19496-2_28

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-19496-2_28

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-19495-5

  • Online ISBN: 978-3-031-19496-2

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation