Skip to main content
SpringerLink
Log in

Hand Gesture Recognition Based Omnidirectional Wheelchair Control Using IMU and EMG Sensors

  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

This paper presents a hand gesture based control of an omnidirectional wheelchair using inertial measurement unit (IMU) and myoelectric units as wearable sensors. Seven common gestures are recognized and classified using shape based feature extraction and Dendogram Support Vector Machine (DSVM) classifier. The dynamic gestures are mapped to the omnidirectional motion commands to navigate the wheelchair. A single IMU is used to measure the wrist tilt angle and acceleration in three axis. EMG signals are extracted from two forearm muscles namely Extensor Carpi Radialis and Flexor Carpi Radialis and processed to provide Root Mean Square (RMS) signal. Initiation and termination of dynamic activities are based on autonomous identification of static to dynamic or dynamic to static transition by setting static thresholds on processed IMU and myoelectric sensor data. Classification involves recognizing the activity pattern based on periodic shape of trajectories of the triaxial wrist tilt angle and EMG-RMS from the two selected muscles. Second order Polynomial coefficients extracted from the sensor trajectory templates during specific dynamic activity cycles are used as features to classify dynamic activities. Classification algorithm and real time navigation of the wheelchair using the proposed algorithm has been tested by five healthy subjects. Classification accuracy of 94% was achieved by DSVM classifier on ‘k’ fold cross validation data of 5 users. Classification accuracy while operating the wheelchair was 90.5%.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Madarasz, R.L., Heiny, L.C., Cromp, R.F., Mazur, N.M.: The design of an autonomous vehicle for the disabled. IEEE J. Robot. Autom. 2(3), 117–126 (1986)

    Article  Google Scholar 

  2. Simpson, R.C.: Smart wheelchair: a literature review. J. Rehab. Res. Develop. 42(4), 423–436 (2005)

    Article  Google Scholar 

  3. Krishnan, R.H., Pugazhenthi, S.: Mobility assistive devices and self-transfer robotic systems for elderly, a review. Intell. Service Robot. 7, 37–49 (2014)

    Article  Google Scholar 

  4. Levine, S.P., Bell, D.A., Jaros, L.A., Simpson, R.C., Koren, Y., Borenstein, J.: The NavChair assistive wheelchair navigation system. IEEE Trans. Rehab. Eng. 7(4), 443–51 (1999)

    Article  Google Scholar 

  5. Borgolte, U., Hoyer, H., Buehler, C., Heck, H., Hoelper, R.: Architectural concepts of a semi-autonomous wheelchair. J. Intell. Robot. Syst. 22(3), 233–53 (1998)

    Article  Google Scholar 

  6. Prassler, E., Scholz, J., Fiorini, P.: A robotic wheelchair for crowded public environments. IEEE Robot. Autom. Mag. 8(1), 38–45 (2001)

    Article  Google Scholar 

  7. Simpson, R.C., LoPresti, E.F., Hayashi, S., Guo, S., Ding, D., Cooper, R.A.: Smart power assistance module for manual wheelchairs. technology and disability: research, design, practice and policy. In: 26th International Annual Conference on Assistive Technology for People with Disabilities (RESNA), 2003 Jun 19–23; Atlanta, GA. RESNA Press, Arlington (2003)

  8. Miller, D.P., Slack, M.G.: Design and testing of a low-cost robotic wheelchair prototype. Auton. Robot. 2 (1), 77–88 (1995)

    Article  Google Scholar 

  9. Fehr, L., Langbein, W.E., Skaar, S.B.: Adequacy of power wheelchair control interfaces for persons with severe disabilities: a clinical survey. J. Rehabil. Res. Dev. 37(3), 353–60 (2000)

    Google Scholar 

  10. Jia, P., Hu, H., Lu, T., Yuan, K.: Head gesture recognition for hand-free control of an intelligent wheelchair. Int. J. Ind. Robot. 34(1), 60–8 (2007)

    Article  Google Scholar 

  11. Nakanishi, S., Kuno, Y., Shimada, N., Shirai, Y.: Robotic wheelchair based on observations of both user and environment. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, Kyongju, Korea, pp. 912–7 (1999)

  12. Katevas, N.I., Sgouros, N.M., Tzafestas, S.G., Papakonstantinou, G., Beattie, P., Bishop, J.M.: The autonomous mobile robot SENARIO: a sensor-aided intelligent navigation system for powered wheelchairs. IEEE Robot. Autom. Mag. 4(4), 60–70 (1997)

    Article  Google Scholar 

  13. Barea, R., Boquete, L., Mazo, M., Lopes, E.: System for assisted mobility using eye movements based on electrooculography. IEEE Trans. Neural Syst. Rehabil. Eng. 10(4), 209–18 (2002)

    Article  Google Scholar 

  14. Tanaka, K., Matsunaga, K., Wang, H.O.: Electroencephalogram based control of an electric wheelchair. IEEE Trans. Robot. 21(4), 762–6 (2005)

    Article  Google Scholar 

  15. Oskoei, M.A., Hu, H.: Myoelectric control systems - a survey. Biomed. Signal Process. Control 2(4), 275–94 (2007)

    Article  Google Scholar 

  16. Ahsan, M.R., Ibrahimy, M.I., Khalifa, O.O.: Advances in electro- myogram signal classification to improve the quality of life for the disabled and aged people. J. Comput. Sci. 7(6), 706–715 (2010)

    Article  Google Scholar 

  17. Khezri, M., Jahed, M.: A novel approach to recognize hand movements via semg patterns. In: 29th Annual International Conference of the IEEE EMBS, pp. 4907–4910 (2007)

  18. Shuman, G.: Using forearm electromyograms to classify hand gestures. In: IEEE International Conference on Bioinformatics and Biomedicine, pp. 261–264 (2009)

  19. Saponas, T.S., Tan, D.S., Morris, D., Balakrishnan, R.: Demonstrating the feasibility of using forearm electromyography for muscle–computer interfaces. In: Proceedings of 26th SIGCHI Conference Human Factors Computing System, Florence, Italy, pp. 515–524 (2008)

  20. Wheeler, K.R., Chang, M.H., Knuth, K.H.: Gesture-based control and EMG decomposition. IEEE Trans. Syst., Man, Cybern. C, Appl. Rev. 36(4), 503–514 (2006)

    Article  Google Scholar 

  21. Sherrill, D.M., Bonato, P., De Luca, C.J.: A neural network approach to monitor motor activities. In: Proceedings of 2nd Joint EMBS/BMES Conference, Houston, TX, vol. 1, pp. 52–53 (2002)

  22. Zhang, X., Chen, X., Li, Y., Lantz, V., Wang, K., Yang, J.: A framework for hand gesture recognition based on accelerometer and EMG sensors. IEEE Trans. Syst. Man Cybern. Syst. Hum. 41(6), 1064–1076 (2011)

    Article  Google Scholar 

  23. Lajnefa, T., Chaibia, S., Rubyb, P., Aguerab, P.-E., Eichenlaubc, J.-B., Sameta, M., Kachouria, A., Jerbib, K.: Learning machines and sleeping brains: automatic sleep stage classification using decision-tree multi-class support vector machines. J. Neurosci. Methods 30(250), 94–105 (2015)

    Article  Google Scholar 

  24. Benabdeslem, K., Bennani, Y.: Dendogram-based SVM for multi-class classification. J. Comput. Inf. Technol. - CIT 14 4, 283–289 (2006). https://doi.org/10.2498/cit.2006.04.03

    Article  Google Scholar 

  25. Allen, F.R., Ambikairajah, E., Lovell, N.H., Celler, B.G.: Classification of a known sequence of motions and postures from accelerometry data using adapted Gaussian mixture models. Physiol. Meas. 27, 935–951 (2006)

    Article  Google Scholar 

  26. Ermes, M., Parkka, J., Mantyjarvi, J., Korhonen, I.: Detection of daily activities and sports with wearable sensors in controlled and uncontrolled conditions. IEEE Trans. Inf. Technol. Biomed. 12(1), 20–26 (2008)

    Article  Google Scholar 

  27. Olguin, D.O., Pentland, A.: Human activity recognition: accuracy across common locations for wearable sensors. In: 10th International Symposium of Wearable Computer (Student Colloquium), Montreux, Switzerland (2006)

  28. Watanabe, K., Shiraishi, Y., Tzafestas, S.G., Tang, J., Fukuda, T.: Feedback control of an omnidirectional autonomous platform for mobile service robots. J. Intell. Robot. Syst. 22(3), 315–330 (1998)

    Article  Google Scholar 

  29. Conceic ao, A.S., Moreira, A.P., Costa, P.J.: Model identification of a four wheeled omni-directional mobile robot. In: Controlo 2006, 7th Portuguese Conference on Automatic Control, Instituto Superior Tecnico, Lisboa, Portugal (2006)

  30. Lajnef, T.: Multiclass SVM classifier. http://www.mathworks.com/matlabcentral/fileexchange/48632-multiclass-svm-classifier. Accessed 12th August, 2016

Download references

Author information

Authors and Affiliations

  1. IIEST, Shibpur, India

    Ananda Sankar Kundu, Oishee Mazumder & Subhasis Bhaumik

  2. NIOH, Kolkata, India

    Prasanna Kumar Lenka

Authors
  1. Ananda Sankar Kundu
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Oishee Mazumder
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Prasanna Kumar Lenka
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Subhasis Bhaumik
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ananda Sankar Kundu.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kundu, A.S., Mazumder, O., Lenka, P.K. et al. Hand Gesture Recognition Based Omnidirectional Wheelchair Control Using IMU and EMG Sensors. J Intell Robot Syst 91, 529–541 (2018). https://doi.org/10.1007/s10846-017-0725-0

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10846-017-0725-0

Keywords

Search

Navigation