Skip to main content

Advertisement

SpringerLink
Log in

High performance computation of human computer interface for neurodegenerative individuals using eye movements and deep learning technique

  • Published:
The Journal of Supercomputing Aims and scope Submit manuscript

Abstract

Disabilities due to neurodegenerative disease are rapidly increasing in number. The need for rehabilitative devices to achieve a normal and comfortable life in the absence of biochannels has also increased. The activities of biochannels can be easily replaced by implementing rehabilitative devices. The electrooculography (EOG)-based human–computer interface (HCI) is one of the most important techniques for enabling disabled persons to enjoy a normal life. The technique of measuring the cornea-retina potential difference is called EOG. The technology for converting thoughts to different control patterns to activate external devices is called the HCI. In this paper, we carried out a study on ten subjects aged 20–30 years using a five-electrode signal acquisition system (AD T26). The subject performances were experimentally verified by implementing periodogram features with a feedforward neural network trained on a nature-inspired algorithm. The analysis was conducted offline and online to evaluate the achievement of the developed HCI. The study showed an average classification accuracy of 93.93% for four tasks, with 95% accuracy for the offline mode and 90.12% for the online mode. The participating subjects drove the mobile robot in all directions frequently and quickly with a recognition accuracy of 98.12%. The study confirmed that the four tasks (related to driving the external device) performed by the subjects were convenient to perform in real time.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14

Similar content being viewed by others

References

  1. Yang JJ, Gang GW, Kim TS (2018) Development of EOG-based human computer interface (HCI) system using piecewise linear approximation (PLA) and support vector regression (SVR). Electronics 7(38):1–18

    Google Scholar 

  2. Xiao J, Qu J, Li Y (2019) An electrooculogram-based interaction method and its music-on-demand application in a virtual reality environment. IEEE Access 7:22059–22070. https://doi.org/10.1109/ACCESS.2019.2898324

    Article  Google Scholar 

  3. Usakli AB, Gurkan S (2010) Design of a novel efficient human–computer interface: an electrooculagram based virtual keyboard. IEEE Trans Instrum Meas 59(8):2099–2108. https://doi.org/10.1109/TIM.2009.2030923

    Article  Google Scholar 

  4. Usakli AB, Gurkan S (2009) A novel electrooculogram-based human computer interface and its application as a virtual keyboard. In: IEEE Conference on National Biomedical Engineering Meeting. https://doi.org/10.1109/BIYOMUT.2009.5130315

  5. Hossain Z, Shuvo MH, Sarker P (2017) Hardware and software implementation of real time electrooculogram (EOG) acquisition system to control computer cursor with eyeball movement. In: IEEE-International Conference on Advances in Electrical Engineering (ICAEE), pp 132–137. https://doi.org/10.1109/ICAEE.2017.8255341

  6. Talukder M, Aktar A, Neelima A, Rahman A (2018) EOG based home automation system by cursor movement using a graphical user interface (GUI). In: IEEE International WIE Conference on Electrical and Computer Engineering (WIECON-ECE). https://doi.org/10.1109/WIECON-ECE.2018.8783025

  7. Zhang R, He S, Yang X, Wang X, Li K, Huang Q, Gu Z, Yu Z, Zhang X, Tang D, Li Y (2019) An EOG-based human machine interface to control a smart home environment for patients with severe spinal cord injuries. IEEE Trans Biomed Eng 66(1):89–100. https://doi.org/10.1109/TBME.2018.2834555

    Article  Google Scholar 

  8. Fang F, Shinozaki T (2018) Electrooculography-based continuous eyewriting recognition system for efficient assistive communication systems. PLoS ONE 13(2):1–20. https://doi.org/10.1371/journal.pone.0192684

    Article  Google Scholar 

  9. Barea R, Vázquez LB, Guillén EL (2018) EOG-based wheelchair control. Smart Wheelchairs Brain-Comput Interfaces, Mob Assist Technol 381–403. https://doi.org/10.1016/B978-0-12-812892-3.00016-9

  10. Huang Q, Chen Y, Zhang Z, He S, Zhang R, Liu J, Zhang Y, Shao M, Li Y (2019) An EOG-based wheelchair robotic arm system for assisting patients with severe spinal cord injuries. J Neural Eng 6:1–11. https://doi.org/10.1088/1741-2552/aafc88

    Article  Google Scholar 

  11. Lin CT, Wu SL, Jiang WL, Liang JW, Chen SA (2013) A wireless Electrooculography-based human-computer interface for baseball game. In: IEEE-International Conference on Information, Communications and Signal Processing. https://doi.org/10.1109/ICICS.2013.6782884

  12. Li L, Wu X (2011) Design and implementation of multimedia control system based on bluetooth and electrooculogram (EOG). In: International Conference on Bioinformatics and Biomedical Engineering. https://doi.org/10.1109/icbbe.2011.5779981

  13. Ramakrishnan J, Mavaluru D, Ramkumar S, Alqahtani AS, Mubarakali A, Retnadhas M (2020) Brain–computer interface for amyotrophic lateral sclerosis patients using deep learning network. Neural Comput Appl. https://doi.org/10.1007/s00521-020-05026-y

    Article  Google Scholar 

  14. Tang W, Wang A, Ramkumar S, Radeep Krishna R (2020) Signal identification system for developing rehabilitative device using deep learning algorithms. Artif Intell Med 102:101755. https://doi.org/10.1016/j.artmed.2019.101755

    Article  Google Scholar 

  15. Teng G, He Y, Zhao H, Liu D, Xiao J, Ramkumar S (2020) Design and development of human computer interface using electrooculogram with deep learning. Artif Intell Med 102:101765. https://doi.org/10.1016/j.artmed.2019.101765

    Article  Google Scholar 

  16. Ramkumar S, Sathesh Kumar K, Maheswari K, Packia Amutha Priya P, Emayavaramban G, Macklin Abraham Navamani J (2020) Offline study for implementing human computer interface for elderly paralysed patients using electrooculography and neural networks. Int J Intell Enterpr 7(1/2/3):306–321. https://doi.org/10.1504/IJIE.2020.104658

    Article  Google Scholar 

  17. Saravanakumar D, Vishnupriya R, Reddy MR (2019) A novel EOG based synchronous and asynchronous visual keyboard system. In: IEEE EMBS International Conference on Biomedical and Health Informatics (BHI). https://doi.org/10.1109/BHI.2019.8834621

  18. Jialu G, Ramkumar S, Emayavaramban G, Thilagaraj M, Muneeswaran V, Rajasekaran MP, Hussein AF (2018) Offline analysis for designing electrooculogram based human computer interface control for paralyzed patients. IEEE Access 6:79151–79161. https://doi.org/10.1109/ACCESS.2018.2884411

    Article  Google Scholar 

  19. Rakshit A, Banerjee A, Tibarewala DN (2016) Electro-oculogram based digit recognition to design assitive communication system for speech disabled patients. In: IEEE International Conference on Microelectronics, Computing and Communications (MicroCom). https://doi.org/10.1109/MicroCom.2016.7522560

  20. Lee KR, Chang WD, Kim S, Im CH (2017) Real-time “eye-writing” recognition using electrooculogram. IEEE Trans Neural Syst Rehab Eng 25(1):37–48. https://doi.org/10.1109/TNSRE.2016.2542524

    Article  Google Scholar 

  21. Kumar D, Sharma A (2016) Electrooculogram-based virtual reality game control using blink detection and gaze calibration. In: International Conference on Advances in Computing, Communications and Informatics (ICACCI). https://doi.org/10.1109/ICACCI.2016.7732407

  22. Teja SSS, Embrandiri SS, Chandrachoodan N, Reddy MR (2015) EOG based virtual keyboard. In: IEEE-Annual Northeast Biomedical Engineering Conference (NEBEC). https://doi.org/10.1109/NEBEC.2015.7117201

  23. Tangsuksant W, Aekmunkhongpaisal C, Cambua P, Charoenpong T, Chanwimalueang T (2012) Directional eye movement detection system for virtual keyboard controller. Biomed Eng Int Conf. https://doi.org/10.1109/BMEiCon.2012.6465432

    Article  Google Scholar 

  24. Deng LY, Hsu CL, Lin TC, Tuan JS, Chang SM (2010) EOG-based human–computer interface system development. Expert Syst Appl 37(4):3337–3343. https://doi.org/10.1016/j.eswa.2009.10.017

    Article  Google Scholar 

  25. Wei L, Hu H, Yuan K (2009) Use of forehead bio-signals for controlling an Intelligent Wheelchair. In: IEEE International Conference on Robotics and Biomimetics, pp 108–113. https://doi.org/10.1109/ROBIO.2009.4912988

  26. Akin M, Kemal Kiymik M (2000) Application of periodogram and AR spectral analysis to EEG signals. J Med Syst 24:247–256. https://doi.org/10.1023/A:1005553931564

    Article  Google Scholar 

  27. Martinez-Murcia FJ, Ortiz A, Morales-Ortega R, López PJ, Luque JL, Castillo-Barnes D, Segovia F, Illan IA, Ortega J, Ramirez J, Gorriz JM (2019) Periodogram connectivity of EEG signals for the detection of dyslexia. In: Understanding the Brain Function and Emotions, Lecture Notes in Computer Science, vol 11486. https://doi.org/10.1007/978-3-030-19591-5_36

  28. Unde SA, Shriram R (2014) Coherence analysis of EEG signal using power spectral density. In: International Conference on Communication Systems and Network Technologies. https://doi.org/10.1109/CSNT.2014.181

  29. Tian T, Liu C, Guo Q, Yuan Y, Li W, Yan Q (2018) An improved ant lion optimization algorithm and its application in hydraulic turbine governing system parameter identification. Energies 11:95. https://doi.org/10.3390/en11010095

    Article  Google Scholar 

  30. Hu H, Li Y, Bai Y, Zhang J, Liu M (2019) The improved antlion optimizer and artificial neural network for Chinese influenza prediction. Complexity 2019:1–12. https://doi.org/10.1155/2019/1480392

    Article  Google Scholar 

  31. Mirjalili S (2015) The ant lion optimizer. Adv Eng Softw 80:80–98. https://doi.org/10.1016/j.advengsoft.2015.01.010

    Article  Google Scholar 

  32. Hema, C. R, Paulraj, M. P. & Ramkumar S, Classification of eye movements using electrooculography and neural networks. International Journal of Human Computer Interaction, Vol. 5 (3), pp. 51–63, 2014.

  33. Hema CR, Ramkumar S, Paulraj MP (2014) Identifying eye movements using neural networks for human computer interaction. Int J Comput Appl 105(8):18–26

    Google Scholar 

  34. Ramkumar S, Sathesh Kumar K, Emayavaramban G (2016) EOG signal classification using neural network for human computer interaction. Int J Control Theory Appl 9(24):223–231

    Google Scholar 

  35. Ramkumar S, Sathesh Kumar K, Emayavaramban G (2017) Nine states HCI using electrooculogram and neural networks. Int J Eng Technol 8(6):3056–3064

    Google Scholar 

  36. Ramkumar S, Sathesh Kumar K, Emayavaramban G (2017) A feasibility study on eye movements using electrooculogram based HCI. In: IEEE-International Conference on Intelligent Sustainable Systems, pp 384–388

  37. Xiaoxiao X, Bin L, Ramkumar S, Saravanan S, Sundar Prakash Balaji M, Dhanasekaran S, Thimmiaraja J (2020) Electroencephalogram based communication system for locked in state person using mentally spelled tasks with optimized network model. Artif Intell Med 102:101766. https://doi.org/10.1016/j.artmed.2019.101766

    Article  Google Scholar 

  38. LiKai S, Ramkumar J, Thimmiaraja S (2020) Diwakaran, Optimized artificial neural network based performance analysis of wheelchair movement for ALS patients. Artif Intell Med 102:101754. https://doi.org/10.1016/j.artmed.2019.101754

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. College of Computer Science and Information Technology, Jazan University, Jazan, Saudi Arabia

    Jayabrabu Ramakrishnan

  2. Faculty of Computer Science, National Defence Academy, Khadakwasla, Pune, India

    Rajesh Doss

  3. Electrical and Computer Engineering, Faculty of Engineering, King Abdulaziz University, Jeddah, Saudi Arabia

    Thangam Palaniswamy

  4. College of Computing and Informatics, Saudi Electronic University, Riyadh, Saudi Arabia

    Raddad Faqihi & Karthik Srinivasan

  5. Department of Basic Sciences, College of Science and Theoretical Studies, Saudi Electronic University, Riyadh, Saudi Arabia

    Dowlath Fathima

Authors
  1. Jayabrabu Ramakrishnan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rajesh Doss
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Thangam Palaniswamy
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Raddad Faqihi
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Dowlath Fathima
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Karthik Srinivasan
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jayabrabu Ramakrishnan.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ramakrishnan, J., Doss, R., Palaniswamy, T. et al. High performance computation of human computer interface for neurodegenerative individuals using eye movements and deep learning technique. J Supercomput 78, 2772–2792 (2022). https://doi.org/10.1007/s11227-021-03932-z

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11227-021-03932-z

Keywords

Search

Navigation