Skip to main content
SpringerLink
Log in

Brain–computer interface for amyotrophic lateral sclerosis patients using deep learning network

  • Original Article
  • Published:
Neural Computing and Applications Aims and scope Submit manuscript

Abstract

Individuals with Motor Neuron Disease were unable to move from one place to another because it gradually reduced all the voluntarily movement due to the degeneration of upper and lower motors neurons. The solution to this problem was to develop rehabilitating devices using biosignals. In this study, we have designed and developed electrooculogram-based wheelchair control using Cross Power Spectral Density. The convolution neural network to verify the performance and recognition accuracy of the wheelchair navigation in the indoor environment by using four trained users and four untrained users between the different age-groups and obtained the accuracy of 91.18% and 86.88% by using four fundamental tasks. From the indoor performance, the subject S4 from trained users outperforms all the trained subjects with an average classification accuracy of 93.51%. To verify the recognition accuracy, we conducted the online performance from the online performances subject S4 from trained subjects outperforms remaining trained subjects at the same time the subject S6 from untrained subjects outperforms all the untrained subjects. From the entire study, we analyzed that classification accuracy of subjects S4 was appreciated compared to other subjects. Through the research, we confirmed that the entire trained subject’s performance was maximum compared to the untrained subjects in all the circumstances.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18
Fig. 19
Fig. 20
Fig. 21
Fig. 22
Fig. 23
Fig. 24
Fig. 25

Similar content being viewed by others

References

  1. Zarei S, Karen C, Luz R, Kelvin D, Orleiquis G, Pablo FA, Wilfredo P, Daud L, Gloria O, Angel C (2015) A comprehensive review of amyotrophic lateral sclerosis. Surg Neurol Int 6:171

    Article  Google Scholar 

  2. Grollemund V, Pradat PF, Querin G, Delbot F, Le Chat G, Pradat-Peyre JF, Bede P (2019) Machine learning in amyotrophic lateral sclerosis: achievements, pitfalls, and future directions. Front Neurosci 13:1–28

    Article  Google Scholar 

  3. Gordon J, Lerner B (2019) Insights into amyotrophic lateral sclerosis from a machine learning perspective. J Clin Med 8(10):1578

    Article  Google Scholar 

  4. Arthur KC, Calvo A, Price TR, Geiger JT, Chio A, Traynor BJ (2016) Projected increase in amyotrophic lateral sclerosis from 2015 to 2040. Nat Commun 7(1):1–6

    Article  Google Scholar 

  5. Zhang R, He S, Yang X, Wang X, Li K, Huang Q, Yu Z, Zhang X, Tang D, Li Y (2019) An EOG-based human–machine interface to control a smart home environment for patients with severe spinal cord injuries. IEEE Trans Biomed Eng 66(1):89–100

    Article  Google Scholar 

  6. Katore M, Bachute MR (2015) Speech based human machine interaction system for home automation. In: IEEE Bombay section symposium (IBSS), pp 1–6

  7. Postelnicu CC, Talaba D (2013) P300-based brain-neuronal computer interaction for spelling applications. IEEE Trans Biomed Eng 60(2):534–543

    Article  Google Scholar 

  8. He S, Li Y (2017) A single-channel EOG-based speller. IEEE Trans Neural Syst Rehabil Eng 25(11):1978–1987

    Article  Google Scholar 

  9. Zhi-Hao W, Yu-Fan K, Chuan-Te C, Shi-Hao L, Gwo-Jia J (2017) Controlling DC motor using eye blink signals based on LabVIEW. In: International conference on electrical, electronics and information engineering (ICEEIE), pp 61–65

  10. Li X, Luo D, Zhao F, Li Y, Luo H (2015) ‘Sensor fusion-based infrastructure independent and agile real-time indoor positioning technology for disabled and elderly people’. In: International symposium on future information and communication technologies for ubiquitous healthcare (Ubi-HealthTech), pp 1–5

  11. Lin M, Mo G (2011) ‘Eye gestures recognition technology in human–computer Interaction’. In: International conference on biomedical engineering and informatics (BMEI), pp 1316–1318

  12. Li L, Wu X (2011) ‘Design and implementation of multimedia control system based on bluetooth and electrooculogram (EOG)’. In: International conference on bioinformatics and biomedical engineering, pp 1–4

  13. Venkataramanan S, Prabhat P, Choudhury SR, Nemade HB, Sahambi JS (2005) ‘Biomedical instrumentation based on electrooculogram (EOG) signal processing and application to a hospital alarm system’. In: Proceedings of the 2nd international conference on intelligent sensing and information processing, pp 535–540

  14. Usakli AB, Gurkan S (2010) Design of a novel efficient human–computer interface: an electrooculagram based virtual keyboard. IEEE Trans Instrum Meas 59(8):2099–2108

    Article  Google Scholar 

  15. Lingegowda DR, Amrutesh K, Ramanujam S (2017) ‘Electrooculography based assistive technology for ALS patients’. In: IEEE international conference on consumer electronics-Asia (ICCE-Asia), pp 36–40

  16. Kuo CH, Chan YC, Chou HC, Siao JW (2009) Eyeglasses based electrooculography human–wheelchair interface. In: IEEE international conference on systems, man, and cybernetics, pp 4746–4751

  17. Hossain MS, Huda K, Rahman S, Mohiuddin Ahmad SM (2015) Implementation of an EOG based security system by analyzing eye movement patterns. In: International conference on advances in electrical engineering (ICAEE), pp 149–152

  18. Akan B, Argunsah AO (2007) A human–computer interface (HCI) based on electrooculogram (EOG) for handicapped. In: International conference on signal processing and communications applications, pp 1–3

  19. Banerjee A, Rakshit A, Tibarewala DN (2016) Application of electrooculography to estimate word count while reading text. In: International conference on systems in medicine and biology, pp 174–177

  20. Banerjee A, Datta S, Das P, Konar A, Tibarewala DN, Janarthanan R (2012) Electrooculogram based online control signal generation for wheelchair. In: International symposium on electronic system design, pp 251–255

  21. Rajesh AN, Chandralingam S, Anjaneyulu T, Satyanarayana K (2014) Eog controlled motorized wheelchair for disabled persons. Int J Med Health Biomed Bioeng Pharm Eng 8(5):302–305

    Google Scholar 

  22. Udhaya Kumar S, Vinod VM (2015) EOG based wheelchair control for quadriplegics. In: IEEE international conference on innovations in information, embedded and communication systems (ICIIECS), pp 1–4

  23. Navarro RB, Vázquez LB, Guillén EL (2018) EOG-based wheelchair control, Elsevier book series—smart wheelchairs and brain-computer interfaces mobile assistive technologies, pp 381–403

  24. Hassan U, Mughal H, Mohsin I, Khan ZH (2018) Real-time control of a mobile robot using electrooculogram based eye tracking system. In: IEEE—international multi-topic ICT conference (IMTIC), pp 1–6

  25. Li X, Luo D, Zhao F, Li Y, Luo H (2015) Sensor fusion-based infrastructure independent and agile real-time indoor positioning technology for disabled and elderly people. In: International symposium on future information and communication technologies for ubiquitous healthcare (Ubi-HealthTech), pp 1–5

  26. Lingegowda DR, Amrutesh K, Ramanujam S (2017) Electrooculography based assistive technology for ALS patients. In: IEEE international conference on consumer electronics-Asia (ICCE-Asia), pp 36–40

  27. Barea R, Boquete L, Mazo M, Lopez E (2002) System for asisted mobility using eye movements based on electrooculography. IEEE Trans Neural Syst Rehabil Eng 10(4):209–218

    Article  Google Scholar 

  28. Li Y, Pan J, Wang F, Yu Z (2013) A hybrid BCI system combining P300 and SSVEP and its application to wheelchair control. IEEE Trans Biomed Eng 60(11):3156–3166

    Article  Google Scholar 

  29. Zhang R et al (2016) Control of a wheelchair in a indoor environment based on a brain computer interface and automated navigation. IEEE Trans Neural Syst Rehabil Eng 24(1):128–139

    Article  Google Scholar 

  30. He S, Zhang R, Wang Q, Chen Y, Yang T, Feng Z, Zhang Y, Shao M, Li Y (2017) A P300-based threshold-free brain switch and its application in wheelchair control. IEEE Trans Neural Syst Rehabil Eng 25(1):37–48

    Article  Google Scholar 

  31. Lee KR (2017) Real-time “eye-writing” recognition using electrooculogram. IEEE Trans Neural Syst Rehabil Eng 25(1):37–48

    Article  Google Scholar 

  32. Ubeda A et al (2013) An integrated electrooculography and desktop input bimodal interface to support robotic arm control. IEEE Trans Hum Mach Syst 43(3):338–342

    Article  MathSciNet  Google Scholar 

  33. Ma J et al (2015) A novel EOG/EEG hybrid human–machine interface adopting eye movements and ERPs application to robot control. IEEE Trans Biomed Eng 62(3):876–889

    Article  Google Scholar 

  34. Ramkumar S, Barathkesavan H (2017) A review on classification of eye movement using electrooculography. Int J Adv Res Comput Commun Eng 6(1):36–40

    Article  Google Scholar 

  35. Ramkumar S, Sathesh Kumar K, Dhiliphan Rajkumar T, Ilayaraja M, Sankar K (2018) A review-classification of electrooculogram based human computer interfaces. Biomed Res 29(6):1078–1084

    Article  Google Scholar 

  36. Kherlopian AR, Gerrein JP, Yue M, Kim KE, Kim JW, Sukumaran M, Sajda P (2006) Electrooculogram based system for computer control using a multiple feature classification model. In: International conference of the IEEE engineering in medicine and biology society, pp 1295–1298

  37. Usakli AB, Gurkan S (2009) A novel electrooculogram-based human computer interface and its application as a virtual keyboard. In: IEEE-national biomedical engineering meeting, pp 1–4

  38. Punsawad Y, Wongsawat Y, Parnichkun M (2010) Hybrid EEG-EOG brain-computer interface system for practical machine control. IN: IEEE international conference of the IEEE engineering in medicine and biology, pp 1360–1363

  39. Tangsuksant W, Aekmunkhongpaisal C, Cambua P, Charoenpong T, Chanwimalueang T (2012) Directional eye movement detection system for virtual keyboard controller. In: IEEE biomedical engineering international conference, pp 1–5

  40. Usakli AB, Gurkan S (2009) A novel electrooculogram-based human computer interface and its application as a virtual keyboard. In: IEEE national biomedical engineering meeting, pp 1–4

  41. Soltani S, Mahnam A (2013) Design of a novel wearable human computer interface based on electrooculograghy. In: IEEE Iranian conference on electrical engineering (ICEE), pp 1–5

  42. Rakshit A, Banerjee A, Tibarewala DN (2016) Electro-oculogram based digit recognition to design assitive communication system for speech disabled patients. In: IEEE international conference on microelectronics, computing and communications, pp 1–5

  43. Chang WD, Cha HS, Kim DY, Kim SH, Im CH (2017) Development of an electrooculogram based eye-computer interface for communication of individuals with amyotrophic lateral sclerosis. J NeuroEng Rehab. https://doi.org/10.1186/s12984-017-0303-5

    Article  Google Scholar 

  44. Huang Q, He S, Wang Q, Gu Z, Peng N, Li K, Zhang Y, Shao M, Li Y (2017) An EOG-based human-machine interface for wheelchair control. IEEE Trans Biomed Eng 65(9):2023–2032. https://doi.org/10.1109/TBME.2017.2732479

    Article  Google Scholar 

  45. Hossain Z, Shuvo MH, Sarker P (2017) Hardware and software implementation of real time electrooculogram (EOG) acquisition system to control computer cursor with eyeball movement. In: IEEE international conference on advances in electrical engineering (ICAEE), pp 132–137

  46. Jialu G, Ramkumar S, Emayavaramban G, Thilagaraj M, Muneeswaran V, Rajasekaran MP, Hussein AF (2018) Offline analysis for designing electrooculogram based human computer interface control for paralyzed patients. IEEE Access. 6:79151–79161

    Article  Google Scholar 

  47. Ishii C, Murooka S, Tajima M (2018) Navigation of an electric wheelchair using EMG, EOG and EEG. IJMERR 7:143–149

    Article  Google Scholar 

  48. Fang F, Shinozaki T (2018) Electrooculography-based continuous eye writing recognition system for efficient assistive communication systems. PLoS ONE 13(2):1–20

    Google Scholar 

  49. Junwei L, Ramkumar S, Emayavaramban G, Thilagaraj M, Muneeswaran V, Rajasekaran MP, Venkataraman V, Hussein AF (2019) Brain computer interface for neurodegenerative person using electroencephalogram. IEEE Access. 7:2439–52

    Article  Google Scholar 

  50. Ramkumar S, Sathesh Kumar K, Maheswari K, P. Packia Amutha Priya, G. Emayavaramban, J. Macklin Abraham Navamani, (2020) Offline study for implementing human computer interface for elderly paralyzed patients using electrooculography and neural networks. Int J Intell Enterprise 7(1):306–321. https://doi.org/10.1504/IJIE.2019.10021809

    Article  Google Scholar 

  51. Alotta G, Paeola MD, Pinnolab F (2017) Cross-correlation and cross-power spectral density representation by complex spectral moments. Int J Non-Linear Mech 94:20–27

    Article  Google Scholar 

  52. Teng G, He Y, Zhao H, Liu D, Xiao J, Ramkumar S (2020) Design and development of human computer interface using electrooculogram with deep learning. Artif Intell Med 102:101765

    Article  Google Scholar 

  53. Xiaoxiao X, Bin L, Ramkumar S, Saravanan S, Balaji MSP, Dhanasekaran S, Thimmiaraja J (2020) Electroencephalogram based communication system for locked in state person using mentally spelled tasks with optimized network model. Artif Intell Med 102:101766

    Article  Google Scholar 

  54. Li K, Ramkumar S, Thimmiaraja J, Diwakaran S (2020) Optimized artificial neural network based performance analysis of wheelchair movement for ALS patients. Artif Intell Med 102:101754

    Article  Google Scholar 

  55. Tang W, Wang A, Ramkumar S, RadhakrishnanNair RK (2020) Signal identification system for developing rehabilittive device using deep learning algorithms. Artif Intell Med 102:101755

    Article  Google Scholar 

  56. http://deeplearning.net/tutorial/lenet.html

  57. Zhang W (1988) Shift-invariant pattern recognition neural network and its optical architecture. In: Proceedings of annual conference of the Japan society of applied physics

  58. Zhang W (1998) Parallel distributed processing model with local space-invariant interconnections and its optical architecture. Appl Opt 29(32):4790–4797

    Article  Google Scholar 

  59. Luo L, Wu Y, Qiao F, Yang Y, Wei Q, Zhou X, Fan Y, Xu S, Liu X, Yang H (2018) Design of FPGA-based accelerator for convolutional neural network under heterogeneous computing framework with OpenCL. Int J Reconfig Comput 2018:1–10

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. College of Computer Science and Information Technology, Jazan University, Jazan, Kingdom of Saudi Arabia

    Jayabrabu Ramakrishnan

  2. College of Computing and Informatics, Saudi Electronic University, Riyadh, Kingdom of Saudi Arabia

    Dinesh Mavaluru & Mervin Retnadhas

  3. School of Computing, Kalasalingam Academy of Research and Education, Krishnankoil, India

    Ramkumar Siva Sakthivel

  4. College of Computer Science and Information System, Najran University, Najran, Kingdom of Saudi Arabia

    Abdulrahman Saad Alqahtani

  5. College of Computer Science, King Khalid University, Abha, Kingdom of Saudi Arabia

    Azath Mubarakali

Authors
  1. Jayabrabu Ramakrishnan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dinesh Mavaluru
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ramkumar Siva Sakthivel
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Abdulrahman Saad Alqahtani
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Azath Mubarakali
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Mervin Retnadhas
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jayabrabu Ramakrishnan.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ramakrishnan, J., Mavaluru, D., Sakthivel, R.S. et al. Brain–computer interface for amyotrophic lateral sclerosis patients using deep learning network. Neural Comput & Applic 34, 13439–13453 (2022). https://doi.org/10.1007/s00521-020-05026-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00521-020-05026-y

Keywords

Search

Navigation