Skip to main content
SpringerLink
Log in

A WiSARD Network Approach for a BCI-Based Robotic Prosthetic Control

  • Published:
International Journal of Social Robotics Aims and scope Submit manuscript

Abstract

There are significant research efforts underway in the area of automatic robotic-prosthesis control based on brain–computer interface aiming at understanding how neural signals can be used to control these assistive devices. Although these approaches have made significant progresses in the ability to control robotic manipulators, the realization of portable and easy of use solutions is still an ongoing research endeavor. In this paper, we propose a novel approach relying on the use of (i) a Weightless Neural Network-based classifier, whose design lends itself to an easy hardware implementation; (ii) a robotic hand designed in order to fit with the main requirements of these kind of technologies (such as low cost, high performance, lightness, etc.) and (iii) a non-invasive light-weight and easy-donning EEG-helmet in order to provide a portable controller interface. The developed interface is connected to a robotic hand for controlling open/close actions. The preliminary results for this system are promising in that they demonstrate that the proposed method achieves similar performance with respect to state-of-the-art classifiers by contemporaneously representing a most suitable and practicable solution due to its portability on hardware devices, which will permit its direct implementation on the helmet board.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9

Similar content being viewed by others

Notes

  1. http://www.touchbionics.com/.

  2. http://www.living-with-michelangelo.com/home/.

  3. http://www.bebionic.com.

  4. http://www.instructables.com/id/Brain-Controlled-Wheelchair.

  5. https://www.emotiv.com.

  6. Wilkes, Stonham and Aleksander Recognition Device.

  7. Available for download at: https://github.com/giordamaug/WiSARD4WEKA.

  8. cA1 with 16, cA2 with 8, cA3 and cD3 with 4 components.

  9. cD1 with 32, cD2 with 16, cD3 with 8, and cD4 and cA4 with 4 components.

  10. Internally in WiSARD-Classifier software.

  11. All datasets used in the experiments are available for download at: https://github.com/giordamaug/WiSARD4WEKA.

  12. http://deeplearning4j.org.

  13. https://sites.google.com/site/projectbci.

  14. Note that the FLDA method could not be applied since this case study is a multi-class classification problem.

  15. The WiSARD model building time is 5.25 s compared to 9.3 s needed to build the MLP model in the case BCI-db1lv3-32 with 17 features; in the BCI-db1lv4-64 case study with 20 features WiSARD building takes 3.5 s compared to 3.1 s needed for MLP.

References

  1. Abdulkader SN, Atia A, Mostafa MSM (2015) Brain computer interfacing: applications and challenges. Egypt Inform J 16(2):213–230. https://doi.org/10.1016/j.eij.2015.06.002

    Article  Google Scholar 

  2. Aha D, Kibler D (1991) Instance-based learning algorithms. Mach Learn 6:37–66. https://doi.org/10.1007/BF00153759

    Article  MATH  Google Scholar 

  3. Al-Fahoum AS, Al-Fraihat AA (2014) Methods of EEG signal features extraction using linear analysis in frequency and time-frequency domains. In: ISRN neuroscience

  4. Aleksander I (1970) Microcircuit learning nets: Hamming-distance behaviour. Electron Lett 6(5):134–136. https://doi.org/10.1049/el:19700092

    Article  Google Scholar 

  5. Aleksander I, Albrow RC (1968) Pattern recognition with adaptive logic elements. In: Proceedings of the IEE-NPL conference on pattern recognition, pp 68–74

  6. Aleksander I, Morton H (1990) An introduction to neural computing. Chapman & Hall, London

    Google Scholar 

  7. Aleksander I, Thomas W, Bowden P (1984) Wisard\(\cdot \)a radical step forward in image recognition. Sens Rev 4(3):120–124. https://doi.org/10.1108/eb007637

    Article  Google Scholar 

  8. Alomari MH, Awada EA, Samaha A, AlKamha K (2014) Wavelet-based feature extraction for the analysis of EEG signals associated with imagined fists and feet movements. Comput Inf Sci 7(2):17–27

    Google Scholar 

  9. Alomari MH, Samaha A, AlKamha K (2013) Automated classification of l/r hand movement EEG signals using advanced feature extraction and machine learning. Int J Adv Comput Sci Appl 4(6):207–212. https://doi.org/10.14569/IJACSA.2013.040628

    Article  Google Scholar 

  10. Athanasiou A, Xygonakis I, Pandria N, Kartsidis P, Arfaras G, Kavazidi KR, Foroglou N, Astaras A, Bamidis PD (2017) Towards rehabilitation robotics: off-the-shelf BCI control of anthropomorphic robotic arms. In: BioMed research international

  11. Badue C, Pedroni F, Souza A (2008) Multi-label text categorization using VG-RAM weightless neural networks. In: Neural networks, 2008. SBRN ’08., pp 105–110. https://doi.org/10.1109/SBRN.2008.29

  12. Bang J, Choi JS, Park K (2013) Noise reduction in brainwaves by using both EEG signals and frontal viewing camera images. Sensors (Switzerland) 13(5):6272–6294. https://doi.org/10.3390/s130506272

    Article  Google Scholar 

  13. Beyrouthy T, Al Kork SK, Korbane JA, Abdulmonem A (2016) EEG mind controlled smart prosthetic arm. In: 2016 IEEE international conference on emerging technologies and innovative business practices for the transformation of societies (EmergiTech). pp 404–409. https://doi.org/10.1109/EmergiTech.2016.7737375

  14. Bi L, Fan X, Liu Y (2013) EEG-based brain-controlled mobile robots: a survey. IEEE Trans Hum Mach Syst 43(2):161–176. https://doi.org/10.1109/TSMCC.2012.2219046

    Article  Google Scholar 

  15. Breiman L (2001) Random forests. Mach Learn 45(1):5–32

    Article  Google Scholar 

  16. Broquère X, Finzi A, Mainprice J, Rossi S, Sidobre D, Staffa M (2014) An attentional approach to human–robot interactive manipulation. Int J Soc Robot 6(4):533–553

    Article  Google Scholar 

  17. Burattini E, Finzi A, Rossi S, Staffa M (2012) Attentional human–robot interaction in simple manipulation tasks. In: Proceedings of the seventh annual ACM/IEEE international conference on human-robot interaction. ACM, Boston, pp 129–130. https://doi.org/10.1145/2157689.2157719

  18. Caesarendra W, Ariyanto M, Pambudi KA, Amri MF, Turnip A (2017) Classification of EEG signals for eye focuses using artificial neural network. Internetworking Indones J 9(1):15–20

    Google Scholar 

  19. Cardoso DO, Carvalho DS, Alves DSF, de Souza DFP, Carneiro HCC, Pedreira CE, Lima PMV, França FMG (2016) Financial credit analysis via a clustering weightless neural classifier. Neurocomputing 183:70–78

    Article  Google Scholar 

  20. Cardoso D, Gama J, De Gregorio M, França FMG (2012) Wips: the wisard indoor positioning system. In: ESANN’12, pp 521–526

  21. Cempini M, Cortese M, Vitiello N (2015) A powered finger-thumb wearable hand exoskeleton with self-aligning joint axes. IEEE/ASME Trans Mechatron 20(2):705–716. https://doi.org/10.1109/TMECH.2014.2315528

    Article  Google Scholar 

  22. Chen X, Zhao B, Wang Y, Xu S, Gao X (2018) Control of a 7-DOF robotic arm system with an SSVEP-based BCI. Int J Neural Syst 28:1850018. https://doi.org/10.1142/S0129065718500181

    Article  Google Scholar 

  23. Colombo R, Pisano F, Micera S, Mazzone A, Delconte C, Carrozza MC, Dario P, Minuco G (2005) Robotic techniques for upper limb evaluation and rehabilitation of stroke patients. IEEE Trans Neural Syst Rehabilit Eng 13(3):311–324. https://doi.org/10.1109/TNSRE.2005.848352

    Article  Google Scholar 

  24. De Gregorio M, Giordano M (2017) Background estimation by weightless neural networks. Pattern Recognit Lett 96:55–65. https://doi.org/10.1016/j.patrec.2017.05.029 Scene Background Modeling and Initialization

    Article  Google Scholar 

  25. De Gregorio M, Giordano M (2018) An experimental evaluation of weightless neural networks for multi-class classification. Appl Soft Comput 72:338–354. https://doi.org/10.1016/j.asoc.2018.07.052

    Article  Google Scholar 

  26. de Aguiar K, França FMG, Barbosa VC, Teixeira CAD (2015) Early detection of epilepsy seizures based on a weightless neural network. In: EMBC, IEEE, pp 4470–4474. http://dblp.uni-trier.de/db/conf/embc/embc2015.html#AguiarFBT15

  27. Ferreira VC, França FMG, Nery AS (2018) A smart disk for in-situ face recognition. In: 2018 IEEE international parallel and distributed processing symposium workshops, pp 1241–1249

  28. Festante F, Vanderwert RE, Sclafani V, Paukner A, Simpson EA, Suomi SJ, Fox NA, Ferrari PF (2018) EEG beta desynchronization during hand goal-directed action observation in newborn monkeys and its relation to the emergence of hand motor skills. Dev Cognit Neurosci 30:142–149. https://doi.org/10.1016/j.dcn.2018.02.010

    Article  Google Scholar 

  29. Ficuciello F (2018) Hand-arm autonomous grasping: synergistic motions to enhance the learning process. Intell Serv Robot 12:17–25. https://doi.org/10.1007/s11370-018-0262-0

    Article  Google Scholar 

  30. Ficuciello F (2018) Synergy-based control of underactuated anthropomorphic hands. IEEE Trans Ind Inf 15:1144–1152. https://doi.org/10.1109/TII.2018.2841043

    Article  Google Scholar 

  31. Ficuciello F, Carloni R, Visser L, Stramigioli S (2010) Port-Hamiltonian modeling for soft-finger manipulation. In: Proceedings of IEEE/RSJ international conference on intelligent robots and systems. Taipei, Taiwan, pp 4281–4286

  32. Ficuciello F, Federico A, Lippiello V, Siciliano B (2018) Synergies evaluation of the SCHUNK S5FH for grasping control. Springer, Cham, pp 225–233. https://doi.org/10.1007/978-3-319-56802-7_24

    Book  Google Scholar 

  33. Ficuciello F, Palli G, Melchiorri C, Siciliano B (2012) Planning and control during reach to grasp using the three predominant UB Hand IV postural synergies. In: Proceedings of IEEE international conference on robotics and automation. Saint Paul, pp 2255–2260

  34. Ficuciello F, Palli G, Melchiorri C, Siciliano B (2012) Postural synergies and neural network for autonomous grasping: a tool for Dextrous prosthetic and robotic hands, chap. converging clinical and engineering research on neurorehabilitation, biosystems and biorobotics. Springer, Berlin, Heidelberg, pp 467–480

  35. Ficuciello F, Palli G, Melchiorri C, Siciliano B (2014) Postural synergies of the UB hand IV for human-like grasping. Robot Auton Syst 62(4):515–527. https://doi.org/10.1016/j.robot.2013.12.008

    Article  Google Scholar 

  36. Ficuciello F, Zaccara D, Siciliano B (2016) Synergy-based policy improvement with path integrals for anthropomorphic hands. In: Proceedings of IEEE international conference on intelligent robots and systems. Daejeon, Korea, pp 1940–1945

  37. Gandolla M, Ferrante S, Ferrigno G, Baldassini D, Molteni F, Guanziroli E, Cotti Cottini M, Seneci C, Pedrocchi A (2016) Artificial neural network EMG classifier for functional hand grasp movements prediction. J Int Med Res 45:1831–1847. https://doi.org/10.1177/0300060516656689

    Article  Google Scholar 

  38. Hastie T, Tibshirani R (1998) Classification by pairwise coupling. In: Jordan MI, Kearns MJ, Solla SA (eds) Advances in neural information processing systems, vol 10. MIT Press, Cambridge

    Google Scholar 

  39. He B, Gao S, Yuan H, Wolpaw J (2013) Brain–computer interfaces. Springer, New York, pp 87–151. https://doi.org/10.1007/9781461452270

    Book  Google Scholar 

  40. He B, Gao S, Yuan H, Wolpaw JR (2013) Brain-computer interfaces. Springer, Boston. https://doi.org/10.1007/978-1-4614-5227-0_2

    Book  Google Scholar 

  41. Hochreiter S, Schmidhuber J (1997) Long short-term memory. Neural Comput 9(8):1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735

    Article  Google Scholar 

  42. Jeong Y, Lee D, Kim K, Park JO (2000) A wearable robotic arm with high force-reflection capability. In: Proceedings 9th IEEE international workshop on robot and human interactive communication. IEEE RO-MAN 2000 (Cat. No.00TH8499), pp 411–416. https://doi.org/10.1109/ROMAN.2000.892639

  43. Keerthi S, Shevade S, Bhattacharyya C, Murthy K (2001) Improvements to Platt’s SMO algorithm for SVM classifier design. Neural Comput 13(3):637–649

    Article  Google Scholar 

  44. Liao K, Xiao R, Gonzalez J, Ding L (2014) Decoding individual finger movements from one hand using human EEG signals. PLOS ONE 9(1):1–12. https://doi.org/10.1371/journal.pone.0085192

    Article  Google Scholar 

  45. Maciejasz P, Eschweiler J, Gerlach-Hahn K, Jansen-Toy A, Leonhardt S (2014) A survey on robotic devices for upper limb rehabilitation. J Neuroeng Rehabilit 11:3. https://doi.org/10.1186/1743-0003-11-3

    Article  Google Scholar 

  46. Mak JN, Wolpaw JR (2009) Clinical applications of brain–computer interfaces: current state and future prospects. IEEE Rev Biomed Eng 2:187–199. https://doi.org/10.1109/RBME.2009.2035356

    Article  Google Scholar 

  47. Mao X, Li M, Li W, Niu L, Xian B, Zeng M, Chen G (2017) Progress in EEG-based brain robot interaction systems. Comput Intell Neurosci. https://doi.org/10.1155/2017/1742862

  48. Mulder T (2007) Motor imagery and action observation: cognitive tools for rehabilitation. J Neural Transm 114(10):1265–1278. https://doi.org/10.1007/s00702-007-0763-z

    Article  Google Scholar 

  49. Narang A, Batra B, Ahuja A, Yadav J, Pachauri N (2018) Classification of EEG signals for epileptic seizures using Levenberg–Marquardt algorithm based multilayer perceptron neural network. J Intell Fuzzy Syst 34:1669–1677. https://doi.org/10.3233/JIFS-169460

    Article  Google Scholar 

  50. Iengo S, Origlia A, Staffa M, Finzi A (2012) Attentional and emotional regulation in human-robot interaction. In: IEEE RO-MAN: The 21st IEEE international symposium on robot and human interactive communication. pp 1135–1140. https://doi.org/10.1109/ROMAN.2012.6343901

  51. Ortner R, Gruenbacher E, Guger C (2018) State of the art in sensors, signals and signal processing. http://www.gtec.at/content/download/1659/10347/file/StateOfTheArt_Physio_SensorsSignals.pdf

  52. Pattnaik PK, Sarraf J (2018) Brain computer interface issues on hand movement. J King Saud Univ Comput Inf Sci 30(1):18–24. https://doi.org/10.1016/j.jksuci.2016.09.006

    Article  Google Scholar 

  53. Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay E (2011) Scikit-learn: machine learning in Python. J Mach Learn Res 12:2825–2830

    MathSciNet  MATH  Google Scholar 

  54. Pedrocchi A, Ferrante S, Ambrosini E, Gandolla M, Casellato C, Schauer T, Klauer C, Pascual J, Vidaurre C, Gföhler M, Reichenfelser W, Karner J, Micera S, Crema A, Molteni F, Rossini M, Palumbo G, Guanziroli E, Jedlitschka A, Hack M, Bulgheroni M, d’Amico E, Schenk P, Zwicker S, Duschau-Wicke A, Miseikis J, Graber L, Ferrigno G (2013) Mundus project: multimodal neuroprosthesis for daily upper limb support. J NeuroEng Rehabilit 10(1):66. https://doi.org/10.1186/1743-0003-10-66

    Article  Google Scholar 

  55. Platt J (1998) Fast training of support vector machines using sequential minimal optimization. In: Schoelkopf B, Burges C, Smola A (eds) Advances in kernel methods–support vector learning. MIT Press, Cambridge. https://pdfs.semanticscholar.org/d1fa/8485ad749d51e7470d801bc1931706597601.pdf

  56. Prochazka A, Kukal J, Vysata O (2008) Wavelet transform use for feature extraction and EEG signal segments classification. In: 2008 3rd International symposium on communications, control and signal processing, pp 719–722. https://doi.org/10.1109/ISCCSP.2008.4537317

  57. Rao RPN, Scherer R (2010) Brain-computer interfacing [in the spotlight]. IEEE Signal Process Mag 27(4):152. https://doi.org/10.1109/MSP.2010.936774

    Article  Google Scholar 

  58. Rossi S, Staffa M, Bove L, Capasso R, Ercolano G (2017) User’s personality and activity influence on HRI comfortable distances. In: Kheddar A, Yoshida E, Ge SS, Suzuki K, Cabibihan JJ, Eyssel F, He H (eds.) ICSR, Lecture Notes in Computer Science, vol. 10652. Springer, pp 167–177. https://doi.org/10.1007/978-3-319-70022-9_17

  59. Rossi S, Staffa M, Giordano M, De Gregorio M, Rossi A, Tamburro A, Vellucci C (2015) User tracking in HRI applications with the human-in-the-loop. In: Proceedings of the tenth annual ACM/IEEE international conference on human–robot interaction extended abstracts, HRI’15 extended abstracts, pp 33–34. ACM, New York, NY, USA. https://doi.org/10.1145/2701973.2701980

  60. Santiago L, Patil VC, Prado CB, Alves TA, Marzulo LA, França FM, Kundu S (2017) Realizing strong PUF from weak PUF via neural computing. In: 2017 IEEE international symposium on defect and fault tolerance in VLSI and nanotechnology systems (DFT), pp 1–6. https://doi.org/10.1109/DFT.2017.8244433

  61. Sequeira S, Diogo C, Ferreira F (2013) EEG-signals based control strategy for prosthetic drive systems. In: IEEE 3rd Portuguese meeting in bioengineering, Braga, pp 1–4

  62. Simões M, Amaral C, França F, Carvalho P, Castelo-Branco M (2019) Applying weightless neural networks to a p300-based brain-computer interface. In: Lhotska L, Sukupova L, Lacković I, Ibbott GS (eds) World congress on medical physics and biomedical engineering 2018. Springer, Singapore, pp 113–117

    Chapter  Google Scholar 

  63. Soekadar SR, Witkowski M, Gómez C, Opisso E, Medina J, Cortese M, Cempini M, Carrozza MC, Cohen LG, Birbaumer N, Vitiello N (2016) Hybrid EEG/EOG-based brain/neural hand exoskeleton restores fully independent daily living activities after quadriplegia. Sci Robot 1(1):eaag3296-1. https://doi.org/10.1126/scirobotics.aag3296

    Article  Google Scholar 

  64. Souza C, Nobre F, Lima P, Silva R, Brindeiro R, França F (2012) Recognition of hIV-1 subtypes and antiretroviral drug resistance using weightless neural networks. In: ESANN’12, pp 429–434

  65. Staffa M, Rossi S, Giordano M, De Gregorio M, Siciliano B (2015) Segmentation performance in tracking deformable objects via WNNs. In: 2015 IEEE international conference on robotics and automation (ICRA), pp 2462–2467. https://doi.org/10.1109/ICRA.2015.7139528

  66. Subasi A, Erçelebi E (2005) Classification of eeg signals using neural network and logistic regression. Comput Methods Prog Biomed 78(2):87–99. https://doi.org/10.1016/j.cmpb.2004.10.009

    Article  MATH  Google Scholar 

  67. Witten IH, Frank E, Hall MA (2011) Data mining: practical machine learning tools and techniques, 3rd edn. Series in Data Management Systems. Morgan Kaufmann, Amsterdam

    Google Scholar 

  68. Wolpaw JR, Birbaumer N, McFarland DJ, Pfurtscheller G, Vaughan TM (2002) Brain–computer interfaces for communication and control. Clin Neurophysiol 113(6):767–791. https://doi.org/10.1016/S1388-2457(02)00057-3

    Article  Google Scholar 

  69. Yang C, Wu H, Li Z, He W, Wang N, Su CY (2018) Mind control of a robotic arm with visual fusion technology. IEEE Trans Ind Inf 14(9):3822–3830

    Article  Google Scholar 

Download references

Funding

Part of this work is funded by the European Project MUSHA (MUltifunctional Smart HAnds: novel insight for new technological insight for mini-invasive surgical tools and artificial anthropomorphic hands) under the Grant Agreement No: 320992, whose Principal Investigator is the third author of this paper, Fanny Ficuciello.

Author information

Authors and Affiliations

  1. Department of Physics “E. Pancini”, University of Naples Federico II, Via Cinthia, 21, 80126, Napoli, Italy

    Mariacarla Staffa

  2. High Performance Computing and Networking Institute, National Research Council of Italy, Via Pietro Castellino 111, Naples, Italy

    Maurizio Giordano

  3. Department of Electrical Engineering and Information Technologies, University of Naples Federico II, via Claudio, 21, 80125, Napoli, Italy

    Fanny Ficuciello

Authors
  1. Mariacarla Staffa
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Maurizio Giordano
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Fanny Ficuciello
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mariacarla Staffa.

Ethics declarations

Conflict of interest

The first author of this paper, Mariacarla Staffa, is part of the Editorial Board of this Special Issues. The authors declare that they have no other conflict of interest to disclose.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Staffa, M., Giordano, M. & Ficuciello, F. A WiSARD Network Approach for a BCI-Based Robotic Prosthetic Control. Int J of Soc Robotics 12, 749–764 (2020). https://doi.org/10.1007/s12369-019-00576-1

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12369-019-00576-1

Keywords

Search

Navigation