Skip to main content
SpringerLink
Log in
SpringerLink
Log in

Human-Robot Cooperation via Brain Computer Interface in Assistive Scenario

  • Conference paper
  • First Online:
Ambient Assisted Living (ForItAAL 2017)

Part of the book series: Lecture Notes in Electrical Engineering ((LNEE,volume 540))

Included in the following conference series:

Abstract

In the last years, the development of robots for assisting and collaborating with people has experienced a large growth. Applications for assistive robots include hospital service robots, factory intelligent assistants and personal homecare robots. Working in shared environments with human beings, these robots require effective ways to achieve an increasing human-robot cooperation. This work presents a possible approach for performing human-robot cooperation, namely recognition of a user selected object by means of Brain Computer Interface (BCI), followed by pick and place via a robotic arm. The object selection is achieved introducing a BCI that allows the user, after a training phase, to choose one among six different objects of common diffusion. The selection is achieved by interpreting the P300 signals generated in the brain, when the image of the object, desidered by the user, appears on a computer screen as a visual stimulus. The robot then recognizes, through a classifier, the selected object among others within its workspace, and inscribes it in a rectangle shape. Finally, the robot arm is moved in correspondence to the object position, the gripper is rotated according to the object orientation and the object grasped and moved into a different position on a desk in front of the robot. This system could support people with limited motor skills or paralysis, playing an important role in structured assistive environments in a near future.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Diamantini, C., Freddi, A., Longhi, S., Potena, D., & Storti, E. (2016, July). A goal-oriented, ontology-based methodology to support the design of AAL environments. Expert Systems with Applications, 64, 117–131.

    Article  Google Scholar 

  2. Longhi, S., Monteriù, A., & Freddi, A. (Eds.). (2017). Human monitoring, smart health and assisted living: techniques and technologies. IET.

    Google Scholar 

  3. Benetazzo, F., Freddi, A., Monteriù, A., & Longhi, S. (2014, September). Respiratory rate detection algorithm based on RGB-D camera: Theoretical background and experimental results. Healthcare Technology Letters, 1(3), 81–86.

    Article  Google Scholar 

  4. Freddi, A., Longhi, S., Monteriù, A., & Ortenzi, D. (2016, September). Redundancy analysis of cooperative dual-arm manipulators. International Journal of Advanced Robotic Systems, 13(5).

    Article  Google Scholar 

  5. Benetazzo, F., Ferracuti, F., Freddi, A., Giantomassi, A., Iarlori, S., Longhi, S., et al. (2015, July) AAL technologies for independent life of elderly people. In Biosystems & biorobotics, ambient assisted living: Italian forum 2014 (Vol. 11, pp. 329–343). Springer International Publishing.

    Google Scholar 

  6. Cavallo, F., Marletta, V., Monteriù, A., & Siciliano, P. (Eds.). (2017). Ambient assisted living: Italian forum 2016. Springer.

    Google Scholar 

  7. Schröer, S., Killmann, I., Frank, B., Völker, M., Fiederer, L., Ball, T., et al. (2015, May) An autonomous robotic assistant for drinking. In 2015 IEEE International Conference on Robotics and Automation (ICRA) (pp. 6482–6487).

    Google Scholar 

  8. Edsinger, A., & Kemp, C. C. (2007, August) Human-robot interaction for cooperative manipulation: Handing objects to one another. In RO-MAN 2007—The 16th IEEE International Symposium on Robot and Human Interactive Communication (pp. 1167–1172).

    Google Scholar 

  9. Wang, H., Ma, H., Yang, C., & Yang, Y. (2015, December) Simulation of one effective human-robot cooperation method based on kinect sensor and uncalibrated visual servoing. In 2015 IEEE International Conference on Robotics and Biomimetics (ROBIO) (pp. 2014–2019).

    Google Scholar 

  10. Donner, P., Mörtl, A., Hirche, S., & Buss, M. (2013, May) Human-robot cooperative object swinging. In 2013 IEEE International Conference on Robotics and Automation (pp. 4343–4349).

    Google Scholar 

  11. Awano, H., Ogata, T., Nishide, S., Takahashi, T., Komatani, K., & Okuno, H.G. (2010, October) Human-robot cooperation in arrangement of objects using confidence measure of neuro-dynamical system. In 2010 IEEE International Conference on Systems, Man and Cybernetics (pp. 2533–2538).

    Google Scholar 

  12. Gomez, R., Nakamura, K., Mizumoto, T., & Nakadai, K. (2015, November) Compensating changes in speaker position for improved voice-based human-robot communication. In 2015 IEEE-RAS 15th International Conference on Humanoid Robots (Humanoids) (pp. 977–982).

    Google Scholar 

  13. Strobel, M., Illmann, J., Kluge, B., & Marrone, F. (2002). Using spatial context knowledge in gesture recognition for commanding a domestic service robot. In Proceedings. 11th IEEE International Workshop on Robot and Human Interactive Communication (pp. 468–473).

    Google Scholar 

  14. Rogalla, O., Ehrenmann, M., Zollner, R., Becher, R., & Dillmann, R. (2002). Using gesture and speech control for commanding a robot assistant. In Proceedings. 11th IEEE International Workshop on Robot and Human Interactive Communication (pp. 454–459).

    Google Scholar 

  15. Hochberg, L., Bacher, D., Jarosiewicz, B., Masse, N. Y., Simeral, J. D., Vogel, J., et al. (2012). Reach and grasp by people with tetraplegia using a neurally controlled robotic arm. Nature, 372–275.

    Article  Google Scholar 

  16. Eleni, A. (2008, October). Control of medical robotics and neurorobotic prosthetics by noninvasive brain-robot interfaces via EEG and RFID technology. In 2008 8th IEEE International Conference on BioInformatics and BioEngineering (pp. 1–4).

    Google Scholar 

  17. Minati, L., Yoshimura, N., & Koike, Y. (2016). Hybrid control of a vision-guided robot arm by EOG, EMG, EEG biosignals and head movement acquired via a consumer-grade wearable device. IEEE Access, 4, 9528–9541.

    Article  Google Scholar 

  18. Ferracuti, F., Freddi, A., Iarlori, S., Longhi, S., & Peretti, P. (2013, November). Auditory paradigm for a P300 BCI system using spatial hearing. In 2013 IEEE/RSJ International Conference on Intelligent Robots and Systems (pp. 871–876).

    Google Scholar 

  19. McFarland, D. J., & Wolpaw, J. R. (2011, May). Brain-computer interfaces for communication and control. Communications of the ACM, 54(5), 60–66.

    Article  Google Scholar 

  20. Sorbello, R., Tramonte, S., Giardina, M., Bella, V. L., Spataro, R., Allison, B., et al. (2017). A human-humanoid interaction through the use of BCI for locked-in ALS patients using neuro-biological feedback fusion. IEEE Transactions on Neural Systems and Rehabilitation Engineering, PP(99), 1–1.

    Google Scholar 

  21. Petit, D., Gergondet, P., Cherubini, A., Meilland, M., Comport, A.I., & Kheddar, A. (2014, June) Navigation assistance for a BCI-controlled humanoid robot. In The 4th Annual IEEE International Conference on Cyber Technology in Automation, Control and Intelligent (pp. 246–251).

    Google Scholar 

  22. Tang, J., Jiang, J., Yu, Y., & Zhou, Z. (2015, November) Humanoid robot operation by a brain-computer interface. In 2015 7th International Conference on Information Technology in Medicine and Education (ITME) (pp. 476–479).

    Google Scholar 

  23. Wijayasekara, D., & Manic, M. (2013, June) Human machine interaction via brain activity monitoring. In 2013 6th International Conference on Human System Interactions (HSI) (pp. 103–109).

    Google Scholar 

  24. Salazar-Gomez, A. F., DelPreto, J., Gil, S., Guenther, F., & Rus, D. (2017). Correcting robot mistakes in real time using EEG signals. In 2017 IEEE International Conference on Robotics and Automation (ICRA).

    Google Scholar 

  25. Jia, W., Huang, D., Bai, O., Pu, H., Luo, X., & Chen, X. (2012). Reliable planning and execution of a human-robot cooperative system based on noninvasive brain-computer interface with uncertainty. In IEEE International Conference on Intelligent Robots and Systems (10).

    Google Scholar 

  26. Foresi, G., Freddi, A., Kyrki, V., Monteriù, A., Muthusamy, R., Ortenzi, D., et al. (2017). An avoidance control strategy for joint-position limits of dual-arm robots. In Proceedings of the 20th IFAC World Congress (to be published into).

    Article  Google Scholar 

  27. Viola, P., & Jones, M. (2001). Rapid object detection using a boosted cascade of simple features. In Proceedings of the IEEE Computer Society Conference on Computer Vision and Pattern Recognition (Vol. 1, pp. I–511–I–518).

    Google Scholar 

  28. Ferreira, A., Celeste, W., auat cheein, F., Bastos, T., Sarcinelli-Filho, M., & Carelli, R. (2008). Human-machine interfaces based on EMG and EEG applied to robotic systems 5(02), 10.

    Google Scholar 

  29. Mao, X., Li, M., Li, W., Niu, L., Xian, B., Zeng, M., et al. (2017). Review article progress in EEG-based brain robot interaction systems 2017(04), 1–25.

    Google Scholar 

  30. Kim, Y. J., Park, S. W., Yeom, H. G., Bang, M. S., Kim, J. S., Chung, C. K., et al. (2015, August). A study on a robot arm driven by three-dimensional trajectories predicted from non-invasive neural signals. BioMedical Engineering OnLine,14(1), 81.

    Google Scholar 

  31. González-Mendoza, A., Pérez-Benítez, J. L., Pérez-Benítez, J. A., Espina-Hernández, J. H. (2015, February). Brain computer interface based on SSVEP for controlling a remote control car. In 2015 International Conference on Electronics, Communications and Computers (CONIELECOMP) (pp. 93–97).

    Google Scholar 

  32. Wu, C. M., Chen, Y. J., Zaeni, I. A. E., & Chen, S. C. (2016, November). A new SSVEP based BCI application on the mobile robot in a maze game. In 2016 International Conference on Advanced Materials for Science and Engineering (ICAMSE) (pp. 550–553).

    Google Scholar 

  33. Sharbrough, F., Chatrian, G. E., Lesser, R. P., Luders, H., Nuwer, M., & Picton, T. W. (1991). American Electroencephalographic Society guidelines for standard electrode position nomenclature. Journal of Clinical Neurophysiology, 8, 200–202.

    Article  Google Scholar 

  34. Schalk, G., McFarland, D. J., Hinterberger, T., Birbaumer, N., & Wolpaw, J. R. (2004, June). Bci2000: A general-purpose brain-computer interface (BCI) system. IEEE Transactions on Biomedical Engineering,51(6), 1034–1043.

    Article  Google Scholar 

  35. Wang, Y. Q. (2014). An analysis of the Viola-Jones face detection algorithm. Image Processing On Line, 4, 128–148.

    Article  Google Scholar 

  36. Neumann, D., Langner, T., Ulbrich, F., Spitta, D., & Goehring, D. (2017, June). Online vehicle detection using Haar-like, LBP and HOG feature based image classifiers with stereo vision preselection. In 2017 IEEE Intelligent Vehicles Symposium (IV) (pp. 773–778).

    Google Scholar 

  37. Kim, B., & Yu, S. C. (2017, February). Imaging sonar based real-time underwater object detection utilizing adaboost method. In 2017 IEEE Underwater Technology (UT) (pp. 1–5).

    Google Scholar 

Download references

Acknowledgements

Authors would like to thank Simone Hanisch, Davide Centioni, Massimo Martini and Diego Retaggi, who contributed to some of the algorithms described in the paper.

Author information

Authors and Affiliations

  1. Università Politecnica delle Marche, Via Brecce Bianche, 60131, Ancona, Italy

    G. Foresi, A. Freddi, S. Iarlori, S. Longhi, A. Monteriù, D. Ortenzi & D. Proietti Pagnotta

Authors
  1. G. Foresi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. A. Freddi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. S. Iarlori
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. S. Longhi
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. A. Monteriù
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. D. Ortenzi
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. D. Proietti Pagnotta
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to S. Iarlori .

Editor information

Editors and Affiliations

  1. University of Genoa, Genoa, Italy

    Niccolò Casiddu

  2. University of Genoa, Genoa, Italy

    Claudia Porfirione

  3. Università Politecnica delle Marche, Ancona, Italy

    Andrea Monteriù

  4. The BioRobotics Institute, Scuola Superiore Sant’Anna, Pontedera, Pisa, Italy

    Filippo Cavallo

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Foresi, G. et al. (2019). Human-Robot Cooperation via Brain Computer Interface in Assistive Scenario. In: Casiddu, N., Porfirione, C., Monteriù, A., Cavallo, F. (eds) Ambient Assisted Living. ForItAAL 2017. Lecture Notes in Electrical Engineering, vol 540. Springer, Cham. https://doi.org/10.1007/978-3-030-04672-9_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-04672-9_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-04671-2

  • Online ISBN: 978-3-030-04672-9

  • eBook Packages: EngineeringEngineering (R0)

Publish with us

Policies and ethics

Search

Navigation