Abstract
In recent studies, electroencephalogram (EEG)-based interfaces that enable to infer human intentions and to detect implicit human evaluation contributed to the development of effective adaptive human-machine interfaces. In this paper, we propose an approach to allow systems to adapt based on implicit human evaluation which can be extracted by using EEGs. In our study, human motion segments are evaluated according to an EEG-based interface. The goal of the presented study is to recognize incorrect motion segments before the motion sequence is completed. This is relevant for early system adaptation or correction. To this end, we recorded EEG data of 10 subjects while they observed human motion sequences. Error-related potentials (ErrPs) are used to recognize observed erroneous human motion. We trained an EEG classifier (i.e., ErrP decoder) that detects erroneous motion segments as part of motion sequences. We achieved a high classification performance, i.e., a mean balanced accuracy of 91% across all subjects. The results show that it is feasible to distinguish between correct and incorrect human motion sequences based on the current intentions of an observer. Further, it is feasible to detect incorrect motion segments in human motion sequences by using ErrPs (i.e., implicit human evaluations) before a motion sequence is completed. This is possible in real time and especially before human motion sequences are completed. Therefore, our results are relevant for human-robot interaction tasks, e.g., in which model adaptation of motion prediction is necessary before the motion sequence is completed.
Supported by the Federal Ministry for Economic Affairs and Climate Action (BMWK) FKZ: 50RA2023 and 50RA2024 and Federal Ministry for Education and Research (BMBF) FKZ: 01IW21002.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Potential artifacts due to the Gibbs phenomenon can be neglected here, as only the classification of the signals and not their shape is of interest.
- 2.
The Riemannian center of mass is also called geometric mean in the field of BCI or Fréchet mean in general.
References
Barachant, A., et al.: pyriemann/pyriemann: v0.3 (2022). https://doi.org/10.5281/zenodo.7547583
Barachant, A., Congedo, M.: A plug &play P300 BCI using information geometry. https://doi.org/10.48550/arXiv.1409.0107
Cartan, E.J.: Groupes simples clos et ouverts et géométrie riemannienne. J. Math. Pures Appl. 8, 1–34 (1929)
Chang, C.C., Lin, C.J.: LIBSVM: a library for support vector machines. ACM Trans. Intell. Syst. Technol. (TIST) 2(3), 1–27 (2011). https://doi.org/10.1145/1961189.1961199
Chavarriaga, R., Sobolewski, A., Millán, J.D.R.: Errare machinale est: the use of error-related potentials in brain-machine interfaces. Front. Neurosci. 8, 208 (2014). https://doi.org/10.3389/fnins.2014.00208
Congedo, M., Barachant, A., Bhatia, R.: Riemannian geometry for EEG-based brain-computer interfaces; a primer and a review. Brain-Comput. Interfaces 4(3), 155–174 (2017). https://doi.org/10.1080/2326263X.2017.1297192
Cortes, C., Vapnik, V.: Support-vector networks. Mach. Learn. 20(3), 273–297 (1995). https://doi.org/10.1007/BF00994018
Ehrlich, S., Cheng, G.: A neuro-based method for detecting context-dependent erroneous robot action. In: 2016 IEEE-RAS 16th International Conference on Humanoid Robots (Humanoids), pp. 477–482 (2016). https://doi.org/10.1109/HUMANOIDS.2016.7803318
Iturrate, I., Montesano, L., Minguez, J.: Robot reinforcement learning using EEG-based reward signals. In: 2010 IEEE International Conference on Robotics and Automation, pp. 4822–4829. IEEE (2010). https://doi.org/10.1109/ROBOT.2010.5509734
Iturrate, I., Montesano, L., Minguez, J.: Single trial recognition of error-related potentials during observation of robot operation. In: 2010 Annual International Conference of the IEEE Engineering in Medicine and Biology, pp. 4181–4184. IEEE (2010). https://doi.org/10.1109/IEMBS.2010.5627380
Iturrate, I., Chavarriaga, R., Montesano, L., Minguez, J., Millán, J.D.R.: Teaching brain-machine interfaces as an alternative paradigm to neuroprosthetics control. Sci. Rep. 5, 13893 (2015). https://doi.org/10.1038/srep13893
Iturrate, I., Grizou, J., Omedes, J., Oudeyer, P.Y., Lopes, M., Montesano, L.: Exploiting task constraints for self-calibrated brain-machine interface control using error-related potentials. PLoS ONE 10(7), e0131491 (2015). https://doi.org/10.1371/journal.pone.0131491
Kappenman, E.S., Luck, S.J.: The Oxford Handbook of Event-Related Potential Components. Oxford University Press, Oxford (2011). https://doi.org/10.1093/oxfordhb/9780195374148.001.0001
Kim, S.K., Kirchner, E.A.: Classifier transferability in the detection of error related potentials from observation to interaction. In: 2013 IEEE International Conference on Systems, Man, and Cybernetics, pp. 3360–3365 (2013). https://doi.org/10.1109/SMC.2013.573
Kim, S.K., Kirchner, E.A.: Handling few training data: classifier transfer between different types of error-related potentials. IEEE Trans. Neural Syst. Rehabil. Eng. 24(3), 320–332 (2016). https://doi.org/10.1109/TNSRE.2015.2507868
Kim, S.K., Kirchner, E.A., Kirchner, F.: Flexible online adaptation of learning strategy using EEG-based reinforcement signals in real-world robotic applications. In: 2020 IEEE International Conference on Robotics and Automation (ICRA), pp. 4885–4891 (2020). https://doi.org/10.1109/ICRA40945.2020.9197538
Kim, S.K., Kirchner, E.A., Schloßmüller, L., Kirchner, F.: Errors in human-robot interactions and their effects on robot learning. Front. Robot. AI 7, 558531 (2020). https://doi.org/10.3389/frobt.2020.558531
Kim, S.K., Kirchner, E.A., Stefes, A., Kirchner, F.: Intrinsic interactive reinforcement learning - using error-related potentials for real world human-robot interaction. Sci. Rep. 7, 1–16 (2017). https://doi.org/10.1038/s41598-017-17682-7
Kirchner, E.A., Fairclough, S.H., Kirchner, F.: Embedded multimodal interfaces in robotics: applications, future trends, and societal implications. In: Monash University, Oviatt, S., Schuller, B., University of Augsburg and Imperial College London, Cohen, P.R., Monash University, Sonntag, D., German Research Center for Artificial Intelligence (DFKI), Potamianos, G., University of Thessaly, Krüger, A., Saarland University and German Research Center for Artificial Intelligence (DFKI) (eds.) The Handbook of Multimodal-Multisensor Interfaces: Language Processing, Software, Commercialization, and Emerging Directions - Volume 3. Association for Computing Machinery (2019). https://doi.org/10.1145/3233795.3233810
Kirchner, E.A., et al.: On the applicability of brain reading for predictive human-machine interfaces in robotics. PLoS ONE 8(12), e81732 (2013). https://doi.org/10.1371/journal.pone.0081732
Krell, M., et al.: pySPACE-a signal processing and classification environment in Python. Front. Neuroinform. 7, 40 (2013). https://doi.org/10.3389/fninf.2013.00040
Ledoit, O., Wolf, M.: A well-conditioned estimator for large-dimensional covariance matrices. J. Multivar. Anal. 88(2), 365–411 (2004). https://doi.org/10.1016/S0047-259X(03)00096-4
Lopes-Dias, C., et al.: Online asynchronous detection of error-related potentials in participants with a spinal cord injury using a generic classifier. J. Neural Eng. 18(4), 046022 (2021). https://doi.org/10.1088/1741-2552/abd1eb
Lopes-Dias, C., Sburlea, A.I., Müller-Putz, G.: Masked and unmasked error-related potentials during continuous control and feedback. J. Neural Eng. 15, 036031 (2018). https://doi.org/10.1088/1741-2552/aab806
Mandel, J.: Generalisation de la theorie de plasticite de WT Koiter. Int. J. Solids Struct. 1(3), 273–295 (1965). https://doi.org/10.1016/0020-7683(65)90034-X
Omedes, J., Iturrate, I., Minguez, J., Montesano, L.: Analysis and asynchronous detection of gradually unfolding errors during monitoring tasks. J. Neural Eng. 12, 056001 (2015). https://doi.org/10.1088/1741-2560/12/5/056001
Pavone, E.F., Tieri, G., Rizza, G., Tidoni, E., Grisoni, L., Aglioti, S.M.: Embodying others in immersive virtual reality: electro-cortical signatures of monitoring the errors in the actions of an avatar seen from a first-person perspective. J. Neurosci. 36(2), 268–279 (2016). https://doi.org/10.1523/JNEUROSCI.0494-15.2016
Rivet, B., Souloumiac, A., Attina, V., Gibert, G.: xDAWN algorithm to enhance evoked potentials: application to brain-computer interface. IEEE Trans. Biomed. Eng. 56(8), 2035–2043 (2009). https://doi.org/10.1109/TBME.2009.2012869
Salazar-Gomez, A.F., DelPreto, J., Gil, S., Guenther, F.H., Rus, D.: Correcting robot mistakes in real time using EEG signals. In: 2017 IEEE International Conference on Robotics and Automation (ICRA), pp. 6570–6577 (2017). https://doi.org/10.1109/ICRA.2017.7989777
Yger, F., Berar, M., Lotte, F.: Riemannian approaches in brain-computer interfaces: a review. IEEE Trans. Neural Syst. Rehabil. Eng. 25(10), 1753–1762 (2017). https://doi.org/10.1109/TNSRE.2016.2627016
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Kim, S.K., Liersch, J., Kirchner, E.A. (2023). Classification of Error-Related Potentials Evoked During Observation of Human Motion Sequences. In: Schmorrow, D.D., Fidopiastis, C.M. (eds) Augmented Cognition. HCII 2023. Lecture Notes in Computer Science(), vol 14019. Springer, Cham. https://doi.org/10.1007/978-3-031-35017-7_10
Download citation
DOI: https://doi.org/10.1007/978-3-031-35017-7_10
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-35016-0
Online ISBN: 978-3-031-35017-7
eBook Packages: Computer ScienceComputer Science (R0)