Abstract
Brain computer interfaces (BCI) have been successful in enabling control of external effectors, such as a computer cursor or robotic arm by people with severe motor impairment. While current usage relies heavily on the users’ ability to reliably utilize visual feedback from the interface, a visual-based BCI can be difficult or even unfeasible to use by people whose paralysis is accompanied by limitations of eye movements or vision. Here we present a novel auditory paradigm, and present pilot data from a person with tetraplegia performing a BCI task with auditory cues.
Inspired by previous studies, we utilize spatial auditory signals - specifically, head-response transfer functions (HRTF) - to provide task-relevant auditory feedback in real-time via headphones to the BCI user. To test the feasibility and reliability of HRTF-based feedback signals, we performed a sound source localization task in lab members and in one BCI participant with tetraplegia using intracortical recordings. Subjects were first tested on multiple azimuths of sound source, instructed to rely on the auditory cues to identify the cued spatial auditory direction, and then to navigate to the cued targets. Initial psychophysics testing suggests cursor navigation is possible in an auditory-reliant manner using the HRTF filters updated in real-time.
The results of this study demonstrate the development and implementation of a novel auditory-reliant intracortical BCI that provides real-time auditory feedback for effector control that has high potential for BCI usage in individuals with severe motor paralysis and sustained visual impairments.
This is a preview of subscription content, log in via an institution.
Buying options
Tax calculation will be finalised at checkout
Purchases are for personal use only
Learn about institutional subscriptionsReferences
Hochberg, L.R., et al.: Neuronal ensemble control of prosthetic devices by a human with tetraplegia. Nature 442(7099), 164–171 (2006). https://doi.org/10.1038/nature04970
Ajiboye, A.B., et al.: Restoration of reaching and grasping movements through brain-controlled muscle stimulation in a person with tetraplegia: a proof-of-concept demonstration. Lancet 389(10081), 1821–1830 (2017). https://doi.org/10.1016/S0140-6736(17)30601-3
Furdea, A., et al.: An auditory oddball (P300) spelling system for brain-computer interfaces. Psychophysiology 46(3), 617–625 (2009). https://doi.org/10.1111/j.1469-8986.2008.00783.x
Kübler, A., Furdea, A., Halder, S., Hammer, E.M., Nijboer, F., Kotchoubey, B.: A brain-computer interface controlled auditory event-related potential (P300) spelling system for locked-in patients. Ann. N. Y. Acad. Sci. 1157(1), 90–100 (2009). https://doi.org/10.1111/j.1749-6632.2008.04122.x
Halder, S., et al.: An auditory oddball brain–computer interface for binary choices. Clin. Neurophysiol. 121(4), 516–523 (2010). https://doi.org/10.1016/j.clinph.2009.11.087
Nambu, I., Ebisawa, M., Kogure, M., Yano, S., Hokari, H., Wada, Y.: Estimating the intended sound direction of the user: toward an auditory brain-computer interface using out-of-head sound localization. PLoS ONE 8(2), e57174 (2013). https://doi.org/10.1371/journal.pone.0057174
Schreuder, M., Blankertz, B., Tangermann, M.: A new auditory multi-class brain-computer interface paradigm: spatial hearing as an informative cue. PLoS ONE 5(4), e9813 (2010). https://doi.org/10.1371/journal.pone.0009813
Nakaizumi, C., Makino, S., Rutkowski, T.M.: Head-related impulse response cues for spatial auditory brain-computer interface. In: 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), Milan, August 2015, pp. 1071–1074. https://doi.org/10.1109/embc.2015.7318550
Hosman, T., et al.: BCI decoder performance comparison of an LSTM recurrent neural network and a Kalman filter in retrospective simulation. In: 2019 9th International IEEE/EMBS Conference on Neural Engineering (NER), San Francisco, CA, USA, March 2019, pp. 1066–1071. https://doi.org/10.1109/ner.2019.8717140
Brandman, D.M., et al.: Rapid calibration of an intracortical brain–computer interface for people with tetraplegia. J. Neural Eng. 15(2), 026007 (2018). https://doi.org/10.1088/1741-2552/aa9ee7
Algazi, V.R., Duda, R.O., Thompson, D.M., Avendano, C.: The CIPIC HRTF database. In: Proceedings of the IEEE Workshop on Applications of Signal Processing to Audio and Acoustics, New Platz, NY, USA, pp. 99–102, October 2001
Simeral, J.D., Kim, S.-P., Black, M.J., Donoghue, J.P., Hochberg, L.R.: Neural control of cursor trajectory and click by a human with tetraplegia 1000 days after implant of an intracortical microelectrode array. J. Neural Eng. 8(2), 025027 (2011). https://doi.org/10.1088/1741-2560/8/2/025027
Steadman, M.A., Kim, C., Lestang, J.-H., Goodman, D.F.M., Picinali, L.: Short-term effects of sound localization training in virtual reality. Sci. Rep. 9(1), 18284 (2019). https://doi.org/10.1038/s41598-019-54811-w
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Thengone, D.J., Hosman, T., Simeral, J.D., Hochberg, L.R. (2020). Auditory-Reliant Intracortical Brain Computer Interfaces for Effector Control by a Person with Tetraplegia. In: Stephanidis, C., Antona, M., Ntoa, S. (eds) HCI International 2020 – Late Breaking Posters. HCII 2020. Communications in Computer and Information Science, vol 1294. Springer, Cham. https://doi.org/10.1007/978-3-030-60703-6_13
Download citation
DOI: https://doi.org/10.1007/978-3-030-60703-6_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-60702-9
Online ISBN: 978-3-030-60703-6
eBook Packages: Computer ScienceComputer Science (R0)