Abstract
Many experiments have successfully demonstrated that prosthetic devices for restoring lost body functions can in principle be controlled by brain signals. However, stable long-term application of these devices, required for paralyzed patients, may suffer substantially from on-going signal changes for example adapting neural activities or movements of the electrodes recording brain activity. These changes currently require tedious re-learning procedures which are conducted and supervised under laboratory conditions, hampering the everyday use of such devices. As an efficient alternative to current methods we here propose an on-line adaptation scheme that exploits a hypothetical secondary signal source from brain regions reflecting the user’s affective evaluation of the current neuro- prosthetic’s performance. For demonstrating the feasibility of our idea, we simulate a typical prosthetic setup controlling a virtual robotic arm. Hereby we use the additional, hypothetical evaluation signal to adapt the decoding of the intended arm movement which is subjected to large non-stationarities. Even with weak signals and high noise levels typically encountered in recording brain activities, our simulations show that prosthetic devices can be adapted successfully during everyday usage, requiring no special training procedures. Furthermore, the adaptation is shown to be stable against large changes in neural encoding and/or in the recording itself.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Andersen RA, Burdick JW, Mussallam S, Pesaran B, Cham JG (2004a) Cognitive neural prosthetics. Trends Cogn Sci 8(11):486–493
Andersen RA, Mussallam S, Pesaran B (2004b) Selecting the signals for a brain-machine interface. Curr Opin Neurobiol 14:720–726
Batista AP, Buneo CA, Snyder LH, Andersen RA (1999) Reach plans in eye-centered coordinates. Science 285:257–260
Black MJ, Bienenstock E, Donoghue JP, Serruya M, Wu W, Gao Y (2003) Connecting brains with machines: the neural control of 2D cursor movement. In: 1st international IEEE/EMBS conference on neural engineering pp 580–583
Blankertz B, Curio G, Mueller K-R (2002) Classifying single trial EEG: towards brain computer interfacing. Adv Neural Inf Proc Syst 14:157–164
Boussaoud D, Jouffrais C, Bremmer F (1998) Eye position effects on the neuronal activity of dorsal premotor cortex in the macaque monkey. J Neurophysiol 80:1132–1150
Diedrichsen J, Hashambhoy YL, Rane T, Shadmehr R (2005) Neural correlates of reach errors. J Neurosci 25(43):9919–9931
Fiorillo CD, Tobler PN, Schultz W (2003) Discrete coding of reward probability and uncertainty by dopamine neurons. Science 299:1898–1902
Fu QG, Flament D, Coltz JD, Ebner TJ (1997) Relationship of cerebellar purkinje cell simple spike discharge to movement kinematics in the monkey. J Neurophysiol 78:478–491
Fujii N, Mushiake H, Tanji J (2000) Rostrocaudal distinction of the dorsal premotor area based on oculomotor involvement. J Neurophysiol 83:1764–1769
Gehring WJ, Goss B, Coles MGH, Meyer DE, Donchin E (1993) A neural system for error detection and compensation. Psychol Sci 4:385–390
Georgopoulos AP, Kalaska JF, Caminiti R, Massey JT (1982) On the relations between the direction of two-dimensional arm movement and cell discharge in primate motor cortex. J Neurosci 2(11):1527–1537
Helms Tillery SI, Taylor DM, Schwartz AB (2003) Training in cortical control of neuroprosthetic devices improves signal extraction from small neuronal ensembles. Rev Neurosci 14(1-2):107–119
Hinterberger T, Schmidt S, Neumann N, Mellinger J, Blankertz B, Curio G, Birbaumer N (2004) Brain-computer communication with slow cortical potentials: Methodology and critical aspects. IEEE Trans Biomed Eng 51(6):1011–1018
Holroyd CB, Coles MGH (2002) The neural basis of human error processing: reinforcement learning, dopamine, and the error-related negativity. Psychol Rev 109:679–709
Ito S, Stuphorn V, Brown JW, Schall JD (2003) Performance monitoring by the anterior cingulate cortex during saccade countermanding. Science 302:120–122
Kobayashi S, Lauwereyns J, Koizumi M, Sakagami M (2002) Influence of reward expectation on visuospatial processing in macaque lateral prefrontal cortex. J Neurophysiol 87:1488–1498
Lebedev MA, Carmena JM, O’Doherty JE, Zacksenhouse M, Henriquez CS, Principe JC, Nicolelis MAL (2005) Cortical ensemble adaptation to represent velocity of an artifical actuator controlled by a brain-machine interface. J Neurosci 25(19):4681–4693
Matsumoto K, Suzuki W, Tanaka K (2003) Neuronal correlates of goal-based motor selection in the preforntal cortex. Science 301:229–232
Mehring C, Rickert J, Vaadia E, Cardoso de Oliveira S, Aertsen A, Rotter S (2003) Inference of hand movements from local field potentials in monkey motor cortex. Nat Neurosci 6:1253–1254
Moran DW, Schwartz AB (1999a). Motor cortical representation of speed and direction during reaching. J Neurophysiol 82:2676–2692
Moran DW, Schwartz AB (1999b) Motor cortical activity during drawing movements: population representation during spiral tracing. J Neurophysiol 82:2693–2704
Murata N, Kawanabe M, Ziehe A, Mueller K-R, Amari S (2002) On-line learning in changing environments with applications in supervised and unsupervised learning. Neural Networks 15:743–760
Musallam S, Corneil BD, Greger B, Scherberger H, Andersen RA (2004) Cognitive control signals for neural prosthetics. Science 305:258–262
Mushiake H, Tanatsugu Y, Tanji J (1997) Neuronal activity in the ventral part of premotor cortex during target-reach movement is modulated by direction of gaze. J Neurophysiol 78:567–571
Nakahara H, Itoh H, Kawagoe R, Takikawa Y, Hikoska O (2004) Dopamine neurons can represent context-dependent prediction error. Neuron 41:269–280
Paninski L, Fellows MR, Hatsopoulos NG, Donoghue JP (2004) Spatiotemporal tuning of motor cortical neurons for hand position and velocity. J Neurophysiol 91:515–532
Platt ML, Glimcher PW (1999) Neural correlates of decision variables in parietal cortex. Nature 400:233–238
Ridderinkhof KR, van den Wildenberg WPM, Segalowitz SJ, Carter CS (2004a) Neurocognitive mechanisms of cognitive control: the role of prefrontal cortex in action selection, response inhibition, performance monitoring, and reward-based learning. Brain Cogn 56:129–140
Ridderinkhof KR, Ullsperger M, Crone EA, Nieuwenhuis S (2004b) The role of the medial frontal cortex in cognitive. Science 306:443–447
Roesch MR, Olson CR (2003) Impact of expected reward on neuronal activity in prefrontal cortex, frontal and supplementary eye fields and premotor cortex. J Neurophysiol 90:1766–1789
Salinas E, Abbott LF (1994) Vector reconstruction from firing rates. J Comput Neurosci 1:89–107
Satoh T, Nakai S, Kimura M (2003) Correlated coding of motivation and outcome of decision by dopamine neurons. J Neurosci 23(30):9913–9923
Schalk G, Wolpaw JR, McFarland DJ, Pfurtscheller G (2000) Eeg-based communication: presence of an error potential. Clin Neurophysiol 111(12):2138–2144
Schultz W (1998) Predictive reward signal of dopamine neurons. J Neurophysiol 80:1–27
Schultz W (2004) Neural coding of basic reward terms of animal learning theory, game theory, microeconomics and behavioural ecology. Curr Opin Neurobiol 14:139–147
Schultz W, Dayan P, Montague PR (1997) A neural substrate of prediction and reward. Science 275:1593–1599
Schwartz AB (2004) Cortical neural prosthetics. Annu Rev Neurosci 27:487–507
Schwartz AB, Taylor DM, Helms Tillery SI (2001) Extraction algorithms for cortical control of arm prosthetics. Curr Opin Neurobiol 11:701–707
Shadlen MN, Newsome WT (1998) The variable discharge of cortical neurons: implications for connectivity, computation, and information coding. J Neurosci 18(10):3870–3896
Shidara M, Aigner TG, Richmond BJ (1998) Neuronal signals in the monkey ventral striatum related to progress through a predictable series of trials. J Neurosci 18(7):2613–2625
Shima K, Tanji J (1998) Role for cingulate motor area cells in voluntary movement selection based on reward. Science 282:1335–1338
Stuphorn V, Bauswein E, Hoffmann K-P (2000) Neurons in the primate superior colliculus coding for arm movements in gaze-related coordinates. J Neurophysiol 83:1283–1299
Sutton RS, Barto AG (1998) Reinforcement learning: an introduction. MIT Press, New York
Taylor DM, Helms Tillery SI, Schwartz AB (2002) Direct cortical control of 3D neuroprosthetic devices. Science 296:1829–1832
Tobler PN, Fiorillo CD, Schultz W (2005) Adaptive coding of reward value by dopamine neurons. Science 307:1642–1645
Tremblay L, Schultz W (1999) Relative reward preference in primate orbitofrontal cortex. Nature 398:704–708
Tremblay L, Schultz W (2000) Reward-related neuronal activity during Go-Nogo task performance in primate orbitofrontal cortex. J Neurophysiol 83:1864–1876
van Schie HT, Mars RB, Coles MG, Bekkering H (2004) Modulation of activity in medial frontal and motor cortices during error observation. Nat Neurosci 7:549–554
van Veen V, Holroyd CB, Cohen JD, Stenger VA, Carter SC (2004) Errors without conflict: implications for performance monitoring theories of anterior cingulate cortex. Brain Cogn 56:267–276
Watanabe M, Hikosaka K, Sakagami M, Shirakawa S-I (2002) Coding and monitoring of motivational context in the primate prefrontal cortex. J Neurosci 22(6):2391–2400
Wolpaw J, McFarland DJ (2004) Control of a two-dimensional movement signal by a noninvasive brain-computer interface in humans. PNAS 101(51):17849–17854
Xie X, Seung HS (2004) Learning in neural networks by reinforcement of irregular spiking. Phys Rev E 69:041909
Yeung N, Botvinick MN, Cohen JD (2004) The neural basis of error detection: conflict monitoring and the error-related negativity. Psychol Rev 111(4):931–959
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Rotermund, D., Ernst, U.A. & Pawelzik, K.R. Towards On-line Adaptation of Neuro-prostheses with Neuronal Evaluation Signals. Biol Cybern 95, 243–257 (2006). https://doi.org/10.1007/s00422-006-0083-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00422-006-0083-7