Abstract
Language plays essential roles in human cognition and social communication, and therefore technology of reading out speech using non-invasively measured brain activity will have both scientific and clinical merits. Here, we examined whether it is possible to decode each syllable from human fMRI activity. Four healthy subjects participated in the experiments. In a decoding session, the subjects repeatedly uttered a syllable presented on a screen at 3Hz for a 12-s block. Nine different syllables are presented in a single experimental run which was repeated 8 times. We also specified the voxels which showed articulation-related activities by utterance of all the syllables in Japanese phonology in a conventional task-rest sequence. Then, we used either all of these voxels or a part of these voxels that exist in anatomically specified ROIs (M1, cerebellum) during decoding sessions as data samples for training and testing a decoder (linear support vector machine) that classifies brain activity patterns for different syllables. To evaluate decoding performance, we performed cross-validation by testing the sample of one decoding session using a decoder trained with the samples of the remaining sessions. As a result, syllables were correctly decoded at above-chance levels. The results suggest the possibility of using non-invasively measured brain activity to read out the intended speech of disabled patients in speech motor control.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Wise, R.J., Greene, J., Buchel, C., Scott, S.K.: Brain regions involved in articulation. Lancet 353, 1057–1061 (1999)
Guenther, F.H., Ghosh, S.S., Tourville, J.A.: Neural modeling and imaging of the cortical interactions underlying syllable production. Brain Lang 96, 280–301 (2006)
Sakai, K.L.: Language acquisition and brain development. Science 310, 815–819 (2005)
Dronkers, N.F.: A new brain region for coordinating speech articulation. Nature 384, 159–161 (1996)
Birn, R.M., Bandettini, P.A., Cox, R.W., Jesmanowicz, A., Shaker, R.: Magnetic field changes in the human brain due to swallowing or speaking. Magn. Reson Med. 40, 55–60 (1998)
Gracco, V.L., Tremblay, P., Pike, B.: Imaging speech production using fMRI. Neuroimage 26, 294–301 (2005)
Kamitani, Y., Tong, F.: Decoding the visual and subjective contents of the human brain. Nat Neurosci 8, 679–685 (2005)
Cox, D.D., Savoy, R.L.: Functional magnetic resonance imaging (fMRI) brain reading: detecting and classifying distributed patterns of fMRI activity in human visual cortex. Neuroimage 19, 261–270 (2003)
Pulvermuller, F., Huss, M., Kherif, F., Del Prado Martin, F.M., Hauk, O., Shtyrov, Y.: Motor cortex maps articulatory features of speech sounds. Proc Natl Acad Sci USA 103, 7865–7870 (2006)
Fox, P.T., Huang, A., Parsons, L.M., Xiong, J.H., Zamarippa, F., Rainey, L., Lancaster, J.L.: Location-probability profiles for the mouth region of human primary motor-sensory cortex: model and validation. Neuroimage 13, 196–209 (2001)
Author information
Authors and Affiliations
Editor information
Rights and permissions
Copyright information
© 2008 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Otaka, Y., Osu, R., Kawato, M., Liu, M., Murata, S., Kamitani, Y. (2008). Decoding Syllables from Human fMRI Activity. In: Ishikawa, M., Doya, K., Miyamoto, H., Yamakawa, T. (eds) Neural Information Processing. ICONIP 2007. Lecture Notes in Computer Science, vol 4985. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-69162-4_102
Download citation
DOI: https://doi.org/10.1007/978-3-540-69162-4_102
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-69159-4
Online ISBN: 978-3-540-69162-4
eBook Packages: Computer ScienceComputer Science (R0)