Abstract
Combining haptics with functional magnetic resonance imaging (Haptic fMRI) has enabled complex motor neuroimaging experiments that non-invasively map real-world motor tasks on to the human brain. The technique’s resolution, fidelity and susceptibility to scanning artifacts, however, have not yet been estimated in a quantitative manner. Here, we demonstrate that unconstrained three degree-of-freedom Haptic fMRI experiments can reliably activate brain regions involved in planning, motor control, haptic perception, and vision. We show that associated neural measurements are reliable, heterogeneous at the millimeter scale, and free from measurable artifacts, and that their anatomical localization is consistent with past neuroscience experiments. In addition, we demonstrate the feasibility of using electromagnetic actuation in Haptic fMRI interfaces to apply high fidelity open-loop three-axis haptic forces (0.5–2N; square or 0.1–65Hz sine waveforms) while maintaining negligible temporal noise in pre-motor, motor, somatosensory, and visual cortex (<1 % of signal). Our results show that Haptic fMRI is a robust and reliable technique for characterizing the human brain’s motor controller.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Logothetis, N.K., Wandell, B.A.: Interpreting the bold signal. Annu. Rev. Physiol. 66, 735–769 (2004)
Logothetis, N.K.: What we can do and what we cannot do with fMRI. Nature 453, 869–878 (2008)
Hale, K.S., Stanney, K.M.: Deriving haptic design guidelines from human physiological, psychophysical, and neurological foundations. IEEE Comput. Graph. Appl. 24, 33–39 (2004)
Kostic, M., Popovic, D., Popovic, M.: Influence of planar manipulandum to the hand trajectory during point to point movement. In: IEEE International Conference on Rehabilitation Robotics, pp. 1–4 (2011)
Khatib, O.: Inertial properties in robotic manipulation: an object-level framework. Int. J. Robot. Res. 14, 19–36 (1995)
Rosenbaum, D.: Human Motor Control. Academic Press, San Diego (2009)
Meier, J.D., Aflalo, T.N., Kastner, S., Graziano, M.S.A.: Complex organization of human primary motor cortex: a high-resolution fMRI study. J. Neurophysiol. 100, 1800–1812 (2008)
Churchland, M.M., Yu, B.M., Ryu, S.I., Santhanam, G., Shenoy, K.V.: Neural variability in premotor cortex provides a signature of motor preparation. J. Neurosci. 26, 3697–3712 (2006)
Aflalo, T.N., Graziano, M.S.A.: Relationship between unconstrained arm movements and single-neuron firing in the macaque motor cortex. J. Neurosci. 27, 2760–2780 (2007)
Kay, K.N., Winawer, J., Mezer, A., Wandell, B.A.: Compressive spatial summation in human visual cortex. J. Neurophysiol. 110, 481–494 (2013)
Mesa-Munera, E., Ramirez-Salazar, J., Boulanger, P., Bischof, W.F., Branch, J.W.: Estimation of vibration and force stimulus thresholds for haptic guidance in MIS training. Rev. Ing. Biomed. 5, 17–22 (2012)
Menon, S., Brantner, G., Aholt, C., Kay, K., Khatib, O.: Haptic fMRI: combining functional neuroimaging with haptics for studying the brain’s motor control representation. In: Proceedings of the 13th Annual Conference of the IEEE Engineering in Medicine and Biology Society, pp. 4137–4142 (2013)
Menon, S., Yu, M., Kay, K., Khatib, O.: Haptic fMRI: accurately estimating neural responses in motor, pre-motor, and somatosensory cortex during complex motor tasks. In: Proceedings of the 14th Annual Conference of the IEEE Engineering in Medicine and Biology Society (2014)
Menon, S., Quigley, P., Yu, M., Khatib, O.: Haptic fMRI: using classification to quantify task-correlated noise during goal-directed reaching motions. In: Proceedings of the 14th Annual Conference of the IEEE Engineering in Medicine and Biology Society (2014)
Vogan, J., Wingert, A., Plante, J., Dubowsky, S., Hafez, M., Kacher, D.: Manipulation in MRI devices using electrostrictive polymer actuators: with an application to reconfigurable imaging coils. In: IEEE International Conference on Robotics and Automation, pp. 2498–2504 (2004)
Diedrichsen, J., Hashambhoy, Y., Rane, T., Shadmehr, R.: Neural correlates of reach errors. J. Neurosci. 25, 9919–9931 (2005)
Menon, S., Stanley, A., Zhu, J., Okamura, A., Khatib, O.: Mapping stiffness perception in the brain with an fMRI-compatible particle-jamming haptic interface. In: Proceedings of the 14th Annual Conference of the IEEE Engineering in Medicine and Biology Society (2014)
Burdet, E., Gassert, R., Gowrishankar, G., Bleuler, H.: fMRI compatible haptic interfaces to investigate human motor control. Exp. Robot. IX 21, 25–34 (2006)
Klare, S., Peer, A., Buss, M.: Development of a 3 DoF MR-compatible Haptic Interface for Pointing and Reaching Movements. Lecture Notes in Computer Science, vol. 6192, pp. 211–218. Springer, Berlin (2010)
Chapuis, D., Gassert, R., Gowrishankar, G., Burdet, E., Bleuler, H.: Investigation of a cable transmission for the actuation of MR compatible haptic interfaces. In: Proceedings of Biomedical Robotics and Biomechatronics, pp. 426–431 (2006)
Hribar, A., Koritnik, B., Munih, M.: Phantom haptic device upgrade for use in fMRI. Med. Biol. Eng. Comput. 47, 677–684 (2009)
Massie, T., Salisbury, J.: The phantom haptic interface: a device for probing virtual objects. In: Proceedings of the ASME Winter Annual Meeting, Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, vol. 55(1), pp. 295–300. IOS Press (1994)
Khatib, O., Warren, J., Sapio, V.D., Sentis, L.: Human-like motion from physiologically-based potential energies. In: Advances in Robot Kinematics, pp. 149–163. Springer, The Netherlands (2004)
Sapio, V.D., Warren, J., Khatib, O., Delp, S.: Simulating the task-level control of human motion: a methodology and framework for implementation. Vis. Comput. 25, 289–302 (2006)
Demircan, E., Besier, T., Menon, S., Khatib, O.: Human motion reconstruction and synthesis of human skills. In: Lenarčič, J., Stanisic, M. (eds.) Advances in Robot Kinematics, pp. 283–292. Springer, Berlin (2010)
Evarts, E.V.: Relation of pyramidal tract activity to force exerted during voluntary movement. J. Neurophysiol. 31, 14–27 (1968)
Georgopoulos, A.P., Kalaska, J.F., Caminiti, R., Massey, J.T.: On the relations between the direction of two-dimensional arm movements and cell discharge in primate motor cortex. J. Neurosci. 2, 1527–1537 (1982)
Georgopoulos, A.P., Schwartz, A.B., Kettner, R.E.: Neuronal population coding of movement direction. Science 233, 1416–1419 (1986)
Georgopoulos, A.P., Ashe, J., Smyrnis, N., Taira, M.: The motor cortex and the coding of force. Science 256, 1692–1695 (1992)
Schwartz, A.B., Kettner, R.E., Georgopoulos, A.P.: Primate motor cortex and free arm movements to visual targets in three-dimensional space. I. relations between single cell discharge and direction of movement. J. Neurosci. 8, 2913–2927 (1988)
Georgopoulos, A.P., Caminiti, R., Kalaska, J.F.: Static spatial effects in motor cortex and area 5: quantitative relations in a two-dimensional space. Exp. Brain Res. 54, 446–454 (1984)
Kettner, R.E., Schwartz, A.B., Georgopoulos, A.P.: Primate motor cortex and free arm movements to visual targets in three- dimensional space. iii. positional gradients and population coding of movement direction from various movement origins. J. Neurosci. 8, 2938–2947 (1988)
Cheney, P.D., Fetz, E.E., Palmer, S.S.: Patterns of facilitation and suppression of antagonist forelimb muscles from motor cortex sites in the awake monkey. J. Neurophysiol. 53, 805–820 (1985)
Kakei, S., Hoffman, D.S., Strick, P.L.: Muscle and movement representations in the primary motor cortex. Science 285, 2136–2139 (1999)
Holdefer, R., Miller, L.: Primary motor cortical neurons encode functional muscle synergies. Exp. Brain Res. 146, 233–243 (2002)
Caminiti, R., Johnson, P.B., Urbano, A.: Making arm movements within different parts of space: dynamic aspects in the primate motor cortex. J. Neurosci. 10, 2039–2058 (1990)
Reina, G.A., Moran, D.W., Schwartz, A.B.: On the relationship between joint angular velocity and motor cortical discharge during reaching. J. Neurophysiol. 85, 2576–2589 (2001)
Fu, Q.G., Suarez, J.I., Ebner, T.J.: Neuronal specification of direction and distance during reaching movements in the superior precentral premotor area and primary motor cortex of monkeys. J. Neurophysiol. 70, 2097–2116 (1993)
Churchland, M.M., Santhanam, G., Shenoy, K.V.: Preparatory activity in premotor and motor cortex reflects the speed of the upcoming reach. J. Neurophysiol. 96, 3130–3146 (2006)
Ashe, J., Georgopoulos, A.P.: Movement parameters and neural activity in motor cortex and area 5. Cereb. Cortex 4, 590–600 (1994)
Boynton, G.M., Engel, S.A., Glover, G.H., Heeger, D.J.: Linear systems analysis of functional magnetic resonance imaging in human V1. J. Neurosci. 16, 4207–4221 (1996)
Graziano, M.S.A., Aflalo, T.N.: Mapping behavioral repertoire onto the cortex. Neuron 56, 239–51 (2007)
Delp, S.L., Anderson, F.C., Arnold, A.S.: OpenSim: open-source software to create and analyze dynamic simulations of movement. IEEE Trans. Biomed. Eng. 54, 1940–1950 (2007)
Friston, K.J., Ashburner, J.T., Kiebel, S.J., Nichols, T.E., Penny, W.D.: Statistical Parametric Mapping: The Analysis of Functional Brain Images. Elsevier, London (2006)
Desikan, R.S., Killiany, R.J., et al.: An automated labeling system for subdividing the human cerebral cortex on MRI scans into gyral based regions of interest. Neuroimage 31, 968–980 (2006)
Kay, K., Rokem, A., Winawer, J., Dougherty, R., Wandell, B.: GLMdenoise: a fast, automated technique for denoising task-based fMRI data. Front. Neurosci. 7 (2013)
Lin, A.L., Fox, P.T., Hardies, J., Duong, T.Q., Gao, J.H.: Nonlinear coupling between cerebral blood flow, oxygen consumption, and ATP production in human visual cortex. Proc. Natl. Acad. Sci. 107, 8446–8451 (2010)
Efron, B., Tibshirani, R.: An Introduction to the Bootstrap, vol. 57. Chapman & Hall, New York (1994)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Appendix
Appendix
1.1 MRI Protocol
All fMRI scans were conducted at Stanford University’s Center for Cognitive and Neurobiological Imaging on a GE Discovery MR750 3T MRI scanner, with a thirty-two channel Nova Medical head coil. The scan protocol was gradient echo EPI with a 16 cm field of view sampled at a \(64\times 64\) resolution (\(2.5\times 2.5\times 2.5\) mm\(^3\) voxels), a 1.57 s repetition time, a 28 ms echo time, and a 72\(^\circ \) flip angle. Each scan run was preceded by 2nd-order polynomial shimming and was sandwiched by spiral fieldmap scans (\(2.5\times 2.5\times 5\) mm\(^3\) voxels). Fieldmap scans were conducted within 10 s of each scan run’s start and end. After scanning, the fMRI images were slice time corrected, motion corrected (SPM [44]), spatially undistorted using fieldmaps, and analyzed to compute temporal noise-to-signal.
1.2 fMRI Analysis
Temporal noise-to-signal computations used the median neural response distribution obtained by regressing out a line from each voxel’s time series, computing the absolute value of the difference between successive time points, computing the median of these absolute differences, dividing the result by the mean of the original time series, and then multiplying by 100. Cortex segmentation used Freesurfer’s Desikan-Killiany atlas [45]. Surface registration was done using Freesurfer, and all surface images were plotted using Freeview. Freeview smoothed the surface plots while rendering (2 steps).
1.3 Estimating fMRI Impulse Response Time Series and \(R^2\)
fMRI measures changes in blood oxygenation induced by neural metabolic activity [1, 2], which have a slower time course than neural computation and persist long after sensory stimuli and motor tasks terminate. Such persistent responses cause raw fMRI measurements to overlap in experiments where consecutive task conditions are not be separated by large time-intervals. Separating task conditions by large time-intervals, however, makes fMRI runs very long, which can induce a variety of unwanted artifacts related to MRI scanner calibration drift, neural adaptation, or subject attention lapses, microsleep and exhaustion. Instead, we optimized our experiments to ensure reliable motor task execution [13], which caused fMRI measurements for different task conditions to overlap.
We segregated neural activation for individual tasks using a finite impulse response (FIR) model (implemented using GLMdenoise [46]). The FIR model works by associating each task type with a unique time course and segregates time courses while assuming that overlapping responses sum linearly. fMRI signal linearity, however, is an active area of research [1, 41, 47]. As such, we randomized inter-task delays and randomly ordered tasks, which made the model’s time series match anatomical expectations based on past research (see Figs. 2 and 3; read [6] for an overview). When tasks were closely spaced in time, as with planning and motion, this method was noisy. The parts of planning that overlap with motion are thus less reliable and the confidence interval for the planning time series estimates is larger after motion starts (but still above zero; see Fig. 1).
We computed \(95\,\%\) confidence intervals by bootstrapping [48] runs (400 bootstraps), fitting FIR models to each, and taking the median percentile estimates across the estimated bootstrap time series. Finally, we computed \(R^2\) values for each voxel by comparing the time series variance with the variance after regressing out median FIR model estimates.
1.4 Data Collection Protocols
See [13] for precise specifications of the motion protocol. Subjects executed one practice run inside the MRI scanner, and then executed at least eight scan runs (S1, 10; S2, 8). Each run was 630 s long.
The force and visual perception experiment protocol involved fixed duration stimuli instances with visual, motor, or visual and motor sensory input. The experiment was divided into runs, and each run was divided into blocks. During each block, the subject started with their hand at rest. Next, they were instructed to move their hand into free space. After a randomized delay period of 3–5 s, the subject experienced two randomly selected stimuli instances. Each stimulus instance was 3–5 s long and was separated from the other by a randomized delay 3–5 s. Finally, the subject was required to rest their hands for a random time interval (4–20 s), and then restarted the process. The subject executed four scan runs with multiple blocks. Each run was 459 s long.
Force magnitudes were set to evenly spaced directions along the x-y plane, with a magnitude of 1.2 N. The force vectors used were (1.2, 0.0), (0.0, 1.2), (\(-\)1.2, 0.0), (0.0, \(-\)1.2), (0.85, 0.0), (0.0, 0.85), (\(-\)0.85, 0.0), and (0.0, \(-\)0.85).
1.5 Haptic and Force Measurement Details
Haptic experiments were conducted with Haptic fMRI Interface [12], a three degree-of-freedom fMRI-compatible device. All motions were right handed, and the haptic control rate was 350 Hz. The reaching task spanned the entire workspace (see [13] for more details), but avoided arm motion artifacts [14].
Visual stimuli were displayed on a 30 in. diagonal (76 cm, 16 : 10 aspect ratio) flat panel display custom built by Resonance Technology. Subjects viewed visual stimuli through a dual-mirror setup. The visual distance from screen to mirror-2 is 184.4 cm, from mirror-2 to mirror-1 is 6 cm, and from mirror-1 to the eye is about 15 cm, for a total viewing distance of about 205 cm. The visual field of view is about 30\(^\circ \), making each visual checkerboard square span about one and a half degrees of the visual field. The display has a native resolution of \(2560\times 1600\) but stimuli were displayed at \(1280\times 800\). The display has a 7 ms temporal response, and 10-bit color rendering. The maximum luminance of the display is 329 cd/m\(^2\) (red is 88, green is 117, and blue is 124 cd/m\(^2\)).
Forces were measured using a JR3 85M35A-U560 63N4S force sensor. The raw sensor data was sampled at 1 KHz, resampled to match HFI’s control rate, and was finally filtered using a 75 Hz low pass filter to remove high frequency sensor noise.
1.6 Human Subjects
Subjects were healthy right-handed males with no history of motor disorders: S1, 29y, 185lb, 5\(^\prime \)9\({^\prime }{^\prime }\); S2, 19y, 170lb, 6\(^\prime \)2\({^\prime }{^\prime }\); S3, 21y, 160lb, 5\(^\prime \)8\({^\prime }{^\prime }\). Informed consent was obtained in advance on a protocol approved by the Institutional Review Board (IRB) at Stanford University.
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Menon, S., Ganti, H., Khatib, O. (2016). Using Haptic fMRI to Enable Interactive Motor Neuroimaging Experiments. In: Hsieh, M., Khatib, O., Kumar, V. (eds) Experimental Robotics. Springer Tracts in Advanced Robotics, vol 109. Springer, Cham. https://doi.org/10.1007/978-3-319-23778-7_7
Download citation
DOI: https://doi.org/10.1007/978-3-319-23778-7_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-23777-0
Online ISBN: 978-3-319-23778-7
eBook Packages: EngineeringEngineering (R0)