Abstract
BCI (Brain Computer Interface) technology shows great promise in the field of assistive robotics. In particular, severely impaired individuals lacking the use of their hands and arms would benefit greatly from a robotic grasping system that can be controlled by a simple and intuitive BCI. In this paper we describe an end-to-end robotic grasping system that is controlled by only four classified facial EMG signals resulting in robust and stable grasps. A front end vision system is used to identify and register objects to be grasped against a database of models. Once the model is aligned, it can be used in a real-time grasp planning simulator that is controlled through a non-invasive and inexpensive BCI interface in both discrete and continuous modes. The user can control the approach direction through the BCI interface, and can also assist the planner in choosing the best grasp. Once the grasp is planned, a robotic hand/arm system can execute the grasp. We show results in using this system to pick up a variety of objects in real-time, from a number of different approach directions, using facial BCI signals exclusively. We believe this system is a working prototype for a fully automated assistive grasping system.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Artemiadis, P.K., Kyriakopoulos, K.J.: A switching regime model for the EMG-based control of a robot arm. IEEE Transactions on Systems, Man, and Cybernetics 41(1), 53–63 (2011)
Bell, C.J., Shenoy, P., Chalodhorn, R., Rao, R.P.N.: Control of a humanoid robot by a noninvasive brain-computer interface in humans. Journal of Neural Engineering 5(2), 214–220 (2008)
Berenson, D., Srinivasa, S.S., Kuffner, J.: Task Space Regions: A framework for pose-constrained manipulation planning. The International Journal of Robotics Research (2011)
Castellini, C., van der Smagt, P.: Surface EMG in advanced hand prosthetics. Biological Cybernetics 100(1), 35–47 (2009)
Ciocarlie, M.T., Allen, P.K.: Hand posture subspaces for dexterous robotic grasping. The International Journal of Robotics Research 28(7), 851–867 (2009)
Cipriani, C., Zaccone, F., Micera, S., Carrozza, M.: On the Shared Control of an EMG-Controlled Prosthetic Hand: Analysis of User Prosthesis Interaction. IEEE Transactions on Robotics 24(1), 170–184 (2008)
Goldfeder, C., Ciocarlie, M., Peretzman, J., Dang, H., Allen, P.K.: Data-driven grasping with partial sensor data. In: IROS 2009: Proceedings of the 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 1278–1283. IEEE Press, Piscataway (2009)
Gomez-Gil, J., San-Jose-Gonzalez, I., Nicolas-Alonso, L.F., Alonso-Garcia, S.: Steering a Tractor by Means of an EMG-Based Human-Machine Interface. Sensors 11(7), 7110–7126 (2011)
Hazrati, M.K., Erfanian, A.: An on-line BCI for control of hand grasp sequence and holding using adaptive probabilistic neural network. In: Conference Proceedings: Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 1009–1012 (2008)
Ho, N.S.K., Tong, K.Y., Hu, X.L., Fung, K.L., Wei, X.J., Rong, W., Susanto, E.A.: An EMG-driven exoskeleton hand robotic training device on chronic stroke subjects: Task training system for stroke rehabilitation. In: 2011 IEEE International Conference on Rehabilitation Robotics, pp. 1–5. IEEE (2011)
Horki, P., Solis-Escalante, T., Neuper, C., Müller-Putz, G.: Combined motor imagery and SSVEP based BCI control of a 2 DoF artificial upper limb. Medical & Biological Engineering & Computing 49(5), 567–577 (2011)
Bryan, M., Thomas, V., Nicoll, G., Chang, L., Smith, J., Rao, R.: What You Think is What You Get: Brain-Controlled Interfacing for the PR2. Tech. rep., Iros 2011: The PR2 Workshop, San Francisco (2011)
Matrone, G., Cipriani, C., Carrozza, M.C., Magenes, G.: Two-channel real-time EMG control of a dexterous hand prosthesis. In: 2011 5th International IEEE/EMBS Conference on Neural Engineering, pp. 554–557 (2011)
Miller, A.T., Allen, P.K.: Graspit!: A versatile simulator for robotic grasping. IEEE Robotics and Automation Magazine 11, 110–122 (2004)
Müller-Putz, G.R., Scherer, R., Pfurtscheller, G., Rupp, R.: EEG-based neuroprosthesis control: a step towards clinical practice. Neuroscience Letters 382(1-2), 169–174 (2005)
Papazov, C., Burschka, D.: An efficient ransac for 3d object recognition in noisy and occluded scenes. In: Kimmel, R., Klette, R., Sugimoto, A. (eds.) ACCV 2010, Part I. LNCS, vol. 6492, pp. 135–148. Springer, Heidelberg (2011)
Postelnicu, C.C., Talaba, D., Toma, M.I.: Controlling a Robotic Arm by Brainwaves and Eye. IFIP International Federation For Information Processing, pp. 157–164 (2011)
Ranky, G.N., Adamovich, S.: Analysis of a commercial EEG device for the control of a robot arm. In: Proc. IEEE Northeast Bioengineering Conference (NEBEC), New York, NY, pp. 1–2 (2010)
Royer, A.S., Rose, M.L., He, B.: Goal selection versus process control while learning to use a brain-computer interface. Journal of Neural Engineering 8(3), 036,012 (2011)
Sagawa, K., Kimura, O.: Control of robot manipulator using EMG generated from face. In: ICMIT 2005: Control Systems and Robotics, vol. 6042, pp. 604,233-604,233–6 (2005)
Santello, M., Flanders, M., Soechting, J.F.: Patterns of hand motion during grasping and the influence of sensory guidance. The Journal of Neuroscience 22(4), 1426–1435 (2002)
Scherer, R., Friedrich, E.C.V., Allison, B., Pröll, M., Chung, M., Cheung, W., Rao, R.P.N., Neuper, C.: Non-invasive brain-computer interfaces: Enhanced gaming and robotic control. In: Cabestany, J., Rojas, I., Joya, G. (eds.) IWANN 2011, Part I. LNCS, vol. 6691, pp. 362–369. Springer, Heidelberg (2011)
Shenoy, P., Miller, K.J., Crawford, B., Rao, R.N.: Online electromyographic control of a robotic prosthesis. IEEE Transactions on Bio-medical Engineering 55(3), 1128–1135 (2008)
Tavella, M., Leeb, R., Rupp, R., Millan, J.D.R.: Towards natural non-invasive hand neuroprostheses for daily living. In: Conference Proceedings: Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 126–129 (2010)
Tsoli, A., Jenkins, O.C.: 2d subspaces for user-driven robot grasping. In: RSS Workshop on Robot Manipulation: Sensing and Adapting to the Real World, Atlanta, GA (2007)
Vogel, J., Haddadin, S., Simeral, J.D., Stavisky, S.D., Bacher, D., Hochberg, L.R., Donoghue, J.P., van der Smagt, P.: Continuous Control of the DLR Light-weight Robot III by a human with tetraplegia using the BrainGate2 Neural Interface System. In: Khatib, O., Kumar, V., Sukhatme, G. (eds.) Experimental Robotics. STAR, vol. 79, pp. 125–136. Springer, Heidelberg (2012)
Waytowich, N., Henderson, A., Krusienski, D., Cox, D.: Robot application of a brain computer interface to staubli TX40 robots - early stages. In: World Automation Congress (WAC), pp. 1–6 (2010)
Wołczowski, A., Kurzyński, M.: Human-machine interface in bioprosthesis control using EMG signal classification. Expert Systems 27(1), 53–70 (2010)
Yang, D., Zhao, J., Gu, Y., Jiang, L., Liu, H.: EMG pattern recognition and grasping force estimation: Improvement to the myocontrol of multi-DOF prosthetic hands. In: 2009 IEEE/RSJ International Conference on Intelligent Robots and Systems, pp. 516–521. IEEE (2009)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2013 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Weisz, J., Shababo, B., Dong, L., Allen, P.K. (2013). Grasping with Your Face. In: Desai, J., Dudek, G., Khatib, O., Kumar, V. (eds) Experimental Robotics. Springer Tracts in Advanced Robotics, vol 88. Springer, Heidelberg. https://doi.org/10.1007/978-3-319-00065-7_30
Download citation
DOI: https://doi.org/10.1007/978-3-319-00065-7_30
Publisher Name: Springer, Heidelberg
Print ISBN: 978-3-319-00064-0
Online ISBN: 978-3-319-00065-7
eBook Packages: EngineeringEngineering (R0)