Abstract
This paper presents a framework for multimodal human-robot interaction. The proposed framework is intended to bring important contributions to the development of human robot interaction to facilitate intuitive programming and to enable easily adapting to changes in robot task without the need of using skilled personnel. The key elements of this system are speech and hand gesture recognition, text programming, and interaction capabilities that allow the user to take over the control of the robot at any given time. Furthermore, our approach is focused on robot tasks. A user can express his/her preference for one or more modalities of interaction so that selected modalities fit user’s personal needs.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Mocan, B., Fulea, M., Brad, S.: Framework for developing a multimodal programming interface used on industrial robots. Robotica Manage. 19(2), 29–33 (2014)
Fulea, M., Popescu, S., Brad, E., Mocan, B., Murar, M.: A literature survey on reconfigurable industrial robotic work cells. In: Proceedings of the 6th International Conference on Robotics ROBOTICS’14. Bucharest (2014)
Mocan, B., Buna, D., Fulea, M., Brad, S.: Increasing the efficiency of robotic manufacturing systems by layout optimization. In: Proceedings of the 6th International Conference on Robotics ROBOTICS’14. Bucharest (2014)
Kongn, J., Zhang, W.Y., Yu, N., Xia, X.J.: Design of human-centric adaptive multimodal interfaces. Int. J. Human-Comput. Stud. 69, 854–869 (2011)
Brad, S., Mocan, B., Brad, E., Fulea, M.: Leading innovation to improve complex process performances by systematic problem analysis with TRIZ, TFC2014, Global Innovation Convention, EPFL, Lausanne, Procedia CIRP. Elsevier, Amsterdam (2014)
Akan, B., Ameri, A., Curuklu, B., Asplund, L.: Intuitive industrial robot programming through incremental multimodal language and augmented reality, 2011. In: IEEE International Conference on Robotics and Automation, pp. 3934–3939. Shanghai (2011)
Mocan, B., Fulea, M., Brad, E., Brad, S.: State-of-the-Art and proposals on reducing energy consumption in the case of industrial robotic systems. In: Proceedings of the 2014 International Conference on Production Research—Regional Conference Africa, Europe and the Middle East, pp. 328–334. Cluj-Napoca, Romania, 1–5 July (2014)
Turk, M.: Multimodal interaction: A review. Pattern Recogn. Lett. 36, 189–195 (2014)
Hirzinger, G., Bals, J., Otter, M., Stelter, J.: The DLR-KUKA success story: Robotics research improves industrial robots. IEEE Robot. Autom. Mag. 12(3), 16–23 (2005)
Mocan, B.: Performance planning of arc welding robotic systems using specific tools for quality planning and systematic introduction of innovation—part I, II, III. J. Calitatea—acces la success 13(127), 80–85 (2012)
Mocan, B., Fulea, M.: Offline Programming of Robotic Arc Welding Systems. Acta Technica Napocensis, S: Appl. Math. Mech. 1(54), 173–178 (2011)
Brick, T., Scheutz, M.: Incremental natural language processing for HRI. In: Proceedings of the ACM/IEEE International Conference on Human-Robot Interaction—HRI ’07, 261–265 (2007)
Brad, S., Fulea, M., Brad, E., Mocan, B.: Smart deployment of demonstrators into successful commercial solutions. In: Proceedings of the 24th CIRP Design Conference, pp. 165–175. Milan, Italy, 14–16 March (2014)
Horst, P., Schreck, G., Willnow, C.: SME-Service networks for cooperative operation of robot installations. In: International Conference Emerging Solutions for Future Manufacturing Systems, pp. 339–346 (2006)
Marin, R., Sanz, P., Nebot, P., Wirz, R.: A multimodal interface to control a robot arm via the web: a case study on remote programming. IEEE Trans. Ind. Electron. 52(6), 1506–1520 (2005)
Dumas, B., Lalanne, D., Oviatt, S.: Multimodal interfaces: A survey of principles, models and frameworks, Human Machine Interaction. Lecture Notes Computer Science, vol. 5440, pp. 3–26. Springer, Berlin (2009)
Van Wassenhove, V., Grant, K.W., Poeppel, D.: Visual speech speeds up the neural processing of auditory speech. In: Proc. Nat. Acad. Sci. 102, 1181–1186 (2005)
Bolt, R.A.: Put-that-there: Voice and gesture at the graphics interface. Int. Conf. Comput. Graph. Interact. Tech. 14, 261–264 (1980)
Xiao, B., Girand, C., Oviatt, S.L.: Multimodal integration patterns in children. In: International Conference on Spoken Language Processing, pp. 629–632 (2002)
Oviatt, S.L., Lunsford, R., Coulston, R.: Individual differences in multimodal integration patterns: What are they and why do they exist? In ACM SIGCHI Conference on Human Factors in Computing Systems, vol. 2, No. 7, 241–249 (2005)
Bohus, D., Horvitz, E.: Facilitating multiparty dialog with gaze, gesture, and speech. In: ACM International Conference on Multimodal Interfaces (ICMIMLMI). Beijing (2010)
Johnston, M., Bangalore, S: Finite-state multimodal parsing and understanding. In: Proceedings of the 18th Conference on Computational Linguistics, pp. 369–375 (2000)
Pires, J.N.: Robotics for small and medium enterprises: control and programming challenges, Industrial Robot (2006)
Fuegen, C., Holzapfel, H., Waibel, A.: Tight coupling of speech recognition and dialog management dialog-context dependent grammar weighting for speech recognition. In: 8th International Conference on Spoken Language Processing, pp. 2–5 (2004)
Schlangen, D., Skantze, G.: A general, abstract model of incremental dialogue processing. In: European Chapter Meeting of the ACL, 710–718 (2009)
Hsiao, K.-Y., Vosoughi, S., Tellex, S., Kubat, R., Roy, D.: Object schemas for responsive robotic language use. In: Proceedings of the 3rd International Conference on Human Robot Interaction—HRI ’08, pp. 231–235 (2008)
Iba, S., Paredis, C.J.J., Khosla, P.K.: Interactive multi-modal robot programming. In: Proceedings 2002 IEEE Intetnational Conference on Robotics and Automation, pp. 161–168 (2002)
McGuire, P.C., Fritsch, J., Steil, J.J., Roethling, F., Fink, G.A., Wachsmuth, S., Sagerer, G., Ritter, H.: Multi-modal human-machine communication for instructing robot grasping tasks. In: International Conference on Intelligent Robots and Systems (IROS), pp. 1082–1089 (2002)
Fang, H., Ong, S.K., Nee, A.Y.-C.: Robot Programming Using Augmented Reality, IEEE, (2009)
Jara, C.A., Candelas, F.A., Gil, P., Fernandez, M., Torres, F.: An augmented reality interface for training robotics through the web. Communication, pp. 189–194 (2005)
Oviatt, S.L.: Ten myths of multimodal interaction. Commun. ACM 42(11), 74–81 (1999)
Marin, R., Sanz, P., Sanchez, J.: A very high level interface to tele-operate a robot via Web including augmented reality. In: Proceedings of IEEE International Conference on Robotics and Automation, (May), pp. 2725–2730 (2002)
Quek, F.: Toward a Vision-Based Hand Gesture Interface, Virtual Reality System Technology Conference, pp. 17–29. Singapore (1994)
Acknowledgments
Support within the project POSDRU/159/1.5/S/137516 PARTING is acknowledged with gratitude.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2016 Springer International Publishing Switzerland
About this paper
Cite this paper
Mocan, B., Fulea, M., Brad, S. (2016). Designing a Multimodal Human-Robot Interaction Interface for an Industrial Robot. In: Borangiu, T. (eds) Advances in Robot Design and Intelligent Control. Advances in Intelligent Systems and Computing, vol 371. Springer, Cham. https://doi.org/10.1007/978-3-319-21290-6_26
Download citation
DOI: https://doi.org/10.1007/978-3-319-21290-6_26
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-21289-0
Online ISBN: 978-3-319-21290-6
eBook Packages: EngineeringEngineering (R0)