End-User Programming of Manipulator Robots in Situated Tangible Programming Paradigm
Pages 319 - 320
Abstract
While the cost of creating robots is declining, deploying them in industry remains expensive. Widespread use of robots, particularly in smaller industries, is more easily realized if robot programming is accessible to non-programmers. Our research explores techniques to lower the barrier to robot programming. In one such attempt, we propose situated tangible robot programming to program a robot by placing specially designed tangible blocks in its workspace. These blocks are used for annotating objects, locations, or regions, and specifying actions and their ordering. The robot compiles a program by detecting blocks and objects in the environment and grouping them into instructions by solving constraints. We designed a preliminary tangible language and blocks and evaluated the intuitiveness and learnability of the approach. Our user studies provide evidence for the promise of situated tangible programming and identify the challenges to address. In addition to improving the block design and extending the language, we are planning to integrate tangible programming into a holistic ecosystem of a programming environment in future.
References
[1]
Universal robots user manual. Universal robots website. https://universal-robots.com/media/8704/ur5_user_manual_gb.pdf, 2014.
[2]
B. Akgun, M. Cakmak, J. W. Yoo, and A. L. Thomaz. Trajectories and keyframes for kinesthetic teaching: A human-robot interaction perspective. In Proceedings of the seventh annual ACM/IEEE international conference on Human-Robot Interaction, pages 391--398, 2012.
[3]
S. Alexamdrova, M. Cakmak, K. Hsaio, and L. Takayama. Robot programming by demonstration with interactive action visualizations. In Robotics: Science and Systems (RSS), 2014.
[4]
S. Alexandrova, Z. Tatlock, and M. Cakmak. Roboflow: A flow-based visual programming language for mobile manipulation tasks. In 2015 IEEE International Conference on Robotics and Automation (ICRA), pages 5537--5544. IEEE, 2015.
[5]
D. Anderson, J. L. Frankel, J. Marks, D. Leigh, E. Sullivan, J. Yedidia, and K. Ryall. Building virtual structures with physical blocks. In Proceedings of the 12th annual ACM symposium on User interface software and technology, pages 71--72. ACM, 1999.
[6]
M. Burnett. Visual programming. Wiley Encyclopedia of Electrical and Electronics Engineering, 1999.
[7]
R. Cantrell, P. Schermerhorn, and M. Scheutz. Learning actions from human-robot dialogues. In 2011 RO-MAN, pages 125--130. IEEE, 2011.
[8]
R. Fang, M. Doering, and J. Y. Chai. Embodied collaborative referring expression generation in situated human-robot interaction. In Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction, pages 271--278. ACM, 2015.
[9]
K. Fischer, F. Kirstein, L. C. Jensen, N. Kr, K. Kukli, M. V. aus der Wieschen, T. R. Savarimuthu, et al. A comparison of types of robot control for programming by demonstration. In 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pages 213--220. IEEE, 2016.
[10]
D. Gallardo, C. F. Julià, and S. Jorda. Turtan: A tangible programming language for creative exploration. In Tabletop, pages 89--92. Citeseer, 2008.
[11]
D. F. Glas, T. Kanda, and H. Ishiguro. Human-robot interaction design using interaction composer: Eight years of lessons learned. In The Eleventh ACM/IEEE International Conference on Human Robot Interation, pages 303--310. IEEE Press, 2016.
[12]
M. S. Horn and R. J. Jacob. Designing tangible programming languages for classroom use. In Proceedings of the 1st international conference on Tangible and embedded interaction, pages 159--162. ACM, 2007.
[13]
J. Huang, T. Lau, and M. Cakmak. Design and evaluation of a rapid programming system for service robots. In 2016 11th ACM/IEEE International Conference on Human-Robot Interaction (HRI), pages 295--302. IEEE, 2016.
[14]
T. Kollar, S. Tellex, D. Roy, and N. Roy. Toward understanding natural language directions. In Human-Robot Interaction (HRI), 2010 5th ACM/IEEE International Conference on, pages 259--266. IEEE, 2010.
[15]
S. Lauria, G. Bugmann, T. Kyriacou, and E. Klein. Mobile robot programming using natural language. Robotics and Autonomous Systems, 38(3):171--181, 2002.
[16]
T. Lozano-Perez. Robot programming. Proceedings of the IEEE, 71(7):821--841, 1983.
[17]
M. Luria, G. Hoffman, B. Megidish, O. Zuckerman, and S. Park. Designing vyo, a robotic smart home assistant: Bridging the gap between device and social agent. In Robot and Human Interactive Communication (RO-MAN), 2016 25th IEEE International Symposium on, pages 1019--1025. IEEE, 2016.
[18]
C. Matuszek, L. Bo, L. Zettlemoyer, and D. Fox. Learning from unscripted deictic gesture and language for human-robot interactions. In AAAI, pages 2556--2563, 2014.
[19]
C. Matuszek, D. Fox, and K. Koscher. Following directions using statistical machine translation. In Proceedings of the 5th ACM/IEEE international conference on Human-robot interaction, pages 251--258. IEEE Press, 2010.
[20]
S. Mohn and J. Laird. Learning goal-oriented hierarchical tasks from situated interactive instruction. In Proceedings of the The Twenty-eighth National Conference on Artificial Intelligence (AAAI), 2014.
[21]
A. Mohseni-Kabir, C. Rich, S. Chernova, C. L. Sidner, and D. Miller. Interactive hierarchical task learning from a single demonstration. In Proceedings of the Tenth Annual ACM/IEEE International Conference on Human-Robot Interaction, pages 205--212. ACM, 2015.
[22]
H. Newton-Dunn, H. Nakano, and J. Gibson. Block jam: a tangible interface for interactive music. In Proceedings of the 2003 conference on New interfaces for musical expression, pages 170--177. National University of Singapore, 2003.
[23]
S. Niekum, S. Chitta, A. Barto, B. Marthi, and S. Osentoski. Incremental semantically grounded learning from demonstration. In Robotics: Science and Systems, volume 9, 2013.
[24]
S. Niekum, S. Osentoski, G. Konidaris, and A. G. Barto. Learning and generalization of complex tasks from unstructured demonstrations. In 2012 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pages 5239--5246, 2012.
[25]
T. Sapounidis, S. Demetriadis, and I. Stamelos. Evaluating children performance with graphical and tangible robot programming tools. Personal and Ubiquitous Computing, 19(1):225--237, 2015.
[26]
E. Schweikardt and M. D. Gross. roblocks: a robotic construction kit for mathematics and science education. In Proceedings of the 8th international conference on Multimodal interfaces, pages 72--75. ACM, 2006.
[27]
L. She, Y. Cheng, J. Y. Chai, Y. Jia, S. Yang, and N. Xi. Teaching robots new actions through natural language instructions. In The 23rd IEEE International Symposium on Robot and Human Interactive Communication, pages 868--873. IEEE, 2014.
[28]
A. Sipitakiat and N. Nusen. Robo-blocks: designing debugging abilities in a tangible programming system for early primary school children. In Proceedings of the 11th International Conference on Interaction Design and Children, pages 98--105. ACM, 2012.
[29]
M. Tykal, A. Montebelli, and V. Kyrki. Incrementally assisted kinesthetic teaching for programming by demonstration. In The Eleventh ACM/IEEE International Conference on Human Robot Interation, pages 205--212. IEEE Press, 2016.
[30]
S. Zhao, K. Nakamura, K. Ishii, and T. Igarashi. Magic cards: a paper tag interface for implicit robot control. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pages 173--182. ACM, 2009.
Index Terms
- End-User Programming of Manipulator Robots in Situated Tangible Programming Paradigm
Recommendations
RobotIST: Interactive Situated Tangible Robot Programming
SUI '18: Proceedings of the 2018 ACM Symposium on Spatial User InteractionSituated tangible robot programming allows programmers to reference parts of the workspace relevant to the task by indicating objects, locations, and regions of interest using tangible blocks. While it takes advantage of situatedness compared to ...
A Survey on End-User Robot Programming
As robots interact with a broader range of end-users, end-user robot programming has helped democratize robot programming by empowering end-users who may not have experience in robot programming to customize robots to meet their individual contextual ...
Situated Tangible Robot Programming
HRI '17: Proceedings of the 2017 ACM/IEEE International Conference on Human-Robot InteractionThis paper introduces situated tangible robot programming, whereby a robot is programmed by placing specially designed tangible "blocks" in its workspace. These blocks are used for annotating objects, locations, or regions, and specifying actions and ...
Comments
Information & Contributors
Information
Published In
March 2018
431 pages
ISBN:9781450356152
DOI:10.1145/3173386
- General Chairs:
- Takayuki Kanda,
- Selma Ŝabanović,
- Program Chairs:
- Guy Hoffman,
- Adriana Tapus
Copyright © 2018 Owner/Author.
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.
Sponsors
- SIGAI: ACM Special Interest Group on Artificial Intelligence
- SIGCHI: ACM Special Interest Group on Computer-Human Interaction
- IEEE-RAS: Robotics and Automation
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 01 March 2018
Check for updates
Author Tags
Qualifiers
- Abstract
Funding Sources
Conference
HRI '18: ACM/IEEE International Conference on Human-Robot Interaction
March 5 - 8, 2018
IL, Chicago, USA
Acceptance Rates
HRI '18 Paper Acceptance Rate 49 of 206 submissions, 24%;
Overall Acceptance Rate 192 of 519 submissions, 37%
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- 0Total Citations
- 310Total Downloads
- Downloads (Last 12 months)62
- Downloads (Last 6 weeks)9
Reflects downloads up to 02 Mar 2025
Other Metrics
Citations
Cited By
View allView Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign in