Taizhou Chen
ABSTRACT
Designing and creating physical computing system can be challenging for novice user.In this paper, we present FritzBot, an intelligent conversational agent offering assistance for novice users on constructing physical-computing systems through natural-language interaction. We create a lexical circuit-event database based on 152 student reports from the undergraduate physical-computing course in a local art school. The LSTM-CRF network of FrzitBot is trained on that database, and is able to extract the input and the output events from the user's description, and generate the circuit and the code along with the construction guidelines. A user study shows that FritzBot can significantly reduce the construction effort and time spent for novice users on physical-computing task.
- F. Anderson, T. Grossman, and G. Fitzmaurice. 2017. Trigger-Action-Circuits: Leveraging Generative Design to Enable Novices to Design and Build Circuitry. In Proc of UIST'17. ACM, 331--342.Google Scholar
- I. Androutsopoulos, G. D. Ritchie, and P. Thanisch. 1995. Natural language interfaces to databases--an introduction. Natural language engineering 1, 1 (1995), 29--81.Google Scholar
- T. Booth, S. Stumpf, J. Bird, and S. Jones. 2016. Crossed wires: Investigating the problems of end-user developers in a physical computing task. In Proc of CHI '16. ACM, 3485--3497.Google Scholar
- J. Coelho, C. Duarte, P. Biswas, and P. Langdon. 2011. Developing accessible TV applications. In Proc of ASSET'11. ACM, 131--138.Google Scholar
- O. N. N. Fernando, A. D. Cheok, N. Ranasinghe, K. Zhu, C. Edirisinghe, and Y. Y. Cao. 2009. Poetry mix-up: a poetry generating system for cultural communication. In Proc of ACE'09. ACM, 396--399.Google Scholar
- M. S. Hsiao. 2018. Automated Program Synthesis from Object-Oriented Natural Language for Computer Games. In Proc of Controlled Natural Language - CNL 2018. 71--74.Google Scholar
- G. Lample, M. Ballesteros, S. Subramanian, K. Kawakami, and C. Dyer. 2016. Neural architectures for named entity recognition. arXiv preprint arXiv:1603.01360 (2016).Google Scholar
- G. Laput, M. Dontcheva, G. Wilensky, W. Chang, A. Agarwala, J. Linder, and E. Adar. 2013. Pixeltone: A multimodal interface for image editing. In Proc of CHI'13. ACM, 2185--2194.Google Scholar
- T. Lau, J. Cerruti, G. Manzato, M. Bengualid, J. P. Bigham, and J. Nichols. 2010. A conversational interface to web automation. In Proc of UIST'10. ACM, 229--238.Google Scholar
- J. Lo, D. Huang, T. Kuo, C. Sun, J. Gong, T. Seyed, X. Yang, and B. Chen. 2019. AutoFritz: Autocomplete for Prototyping Virtual Breadboard Circuits. In Proc of CHI'19. ACM, 403.Google Scholar
- D. A. Mellisy, L. Buechley, M. Resnick, and B. Hartmann. 2016. Engaging amateurs in the design, fabrication, and assembly of electronic devices. Proc of DIS'16 (2016), 1270--1281.Google ScholarDigital Library
- George A Miller. 1998. WordNet: An electronic lexical database. MIT press.Google Scholar
- Claude Elwood Shannon. 1948. A mathematical theory of communication. Bell system technical journal 27, 3 (1948), 379--423.Google Scholar
- A. Srinivasan, M. Dontcheva, E. Adar, and S. Walker. 2019. Discovering natural language commands in multimodal interfaces. In Proc of IUI'19. ACM, 661--672.Google Scholar
- K. Zhu, H. Nii, O. N. N. Fernando, and A. D. Cheok. 2011. Selective inductive powering system for paper computing. In Proc of ACE'11. ACM, 59.Google Scholar
Index Terms
- Facilitating Physical-Computer System Design through Data-Driven Natural-Language Interaction
Recommendations
Natural Language, Mixed-initiative Personal Assistant Agents
IMCOM '18: Proceedings of the 12th International Conference on Ubiquitous Information Management and CommunicationThe increasing popularity and use of personal voice assistant technologies, such as Siri and Google Now, is driving and expanding progress toward the long-term and lofty goal of using artificial intelligence to build human-computer dialog systems ...
Towards computer-human interaction in natural language
Estonian human-human calls (directory inquiries) are analysed with the further aim to develop a computer-human dialogue system that interacts with a user in natural language. The analysis is based on the Estonian Dialogue Corpus. Linguistic features of ...
Comments