skip to main content
10.1145/3197768.3197776acmotherconferencesArticle/Chapter ViewAbstractPublication PagespetraConference Proceedingsconference-collections
research-article

Programming Intelligent Environments in Natural Language: An Extensible Interactive Approach

Published: 26 June 2018 Publication History

Abstract

The proliferation of Ambient Intelligence (AmI) devices and services and their integration in smart environments creates the need for a simple yet effective way of controlling and communicating with them. Towards that direction, the application of the Trigger -- Action model has attracted a lot of research with many systems and applications having been developed following that approach. This work introduces ParlAmI, a multimodal conversational interface aiming to give its users the ability to determine the behavior of AmI environments, by creating rules using natural language as well as a GUI. The paper describes ParlAmI, its requirements and functionality, and presents the findings of a user-based evaluation which was conducted.

References

[1]
Resonance AI. Resonance | Predicting Human Behaviours. Retrieved October 7, 2017 from https://www.resonanceai.com/
[2]
Tom Bocklisch, Joey Faulker, Nick Pawlowski, and Alan Nichol. 2017. Rasa: Open Source Language Understanding and Dialogue Management. ArXiv Prepr. ArXiv171205181 (2017).
[3]
Tom Brinck, Darren Gergle, and Scott D. Wood. 2001. Usability for the Web: designing Web sites that work. Morgan Kaufmann.
[4]
Michal Gordon and David Harel. 2014. Steps Towards Scenario-Based Programming with a Natural Language Interface. In From Programs to Systems. The Systems perspective in Computing. Springer, 129--144.
[5]
Ting-Hao Kenneth Huang, Amos Azaria, and Jeffrey P. Bigham. 2016. Instructablecrowd: Creating if-then rules via conversations with the crowd. In Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems, 1555--1562.
[6]
Tina Klüwer. 2011. From chatbots to dialog systems. Conversational Agents Nat. Lang. Interact. Tech. Eff. Pract. (2011), 1--22.
[7]
M. Korozi, M. Antona, A. Ntagianta, A. Leonidis, and C. Stephanidis. 2017. LECTORSTUDIO: CREATING INATTENTION ALARMS AND INTERVENTIONS TO REENGAGE THE STUDENTS IN THE EDUCATIONAL PROCESS. In ICERI2017 Proceedings (10th annual International Conference of Education, Research and Innovation), 4486--4495.
[8]
Maria Korozi, Asterios Leonidis, Margherita Antona, and Constantine Stephanidis. 2017. LECTOR: Towards Reengaging Students in the Educational Process Inside Smart Classrooms. In International Conference on Intelligent Human Computer Interaction, 137--149.
[9]
Asterios Leonidis, Dimitrios Arampatzis, Nikolaos Louloudakis, and Constantine Stephanidis. 2017. The AmI-Solertis System: Creating User Experiences in Smart Environments. In Proceedings of the 13th IEEE International Conference on Wireless and Mobile Computing, Networking and Communications.
[10]
Michael McTear, Zoraida Callejas, and David Griol. 2016. Introducing the Conversational Interface. In The Conversational Interface. Springer, 1--7.
[11]
Michael McTear, Zoraida Callejas, and David Griol. 2016. Creating a Conversational Interface Using Chatbot Technology. In The Conversational Interface. Springer, 125--159.
[12]
Mark W. Newman, Ame Elliott, and Trevor F. Smith. 2008. Providing an Integrated User Experience of Networked Media, Devices, and Services through End-User Composition. In Pervasive Computing (Lecture Notes in Computer Science), 213--227.
[13]
Jakob Nielsen. 1994. Usability inspection methods. In Conference companion on Human factors in computing systems, 413--414.
[14]
Chris Quirk, Raymond J. Mooney, and Michel Galley. 2015. Language to Code: Learning Semantic Parsers for If-This-Then-That Recipes. In ACL (1), 878--888.
[15]
P. Remagnino and G. L. Foresti. 2005. Ambient Intelligence: A New Multidisciplinary Paradigm. IEEE Trans. Syst. Man Cybern. - Part Syst. Hum. 35, 1 (January 2005), 1--6.
[16]
Maj Stenmark and Pierre Nugues. 2013. Natural language programming of industrial robots. In Robotics (ISR), 2013 44th International Symposium on, 1--5.
[17]
Biljana L. Risteska Stojkoska and Kire V. Trivodaliev. 2017. A review of Internet of Things for smart home: Challenges and solutions. J. Clean. Prod. 140, (2017), 1454--1464.
[18]
Blase Ur, Elyse McManus, Melwyn Pak Yong Ho, and Michael L. Littman. 2014. Practical Trigger-action Programming in the Smart Home. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14), 803--812.
[19]
Blase Ur, Melwyn Pak Yong Ho, Stephen Brawner, Jiyun Lee, Sarah Mennicken, Noah Picard, Diane Schulze, and Michael L. Littman. 2016. Trigger-Action Programming in the Wild: An Analysis of 200,000 IFTTT Recipes. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16), 3227--3231.
[20]
Marcel Walch, Michael Rietzler, Julia Greim, Florian Schaub, Björn Wiedersheim, and Michael Weber. 2013. homeBLOX: making home automation usable. In Proceedings of the 2013 ACM conference on Pervasive and ubiquitous computing adjunct publication, 295--298.
[21]
B Wilcox. 2011. Chatscript.
[22]
IFTTT. Retrieved October 5, 2017 from https://ifttt.com/
[23]
Zapier. The best apps. Better together. - Zapier. Retrieved October 7, 2017 from https://zapier.com/
[24]
Zipato. Zipato- Home Control and Automation. Retrieved October 7, 2017 from https://www.zipato.com/
[25]
Supermechanical: Twine. Listen to your world. Talk to the web. Retrieved October 6, 2017 from http://supermechanical.com/twine/technical.html
[26]
WigWag. Retrieved October 6, 2017 from https://www.wigwag.com/
[27]
Pandorabots. Retrieved December 19, 2017 from https://www.pandorabots.com/
[28]
Language Understanding with rasa NLU --- rasa NLU 0.10.5 documentation. Retrieved December 22, 2017 from https://rasa-nlu.readthedocs.io/en/latest/index.html#
[29]
spaCy - Industrial-strength Natural Language Processing in Python. Retrieved December 22, 2017 from https://spacy.io/index
[30]
Thinking Aloud: The #1 Usability Tool. Nielsen Norman Group. Retrieved October 31, 2017 from https://www.nngroup.com/articles/thinking-aloud-the-1-usability-tool/
[31]
Why You Only Need to Test with 5 Users. Retrieved from https://www.nngroup.com/articles/why-you-only-need-to-test-with-5-users/

Cited By

View all
  • (2023)Conversational Interfaces in IoT Ecosystems: Where We Are, What Is Still MissingProceedings of the 22nd International Conference on Mobile and Ubiquitous Multimedia10.1145/3626705.3627775(279-293)Online publication date: 3-Dec-2023
  • (2023)Understanding In-Situ Programming for Smart Home AutomationProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35962547:2(1-31)Online publication date: 12-Jun-2023
  • (2021)AdaReNet: Adaptive Reweighted Semi-supervised Active Learning to Accelerate Label AcquisitionProceedings of the 14th PErvasive Technologies Related to Assistive Environments Conference10.1145/3453892.3461321(431-438)Online publication date: 29-Jun-2021
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
PETRA '18: Proceedings of the 11th PErvasive Technologies Related to Assistive Environments Conference
June 2018
591 pages
ISBN:9781450363907
DOI:10.1145/3197768
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

In-Cooperation

  • NSF: National Science Foundation

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 26 June 2018

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. AmI
  2. Ambient Intelligence
  3. Chat-bot
  4. Conversational Interface
  5. Natural Language Programming
  6. Trigger-Action

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Conference

PETRA '18

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)11
  • Downloads (Last 6 weeks)1
Reflects downloads up to 10 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2023)Conversational Interfaces in IoT Ecosystems: Where We Are, What Is Still MissingProceedings of the 22nd International Conference on Mobile and Ubiquitous Multimedia10.1145/3626705.3627775(279-293)Online publication date: 3-Dec-2023
  • (2023)Understanding In-Situ Programming for Smart Home AutomationProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/35962547:2(1-31)Online publication date: 12-Jun-2023
  • (2021)AdaReNet: Adaptive Reweighted Semi-supervised Active Learning to Accelerate Label AcquisitionProceedings of the 14th PErvasive Technologies Related to Assistive Environments Conference10.1145/3453892.3461321(431-438)Online publication date: 29-Jun-2021
  • (2020)Empowering Caregivers to Customizing the Assistive Computing Support of Older Adults - an End-User Domain-Specific ApproachInternational Journal of Human–Computer Interaction10.1080/10447318.2020.175247936:15(1447-1459)Online publication date: 21-Apr-2020
  • (2019)ParlAmI: A Multimodal Approach for Programming Intelligent EnvironmentsTechnologies10.3390/technologies70100117:1(11)Online publication date: 11-Jan-2019
  • (2019)Ambient Intelligence in the Living RoomSensors10.3390/s1922501119:22(5011)Online publication date: 16-Nov-2019
  • (2019)CasandRA: A Screenplay Approach to Dictate the Behavior of Virtual Humans in AmI EnvironmentsHCI International 2019 – Late Breaking Posters10.1007/978-3-030-30712-7_8(57-66)Online publication date: 20-Sep-2019
  • (2019)A Short Survey on Chatbot Technology: Failure in Raising the State of the ArtDistributed Computing and Artificial Intelligence, 16th International Conference10.1007/978-3-030-23887-2_4(28-36)Online publication date: 22-Jun-2019

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media