skip to main content
10.1145/3615834.3615852acmotherconferencesArticle/Chapter ViewAbstractPublication PagesiwoarConference Proceedingsconference-collections
poster
Public Access

Challenges in Modelling Cooking Task Execution for User Assistance

Published: 11 October 2023 Publication History

Abstract

Executing a complex physical task according to an instruction or a checklist is typical for various fields, such as aviation or healthcare. It is possible that the person is inexperienced or under stress and therefore unable to promptly consult the instruction text for correct execution of the task. To address this problem different works propose the usage of automated assistive systems that guide the user through the task execution. This is also addressed by the Defense Advanced Research Projects Agency (DARPA) Perceptually-enabled Task Guidance (PTG) program, aiming to provide users with augmented reality goggle capable of tracking the state of the user during task execution and to display hints relevant for the task.
In this paper we discuss one important part of this project, namely, the ability to track the state of the user and the environment in order to be able to assist them. We discuss our modelling approach for the development of a probabilistic state tracker, which uses sensor observations to identify the current state and actions of the user, as well as the goal they are pursuing. The paper provides useful insights into the modelling of user behaviour, the challenges associated with modelling multi-user behaviour and how to tackle them.

References

[1]
Chris L. Baker, Rebecca Saxe, and Joshua B. Tenenbaum. 2009. Action understanding as inverse planning. Cognition 113, 3 (2009), 329–349. https://doi.org/10.1016/j.cognition.2009.07.005 Reinforcement learning and higher cognition.
[2]
Jesse Hoey, Thomas Plötz, Dan Jackson, Andrew Monk, Cuong Pham, and Patrick Olivier. 2011. Rapid specification and automated generation of prompting systems to assist people with dementia. Pervasive and Mobile Computing 7, 3 (2011), 299–318. https://doi.org/10.1016/j.pmcj.2010.11.007 Knowledge-Driven Activity Recognition in Intelligent Environments.
[3]
Jesse Hoey, Pascal Poupart, Axel von Bertoldi, Tammy Craig, Craig Boutilier, and Alex Mihailidis. 2010. Automated handwashing assistance for persons with dementia using video and a partially observable Markov decision process. Computer Vision and Image Understanding 114, 5 (2010), 503–519. https://doi.org/10.1016/j.cviu.2009.06.008 Special issue on Intelligent Vision Systems.
[4]
Frank Krüger, Martin Nyolt, Kristina Yordanova, Albert Hein, and Thomas Kirste. 2014. Computational State Space Models for Activity and Intention Recognition. A Feasibility Study. PLOS ONE 9, 11 (11 2014), 1–24. https://doi.org/10.1371/journal.pone.0109381
[5]
Teodor Stoev, Tomasz Sosnowski, and Kristina Yordanova. 2023. A Tool for Automated Generation of Domain Specific Symbolic Models From Texts. In 2023 IEEE International Conference on Pervasive Computing and Communications Workshops and other Affiliated Events (PerCom Workshops). 276–278. https://doi.org/10.1109/PerComWorkshops56833.2023.10150252
[6]
Kristina Yordanova. 2018. Extracting Planning Operators from Instructional Texts for Behaviour Interpretation. In German Conference on Artificial Intelligence. Berlin, Germany, 215–228. https://doi.org/10.1007/978-3-030-00111-7_19
[7]
Kristina Yordanova , Stefan Lüdtke, Samuel Whitehouse, Frank Krüger, Adeline Paiement, Majid Mirmehdi, Ian Craddock, and Thomas Kirste. 2019. Analysing Cooking Behaviour in Home Settings: Towards Health Monitoring. Sensors 19, 3 (2019). https://doi.org/10.3390/s19030646

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
iWOAR '23: Proceedings of the 8th international Workshop on Sensor-Based Activity Recognition and Artificial Intelligence
September 2023
171 pages
ISBN:9798400708169
DOI:10.1145/3615834
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 11 October 2023

Check for updates

Author Tags

  1. activity recognition
  2. guidance
  3. state tracking
  4. user modelling

Qualifiers

  • Poster
  • Research
  • Refereed limited

Funding Sources

Conference

iWOAR 2023

Acceptance Rates

Overall Acceptance Rate 46 of 73 submissions, 63%

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • 0
    Total Citations
  • 62
    Total Downloads
  • Downloads (Last 12 months)47
  • Downloads (Last 6 weeks)16
Reflects downloads up to 22 Jan 2025

Other Metrics

Citations

View Options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Login options

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media