abstract

Cognitive Aid: Task Assistance Based On Mental Workload Estimation

Authors:

Namrata Srivastava,

Tilman Dingler,

Jorge Goncalves,

Eduardo VellosoAuthors Info & Claims

CHI EA '19: Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems

Paper No.: LBW2315, Pages 1 - 6

https://doi.org/10.1145/3290607.3313010

Published: 02 May 2019 Publication History

Abstract

In this work, we evaluate the potential of using wearable non-contact (infrared) thermal sensors through a user study (N=12) to measure mental workload. Our results indicate the possibility of mental workload estimation through the temperature changes detected using the prototype as participants perform two task variants with increasing difficulty levels. While the sensor accuracy and the design of the prototype can be further improved, the prototype showed the potential of building AR-based systems with cognitive aid technology for ubiquitous task assistance from the changes in mental workload demands. As such, we demonstrate our next steps by integrating our prototype into an existing AR headset (i.e. Microsoft HoloLens).

References

[1]

Yomna Abdelrahman, Eduardo Velloso, Tilman Dingler, Albrecht Schmidt, and Frank Vetere. 2017. Cognitive heat: exploring the usage of thermal imaging to unobtrusively estimate cognitive load. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies 1, 3 (2017), 33.

Digital Library

[2]

Jackson Beatty and Brennis Lucero-Wagoner. 2000. The pupillary system. Handbook of psychophysiology 2, 142--162 (2000).

[3]

Leonardo Bonanni, Chia-Hsun Lee, and Ted Selker. 2005. Attention-based Design of Augmented Reality Interfaces. In CHI '05 Extended Abstracts on Human Factors in Computing Systems (CHI EA '05). ACM, New York, NY, USA, 1228--1231.

Digital Library

[4]

Roland Brunken, Jan L Plass, and Detlev Leutner. 2003. Direct measurement of cognitive load in multimedia learning. Educational psychologist 38, 1 (2003), 53--61.

[5]

Douglas C Engelbart. 2001. Augmenting human intellect: a conceptual framework (1962). PACKER, Randall and JORDAN, Ken. Multimedia. From Wagner to Virtual Reality. New York: WW Norton & Company (2001), 64--90.

[6]

Stephen H Fairclough. 2009. Fundamentals of physiological computing. Interacting with computers 21, 1--2 (2009), 133--145.

Digital Library

[7]

Rudolph Flesch. 1948. A new readability yardstick. Journal of applied psychology 32, 3 (1948), 221.

[8]

Lynda Gerry, Barrett Ens, Adam Drogemuller, Bruce Thomas, and Mark Billinghurst. {n. d.}. Levity: A Virtual Reality System that Responds to Cognitive Load. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, LBW610.

Digital Library

[9]

Martijn Haak, Steven Bos, Sacha Panic, and LJM Rothkrantz. 2009. Detecting stress using eye blinks and brain activity from EEG signals. Proceeding of the 1st driver car interaction and interface (DCII 2008) (2009), 35--60.

[10]

SD Jenkins and RDH Brown. 2014. A correlational analysis of human cognitive activity using Infrared Thermography of the supraorbital region, frontal EEG and self-report of core affective state. QIRT.

[11]

Calvin KL Or and Vincent G Duffy. 2007. Development of a facial skin temperature-based methodology for non-intrusive mental workload measurement. Occupational Ergonomics 7, 2 (2007), 83--94.

[12]

Fred G Paas. 1992. Training strategies for attaining transfer of problem-solving skill in statistics: A cognitive-load approach. Journal of educational psychology 84, 4 (1992), 429.

[13]

Rosalind Wright Picard et al. 1995. Affective computing. (1995).

[14]

John Stemberger, Robert S Allison, and Thomas Schnell. 2010. Thermal imaging as a way to classify cognitive workload. In Computer and Robot Vision (CRV), 2010 Canadian Conference on. IEEE, 231--238.

Digital Library

[15]

J Ridley Stroop. 1935. Studies of interference in serial verbal reactions. Journal of experimental psychology 18, 6 (1935), 643.

[16]

John Sweller. 1988. Cognitive load during problem solving: Effects on learning. Cognitive science 12, 2 (1988), 257--285.

[17]

John Sweller, Jeroen JG Van Merrienboer, and Fred GWC Paas. 1998. Cognitive architecture and instructional design. Educational psychology review 10, 3 (1998), 251--296.

[18]

Robert B Zajonc, Sheila T Murphy, and Marita Inglehart. 1989. Feeling and facial efference: implications of the vascular theory of emotion. Psychological review 96, 3 (1989), 395.

Cited By

Srivastava NNawaz SLodge JVelloso EErfani SBailey JRensing CDrachsler HKovanović VPinkwart NScheffel MVerbert K(2020)Exploring the usage of thermal imaging for understanding video lecture designs and students' experiencesProceedings of the Tenth International Conference on Learning Analytics & Knowledge10.1145/3375462.3375514(250-259)Online publication date: 23-Mar-2020
https://dl.acm.org/doi/10.1145/3375462.3375514
Babaei ESrivastava NNewn JZhou QDingler TVelloso EBernhaupt RMueller FVerweij DAndres JMcGrenere JCockburn AAvellino IGoguey ABjørn PZhao SSamson BKocielnik R(2020)Faces of Focus: A Study on the Facial Cues of Attentional StatesProceedings of the 2020 CHI Conference on Human Factors in Computing Systems10.1145/3313831.3376566(1-13)Online publication date: 21-Apr-2020
https://dl.acm.org/doi/10.1145/3313831.3376566
Abdelrahman YKhan ANewn JVelloso ESafwat SBailey JBulling AVetere FSchmidt A(2019)Classifying Attention Types with Thermal Imaging and Eye TrackingProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/33512273:3(1-27)Online publication date: 9-Sep-2019
https://dl.acm.org/doi/10.1145/3351227
Show More Cited By

Recommendations

Cognitive Heat: Exploring the Usage of Thermal Imaging to Unobtrusively Estimate Cognitive Load

Current digital systems are largely blind to users’ cognitive states. Systems that adapt to users’ states show great potential for augmenting cognition and for creating novel user experiences. However, most approaches for sensing cognitive states, and ...
A Survey on Measuring Cognitive Workload in Human-Computer Interaction
The ever-increasing number of computing devices around us results in more and more systems competing for our attention, making cognitive workload a crucial factor for the user experience of human-computer interfaces. Research in Human-Computer Interaction ...
One does not Simply RSVP: Mental Workload to Select Speed Reading Parameters using Electroencephalography
CHI '20: Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems

Rapid Serial Visual Presentation (RSVP) has gained popularity as a method for presenting text on wearable devices with limited screen space. Nonetheless, it remains unclear how to calibrate RSVP display parameters, such as spatial alignments or ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI EA '19: Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems

May 2019

3673 pages

ISBN:9781450359719

DOI:10.1145/3290607

General Chairs:
Stephen Brewster
University of Glasgow, Scotland, UK
,
Geraldine Fitzpatrick
TU Wien, Austria
,
Program Chairs:
Anna Cox
University College London, UK
,
Vassilis Kostakos
University of Melbourne, Australia

Copyright © 2019 Owner/Author.

Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 May 2019

Check for updates

Author Tags

Qualifiers

Abstract

Funding Sources

Australian Research Council Discovery Early Career Award

Conference

CHI '19

Sponsor:

SIGCHI

CHI '19: CHI Conference on Human Factors in Computing Systems

May 4 - 9, 2019

Glasgow, Scotland Uk

Acceptance Rates

Overall Acceptance Rate 6,164 of 23,696 submissions, 26%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

5
Total Citations
View Citations
369
Total Downloads

Downloads (Last 12 months)20
Downloads (Last 6 weeks)5

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Srivastava NNawaz SLodge JVelloso EErfani SBailey JRensing CDrachsler HKovanović VPinkwart NScheffel MVerbert K(2020)Exploring the usage of thermal imaging for understanding video lecture designs and students' experiencesProceedings of the Tenth International Conference on Learning Analytics & Knowledge10.1145/3375462.3375514(250-259)Online publication date: 23-Mar-2020
https://dl.acm.org/doi/10.1145/3375462.3375514
Babaei ESrivastava NNewn JZhou QDingler TVelloso EBernhaupt RMueller FVerweij DAndres JMcGrenere JCockburn AAvellino IGoguey ABjørn PZhao SSamson BKocielnik R(2020)Faces of Focus: A Study on the Facial Cues of Attentional StatesProceedings of the 2020 CHI Conference on Human Factors in Computing Systems10.1145/3313831.3376566(1-13)Online publication date: 21-Apr-2020
https://dl.acm.org/doi/10.1145/3313831.3376566
Abdelrahman YKhan ANewn JVelloso ESafwat SBailey JBulling AVetere FSchmidt A(2019)Classifying Attention Types with Thermal Imaging and Eye TrackingProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/33512273:3(1-27)Online publication date: 9-Sep-2019
https://dl.acm.org/doi/10.1145/3351227
Zhou QNewn JTag BLee HWang CVelloso EHarle RFarrahi KLane N(2019)Ubiquitous smart eyewear interactions using implicit sensing and unobtrusive information outputAdjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers10.1145/3341162.3348392(661-666)Online publication date: 9-Sep-2019
https://dl.acm.org/doi/10.1145/3341162.3348392
Newn JTag BSingh RVelloso EVetere FHarle RFarrahi KLane N(2019)AI-mediated gaze-based intention recognition for smart eyewearAdjunct Proceedings of the 2019 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2019 ACM International Symposium on Wearable Computers10.1145/3341162.3348387(637-642)Online publication date: 9-Sep-2019
https://dl.acm.org/doi/10.1145/3341162.3348387

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten