Xiao Zhang
ABSTRACT
Physiological data are nowadays frequently used to recognize the affective state of subjects while performing different tasks. Measuring cognitive load using simple device plays an important role in everyday life such as intelligent human-computer interaction, physical health monitoring, and mental health monitoring. However, due to the nature of the experiments involving subjects, the obtained data base is often low, making it difficult to train deep learning methods from scratch. In this paper, We conducted a method to recognize the cognitive load based on the Photoplethysmogram(PPG) data and the application of Long Short Term Memory (LSTM). We tested this method on the PPG data from an experiment where 19 subjects were involved in arithmetic calculation tasks of two different cognitive load levels. The method successfully achieve btter accuracy of recognition compared with the traditional machine learning classifiers with features artificially extracted from the image and pre-trained CNN method, 92.3% binary classification accuracy was reached and about 3.8% binary classification accuracy was improved.
- Aamir Arsalan, Syed Muhammad Anwar, and Muhammad Majid. 2022. Mental stress detection using data from wearable and non-wearable sensors: a review. arXiv preprint arXiv:2202.03033 (2022).Google Scholar
- D Berrar. 2018. Cross-validation. Encyclopedia of Bioinformatics and Computational Biology: ABC of Bioinformatics, 1–3, 542–545.Google Scholar
- Karel A Brookhuis and Dick De Waard. 2010. Monitoring drivers’ mental workload in driving simulators using physiological measures. Accident Analysis & Prevention 42, 3 (2010), 898–903.Google ScholarCross Ref
- Roland Brünken, Susan Steinbacher, Jan L Plass, and Detlev Leutner. 2002. Assessment of cognitive load in multimedia learning using dual-task methodology.Experimental psychology 49, 2 (2002), 109.Google Scholar
- Francesca Gasparini, Alessandra Grossi, and Stefania Bandini. 2021. A deep learning approach to recognize cognitive load using ppg signals. In The 14th PErvasive Technologies Related to Assistive Environments Conference. 489–495.Google Scholar
- Luis Cabañero Gómez, Ramón Hervás, Ivan Gonzalez, and Vladimir Villarreal. 2021. Studying the generalisability of cognitive load measured with EEG. Biomedical Signal Processing and Control 70 (2021), 103032.Google ScholarCross Ref
- Shannan Guan, Haiyan Lu, Linchao Zhu, and Gengfa Fang. 2022. AFE-CNN: 3D Skeleton-based Action Recognition with Action Feature Enhancement. Neurocomputing 514 (2022), 256–267.Google ScholarDigital Library
- Joshua Harrison, Kurtuluş İzzetoğlu, Hasan Ayaz, Ben Willems, Sehchang Hah, Ulf Ahlstrom, Hyun Woo, Patricia A Shewokis, Scott C Bunce, and Banu Onaral. 2014. Cognitive workload and learning assessment during the implementation of a next-generation air traffic control technology using functional near-infrared spectroscopy. IEEE Transactions on Human-Machine Systems 44, 4 (2014), 429–440.Google ScholarCross Ref
- Woo-Hyun Hwang, Dong-Hyun Kang, and Deok-Hwan Kim. 2022. Brain lateralisation feature extraction and ant colony optimisation-bidirectional LSTM network model for emotion recognition. IET Signal Processing 16, 1 (2022), 45–61.Google ScholarCross Ref
- Jeff Klingner, Barbara Tversky, and Pat Hanrahan. 2011. Effects of visual and verbal presentation on cognitive load in vigilance, memory, and arithmetic tasks. Psychophysiology 48, 3 (2011), 323–332.Google ScholarCross Ref
- Long Lin and Liang Tan. 2022. Multi-Distributed Speech Emotion Recognition Based on Mel Frequency Cepstogram and Parameter Transfer. Chinese Journal of Electronics 31, 1 (2022), 155–167.Google Scholar
- Yongqiang Lyu, Xiaomin Luo, Jun Zhou, Chun Yu, Congcong Miao, Tong Wang, Yuanchun Shi, and Ken-ichi Kameyama. 2015. Measuring photoplethysmogram-based stress-induced vascular response index to assess cognitive load and stress. In Proceedings of the 33rd annual ACM conference on human factors in computing systems. 857–866.Google ScholarDigital Library
- Nargess Nourbakhsh, Yang Wang, Fang Chen, and Rafael A Calvo. 2012. Using galvanic skin response for cognitive load measurement in arithmetic and reading tasks. In Proceedings of the 24th australian computer-human interaction conference. 420–423.Google ScholarDigital Library
- Sharon Oviatt. 2006. Human-centered design meets cognitive load theory: designing interfaces that help people think. In Proceedings of the 14th ACM international conference on Multimedia. 871–880.Google ScholarDigital Library
- Rachel L Piferi, Keith A Kline, Jarred Younger, and Kathleen A Lawler. 2000. An alternative approach for achieving cardiovascular baseline: viewing an aquatic video. International Journal of Psychophysiology 37, 2 (2000), 207–217.Google ScholarCross Ref
- Cornelia Setz, Bert Arnrich, Johannes Schumm, Roberto La Marca, Gerhard Tröster, and Ulrike Ehlert. 2009. Discriminating stress from cognitive load using a wearable EDA device. IEEE Transactions on information technology in biomedicine 14, 2 (2009), 410–417.Google ScholarDigital Library
- David Sharek and Eric Wiebe. 2012. Embedding secondary tasks in video games to measure real-time cognitive load: An approach to developing adaptive video games. In Proceedings of the Human Factors and Ergonomics Society Annual Meeting, Vol. 56. SAGE Publications Sage CA: Los Angeles, CA, 2147–2151.Google ScholarCross Ref
- Lovejit Singh. 2022. Deep bi-directional LSTM network with CNN features for human emotion recognition in audio-video signals. International Journal of Swarm Intelligence 7, 1 (2022), 110–122.Google ScholarCross Ref
- John Sweller. 1988. Cognitive load during problem solving: Effects on learning. Cognitive science 12, 2 (1988), 257–285.Google Scholar
- Aldert Vrij, Ronald Fisher, Samantha Mann, and Sharon Leal. 2008. A cognitive load approach to lie detection. Journal of Investigative Psychology and Offender Profiling 5, 1-2 (2008), 39–43.Google ScholarCross Ref
- Shiyan Yang, Jonny Kuo, Michael G Lenné, Michael Fitzharris, Timothy Horberry, Kyle Blay, Darren Wood, Christine Mulvihill, and Carine Truche. 2021. The impacts of temporal variation and individual differences in driver cognitive workload on ECG-based detection. Human factors 63, 5 (2021), 772–787.Google Scholar
- Xiao Zhang, Yongqiang Lyu, Xin Hu, Ziyue Hu, Yuanchun Shi, and Hao Yin. 2018. Evaluating photoplethysmogram as a real-time cognitive load assessment during game playing. International Journal of Human–Computer Interaction 34, 8 (2018), 695–706.Google ScholarCross Ref
- Xiao Zhang, Yongqiang Lyu, Xiaomin Luo, Jingyu Zhang, Chun Yu, Hao Yin, and Yuanchun Shi. 2018. Touch sense: Touch screen based mental stress sense. Proceedings of the ACM on interactive, mobile, wearable and ubiquitous technologies 2, 2 (2018), 1–18.Google ScholarDigital Library
- Xiao Zhang, Yongqiang Lyu, Tong Qu, Pengfei Qiu, Xiaomin Luo, Jingyu Zhang, Shunjie Fan, and Yuanchun Shi. 2019. Photoplethysmogram-based Cognitive Load Assessment Using Multi-Feature Fusion Model. ACM Transactions on Applied Perception (TAP) 16, 4 (2019), 1–17.Google ScholarDigital Library
Index Terms
- Photoplethysmogram Based Cognitive Load Recognition Using Lstm
Recommendations
Measuring Photoplethysmogram-Based Stress-Induced Vascular Response Index to Assess Cognitive Load and Stress
CHI '15: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing SystemsQuantitative assessment for cognitive load and mental stress is very important in optimizing human-computer system designs to improve performance and efficiency. Traditional physiological measures, such as heart rate variation (HRV), blood pressure and ...
Photoplethysmogram-based Cognitive Load Assessment Using Multi-Feature Fusion Model
Cognitive load assessment is crucial for user studies and human--computer interaction designs. As a noninvasive and easy-to-use category of measures, current photoplethysmogram- (PPG) based assessment methods rely on single or small-scale predefined ...
Study of skin flow motion pattern using photoplethysmogram
Microcirculatory dysfunction is related to many diseases and occurs long before their clinical manifestation. We used wavelet transform to study the microcirculatory regulatory mechanism in three different groups (18-diabetic, 8-peripheral arterial ...
Comments