skip to main content
10.1145/3453892.3461625acmotherconferencesArticle/Chapter ViewAbstractPublication PagespetraConference Proceedingsconference-collections
research-article

A Deep Learning Approach to Recognize Cognitive Load using PPG Signals

Published: 29 June 2021 Publication History

Abstract

Physiological data are nowadays frequently used to recognize the affective state of subjects while performing different tasks. Automatic recognition of a stressful state as a consequence of a high level of cognitive load is significant to prevent illnesses like depression, anxiety and sleep disorders that are often due to excessive workload. The spread of wearable sensors that are increasingly reliable and comfortable makes them easy to use in day-life activities. However, due to the nature of experiments that involve subjects, the cardinality of the acquired data is often low, making difficult to train deep learning methods from the scratch. In this paper we consider the photopletismography (PPG) that measures the blood volume registered just under the skin, which can be used to obtain the heart rate of the subject. It is well known that PPG data are particularly relevant to detect high level of arousal that is activated by stress. We show that, converting monodimensional photopletismography (PPG) data into bidimensional signals it is possible to apply a pretrained CNN, obtaining deep features that outperform handcrafted ones in classification tasks, especially introducing feature selections strategies to avoid curse of dimensionality.

References

[1]
J-P Antoine, Pierre Carrette, Romain Murenzi, and Bernard Piette. 1993. Image analysis with two-dimensional continuous wavelet transform. Signal processing 31, 3 (1993), 241–272.
[2]
Daniel Berrar. 2019. Cross-validation. Encyclopedia of bioinformatics and computational biology 1 (2019), 542–545.
[3]
Anita Biswas, Monalisa Singha Roy, and Rajarshi Gupta. 2019. Motion artifact reduction from finger photoplethysmogram using discrete wavelet transform. In Recent Trends in Signal and Image Processing. Springer, 89–98.
[4]
Patricia J Bota, Chen Wang, Ana LN Fred, and Hugo Plácido Da Silva. 2019. A review, current challenges, and future possibilities on emotion recognition using machine learning and physiological signals. IEEE Access 7(2019), 140990–141020.
[5]
Adrian Burns, Emer P Doheny, Barry R Greene, Timothy Foran, Daniel Leahy, Karol O’Donovan, and Michael J McGrath. 2010. SHIMMER™: an extensible platform for physiological signal capture. In 2010 annual international conference of the IEEE engineering in medicine and biology. IEEE, 3759–3762.
[6]
Yekta Said Can, Bert Arnrich, and Cem Ersoy. 2019. Stress detection in daily life scenarios using smart phones and wearable sensors: A survey. Journal of biomedical informatics 92 (2019), 103139.
[7]
Silvia Corchs, Elisabetta Fersini, and Francesca Gasparini. 2019. Ensemble learning on visual and textual data for social image emotion classification. International Journal of Machine Learning and Cybernetics 10, 8(2019), 2057–2070.
[8]
Mengxi Dai, Dezhi Zheng, Rui Na, Shuai Wang, and Shuailei Zhang. 2019. EEG classification of motor imagery using a novel deep learning framework. Sensors 19, 3 (2019), 551.
[9]
David L Donoho and Jain M Johnstone. 1994. Ideal spatial adaptation by wavelet shrinkage. biometrika 81, 3 (1994), 425–455.
[10]
P. Du, E. Li, J. Xia, A. Samat, and X. Bai. 2019. Feature and Model Level Fusion of Pretrained CNN for Remote Sensing Scene Classification. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing 12, 8 (2019), 2600–2611. https://doi.org/10.1109/JSTARS.2018.2878037
[11]
Rakkrit Duangsoithong and Terry Windeatt. 2010. Correlation-based and causal feature selection analysis for ensemble classifiers. In IAPR Workshop on Artificial Neural Networks in Pattern Recognition. Springer, 25–36.
[12]
Daniel Joseph France, Richard G Shiavi, Stephen Silverman, Marilyn Silverman, and M Wilkes. 2000. Acoustical properties of speech as indicators of depression and suicidal risk. IEEE transactions on Biomedical Engineering 47, 7 (2000), 829–837.
[13]
Louis Guttman. 1974. The basis for scalogram analysis. Bobbs-Merrill, College Division.
[14]
Matthias Holschneider, Richard Kronland-Martinet, Jean Morlet, and Ph Tchamitchian. 1990. A real-time algorithm for signal analysis with the help of the wavelet transform. In Wavelets. Springer, 286–297.
[15]
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. Advances in neural information processing systems 25 (2012), 1097–1105.
[16]
Chul Min Lee and Shrikanth S Narayanan. 2005. Toward detecting emotions in spoken dialogs. IEEE transactions on speech and audio processing 13, 2(2005), 293–303.
[17]
Wei Li. 2018. Wavelets for electrocardiogram: overview and taxonomy. IEEE Access 7(2018), 25627–25649.
[18]
Yongbo Liang, Zhencheng Chen, Rabab Ward, and Mohamed Elgendi. 2018. Photoplethysmography and deep learning: enhancing hypertension risk stratification. Biosensors 8, 4 (2018), 101.
[19]
J. M. Lilly and S. C. Olhede. 2012. Generalized Morse Wavelets as a Superfamily of Analytic Wavelets. IEEE Transactions on Signal Processing 60, 11 (2012), 6036–6041. https://doi.org/10.1109/TSP.2012.2210890
[20]
Cherie Madarash-Hill and JB Hill. 2004. Enhancing access to IEEE Conference Proceedings: a case study in the application of IEEE Xplore full text and table of contents enhancements. Science & Technology Libraries 24, 3-4 (2004), 389–399.
[21]
Guy P Nason and Bernard W Silverman. 1995. The stationary wavelet transform and some statistical applications. In Wavelets and statistics. Springer, 281–299.
[22]
Aaqib Saeed, Stojan Trajanovski, Maurice Van Keulen, and Jan Van Erp. 2017. Deep physiological arousal detection in a driving simulator using wearable sensors. In 2017 IEEE International Conference on Data Mining Workshops (ICDMW). IEEE, 486–493.
[23]
Ali Sharif Razavian, Hossein Azizpour, Josephine Sullivan, and Stefan Carlsson. 2014. CNN features off-the-shelf: an astounding baseline for recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition workshops. 806–813.
[24]
Siddharth Siddharth, Tzyy-Ping Jung, and Terrence J Sejnowski. 2019. Utilizing deep learning towards multi-modal bio-sensing and vision-based affective computing. IEEE Transactions on Affective Computing(2019).
[25]
Phyllis K Stein, Matthew S Bosner, Robert E Kleiger, and Brooke M Conger. 1994. Heart rate variability: a measure of cardiac autonomic tone. American heart journal 127, 5 (1994), 1376–1381.
[26]
Manida Swangnetr and David B Kaber. 2012. Emotional state classification in patient–robot interaction using wavelet analysis and statistics-based feature selection. IEEE Transactions on Human-Machine Systems 43, 1 (2012), 63–75.
[27]
Terumi Umematsu, Akane Sano, Sara Taylor, and Rosalind W Picard. 2019. Improving students’ daily life stress forecasting using lstm neural networks. In 2019 IEEE EMBS International Conference on Biomedical & Health Informatics (BHI). IEEE, 1–4.
[28]
Ryan J Urbanowicz, Melissa Meeker, William La Cava, Randal S Olson, and Jason H Moore. 2018. Relief-based feature selection: Introduction and review. Journal of biomedical informatics 85 (2018), 189–203.

Cited By

View all
  • (2025)A Unified Biosensor–Vision Multi-Modal Transformer network for emotion recognitionBiomedical Signal Processing and Control10.1016/j.bspc.2024.107232102(107232)Online publication date: Apr-2025
  • (2024)Remote Heart Rate Monitoring in Smart Environments From Videos With Self-Supervised PretrainingIEEE Internet of Things Journal10.1109/JIOT.2023.332762311:6(10279-10294)Online publication date: 15-Mar-2024
  • (2023)Privacy-Preserving Remote Heart Rate Estimation from Facial Videos2023 IEEE International Conference on Systems, Man, and Cybernetics (SMC)10.1109/SMC53992.2023.10394350(706-712)Online publication date: 1-Oct-2023
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences
PETRA '21: Proceedings of the 14th PErvasive Technologies Related to Assistive Environments Conference
June 2021
593 pages
ISBN:9781450387927
DOI:10.1145/3453892
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 June 2021

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. CNN
  2. Photopletysmography
  3. cognitive load
  4. deep features
  5. handcrafted features

Qualifiers

  • Research-article
  • Research
  • Refereed limited

Funding Sources

  • cariplo

Conference

PETRA '21

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)79
  • Downloads (Last 6 weeks)8
Reflects downloads up to 27 Feb 2025

Other Metrics

Citations

Cited By

View all
  • (2025)A Unified Biosensor–Vision Multi-Modal Transformer network for emotion recognitionBiomedical Signal Processing and Control10.1016/j.bspc.2024.107232102(107232)Online publication date: Apr-2025
  • (2024)Remote Heart Rate Monitoring in Smart Environments From Videos With Self-Supervised PretrainingIEEE Internet of Things Journal10.1109/JIOT.2023.332762311:6(10279-10294)Online publication date: 15-Mar-2024
  • (2023)Privacy-Preserving Remote Heart Rate Estimation from Facial Videos2023 IEEE International Conference on Systems, Man, and Cybernetics (SMC)10.1109/SMC53992.2023.10394350(706-712)Online publication date: 1-Oct-2023
  • (2023)Human-robot teaming in construction: Evaluative safety training through the integration of immersive technologies and wearable physiological sensingSafety Science10.1016/j.ssci.2022.106019159(106019)Online publication date: Mar-2023
  • (2022)Machine Learning Based Real-Time Diagnosis of Mental Stress Using PhotoplethysmographyJournal of Biomimetics, Biomaterials and Biomedical Engineering10.4028/p-01r9mn55(154-167)Online publication date: 28-Mar-2022
  • (2022)Personalized PPG Normalization Based on Subject Heartbeat in Resting State ConditionSignals10.3390/signals30200163:2(249-265)Online publication date: 18-Apr-2022
  • (2022)Survey on Emotion Sensing Using Mobile DevicesIEEE Transactions on Affective Computing10.1109/TAFFC.2022.322048414:4(2678-2696)Online publication date: 8-Nov-2022
  • (2022)EMD-based Features for Cognitive Load and Stress Assessment from PPG Signals2021 International Conference on Biomedical Innovations and Applications (BIA)10.1109/BIA52594.2022.9831344(62-65)Online publication date: 2-Jun-2022
  • (2021)The Application of Deep Learning Algorithms for PPG Signal Processing and ClassificationComputers10.3390/computers1012015810:12(158)Online publication date: 25-Nov-2021

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Figures

Tables

Media

Share

Share

Share this Publication link

Share on social media