Skip to main content
SpringerLink
Log in

Machine learning approaches for boredom classification using EEG

  • Original Research
  • Published:
Journal of Ambient Intelligence and Humanized Computing Aims and scope Submit manuscript

Abstract

Recently, commercial physiological sensors and computing devices have become cheaper and more accessible, while computer systems have become increasingly aware of their contexts, including but not limited to users’ emotions. Consequently, many studies on emotion recognition have been conducted. However, boredom has received relatively little attention as a target emotion due to its diverse nature. Moreover, only a few researchers have tried classifying boredom using electroencephalogram (EEG). In this study, to perform this classification, we first reviewed studies that tried classifying emotions using EEG. Further, we designed and executed an experiment, which used a video stimulus to evoke boredom and non-boredom, and collected EEG data from 28 Korean adult participants. After collecting the data, we extracted its absolute band power, normalized absolute band power, differential entropy, differential asymmetry, and rational asymmetry using EEG, and trained these on three machine learning algorithms: support vector machine, random forest, and k-nearest neighbors (k-NN). We validated the performance of each training model with 10-fold cross validation. As a result, we achieved the highest accuracy of 86.73% using k-NN. The findings of this study can be of interest to researchers working on emotion recognition, physiological signal processing, machine learning, and emotion-aware system development.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  • Aha DW, Kibler D, Albert MK (1991) Instance-based learning algorithms. Mach Learn 6(1):37–66. https://doi.org/10.1007/BF00153759

    Article  Google Scholar 

  • van der Akker J, Keursten P, Plomp T (1992) The integration of computer use in education. Int J Educ Res 17(1):65–76

    Article  Google Scholar 

  • Aksoy S, Haralick RM (2001) Feature normalization and likelihood-based similarity measures for image retrieval. Pattern Recogn Lett 22(5):563–582

    Article  MATH  Google Scholar 

  • Alm CO, Roth D, Sproat R (2005) Emotions from text: machine learning for text-based emotion prediction. In: Proceedings of the conference on human language technology and Empirical methods in natural language processing, association for computational Linguistics, Stroudsburg, PA, USA, HLT ’05, pp 579–586, https://doi.org/10.3115/1220575.1220648

  • Altman NS (1992) An introduction to kernel and nearest-neighbor nonparametric regression. Am Stat 46(3):175–185. https://doi.org/10.1080/00031305.1992.10475879

    Article  MathSciNet  Google Scholar 

  • Baker R, D’Mello S, Rodrigo M, Graesser A (2010) Better to be frustrated than bored: the incidence and persistence of affect during interactions with three different computer-based learning environments. Int J Hum Comput Stud 68(4):223–241

    Article  Google Scholar 

  • Barandiaran I (1998) The random subspace method for constructing decision forests. IEEE Trans Pattern Anal Mach Intell 20(8):56

    Google Scholar 

  • Baumgartner T, Esslen M, Jäncke L (2006) From emotion perception to emotion experience: emotions evoked by pictures and classical music. Int J Psychophysiol 60(1):34–43. https://doi.org/10.1016/j.ijpsycho.2005.04.007

    Article  Google Scholar 

  • Bechara A, Damasio H, Damasio AR (2000) Emotion, decision making and the orbitofrontal cortex. Cerebral Cortex 10(3):295–307. https://doi.org/10.1093/cercor/10.3.295

    Article  Google Scholar 

  • Bednarik R, Vrzakova H, Hradis M (2012) What do you want to do next: a novel approach for intent prediction in gaze-based interaction. In: Proceedings of the symposium on eye tracking research and applications, ACM, pp 83–90

  • Bereiter C (2002) Education and mind in the knowledge age. Routledge, Abingdon

    Google Scholar 

  • Bruce BC, Rubin A (1993) Electronic quills a situated evaluation of using computers for writing in classrooms. Routledge, Abingdon

    Google Scholar 

  • Busso C, Deng Z, Yildirim S, Bulut M, Lee CM, Kazemzadeh A, Lee S, Neumann U, Narayanan S (2004) Analysis of emotion recognition using facial expressions, speech and multimodal information. In: Proceedings of the 6th international conference on Multimodal interfaces—ICMI ’04, https://doi.org/10.1145/1027933.1027968,arXiv:1011.1669v3

  • Castellano G, Kessous L, Caridakis G (2008) Emotion recognition through multiple modalities: face, body gesture, speech. Affect and emotion in human-computer interaction. Springer, Berlin, pp 92–103

    Google Scholar 

  • Chanel G, Kronegg J, Grandjean D, Pun T (2006) Emotion assessment: arousal evaluation using EEG’s and peripheral physiological signals. In: Gunsel B, Jain AK, Tekalp AM, Sankur B (eds) Multimedia content representation, classification and security. MRCS 2006. Lecture notes in computer science, vol 4105. Springer, Berlin, Heidelberg

    Google Scholar 

  • Chi TS, Yeh LY, Hsu CC (2012) Robust emotion recognition by spectro-temporal modulation statistic features. J Ambient Intell Hum Comput 3(1):47–60. https://doi.org/10.1007/s12652-011-0088-5

    Article  Google Scholar 

  • Chuang ZJ, Wu Ch (2004) Multi-modal emotion recognition from speech and text. J Comput Linguist Chin Lang Process 9(2):45–62

    Google Scholar 

  • Cole M (1998) Cultural psychology: a once and future discipline. Harvard University Press, Cambridge

    Google Scholar 

  • Davidson RJ (1992) Anterior cerebral asymmetry and the nature of emotion. Brain Cogn 20(1):125–151

    Article  MathSciNet  Google Scholar 

  • Davidson RJ, Fox NA (1982) Asymmetrical brain activity discriminates between positive and negative affective stimuli in human infants. Science 218(4578):1235–1237

    Article  Google Scholar 

  • Eastwood JD, Frischen A, Fenske MJ, Smilek D (2012) The unengaged mind defining boredom in terms of attention. Perspect Psychol Sci 7(5):482–495. https://doi.org/10.1177/1745691612456044

    Article  Google Scholar 

  • Fagerberg P, Ståhl A, Höök K (2004) EMoto: emotionally engaging interaction. Pers Ubiquitous Comput 8(5):377–381

    Article  Google Scholar 

  • Fahlman SA, Mercer-Lynn KB, Flora DB, Eastwood JD (2013) Development and validation of the multidimensional state boredom scale. Assessment 20(1):68–85. https://doi.org/10.1177/1073191111421303

    Article  Google Scholar 

  • Feldman L (1995) Variations in the circumplex structure of mood. Personal Soc Psychol Bull 21(8):806–817

    Article  Google Scholar 

  • Giakoumis D, Vogiannou A, Kosunen I, Moustakas K, Tzovaras D, Hassapis G (2010) Identifying psychophysiological correlates of boredom and negative mood induced during HCI. In: Bio-inspired human-machine interfaces and healthcare applications, January 2010, pp 3–12, https://doi.org/10.5220/0002812600030012

  • Giakoumis D, Tzovaras D, Moustakas K, Hassapis G (2011) Automatic recognition of boredom in video games using novel biosignal moment-based features. IEEE Trans Affect Comput 2(3):119–133. https://doi.org/10.1109/T-AFFC.2011.4

    Article  Google Scholar 

  • Glowinski D, Camurri A, Volpe G, Dael N, Scherer K (2008) Technique for automatic emotion recognition by body gesture analysis. In: 2008 IEEE computer society conference on computer vision and pattern recognition workshops, CVPR Workshops pp 1–6, https://doi.org/10.1109/CVPRW.2008.4563173

  • Gruber S, Peyton JK, Bruce BC (1994) Collaborative writing in multiple discourse contexts. CSCW 3(3–4):247–269. https://doi.org/10.1007/BF00750742

    Article  Google Scholar 

  • Gunes H, Piccardi M (2007) Bi-modal emotion recognition from expressive face and body gestures. J Netw Comput Appl 30(4):1334–1345. https://doi.org/10.1016/j.jnca.2006.09.007

    Article  Google Scholar 

  • Hall M, Frank E, Holmes G, Pfahringer B, Reutemann P, Witten IH (2009) The weka data mining software: an update. ACM SIGKDD Explor Newslett 11(1):10–18

    Article  Google Scholar 

  • Ho TK (1995) Random decision forests. In: Document analysis and recognition, 1995, proceedings of the third international conference on, IEEE, vol 1, pp 278–282

  • Horlings R, Datcu D, Rothkrantz LJM (2008) Emotion recognition using brain activity. In: Proceedings of the 9th international conference on computer systems and technologies and workshop for PhD students in computing, ACM, March, p 6

  • Jang EH, Park BJ, Park MS, Kim SH, Sohn JH (2015) Analysis of physiological signals for recognition of boredom, pain, and surprise emotions. J Physiol Anthropol 34(1):1–12. https://doi.org/10.1186/s40101-015-0063-5

    Article  Google Scholar 

  • Jaques N, Conati C, Harley JM, Azevedo R (2014) Predicting affect from gaze data during interaction with an intelligent tutoring system. Lect Notes Comput Sci 8474:29–38

  • Jasper H (1958) Report of the committee on methods of clinical examination in electroencephalography: 1957. Electroencephalogr Clin Neurophysiol 10(2):370–375. https://doi.org/10.1016/0013-4694(58)90053-1

    Article  Google Scholar 

  • Jiang R, Ho AT, Cheheb I, Al-Maadeed N, Al-Maadeed S, Bouridane A (2017) Emotion recognition from scrambled facial images via many graph embedding. Pattern Recogn 67:245–251. https://doi.org/10.1016/j.patcog.2017.02.003

    Article  Google Scholar 

  • Kanevsky LS (1994) A comparative study of children’s learning in the zone of proximal development. Eur J High Abil 5(2):163–175. https://doi.org/10.1080/0937445940050206

    Article  Google Scholar 

  • Katahira K, Yamazaki Y, Yamaoka C, Ozaki H, Nakagawa S, Nagata N (2018) Eeg correlates of the flow state: a combination of increased frontal theta and moderate frontocentral alpha rhythm in the mental arithmetic task. Front Psychol 9:300. https://doi.org/10.3389/fpsyg.2018.00300

    Article  Google Scholar 

  • Khalili Z, Moradi MH (2008) Emotion detection using brain and peripheral signals. In: 2008 Cairo international biomedical engineering conference, pp 1–4, https://doi.org/10.1109/CIBEC.2008.4786096

  • Kim D, Kim Y, Kim S, Park Y, Park J, Bae K, Lee S, Lee J, Lim C, Jeon Y, Jin S, Chae J, Hwang H (2017) Understanding and application of EEG. Hakjisa, Seogyo-dong

    Google Scholar 

  • Kim DJ, Kim YS (2010) Pattern classification of four emotions using EEG. J Korea Inst Inf Electron Commun Technol 3:23–27

    Google Scholar 

  • Kim HD, Sim KB (2007) Brain-wave analysis using fMRI, TRS and EEG for human emotion recognition. In: Proceedings of KFIS autumn conference 2007, Korean Institute of Intelligent Systems, vol 17, pp 7–10

  • Kim J, Seo J, Laine TH (2018) Detecting Boredom from Eye Gaze and EEG. Biomed Signal Process Control 46:302–313

    Article  Google Scholar 

  • Lang PJ, Bradley MM, Cuthbert BN (2008) International affective picture system (IAPS): affective ratings of pictures and instruction manual, Technical Report. University of Florida, Gainesville, Florida

  • Lee CC, Shih CY, Lai WP, Lin PC (2012) An improved boosting algorithm and its application to facial emotion recognition. J Ambient Intell Hum Comput 3(1):11–17. https://doi.org/10.1007/s12652-011-0085-8

    Article  Google Scholar 

  • Lee HJ, Shin DI, Shin DK (2013) A Study on an emotion-classification algorithm of users adapting Brainwave. In: Proceedings of symposium of the Korean Institute of communications and Information Sciences, Korea Institute Of Communication Sciences, pp 786–787

  • Lee HJ, Shin DI, Shin DK (2014) The classification algorithm of users emotion using brain-wave. J Kor Inst Commun Inf Sci 39(02):122–129

    Google Scholar 

  • Lee S, Song K (2007) The effects of affective feedbacks according to the learner’s emotions in e-learning. J Kor Soc Comput Inf 10(4):56

    Google Scholar 

  • Li M, Lu BL (2009) Emotion classification based on gamma-band EEG. In: 2009 Annual International Conference of the IEEE Engineering in medicine and biology society pp 1223–1226, https://doi.org/10.1109/IEMBS.2009.5334139

  • Lin HCK, Hsieh MC, Loh LC, Wang CH (2012) An emotion recognition mechanism based on the combination of mutual information and semantic clues. J Ambient Intell Hum Comput 3(1):19–29. https://doi.org/10.1007/s12652-011-0086-7

    Article  Google Scholar 

  • Lin YP, Wang CH, Wu TL, Jeng SK, Chen JH (2008) Support vector machine for EEG signal classification during listening to emotional music. In: 2008 IEEE 10th workshop on multimedia signal processing, pp 127–130, https://doi.org/10.1109/MMSP.2008.4665061

  • Lin YP, Wang CH, Wu TL, Jeng SK, Chen JH (2009) EEG-based emotion recognition in music listening: a comparison of schemes for multiclass support vector machine. In: 2009 IEEE international conference on Acoustics, speech and signal processing pp 489–492, https://doi.org/10.1109/ICASSP.2009.4959627

  • Lin YP, Wang CH, Jung TP, Wu TL, Jeng SK, Duann JR, Chen JH (2010) EEG-based emotion recognition in music listening. IEEE Trans Biomed Eng 57(7):1798–1806. https://doi.org/10.1109/TBME.2010.2048568

    Article  Google Scholar 

  • Logeswaran N, Bhattacharya J (2009) Crossmodal transfer of emotion by music. Neurosci Lett 455(2):129–133. https://doi.org/10.1016/j.neulet.2009.03.044

    Article  Google Scholar 

  • Mandryk RL, Atkins MS (2007) A fuzzy physiological approach for continuously modeling emotion during interaction with play technologies. Int J Hum Comput Stud 65(4):329–347

    Article  Google Scholar 

  • Mao X, Li Z (2009) Implementing emotion-based user-aware e-learning. Int Conf Hum Factors Comput Syst 56:3787–3792

    Google Scholar 

  • Mistry K, Zhang L, Neoh SC, Lim CP, Fielding B (2017) A micro-GA embedded PSO feature selection approach to intelligent facial emotion recognition. IEEE Trans Cybern 47(6):1496–1509. https://doi.org/10.1109/TCYB.2016.2549639

    Article  Google Scholar 

  • Mohammadpour M, Hashemi SMR, Houshmand N (2017) Classification of EEG-based emotion for BCI applications. In: 2017 artificial intelligence and robotics (IRANOPEN), pp 127–131, https://doi.org/10.1109/RIOS.2017.7956455

  • Murugappan M, Rizon M, Nagarajan R, Yaacob S, Zunaidi I, Hazry D (2008) Lifting scheme for human emotion recognition using EEG. In: 2008 International symposium on information technology 2:0–6, https://doi.org/10.1109/ITSIM.2008.4631646

  • MUSE (2018) MUSE for developer. http://developer.choosemuse.com/ Accessed: 2018-11-30

  • Nicolopoulou A, Cole M (1996) Generation and transmission of shared knowledge in the culture of collaborative learning: the Fifth Dimension, its play-world and its institutional contexts. Oxford University Press, Oxford

    Google Scholar 

  • Nie D, Wang XW, Shi LC, Lu BL (2011) EEG-based emotion recognition during watching movies. In: 2011 5th International IEEE/EMBS conference on neural engineering, pp 667–670, https://doi.org/10.1109/NER.2011.5910636

  • Nwe TL, Foo SW, De Silva LC (2003) Speech emotion recognition using hidden Markov models. Speech Commun 41(4):603–623. https://doi.org/10.1016/S0167-6393(03)00099-2

    Article  Google Scholar 

  • Oostenveld R, Praamstra P (2001) The five percent electrode system for high-resolution eeg and erp measurements. Clin Neurophysiol 112(4):713–719. https://doi.org/10.1016/S1388-2457(00)00527-7

    Article  Google Scholar 

  • Oroujlou N, Vahedi M (2011) Motivation, attitude, and language learning. Procedia Soc Beh Sci 29:994–1000. https://doi.org/10.1016/j.sbspro.2011.11.333

    Article  Google Scholar 

  • Petrantonakis PC, Hadjileontiadis LJ (2010) Emotion recognition from brain signals using hybrid adaptive filtering and higher order crossings analysis. IEEE Trans Affect Comput 1(2):81–97. https://doi.org/10.1109/T-AFFC.2010.7

    Article  Google Scholar 

  • Petrantonakis PC, Hadjileontiadis LJ (2011) A novel emotion elicitation index using frontal brain asymmetry for enhanced EEG-Based emotion recognition. IEEE Trans Inf Technol Biomed 15(5):737–746. https://doi.org/10.1109/TITB.2011.2157933

    Article  Google Scholar 

  • Picard RW (1995) Affective computing. Tech Rep 5:6–12. https://doi.org/10.1007/BF01238028

    Article  MathSciNet  Google Scholar 

  • Rosenblatt F (1961) Principles of neurodynamics perceptrons and the theory of brain mechanisms. Cornell aeronautical lab, Buffalo

    Book  MATH  Google Scholar 

  • Russell JA (1980) A circumplex model of affect. J Personal Soc Psychol 39(6):1161–1178. https://doi.org/10.1037/h0077714

    Article  Google Scholar 

  • Sanei S, Chambers JA (2013) EEG signal processing. Wiley, Hoboken

    Google Scholar 

  • Scardamalia M, Bereiter C (1994) Computer support for knowledge-building communities. J Learn Sci 3(3):265–283. https://doi.org/10.1207/s15327809jls0303_3

    Article  Google Scholar 

  • Schmidt LA, Trainor LJ (2001) Frontal brain electrical activity (EEG) distinguishes valence and intensity of musical emotions. Cogn Emot 15(4):487–500. https://doi.org/10.1080/02699930126048

    Article  Google Scholar 

  • Schuller B, Rigoll G, Lang M (2003) Hidden markov model-based speech emotion recognition. In: 2003 IEEE international conference on Acoustics, speech, and signal processing, 2003. Proceedings. (ICASSP ’03)., vol 2, pp II–1, https://doi.org/10.1109/ICASSP.2003.1202279

  • Shen L, Leon E, Callaghan V, Shen R (2007) Exploratory research on an Affective e-Learning Model. In: Proceedings of workshop on blended learning, vol 267-278, 10.1007/978-3-540-78139-4, 9780201398298

  • Shen L, Wang M, Shen R (2009) Affective e-learning: using emotional data to improve learning in pervasive learning environment related work and the pervasive e-learning platform. Educ Technol Soc 12(2):176–189

    Google Scholar 

  • Sidney KD, Craig SD, Gholson B, Franklin S, Picard R, Graesser AC (2005) Integrating affect sensors in an intelligent tutoring system. In: In affective interactions: the computer in the affective loop workshop at 2005 international conference on intelligent user interfaces, pp 7–13

  • Singh G, Jati A, Khasnobish A, Bhattacharyya S, Konar A, Tibarewala DN, Janarthanan R (2012) Negative emotion recognition from stimulated EEG signals. In: Computing communication networking technologies (ICCCNT), 2012 third international conference on, pp 1–8, https://doi.org/10.1109/ICCCNT.2012.6395891

  • Sottilare R, Goldberg B (2012) Designing adaptive computer-based tutoring systems to accelerate learning and facilitate retention. J Cogn Technol 17(1):19–33

    Google Scholar 

  • Takahashi K (2005) Remarks on emotion recognition from bio-potential signals. In: in 2nd Int. Conf. on Autonomous Robots and Agents

  • Tan JW, Walter S, Scheck A, Hrabal D, Hoffmann H, Kessler H, Traue HC (2012) Repeatability of facial electromyography (EMG) activity over corrugator supercilii and zygomaticus major on differentiating various emotions. J Ambient Intell Hum Comput 3(1):3–10. https://doi.org/10.1007/s12652-011-0084-9

    Article  Google Scholar 

  • Timmermann M, Jeung H, Schmitt R, Boll S, Freitag CM, Bertsch K, Herpertz SC (2017) Oxytocin improves facial emotion recognition in young adults with antisocial personality disorder. Psychoneuroendocrinology 85(July):158–164. https://doi.org/10.1016/j.psyneuen.2017.07.483

    Article  Google Scholar 

  • Vijayan AE, Sen D, Sudheer A (2015) EEG-Based Emotion Recognition Using Statistical Measures and Auto-Regressive Modeling. In: 2015 IEEE international conference on computational intelligence and communication technology, pp 587–591, https://doi.org/10.1109/CICT.2015.24

  • Vogel-Walcutt JJ, Fiorella L, Carper T, Schatz S (2012) The definition, assessment, and mitigation of state boredom within educational settings: a comprehensive review. Educ Psychol Rev 24(1):89–111. https://doi.org/10.1007/s10648-011-9182-7

    Article  Google Scholar 

  • Wu CH, Chuang ZJ, Lin YC (2006) Emotion recognition from text using semantic labels and separable mixture models. ACM Trans Asian Lang Inf Process 5(2):165–183. https://doi.org/10.1145/1165255.1165259

    Article  Google Scholar 

  • Wundt W (1897) Outline of psychology (CH Judd, Trans.). Williams and Norgate, London, Great Britain (Wilhelm Engelmann, Leipzig, Germany)

  • Yeager DS, Henderson MD, Paunesku D, Walton GM, D'Mello S, Spitzer BJ, Duckworth AL (2014) Boring but important: a self-transcendent purpose for learning fosters academic self-regulation. J Pers Soc Psychol 107(4):559

    Article  Google Scholar 

  • Zhao Y, Wang X, Goubran M, Whalen T, Petriu EM (2013) Human emotion and cognition recognition from body language of the head using soft computing techniques. J Ambient Intell Hum Comput 4(1):121–140. https://doi.org/10.1007/s12652-012-0107-1

    Article  Google Scholar 

  • Zheng WL, Lu BL (2015) Investigating critical frequency bands and channels for EEG-based emotion recognition with deep neural networks. IEEE Trans Auton Ment Dev 7(3):162–175. https://doi.org/10.1109/TAMD.2015.2431497

    Article  Google Scholar 

  • Zheng WL, Zhu JY, Lu BL (2017) Identifying stable patterns over time for emotion recognition from EEG. In: IEEE transactions on affective computing PP(99):1, https://doi.org/10.1109/TAFFC.2017.2712143

Download references

Acknowledgements

This research was supported by the MSIT (Ministry of Science and ICT), Korea, under the ITRC (Information Technology Research Center) support program (IITP-2018-2018-0-01431) supervised by the IITP (Institute for Information & communications Technology Promotion). We would like to show our appreciation to the participants in our data collection. Special thanks to Ms. Wooryeon Go (Emily), Dr. Carolina Islas Sedano, Dr. Hana Vrzakova, Dr. Roman Bednarik, and Dr. Bednarik’s research group members at the University of Eastern Finland. Furthermore, we would like to thank Mr. Joochan Kim, at the Luleå University of Technology in Sweden.

Author information

Authors and Affiliations

  1. Department of Computer Engineering, Ajou University, Suwon, Republic of Korea

    Jungryul Seo & Kyung-Ah Sohn

  2. Department of Computer Science, Electrical and Space Engineering, Luleå University of Technology, Skellefteå, Sweden

    Teemu H. Laine

Authors
  1. Jungryul Seo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Teemu H. Laine
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Kyung-Ah Sohn
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kyung-Ah Sohn.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Seo, J., Laine, T.H. & Sohn, KA. Machine learning approaches for boredom classification using EEG. J Ambient Intell Human Comput 10, 3831–3846 (2019). https://doi.org/10.1007/s12652-019-01196-3

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12652-019-01196-3

Keywords

Search

Navigation