Skip to main content

Advertisement

SpringerLink
Log in

A semi-supervised inattention detection method using biological signal

  • OR in Neuroscience
  • Published:
Annals of Operations Research Aims and scope Submit manuscript

Abstract

Recently, operations research methods have been utilized for biological data analysis as a huge amount of biological data becomes available. One of popular applications of the data analysis is inattention detection of operators in human–machine interaction systems using electroencephalography (EEG) signal. Most of the previous studies on the inattention detection employed supervised learning approaches, but their results have potential bias since they rely on imperfect assumptions for the acquisition of mental state labels, attention and inattention, due to the absence of the standardized measure for the mental states. Instead, we consider unsupervised learning approach, where no labeled data is required. In order to address the low performance of unsupervised learning approaches, attention duration for which an operator sustains his/her attention from the beginning of performing a task and relevance levels between four attributes of EEG signal and mental states are exploited. In this regard, we propose a semi-supervised inattention detection method (SID), in which attention duration and attributes-weights of EEG signal are respectively utilized as a small portion of labeled data for semi-supervised learning and weights for similarity calculation. Specifically, cumulative sum algorithm is used for the determination of the attention duration, and constrained attributes-weighting clustering algorithm is used for the estimation of attributes-weights. From experiments using real-world dataset, SID outperformed the compared methods, and it is expected that the adoption of SID will contribute to the enhancement of the operators’ safety.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6

Similar content being viewed by others

References

  • Akin, M., Kurt, M. B., Sezgin, N., & Bayram, M. (2008). Estimating vigilance level by using EEG and EMG signals. Neural Computing and Applications, 17(3), 227–236.

    Article  Google Scholar 

  • Alfaro, C., Cano-Montero, J., Gmez, J., Moguerza, J. M., & Ortega, F. (2013). A multi-stage method for content classification and opinion mining on weblog comments. Annals of Operations Research, 236(1), 1–17.

    Google Scholar 

  • Balasubramanian, V., Adalarasu, K., & Gupta, A. (2011). EEG based analysis of cognitive fatigue during simulated driving. International Journal of Industrial and Systems Engineering, 7(2), 135–149.

    Article  Google Scholar 

  • Basu, S., Banerjee, A., & Mooney, R. J. (2001). Semi-supervised clustering by seeding. In Proceedings of the international conference on machine learning (pp. 27–34).

  • Belyavin, A., & Wright, N. A. (1987). Changes in electrical activity of the brain with vigilance. Electroencephalography and Clinical Neurophysiology, 66(2), 137–144.

    Article  Google Scholar 

  • Berka, C., Levendowski, D. J., Lumicao, M. N., Yau, A., Davis, G., Zivkovic, V. T., et al. (2007). EEG correlates of task engagement and mental workload in vigilance, learning, and memory tasks. Aviation, Space, and Environmental Medicine, 78(5), B231–B244.

    Google Scholar 

  • Caldwell, J., Hall, K., & Erickson, B. (2002). EEG data collected from helicopter pilots in flight are sufficiently sensitive to detect increased fatigue from sleep deprivation. The International Journal of Aviation Psychology, 12(1), 19–32.

    Article  Google Scholar 

  • Chan, E. Y., Ching, W. K., Ng, M. K., & Huang, J. Z. (2004). An optimization algorithm for clustering using weighted dissimilarity measures. Pattern Recognition, 37(5), 943–952.

    Article  Google Scholar 

  • Chaovalitwongse, W. A., Prokopyev, O. A., & Pardalos, P. M. (2006). Electroencephalogram (EEG) time series classification: Applications in epilepsy. Annals of Operations Research, 148(1), 227–250.

    Article  Google Scholar 

  • Choi, Y., Kwon, N., Lee, S., Shin, Y., Ryo, C. Y., & Park, J., et al. (2014). Hypovigilance detection for ucav operators based on a hidden Markov model. Computational and Mathematical Methods in Medicine, 2014, 1–13.

  • Choi, Y., Kwon, N., Yoon, J., Jeon, S., Paeng, B. H., & Park, J., et al. (2013a). A semi-supervised attributes-weighting clustering method for inattention prediction in human–machine interaction systems. In Proceedings of the Asia pacific industrial engineering and management systems conference.

  • Choi, Y., Kwon, N., Yoon, J., Park, J., Lee, H., Kim, Y., et al. (2013b). A BCI based ground control framework for attention maintenance of UAV operators. Entrue Journal of Information Technology, 12(1), 101–115.

    Google Scholar 

  • Craig, A., Tran, Y., Wijesuriya, N., & Nguyen, H. (2012). Regional brain wave activity changes associated with fatigue. Psychophysiology, 49(4), 574–582.

    Article  Google Scholar 

  • Dempster, A. P,. Laird, N. M., & Rubin, D. B. (1977). Maximum likelihood from incomplete data via the em algorithm. Journal of the Royal Statistical Society Series B (Methodological), 39(1), 1–38.

  • Doroshenkov, L. G., Konyshev, V. A., & Selishchev, S. V. (2007). Classification of human sleep stages based on EEG processing using hidden Markov models. Biomedical Engineering, 41(1), 25–28.

    Article  Google Scholar 

  • Egner, T., & Gruzelier, J. H. (2004). EEG biofeedback of low beta band components: Frequency-specific effects on variables of attention and event-related brain potentials. Clinical Neurophysiology, 115(1), 131–139.

    Article  Google Scholar 

  • Elkan, C., & Noto, K. (2008). Learning classifiers from only positive and unlabeled data. In Proceedings of the ACM international conference on knowledge discovery and data mining (pp. 213–220).

  • Endsley, M. (1995). Toward a theory of situation awareness in dynamic systems. Human Factors: The Journal of the Human Factors and Ergonomics Society, 37(1), 32–64.

    Article  Google Scholar 

  • Friedrichs, F., & Yang, B. (2010). Camera-based drowsiness reference for driver state classification under real driving conditions. In Proceedings of the IEEE intelligent vehicles symposium (pp. 101–106).

  • Frigui, H., & Nasraoui, O. (2004). Unsupervised learning of prototypes and attribute weights. Pattern Recognition, 37(3), 567–581.

    Article  Google Scholar 

  • Harmony, T., Fernndez, T., Silva, J., Bernal, J., Daz-Comas, L., Reyes, A., et al. (1996). EEG delta activity: An indicator of attention to internal processing during performance of mental tasks. International Journal of Psychophysiology, 24(12), 161–171.

    Article  Google Scholar 

  • Haufe, S., Treder, M. S., Gugler, M. F., Sagebaum, M., Curio, G., & Blankertz, B. (2011). EEG potentials predict upcoming emergency brakings during simulated driving. Journal of Neural Engineering, 8(5). doi:10.1088/1741-2560/8/5/056001.

  • Huang, R. S., Kuo, C. J., Tsai, L. L., & Chen, O. T. C. (1996). EEG pattern recognition-arousal states detection and classification. In Proceedings of the IEEE international conference on neural networks (pp. 641–646).

  • Jap, B. T., Lal, S., & Fischer, P. (2011). Comparing combinations of EEG activity in train drivers during monotonous driving. Expert Systems with Applications, 38(1), 996–1003.

    Article  Google Scholar 

  • Ji, Q., & Yang, X. (2001). Real time visual cues extraction for monitoring driver vigilance. Lecture Notes in Computer Science, 2095, 107–124.

    Article  Google Scholar 

  • Kaida, K., Takahashi, M., Akerstedt, T., Nakata, A., Otsuka, Y., Haratani, T., et al. (2006). Validation of the karolinska sleepiness scale against performance and EEG variables. Clinical Neurophysiology, 117(7), 1574–1581.

    Article  Google Scholar 

  • Kim, K., Choi, Y., & Park, J. (2014a). A conditional feature utilization approach to itemset retrieval in online shopping services. Journal of Electronic Commerce Research, 15(4), 317.

    Google Scholar 

  • Kim, K., Chung, B. S., Choi, Y., Lee, S., Jung, J. Y., & Park, J. (2014b). Language independent semantic kernels for short-text classification. Expert Systems with Applications, 41(2), 735–743.

    Article  Google Scholar 

  • Klimesch, W., Doppelmayr, M., Russegger, H., Pachinger, T., & Schwaiger, J. (1998). Induced alpha band power changes in the human EEG and attention. Neuroscience Letters, 244(2), 73–76.

    Article  Google Scholar 

  • Lal, S. K. L., & Craig, A. (2002). Driver fatigue: Electroencephalography and psychological assessment. Psychophysiology, 39(3), 313–321.

    Article  Google Scholar 

  • Larue, G., Rakotonirainy, A., & Pettitt, A. (2010). Real-time performance modelling of a sustained attention to response task. Ergonomics, 53(10), 1205–1216.

    Article  Google Scholar 

  • Lee, S., Ha, J., Na, O., & Na, S. (2003). The cusum test for parameter change in time series models. Scandinavian Journal of Statistics, 30(4), 781–796.

    Article  Google Scholar 

  • Liu, J., Zhang, C., & Zheng, C. (2010). EEG-based estimation of mental fatigue by using KPCAHMM and complexity parameters. Biomedical Signal Processing and Control, 5(2), 124–130.

    Article  Google Scholar 

  • Lukasiak, P., Baewicz, J., & Miostan, M. (2009). Some operations research methods for analyzing protein sequences and structures. Annals of Operations Research, 175(1), 9–35.

    Article  Google Scholar 

  • Mackworth, N. H. (1948). The breakdown of vigilance durning prolonged visual search. Quarterly Journal of Experimental Psychology, 1(1), 6–21.

    Article  Google Scholar 

  • Mishchenko, Y., & Kaya, M. (2015). Detecting the attention state of an operator in continuous attention task using EEG-based brain-computer interface. In Proceedings of the signal processing and communications applications conference (pp. 232–235).

  • Nilsson, T., Nelson, T., & Carlson, D. (1997). Development of fatigue symptoms during simulated driving. Accident Analysis and Prevention, 29(4), 479–488.

    Article  Google Scholar 

  • Nuechterlein, K., Parasuraman, R., & Jiang, Q. (1983). Visual sustained attention: Image degradation produces rapid sensitivity decrement over time. Science, 220(4594), 327–329.

    Article  Google Scholar 

  • Pattyn, N., Neyt, X., Henderickx, D., & Soetens, E. (2008). Psychophysiological investigation of vigilance decrement: Boredom or cognitive fatigue? Physiology and Behavior, 93(1), 369–378.

    Article  Google Scholar 

  • Peiris, M., Jones, R., Davidson, P., Carroll, G., & Bones, P. (2006). Frequent lapses of responsiveness during an extended visuomotor tracking task in nonsleepdeprived subjects. Journal of Sleep Research, 15(3), 291–300.

    Article  Google Scholar 

  • Plotnikov, A., Stakheika, N., De Gloria, A., Schatten, C., Bellotti, F., & Berta, R., et al. (2012). Exploiting real-time eeg analysis for assessing flow in games. In Proceedings of the IEEE international conference on advanced learning technologies (pp. 688–689).

  • Punsawad, Y., Aempedchr, S., Wongsawat, Y., & Panichkun, M. (2011). Weighted-frequency index for EEG-based mental fatigue alarm system. International Journal of Applied Biomedical Engineering, 4(1), 36–41.

    Google Scholar 

  • Putman, P., Verkuil, B., Arias-Garcia, E., Pantazi, I., & van Schie, C. (2014). EEG theta/beta ratio as a potential biomarker for attentional control and resilience against deleterious effects of stress on attention. Cognitive, Affective, and Behavioral Neuroscience, 14(2), 782–791.

    Article  Google Scholar 

  • Sahin, H., & Feng, Q. (2009). A mathematical framework for sequential passenger and baggage screening to enhance aviation security. Computers and Industrial Engineering, 57(1), 148–155.

    Article  Google Scholar 

  • Sauter, C., Danker-Hopfe, H., Loretz, E., Zeitlhofer, J., Geisler, P., & Popp, R. (2013). The assessment of vigilance: Normative data on the siesta sustained attention test. Sleep Medicine, 14(6), 542–548.

    Article  Google Scholar 

  • Shi, L. C., & Lu, B. L. (2008). Dynamic clustering for vigilance analysis based on EEG. In Proceedings of the international conference of the IEEE engineering in medicine and biology society (pp. 54–57).

  • Shi, L. C., Yu, H., & Lu, B. L. (2007). Semi-supervised clustering for vigilance analysis based on EEG. In Proceedings of the international joint conference on neural networks (pp. 1518–1523).

  • Sun, H., Bi, L., Lu, X., Fan, B., & Guo, Y. (2015a). Vigilance analysis based on EEG band power using support vector machine. In Proceedings of the international congress on image and signal processing (pp. 1090–1094).

  • Sun, H., Yang, Y., Sourina, O., Huang, G. B., Klanner, F., & Denk, C., et al. (2015b). Estimating vigilance from EEG using manifold clustering guided by instantaneous lapse rate. In Proceedings of the international conference on information, communications and signal processing (pp. 1–5).

  • Thiffault, P., & Bergeron, J. (2003). Monotony of road environment and driver fatigue: A simulator study. Accident Analysis and Prevention, 35(3), 381–391.

    Article  Google Scholar 

  • Ting, P. H., Hwang, J. R., Doong, J. L., & Jeng, M. C. (2008). Driver fatigue and highway driving: A simulator study. Physiology and Behavior, 94(3), 448–453.

    Article  Google Scholar 

  • Upadhyay, R., Padhy, P. K., & Kankar, P. K. (2016). Application of s-transform for automated detection of vigilance level using EEG signals. Journal of Biological Systems, 24(1), 1–27.

    Article  Google Scholar 

  • Uusberg, A., Uibo, H., Kreegipuu, K., & Allik, J. (2013). EEG alpha and cortical inhibition in affective attention. International Journal of Psychophysiology, 89(1), 26–36.

    Article  Google Scholar 

  • Vijay, J., & Subhashini, J. (2013). An efficient brain tumor detection methodology using K-means clustering algoriftnn. In Proceedings of the international conference on communications and signal processing (pp. 653–657).

  • Wang, L., & Pan, C. (2014). Robust level set image segmentation via a local correntropy-based K-means clustering. Pattern Recognition, 47(5), 1917–1925.

    Article  Google Scholar 

  • West, J. V., & Lane-Cummings, K. (2007). Microsoft flight simulator X for pilots: Real world training. New York: Wiley.

    Google Scholar 

  • Yeo, M. V. M., Li, X., Shen, K., & Wilder-Smith, E. P. V. (2009). Can svm be used for automatic eeg detection of drowsiness during car driving? Safety Science, 47(1), 115–124.

    Article  Google Scholar 

  • Yusoff, I. A., Isa, N. A. M., & Hasikin, K. (2013). Automated two-dimensional K-means clustering algorithm for unsupervised image segmentation. Computers and Electrical Engineering, 39(3), 907–917.

    Article  Google Scholar 

  • Zhang, B., Wang, J., & Fuhlbrigge, T. (2010). A review of the commercial brain-computer interface technology from perspective of industrial robotics. In Proceedings of the IEEE international conference on automation and logistics (pp. 379–384).

  • Zhang, Z., Luo, D., Rasim, Y., Li, Y., Meng, G., Xu, J., et al. (2016). A vehicle active safety model: Vehicle speed control based on driver vigilance detection using wearable EEG and sparse representation. Sensors, 16(2), 242.

    Article  Google Scholar 

  • Zhao, C., Zhao, M., Liu, J., & Zheng, C. (2012). Electroencephalogram and electrocardiograph assessment of mental fatigue in a driving simulator. Accident Analysis and Prevention, 45, 83–90.

    Article  Google Scholar 

Download references

Acknowledgements

This work was supported by the BK21 Plus Program (Center for Sustainable and Innovative Industrial Systems, Department of Industrial Engineering, Seoul National University) funded by the Ministry of Education, Korea (No. 21A20130012638) and by Basic Science Research Program through the National Research Foundation of Korea (NRF) funded by the Ministry of Science, ICT and future Planning (No. 2013R1A2A2A03013947).

Author information

Authors and Affiliations

  1. Department of Industrial and Management Engineering, Kyonggi University, Suwon, Gyeonggi, 16227, Republic of Korea

    Yerim Choi

  2. Department of Industrial Engineering & Institute for Industrial Systems Innovation, Seoul National University, Seoul, 151-744, Republic of Korea

    Jonghun Park

  3. Department of Industrial and Management Engineering, Hanyang University, Ansan, Gyeonggi, 425-791, Republic of Korea

    Dongmin Shin

Authors
  1. Yerim Choi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jonghun Park
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Dongmin Shin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dongmin Shin.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Choi, Y., Park, J. & Shin, D. A semi-supervised inattention detection method using biological signal. Ann Oper Res 258, 59–78 (2017). https://doi.org/10.1007/s10479-017-2406-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10479-017-2406-6

Keywords

Search

Navigation