Skip to main content
SpringerLink
Log in
Book cover

International Conference on Human Centered Computing

HCC 2019: Human Centered Computing pp 659–670Cite as

Modelling Mental States via Computational Psychophysiology: Benefits and Challenges

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 11956))

Abstract

The human psychophysiological processes are complex phenomenon built upon the physical scaffolding of the body. Machine learning approaches facilitate the understanding of numerous physiological processes underlying complex human mental states and behavior, leading to a new research direction named Computational Psychophysiology. Computational Psychophysiology aims to reveal the psychophysiological processes underlying complex human emotion and mental states from a computational perspective, and can be used to predict affective and psychological outcomes based on different physiological features or experimental manipulations. In this paper, we discuss the benefits and challenges in the future of bringing computing technologies into decoding human mental states.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

References

  1. Fowles, D.C.: Psychophysiology and psychopathology: a motivational approach. Psychophysiology 25, 373–391 (1988)

    Article  Google Scholar 

  2. Stern, R.M., Ray, W.J., Quigley, K.S.: Psychophysiological Recording. Oxford University Press, Oxford (2001)

    Google Scholar 

  3. Allen, J.J., Kline, J.P.: Frontal EEG asymmetry, emotion, and psychopathology: the first, and the next 25 years. Biol. Psychol. 67, 1–5 (2004)

    Article  Google Scholar 

  4. Dolan, R.J.: Emotion, cognition, and behavior. Science 298, 1191–1194 (2002)

    Article  Google Scholar 

  5. Picard, R.W., Vyzas, E., Healey, J.: Toward machine emotional intelligence: analysis of affective physiological state. IEEE Trans. Pattern Anal. Mach. Intell. 23, 1175–1191 (2001)

    Article  Google Scholar 

  6. Hu, B., Zheng, W.: A review of computational psychophysiology: the progress and trends. Commun. CCF 14, 31–34 (2018)

    Google Scholar 

  7. American Association for the Advancement of Science: Advances in computational psychophysiology. Science 350, 114 (2015)

    Google Scholar 

  8. Bzdok, D., Meyer-Lindenberg, A.: Machine learning for precision psychiatry: opportunities and challenges. Biol. Psychiatry Cogn. Neurosci. Neuroimaging 3, 223–230 (2018)

    Article  Google Scholar 

  9. Cacioppo, J.T., Tassinary, L.G., Berntson, G.: Handbook of Psychophysiology. Cambridge University Press, Cambridge (2007)

    Google Scholar 

  10. Boveroux, P., et al.: Breakdown of within- and between-network resting state functional magnetic resonance imaging connectivity during propofol-induced loss of consciousness. Anesthesiol. J. Am. Soc. Anesthesiol. 113, 1038–1053 (2010)

    Google Scholar 

  11. Kelly, J.E.: Computing, cognition and the future of knowing. Whitepaper, IBM Reseach 2 (2015)

    Google Scholar 

  12. Modha, D.S., Ananthanarayanan, R., Esser, S.K., Ndirango, A., Sherbondy, A.J., Singh, R.: Cognitive computing. Commun. ACM 54, 62–71 (2011)

    Article  Google Scholar 

  13. Huys, Q.J., Maia, T.V., Frank, M.J.: Computational psychiatry as a bridge from neuroscience to clinical applications. Nat. Neurosci. 19, 404 (2016)

    Article  Google Scholar 

  14. Solo, A.M., Gupta, M.M.: Uncertainty in computational perception and cognition. In: Nikravesh, M., Kacprzyk, J., Zadeh, L.A. (eds.) Forging New Frontiers: Fuzzy Pioneers I, pp. 251–266. Springer, Heidelberg (2007)

    Chapter  Google Scholar 

  15. Chen, Y., Argentinis, J.E., Weber, G.: IBM Watson: how cognitive computing can be applied to big data challenges in life sciences research. Clin. Ther. 38, 688–701 (2016)

    Article  Google Scholar 

  16. Adams, R.A., Huys, Q.J., Roiser, J.P.: Computational psychiatry: towards a mathematically informed understanding of mental illness. J. Neurol. Neurosurg. Psychiatry 87, 53–63 (2016)

    Article  Google Scholar 

  17. Hyvärinen, A., Oja, E.: Independent component analysis: algorithms and applications. Neural Netw. 13, 411–430 (2000)

    Article  Google Scholar 

  18. Calhoun, V.D., Liu, J., Adali, T.: A review of group ICA for fMRI data and ICA for joint inference of imaging, genetic, and ERP data. Neuroimage 45, S163–S172 (2009)

    Article  Google Scholar 

  19. Yao, Z., et al.: Resting-state time-varying analysis reveals aberrant variations of functional connectivity in autism. Front. Hum. Neurosci. 10, 463 (2016)

    Article  Google Scholar 

  20. Dhall, A., Asthana, A., Goecke, R., Gedeon, T.: Emotion recognition using PHOG and LPQ features. In: Face and Gesture 2011, pp. 878–883. IEEE (2011)

    Google Scholar 

  21. Jenke, R., Peer, A., Buss, M.: Feature extraction and selection for emotion recognition from EEG. IEEE Trans. Affect. Comput. 5, 327–339 (2014)

    Article  Google Scholar 

  22. Kim, K.H., Bang, S.W., Kim, S.R.: Emotion recognition system using short-term monitoring of physiological signals. Med. Biol. Eng. Comput. 42, 419–427 (2004)

    Article  Google Scholar 

  23. Wager, T.D., Atlas, L.Y., Lindquist, M.A., Roy, M., Woo, C.-W., Kross, E.: An fMRI-based neurologic signature of physical pain. N. Engl. J. Med. 368, 1388–1397 (2013)

    Article  Google Scholar 

  24. Wager, T.D., Atlas, L.Y., Leotti, L.A., Rilling, J.K.: Predicting individual differences in placebo analgesia: contributions of brain activity during anticipation and pain experience. J. Neurosci. 31, 439–452 (2011)

    Article  Google Scholar 

  25. Querbes, O., et al.: Early diagnosis of Alzheimer’s disease using cortical thickness: impact of cognitive reserve. Brain 132, 2036–2047 (2009)

    Article  Google Scholar 

  26. Tong, T., et al.: A novel grading biomarker for the prediction of conversion from mild cognitive impairment to Alzheimer’s disease. IEEE Trans. Biomed. Eng. 64, 155–165 (2017)

    Article  Google Scholar 

  27. Wee, C.Y., Wang, L., Shi, F., Yap, P.T., Shen, D.: Diagnosis of autism spectrum disorders using regional and interregional morphological features. Hum. Brain Mapp. 35, 3414–3430 (2014)

    Article  Google Scholar 

  28. Yao, Z., Hu, B., Nan, H., Zheng, W., Xie, Y.: Individual metabolic network for the accurate detection of Alzheimer’s disease based on FDGPET imaging. In: 2016 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pp. 1328–1335. IEEE (2016)

    Google Scholar 

  29. Zhang, Z., et al.: Frequency-specific functional connectivity density as an effective biomarker for adolescent generalized anxiety disorder. Front. Hum. Neurosci. 11, 549 (2017)

    Article  Google Scholar 

  30. Zhao, Y., et al.: Predicting MCI progression with individual metabolic network based on longitudinal FDG-PET. In: 2017 IEEE International Conference on Bioinformatics and Biomedicine (BIBM), pp. 1894–1899. IEEE (2017)

    Google Scholar 

  31. Zheng, W., et al.: Multi-feature based network revealing the structural abnormalities in autism spectrum disorder. IEEE Trans. Affect. Comput. 1 (2019)

    Google Scholar 

  32. Zheng, W., Yao, Z., Hu, B., Gao, X., Cai, H., Moore, P.: Novel cortical thickness pattern for accurate detection of Alzheimer’s disease. J. Alzheimers Dis. 48, 995–1008 (2015)

    Article  Google Scholar 

  33. Zheng, W., Yao, Z., Li, Y., Wu, D., Hu, B.: Prediction of Alzheimer’s disease in patients with mild cognitive impairment using connectivity extracted from multi-modal brain imaging (under review)

    Google Scholar 

  34. Zheng, W., Yao, Z., Xie, Y., Fan, J., Hu, B.: Identification of Alzheimer’s disease and mild cognitive impairment using networks constructed based on multiple morphological brain features. Biol. Psychiatry Cogn. Neurosci. Neuroimaging 3, 887–897 (2018)

    Article  Google Scholar 

  35. Norman, K.A., Polyn, S.M., Detre, G.J., Haxby, J.V.: Beyond mind-reading: multi-voxel pattern analysis of fMRI data. Trends Cogn. Sci. 10, 424–430 (2006)

    Article  Google Scholar 

  36. Peelen, M.V., Downing, P.E.: Using multi-voxel pattern analysis of fMRI data to interpret overlapping functional activations. Trends Cogn. Sci. 11, 4 (2007)

    Article  Google Scholar 

  37. Hawkins, D.M.: The problem of overfitting. J. Chem. Inf. Comput. Sci. 44, 1–12 (2004)

    Article  Google Scholar 

  38. Emerson, R.W., et al.: Functional neuroimaging of high-risk 6-month-old infants predicts a diagnosis of autism at 24 months of age. Sci. Transl. Med. 9, eaag2882 (2017)

    Article  Google Scholar 

  39. Maia, T.V.: Introduction to the series on computational psychiatry. J. Clin. Psychol. Sci. 3, 374–377 (2015)

    Article  Google Scholar 

  40. Jolliffe, I.: Principal Component Analysis. Springer, New York (2011). https://doi.org/10.1007/b98835

    Book  MATH  Google Scholar 

  41. Guyon, I., Elisseeff, A.: An introduction to variable and feature selection. J. Mach. Learn. Res. 3, 1157–1182 (2003)

    MATH  Google Scholar 

  42. Zou, H., Hastie, T.: Regularization and variable selection via the elastic net. J. R. Stat. Soc. Ser. B (Stat. Methodol.) 67, 301–320 (2005)

    Article  MathSciNet  MATH  Google Scholar 

  43. Tibshirani, R.: Regression shrinkage and selection via the lasso. J. R. Stat. Soc. Ser. B (Methodol.) 58, 267–288 (1996)

    MathSciNet  MATH  Google Scholar 

  44. Liu, H., Motoda, H., Setiono, R., Zhao, Z.: Feature selection: an ever evolving Frontier in data mining. In: Feature Selection in Data Mining, pp. 4–13 (2010)

    Google Scholar 

  45. Saeys, Y., Inza, I., Larrañaga, P.: A review of feature selection techniques in bioinformatics. Bioinformatics 23, 2507–2517 (2007)

    Article  Google Scholar 

  46. Wang, W., Zheng, W., Ma, Y.: 3D facial expression recognition based on combination of local features and globe information. In: 2014 Sixth International Conference on Intelligent Human-Machine Systems and Cybernetics, pp. 20–25. IEEE (2014)

    Google Scholar 

  47. Zheng, W.H., Wang, W., Ma, Y.D.: Facial expression recognition based on the texture features of global principal component and local boundary. Appl. Mech. Mater. 548–549, 1110–1117 (2014)

    Article  Google Scholar 

  48. Ali, M., Mosa, A.H., Machot, F.A., Kyamakya, K.: Emotion recognition involving physiological and speech signals: a comprehensive review. In: Kyamakya, K., Mathis, W., Stoop, R., Chedjou, J.C., Li, Z. (eds.) Recent Advances in Nonlinear Dynamics and Synchronization. SSDC, vol. 109, pp. 287–302. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-58996-1_13

    Chapter  Google Scholar 

  49. Han, J., Zhang, Z., Cummins, N., Schuller, B.: Adversarial training in affective computing and sentiment analysis: recent advances and perspectives. arXiv preprint: arXiv:1809.08927 (2018)

  50. Krueger, R.F., et al.: Progress in achieving quantitative classification of psychopathology. World Psychiatry 17, 282–293 (2018)

    Article  Google Scholar 

  51. Swain, M., Routray, A., Kabisatpathy, P.: Databases, features and classifiers for speech emotion recognition: a review. Int. J. Speech Technol. 21, 93–120 (2018)

    Article  Google Scholar 

  52. Wiecki, T.V., Poland, J., Frank, M.J.: Model-based cognitive neuroscience approaches to computational psychiatry: clustering and classification. Clin. Psychol. Sci. 3, 378–399 (2015)

    Article  Google Scholar 

  53. Dong, Y., Su, H., Zhu, J., Zhang, B.: Improving interpretability of deep neural networks with semantic information. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4306–4314 (2017)

    Google Scholar 

  54. Cui, X., Bray, S., Bryant, D.M., Glover, G.H., Reiss, A.L.: A quantitative comparison of NIRS and fMRI across multiple cognitive tasks. Neuroimage 54, 2808–2821 (2011)

    Article  Google Scholar 

  55. Fan, T., Wu, X., Yao, L., Dong, J.: Abnormal baseline brain activity in suicidal and non-suicidal patients with major depressive disorder. Neurosci. Lett. 534, 35–40 (2013)

    Article  Google Scholar 

  56. Schaefer, R.S., Vlek, R.J., Desain, P.: Music perception and imagery in EEG: alpha band effects of task and stimulus. Int. J. Psychophysiol. 82, 254–259 (2011)

    Article  Google Scholar 

  57. Finn, E.S., Scheinost, D., Finn, D.M., Shen, X., Papademetris, X., Constable, R.T.: Can brain state be manipulated to emphasize individual differences in functional connectivity? Neuroimage 160, 140–151 (2017)

    Article  Google Scholar 

  58. Geerligs, L., Rubinov, M., Henson, R.N.: State and trait components of functional connectivity: individual differences vary with mental state. J. Neurosci. 35, 13949–13961 (2015)

    Article  Google Scholar 

  59. American Psychiatric Association: Diagnostic and Statistical Manual of Mental Disorders (DSM-5®). American Psychiatric Publishing (2013)

    Google Scholar 

  60. World Health Organization: International Classification of Diseases. World Health Organization Press (1990)

    Google Scholar 

  61. Acar, E., Levin-Schwartz, Y., Calhoun, V.D., Adali, T.: ACMTF for fusion of multi-modal neuroimaging data and identification of biomarkers. In: 2017 25th European Signal Processing Conference (EUSIPCO), pp. 643–647. IEEE (2017)

    Google Scholar 

  62. Bänziger, T., Grandjean, D., Scherer, K.R.: Emotion recognition from expressions in face, voice, and body: the Multimodal Emotion Recognition Test (MERT). Emotion 9, 691 (2009)

    Article  Google Scholar 

  63. Sebe, N., Cohen, I., Gevers, T., Huang, T.S.: Multimodal approaches for emotion recognition: a survey. In: Internet Imaging VI, pp. 56–68. International Society for Optics and Photonics (2005)

    Google Scholar 

  64. Tzirakis, P., Trigeorgis, G., Nicolaou, M.A., Schuller, B.W., Zafeiriou, S.: End-to-end multimodal emotion recognition using deep neural networks. IEEE J. Sel. Top. Sig. Process. 11, 1301–1309 (2017)

    Article  Google Scholar 

  65. Argyriou, A., Evgeniou, T., Pontil, M.: Multi-task feature learning. In: Advances in Neural Information Processing Systems, pp. 41–48 (2007)

    Google Scholar 

  66. Poria, S., Chaturvedi, I., Cambria, E., Hussain, A.: Convolutional MKL based multimodal emotion recognition and sentiment analysis. In: 2016 IEEE 16th International Conference on Data Mining (ICDM), pp. 439–448. IEEE (2016)

    Google Scholar 

  67. Zhang, D., Wang, Y., Zhou, L., Yuan, H., Shen, D., Alzheimer’s Disease Neuroimaging Initiative: Multimodal classification of Alzheimer’s disease and mild cognitive impairment. Neuroimage 55, 856–867 (2011)

    Article  Google Scholar 

  68. Sui, J., et al.: Multimodal neuromarkers in schizophrenia via cognition-guided MRI fusion. Nat. Commun. 9, 3028 (2018)

    Article  Google Scholar 

Download references

Acknowledgment

This work was supported by the National Natural Science Foundation of China (grant number: 61210010 and 61632014, to B.H.), the National key research and development program of China (grant number: 2016YFC1307203), and the Program of Beijing Municipal Science & Technology Commission (grant number: Z171100000117005, to B.H.). The authors declare there is no conflict of interest in relation to this work.

Author information

Authors and Affiliations

  1. School of Information Science and Engineering, Lanzhou University, Lanzhou, 730000, People’s Republic of China

    Weihao Zheng, Hanshu Cai, Zhijun Yao, Xiaowei Zhang, Xiaowei Li & Bin Hu

  2. College of Biomedical Engineering and Instrument Science, Zhejiang University, Hangzhou, 310027, People’s Republic of China

    Weihao Zheng

Authors
  1. Weihao Zheng
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Hanshu Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Zhijun Yao
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Xiaowei Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xiaowei Li
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Bin Hu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Bin Hu .

Editor information

Editors and Affiliations

  1. University of Kragujevac, Čačak, Serbia

    Danijela Milošević

  2. South China Normal University, Guangzhou, China

    Yong Tang

  3. Wuhan University of Technology, Wuhan, Hubei, China

    Qiaohong Zu

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Zheng, W., Cai, H., Yao, Z., Zhang, X., Li, X., Hu, B. (2019). Modelling Mental States via Computational Psychophysiology: Benefits and Challenges. In: Milošević, D., Tang, Y., Zu, Q. (eds) Human Centered Computing. HCC 2019. Lecture Notes in Computer Science(), vol 11956. Springer, Cham. https://doi.org/10.1007/978-3-030-37429-7_67

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-37429-7_67

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-37428-0

  • Online ISBN: 978-3-030-37429-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation