skip to main content
chapter

Multimodal assessment of depression from behavioral signals

Published:01 October 2018Publication History
First page image

References

  1. L. A. Abel, L. Friedman, J. Jesberger, A. Malki, and H. Y. Meitzer. 1991. Quantitative assessment of smooth pursuit gain and catch-up saccades in schizophrenia and affective disorders. Biological Psychiatry 29: 1063--1072. 394Google ScholarGoogle ScholarCross RefCross Ref
  2. S. Alghowinem, R. Goecke, M. Wagner, J. Epps, M. Breakspear, and G. Parker. 2012. From joyous to clinically depressed: mood detection using spontaneous speech. In Proceedings of the International FLAIRS Conference AAAI, Marco Island, FL. pp. 141--146. 387, 390, 391, 399Google ScholarGoogle Scholar
  3. S. Alghowinem, R. Goecke, M. Wagner, G. Parker, and M. Breakspear. 2013a. Eye movement analysis for depression detection. In Proceedings of the IEEE International Conference on Image Processing, pp. 4220--4224. 394Google ScholarGoogle Scholar
  4. S. Alghowinem, R. Goecke, M. Wagner, G. Parker, and M. Breakspear. 2013b. Head pose and movement analysis as an indicator of depression. In Proceedings of the International Conference on Affective Computing and Intelligent Interaction. pp. 283--288. 394 Google ScholarGoogle ScholarDigital LibraryDigital Library
  5. S. Alghowinem, R. Goecke, M. Wagner, and J. Epps. 2013c. A comparative of different classifiers for detecting depression from spontaneous speech. In Proceedings of the IEEE International Confereence on Acoustics, Speech and Signal Processing. pp. 8022--8026. 390Google ScholarGoogle Scholar
  6. S. Alghowinem, R. Goecke, J. Epps, M. Wagner, and J. F. Cohn. 2016. Cross-cultural depression recognition from vocal biomarkers. in Proceedings of INTERSPEECH. pp. 1943--1947. 402Google ScholarGoogle Scholar
  7. M. Alpert, E. R. Pouget, and R. R. Silva. 2001. Reflections of depression in acoustic measures of the patient's speech. Journal of Affective Disorders 66: 59--69. 387Google ScholarGoogle ScholarCross RefCross Ref
  8. American Psychiatric Association. 2015. Diagnostic and Statistical Manual of Mental Disorders?: DSM-V. 5th ed. American Psychiatric Association, Washington, DC. 376, 379Google ScholarGoogle Scholar
  9. A. Asthana, A. Khwaja, and R. Goecke. 2009. Automatic frontal face annotation and AAM building for arbitrary expressions from a single frontal image only. In Proceedings of the IEEE International Conference on Image Processing, Cairo, Egypt. pp. 2445--2448. 383 Google ScholarGoogle ScholarDigital LibraryDigital Library
  10. A. Asthana, S. Lucey, and R. Goecke. 2011. Regression based automatic face annotation for deformable model building. Pattern Recognition 44(10-11): 2598--2613. Google ScholarGoogle ScholarDigital LibraryDigital Library
  11. S. Baker, I. Matthews, and J. Schneider. 2004. Automatic construction of active appearance models as an image coding problems. IEEE Transactions on Pattern Analysis and Machine Intelligence, 26(10), 1380--1384. 383 Google ScholarGoogle ScholarDigital LibraryDigital Library
  12. T. Baltrusaitis, P. Robinson, and L.-P. Morency. 2012. 3D constrained local model for rigid and non-rigid facial tracking. In Proceedings of IEEE Computer Vision and Pattern Recognition, Providence, RI. 383 Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. A. T. Beck, R. A. Steer, R. Ball, and W. F. Ranieri. 1996. Comparison of Beck depression inventories-IA and-II in psychiatric outpatients. Journal of Personality Assessment 67: 588--597. 379Google ScholarGoogle Scholar
  14. G. W. Blair-West, C. H. Cantor, G. W. Mellsop, and M. L. Eyeson-Annan. 1999. Lifetime suicide risk in major depression: sex and age determinants. Journal of Affective Disorders 55(2-3): 171--178. 376Google ScholarGoogle ScholarCross RefCross Ref
  15. M. Blais and L. Baer. 2010. Understanding rating scales and assessment instruments. In L. Baer and M.A. Blais, editors, Handbook of Clinical Rating Scales and Assessment in Psychiatry and Mental Health, Current Clinical Psychiatry. Humana Press, New York. 379Google ScholarGoogle Scholar
  16. K. L. Burkhouse, G. J. Siegle, M. L. Woody, A. Y. Kudinova, and B. E. Gibb. 2015. Pupillary reactivity to sad stimuli as a biomarker of depression risk: Evidence from a prospective study of children. Journal of Abnormal Psychology 124(3): 498. 394Google ScholarGoogle ScholarCross RefCross Ref
  17. L. M. Bylsma, B. H. Morris, and J. Rottenberg. 2008. A meta-analysis of emotional reactivity in major depressive disorder. Clinical Psychology Review 28(4): 676--691. 392Google ScholarGoogle Scholar
  18. M. E. Calkins, W. G. Iacono, and D. S. Ones. 2008. Eye movement dysfunction in first-degree relatives of patients with schizophrenia: A meta-analytic evaluation of candidate endophenotypes. Brain and Cognition 68: 436--461. 394Google ScholarGoogle ScholarCross RefCross Ref
  19. S. B. Campbell and J. F. Cohn. 1991. Prevalence and correlates of postpartum depression in first-time mothers. Journal of Abnormal Psychology 100(4): 594--599.Google ScholarGoogle ScholarCross RefCross Ref
  20. B. J. Carroll. 1984. Dexamethasone suppression test. In R. C. W. Hall and T. P. Beresford, editors, Handbook of Psychiatric Diagnosis, vol. 1, pp. 3--28. Springer, Netherlands. 379Google ScholarGoogle Scholar
  21. M. De Choudhury, M. Gamon, S. Counts, and E. Horvitz. 2013. Predicting depression via social media. In Proceedings of the AAAI Conference on Web and Social Media.Google ScholarGoogle Scholar
  22. W.-S. Chu, F. De la Torre, and J. F. Cohn. 2016. Selective transfer machine for personalized facial action unit detection. IEEE Transactions on Pattern Analysis and Machine Intelligence 3515--3522. 384Google ScholarGoogle Scholar
  23. J. F. Cohn, Z. Ambadar, and P. Ekman. 2007. Observer-based measurement of facial expression with the facial action coding system. In J. A. Coan and J. J. B. Allen, editors, Handbook of Emotion Elicitation and Assessment. Oxford University Press Series in Affective Science, pp. 203--221. Oxford University, New York. 382Google ScholarGoogle Scholar
  24. J. F. Cohn, T. S. Kruez, I. Matthews, Y. Yang, M. H. Nguyen, M. T. Padilla, F. Zhou, and F. De la Torre. 2009. Detecting depression from facial actions and vocal prosody. In Proceedings of the International Conference on Affective Computing and Intelligent Interaction. pp. 1--7. 382, 383, 390, 391Google ScholarGoogle Scholar
  25. J. F. Cohn and F. De la Torre. 2015. Automated face analysis for affective computing. In R. A. Calvo, S. K. D'Mello, J. Gratch, and A. Kappas, editors, Handbook of Affective Computing, pp. 131--150. Oxford, New York. 384Google ScholarGoogle Scholar
  26. T. F. Cootes, G. V. Wheeler, K. N. Walker, and C. J. Taylor. 2002. View-based active appearance models. Image and Vision Computing 20(9-10): 657--664. 383Google ScholarGoogle ScholarCross RefCross Ref
  27. T. F. Cootes, and C. J. Taylor. 2004. Statistical models of appearance for computer vision. Technical Report, University of Manchester.Google ScholarGoogle Scholar
  28. C. Corneanu, M. Oliu, J. F. Cohn, and S. Escalera. 2015. Survey on RGB, thermal, and multimodal approaches for facial expression analysis: History, trends, and affect-related applications. IEEE Transactions on Pattern Analysis and Machine Intelligence, 36 (8), 1548--1568. 384Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. N. Cummins. 2016. Automatic assessment of depression from speech: paralinguistic analysis, modelling and machine learning. Ph.D. Thesis, UNSW Australia. 389, 391, 398Google ScholarGoogle Scholar
  30. N. Cummins, J. Epps, M. Breakspear, and R. Goecke. 2011. An investigation of depressed speech detection: features and normalization. In Proceedings of INTERSPEECH, Florence, Italy. pp. 2997--3000. 390, 391Google ScholarGoogle Scholar
  31. N. Cummins, J. Epps, V. Sethu, M. Breakspear, and R. Goecke. 2013a. Modeling spectral variability for the classification of depressed speech. In Proceedings of INTERSPEECH, Lyon, France. pp. 857--861. 390Google ScholarGoogle Scholar
  32. N. Cummins, J. Epps, V. Sethu, and J. Krajewski. 2014. Variability compensation in small data: oversampled extraction of i-vectors for the classification of depressed speech. In Proceedings of IEEE International Conference on Acoustics, Speech and Signal Processing, Florence, Italy. pp. 970--974. 390, 391, 403Google ScholarGoogle Scholar
  33. N. Cummins, J. Joshi, A. Dhall, V., Sethu, R. Goecke, and J. Epps. 2013. Diagnosis of depression by behavioral signals: a multimodal approach. In Proceedings of the ACM International Workshop on Audio/Visual Emotion Challenge (AVEC'13), Barcelona, Spain. pp. 11--20. 381, 390, 391, 397 Google ScholarGoogle ScholarDigital LibraryDigital Library
  34. N. Cummins, S. Scherer, J. Krajewski, S. Schnieder, J. Epps, and T.F. Quatieri. 2015a. A review of depression and suicide risk assessment using speech analysis. Speech Communication 71: 10--49. 380, 385, 386, 387, 388 Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. N. Cummins, V. Sethu, J. Epps, and J. Krajewski. 2015b. Relevance vector machine for depression prediction. In Proceedings of INTERSPEECH, Dresden, Germany. pp. 110--114. 391, 399Google ScholarGoogle Scholar
  36. N. Cummins, V. Sethu, J. Epps, S. Schnieder, and J. Krajewski. 2015c. Analysis of acoustic space variability in speech affected by depression. Speech Communication 75: 27--49. 388, 390 Google ScholarGoogle ScholarDigital LibraryDigital Library
  37. B. N. Cuthbert and T. R. Insel. 2013. Toward the future of psychiatric diagnosis: the seven pillars of RDoC. BMC Medicine, 11: 126. 380Google ScholarGoogle ScholarCross RefCross Ref
  38. M. E. Dawson, A. M. Schell, J. R. Braaten, and J. J. Catania. 1985. Diagnostic utility of autonomic measures for major depressive disorders. Psychiatry Research 15.4: 261--270. 394Google ScholarGoogle ScholarCross RefCross Ref
  39. M. De Choudhury, M. Gamon, S. Counts, and E. Horvitz. 2013. Predicting depression via social media. In Proceedings of the AAAI Conference on Web and Social Media. 395Google ScholarGoogle Scholar
  40. D. DeVault, R. Artstein, G. Benn, T. Dey, E. Fast, A. Gainer, K. Georgilia, J. Gratch, A. Hartholt, M. Lhommet, G. Lucas, S. Marsella, F. Morbini, A. Nazarian, S. Scherer, G. Stratou, A. Suri, D. Traum, R. Wood, Y. Xu, A. Rizzo, and L.P. Morency. 2014. Simsensei kiosk: A virtual human interviewer for healthcare decision support. In Proceedings of Autonomous Agents and Multiagent Systems. pp. 1061--1068. 398, 399 Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. H. Dibeklioglu, Z. Hammal, Y. Yang, and J.F. Cohn. 2015. Multimodal detection of depression in clinical interviews. In Proceedings of the ACM International Conference on Multimodal Interaction, Seattle, WA. 384, 385 Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. H. Dibeklioglu, Z. Hammal, and J. F. Cohn. 2017. Multimodal measurement of depression severity in the context of clinical interviews. IEEE Journal of Biomedical Health Informatics. 402Google ScholarGoogle Scholar
  43. T. Donker, K. Petrie, J. Proudfoot, J. Clarke, Birch, and H. Christensen. 2013. Smartphones for smarter delivery of mental health programs: a systematic review. Journal of Medical Internet Research 15(11). 401Google ScholarGoogle ScholarCross RefCross Ref
  44. W. Eaton, C. Smith, M. Ybarra, C. Muntaner, and A. Tien. 2004. Center for epidemiologic studies depression scale: review and revision (CESD and CESD-R). In M. E. Maruish, editor, The Use of Psychological Testing for Treatment Planning and Outcomes Assessment, Lawrence Erlbaum, pp. 363--377. Lawrence Erlbaum, Mahawah, NJ. 379Google ScholarGoogle Scholar
  45. G. J. Edwards, C. J. Taylor, and T. F. Cootes. 1998. Interpreting face images using active appearance models. Proceedings of the Third IEEE International Conference on Automatic Face and Gesture Recognition, Nara, Japan, 300--305. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. P. Ekman and W. V. Friesen. 1969. The repertoire of nonverbal behavior: Categories, origins, usage, and coding. Semiotica 1(1): 49--98. 393Google ScholarGoogle Scholar
  47. P. Ekman, and W. V. Friesen. 1978. Facial action coding system. Palo Alto, CA: Consulting Psychologists Press.Google ScholarGoogle Scholar
  48. P. Ekman, W. V. Friesen, and J. C. Hager. 2002. Facial Action Coding System: Research Nexus. Network Research Information, Salt Lake City, UT 382Google ScholarGoogle Scholar
  49. L. A. Fairbanks, M. T. McGuire, and C. J. Harris. 1982. Nonverbal interaction of patients and therapists during psychiatric interviews. Journal of Abnormal Psychology 91(2): 109. 392, 393Google ScholarGoogle ScholarCross RefCross Ref
  50. M. B. First, J. B. W. Williams, R. S. Karg, and R. L. Spitzer. 2015. Structured Clinical Interview for DSM-5---Research Version. American Psychiatric Association, Arlington, VA. 379Google ScholarGoogle Scholar
  51. A. J. Flint, S. E. Black, I. Campbell-Taylor, G. F. Gailey, and C. Levinton. 1993. Abnormal speech articulation, psychomotor retardation, and subcortical dysfunction in major depression. Journal of Psychiatric Research 27: 309--319. 387Google ScholarGoogle ScholarCross RefCross Ref
  52. J. P. Forgas. 2014. Four Ways Sadness May be Good for You. The Greater Good Science Center. University of California Berkeley, Berkeley, California. 376Google ScholarGoogle Scholar
  53. D. J. France, R. G. Shiavi, S. Silverman, M. Silverman, and M. Wilkes. 2000. Acoustical properties of speech as indicators of depression and suicidal risk. IEEE Transactions on Biomedical Engineering 47: 829--837. 387, 388Google ScholarGoogle Scholar
  54. S. Ghosh, E. Laksana, S. Scherer, and L.-P. Morency. 2015. A multi-label convolutional neural network approach to cross-domain action unit detection. In Proceedings of the International Conference on Affective Computing and Intelligent Interaction, pp. 609--615. 384 Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. R. D. Gibbons, D. J. Weiss, P. A. Pilkonis, E. Frank, T. Moore, J. B. Kim, and D. J. Kupfer. 2012. Development of a computerized adaptive test for depression. Archives of General Psychiatry. 69(11): 11041112. 379Google ScholarGoogle ScholarCross RefCross Ref
  56. J. M. Girard, J. F. Cohn, M. H. Mahoor, S. Mavadati, and D. P. Rosenwald. 2013. Social risk and depression: Evidence from manual and automatic facial expression analysis. In Proceedings of. IEEE International Conference on Automatic Face and Gesture Recognition, pp. 1--8. 381, 382, 383Google ScholarGoogle Scholar
  57. J. M. Girard, J. F. Cohn, L. A. Jeni, M. A. Sayette, and F. De la Torre. 2014a. Spontaneous facial expression can be measured automatically. Behavior Research Methods 47(4): 1136--1147. 384, 402Google ScholarGoogle ScholarCross RefCross Ref
  58. J. M. Girard, J. F. Cohn, M. H. Mahoor, S. M. Mavadati, Z. Hammal, and D. P. Rosenwald. 2014b. Nonverbal social withdrawal in depression: Evidence from manual and automatic analyses. Image and Vision Computing 32(10): 641--647. 382, 392Google ScholarGoogle ScholarCross RefCross Ref
  59. J. M. Girard and J. F. Cohn. 2015. Automated audiovisual depression analysis. Current Opinion in Psychology 4: 75--79. 380Google ScholarGoogle ScholarCross RefCross Ref
  60. T. Glenn and S. Monteith. 2014. New measures of mental state and behavior based on data collected from sensors, smartphones, and the Internet. Current Psychiatry Reports 16(12): 1--10. 395, 400Google ScholarGoogle ScholarCross RefCross Ref
  61. I. H. Gotlib and C. L. Hammen. 2002. Handbook of Depression. Guilford, New York.Google ScholarGoogle Scholar
  62. J. Gratch, R. Artstein, G. Lucas, S. Scherer, A. Nazarian, R. Wood, J. Boberg, D. DeVault, S. Marsella, D. Traum, A. Rizzo, and L.-P. Morency. 2014. The distress analysis interview corpus of human and computer interviews. In Proceedings of Language Resources and Evaluation Conference, pp. 3123--3128. 399Google ScholarGoogle Scholar
  63. R. Gupta, N. Malandrakis, B. Xiao, T. Guha, M. Van Segbroeck, M. Black, A. Potamianos, and S. Narayanan. 2014. Multimodal prediction of affective dimensions and depression in human-computer interactions. In Proceedings of the ACM International Workshop on Audio/Visual Emotion Challenge {AVEC'14}, Orlando, FL. 397 Google ScholarGoogle ScholarDigital LibraryDigital Library
  64. J. A. Hall, J. A. Harrigan, and R. Rosenthal. 1995. Nonverbal behavior in clinician---patient interaction. Applied and Preventative Psychology 4: 21--37. 386, 393Google ScholarGoogle ScholarCross RefCross Ref
  65. H. Hamilton. 1960. HAMD: A rating scale for depression. Neurosurgy and Psychiatry 23: 56--62.Google ScholarGoogle ScholarCross RefCross Ref
  66. B. S. Helfer, T. F. Quatieri, J. R. Williamson, D. D. Mehta, R. Horwitz, and B. Yu. 2013. Classification of depression state based on articulatory precision. In Proceedings in INTERSPEECH, Lyon, France, pp. 2172--2176. 387, 390, 391Google ScholarGoogle Scholar
  67. HiTOP. (Undated). The hierarchical taxonomy of psychopathology {HiTOP}. http://medicine.stonybrookmedicine.edu/HITOP. 376, 379, 380Google ScholarGoogle Scholar
  68. F. Hönig, A. Batliner, E. Nöth, S. Schnieder, and J. Krajewski. 2014. Automatic modelling of depressed speech: relevant features and relevance of gender. In Proceedings in INTERSPEECH, Singapore, pp. 1248--1252. 387Google ScholarGoogle Scholar
  69. L. Hong, K. Turano, H. O'Neil, L. Hao, I. Wonodi, and R. McMahon. 2008. Refining the predictive pursuit endophenotype in schizophrenia. Biological Psychiatry 63: 458--464. 394Google ScholarGoogle ScholarCross RefCross Ref
  70. R. Horwitz, T. F. Quatieri, B. S. Helfer, B. Yu, J. R. Williamson, and J. Mundt. 2013. On the relative importance of vocal source, system, and prosody in human depression. In Proceedings of IEEE International Conference on Body Sensor Networks, Cambridge, MA, pp. 1--6. 387Google ScholarGoogle Scholar
  71. L. A. Jeni, J. F. Cohn, and T. Kanade. 2015. Dense 3D face alignment from 2d videos in real-time. In Proc. 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition (FG). 384Google ScholarGoogle Scholar
  72. J. Joshi, A. Dhall, R. Goecke, M. Breakspear, and G. Parker. 2012. Neural-net classification for spatio-temporal descriptor based depression analysis. In Proceedings of the International Conference on Pattern Recognition, Tsukuba, Japan, pp. 2634--2638. 386Google ScholarGoogle Scholar
  73. J. Joshi, R. Goecke, S. Alghowinem, A. Dhall, M. Wagner, J. Epps, G. Parker, and M. Breakspear. 2013a. Multimodal Assistive Technologies for Depression Diagnosis and Monitoring. Journal on Multimodal User Interfaces (Special Issue on Multimodal Interfaces for Pervasive Assistance) 7(3): 217--228. 381, 385, 396, 397Google ScholarGoogle ScholarCross RefCross Ref
  74. J. Joshi, R. J., Goecke, M. Breakspear, and G. Parker. 2013b. Can body expressions contribute to automatic depression analysis? In Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition, Shanghai, China. 384Google ScholarGoogle Scholar
  75. J. Joshi, A. Dhall, R. Goecke, and J.F. Cohn. 2013c. Relative body parts movement for automatic depression analysis. In Proceedings of the Conference. on Affective Computing and Intelligent Interaction, pp. 492--497. 392, 393 Google ScholarGoogle ScholarDigital LibraryDigital Library
  76. H. Kaya, F. Çilli, and A. Salah. 2014a. Ensemble CCA for Continuous Emotion Prediction. In Proceedings of the ACM International Workshop on Audio/Visual Emotion Challenge {AVEC'14}, Orlando, FL, pp. 19--26. 391 Google ScholarGoogle ScholarDigital LibraryDigital Library
  77. H. Kaya, F. Eyben, and A.A. Salah. 2014b. CCA based Feature Selection with Application to Continuous Depression Recognition from Acoustic Speech Features. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, Florence, Italy, pp. 3757--3761. 391Google ScholarGoogle Scholar
  78. R. D. Kent and Y.-J. Kim. 2003. Toward an acoustic typology of motor speech disorders. Clinical Linguistics & Phonetics 17: 427--445. 388, 389Google ScholarGoogle ScholarCross RefCross Ref
  79. R. C. Kessler, W. T. Chiu, O. Demler, and E. E. Walters. 2005. Prevalence, severity, and comorbidity of 12-month DSM-IV disorders in the National Comorbidity Survey Replication. Archives of General Psychiatry 62: 617--627. 376Google ScholarGoogle ScholarCross RefCross Ref
  80. A. Kirsch and S. Brunnhuber. 2007. Facial expression and experience of emotions in psychodynamic interviews with patients with PTSD in comparison to healthy subjects. Psychopathology 40(5): 296--302. 393Google ScholarGoogle ScholarCross RefCross Ref
  81. R. Kotov, W. Gamez, F. Schmidt, and D. Watson. 2010. Linking big personality traits to anxiety, depressive, and substance use disorders: A meta-analysis. Psychological Bulletin 136(5): 768--821. . 376Google ScholarGoogle ScholarCross RefCross Ref
  82. E. Kraepelin. 1921. Manic depressive insanity and paranoia. The Journal of Nervous and Mental Disease 53: 350. 385, 386Google ScholarGoogle ScholarCross RefCross Ref
  83. K. Kroenke, R. L. Spitzer, and J. B. W. Williams. 2001. The PHQ-9. Journal of General Internal Medicine 16: 606--613. 379Google ScholarGoogle ScholarDigital LibraryDigital Library
  84. M. E. Larsen, N. Cummins, T. W. Boonstra, B. O' Dea, J. Tighe, F. Shand, J. Epps, and H. Christensen. 2015. The use of technology in suicide prevention. In Proceedings of the IEEE International Conference on Engineering in Medicine and Biology Society, Milan, Italy, pp. 7316--7319. 395Google ScholarGoogle Scholar
  85. D. L. Levy, P. S. Holzman, S. Matthysse, and N. R. Mendell. 1993. Eye tracking dysfunction and schizophrenia: A critical perspective. Schizophrenia Bulletin 19: 461--536. 394Google ScholarGoogle ScholarCross RefCross Ref
  86. L. S. A. Low, N. C. Maddage, M. Lech, L. Sheeber, and N. Allen. 2010. Influence of acoustic low-level descriptors in the detection of clinical depression in adolescents. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, Dallas, Texas, pp. 5154--5157. 390, 391Google ScholarGoogle Scholar
  87. L. S. A. Low, N. C. Maddage, M. Lech, L. B. Sheeber, and N. B. Allen. 2011. Detection of clinical depression in adolescents; speech during family interactions. IEEE Transactions on Biomedical Engineering 58: 574--586. 387, 390Google ScholarGoogle ScholarCross RefCross Ref
  88. C. D. Mathers and D. Loncar. 2011. Projections of global mortality and burden of disease from 2002 to 2030. PLoS Medicine 3(11). 376Google ScholarGoogle Scholar
  89. I. Matthews, and S. Baker. 2004. Active appearance models revisited. International Journal of Computer Vision, 60(2), 135--164. Google ScholarGoogle ScholarDigital LibraryDigital Library
  90. G. McIntyre, R. Göcke, M. Hyett, M. Green, and M. Breakspear. 2009. An approach for automatically measuring facial activity in depressed subjects. In Proceedings of the International Conference on Affective Computing and Intelligent Interaction, pp. 1--8. 400Google ScholarGoogle Scholar
  91. G. McIntyre, R. Göcke, M. Breakspear, and G. Parker. 2011. Facial response to video content in depression. In Proceedings of the ICMI Workshop on Inferring Cognitive and Emotional States from Multimodal Measures. 403Google ScholarGoogle Scholar
  92. P. A. McRae, K. Tjaden, and B. Schoonings. 2002. Acoustic and perceptual consequences of articulatory rate change in Parkinson disease. Journal of Speech, Language, and Hearing Research 45: 35--50. 390Google ScholarGoogle ScholarCross RefCross Ref
  93. D. Milne, C. Paris, H. Christensen, P. Batterham, and B. O'Dea. 2015. We feel: taking the emotional pulse of the world. In Proceedings of the Congress of the International Ergonomics Association, Melbourne, Australia. 395Google ScholarGoogle Scholar
  94. V. Mitra, E. Shriberg, M. McLaren, A. Kathol, C. Richey, D. Vergyri, and M. Graciarena. 2014. The SRI AVEC-2014 evaluation system. In Proceedings of the ACM International Workshop on Audio/Visual Emotion Challenge (AVEC'14), Orlando, Florida, pp. 93--101. 391 Google ScholarGoogle ScholarDigital LibraryDigital Library
  95. R. Menke. 2011. Examining nonverbal shame markers among post-pregnancy women with maltreatment histories. Doctoral dissertation, Wayne State University, Detroit, Michigan. 393Google ScholarGoogle Scholar
  96. E. Moore, M. A. Clements, J. W. Peifer, and L. Weisser. 2008. Critical analysis of the impact of glottal features in the classification of clinical depression in speech. IEEE Transactions on Biomedical Engineering 55: 96--107. 391Google ScholarGoogle ScholarCross RefCross Ref
  97. J. C. Mundt, P. J. Snyder, M. S. Cannizzaro, K. Chappie, and D. S. Geralts. 2007. Voice acoustic measures of depression severity and treatment response collected via interactive voice response (IVR) technology. Journal of Neurolinguistics 20: 50--64. 387Google ScholarGoogle ScholarCross RefCross Ref
  98. J. C. Mundt, A. P. Vogel, D. E. Feltner, and W. R. Lenderking. 2012. Vocal acoustic biomarkers of depression severity and treatment response. Biological Psychiatry, 72: 580--587. 387Google ScholarGoogle ScholarCross RefCross Ref
  99. T. Nguyen, D. Phung, B. Dao, S. Venkatesh, and M. Berk. 2014. Affective and content analysis of online depression communities. IEEE Transactions on Affective Computing 5(3): 217--226. 395Google ScholarGoogle Scholar
  100. T. Nguyen, B. O'Dea, M. Larsen, D. Phung, S. Venkatesh, and H. Christensen. 2017. Using linguistic and topic analysis to classify sub-groups of online depression communities. Multimedia Tools and Applications 1--24. 395 Google ScholarGoogle ScholarDigital LibraryDigital Library
  101. A. Nilsonne, J. Sundberg, S. Ternstrom, and A. Askenfelt. 1987. Measuring the rate of change of voice fundamental frequency in fluent speech during mental depression. The Journal of the Acoustical Society of America 83: 716--728. 387Google ScholarGoogle ScholarCross RefCross Ref
  102. M. W. O'Hara, E. M. Zekoski, L. H. Philipps, and E. J. Wright. 1990. Controlled prospective study of postpartum mood disorders: comparison of childbearing and nonchild-bearing women. Journal of Abnormal Psychology 99(1): 3--15.Google ScholarGoogle ScholarCross RefCross Ref
  103. K. E. B. Ooi, M. Lech, and N. B. Allen. 2013. Multichannel weighted speech classification system for prediction of major depression in adolescents. IEEE Transactions on Biomedical Engineering 60: 497--506. 390, 391Google ScholarGoogle ScholarCross RefCross Ref
  104. J. R. Orozco-Arroyave, E. A. Belalcázar-Bolanos, J. D. Arias-Londoño, J. F. Vargas-Bonilla, T. Haderlein, and E. Nöth. 2014. Phonation and articulation analysis of spanish vowels for automatic detection of Parkinson's Disease. In. P. Sojka, A. Horák, I. Kopeček, and K. Pala, editors, Text, Speech and Dialogue, Lecture Notes in Computer Science, pp. 374--381. Springer International Publishing, Cham, Switzerland. 390Google ScholarGoogle Scholar
  105. S. D. Østergaard, S. O. W. Jensen, and P. Bech. 2011. The heterogeneity of the depressive syndrome: when numbers get serious. Acta Psychiatrica Scandinavica 124: 495--496. 379Google ScholarGoogle ScholarCross RefCross Ref
  106. A. Ozdas, R. G. Shiavi, S. E. Silverman, M. K. Silverman, and D. M. Wilkes. 2004. Investigation of vocal jitter and glottal flow spectrum as possible cues for depression and near-term suicidal risk. IEEE Transactions on Biomedical Engineering 51: 1530--1540. 388Google ScholarGoogle ScholarCross RefCross Ref
  107. M. C. Pachucki, E. J. Ozer, A. Barrat, and C. Cattuto. 2015. Mental health and social networks in early adolescence: a dynamic study of objectively-measured social interaction behaviors. Social Science & Medicine 125: 40--50. 395Google ScholarGoogle ScholarCross RefCross Ref
  108. M. Park, C. Cha, and M. Cha. 2012. Depressive moods of users portrayed in twitter. In Proceedings of the ACM SIGKDD Workshop on Healthcare Informatics, pp. 1--8. 395Google ScholarGoogle Scholar
  109. J. E. Perez and R. E. Riggio. 2003. Nonverbal social skills and psychopathology. Nonverbal Behavior in Clinical Settings, 17--44. 392, 393, 394Google ScholarGoogle Scholar
  110. H. Pérez, H. J. Escalante, L. Villaseñor-Pineda, M. Montes-y-Gómez, D. Pinto-Avedaño, and V. Reyes-Meza. 2014. Fusing affective dimensions and audio-visual features from segmented video for depression recognition. In Proceedings of the ACM International Workshop on Audio/Visual Emotion Challenge (AVEC'14), Orlando, FL, pp. 49--55. 391 Google ScholarGoogle ScholarDigital LibraryDigital Library
  111. J. Proudfoot, J. Clarke, M.-R. Birch, A.E. Whitton, G. Parker, V. Manicavasagar, V. Harrison, H. Christensen, and D. Hadzi-Pavlovic. 2013. Impact of a mobile phone and web program on symptom and functional outcomes for people with mild-to-moderate depression, anxiety and stress: a randomised controlled trial. BMC Psychiatry 13.1: 1. 395Google ScholarGoogle ScholarCross RefCross Ref
  112. T. F. Quatieri and N. Malyska. 2012. Vocal-source biomarkers for depression: a link to psychomotor activity. In Proceedings of INTERSPEECH, Portland, OR, pp. 1059--1062. 387, 388Google ScholarGoogle Scholar
  113. L. S. Radloff. 1977. The CES-D scale: A self-report depression scale for research in the general population. Applied Psychological Measurement 1(3):385--401. 379Google ScholarGoogle Scholar
  114. L. I. Reed, M. A. Sayette, and J. F. Cohn. 2007. Impact of depression on response to comedy: A dynamic facial coding analysis. Journal of Abnormal Psychology 116(4): 804--809. 382Google ScholarGoogle ScholarCross RefCross Ref
  115. J. N. Rosenquist, J. H. Fowler, and N. A. Christakis. 2011. Social network determinants of depression. Molecular Psychiatry 16: 273--281. 395Google ScholarGoogle ScholarCross RefCross Ref
  116. S. Sapir, L. O. Ramig, J. L. Spielman, and C. Fox. 2010. Formant centralization ratio: a proposal for a new acoustic measure of dysarthric speech. Journal of Speech, Language, and Hearing Research 53: 114--125. ). 390Google ScholarGoogle ScholarCross RefCross Ref
  117. J. M. Saragih, S. Lucey, and J. F. Cohn. 2009. Subspace constrained mean-shift. In Proceedings of the IEEE International Conference on Computer Vision, Kyoto, Japan. 383Google ScholarGoogle Scholar
  118. J. T. M. Schelde. 1998. Major depression: Behavioral markers of depression and recovery. The Journal of Nervous and Mental Disease 186(3): 133--140. 392, 394Google ScholarGoogle ScholarCross RefCross Ref
  119. S. Scherer, G. Stratou, J. Gratch, and L.-P. Morency. 2013a. Investigating voice quality as a speaker-independent indicator of depression and PTSD. In Proceedings of INTERSPEECH, Lyon, France, pp. 847--851. 387, 390, 391Google ScholarGoogle Scholar
  120. S. Scherer, G. Stratou, M. Mahmoud, J. Boberg, and J. Gratch. 2013b. Automatic behavior descriptors for psychological disorder analysis. In Proceedings of the IEEE International Conference on Automatic Face and Gesture Recognition, Shanghai, China, pp. 1--8. 387, 390, 392, 393, 394Google ScholarGoogle Scholar
  121. S. Scherer, G. Stratou, and L.P. Morency. 2013c. Audiovisual Behavior Descriptors for Depression Assessment. In Proceedings of the ACM International Conference on Multimodal Interaction, Sydney, Australia, pp. 135--140. 387, 391, 394 Google ScholarGoogle ScholarDigital LibraryDigital Library
  122. S. Scherer, G. Stratou, G. Lucus, M. Mahmoud, J. Boberg, J. Gratch, A. (`Skip')_ Rizzo, L.-P. Morency, and G. Lucas. 2014. Automatic audiovisual behavior descriptors for psychological disorder analysis. Image and Vision Computing 32: 1--21. 381, 382, 392, 393, 394, 402Google ScholarGoogle Scholar
  123. S. Scherer, L. P. Morency, J. Gratch, and J. Pestian. 2015. Reduced vowel space is a robust indicator of psychological distress: A cross-corpus analysis. In Proceedings of the IEEE International Conference on Acoustics, Speech and Signal Processing, Brisbane, Australia, pp. 4789--4793. 388, 390Google ScholarGoogle Scholar
  124. S. Scherer, G. M. Lucas, J. Gratch, A. Skip Rizzo and L. P. Morency. 2016. Self-reported symptoms of depression and PTSD are associated with reduced vowel space in screening interviews. In IEEE Transactions on Affective Computing 7(1): 59--73. 387, 388, 390, 393 Google ScholarGoogle ScholarDigital LibraryDigital Library
  125. S. M. Schueller, M. Begale, F. J. Penedo, and D. C. Mohr. 2014. Purple: A modular system for developing and deploying behavioral intervention technologies. Journal of Medical Internet Research 16.7: e181. 395Google ScholarGoogle ScholarCross RefCross Ref
  126. M. Senoussaoui, M. Sarria-Paja, J. F. Santos, and T. H. Falk. 2014. Model fusion for multimodal depression classification and level detection. In Proceedings of the ACM International Workshop on Audio/Visual Emotion Challenge (AVEC'14), Orlando, FL, pp. 57--63. 391 Google ScholarGoogle ScholarDigital LibraryDigital Library
  127. D. V. Sheehan, Y. Lecrubier, K. Harnett-Sheehan, J. Janavs, E. Weiller, L. Bonora, A. Keskiner, J. Schinka, E. Knapp, M. F. Sheehan, and G. C. Dunbar. 1997. Reliability and validity of the M.I.N.I. International Neuropsychiatric Interview (M.I.N.I.). According to the SCID-P. European Psychiatry 12: 232--241. 379Google ScholarGoogle ScholarCross RefCross Ref
  128. M. Sidorov and W. Minker. 2014. Emotion recognition and depression diagnosis by acoustic and visual features: a multimodal approach. In Proceedings of the ACM Internatinal Workshop on Audio/Visual Emotion Challenge (AVEC'14), Orlando, FL, pp. 81--86. 391 Google ScholarGoogle ScholarDigital LibraryDigital Library
  129. G. J. Siegle, E. Granholm, R. E. Ingram, and G. E. Matt. 2001. Pupillary and reaction time measures of sustained processing of negative information in depression. Biological Psychiatry, 49.7: 624--636. 394Google ScholarGoogle ScholarCross RefCross Ref
  130. G. J. Siegle, S. R. Steinhauer, C. S. Carter, W. Ramel, and M. E. Thase. 2003. Do the seconds turn into hours? Relationships between sustained pupil dilation in response to emotional information and self-reported rumination. Cognitive Therapy and Research 27(3): 365--382. 394Google ScholarGoogle ScholarCross RefCross Ref
  131. S. Skodda, W. Grönheit, and U. Schlegel. 2012. Impairment of vowel articulation as a possible marker of disease progression in Parkinson's disease. PLoS One 7: e32132. 390Google ScholarGoogle ScholarCross RefCross Ref
  132. C. Sobin and H. A. Sackeim. 1997. Psychomotor symptoms of depression. American Journal of Psychiatry 154: 4--17. 386Google ScholarGoogle ScholarCross RefCross Ref
  133. H. H. Stassen, M. Albers, J. Püschel, C. Scharfetter, M. Tewesmeier, and B. Woggon. 1995. Speaking behavior and voice sound characteristics associated with negative schizophrenia. Journal of Psychiatriatric Research 29: 277--296. 387Google ScholarGoogle ScholarCross RefCross Ref
  134. H. H. Stassen, G. Bomben, and E. Gunther. 1991. Speech characteristics in depression. Psychopathology 24: 88--105. 387Google ScholarGoogle ScholarCross RefCross Ref
  135. D. Sturim, P. A. Torres-Carrasquillo, T. F. Quatieri, N. Malyska, and A. McCree. 2011. Automatic detection of depression in speech using Gaussian mixture modeling with factor analysis. In Proceedings of. INTERSPEECH, Florence, Italy, pp. 2983--2986. 390Google ScholarGoogle Scholar
  136. F. Tolkmitt, H. Helfrich, R. Standke, and K. R. Scherer. 1982. Vocal indicators of psychiatric treatment effects in depressives and schizophrenics. Journal of Communication. Disorders 15: 209--222. 388Google ScholarGoogle ScholarCross RefCross Ref
  137. A. Trevino, T. Quatieri, and N. Malyska. 2011. Phonologically-based biomarkers for major depressive disorder. EURASIP Journal of Advanced Signal Processing 1--18. 387, 390, 403Google ScholarGoogle Scholar
  138. M. Valstar, B. Schuller, K. Smith, F. Eyben, B. Jiang, S. Bilakhia, S. Schnieder, R. Cowie, and M. Pantic. 2013. AVEC 2013: the continuous audio/visual emotion and depression recognition challenge. In Proceedings of the ACM International Workshop on Audio/Visual Emotion Challenge (AVEC'13), pp. 3--10. 388, 390, 391, 397 Google ScholarGoogle ScholarDigital LibraryDigital Library
  139. M. Valstar. 2014a. Automatic behavior understanding in medicine. In Proceedings of the ACM Workshop on Roadmapping the Future of Multimodal Interaction Research including Business Opportunities and Challenges, pp. 57--60. 381 Google ScholarGoogle ScholarDigital LibraryDigital Library
  140. M. Valstar, B. Schuller, K. Smith, T. Almaev, F. Eyben, J. Krajewski, R. Cowie, and M. Pantic. 2014b. AVEC 2014: 3D dimensional affect and depression recognition challenge. Proceedings of the ACM International Workshop on Audio/Visual Emotion Challenge (AVEC'14), pp. 3--10. 388, 391, 397 Google ScholarGoogle ScholarDigital LibraryDigital Library
  141. M. Valstar, J. Gratch, B. Schuller, F. Ringeval, D. Lalanne, M. Torres, S. Scherer, G. Stratou, R. Cowie, and M. Pantic., 2016. AVEC 2016-depression, mood, and emotion recognition workshop and Challenge. In Proceedings of the ACM International Workshop on Audio/Visual Emotion Challenge {AVEC'16}, pp.3--10. 399 Google ScholarGoogle ScholarDigital LibraryDigital Library
  142. D. Watson and L. A. Clark. 2006. Distress and fear disorders: An alternative empirically based taxonomy of the `mood' and `anxiety' disorders. British Journal of Psychiatry 189(6): 481--483. 379Google ScholarGoogle ScholarCross RefCross Ref
  143. D. Watson, M. W. O'Hara, L. J. Simms, R. Kotov, M. Chmielewski, E. A. McDade-Montez, W. Gamez and S. Stuart. 2007. Development and validation of the inventory of depression and anxiety symptoms (IDAS). Psychological Assesment 9(3): 253--268. 379Google ScholarGoogle ScholarCross RefCross Ref
  144. P. Waxer. 1974. Nonverbal cues for depression. Journal of Abnormal Psychology 83(3): 319. 394Google ScholarGoogle ScholarCross RefCross Ref
  145. J. R. Williamson, T. F. Quatieri, B. S. Helfer, R. Horwitz, B. Yu, and D. D. Mehta. 2013. Vocal biomarkers of depression based on motor incoordination. In Proceedings of the ACM International Workshop on Audio/Visual Emotion Challenge (AVEC'13), Barcelona, Spain, pp. 41--48. 387, 388, 390, 391, 403 Google ScholarGoogle ScholarDigital LibraryDigital Library
  146. J. Williamson, T. Quatieri, B. Helfer, G. Ciccarelli, and D. D. Mehta. 2014. Vocal and facial biomarkers of depression based on motor incoordination and timing. In Proceedings of the ACM International Workshop on Audio/Visual Emotion Challenge (AVEC'14), Orlando, FL, pp. 65--72. 381, 390, 391, 397, 398 Google ScholarGoogle ScholarDigital LibraryDigital Library
  147. World Health Organization. 2014. Global Estimates 2014 Summary Tables. World Health Organization, Geneva, Switzerland. 376Google ScholarGoogle Scholar
  148. X. Xiong and F. De la Torre. 2013. Supervised descent method and its applications to face alignment. In Proceedings of the IEEE International Conference on Computer Vision and Pattern Recognition, Portland, OR. 384 Google ScholarGoogle ScholarDigital LibraryDigital Library
  149. Y. Yang and D. Ramana. 2011. Articulated pose estimation with flexible mixtures-of-parts. In Proceedings of the International Conference on Computer Vision and Pattern Recognition, pp. 1385--1392. 392 Google ScholarGoogle ScholarDigital LibraryDigital Library
  150. Y. Yang, C. Fairbairn, and J. Cohn. 2013. Detecting depression severity from vocal prosody. IEEE Transactions on Affective Computing 4: 142--150. 387 Google ScholarGoogle ScholarDigital LibraryDigital Library
  151. T. Yingthawornsuk, H. K. Keskinpala, D. France, D. M. Wilkes, R. G. Shiavi, and R. M. Salomon. 2006. Objective estimation of suicidal risk using vocal output characteristics. In Proceedings of INTERSPEECH, Pittsburgh, PA, pp. 649--652. 388Google ScholarGoogle Scholar
  152. M. Zimmerman, I. Chelminski, and M. Posternak. 2004. A review of studies of the Hamilton depression rating scale in healthy controls: Implications for the definition of remission in treatment studies of depression. Journal of Nervous & Mental Disease 192(9) 595--601.Google ScholarGoogle ScholarCross RefCross Ref

Index Terms

  1. Multimodal assessment of depression from behavioral signals
    Index terms have been assigned to the content through auto-classification.

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in

    Full Access

    • Published in

      cover image ACM Books
      The Handbook of Multimodal-Multisensor Interfaces: Signal Processing, Architectures, and Detection of Emotion and Cognition - Volume 2
      October 2018
      2034 pages
      ISBN:9781970001716
      DOI:10.1145/3107990

      Publisher

      Association for Computing Machinery and Morgan & Claypool

      Publication History

      • Published: 1 October 2018

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • chapter

      Appears In

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader