research-article

Early Identification of Depression Severity Levels on Reddit Using Ordinal Classification

Authors:

Matloob KhushiAuthors Info & Claims

WWW '22: Proceedings of the ACM Web Conference 2022

Pages 2563 - 2572

https://doi.org/10.1145/3485447.3512128

Published: 25 April 2022 Publication History

Abstract

User-generated text on social media is a promising avenue for public health surveillance and has been actively explored for its feasibility in the early identification of depression. Existing methods in the identification of depression have shown promising results; however, these methods were all focused on treating the identification as a binary classification problem. To date, there has been little effort towards identifying users’ depression severity level and disregard the inherent ordinal nature across these fine-grain levels. This paper aims to make early identification of depression severity levels on social media data. To accomplish this, we built a new dataset based on the inherent ordinal nature over depression severity levels using clinical depression standards on Reddit posts. The posts were classified into 4 depression severity levels covering the clinical depression standards on social media. Accordingly, we reformulate the early identification of depression as an ordinal classification task over clinical depression standards such as Beck’s Depression Inventory and the Depressive Disorder Annotation scheme to identify depression severity levels. With these, we propose a hierarchical attention method optimized to factor in the increasing depression severity levels through a soft probability distribution. We experimented using two datasets (a public dataset having more than one post from each user and our built dataset with a single user post) using real-world Reddit posts that have been classified according to questionnaires built by clinical experts and demonstrated that our method outperforms state-of-the-art models. Finally, we conclude by analyzing the minimum number of posts required to identify depression severity level followed by a discussion of empirical and practical considerations of our study.

References

[1]

Aaron T Beck, Calvin H Ward, Mock Mendelson, Jeremiah Mock, and John Erbaugh. 1961. An inventory for measuring depression. Archives of general psychiatry 4, 6 (1961), 561–571.

[2]

Iz Beltagy, Matthew E Peters, and Arman Cohan. 2020. Longformer: The long-document transformer. arXiv preprint arXiv:2004.05150(2020).

[3]

Adrian Benton, Glen Coppersmith, and Mark Dredze. 2017. Ethical research protocols for social media health research. In Proceedings of the First ACL Workshop on Ethics in Natural Language Processing. 94–102.

[4]

David M Blei, Andrew Y Ng, and Michael I Jordan. 2003. Latent dirichlet allocation. the Journal of machine Learning research 3 (2003), 993–1022.

[5]

Leo Breiman. 2001. Random forests. Machine learning 45, 1 (2001), 5–32.

Digital Library

[6]

Sergio G Burdisso, Marcelo Luis Errecalde, and Manuel Montes y Gómez. 2021. Using Text Classification to Estimate the Depression Level of Reddit Users. Journal of Computer Science & Technology 21 (2021).

[7]

Stevie Chancellor, Michael L Birnbaum, Eric D Caine, Vincent MB Silenzio, and Munmun De Choudhury. 2019. A taxonomy of ethical tensions in inferring mental health states from social media. In Proceedings of the conference on fairness, accountability, and transparency. 79–88.

Digital Library

[8]

Stevie Chancellor and Munmun De Choudhury. 2020. Methods in predictive techniques for mental health status on social media: a critical review. NPJ digital medicine 3, 1 (2020), 1–11.

[9]

Junyoung Chung, Çaglar Gülçehre, KyungHyun Cho, and Yoshua Bengio. 2014. Empirical Evaluation of Gated Recurrent Neural Networks on Sequence Modeling. CoRR abs/1412.3555(2014). arxiv:1412.3555http://arxiv.org/abs/1412.3555

[10]

Glen Coppersmith, Ryan Leary, Patrick Crutchley, and Alex Fine. 2018. Natural language processing of social media as screening for suicide risk. Biomedical informatics insights 10 (2018), 1178222618792860.

[11]

Corinna Cortes and Vladimir Vapnik. 1995. Support-vector networks. Machine learning 20, 3 (1995), 273–297.

Digital Library

[12]

Munmun De Choudhury, Scott Counts, and Eric Horvitz. 2013. Predicting postpartum changes in emotion and behavior via social media. In Proceedings of the SIGCHI conference on human factors in computing systems. 3267–3276.

Digital Library

[13]

Munmun De Choudhury, Michael Gamon, Scott Counts, and Eric Horvitz. 2013. Predicting depression via social media. In Proceedings of the International AAAI Conference on Web and Social Media, Vol. 7.

[14]

Munmun De Choudhury and Emre Kiciman. 2017. The language of social support in social media and its effect on suicidal ideation risk. In Proceedings of the International AAAI Conference on Web and Social Media, Vol. 11.

[15]

Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2019. BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. In Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers). Association for Computational Linguistics, Minneapolis, Minnesota, 4171–4186. https://doi.org/10.18653/v1/N19-1423

[16]

Adam G Dunn, Kenneth D Mandl, and Enrico Coiera. 2018. Social media interventions for precision public health: promises and risks. NPJ digital medicine 1, 1 (2018), 1–4.

[17]

Jacob Eisenstein, Amr Ahmed, and Eric P Xing. 2011. Sparse additive generative models of text. In Proceedings of the 28th international conference on machine learning (ICML-11). Citeseer, 1041–1048.

[18]

Sindhu Kiranmai Ernala, Michael L Birnbaum, Kristin A Candan, Asra F Rizvi, William A Sterling, John M Kane, and Munmun De Choudhury. 2019. Methodological gaps in predicting mental health states from social media: triangulating diagnostic signals. In Proceedings of the 2019 chi conference on human factors in computing systems. 1–16.

Digital Library

[19]

Eibe Frank and Mark Hall. 2001. A simple approach to ordinal classification. In European Conference on Machine Learning. Springer, 145–156.

Digital Library

[20]

Manas Gaur, Amanuel Alambo, Joy Prakash Sain, Ugur Kursuncu, Krishnaprasad Thirunarayan, Ramakanth Kavuluru, Amit Sheth, Randy Welton, and Jyotishman Pathak. 2019. Knowledge-aware assessment of severity of suicide risk for early intervention. In The World Wide Web Conference. 514–525.

Digital Library

[21]

Sepp Hochreiter and Jürgen Schmidhuber. 1997. Long Short-Term Memory. Neural Computation 9, 8 (1997), 1735–1780. https://doi.org/10.1162/neco.1997.9.8.1735

Digital Library

[22]

Y Kim. 2014. Convolutional neural networks for sentence classification., arXiv. preprint (2014).

[23]

Thomas N Kipf and Max Welling. 2016. Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907(2016).

[24]

Zhenzhong Lan, Mingda Chen, Sebastian Goodman, Kevin Gimpel, Piyush Sharma, and Radu Soricut. 2019. ALBERT: A Lite BERT for Self-supervised Learning of Language Representations. arxiv:1909.11942 [cs.CL]

[25]

Xien Liu, Xinxin You, Xiao Zhang, Ji Wu, and Ping Lv. 2020. Tensor graph convolutional networks for text classification. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 34. 8409–8416.

[26]

David E Losada and Fabio Crestani. 2016. A test collection for research on depression and language use. In International Conference of the Cross-Language Evaluation Forum for European Languages. Springer, 28–39.

[27]

David E Losada, Fabio Crestani, and Javier Parapar. 2017. eRISK 2017: CLEF lab on early risk prediction on the internet: experimental foundations. In International Conference of the Cross-Language Evaluation Forum for European Languages. Springer, 346–360.

[28]

David E Losada, Fabio Crestani, and Javier Parapar. 2018. Overview of eRisk: early risk prediction on the internet. In International conference of the cross-language evaluation forum for european languages. Springer, 343–361.

[29]

David E Losada, Fabio Crestani, and Javier Parapar. 2019. Overview of erisk 2019 early risk prediction on the internet. In International Conference of the Cross-Language Evaluation Forum for European Languages. Springer, 340–357.

Digital Library

[30]

Danielle L Mowery, Craig Bryan, and Mike Conway. 2015. Towards developing an annotation scheme for depressive disorder symptoms: A preliminary study using Twitter data. In Proceedings of the 2nd Workshop on Computational Linguistics and Clinical Psychology: From Linguistic Signal to Clinical Reality. 89–98.

[31]

Usman Naseem, Matloob Khushi, Jinman Kim, and Adam G Dunn. 2021. Classifying vaccine sentiment tweets by modelling domain-specific representation and commonsense knowledge into context-aware attentive GRU. arXiv preprint arXiv:2106.09589(2021).

[32]

Usman Naseem and Katarzyna Musial. 2019. Dice: Deep intelligent contextual embedding for twitter sentiment analysis. In 2019 International conference on document analysis and recognition (ICDAR). IEEE, 953–958.

[33]

Usman Naseem, Imran Razzak, Peter Eklund, and Katarzyna Musial. 2020. Towards improved deep contextual embedding for the identification of irony and sarcasm. In 2020 International Joint Conference on Neural Networks (IJCNN). IEEE, 1–7.

[34]

Usman Naseem, Imran Razzak, and Peter W Eklund. 2021. A survey of pre-processing techniques to improve short-text quality: a case study on hate speech detection on twitter. Multimedia Tools and Applications 80, 28 (2021), 35239–35266.

Digital Library

[35]

Usman Naseem, Imran Razzak, Matloob Khushi, Peter W Eklund, and Jinman Kim. 2021. Covidsenti: A large-scale benchmark Twitter data set for COVID-19 sentiment analysis. IEEE Transactions on Computational Social Systems (2021).

[36]

Mark Olfson, Carlos Blanco, and Steven C Marcus. 2016. Treatment of adult depression in the United States. JAMA internal medicine 176, 10 (2016), 1482–1491.

[37]

Ahmed Husseini Orabi, Prasadith Buddhitha, Mahmoud Husseini Orabi, and Diana Inkpen. 2018. Deep learning for depression detection of twitter users. In Proceedings of the Fifth Workshop on Computational Linguistics and Clinical Psychology: From Keyboard to Clinic. 88–97.

[38]

Minsu Park, Chiyoung Cha, and Meeyoung Cha. 2012. Depressive moods of users portrayed in Twitter. (2012).

[39]

Minsu Park, David McDonald, and Meeyoung Cha. 2013. Perception differences between the depressed and non-depressed users in twitter. In Proceedings of the International AAAI Conference on Web and Social Media, Vol. 7.

[40]

James W Pennebaker, Martha E Francis, and Roger J Booth. 2001. Linguistic inquiry and word count: LIWC 2001. Mahway: Lawrence Erlbaum Associates 71, 2001 (2001), 2001.

[41]

Hassan Ramchoun, Mohammed Amine Janati Idrissi, Youssef Ghanou, and Mohamed Ettaouil. 2016. Multilayer Perceptron: Architecture Optimization and Training.Int. J. Interact. Multim. Artif. Intell. 4, 1 (2016), 26–30.

[42]

Guozheng Rao, Chengxia Peng, Li Zhang, Xin Wang, and Zhiyong Feng. 2020. A Knowledge Enhanced Ensemble Learning Model for Mental Disorder Detection on Social Media. In International Conference on Knowledge Science, Engineering and Management. Springer, 181–192.

Digital Library

[43]

Andrew G Reece, Andrew J Reagan, Katharina LM Lix, Peter Sheridan Dodds, Christopher M Danforth, and Ellen J Langer. 2017. Forecasting the onset and course of mental illness with Twitter data. Scientific reports 7, 1 (2017), 1–11.

[44]

Esteban A Ríssola, David E Losada, and Fabio Crestani. 2021. A Survey of Computational Methods for Online Mental State Assessment on Social Media. ACM Transactions on Computing for Healthcare 2, 2 (2021), 1–31.

Digital Library

[45]

Gerard Salton and Michael J McGill. 1983. Introduction to modern information retrieval. mcgraw-hill.

[46]

Ramit Sawhney, Harshit Joshi, Saumya Gandhi, and Rajiv Ratn Shah. 2021. Towards Ordinal Suicide Ideation Detection on Social Media. In Proceedings of the 14th ACM International Conference on Web Search and Data Mining. 22–30.

Digital Library

[47]

Han-Chin Shing, Philip Resnik, and Douglas W Oard. 2020. A prioritization model for suicidality risk assessment. In Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics. 8124–8137.

[48]

Ruba Skaik and Diana Inkpen. 2020. Using Social Media for Mental Health Surveillance: A Review. ACM Computing Surveys (CSUR) 53, 6 (2020), 1–31.

Digital Library

[49]

Hajime Sueki. 2015. The association of suicide-related Twitter use with suicidal behaviour: a cross-sectional study of young internet users in Japan. Journal of affective disorders 170 (2015), 155–160.

[50]

Yoshihiko Suhara, Yinzhan Xu, and Alex Sandy Pentland. 2017. Deepmood: Forecasting depressed mood based on self-reported histories via recurrent neural networks. In Proceedings of the 26th International Conference on World Wide Web. Springer, 715–724.

Digital Library

[51]

Michael M Tadesse, Hongfei Lin, Bo Xu, and Liang Yang. 2019. Detection of depression-related posts in reddit social media forum. IEEE Access 7(2019), 44883–44893.

[52]

Truyen Tran, Dinh Phung, Wei Luo, Richard Harvey, Michael Berk, and Svetha Venkatesh. 2013. An integrated framework for suicide risk prediction. In Proceedings of the 19th ACM SIGKDD international conference on Knowledge discovery and data mining. 1410–1418.

Digital Library

[53]

Elsbeth Turcan and Kathleen McKeown. 2019. Dreaddit: A Reddit dataset for stress analysis in social media. arXiv preprint arXiv:1911.00133(2019).

[54]

Ronghua Xu and Qingpeng Zhang. 2016. Understanding online health groups for depression: social network and linguistic perspectives. Journal of medical Internet research 18, 3 (2016), e63.

[55]

Zichao Yang, Diyi Yang, Chris Dyer, Xiaodong He, Alexander J. Smola, and Eduard H. Hovy. 2016. Hierarchical Attention Networks for Document Classification. In HLT-NAACL.

[56]

Liang Yao, Chengsheng Mao, and Yuan Luo. 2019. Graph convolutional networks for text classification. In Proceedings of the AAAI Conference on Artificial Intelligence, Vol. 33. 7370–7377.

Digital Library

[57]

Andrew Yates, Arman Cohan, and Nazli Goharian. 2017. Depression and self-harm risk assessment in online forums. arXiv preprint arXiv:1709.01848(2017).

[58]

Xujuan Zhou, Enrico Coiera, Guy Tsafnat, Diana Arachi, Mei-Sing Ong, Adam G Dunn, 2015. Using social connection information to improve opinion mining: Identifying negative sentiment about HPV vaccines on Twitter. (2015).

[59]

Mark Zimmerman, Jennifer H Martinez, Diane Young, Iwona Chelminski, and Kristy Dalrymple. 2013. Severity classification on the Hamilton depression rating scale. Journal of affective disorders 150, 2 (2013), 384–388.

[60]

Hamad Zogan, Imran Razzak, Shoaib Jameel, and Guandong Xu. 2021. DepressionNet: A Novel Summarization Boosted Deep Framework for Depression Detection on Social Media. arXiv preprint arXiv:2105.10878(2021).

Cited By

Qasim AMehak GHussain NGelbukh ASidorov G(2025)Detection of Depression Severity in Social Media Text Using Transformer-Based ModelsInformation10.3390/info1602011416:2(114)Online publication date: 7-Feb-2025
https://doi.org/10.3390/info16020114
Khowaja SNkenyereye LKhuwaja PAl Hamadi HDev K(2025)Depression Detection From Social Media Posts Using Emotion Aware Encoders and Fuzzy Based Contrastive NetworksIEEE Transactions on Fuzzy Systems10.1109/TFUZZ.2024.346177633:1(43-53)Online publication date: Jan-2025
https://doi.org/10.1109/TFUZZ.2024.3461776
Dalal STilwani DGaur MJain SShalin VSheth A(2025)A Cross Attention Approach to Diagnostic Explainability Using Clinical Practice Guidelines for DepressionIEEE Journal of Biomedical and Health Informatics10.1109/JBHI.2024.348357729:2(1333-1342)Online publication date: Feb-2025
https://doi.org/10.1109/JBHI.2024.3483577
Show More Cited By

Index Terms

Early Identification of Depression Severity Levels on Reddit Using Ordinal Classification
1. Applied computing
  1. Law, social and behavioral sciences
  2. Life and medical sciences
2. Information systems
  1. Information retrieval
  2. Information systems applications

Index terms have been assigned to the content through auto-classification.

Recommendations

Facebook use, envy, and depression among college students

A survey of 736 college students found that Facebook use can trigger feelings of envy.Feelings of envy were found to predict depression symptoms.The effect of surveillance use of Facebook on depression is mediated by feelings of envy.Surveillance use of ...
A content analysis of depression-related tweets

This study examines depression-related chatter on Twitter to glean insight into social networking about mental health. We assessed themes of a random sample (n = 2000) of depression-related tweets (sent 4-11 to 5-4-14). Tweets were coded for expression ...
Expression of Depression Among Arab Twitter Users Using Arabic Corpus Analysis
Abstract
Expressing emotions on social media is becoming popular by the day. Several studies have documented the relation between how people express their emotions to their mental state e.g., depression. Most of these studies were conducted in English, ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

WWW '22: Proceedings of the ACM Web Conference 2022

April 2022

3764 pages

ISBN:9781450390965

DOI:10.1145/3485447

Editors:
Frédérique Laforest
INSA Lyon, France
,
Raphaël Troncy
EURECOM, France
,
Elena Simperl
King’s College London, UK
,
Deepak Agarwal
Pinterest, USA
,
Aristides Gionis
KTH Royal Institute of Technology, Sweden
,
Ivan Herman
W3C / retired
,
Lionel Médini
Université Lyon 1, France

Copyright © 2022 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGWEB: ACM Special Interest Group on Hypertext, Hypermedia, and Web

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 25 April 2022

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article
Research
Refereed limited

Conference

WWW '22

Sponsor:

SIGWEB

WWW '22: The ACM Web Conference 2022

April 25 - 29, 2022

Virtual Event, Lyon, France

Acceptance Rates

Overall Acceptance Rate 1,899 of 8,196 submissions, 23%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

34
Total Citations
View Citations
1,369
Total Downloads

Downloads (Last 12 months)410
Downloads (Last 6 weeks)49

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Qasim AMehak GHussain NGelbukh ASidorov G(2025)Detection of Depression Severity in Social Media Text Using Transformer-Based ModelsInformation10.3390/info1602011416:2(114)Online publication date: 7-Feb-2025
https://doi.org/10.3390/info16020114
Khowaja SNkenyereye LKhuwaja PAl Hamadi HDev K(2025)Depression Detection From Social Media Posts Using Emotion Aware Encoders and Fuzzy Based Contrastive NetworksIEEE Transactions on Fuzzy Systems10.1109/TFUZZ.2024.346177633:1(43-53)Online publication date: Jan-2025
https://doi.org/10.1109/TFUZZ.2024.3461776
Dalal STilwani DGaur MJain SShalin VSheth A(2025)A Cross Attention Approach to Diagnostic Explainability Using Clinical Practice Guidelines for DepressionIEEE Journal of Biomedical and Health Informatics10.1109/JBHI.2024.348357729:2(1333-1342)Online publication date: Feb-2025
https://doi.org/10.1109/JBHI.2024.3483577
Tahir WKhalid SAlmutairi SAbohashrh MMemon SKhan J(2025)Depression Detection in Social Media: A Comprehensive Review of Machine Learning and Deep Learning TechniquesIEEE Access10.1109/ACCESS.2025.353086213(12789-12818)Online publication date: 2025
https://doi.org/10.1109/ACCESS.2025.3530862
Ding TQu TZou ZDing C(2024)A novel multi-model feature generation technique for suicide detectionPeerJ Computer Science10.7717/peerj-cs.230110(e2301)Online publication date: 28-Oct-2024
https://doi.org/10.7717/peerj-cs.2301
Kumari MSingh GPande S(2024)Depressonify: BERT a deep learning approach of detection of depressionEAI Endorsed Transactions on Pervasive Health and Technology10.4108/eetpht.10.551310Online publication date: 22-Mar-2024
https://doi.org/10.4108/eetpht.10.5513
Liu ZBao YZeng SQian RDeng MGu ALi JWang WCai WLi WWang HXu DLin G(2024)Large Language Models in Psychiatry: Current Applications, Limitations, and Future ScopeBig Data Mining and Analytics10.26599/BDMA.2024.90200467:4(1148-1168)Online publication date: Dec-2024
https://doi.org/10.26599/BDMA.2024.9020046
Xu XYao BDong YGabriel SYu HHendler JGhassemi MDey AWang D(2024)Mental-LLMProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435408:1(1-32)Online publication date: 6-Mar-2024
https://dl.acm.org/doi/10.1145/3643540
Jain MJain SJain AGarg B(2024)CDME-GAT: Context-Aware Depression Detection Using Multiembedding and Graph Attention Networks in Social Media TextIEEE Transactions on Computational Social Systems10.1109/TCSS.2024.340268911:6(7212-7222)Online publication date: Dec-2024
https://doi.org/10.1109/TCSS.2024.3402689
Ilias LMouzakitis SAskounis D(2024)Calibration of Transformer-Based Models for Identifying Stress and Depression in Social MediaIEEE Transactions on Computational Social Systems10.1109/TCSS.2023.328300911:2(1979-1990)Online publication date: Apr-2024
https://doi.org/10.1109/TCSS.2023.3283009
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten