skip to main content
10.1145/3025453.3025932acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article
Public Access

Modeling and Understanding Visual Attributes of Mental Health Disclosures in Social Media

Published: 02 May 2017 Publication History

Abstract

Content shared on social media platforms has been identified to be valuable in gaining insights into people's mental health experiences. Although there has been widespread adoption of photo-sharing platforms such as Instagram in recent years, the role of visual imagery as a mechanism of self-disclosure is less understood. We study the nature of visual attributes manifested in images relating to mental health disclosures on Instagram. Employing computer vision techniques on a corpus of thousands of posts, we extract and examine three visual attributes: visual features (e.g., color), themes, and emotions in images. Our findings indicate the use of imagery for unique self-disclosure needs, quantitatively and qualitatively distinct from those shared via the textual modality: expressions of emotional distress, calls for help, and explicit display of vulnerability. We discuss the relationship of our findings to literature in visual sociology, in mental health self disclosure, and implications for the design of health interventions.

References

[1]
Saeed Abdullah, Elizabeth L Murnane, Jean MR Costa, and Tanzeem Choudhury. 2015. Collective smile: Measuring societal happiness from geolocated images. In Proc. CSCW. 361--374.
[2]
Irwin Altman and Dalmas A Taylor. 1973. Social penetration: The development of interpersonal relationships. Holt, Rinehart & Winston.
[3]
Nazanin Andalibi, Pinar Ozturk, and Andrea Forte. 2015. Depression-related Imagery on Instagram. In CSCW15 Companion.
[4]
Nazanin Andalibi, Pinar Ozturk, and Andrea Forte. 2017. Sensitive Self-disclosures, Responses, and Social Support on Instagram: the case of #depression. In Forthcoming in Proc.CSCW.
[5]
American Psychiatric Association and others. 2013. Diagnostic and statistical manual of mental disorders. American Psychiatric Pub.
[6]
Saeideh Bakhshi Bakhshi, David Shamma, Lyndon Kennedy, and Eric Gilbert. 2015. Why We Filter Our Photos and How It Impacts Engagement. In Proc. ICWSM.
[7]
Sairam Balani and Munmun De Choudhury. 2015. Detecting and characterizing mental health related self-disclosure in social media. In Proc. CHI Extended Abstracts. 1373--1378.
[8]
Herbert Bay, Tinne Tuytelaars, and Luc Van Gool. 2006. Surf: Speeded up robust features. In Proc. ECCV 2006.
[9]
Jay Callahan. 1996. A specific therapeutic approach to suicide risk in borderline clients. Clinical Social Work Journal 24, 4 (1996), 443--459.
[10]
Stevie Chancellor, Zhiyuan Jerry Lin, Erica L Goodman, Stephanie Zerwas, and Munmun De Choudhury. 2016. Quantifying and Predicting Mental Illness Severity in Online Pro-Eating Disorder Communities. In Proc. CSCW.
[11]
Paul C Cozby. 1973. Self-disclosure: a literature review. Psychological bulletin 79, 2 (1973), 73.
[12]
Munmun De Choudhury, Scott Counts, Eric Horvitz, and Aaron Hoff. 2014. Characterizing and Predicting Postpartum Depression from Facebook Data. In Proc. CSCW.
[13]
Munmun De Choudhury and Sushovan De. 2014. Mental Health Discourse on reddit: Self-disclosure, Social Support, and Anonymity. In Proc. ICWSM.
[14]
Munmun De Choudhury, Michael Gamon, Scott Counts, and Eric Horvitz. 2013. Predicting depression via social media. In Proc. ICWSM.
[15]
Maeve Duggan, Nicole B Ellison, Cliff Lampe, Amanda Lenhart, and Mary Madden. 2015. Social media update 2014. Pew Research Center 19 (2015).
[16]
Darren Ellis and John Cromby. 2012. Emotional inhibition: A discourse analysis of disclosure. Psychology & health 27, 5 (2012), 515--532.
[17]
Gunther Eysenbach, John Powell, Marina Englesakis, Carlos Rizo, and Anita Stern. 2004. Health related virtual communities and electronic support groups: systematic review of the effects of online peer to peer interactions. Bmj 328, 7449 (2004), 1166.
[18]
Robert Douglas Ferguson, Michael Massimi, Emily Crist, and Karyn Moffatt. 2014. Craving, Creating, and Constructing Comfort: Insights and Opportunities for Technology in Hospice. In Proc. CSCW.
[19]
Venkata Rama Kiran Garimella, Abdulrahman Alfayad, and Ingmar Weber. 2016. Social Media Image Analysis for Public Health. In Proc. CHI.
[20]
David M Garner and Paul E Garfinkel. 1982. Body image in anorexia nervosa: Measurement, theory and clinical implications. The International Journal of Psychiatry in Medicine 11, 3 (1982), 263--284.
[21]
Val Gillies, Angela Harden, Katherine Johnson, Paula Reavey, Vicky Strange, and Carla Willig. 2005. Painting pictures of embodied experience: The use of nonverbal data production for the study of embodiment. Qualitative research in psychology 2, 3 (2005), 199--212.
[22]
Erving Goffman. 2009. Stigma: Notes on the management of spoiled identity. Simon and Schuster.
[23]
Jacob Goldberger, Hayit Greenspan, and Shiri Gordon. 2002. Unsupervised image clustering using the information bottleneck method. In Joint Pattern Recognition Symposium. Springer, 158--165.
[24]
Jonathan Harel, Christof Koch, and Pietro Perona. 2006. Graph-based visual saliency. In Proc. NIPS.
[25]
Douglas Harper. 2002. Talking about pictures: A case for photo elicitation. Visual studies 17, 1 (2002), 13--26.
[26]
Barbara Harrison. 2002. Seeing health and illness worlds--using visual methodologies in a sociology of health and illness: a methodological review. Sociology of Health & Illness 24, 6 (2002), 856--872.
[27]
Christopher M Homan, Naiji Lu, Xin Tu, Megan C Lytle, and Vincent Silenzio. 2014. Social structure and depression in TrevorSpace. In Proc. CSCW.
[28]
Yuheng Hu, Lydia Manikonda, and Subbarao Kambhampati. 2014. What We Instagram: A First Analysis of Instagram Photo Content and User Types. In Proc. ICWSM.
[29]
Jina Huh and Mark S Ackerman. 2012. Collaborative help in chronic disease management: supporting individualized problems. In Proc. CSCW.
[30]
Adam N Joinson and Carina B Paine. 2007. Self-disclosure, privacy and the Internet. The Oxford handbook of Internet psychology (2007), 2374252.
[31]
Dhiraj Joshi, Ritendra Datta, Elena Fedorovskaya, Quang-Tuan Luong, James Z Wang, Jia Li, and Jiebo Luo. 2011. Aesthetics and emotions in images. Signal Processing Magazine, IEEE 28, 5 (2011), 94--115.
[32]
Sidney M Jourard. 1959. Healthy personality and self-disclosure. Mental Hygiene. NY (1959).
[33]
Dacher Keltner. 2004. Ekman, emotional expression, and the art of empirical epiphany. Journal of Research in Personality 38, 1 (2004), 37--44.
[34]
Lyndon Kennedy, Mor Naaman, Shane Ahern, Rahul Nair, and Tye Rattenbury. 2007. How flickr helps us make sense of the world: context and content in community-contributed media collections. In Proc. ACM MM.
[35]
E David Klonsky. 2007. The functions of deliberate self-injury: A review of the evidence. Clinical psychology review 27, 2 (2007), 226--239.
[36]
Alex Krizhevsky, Ilya Sutskever, and Geoffrey E Hinton. 2012. Imagenet classification with deep convolutional neural networks. In Proc. NIPS.
[37]
Diana MacLean, Sonal Gupta, Anna Lembke, Christopher Manning, and Jeffrey Heer. 2015. Forum77: An Analysis of an Online Health Forum Dedicated to Addiction Recovery. In Proc. CSCW.
[38]
Lena Mamykina, Andrew D Miller, Elizabeth D Mynatt, and Daniel Greenblatt. 2010. Constructing identities through storytelling in diabetes management. In Proc. CHI.
[39]
Lydia Manikonda, Vamsi Meduri, and Subbarao Kambhampati. 2016. Tweeting the Mind and Instagramming the Heart: Exploring Differentiated Content Sharing on Social Media. In Proc. ICWSM.
[40]
Alice E Marwick. 2015. Instafame: Luxury selfies in the attention economy. Public Culture 27, 1 75 (2015), 137--160.
[41]
Ran Pang, Agustin Baretto, Henry Kautz, and Jiebo Luo. 2015. Monitoring adolescent alcohol use via multimodal analysis in social multimedia. In Proc. IEEE Big Data.
[42]
James W Pennebaker. 1990. Opening up: The healing power of confiding in others. William Morrow.
[43]
James W Pennebaker and Cindy K Chung. 2007. Expressive writing, emotional upheavals, and health. Foundations of health psychology (2007), 263--284.
[44]
A. G. Reece and C. M. Danforth. 2016. Instagram photos reveal predictive markers of depression. ArXiv e-prints (aug 2016).
[45]
Michael Rich, Jennifer Patashnick, and Richard Chalfen. 2002. Visual illness narratives of asthma: explanatory models and health-related behavior. American journal of health behavior 26, 6 (2002), 442--453.
[46]
Meredith M Skeels, Kenton T Unruh, Christopher Powell, and Wanda Pratt. 2010. Catalyzing social support for breast cancer patients. In Proc. CHI.
[47]
Brian K Smith, Jeana Frost, Meltem Albayrak, and Rajneesh Sudhakar. 2006. Facilitating narrative medical discussions of type 1 diabetes with computer visualizations and photography. Patient Education and Counseling 64, 1 (2006), 313--321.
[48]
Joshua M Smyth. 1998. Written emotional expression: effect sizes, outcome types, and moderating variables. Journal of consulting and clinical psychology 66, 1 (1998), 174.
[49]
Thomas Stephens. 1988. Physical activity and mental health in the USA and Canada: evidence from four population surveys. Preventive medicine 17, 1 (1988), 35--47.
[50]
Markus A Stricker and Markus Orengo. 1995. Similarity of color images. In IS&T/SPIEs Symposium on Electronic Imaging: Science & Technology. International Society for Optics and Photonics, 381--392.
[51]
Alise Tifentale and Lev Manovich. 2015. Selfiecity: Exploring photography and self-fashioning in social media. In Postdigital Aesthetics. Springer, 109--122.
[52]
Ingmar Weber and Yelena Mejova. 2016. Crowdsourcing Health Labels: Inferring Body Weight from Profile Pictures. In Proc. ACM Digital Health.
[53]
Paul Wicks, Michael Massagli, Jeana Frost, Catherine Brownstein, Sally Okun, Timothy Vaughan, Richard Bradley, and James Heywood. 2010. Sharing health data for better outcomes on PatientsLikeMe. Journal of medical Internet research 12, 2 (2010).
[54]
Wayne Xin Zhao, Jing Jiang, Jianshu Weng, Jing He, Ee-Peng Lim, Hongfei Yan, and Xiaoming Li. 2011. Comparing Twitter and Traditional Media Using Topic Models. In Proc. ECIR.

Cited By

View all
  • (2025)Sentiment Analysis of #Meanspo Tweets: Humans vs. Automatic ClassificationsProceedings of the ACM on Human-Computer Interaction10.1145/37012079:1(1-26)Online publication date: 10-Jan-2025
  • (2024)The Subtleties of Self-Presentation: A study of sensitive disclosure among sexual minority adolescentsProceedings of the ACM on Human-Computer Interaction10.1145/36374088:CSCW1(1-27)Online publication date: 26-Apr-2024
  • (2024)Images Connect Us Together: Navigating a COVID-19 Local Outbreak in China Through Social Media ImagesProceedings of the ACM on Human-Computer Interaction10.1145/36373498:CSCW1(1-32)Online publication date: 26-Apr-2024
  • Show More Cited By

Index Terms

  1. Modeling and Understanding Visual Attributes of Mental Health Disclosures in Social Media

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    CHI '17: Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems
    May 2017
    7138 pages
    ISBN:9781450346559
    DOI:10.1145/3025453
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 02 May 2017

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. instagram
    2. mental health
    3. social media
    4. visual attributes

    Qualifiers

    • Research-article

    Funding Sources

    Conference

    CHI '17
    Sponsor:

    Acceptance Rates

    CHI '17 Paper Acceptance Rate 600 of 2,400 submissions, 25%;
    Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

    Upcoming Conference

    CHI 2025
    ACM CHI Conference on Human Factors in Computing Systems
    April 26 - May 1, 2025
    Yokohama , Japan

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)530
    • Downloads (Last 6 weeks)69
    Reflects downloads up to 13 Feb 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2025)Sentiment Analysis of #Meanspo Tweets: Humans vs. Automatic ClassificationsProceedings of the ACM on Human-Computer Interaction10.1145/37012079:1(1-26)Online publication date: 10-Jan-2025
    • (2024)The Subtleties of Self-Presentation: A study of sensitive disclosure among sexual minority adolescentsProceedings of the ACM on Human-Computer Interaction10.1145/36374088:CSCW1(1-27)Online publication date: 26-Apr-2024
    • (2024)Images Connect Us Together: Navigating a COVID-19 Local Outbreak in China Through Social Media ImagesProceedings of the ACM on Human-Computer Interaction10.1145/36373498:CSCW1(1-32)Online publication date: 26-Apr-2024
    • (2024)"Unrest and trauma stays with you!": Navigating mental health and professional service-seeking in KashmirProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642507(1-17)Online publication date: 11-May-2024
    • (2024)"I'm gonna KMS": From Imminent Risk to Youth Joking about Suicide and Self-Harm via Social MediaProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642489(1-18)Online publication date: 11-May-2024
    • (2024)Predicting Depression with Social Media Images2024 IEEE 1st Karachi Section Humanitarian Technology Conference (KHI-HTC)10.1109/KHI-HTC60760.2024.10481967(1-8)Online publication date: 8-Jan-2024
    • (2024)The collective emotion of mentally ill individuals within Facebook groups during Covid-19 pandemicAslib Journal of Information Management10.1108/AJIM-08-2023-0320Online publication date: 28-Feb-2024
    • (2023)A Fast and Minimal System to Identify Depression Using Smartphones: Explainable Machine Learning–Based ApproachJMIR Formative Research10.2196/288487(e28848)Online publication date: 10-Aug-2023
    • (2023)Understanding Emotional Disclosure via Diary-keeping in Quarantine on Social MediaProceedings of the Eleventh International Symposium of Chinese CHI10.1145/3629606.3629623(169-181)Online publication date: 13-Nov-2023
    • (2023)“Help Me:” Examining Youth’s Private Pleas for Support and the Responses Received from Peers via Instagram Direct MessagesProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581233(1-14)Online publication date: 19-Apr-2023
    • Show More Cited By

    View Options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Login options

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media