skip to main content
10.1145/3411763.3451799acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
poster

SAD: A Stress Annotated Dataset for Recognizing Everyday Stressors in SMS-like Conversational Systems

Published: 08 May 2021 Publication History

Abstract

There is limited infrastructure for providing stress management services to those in need. To address this problem, chatbots are viewed as a scalable solution. However, one limiting factor is having clear definitions and examples of daily stress on which to build models and methods for routing appropriate advice during conversations. We developed a dataset of 6850 SMS-like sentences that can be used to classify input using a scheme of 9 stressor categories derived from: stress management literature, live conversations from a prototype chatbot system, crowdsourcing, and targeted web scraping from an online repository. In addition to releasing this dataset, we show results that are promising for classification purposes. Our contributions include: (i) a categorization of daily stressors, (ii) a dataset of SMS-like sentences, (iii) an analysis of this dataset that demonstrates its potential efficacy, and (iv) a demonstration of its utility for implementation via a simulation of model response times.

References

[1]
David M Almeida. 2005. Resilience and Vulnerability to Daily Stressors Assessed via Diary Methods. Current Directions in Psychological Science 14, 2 (2005), 64–68. https://doi.org/10.1111/j.0963-7214.2005.00336.x
[2]
Timothy W Bickmore, Ha Trinh, Stefan Olafsson, Teresa K O’Leary, Reza Asadi, Nathaniel M Rickles, and Ricardo Cruz. 2018. Patient and consumer safety risks when using conversational assistants for medical information: an observational study of Siri, Alexa, and Google Assistant. Journal of medical Internet research 20, 9 (2018), e11510.
[3]
Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, and Dario Amodei. 2020. Language Models are Few-Shot Learners. arxiv:2005.14165 [cs.CL]
[4]
Jeffrey Dastin. 2018. ’Kill your foster parents’: Amazon’s Alexa talks murder, sex in AI experiment. Reuters (Dec 2018). https://www.reuters.com/article/us-amazon-com-alexa-insight/kill-your-foster-parents-amazons-alexa-talks-murder-sex-in-ai-experiment-idUSKCN1OK1AJ
[5]
Ryan Daws. 2020. Medical chatbot using OpenAI’s GPT-3 told a fake patient to kill themselves. AI News (Oct 2020). https://artificialintelligence-news.com/2020/10/28/medical-chatbot-openai-gpt3-patient-kill-themselves/
[6]
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. 2018. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805(2018).
[7]
Emily Dinan, Angela Fan, Adina Williams, Jack Urbanek, Douwe Kiela, and Jason Weston. 2020. Queens are powerful too: Mitigating gender bias in dialogue generation.
[8]
Elissa S Epel, Elizabeth H Blackburn, Jue Lin, Firdaus S Dhabhar, Nancy E Adler, Jason D Morrow, and Richard M Cawthon. 2004. Accelerated telomere shortening in response to life stress.Proceedings of the National Academy of Sciences of the United States of America 101, 49 (2004), 17312–17315. https://doi.org/10.1073/pnas.0407162101
[9]
Kathleen Kara Fitzpatrick, Alison Darcy, and Molly Vierhile. 2017. Delivering Cognitive Behavior Therapy to Young Adults With Symptoms of Depression and Anxiety Using a Fully Automated Conversational Agent (Woebot): A Randomized Controlled Trial. JMIR Mental Health 4, 2 (2017). https://doi.org/10.2196/mental.7785
[10]
Luciano Floridi and Massimo Chiriatti. 2020. GPT-3: Its nature, scope, limits, and consequences. Minds and Machines (2020), 1–14.
[11]
Russell Fulmer, Angela Joerin, Breanna Gentile, Lysanne Lakerink, and Michiel Rauws. 2018. Using Psychological Artificial Intelligence (Tess) to Relieve Symptoms of Depression and Anxiety: Randomized Controlled Trial. JMIR Ment Health 5, 4 (dec 2018), e64. https://doi.org/10.2196/mental.9782
[12]
Jennifer Golbeck, Matthew Mauriello, Brooke Auxier, Keval H. Bhanushali, Christopher Bonk, Mohamed Amine Bouzaghrane, Cody Buntain, Riya Chanduka, Paul Cheakalos, Jennine B. Everett, Waleed Falak, Carl Gieringer, Jack Graney, Kelly M. Hoffman, Lindsay Huth, Zhenya Ma, Mayanka Jha, Misbah Khan, Varsha Kori, Elo Lewis, George Mirano, William T. Mohn IV, Sean Mussenden, Tammie M. Nelson, Sean Mcwillie, Akshat Pant, Priya Shetye, Rusha Shrestha, Alexandra Steinheimer, Aditya Subramanian, and Gina Visnansky. 2018. Fake News vs Satire: A Dataset and Analysis. In Proceedings of the 10th ACM Conference on Web Science (Amsterdam, Netherlands) (WebSci ’18). Association for Computing Machinery, New York, NY, USA, 17–21. https://doi.org/10.1145/3201064.3201100
[13]
Kotaro Hara, Vicki Le, and Jon Froehlich. 2013. Combining Crowdsourcing and Google Street View to Identify Street-Level Accessibility Problems. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Paris, France) (CHI ’13). Association for Computing Machinery, New York, NY, USA, 631–640. https://doi.org/10.1145/2470654.2470744
[14]
Becky Inkster, Shubhankar Sarda, and Vinod Subramanian. 2018. An empathy-driven, conversational artificial intelligence agent (Wysa) for digital mental well-being: real-world data evaluation mixed-methods study. JMIR mHealth and uHealth 6, 11 (2018), e12106.
[15]
Gierad Laput, Walter S. Lasecki, Jason Wiese, Robert Xiao, Jeffrey P. Bigham, and Chris Harrison. 2015. Zensors: Adaptive, Rapidly Deployable, Human-Intelligent Sensor Feeds. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems(Seoul, Republic of Korea) (CHI ’15). Association for Computing Machinery, New York, NY, USA, 1935–1944. https://doi.org/10.1145/2702123.2702416
[16]
Haochen Liu, Jamell Dacon, Wenqi Fan, Hui Liu, Zitao Liu, and Jiliang Tang. 2020. Does Gender Matter? Towards Fairness in Dialogue Systems. In Proceedings of the 28th International Conference on Computational Linguistics. 4403–4416.
[17]
B S McEwen. 1998. Stress, adaptation, and disease. Allostasis and allostatic load.Annals of the New York Academy of Sciences 840 (1998), 33–44. https://doi.org/10.1111/j.1749-6632.1998.tb09546.x
[18]
Adam S Miner, Arnold Milstein, Stephen Schueller, Roshini Hegde, Christina Mangurian, and Eleni Linos. 2016. Smartphone-Based Conversational Agents and Responses to Questions About Mental Health, Interpersonal Violence, and Physical Health. JAMA Internal Medicine 176, 5 (2016), 619. https://doi.org/10.1001/jamainternmed.2016.0400 arxiv:15334406
[19]
Peter A Noone. 2017. The Holmes–Rahe Stress Inventory. Occupational Medicine 67, 7 (2017), 581–582.
[20]
Pablo Paredes, Ana Rufino Ferreira, Cory Schillaci, Gene Yoo, Pierre Karashchuk, Dennis Xing, Coye Cheshire, and John Canny. 2017. Inquire: Large-Scale Early Insight Discovery for Qualitative Research. In Proceedings of the 2017 ACM Conference on Computer Supported Cooperative Work and Social Computing(Portland, Oregon, USA) (CSCW ’17). Association for Computing Machinery, New York, NY, USA, 1562–1575. https://doi.org/10.1145/2998181.2998363
[21]
Pablo E Paredes, Nantanick Tantivasadakarn, Grace Hon, Emmanuel Thierry Lincoln, Nikhil Gowda, Marco A Mora-Mendoza, and Matthew Louis Mauriello. 2020. Towards PopBots: A Suite of Conversational Agents for Daily Stress. CH 2020 Workshop on Conversational Agents for Health and Wellbeing (2020). https://www.researchgate.net/publication/344771066_Towards_PopBots_A_Suite_of_Conversational_Agents_for_Daily_Stress/
[22]
F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. 2011. Scikit-learn: Machine Learning in Python. Journal of Machine Learning Research 12 (2011), 2825–2830.
[23]
Jeffrey Pennington, Richard Socher, and Christopher Manning. 2014. GloVe: Global Vectors for Word Representation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP). Association for Computational Linguistics, Doha, Qatar, 1532–1543. https://doi.org/10.3115/v1/D14-1162
[24]
Jennifer R Piazza, Susan T Charles, Martin J Sliwinski, Jacqueline Mogle, and David M Almeida. 2013. Affective reactivity to daily stressors and long-term risk of reporting a chronic physical health condition. Annals of behavioral medicine : a publication of the Society of Behavioral Medicine (feb 2013). https://doi.org/10.1007/s12160-012-9423-0
[25]
Hannah Rashkin, Eric Michael Smith, Margaret Li, and Y-Lan Boureau. 2019. Towards Empathetic Open-domain Conversation Models: A New Benchmark and Dataset. arXiv preprint arXiv:1811.00207(2019), 5370–5381. https://www.aclweb.org/anthology/P19-1534
[26]
Nils Reimers and Iryna Gurevych. 2019. Sentence-BERT: Sentence Embeddings using Siamese BERT-Networks. In Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing. Association for Computational Linguistics. https://arxiv.org/abs/1908.10084
[27]
Han-Chin Shing, Suraj Nair, Ayah Zirikly, Meir Friedenberg, Hal Daumé III, and Philip Resnik. 2018. Expert, crowdsourced, and machine assessment of suicide risk via online postings. In Proceedings of the Fifth Workshop on Computational Linguistics and Clinical Psychology: From Keyboard to Clinic. 25–36.
[28]
Elsbeth Turcan and Kathleen McKeown. 2019. Dreaddit: A Reddit dataset for stress analysis in social media. arXiv preprint arXiv:1911.00133(2019).
[29]
Joseph Weizenbaum. 1966. ELIZA—a Computer Program for the Study of Natural Language Communication between Man and Machine. Commun. ACM 9, 1 (jan 1966), 36–45. https://doi.org/10.1145/365153.365168
[30]
Ze Yang, Linjun Shou, Ming Gong, Wutao Lin, and Daxin Jiang. 2020. Model Compression with Two-Stage Multi-Teacher Knowledge Distillation for Web Question Answering System. In Proceedings of the 13th International Conference on Web Search and Data Mining (Houston, TX, USA) (WSDM ’20). Association for Computing Machinery, New York, NY, USA, 690–698. https://doi.org/10.1145/3336191.3371792
[31]
Ayah Zirikly, Philip Resnik, Özlem Uzuner, and Kristy Hollingshead. 2019. CLPsych 2019 Shared Task: Predicting the Degree of Suicide Risk in Reddit Posts. In Proceedings of the Sixth Workshop on Computational Linguistics and Clinical Psychology(Minneapolis).

Cited By

View all
  • (2025)Depression Detection From Social Media Posts Using Emotion Aware Encoders and Fuzzy Based Contrastive NetworksIEEE Transactions on Fuzzy Systems10.1109/TFUZZ.2024.346177633:1(43-53)Online publication date: Jan-2025
  • (2024)Intent aware data augmentation by leveraging generative AI for stress detection in social media textsPeerJ Computer Science10.7717/peerj-cs.215610(e2156)Online publication date: 8-Jul-2024
  • (2024)Large Language Models in Psychiatry: Current Applications, Limitations, and Future ScopeBig Data Mining and Analytics10.26599/BDMA.2024.90200467:4(1148-1168)Online publication date: Dec-2024
  • Show More Cited By

Recommendations

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences
CHI EA '21: Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems
May 2021
2965 pages
ISBN:9781450380959
DOI:10.1145/3411763
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 08 May 2021

Permissions

Request permissions for this article.

Check for updates

Author Tags

  1. Classification
  2. Conversational Agents
  3. Daily Stress
  4. Datasets
  5. Stress Management
  6. Stressors

Qualifiers

  • Poster
  • Research
  • Refereed limited

Conference

CHI '21
Sponsor:

Acceptance Rates

Overall Acceptance Rate 6,164 of 23,696 submissions, 26%

Upcoming Conference

CHI 2025
ACM CHI Conference on Human Factors in Computing Systems
April 26 - May 1, 2025
Yokohama , Japan

Contributors

Other Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

  • Downloads (Last 12 months)237
  • Downloads (Last 6 weeks)22
Reflects downloads up to 20 Jan 2025

Other Metrics

Citations

Cited By

View all
  • (2025)Depression Detection From Social Media Posts Using Emotion Aware Encoders and Fuzzy Based Contrastive NetworksIEEE Transactions on Fuzzy Systems10.1109/TFUZZ.2024.346177633:1(43-53)Online publication date: Jan-2025
  • (2024)Intent aware data augmentation by leveraging generative AI for stress detection in social media textsPeerJ Computer Science10.7717/peerj-cs.215610(e2156)Online publication date: 8-Jul-2024
  • (2024)Large Language Models in Psychiatry: Current Applications, Limitations, and Future ScopeBig Data Mining and Analytics10.26599/BDMA.2024.90200467:4(1148-1168)Online publication date: Dec-2024
  • (2024)Mental-LLMProceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies10.1145/36435408:1(1-32)Online publication date: 6-Mar-2024
  • (2024)Detection and Analysis of Stress-Related Posts in Reddit’s Acamedic CommunitiesIEEE Access10.1109/ACCESS.2024.335766212(14932-14948)Online publication date: 2024
  • (2024)Enhancing pre-trained contextual embeddings with triplet loss as an effective fine-tuning method for extracting clinical features from electronic health record derived mental health clinical notesNatural Language Processing Journal10.1016/j.nlp.2023.1000456(100045)Online publication date: Mar-2024
  • (2024)CrisisTransformersKnowledge-Based Systems10.1016/j.knosys.2024.111916296:COnline publication date: 19-Jul-2024
  • (2024)LLMental: Classification of Mental Disorders with Large Language ModelsFoundations of Intelligent Systems10.1007/978-3-031-62700-2_4(35-44)Online publication date: 17-Jun-2024
  • (2023)Just Do Something: Comparing Self-proposed and Machine-recommended Stress Interventions among Online Workers with Home Sweet OfficeProceedings of the 2023 CHI Conference on Human Factors in Computing Systems10.1145/3544548.3581319(1-20)Online publication date: 19-Apr-2023
  • (2023)Data Augmentation Impact on Deep learning Performance for Stress Detection2023 Eleventh International Conference on Intelligent Computing and Information Systems (ICICIS)10.1109/ICICIS58388.2023.10391179(160-165)Online publication date: 21-Nov-2023

View Options

Login options

View options

PDF

View or Download as a PDF file.

PDF

eReader

View online with eReader.

eReader

HTML Format

View this article in HTML Format.

HTML Format

Media

Figures

Other

Tables

Share

Share

Share this Publication link

Share on social media