Skip to main content
SpringerLink
Log in

How Do You Feel? Information Retrieval in Psychotherapy and Fair Ranking Assessment

  • Conference paper
  • First Online:
Advances in Bias and Fairness in Information Retrieval (BIAS 2023)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1840))

  • 240 Accesses

Abstract

The recent pandemic Coronavirus Disease 2019 (COVID-19) led to an unexpectedly imposed social isolation, causing an enormous disruption of daily routines for the global community and posing a potential risk to the mental well-being of individuals. However, resources for supporting people with mental health issues remain extremely limited, raising the matter of providing trustworthy and relevant psychotherapeutic content publicly available. To bridge this gap, this paper investigates the application of information retrieval in the mental health domain to automatically filter therapeutical content by estimated quality. We have used AnnoMI, an expert annotated counseling dataset composed of high- and low-quality Motivational Interviewing therapy sessions. First, we applied state-of-the-art information retrieval models to evaluate their applicability in the psychological domain for ranking therapy sessions by estimated quality. Then, given the sensitive psychological information associated with each therapy session, we analyzed the potential risk of unfair outcomes across therapy topics, i.e., mental issues, under a common fairness definition. Our experimental results show that the employed ranking models are reliable for systematically ranking high-quality content above low-quality one, while unfair outcomes across topics are model-dependent and associated low-quality content distribution. Our findings provide preliminary insights for applying information retrieval in the psychological domain, laying the foundations for incorporating publicly available high-quality resources to support mental health. Source code available at https://github.com/jackmedda/BIAS-FairAnnoMI.

V. Kumar and G. Medda—These authors contributed equally to this work.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 54.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 69.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Data available at https://github.com/vsrana-ai/AnnoMI.

  2. 2.

    https://github.com/NTMC-Community/MatchZoo.

References

  1. Abd-Alrazaq, A.A., Alajlani, M., Ali, N., Denecke, K., Bewick, B.M., Househ, M.: Perceptions and opinions of patients about mental health chatbots: scoping review. J. Med. Internet Res. 23(1), e17828 (2021)

    Article  Google Scholar 

  2. Balloccu, G., Boratto, L., Fenu, G., Marras, M.: Post processing recommender systems with knowledge graphs for recency, popularity, and diversity of explanations. In: Amigó, E., Castells, P., Gonzalo, J., Carterette, B., Culpepper, J.S., Kazai, G. (eds.) SIGIR ’22: The 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, Madrid, Spain, 11–15 July 2022, pp. 646–656. ACM (2022). https://doi.org/10.1145/3477495.3532041

  3. Bhandari, A., Kumar, V., Thien Huong, P.T., Thanh, D.N.: Sentiment analysis of covid-19 tweets: Leveraging stacked word embedding representation for identifying distinct classes within a sentiment. In: Artificial Intelligence in Data and Big Data Processing: Proceedings of ICABDE 2021, pp. 341–352. Springer (2022). https://doi.org/10.1007/978-3-030-97610-1_27

  4. Boratto, L., Fenu, G., Marras, M., Medda, G.: Consumer fairness in recommender systems: contextualizing definitions and mitigations. In: Hagen, M., Verberne, S., Macdonald, C., Seifert, C., Balog, K., Nørvåg, K., Setty, V. (eds.) ECIR 2022. LNCS, vol. 13185, pp. 552–566. Springer, Cham (2022). https://doi.org/10.1007/978-3-030-99736-6_37

    Chapter  Google Scholar 

  5. Boratto, L., Fenu, G., Marras, M., Medda, G.: Practical perspectives of consumer fairness in recommendation. Inf. Process. Manage. 60(2), 103208 (2023). https://doi.org/10.1016/j.ipm.2022.103208. https://www.sciencedirect.com/science/article/pii/S0306457322003090

  6. Buechel, S., Buffone, A., Slaff, B., Ungar, L., Sedoc, J.: Modeling empathy and distress in reaction to news stories. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, pp. 4758–4765 (2018)

    Google Scholar 

  7. Cabitza, F., Ciucci, D., Pasi, G., Viviani, M.: Responsible AI in healthcare. CoRR abs/2203.03616 (2022). https://doi.org/10.48550/arXiv.2203.03616

  8. Chen, R.J., et al.: Algorithm fairness in AI for medicine and healthcare. CoRR abs/2110.00603 (2021). https://arxiv.org/abs/2110.00603

  9. Currie, G., Hawk, K.E.: Ethical and legal challenges of artificial intelligence in nuclear medicine. Semin. Nucl. Med. 51(2), 120–125 (2020)

    Article  Google Scholar 

  10. Dessì, D., Helaoui, R., Kumar, V., Recupero, D.R., Riboni, D.: TF-IDF vs word embeddings for morbidity identification in clinical notes: An initial study. In: Consoli, S., ecupero, D.R., Riboni, D. (eds.) Proceedings of the First Workshop on Smart Personal Health Interfaces co-located with 25th International Conference on Intelligent User Interfaces, SmartPhil@IUI 2020, Cagliari, Italy, March 17, 2020. CEUR Workshop Proceedings, vol. 2596, pp. 1–12. CEUR-WS.org (2020), http://ceur-ws.org/Vol-2596/paper1.pdf

  11. Diao, J.A., et al.: Clinical implications of removing race from estimates of kidney function. JAMA 325(2), 184–186 (2021)

    Google Scholar 

  12. Gómez, E., Zhang, C.S., Boratto, L., Salamó, M., Marras, M.: The winner takes it all: Geographic imbalance and provider (un)fairness in educational recommender systems. In: Diaz, F., Shah, C., Suel, T., Castells, P., Jones, R., Sakai, T. (eds.) SIGIR ’21: The 44th International ACM SIGIR Conference on Research and Development in Information Retrieval, Virtual Event, Canada, 11–15 July 2021, pp. 1808–1812. ACM (2021). https://doi.org/10.1145/3404835.3463235,https://doi.org/10.1145/3404835.3463235

  13. Gómez, E., Zhang, C.S., Boratto, L., Salamó, M., Ramos, G.: Enabling cross-continent provider fairness in educational recommender systems. Future Gener. Comput. Syst. 127, 435–447 (2022). https://doi.org/10.1016/j.future.2021.08.025

  14. Guo, J., Fan, Y., Ji, X., Cheng, X.: Matchzoo: A learning, practicing, and developing system for neural text matching. In: Piwowarski, B., Chevalier, M., Gaussier, É., Maarek, Y., Nie, J., Scholer, F. (eds.) Proceedings of the 42nd International ACM SIGIR Conference on Research and Development in Information Retrieval, SIGIR 2019, Paris, France, 21–25 July 2019, pp. 1297–1300. ACM (2019). https://doi.org/10.1145/3331184.3331403

  15. Han, S., Wang, X., Bendersky, M., Najork, M.: Learning-to-rank with BERT in tf-ranking. CoRR abs/2004.08476 (2020). https://arxiv.org/abs/2004.08476

  16. Hu, B., Lu, Z., Li, H., Chen, Q.: Convolutional neural network architectures for matching natural language sentences. In: Ghahramani, Z., Welling, M., Cortes, C., Lawrence, N.D., einberger, K.Q. (eds.) Advances in Neural Information Processing Systems 27: Annual Conference on Neural Information Processing Systems 2014(December), pp. 8–13, 2014. Montreal, Quebec, Canada, pp. 2042–2050 (2014). https://proceedings.neurips.cc/paper/2014/hash/b9d487a30398d42ecff55c228ed5652b-Abstract.html

  17. Kumar, V., Mishra, B.K., Mazzara, M., Thanh, D.N., Verma, A.: Prediction of malignant and benign breast cancer: a data mining approach in healthcare applications. In: Advances in data science and management. Springer (2020)

    Google Scholar 

  18. Kumar, V., Recupero, D.R., Helaoui, R., Riboni, D.: K-lm: knowledge augmenting in language models within the scholarly domain. IEEE Access 10, 91802–91815 (2022)

    Article  Google Scholar 

  19. Kumar, V., Recupero, D.R., Riboni, D., Helaoui, R.: Ensembling classical machine learning and deep learning approaches for morbidity identification from clinical notes. IEEE Access 9, 7107–7126 (2020)

    Article  Google Scholar 

  20. Le Glaz, A., Haralambous, Y., Kim-Dufor, D.H., Lenca, P., Billot, R., Ryan, T.C., Marsh, J., Devylder, J., Walter, M., Berrouiguet, S., et al.: Machine learning and natural language processing in mental health: systematic review. J. Med. Internet Res. 23(5), e15708 (2021)

    Article  Google Scholar 

  21. Locke, S., Bashall, A., Al-Adely, S., Moore, J., Wilson, A., Kitchen, G.B.: Natural language processing in medicine: a review. Trends in Anaesthesia and Critical Care 38, 4–9 (2021)

    Article  Google Scholar 

  22. Lopez, Leo, I., Hart, Louis H., I., Katz, M.H.: Racial and ethnic health disparities related to COVID-19. JAMA 325(8), 719–720 (2021). https://doi.org/10.1001/jama.2020.26443

  23. Luo, M., Mitra, A., Gokhale, T., Baral, C.: Improving biomedical information retrieval with neural retrievers. In: Thirty-Sixth AAAI Conference on Artificial Intelligence, AAAI 2022, Thirty-Fourth Conference on Innovative Applications of Artificial Intelligence, IAAI 2022, The Twelveth Symposium on Educational Advances in Artificial Intelligence, EAAI 2022 Virtual Event, 22 February–1 March 2022, pp. 11038–11046. AAAI Press (2022). https://ojs.aaai.org/index.php/AAAI/article/view/21352

  24. Marras, M., Boratto, L., Ramos, G., Fenu, G.: Equality of learning opportunity via individual fairness in personalized recommendations. Int. J. Artif. Intell. Educ. 32(3), 636–684 (2022). https://doi.org/10.1007/s40593-021-00271-1

  25. Mhasawade, V., Zhao, Y., Chunara, R.: Machine learning and algorithmic fairness in public and population health. Nat. Mach. Intell. 3(8), 659–666 (2021). https://doi.org/10.1038/s42256-021-00373-4

  26. D Mitra, B., Diaz, F., Craswell, N.: Learning to match using local and distributed representations of text for web search. In: Barrett, R., Cummings, R., Agichtein, E., Gabrilovich, E. (eds.) Proceedings of the 26th International Conference on World Wide Web, WWW 2017, Perth, Australia, 3–7 April 2017, pp. 1291–1299. ACM (2017). https://doi.org/10.1145/3038912.3052579

  27. Morahan-Martin, J.: How internet users find, evaluate, and use online health information: A cross-cultural review. Cyberpsychology Behav. Soc. Netw. 7(5), 497–510 (2004). https://doi.org/10.1089/cpb.2004.7.497

  28. Morahan-Martin, J., Anderson, C.D.: Information and misinformation online: recommendations for facilitating accurate mental health information retrieval and evaluation. Cyberpsychology Behav. Soc. Netw. 3(5), 731–746 (2000). https://doi.org/10.1089/10949310050191737

    Article  Google Scholar 

  29. Patel, D., Msosa, Y., Wang, T., Mustafa, O.G., Gee, S., Williams, J., Roberts, A., Dobson, R.J.B., Gaughran, F.: An implementation framework and a feasibility evaluation of a clinical decision support system for diabetes management in secondary mental healthcare using cogstack. BMC Medical Informatics Decis. Mak. 22(1), 100 (2022). https://doi.org/10.1186/s12911-022-01842-5

    Article  Google Scholar 

  30. Progga, F.T., Rubya, S.: "just like therapy!": Investigating the potential of storytelling in online postpartum depression communities. In: Fiesler, C., de Carvalho, A.F.P. (eds.) The 2023 ACM International Conference on Supporting Group Work, GROUP ’23, Companion, Hilton Head, SC, USA, 8–11 January 2023, pp. 18–20. ACM (2023). https://doi.org/10.1145/3565967.3570977

  31. Raj, A., Ekstrand, M.D.: Measuring fairness in ranked results: An analytical and empirical comparison. In: Amigó, E., Castells, P., Gonzalo, J., Carterette, B., Culpepper, J.S., Kazai, G. (eds.) SIGIR ’22: The 45th International ACM SIGIR Conference on Research and Development in Information Retrieval, Madrid, Spain, 11–15 July 2022, pp. 726–736. ACM (2022). https://doi.org/10.1145/3477495.3532018,https://doi.org/10.1145/3477495.3532018

  32. Rashkin, H., Smith, E.M., Li, M., Boureau, Y.L.: Towards empathetic open-domain conversation models: a new benchmark and dataset. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (2019)

    Google Scholar 

  33. Snowden, L.R.: Bias in mental health assessment and intervention: theory and evidence. Am. J. Public Health 93(2), 239–243 (2003). https://doi.org/10.2105/AJPH.93.2.239,pMID: 12554576

  34. Talman, A., Yli-Jyrä, A., Tiedemann, J.: Sentence embeddings in NLI with iterative refinement encoders. Nat. Lang. Eng. 25(4), 467–482 (2019). https://doi.org/10.1017/S1351324919000202

    Article  Google Scholar 

  35. Wells, K., Klap, R., Koike, A., Sherbourne, C.: Ethnic disparities in unmet need for alcoholism, drug abuse, and mental health care. Am. J. Psychiatry 158(12), 2027–2032 (2001)

    Article  Google Scholar 

  36. Wu, H., Ma, C., Mitra, B., Diaz, F., Liu, X.: A multi-objective optimization framework for multi-stakeholder fairness-aware recommendation. ACM Trans. Inf. Syst. 41(2) (2022). https://doi.org/10.1145/3564285

  37. Wu, Z., Balloccu, S., Kumar, V., Helaoui, R., Reiter, E., Recupero, D.R., Riboni, D.: Anno-mi: a dataset of expert-annotated counselling dialogues. In: ICASSP 2022–2022 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 6177–6181. IEEE (2022)

    Google Scholar 

  38. Wu, Z., Helaoui, R., Kumar, V., Reforgiato Recupero, D., Riboni, D.: Towards detecting need for empathetic response in motivational interviewing. In: Companion Publication of the 2020 International Conference on Multimodal Interaction, pp. 497–502 (2020)

    Google Scholar 

  39. Xiong, C., Dai, Z., Callan, J., Liu, Z., Power, R.: End-to-end neural ad-hoc ranking with kernel pooling. In: Kando, N., Sakai, T., Joho, H., Li, H., de Vries, A.P., White, R.W. (eds.) Proceedings of the 40th International ACM SIGIR Conference on Research and Development in Information Retrieval, Shinjuku, Tokyo, Japan, 7–11 August 2017, pp. 55–64. ACM (2017). https://doi.org/10.1145/3077136.3080809

  40. Yang, Z., Lan, Q., Guo, J., Fan, Y., Zhu, X., Lan, Y., Wang, Y., Cheng, X.: A deep Top-K relevance matching model for ad-hoc retrieval. In: Zhang, S., Liu, T.-Y., Li, X., Guo, J., Li, C. (eds.) CCIR 2018. LNCS, vol. 11168, pp. 16–27. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01012-6_2

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Cagliari, Via Ospedale 72, 09124, Cagliari, Italy

    Vivek Kumar, Giacomo Medda, Diego Reforgiato Recupero, Daniele Riboni & Gianni Fenu

  2. Philips Research, High-Tech Campus, Eindhoven, Netherlands

    Rim Helaoui

Authors
  1. Vivek Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Giacomo Medda
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Diego Reforgiato Recupero
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Daniele Riboni
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Rim Helaoui
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Gianni Fenu
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Giacomo Medda .

Editor information

Editors and Affiliations

  1. University of Cagliari, Cagliari, Italy

    Ludovico Boratto

  2. Sapienza University of Rome, Rome, Italy

    Stefano Faralli

  3. University of Cagliari, Cagliari, Italy

    Mirko Marras

  4. University of L' Aquila, L’Aquila, L'Aquila, Italy

    Giovanni Stilo

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Kumar, V., Medda, G., Recupero, D.R., Riboni, D., Helaoui, R., Fenu, G. (2023). How Do You Feel? Information Retrieval in Psychotherapy and Fair Ranking Assessment. In: Boratto, L., Faralli, S., Marras, M., Stilo, G. (eds) Advances in Bias and Fairness in Information Retrieval. BIAS 2023. Communications in Computer and Information Science, vol 1840. Springer, Cham. https://doi.org/10.1007/978-3-031-37249-0_10

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-37249-0_10

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-37248-3

  • Online ISBN: 978-3-031-37249-0

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation