Skip to main content
SpringerLink
Log in

Experiencing Ethics and Values in the Design Process of AI-Enabled Medical Devices and Software

  • Conference paper
  • First Online:
HCI International 2023 – Late Breaking Papers (HCII 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14059))

Included in the following conference series:

  • 780 Accesses

Abstract

The interaction design of medical devices has to engage and cater to the needs of a wide range of stakeholders, from patients to physicians. The present paper aims to show by example the methodological integration of critical reflection on ethics and values during the design process of AI-enabled tools. The question of how to promote awareness of ethical responsibilities in the context of AI in medical devices is raised, and the use of core frameworks for ethical and value-based design as well as reflexive practice are being evaluated. To this end we conducted multiple empirical studies based on multiple university research projects, including joint research with healthcare industry partners, connecting the fields of medicine, engineering psychology, media informatics, and design research. We present a questionnaire-based self-reflection tool—VaPeMeT—designed to encourage ethical reflection and use of frameworks across project phases and team members. Our studies combine qualitative and quantitative research in order to 1) connect reported participant actions with theoretical considerations from value-based design and 2) evaluate the completeness and usefulness of the VaPeMeT questionnaire. By conceptualizing process design implications and suggesting a questionnaire-based self-reflection tool, our research aims to contribute to integrated, actionable and responsible transdisciplinary cooperation in this sensitive context.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Apple, J.M., Guerci, J.C., Seligson, N.D., Curtis, S.D.: Adding the second T: elevating STAR to START for behavioral interviewing. Am. J. Health Syst. Pharm. 78(1), 18–21 (2021)

    Article  Google Scholar 

  2. Bencherki, N.: Actor-network theory, 1st edn. In: Scott, C.R., Barker, J.R., Kuhn, T., Keyton, J., Turner, P.K., Lewis, L.K. (eds.) The International Encyclopedia of Organizational Communication, 8 March 2017, vol. 1 (4 vols.), pp. 1–13. Wiley (2017). https://doi.org/10.1002/9781118955567.wbieoc002

  3. Birks, M., Mills, J.: Grounded Theory: A Practical Guide. SAGE Publications (2015)

    Google Scholar 

  4. Callon, M.: Actor network theory. In: International Encyclopedia of the Social & Behavioral Sciences, pp. 62–66. Elsevier (2001). https://doi.org/10.1016/B0-08-043076-7/03168-5

  5. Creswell, J.W., Plano Clark, V.L.: Choosing a mixed methods research design, 2nd edn. In: Designing and Conducting Mixed Methods Research, pp. 53–106. SAGE Publications, Los Angeles (2011)

    Google Scholar 

  6. Cruz-Martínez, R.R., Wentzel, J., Bente, B.E., Sanderman, R., Van Gemert-Pijnen, J.E.: Toward the value sensitive design of ehealth technologies to support self-management of cardiovascular diseases: content analysis. JMIR Cardio. 5(2), e31985 (2021). https://doi.org/10.2196/31985

    Article  Google Scholar 

  7. Daku, M.: Ethics beyond ethics: the need for virtuous researchers. BMC Med. Ethics 19(1), 42 (2018). https://doi.org/10.1186/s12910-018-0281-6

    Article  Google Scholar 

  8. Dalsgaard, P., Halskov, K.: Reflective design documentation. In: Proceedings of the Designing Interactive Systems Conference, DIS 2012, pp. 428–437. Association for Computing Machinery, New York (2012). https://doi.org/10.1145/2317956.2318020

  9. Freeman, R.E.: Strategic Management: A Stakeholder Approach. Cambridge University Press, 2010 (1984)

    Google Scholar 

  10. Friedman, B., Hendry, D.G., Borning, A.: A survey of value sensitive design methods. Found. Trends® Hum. Comput. Interact. 11(2), 63–125 (2017). https://doi.org/10.1561/1100000015

  11. Friedman, B., Kahn, P.H., Borning, A., Huldtgren, A.: Value sensitive design and information systems. In: Doorn, N., Schuurbiers, D., van de Poel, I., Gorman, M.E. (eds.) Early Engagement and New Technologies: Opening Up the Laboratory. PET, vol. 16, pp. 55–95. Springer, Dordrecht (2013). https://doi.org/10.1007/978-94-007-7844-3_4

    Chapter  Google Scholar 

  12. Gunning, D., Stefik, M., Choi, J., Miller, T., Stumpf, S., Yang, G.-Z.: XAI-explainable artificial intelligence. Sci. Robot. 4(37), eaay7120 (2019). https://doi.org/10.1126/scirobotics.aay7120

  13. Hankins, J.: Grass-roots case studies in ‘poiesis-intensive’ responsible innovation (PIRI). In: International Handbook on Responsible Innovation, pp. 393–404. Edward Elgar Publishing (2019). https://doi.org/10.4337/9781784718862.00036

  14. Jonas, W.: Systems design thinking: theoretical, methodological, and methodical considerations. A German narrative. In: Jones, P., Kijima, K. (eds.) Systemic Design. TSS, vol. 8, pp. 89–117. Springer, Tokyo (2018). https://doi.org/10.1007/978-4-431-55639-8_4

    Chapter  Google Scholar 

  15. Jongsma, K.R., Jongepier, F.: Value-sensitive design and global digital health. Bull. World Health Organ. 98(8), 579–580 (2020). https://doi.org/10.2471/BLT.19.237362

    Article  Google Scholar 

  16. Lee, J.: Design rationale systems: understanding the issues. IEEE Exp. 12, 78–85 (1997)

    Article  Google Scholar 

  17. Luhmann, N.: Social Systems. Stanford University Press, 1995 (1984)

    Google Scholar 

  18. Olivier, E., Urs, G., Marc, Z.S.: Stakeholder als sozialwissenschaftliches Konzept: Begrifflichkeit und Operationalisierung (2006). https://doi.org/10.5167/UZH-76814

  19. Papoutsi, C., Wherton, J., Shaw, S., Morrison, C., Greenhalgh, T.: Putting the social back into sociotechnical: case studies of co-design in digital health. J. Am. Med. Inf. Assoc. 28, 284–293 (2020). https://doi.org/10.1093/jamia/ocaa197

    Article  Google Scholar 

  20. Parameswaran, U.D., Ozawa-Kirk, J.L., Latendresse, G.: To live (code) or to not: a new method for coding in qualitative research. Qual. Soc. Work. 19(4), 630–644 (2020). https://doi.org/10.1177/1473325019840394

    Article  Google Scholar 

  21. Quintal, A., Messier, V., Rabasa-Lhoret, R., Racine, E.: A critical review and analysis of ethical issues associated with the artificial pancreas. Diab. Metab. 45(1), 1–10 (2019). https://doi.org/10.1016/j.diabet.2018.04.003

    Article  Google Scholar 

  22. Reijers, W., et al.: Methods for practising ethics in research and innovation: a literature review, critical analysis and recommendations. Sci. Eng. Ethics 24(5), 1437–1481 (2017). https://doi.org/10.1007/s11948-017-9961-8

    Article  Google Scholar 

  23. Saldaña, J.: The Coding Manual for Qualitative Researchers, 3rd edn., 339 pp. SAGE Publications (2016)

    Google Scholar 

  24. Shneiderman, B.: Human-centered artificial intelligence: reliable, safe & trustworthy. Int. J. Hum. Comput. Interact. 36(6), 495–504 (2020). https://doi.org/10.1080/10447318.2020.1741118

    Article  Google Scholar 

  25. Simonsen, J., Robertson, T. (eds.): Routledge International Handbook of Participatory Design. Routledge, 12 October 2012. https://doi.org/10.4324/9780203108543

  26. Timmermans, J., Zhao, Y., Van Den Hoven, J.: Ethics and nanopharmacy: value sensitive design of new drugs. NanoEthics 5(3), 269–283 (2011). https://doi.org/10.1007/s11569-011-0135-x

    Article  Google Scholar 

  27. Timmermans, S., Tavory, I.: Theory construction in qualitative research: from grounded theory to abductive analysis. Sociol. Theor. 30(3), 167–186 (2012). https://doi.org/10.1177/0735275112457914

    Article  Google Scholar 

  28. Valentinov, V., Roth, S., Will, M.G.: Stakeholder theory: a Luhmannian perspective. Adm. Soc. 51(5), 826–849 (2019). https://doi.org/10.1177/0095399718789076

    Article  Google Scholar 

  29. Van Andel, J., Leijten, F., Van Delden, H., Van Thiel, G.: What makes a good home-based nocturnal seizure detector? A value sensitive design. PLoS ONE 10(4), e0121446 (2015). https://doi.org/10.1371/journal.pone.0121446

  30. von Schomberg, R., Hankins, J.: International Handbook on Responsible Innovation: A Global Resource. Edward Elgar Publishing (2019)

    Google Scholar 

  31. Weber, K.: Extended model for ethical evaluation. In: Karafillidis, A., Weidner, R. (eds.) Developing Support Technologies. BB, vol. 23, pp. 257–263. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01836-8_25

    Chapter  Google Scholar 

  32. Winkler, T., Spiekermann, S.: Twenty years of value sensitive design: a review of methodological practices in VSD projects. Ethics Inf. Technol. 23(1), 17–21 (2018). https://doi.org/10.1007/s10676-018-9476-2

    Article  Google Scholar 

Download references

Acknowledgment

This research has been funded in part by the Federal Ministry of Education and Research of Germany in the framework of the project CoCoAI (Cooperative and Communicating AI, project number 01GP1908).

Author information

Authors and Affiliations

  1. University of Vechta, Driverstraße 22, 49377, Vechta, Germany

    Benjamin Schwarz

  2. Institute for Multimedia and Interactive Systems, University of Lübeck, Ratzeburger Allee 160, 23562, Lübeck, Germany

    Benjamin Schwarz, Tim Schrills & Thomas Franke

Authors
  1. Benjamin Schwarz
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tim Schrills
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Thomas Franke
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Benjamin Schwarz .

Editor information

Editors and Affiliations

  1. Siemens Corporation, Princeton, NJ, USA

    Helmut Degen

  2. Foundation for Research and Technology – Hellas (FORTH), Heraklion, Crete, Greece

    Stavroula Ntoa

  3. San Jose State University, San Jose, CA, USA

    Abbas Moallem

A Study Materials and Detailed Results

A Study Materials and Detailed Results

Table 2. Interview Items (Study 1)
Full size table
Table 3. Meta Items Surrounding VaPeMeT Questionnaire in Study 3
Full size table
Table 4. Descriptive Data of VaPeMeT Items Regarding Components Values, Processes and Documentation (Study 2, N = 22)
Full size table

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Schwarz, B., Schrills, T., Franke, T. (2023). Experiencing Ethics and Values in the Design Process of AI-Enabled Medical Devices and Software. In: Degen, H., Ntoa, S., Moallem, A. (eds) HCI International 2023 – Late Breaking Papers. HCII 2023. Lecture Notes in Computer Science, vol 14059. Springer, Cham. https://doi.org/10.1007/978-3-031-48057-7_15

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-48057-7_15

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-48056-0

  • Online ISBN: 978-3-031-48057-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation