research-article

Effects-driven IT development: an instrument for supporting sustained participatory design

Authors:

Morten Hertzum,

Jesper SimonsenAuthors Info & Claims

PDC '10: Proceedings of the 11th Biennial Participatory Design Conference

Pages 61 - 70

https://doi.org/10.1145/1900441.1900451

Published: 29 November 2010 Publication History

Abstract

We present effects-driven IT development as an instrument for pursuing and reinforcing Participatory Design (PD) when it is applied in commercial information technology (IT) projects. Effects-driven IT development supports the management of a sustained PD process throughout design and organizational implementation. The focus is on the effects to be achieved by users through their adoption and use of a system. The overall idea is to (a) specify the purpose of a system as effects that are both measurable and meaningful to the users, and (b) evaluate the absence or presence of these effects during real use of the system. Effects are formulated in a user-oriented terminology, and they can be evaluated and revised with users in an iterative and incremental systems-development process that involves pilot implementations. In this paper we investigate the design, pilot implementation, and effects assessment of an electronic patient record. Effects concerning, among other things, clinicians' mental workload were specified and measured, but apart from the planned changes associated with these effects the pilot implementation also gave rise to emergent, opportunity-based, and curtailed changes. We discuss our experiences regarding conditions for making the specification of effects and their real-use evaluation central activities in IT projects.

References

[1]

Berg, M., Aarts, J., and Lei, J. v. d. ICT in health care: Sociotechnical approaches. Methods of Information in Medicine, 2003, 42(4): 297--301.

[2]

Bødker, K., Kensing, F., and Simonsen, J. Participatory IT design: Designing for business and workplace realities. MIT Press, Cambridge, MA. 2004.

Digital Library

[3]

Connell, K., Andal, D., and Brown, C. L. Performance based procurement. Another model for California. California Franchise Tax Board, Sacramento, CA. 1995.

[4]

Davis, F. D. Perceived usefulness, perceived ease of use, and user acceptance of information technology. MIS Quarterly, 1989, 13(3): 319--340.

Digital Library

[5]

Dumas, J. S., and Redish, J. C. A practical guide to usability testing. Revised edition. Intellect Books, Exeter, UK. 1999.

Digital Library

[6]

Fichman, R. G., and Moses, S. A. An incremental process for software implementation. Sloan Management Review, 1999, 40(2): 39--52.

[7]

Gagnon, M., Hébert, R., Dubé, M., and Dubois, M.-F. Development and validation of an instrument measuring individual empowerment in relation to personal health care: The Health Care Empowerment Questionnaire (HCEQ). American Journal of Health Promotion, 2006, 20(6): 429--435.

[8]

Glass, R. L. Pilot studies: What, why, and how. Journal of Systems and Software, 1997, 36(1): 85--97.

Digital Library

[9]

Good, M., Spine, T. M., Whiteside, J., and George, P. User-derived impact analysis as a tool for usability engineering. In Proc. CHI'86, pp. 241--246. ACM Press, New York, 1986.

Digital Library

[10]

Granlien, M. F., Hertzum, M., and Gudmundsen, J. The gap between actual and mandated use of an electronic medication record three years after deployment. In S. K. Andersen, G. O. Klein, S. Schulz, J. Arts, and M. C. Mazzoleni (eds), Proc. MIE'2008, pp. 419--424. IOS Press, Amsterdam, 2008.

[11]

Granlien, M. S., and Hertzum, M. Implementing new ways of working: Interventions and their effect on the use of an electronic medication record. In Proc. GROUP'2009, pp. 321--330. ACM Press, New York, 2009.

Digital Library

[12]

Hamilton, S., and Chervany, N. L. Evaluating information system effectiveness - Part I: Comparing evaluation approaches. MIS Quarterly, 1981, 5(3): 55--69.

Digital Library

[13]

Harlen, W., and James, M. Assessment and learning: Differences and relationships between formative and summative assessment. Assessment in Education: Principles, Policy & Practice, 1997, 4(3): 365--379.

[14]

Hart, S. G., and Staveland, L. E. Development of NASA-TLX (task load index): Results of empirical and theoretical research. In P. A. Hancock and N. Meshkati (eds), Human Mental Workload, pp. 139--183. North-Holland, Amsterdam, 1988.

[15]

Hartswood, M., Procter, R., Slack, R., Voss, A., Büscher, M., Rouncefield, M., and Rouchy, P. Co-realization: Toward a principled synthesis of ethnomethodology and participatory design. In M. S. Ackerman, C. A. Halverson, T. D. Erickson, and W. A. Kellogg (eds), Resources, Co-Evolution and artifacts, pp. 59--94. Springer, London, 2008.

[16]

Hertzum, M., and Simonsen, J. Positive effects of electronic patient records on three clinical activities. International Journal of Medical Informatics, 2008, 77(12): 809--817.

[17]

ISO 9241 Ergonomic requirements for office work with visual display terminals (VDTs) - Part 11: Guidance on usability. International Standard Organization, Geneva, CH. 1998.

[18]

Leveson, N. G. Intent specifications: An approach to building human-centered specifications. IEEE Transactions on Software Engineering, 2000, 26(1): 15--35.

Digital Library

[19]

Markus, M. L. Technochange management: Using IT to drive organizational change. Journal of Information Technology, 2004, 19(1): 4--20.

[20]

Maulsby, D., Greenberg, S., and Mander, R. Prototyping an intelligent agent through Wizard of Oz. In Proc. INTERCHI '93, pp. 277--284. ACM Press, New York, 1993.

Digital Library

[21]

Mechling, J. Better funding for government IT: Views from the front line. Journal of the American Society for Information Science, 1999, 50(4): 305--313.

Digital Library

[22]

Orlikowski, W. J., and Hofman, J. D. An improvisational model for change management: The case of groupware technologies. Sloan Management Review, 1997, 38(2): 11--22.

[23]

Ottersten, I., and Balic, M. Effect managing IT. Liber, Køge. 2007.

Digital Library

[24]

Ramsay, C. R., Grant, A. M., Wallace, S. A., Garthwaite, P. H., Monk, A. F., and Russell, I. T. Assessment of the learning curve in health technologies: A systematic review. International Journal of Technology Assessment in Health Care, 2000, 16(4): 1095--1108.

[25]

Robinson, M. Design for unanticipated use... In G. De Michelis, C. Simone, and K. Schmidt (eds), Proc. ECSCW'93, pp. 187--202. Springer, Berlin, 1993.

Digital Library

[26]

Rosson, M. B., and Carroll, J. M. Usability engineering: Scenario-based development of human-computer interaction. Morgan Kaufmann, San Francisco, CA. 2002.

Digital Library

[27]

Rzevski, G. Prototypes versus pilot systems: Strategies for evolutionary information system development. In R. Budde, K. Kuhlenkamp, L. Mathiassen, and L. Zullighoven (eds), Approaches to Prototyping: Proceedings on the Working Conference on Prototyping, pp. 356--367. Springer, Heidelberg, 1984.

[28]

Sarti, C., Rastenyte, D., Cepaitis, Z., and Tuomilehto, J. International trends in mortality from stroke, 1968 to 1994. Stroke, 2000, 31(7): 1588--1601.

[29]

Shapiro, D. Participatory design: The will to succeed. In Proc. AARHUS'2005 Conference on Critical Computing, pp. 29--38. ACM Press, New York, 2005.

Digital Library

[30]

Simonsen, J. The role of ethnography in the design and implementation of IT systems. Design Principles and Practices, an International Journal, 2009, 3(3): 251--264.

[31]

Simonsen, J., and Hertzum, M. Participative design and the challenges of large-scale systems: Extending the iterative PD approach. In J. Simonsen, T. Robertson, and D. Hakken (eds), Proc. PDC'2008, pp. 1--10. ACM Press, New York, 2008.

Digital Library

[32]

Simonsen, J., and Hertzum, M. Iterative participatory design. In J. Simonsen, J. O. Bærenholdt, M. Büscher, and J. D. Scheuer (eds), Design Research: Synergies from Interdisciplinary Perspectives, pp. 16--32. Routledge, London, 2010.

[33]

Sommerville, I. Software engineering. Seventh edition. Addison Wesley, San Francisco, CA. 2004.

Digital Library

[34]

Steinberg, D. H., and Palmer, D. W. Extreme software engineering. Pearson, Upper Saddle River, NJ. 2004.

Digital Library

[35]

Suchman, L. A. Human-machine reconfigurations: Plans and situated action, 2nd edition. Cambridge University Press, Cambridge, UK. 2007.

Digital Library

[36]

Turner, J. R. The role of pilot studies in reducing risk on projects and programmes. International Journal of Project Management, 2005, 23(1): 1--6.

[37]

Tyre, M. J., and Orlikowski, W. J. Windows of opportunity: Temporal patterns of technological adaptation in organizations. Organization Science, 1994, 5(1): 98--118.

Digital Library

[38]

Vicente, K. J. Cognitive work analysis: Toward safe, productive, and healthy computer-based work. Erlbaum, Mahwah, NJ. 1999.

Digital Library

[39]

Ward, J., and Daniel, E. Benefits management: Delivering value from IS & IT investments. Wiley, Chichester, UK. 2006.

[40]

Whiteside, J., Bennett, J., and Holtzblatt, K. Usability engineering: Our experience and evolution. In M. Helander (ed), Handbook of Human-Computer Interaction, pp. 791--817. Elsevier, Amsterdam, 1988.

[41]

Whyte, W. F. (ed). Participatory action research. Sage, Newbury Park, CA. 1991.

Cited By

Yang SDortheimer JSprecher AYang Q(2024)When design workshops meet chatbots: Meaningful participation at scale?International Journal of Architectural Computing10.1177/14780771241253440Online publication date: 30-May-2024
https://doi.org/10.1177/14780771241253440
Nguyen QJaspaert EHarthum B(2024)Talk the Talk, Walk the Walk: A Case Study on Systematic Evaluation in Participatory DesignProceedings of the Participatory Design Conference 2024: Full Papers - Volume 110.1145/3666094.3666103(133-144)Online publication date: 11-Aug-2024
https://dl.acm.org/doi/10.1145/3666094.3666103
Nguyen QWindisch AKriglstein S(2024)Between Two Worlds: Analysing the Effects of Immersive and Non-Immersive Prototyping for Participatory DesignProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3660715(2198-2212)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3660715
Show More Cited By

Effects-driven IT development: an instrument for supporting sustained participatory design
1. Applied computing
  1. Life and medical sciences
  2. Operations research

Recommendations

Real-use evaluation of effects: emergency departments aiming for 'Warm Hands'
PDC '12: Proceedings of the 12th Participatory Design Conference: Exploratory Papers, Workshop Descriptions, Industry Cases - Volume 2

Embracing real use in an iterative approach calls for systematic formative evaluation. Effects-driven IT Development has been suggested as a way of supporting a Participatory Design (PD) process involving implementations that expose mature prototypes to ...
On the development of electronic design tools and associated guidelines for supporting the early stages of the design process
DESIRE '11: Procedings of the Second Conference on Creativity and Innovation in Design

According to the user-centred design (UCD) philosophy, users should be involved as much as possible in the different stages of the design process, to optimize the likelihood that design proposals will be appreciated by users. In this paper, first we ...
The innovation-driven hackathon: one means for accelerating innovation
EuroPlop '16: Proceedings of the 21st European Conference on Pattern Languages of Programs

Established companies often have difficulties developing new innovative products in a fast and creative way because of their fix structure and organizational complexity. With decision processes divided across several departments it takes too long to get ...

Comments

Information & Contributors

Information

Published In

cover image ACM Other conferences

PDC '10: Proceedings of the 11th Biennial Participatory Design Conference

November 2010

314 pages

ISBN:9781450301312

DOI:10.1145/1900441

Conference Chair:
Toni Robertson
Univ. of Technology Sydney, Australia
,
Program Chairs:
Keld Bødker
Roskilde University, Denmark
,
Tone Bratteteig
University of Oslo, Norway
,
Daria Loi
Intel Corporation

Copyright © 2010 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than ACM must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected]

Sponsors

DE: Digital Eskimo
UTS-HCTDRS: The UTS Human Centred Technology Design Research Strength
University of Technology Sydney
Roskilde University
SIGCHI-Australia: ACM SIGCHI Australia
Zumio: Zumio

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 29 November 2010

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

PDC '10

Sponsor:

DE
UTS-HCTDRS
SIGCHI-Australia
Zumio

PDC '10: The 11th Biennial Participatory Design Conference

November 29 - December 3, 2010

Sydney, Australia

Acceptance Rates

Overall Acceptance Rate 49 of 289 submissions, 17%

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

18
Total Citations
View Citations
451
Total Downloads

Downloads (Last 12 months)9
Downloads (Last 6 weeks)0

Reflects downloads up to 05 Mar 2025

Other Metrics

View Author Metrics

Citations

Cited By

Yang SDortheimer JSprecher AYang Q(2024)When design workshops meet chatbots: Meaningful participation at scale?International Journal of Architectural Computing10.1177/14780771241253440Online publication date: 30-May-2024
https://doi.org/10.1177/14780771241253440
Nguyen QJaspaert EHarthum B(2024)Talk the Talk, Walk the Walk: A Case Study on Systematic Evaluation in Participatory DesignProceedings of the Participatory Design Conference 2024: Full Papers - Volume 110.1145/3666094.3666103(133-144)Online publication date: 11-Aug-2024
https://dl.acm.org/doi/10.1145/3666094.3666103
Nguyen QWindisch AKriglstein S(2024)Between Two Worlds: Analysing the Effects of Immersive and Non-Immersive Prototyping for Participatory DesignProceedings of the 2024 ACM Designing Interactive Systems Conference10.1145/3643834.3660715(2198-2212)Online publication date: 1-Jul-2024
https://dl.acm.org/doi/10.1145/3643834.3660715
Nguyen Q(2022)Evaluation in Participatory Design – The Whys and the NotsProceedings of the Participatory Design Conference 2022 - Volume 210.1145/3537797.3537828(161-166)Online publication date: 19-Aug-2022
https://dl.acm.org/doi/10.1145/3537797.3537828
Clemmensen T(2022)Dropping a Bomb or Providing a Gentle Loving Touch? Towards a Relation Artefact Theory of Pilot ImplementationSense, Feel, Design10.1007/978-3-030-98388-8_39(429-439)Online publication date: 20-Mar-2022
https://doi.org/10.1007/978-3-030-98388-8_39
Herbæk LHansen CClemmensen T(2022)Pilot Implementation: Organizational Alignment When Implementing an IT-SystemSense, Feel, Design10.1007/978-3-030-98388-8_34(391-396)Online publication date: 20-Mar-2022
https://doi.org/10.1007/978-3-030-98388-8_34
Bødker SMina CIversen OSmith R(2021)Participatory DesignSynthesis Lectures on Human-Centered Informatics10.2200/S01136ED1V01Y202110HCI05214:5Online publication date: 9-Nov-2021
https://doi.org/10.2200/S01136ED1V01Y202110HCI052
Simonsen JZahoor Malik AFrom GFilippson Parslov MTue Sørensen L(2020)A Checklist For A Successful PD Student ProjectProceedings of the 16th Participatory Design Conference 2020 - Participation(s) Otherwise - Volume 210.1145/3384772.3385132(119-123)Online publication date: 15-Jun-2020
https://dl.acm.org/doi/10.1145/3384772.3385132
Hansen NDindler CHalskov KIversen OBossen CBasballe DSchouten B(2019)How Participatory Design WorksProceedings of the 31st Australian Conference on Human-Computer-Interaction10.1145/3369457.3369460(30-41)Online publication date: 2-Dec-2019
https://dl.acm.org/doi/10.1145/3369457.3369460
Tolmie P(2019)Orienting to the WildInto the Wild: Beyond the Design Research Lab10.1007/978-3-030-18020-1_10(195-235)Online publication date: 4-Jul-2019
https://doi.org/10.1007/978-3-030-18020-1_10
Show More Cited By

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

Figures

Tables

Media

View Table of Conten