Cited By
View all- Robinson RMekler E(2024)Reflections Towards More Thoughtful Engagement with Literature Reviews in HCIProceedings of the Halfway to the Future Symposium10.1145/3686169.3686183(1-4)Online publication date: 21-Oct-2024
"That's Not Good Science!": An Argument for the Thoughtful Use of Formative Situations in Research Through Design
Authors: 
Raquel B Robinson, 
Anya Osborne, Chen Ji, James Collin Fey, Ella Dagan, 
Katherine IsbisterAuthors Info & Claims
Raquel B Robinson, Anya Osborne, Chen Ji, James Collin Fey, Ella Dagan, Katherine IsbisterAuthors Info & ClaimsArticle No.: 545, Pages 1 - 8
Abstract
Most currently accepted approaches to evaluating Research through Design (RtD) presume that design prototypes are finalized and ready for robust testing in laboratory or in-the-wild settings. However, it is also valuable to assess designs at intermediate phases with mid-fidelity prototypes, not just to inform an ongoing design process, but also to glean knowledge of broader use to the research community. We propose ’formative situations’ as a frame for examining mid-fidelity prototypes-in-process in this way. We articulate a set of criteria to help the community better assess the rigor of formative situations, in the service of opening conversation about establishing formative situations as a valuable contribution type within the RtD community.
Supplemental Material
- Download
- 199.35 MB
- Transcript
References
[1]
[n. d.]. CHI 2024. https://chi2024.acm.org/
[2]
Eric P S Baumer, Mark Blythe, and Theresa Jean Tanenbaum. 2020. Evaluating Design Fiction: The Right Tool for the Job. In Proceedings of the 2020 ACM Designing Interactive Systems Conference(DIS ’20). Association for Computing Machinery, New York, NY, USA, 1901–1913. https://doi.org/10.1145/3357236.3395464
[3]
Roni Berger. 2013. Now I see it, now I don’t: researcher’s position and reflexivity in qualitative research. Qualitative Research 15, 2 (1 2013), 219–234. https://doi.org/10.1177/1468794112468475
[4]
Kirsten Boehner, Rogério DePaula, Paul Dourish, and Phoebe Sengers. 2007. How Emotion is Made and Measured. Int. J. Hum.-Comput. Stud. 65, 4 (4 2007), 275–291. https://doi.org/10.1016/j.ijhcs.2006.11.016
[5]
D.T. Campbell and J.C. Stanley. 1966. Experimental And Quasi-Experimental Designs For Research. (1966).
[6]
Joey Campbell. 2018. Exploring the relationship between vr immersion, adaptive resistance and physical exertion. Conference on Human Factors in Computing Systems - Proceedings 2018-April (2018), 1–4. https://doi.org/10.1145/3170427.3173019
[7]
Alan Chamberlain, Andy Crabtree, Tom Rodden, Matt Jones, and Yvonne Rogers. 2012. Research in the Wild: Understanding ’in the Wild’ Approaches to Design and Development. In Proceedings of the Designing Interactive Systems Conference(DIS ’12). Association for Computing Machinery, New York, NY, USA, 795–796. https://doi.org/10.1145/2317956.2318078
[8]
Nigel Cross. 2001. Designerly Ways of Knowing: Design Discipline versus Design Science. Design Issues 17, 3 (2001), 49–55. http://www.jstor.org/stable/1511801
[9]
Ella Dagan, James Fey, Sanoja Kikkeri, Charlene Hoang, Rachel Hsiao, and Katherine Isbister. 2020. Flippo the Robo-Shoe-Fly: A Foot Dwelling Social Wearable Companion. In CHI ’20: CHI Conference on Human Factors in Computing Systems. ACM, 1–10. https://doi.org/10.1145/3334480.3382928
[10]
Ella Dagan and Katherine Isbister. 2021. Synergistic Social Technology: Designing Systems with ‘Needs’ That Encourage and Support Social Interaction. In Proceedings of the 2021 ACM Designing Interactive Systems Conference(DIS ’21). Association for Computing Machinery, New York, NY, USA, 1419–1432. https://doi.org/10.1145/3461778.3462021
[11]
Alan Dix, Janet E Finlay, Gregory D Abowd, and Russell Beale. 2003. Human-Computer Interaction (3rd Edition). Prentice-Hall, Inc., USA.
[12]
Paul Dourish. 2006. Implications for Design. In Conference on Human Factors in Computing Systems (CHI).
[13]
Anthony Dunne. 2006. Hertzian Tales: Electronic Products, Aesthetic Experience, and Critical Design. The MIT Press.
[14]
Anthony Dunne and Fiona Raby. 2013. Speculative Everything: Design, Fiction, and Social Dreaming. The MIT Press. http://www.jstor.org/stable/j.ctt9qf7j7
[15]
Daniel Fallman. 2003. Design-oriented human - Computer interaction. In Conference on Human Factors in Computing Systems - Proceedings. 225–232. https://doi.org/10.1145/642611.642652
[16]
Daniel Fallman. 2007. Why Research-Oriented Design Isn’t Design-Oriented Research: On the Tensions Between Design and Research in an Implicit Design Discipline. Knowledge, Technology & Policy 20, 3 (10 2007), 193–200. https://doi.org/10.1007/s12130-007-9022-8
[17]
Christopher Frayling. 1993. Research in Art and Design., 5 pages.
[18]
Bill Gaver and John Bowers. 2012. Annotated Portfolios. Interactions (2012).
[19]
William Gaver. 2012. What Should We Expect from Research through Design?. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems(CHI ’12). Association for Computing Machinery, New York, NY, USA, 937–946. https://doi.org/10.1145/2207676.2208538
[20]
William W. Gaver, Peter Gall Krogh, Andy Boucher, and David Chatting. 2022. Emergence as a Feature of Practice-based Design Research. In DIS 2022 - Proceedings of the 2022 ACM Designing Interactive Systems Conference: Digital Wellbeing. Association for Computing Machinery, Inc, 517–526. https://doi.org/10.1145/3532106.3533524
[21]
Egon G Guba Yvonn and A S Lincoln. 1994. Competing Paradigms in Qualitative Research. Handbook of qualitative research, Sage Publications (1994).
[22]
Kristina Höök and Jonas Löwgren. 2012. Strong Concepts: Intermediate-Level Knowledge in Interaction Design Research. ACM Trans. Comput.-Hum. Interact. 19 (10 2012), 23:1–23:18. https://doi.org/10.1145/2362364.2362371
[23]
Hilary Hutchinson, Wendy Mackay, Bosse Westerlund, Benjamin B Bederson, Allison Druin, Catherine Plaisant, Michel Beaudouin-Lafon, Stéphane Conversy, Helen Evans, Heiko Hansen, Nicolas Roussel, Björn Eiderbäck, Sinna Lindquist, and Yngve Sundblad. [n. d.]. Technology Probes: Inspiring Design for and with Families. Technical Report.
[24]
Katherine Isbister, Peter Cottrell, Alessia Cecchet, Ella Dagan, Nikki Theofanopoulou, Ferran Altarriba Bertran, Aaron J. Horowitz, Nick Mead, Joel B. Schwartz, and Petr Slovak. 2022. Design (Not) Lost in Translation: A Case Study of an Intimate-Space Socially Assistive "robot" for Emotion Regulation. ACM Transactions on Computer-Human Interaction 29, 4 (8 2022). https://doi.org/10.1145/3491083
[25]
Chen Ji and Katherine Isbister. 2022. AR Fidget: Augmented Reality Experiences That Support Emotion Regulation through Fidgeting. In Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems(CHI EA ’22). Association for Computing Machinery, New York, NY, USA. https://doi.org/10.1145/3491101.3519874
[26]
Larry Laudan. 1977. Progress and its Problems: Toward a Theory of Scientific Growth. Vol. 87. University of California Press.
[27]
Brenda Laurel. 2003. Design Research: Methods and Perspectives. The MIT Press.
[28]
Jonathan Lazar, Jinjuan Heidi Feng, and Harry Hochheiser. 2017. Research Methods in Human-Computer Interaction (2 ed.). Morgan Kaufmann, Cambridge, MA. https://doi.org/science/book/9780128053904
[29]
David J. Lewkowicz. 2001. The Concept of Ecological Validity: What are Its Limitations and is It Bad to Be Invalid?Infancy 2, 4 (2001), 437–450. https://doi.org/10.1207/S15327078IN0204_03
[30]
Adam Lobel, Marientina Gotsis, and Erin Reynolds. 2016. Nevermind: Emotion Regulation in a Biofeedback Video Game. In CHI EA ’16: Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems. ACM, San Jose, California, 1945–1951. https://doi.org/10.13140/RG.2.1.4372.6249
[31]
I Scott MacKenzie. 2013. Human-Computer Interaction: An Empirical Research Perspective (1st ed.). Morgan Kaufmann Publishers Inc., San Francisco, CA, USA.
[32]
David Moher, Alejandro R. Jadad, and Peter Tugwell. 1996. Assessing the Quality of Randomized Controlled Trials: Current Issues and Future Directions. International Journal of Technology Assessment in Health Care 12, 2 (3 1996), 195–208. https://doi.org/10.1017/S0266462300009570
[33]
William Odom, Ron Wakkary, Youn-kyung Lim, Audrey Desjardins, Bart Hengeveld, and Richard Banks. 2016. From Research Prototype to Research Product. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems(CHI ’16). Association for Computing Machinery, New York, NY, USA, 2549–2561. https://doi.org/10.1145/2858036.2858447
[34]
Judith S Olson and Wendy A Kellogg. 2014. Ways of Knowing in HCI. Springer Publishing Company, Incorporated.
[35]
Antti Oulasvirta and Kasper Hornbæk. 2016. HCI research as problem-solving. In Conference on Human Factors in Computing Systems - Proceedings. Association for Computing Machinery, 4956–4967. https://doi.org/10.1145/2858036.2858283
[36]
Wanda Pillow. 2003. Confession, catharsis, or cure? Rethinking the uses of reflexivity as methodological power in qualitative research. International Journal of Qualitative Studies in Education 16, 2 (3 2003), 175–196. https://doi.org/10.1080/0951839032000060635
[37]
Isabel Prochner and Danny Godin. 2022. Quality in research through design projects: Recommendations for evaluation and enhancement. Design Studies 78 (1 2022). https://doi.org/10.1016/j.destud.2021.101061
[38]
Horst W J Rittel and Melvin M Webber. 1973. Dilemmas in a General Theory of Planning. Technical Report 2. 155–169 pages.
[39]
Yvonne Rogers. 2011. Interaction Design Gone Wild: Striving for Wild Theory. Interactions 18, 4 (7 2011), 58–62. https://doi.org/10.1145/1978822.1978834
[40]
Yvonne Rogers and Paul Marshall. 2017. Research in the Wild. Synthesis Lectures on Human-Centered Informatics 10, 3 (4 2017), i–97. https://doi.org/10.2200/S00764ED1V01Y201703HCI037
[41]
Yvonne Rogers and Helen Sharp. 2002. Interaction Design: Beyond Human-Computer Interaction. Wiley.
[42]
Donald A. Schön. 1987. Educating the Reflective Practitioner: Toward a New Design for Teaching and Learning in the Professions (first edition ed.). Jossey-Bass Publishers, San Francisco.
[43]
Donald A. Schön. 2017. The Reflective Practitioner. Routledge. https://doi.org/10.4324/9781315237473
[44]
Katie Siek, Gillian Hayes, Mark Newman, and John Tang. 2014. Field Deployments: Knowing from Using in Context. 119–142. https://doi.org/10.1007/978-1-4939-0378-8_6
[45]
Erik Stolterman. 2008. The Nature of Design Practice and Implications for Interaction Design Research. International Journal of Design; Vol 2, No 1 (2008) (2008). http://www.ijdesign.org/index.php/IJDesign/article/view/240/148
[46]
Niels Van Berkel and Kasper Hornbæk. 2023. Implications of Human-Computer Interaction Research. Interactions (2023).
[47]
Jacob O Wobbrock and Julie A Kientz. 2016. Research Contributions in Human-Computer Interaction. Interactions 23, 3 (4 2016), 38–44. https://doi.org/10.1145/2907069
[48]
John Zimmerman and Jodi Forlizzi. 2014. Research Through Design in HCI. In Ways of Knowing in HCI.
[49]
John Zimmerman, Jodi Forlizzi, and Shelley Evenson. 2007. Research through design as a method for interaction design research in HCI. Conference on Human Factors in Computing Systems - ProceedingsJanuary (2007), 493–502. https://doi.org/10.1145/1240624.1240704
Index Terms
- "That's Not Good Science!": An Argument for the Thoughtful Use of Formative Situations in Research Through Design
Recommendations
An analysis and critique of Research through Design: towards a formalization of a research approach
DIS '10: Proceedings of the 8th ACM Conference on Designing Interactive SystemsThe field of HCI is experiencing a growing interest in Research through Design (RtD), a research approach that employs methods and processes from design practice as a legitimate method of inquiry. We are interested in expanding and formalizing this ...
Teaching interaction design by research through design
OzCHI '18: Proceedings of the 30th Australian Conference on Computer-Human InteractionResearch-through-design (RtD) has become a well-established research approach within HCI research. In this paper we discuss how research-through-design can be applied as an explicit strategy for teaching interaction design. RtD is productive when ...
The [Everyday] Future by Design: Opportunities for the Design Exploration of Everyday Sustainability
DIS '16 Companion: Proceedings of the 2016 ACM Conference Companion Publication on Designing Interactive SystemsSituated within the interaction design and Human-Computer Interaction spaces, this design led research is concerned with the development of ongoing creative and adaptable approaches towards everyday sustainability. Grounded in the theoretical concepts ...
Comments
Information & Contributors
Information
Published In
May 2024
4761 pages
ISBN:9798400703317
DOI:10.1145/3613905
Copyright © 2024 Owner/Author.
Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the Owner/Author.
Sponsors
Publisher
Association for Computing Machinery
New York, NY, United States
Publication History
Published: 11 May 2024
Check for updates
Author Tags
Qualifiers
- Extended-abstract
- Research
- Refereed limited
Conference
Acceptance Rates
Overall Acceptance Rate 6,164 of 23,696 submissions, 26%
Upcoming Conference
CHI 2025
- Sponsor:
- sigchi
Contributors
Other Metrics
Bibliometrics & Citations
Bibliometrics
Article Metrics
- View Citations1Total Citations
- 430Total Downloads
- Downloads (Last 12 months)430
- Downloads (Last 6 weeks)78
Reflects downloads up to 28 Feb 2025
Other Metrics
Citations
Cited By
View all- Robinson RMekler E(2024)Reflections Towards More Thoughtful Engagement with Literature Reviews in HCIProceedings of the Halfway to the Future Symposium10.1145/3686169.3686183(1-4)Online publication date: 21-Oct-2024
View Options
Login options
Check if you have access through your login credentials or your institution to get full access on this article.
Sign inFull Access
View options
View or Download as a PDF file.
PDFeReader
View online with eReader.
eReaderFull Text
View this article in Full Text.
Full TextHTML Format
View this article in HTML Format.
HTML Format