Jordan Wirfs-Brock
Laura Devendorf
Brian C Keegan
ABSTRACT
This workshop engages the phenomenon of synesthesia to explore how translating between sensory modalities might uncover new ways to experience and represent data: What does it mean to taste a timeline, hear a network, or touch a categorical scale? We employ the method of sketching, which traditionally favors visual representations, and consider what it means to ‘sketch’ in other modalities like sound, taste, and touch. Through a series of rapid, playful activities ideating data representations across sensory modalities, we will explore how the affordances of sketching—like intentional ambiguity—might help designers creatively map data to experience. We will also discuss challenges for sensory sketching in remote, collaborative environments and brainstorm suggestions for digital tools. The outcome of this workshop will be a series of exercises and examples that serve as a toolkit for designers, researchers, and data practitioners to incorporate sketching across the senses into their work.
- AirNow.gov. 2018. Air Quality Index (AQI) Basics. Retrieved October 8, 2021 from https://www.airnow.gov/aqi/aqi-basics/Google Scholar
- Kristina Andersen, Laura Devendorf, James Pierce, Ron Wakkary, and Daniela K. Rosner. 2018. Disruptive Improvisations: Making Use of Non-Deterministic Art Practices in HCI. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems (Montreal QC, Canada) (CHI EA ’18). Association for Computing Machinery, New York, NY, USA, 1–8. https://doi.org/10.1145/3170427.3170630Google Scholar
- Leonardo Angelini, Zuzanna Lechelt, Eva Hornecker, Paul Marshall, Can Liu, Margot Brereton, Alessandro Soro, Nadine Couture, Omar Abou Khaled, and Elena Mugellini. 2018. Internet of Tangible Things: Workshop on Tangible Interaction with the Internet of Things. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems(CHI EA ’18). Association for Computing Machinery, New York, NY, USA, 1–8. https://doi.org/10.1145/3170427.3170634Google ScholarDigital Library
- Maryam Bandukda, Aneesha Singh, Catherine Holloway, Nadia Berthouze, Emeline Brulé, Ana Tajadura-Jiménez, Oussama Metatla, Ana Javornik, and Anja Thieme. 2021. Rethinking the Senses: A Workshop on Multisensory Embodied Experiences and Disability Interactions. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, 1–5. https://doi.org/10.1145/3411763.3441356Google Scholar
- Joanna Berzowska, Aisling Kelliher, Daniela K. Rosner, Matt Ratto, and Suzanne Kite. 2019. Critical Materiality: Creating Toolkits and Methods for Engaging Materiality in HCI. In Proceedings of the Thirteenth International Conference on Tangible, Embedded, and Embodied Interaction (Tempe, Arizona, USA) (TEI ’19). Association for Computing Machinery, New York, NY, USA, 691–694. https://doi.org/10.1145/3294109.3295656Google ScholarDigital Library
- Jas Brooks, Pedro Lopes, Judith Amores, Emanuela Maggioni, Haruka Matsukura, Marianna Obrist, Roshan Lalintha Peiris, and Nimesha Ranasinghe. 2021. Smell, Taste, and Temperature Interfaces. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, 1–6. https://doi.org/10.1145/3411763.3441317Google ScholarDigital Library
- Bill Buxton. 2010. Sketching user experiences: getting the design right and the right design. Morgan Kaufmann.Google Scholar
- Matthew Chalmers and Areti Galani. 2004. Seamful interweaving: heterogeneity in the theory and design of interactive systems. In Proceedings of the 5th conference on Designing interactive systems: processes, practices, methods, and techniques(DIS ’04). Association for Computing Machinery, New York, NY, USA, 243–252. https://doi.org/10.1145/1013115.1013149Google ScholarDigital Library
- Jialin Deng, Yan Wang, Carlos Velasco, Ferran Altarriba Altarriba Bertran, Rob Comber, Marianna Obrist, Katherine Isbister, Charles Spence, and Florian ’Floyd’ Mueller. 2021. The Future of Human-Food Interaction. In Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, New York, NY, USA, 1–6. https://doi.org/10.1145/3411763.3441312Google Scholar
- Audrey Desjardins, Jena McWhirter, Min Jung Koo, Philbert Widjaja, and Nick Logler. 2021. Data Bakery, url =https://www.studiotilt.design/data-bakery, lastaccessed =October 13, 2021,.Google Scholar
- Laura Devendorf. 2021. Wear. Retrieved October 13, 2021 from http://artfordorks.com/wear/Google Scholar
- Laura Devendorf, Kristina Andersen, and Aisling Kelliher. 2020. Making Design Memoirs: Understanding and Honoring Difficult Experiences. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems(CHI ’20). Association for Computing Machinery, New York, NY, USA, 1–12. https://doi.org/10.1145/3313831.3376345Google ScholarDigital Library
- Inger Ekman and Michal Rinott. 2010. Using Vocal Sketching for Designing Sonic Interactions. In Proceedings of the 8th ACM Conference on Designing Interactive Systems (Aarhus, Denmark) (DIS ’10). Association for Computing Machinery, New York, NY, USA, 123–131. https://doi.org/10.1145/1858171.1858195Google ScholarDigital Library
- Mikhaila Friske, Jordan Wirfs-Brock, and Laura Devendorf. 2020. Entangling the Roles of Maker and Interpreter in Interpersonal Data Narratives: Explorations in Yarn and Sound. In Proceedings of the 2020 ACM Designing Interactive Systems Conference (Eindhoven, Netherlands) (DIS ’20). Association for Computing Machinery, New York, NY, USA, 297–310. https://doi.org/10.1145/3357236.3395442Google ScholarDigital Library
- Courtney Lynn Gallagher. 2017. Sketching for Ideation: A Structured Approach for Increasing Divergent Thinking. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems(CHI EA ’17). Association for Computing Machinery, New York, NY, USA, 106–111. https://doi.org/10.1145/3027063.3048424Google ScholarDigital Library
- Gabriela Goldschmidt. 1991. The dialectics of sketching. Creativity Research Journal 4, 2 (Jan. 1991), 123–143. https://doi.org/10.1080/10400419109534381 Publisher: Routledge _eprint: https://doi.org/10.1080/10400419109534381.Google ScholarCross Ref
- Gabriela Goldschmidt. 2017. Manual Sketching: Why Is It Still Relevant?In The Active Image: Architecture and Engineering in the Age of Modeling, Sabine Ammon and Remei Capdevila-Werning (Eds.). Springer International Publishing, Cham, 77–97. https://doi.org/10.1007/978-3-319-56466-1_4Google Scholar
- John E Harrison. 2001. Synaesthesia: The strangest thing. Oxford University Press, USA.Google Scholar
- Douglas R Hofstadter. 1997. Le ton beau de Marot: In praise of the music of language. Basic Books New York.Google Scholar
- Vincent Isnard, Marine Taffou, Isabelle Viaud-Delmon, and Clara Suied. 2016. Auditory sketches: very sparse representations of sounds are still recognizable. PloS one 11, 3 (2016), e0150313.Google ScholarCross Ref
- Chang Hee Lee, Dan Lockton, John Stevens, Stephen Jia Wang, and SungHee Ahn. 2019. Synaesthetic-Translation Tool: Synaesthesia as an Interactive Material for Ideation. In Extended Abstracts of the 2019 CHI Conference on Human Factors in Computing Systems(Glasgow, Scotland Uk) (CHI EA ’19). Association for Computing Machinery, New York, NY, USA, 1–6. https://doi.org/10.1145/3290607.3312849Google ScholarDigital Library
- Giorgia Lupi and Stefanie Posavec. 2018. Observe, Collect, Draw!: A Visual Journal (illustrated edition ed.). Princeton Architectural Press.Google Scholar
- Yuri Ono Miranda July, Harrell Fletcher. 2002. Learning to Love You More. Retrieved October 8, 2021 from http://www.learningtoloveyoumore.com/index.phpGoogle Scholar
- Sara Price, Kerstin Leder Mackley, Carey Jewitt, Gijs Huisman, Bruna Petreca, Nadia Berthouze, Domenico Prattichizzo, and Vincent Hayward. 2018. Reshaping Touch Communication: An Interdisciplinary Research Agenda. In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems(CHI EA ’18). Association for Computing Machinery, New York, NY, USA, 1–8. https://doi.org/10.1145/3170427.3170603Google ScholarDigital Library
- Hugo Scurto, Baptiste Caramiaux, and Frederic Bevilacqua. 2021. Prototyping Machine Learning Through Diffractive Art Practice. In Designing Interactive Systems Conference 2021(DIS ’21). Association for Computing Machinery, New York, NY, USA, 2013–2025. https://doi.org/10.1145/3461778.3462163Google ScholarDigital Library
- Jakob Tholander, Klas Karlgren, Robert Ramberg, and Per Sökjer. 2008. Where all the interaction is: sketching in interaction design as an embodied practice. In Proceedings of the 7th ACM conference on Designing interactive systems(DIS ’08). Association for Computing Machinery, New York, NY, USA, 445–454. https://doi.org/10.1145/1394445.1394493Google ScholarDigital Library
Index Terms
- Sketching Across the Senses: Exploring Sensory Translation as a Generative Practice for Designing Data Representations
Index terms have been assigned to the content through auto-classification.
Recommendations
Sketching user experiences tutorial: stories, strategies, surfaces
ITS '13: Proceedings of the 2013 ACM international conference on Interactive tabletops and surfacesPaper-pencil sketches are a valuable tool during different stages of experience design in human-computer interaction. This hands-on tutorial will demonstrate how to integrate sketching into researchers' and interaction designers' everyday practice -- ...
Visualization and the art of metaphor
C&C '07: Proceedings of the 6th ACM SIGCHI conference on Creativity & cognitionThis tutorial is designed for students in various disciplines (arts & sciences) that traditionally work in other media and would like to integrate visualization into their work to enhance communication of ideas and concepts. Attendees that are already ...
Hands-on sketching course
CHI EA '14: CHI '14 Extended Abstracts on Human Factors in Computing SystemsSketching as a technique to quickly draw something on a piece of paper is used to explore and communicate ideas. This course introduces basic sketching techniques and a visual language which participants can apply in research and practice. It is a hands-...
Comments