Eleftherios Papachristos
Dimitrios Raptis
Anders Bruun
ABSTRACT
User interfaces utilising multiple modalities or even multisensory feedback are more common, creating the need for evaluation techniques that can consider various quality dimensions. This paper demonstrates how the trained panel method can support the design and evaluation of physical or complex technological artefacts by mapping out design spaces based on their descriptive attributes. It is an expert-based method, and the goal is to derive a comprehensive description of a sample of existing artefacts or prototypes. The method entails training as well as multiple feedback sessions to ensure consensus among panel participants. We describe the advantages and limitations of the method by presenting how it was applied to identify important attributes in the design or evaluation of smartwatches. Apart from the specific case described in detail, we also discuss how and in what context the trained panel method can provide value in HCI research and practice.
- Daniel E Berlyne. 1973. Aesthetics and psychobiology. Journal of Aesthetics and Art Criticism 31, 4 (1973).Google Scholar
- Michael Braun, Anja Mainz, Ronee Chadowitz, Bastian Pfleging, and Florian Alt. 2019. At your service: Designing voice assistant personalities to improve automotive user interfaces. In Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems. 1–11. Google ScholarDigital Library
- Douglas J. Carroll. 1972. Individual differences and multidimensional scaling. Multidimesional scaling. Theory and applications in the behavioral sciences, Theory 1(1972), 105–155.Google Scholar
- Tobias Dahl, Oliver Tomic, Jens P Wold, and Tormod Næs. 2008. Some new tools for visualising multi-way sensory data. Food quality and preference 19, 1 (2008), 103–113. https://doi.org/10.1016/j.foodqual.2007.07.001Google Scholar
- David Hirst and Tormod Næs. 1994. A graphical technique for assessing differences among a set of rankings. Journal of Chemometrics 8, 1 (1994), 81–93. https://doi.org/10.1002/cem.1180080108Google ScholarCross Ref
- F Husson, S Le Dien, and J Pagès. 2001. Which value can be granted to sensory profiles given by consumers? Methodology and results. Food quality and preference 12, 5-7 (2001), 291–296.Google Scholar
- William Jones, Robert Capra, Anne Diekema, Jaime Teevan, Manuel Pérez-Quiñones, Jesse David Dinneen, and Bradley Hemminger. 2015. ”For Telling” the Present: Using the Delphi Method to Understand Personal Information Management Practices. Association for Computing Machinery, New York, NY, USA, 3513–3522. https://doi.org/10.1145/2702123.2702523 Google ScholarDigital Library
- Talia Lavie and Noam Tractinsky. 2004. Assessing dimensions of perceived visual aesthetics of web sites. International Journal of Human-Computer Studies 60, 3 (2004), 269–298. https://doi.org/10.1016/j.ijhcs.2003.09.002 Google ScholarDigital Library
- Harry T Lawless, Hildegarde Heymann, 2010. Sensory evaluation of food: principles and practices. Vol. 2. Springer.Google Scholar
- Robert L Mack and Jakob Nielsen. 1995. Usability inspection methods: Executive summary. In Readings in Human–Computer Interaction. Elsevier, 170–181. Google ScholarDigital Library
- Ville-Veikko Mattila. 2002. Ideal point modelling of speech quality in mobile communications based on multidimensional scaling (MDS). In Audio Engineering Society Convention 112. Audio Engineering Society.Google Scholar
- Tormod Næs, Per Bruun Brockhoff, and Oliver Tomic. 2011. Statistics for sensory and consumer science. John Wiley & Sons.Google Scholar
- Tormod Naes and Einar Risvik. 1997. Multivariate analysis of data in sensory science. Chemometrics and Intelligent Laboratory Systems 36, 1 (1997), 75–76. https://doi.org/10.1016/S0169-7439(96)00072-XGoogle ScholarCross Ref
- Tormod Næs and Ragnhild Solheim. 1991. Detection and interpretation of variation within and between assessors in sensory profiling. Journal of sensory studies 6, 3 (1991), 159–177. https://doi.org/10.1111/j.1745-459X.1991.tb00512.xGoogle ScholarCross Ref
- Jakob Nielsen. 1994. Usability engineering. Morgan Kaufmann.Google Scholar
- Jakob Nielsen. 1994. Usability inspection methods. In Conference companion on Human factors in computing systems. 413–414. Google ScholarDigital Library
- Marianna Obrist, Alexandre N. Tuch, and Kasper Hornbaek. 2014. Opportunities for Odor: Experiences with Smell and Implications for Technology. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (Toronto, Ontario, Canada) (CHI ’14). Association for Computing Machinery, New York, NY, USA, 2843–2852. https://doi.org/10.1145/2556288.2557008 Google ScholarDigital Library
- Eleftherios Papachristos and Nikolaos Avouris. 2009. The Subjective and Objective Nature of Website Aesthetic Impressions. In Human-Computer Interaction – INTERACT 2009, Tom Gross, Jan Gulliksen, Paula Kotzé, Lars Oestreicher, Philippe Palanque, Raquel Oliveira Prates, and Marco Winckler (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 119–122. Google ScholarDigital Library
- Eleftherios Papachristos and Nikolaos Avouris. 2011. The Application of Preference Mapping in Aesthetic Website Evaluation. In Human-Computer Interaction – INTERACT 2011, Pedro Campos, Nicholas Graham, Joaquim Jorge, Nuno Nunes, Philippe Palanque, and Marco Winckler (Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 616–619. Google ScholarDigital Library
- Eleftherios Papachristos and Nikolaos Avouris. 2013. The Influence of Website Category on Aesthetic Preferences. In Human-Computer Interaction – INTERACT 2013, Paula Kotzé, Gary Marsden, Gitte Lindgaard, Janet Wesson, and Marco Winckler(Eds.). Springer Berlin Heidelberg, Berlin, Heidelberg, 445–452.Google ScholarCross Ref
- Eleftherios Papachristos, Christos Katsanos, Nikolaos Karousos, Ioannis Ioannidis, Christos Fidas, and Nikolaos Avouris. 2014. Deployment, Usage and Impact of Social Media Tools in Small and Medium Enterprises: A Case Study. In Social Computing and Social Media, Gabriele Meiselwitz (Ed.). Springer International Publishing, Cham, 79–90. Google ScholarDigital Library
- Anna Perry, Laura Malinin, Eulanda Sanders, Yan Li, and Katharine Leigh. 2017. Explore consumer needs and design purposes of smart clothing from designers’ perspectives. International Journal of Fashion Design, Technology and Education 10, 3(2017), 372–380. https://doi.org/10.1080/17543266.2016.1278465Google ScholarCross Ref
- Dimitrios Raptis, Eleftherios Papachristos, Anders Bruun, and Jesper Kjeldskov. 2020. Why did you pick that? A study on smartwatch design qualities and people’s preferences. Behaviour & Information Technology 0, 0 (2020), 1–18. https://doi.org/10.1080/0144929X.2020.1836259Google ScholarCross Ref
- Kai Riemer, Judy Kay, 2019. Mapping beyond the uncanny valley: A Delphi study on aiding adoption of realistic digital faces. In Proceedings of the 52nd Hawaii International Conference on System Sciences. https://doi.org/10.24251/HICSS.2019.577Google Scholar
- Herbert Stone and Joel L. Sidel. 2004. Special Problems. In Sensory Evaluation Practices(third edition ed.), Herbert Stone and Joel L. Sidel (Eds.). Academic Press, San Diego, 279–335. https://doi.org/10.1016/B978-012672690-9/50012-3Google Scholar
- Oliver Tomic, Giorgio Luciano, Asgeir Nilsen, Grethe Hyldig, Kirsten Lorensen, and Tormod Næs. 2010. Analysing sensory panel performance in a proficiency test using the PanelCheck software. European Food Research and Technology 230, 3 (2010), 497–511. https://doi.org/10.1007/s00217-009-1185-yGoogle ScholarCross Ref
- Ellen Van Kleef, Hans CM Van Trijp, and Pieternel Luning. 2006. Internal versus external preference analysis: An exploratory study on end-user evaluation. 17, 5 (2006), 387–399. https://doi.org/10.1016/j.foodqual.2005.05.001Google Scholar
- Carlos Velasco, Marianna Obrist, Olivia Petit, and Charles Spence. 2018. Multisensory Technology for Flavor Augmentation: A Mini Review. Frontiers in Psychology 9 (2018), 26. https://doi.org/10.3389/fpsyg.2018.00026Google ScholarCross Ref
- Susan Weinschenk and Dean T Barker. 2000. Designing effective speech interfaces. John Wiley & Sons, Inc. Google ScholarDigital Library
- Thierry Worch, Sébastien Lê, and Pieter Punter. 2010. How reliable are the consumers? Comparison of sensory profiles from consumers and experts. Food quality and preference 21, 3 (2010), 309–318.Google Scholar
Index Terms
- The Trained Panel Method and Its Application in HCI Research
Index terms have been assigned to the content through auto-classification.
Recommendations
A postphenomenological method for HCI research
OzCHI '18: Proceedings of the 30th Australian Conference on Computer-Human InteractionThis paper presents an analysis of the presence and potential of a postphenomenology as a research method in human-computer interaction (HCI). Specifically, we introduce Rosenberger's method of variational cross-examination; an empirical approach that ...
Evaluating HCI Research beyond Usability
CHI EA '18: Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing SystemsEvaluating research artefacts is an important step to showcase the validity of a chosen approach. The CHI community has developed and agreed upon a large variety of evaluation methods for HCI research; however, sometimes those methods are not applicable ...
Affordances in HCI: toward a mediated action perspective
CHI '12: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsInterpretations of the concept of "affordances" in HCI are becoming increasingly diverse, extending well beyond the original Gibsonian meaning. We discuss some of the key analyses of affordances in HCI research and make three related claims. First, we ...
Comments