Joni Salminen
Ying-Hsang Liu
Bernard J. Jansen
ABSTRACT
In an experiment, we present 38 marketing and data analysts professionals with two online AI-driven persona interfaces, one using numbers and the other using text. We employ eye tracking, think-aloud, and a post-engagement survey for data collection to measure perception and visual engagement with the personas along 7 constructs. Results show that the use of numbers has a mixed effect on the perceptions and visual engagement of the persona profile, with job role as a determining factor on whether numbers/text affect end users for 2 of the constructs. The use of numbers has a significant positive effect on user perceptions of usefulness by analysts but a significantly negative effect on user perceptions of completeness for both marketers and analysts. The use of numbers decreases the perceived completeness of the personas for both marketer and analysts. This research has both theoretical and practical consequences for AI-driven persona development and their interface design, suggesting that the inclusion of numbers can have a desirable effect for certain roles but with possible negative effects on user perceptions.
- T. Adlin and J. Pruitt, The Essential Persona Lifecycle: Your Guide to Building and Using Personas: Morgan Kaufmann Publishers Inc., 2010.Google Scholar
Digital Library
- O. Alhadreti and P. Mayhew, "To Intervene or Not to Intervene: An Investigation of Three Think-aloud Protocols in Usability Testing," Journal of Usability Studies,, vol. 12, pp. 111--132, 2017.Google ScholarDigital Library
- N. S. Ali, "Information Behaviour of Sell-side and Other Analysts in Financial Institutions in Toronto, Canada," ed, 2017.Google Scholar
- S. Alspaugh, N. Zokaei, A. Liu, C. Jin, and M. A. Hearst, "Futzing and Moseying: Interviews with Professional Data Analysts on Exploration Practices," IEEE Transactions on Visualization and Computer Graphics, vol. 25, pp. 22--31, 2019.Google ScholarDigital Library
- J. An, H. Kwak, J. Salminen, S. G. Jung, and B. J. Jansen, "Customer segmentation using online platforms: isolating behavioral and demographic segments for persona creation via aggregated user data," Social Network Analysis and Mining, vol. 8, p. 54, 2018.Google ScholarCross Ref
- J. An, H. Kwak, J. Salminen, S. G. Jung, and B. J. Jansen, "Imaginary People Representing Real Numbers: Generating Personas from Online Social Media Data," ACM Transactions on the Web, vol. 12, p. Article 27, 2018.Google ScholarDigital Library
- N. S. Baldwin and R. E. Rice, "Information-seeking behavior of securities analysts: Individual and institutional influences, information sources and channels, and outcomes," Journal of the American Society for Information Science, vol. 48, pp. 674--693, 1997.Google ScholarDigital Library
- T. Blascheck, M. John, S. Koch, L. Bruder, and T. Ertl, "Triangulating User Behavior Using Eye Movement, Interaction, and Think Aloud Data," in The Ninth Biennial ACM Symposium on Eye Tracking Research & Applications, 2016, pp. 175--182.Google ScholarDigital Library
- S. Bødker, E. Christiansen, T. Nyvang, and P.-O. Zander, "Personas, people and participation: challenges from the trenches of local government," presented at the Proceedings of the 12th Participatory Design Conference: Research Papers - Volume 1, Roskilde, Denmark, 2012.Google Scholar
- J. Brickey, S. Walczak, and T. Burgess, "A Comparative Analysis of Persona Clustering Methods," in Americas Conference on Information Systems (AMCIS2010), 2010, p. Article 217 http://aisel.aisnet.org/amcis2010/217.Google Scholar
- Y. Chang, Y. Lim, and E. Stolterman, "Personas: From Theory to Practices," in the 5th Nordic Conference on Human-computer Interaction: Building Bridges, New York, NY, 2008, pp. 439--442.Google Scholar
- C. N. Chapman, E. Love, R. P. Milham, P. ElRif, and J. L. Alford, "Quantitative Evaluation of Personas as Information," in Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 2008, pp. 1107--1111.Google ScholarCross Ref
- C. N. Chapman and R. P. Milham, "The Personas' New Clothes: Methodological and Practical Arguments against a Popular Method," in Proceedings of the Human Factors and Ergonomics Society Annual Meeting, 2006, pp. 634--636.Google ScholarCross Ref
- A. Cooper, The Inmates Are Running the Asylum: Why High Tech Products Drive Us Crazy and How to Restore the Sanity (2nd Edition): Pearson Higher Education, 2004.Google ScholarDigital Library
- N. De Voil. (2010, 14 Aug). Personas considered harmful. Available: http://www.devoil.com/papers/PersonasConsideredHarmful.pdf.Google Scholar
- J. T. DeCuir-Gunby, P. L. Marshall, and A. W. McCulloch, "Developing and using a codebook for the analysis of interview data: An example from a professional development research project," Field Methods, vol. 23, pp. 136--155, 2011.Google ScholarCross Ref
- J. T. Du, "The information journey of marketing professionals: Incorporating work task-driven information seeking, information judgments, information use, and information sharing," Journal of the Association for Information Science and Technology, vol. 65, pp. 1850--1869, 2014/09/01 2014.Google ScholarDigital Library
- J. T. Du, Y. H. Liu, Q. Zhu, and Y. Chen, "Modelling Marketing Professionals' Information Behaviour in the Workplace: Towards a Holistic Understanding," Information Research, vol. 18, 2013.Google Scholar
- N. Du, D. V. Budescu, M. K. Shelly, and T. C. Omer, "The appeal of vague financial forecasts," Organizational Behavior and Human Decision Processes, vol. 114, pp. 179--189, 2011.Google ScholarCross Ref
- A. T. Duchowski, Eye Tracking Methodology: Theory and Practice. London: Springer, 2009.Google ScholarDigital Library
- M. Eiband, H. Schneider, M. Bilandzic, J. Fazekas-Con, M. Haug, and H. Hussmann, "Bringing transparency design into practice," in Proceedings of the 23rd International Conference on Intelligent User Interfaces, 2018, pp. 211--223.Google Scholar
- S. Elling, L. Lentz, and M. De Jong, "Combining concurrent think-aloud protocols and eye-tracking observations: An analysis of verbalizations and silences," IEEE Transactions on Professional Communication, vol. 55, pp. 206--220, 2012.Google ScholarCross Ref
- K. A. Ericsson and H. A. Simon, Protocol analysis: Verbal reports as data, Rev. ed. Cambridge, MA.: MIT Press, 1993.Google ScholarCross Ref
- S. Fan, T.-T. Ng, J. S. Herberg, B. L. Koenig, and S. Xin, "Real or Fake?: human judgments about photographs and computer-generated images of faces," presented at the SIGGRAPH Asia 2012 Technical Briefs, Singapore, Singapore, 2012.Google ScholarDigital Library
- R. Fidel, Human Information Interaction: An Ecological Approach to Information Behavior. Cambridge, MA: The MIT Press, 2012.Google ScholarCross Ref
- I. R. Floyd, C. M. Jones, and M. B. Twidale, "Resolving Incommensurable Debates: A Preliminary Identification of Persona Kinds, Attributes, and Characteristics," Artifact vol. 2, pp. 12--26, 2008.Google ScholarCross Ref
- E. Friess, "Personas and Decision Making in the Design Process: An Ethnographic Case Study," in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Austin, Texas, USA, 2012, pp. 1209--1218.Google Scholar
- J. Gothelf, "Using proto-personas for executive alignment," UX Magazine, p. Article No: 821, 2012.Google Scholar
- J. F. Hair, W. C. Black, B. J. Babin, R. E. Anderson, and R. L. Tatham, Multivariate data analysis, 7 ed.: Pearson, 2007.Google Scholar
- C. G. Hill, M. Haag, A. Oleson, C. Mendez, N. Marsden, A. Sarma, et al., "Gender-Inclusiveness Personas vs. Stereotyping: Can We Have it Both Ways?," in Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, Denver, Colorado, USA, 2017, pp. 6658--6671.Google Scholar
- T. W. Howard, "Are Personas Really Usable?," Communication Design Quarterly Review, vol. 3, pp. 20--26, 2015.Google ScholarDigital Library
- A. Hutchison and L. Gerstein, "Emotion Recognition, Emotion Expression, and Cultural Display Rules: Implications for Counseling," Journal of Asia Pacific Counseling, vol. 7, pp. 19--35, 2017.Google ScholarCross Ref
- T. Jiang, Q. Guo, Y. Xu, and S. Fu, "A diary study of information encountering triggered by visual stimuli on micro-blogging services," Information Processing & Management, vol. 56, pp. 29--42, 2019.Google ScholarCross Ref
- S. Jung, J. An, H. Kwak, M. Ahmad, L. Nielsen, and B. J. Jansen, "Persona Generation from Aggregated Social Media Data," in ACM Conference on Human Factors in Computing Systems 2017 (CHI2017), Denver, CO, 2017, pp. 1748--1755.Google Scholar
- S. G. Jung, J. Salminen, J. An, H. Kwak, and B. J. Jansen, "Automatically Conceptualizing Social Media Analytics Data via Personas (Demo Paper)," in International AAAI Conference on Web and Social Media (ICWSM 2018), Stanford, CA, USA, 2018, pp. 715--716.Google Scholar
- P. T. A. Junior and L. V. L. Filgueiras, "User modeling with personas," in Proceedings of the 2005 Latin American conference on Human-Computer Interaction, Cuernavaca, Mexico, 2005, pp. 277--282.Google Scholar
- D. Kahan, E. Peters, and E. Dawson, "Motivated numeracy and enlightened self-government," Behavioural Public Policy, vol. 1, pp. 54--86, 2017.Google ScholarCross Ref
- S. Kandel, A. Paepcke, J. M. Hellerstein, and J. Heer, "Enterprise Data Analysis and Visualization: An Interview Study," IEEE Transactions on Visualization and Computer Graphics, vol. 18, pp. 2917--2926, 2012.Google ScholarDigital Library
- C. C. Kuhlthau, "The role of experience in the information search process of an early career information worker: Perceptions of uncertainty, complexity, construction, and sources," Journal of the American Society for Information Science, vol. 50, pp. 399--412, 1999.Google ScholarCross Ref
- C. C. Kuhlthau, "Towards collaboration between information seeking and information retrieval," Information Research-an International Electronic Journal, vol. 10, p. http://InformationR.net/ir/10, 2005.Google Scholar
- H. Kwak, J. An, and B. J. Jansen, "Automatic Generation of Personas Using YouTube Social Media Data," in Hawaii International Conference on System Sciences (HICSS-50), Waikoloa, Hawaii, 2017, pp. 833--842.Google Scholar
- S. Lalle and C. Conati, "The role of user differences in customization: a case study in personalization for infovis-based content," presented at the Proceedings of the 24th International Conference on Intelligent User Interfaces, Marina del Ray, California, 2019.Google Scholar
- G. J. Leckie, K. E. Pettigrew, and C. Sylvain, "Modeling the Information Seeking of Professionals : A General Model Derived from Research on Engineers, Health Care Professionals, and Lawyers Author ( s ): Gloria J. Leckie, Karen E. Pettigrew, Christian Sylvain Published by : The University of Chi," The Library Quarterly, vol. 66, pp. 161--193, 1996.Google ScholarCross Ref
- J. Li, M. X. Zhou, H. Yang, and G. Mark, "Confiding in and listening to virtual agents: The effect of personality," in Proceedings of the 22nd International Conference on Intelligent User Interfaces, 2017, pp. 275--286.Google Scholar
- Lieve Laporte, Karin Slegers, and D. D. Grooff, "Using correspondence analysis to monitor the persona segmentation process," in The 7th Nordic Conference on Human-Computer Interaction: Making Sense Through Design (NordiCHI '12), 2012, pp. 265--274.Google Scholar
- C. Liu, Y.-H. Liu, T. Gedeon, Y. Zhao, Y. Wei, and F. Yang, "The effects of perceived chronic pressure and time constraint on information search behaviors and experience," Information Processing & Management, vol. 56, pp. 1667--1679, 2019.Google ScholarDigital Library
- H. Liu and P. Maes, "What would they think?: A computational model of attitudes," in Proceedings of the 9th International Conference on Intelligent User Interfaces, 2004, pp. 38--45.Google Scholar
- Y.-H. Liu, P. Thomas, M. Bacic, T. Gedeon, and X. Li, "Natural search user interfaces for complex biomedical search: An eye tracking study," Journal of the Australian Library and Information Association, vol. 66, pp. 364--381, 2017/11/08 2017.Google ScholarCross Ref
- Y.-H. Liu and N. Wacholder, "Evaluating the impact of MeSH (Medical Subject Headings) terms on different types of searchers," Information Processing & Management, vol. 53, pp. 851--870, 2017.Google ScholarDigital Library
- E. Løhre and K. H. Teigen, "Probabilities associated with precise and vague forecasts," Journal of Behavioral Decision Making, vol. 30, pp. 1014--1026, 2017.Google ScholarCross Ref
- N. Marsden and M. Haag, "Stereotypes and Politics: Reflections on Personas," presented at the Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems, San Jose, California, USA, 2016.Google Scholar
- T. Matthews, T. Judge, and S. Whittaker, "How Do Designers and User Experience Professionals Actually Perceive and Use Personas?," in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, 2012, pp. 1219--1228.Google ScholarDigital Library
- J. McGinn and N. Kotamraju, "Data-driven Persona Development," in Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Florence, Italy, 2008, pp. 1521--1524.Google Scholar
- T. Miaskiewicz and K. A. Kozar, "Personas and user-centered design: How can personas benefit product design processes?," Design Studies, vol. 32, pp. 417--430, 2011.Google ScholarCross Ref
- T. Miaskiewicz, T. Sumner, and K. A. Kozar, "A latent semantic analysis methodology for the identification and creation of personas," in SIGCHI Conference on Human Factors in Computing Systems, Florence, Italy, 2008, pp. 1501--1510.Google Scholar
- L. Nielsen, Personas - User Focused Design. London: Springer-Verlag, 2013.Google ScholarCross Ref
- L. Nielsen and K. S. Hansen, "Personas is applicable: a study on the use of personas in Denmark," presented at the Proceedings of the 32nd annual ACM conference on Human factors in computing systems, Toronto, Ontario, Canada, 2014.Google ScholarDigital Library
- L. Nielsen, K. S. Hansen, J. Stage, and J. Billestrup, "A Template for Design Personas: Analysis of 47 Persona Descriptions from Danish Industries and Organizations," Int. J. Sociotechnology Knowl. Dev., vol. 7, pp. 45--61, 2015.Google ScholarDigital Library
- D. Norman. (2004, 1 Sep). Ad-Hoc Personas & Empathetic Focus. Available: http://www.jnd.org/dn.mss/personas_empath.htmlGoogle Scholar
- T. M. Pavia and J. A. Costa, "The winning number: Consumer perceptions of alpha-numeric brand names," The Journal of Marketing, vol. 57, pp. 85--98, 1993.Google ScholarCross Ref
- K. Pernice, K. Whitenton, and J. Nielsen. (2014). How People Read on the Web: The Eyetracking Evidence. Available: https://www.nngroup.com/reports/how-people-read-web-eyetracking-evidence/Google Scholar
- C. Peters, G. Castellano, and S. d. Freitas, "An exploration of user engagement in HCI," presented at the Proceedings of the International Workshop on Affective-Aware Virtual Agents and Social Robots, Boston, Massachusetts, 2009.Google Scholar
- K. Rönkkö, M. Hellman, B. Kilander, and Y. Dittrich, "Personas is Not Applicable: Local Remedies Interpreted in a Wider Context," in Proceedings of the Eighth Conference on Participatory Design: Artful Integration: Interweaving Media, Materials and Practices, New York, NY, USA, 2004, pp. 112--120.Google Scholar
- J. E. Russo, E. J. Johnson, and D. L. Stephens, "The validity of verbal protocols," Memory & Cognition, vol. 17, pp. 759--769, 1989.Google ScholarCross Ref
- J. Salminen, S. Jung, J. An, H. Kwak, L. Nielsen, and B. J. Jansen, "Confusion and information triggered by photos in persona profiles," International Journal of Human-Computer Studies, vol. 129, pp. 1--14, 2019.Google ScholarDigital Library
- J. Salminen, H. Kwak, J. An, S. G. Jung, and B. J. Jansen, "Are personas done? Evaluating their usefulness in the age of digital analytics," Persona Studies, vol. 4, pp. 47--65, 2018.Google ScholarCross Ref
- J. Salminen, H. Kwak, J. M. Santos, S. G. Jung, J. An, and B. J. Jansen, "Persona Perception Scale: Developing and Validating an Instrument for Human-Like Representations of Data," in ACM CHI Conference on Human Factors in Computing Systems (CHI2018) (Extended Abstract), Montreal, Canada, 2018, p. LBW075.Google Scholar
- J. Salminen, L. Nielsen, S.-G. Jung, J. An, H. Kwak, and B. J. Jansen, "Is More Better?: Impact of Multiple Photos on Perception of Persona Profiles," presented at the Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems, Montreal QC, Canada, 2018.Google Scholar
- J. Salminen, J. M. Santos, S.-G. Jung, M. Eslami, and B. J. Jansen, "Persona Transparency: Analyzing the Impact of Explanations on Perceptions of Data-Driven Personas," International Journal of Human-Computer Interaction, vol. 0, pp. 1--13, 2019.Google ScholarCross Ref
- J. Salminen, S. Sengun, S. G. Jung, and B. J. Jansen, "Design Issues in Automatically Generated Persona Profiles: A Qualitative Analysis from 38 Think-Aloud Transcripts," in The ACM SIGIR Conference on Human Information Interaction and Retrieval (CHIIR2019), Glasgow, UK., 2019, pp. 225--229.Google Scholar
- J. Salminen, S. Şengün, H. Kwak, B. J. Jansen, J. An, S. Jung, et al., "From 2,772 segments to five personas: Summarizing a diverse online audience by generating culturally adapted personas," First Monday, vol. 23, p. http://firstmonday.org/ojs/index.php/fm/article/view/8415, 2018.Google Scholar
- J. Shaughnessy, E. Zechmeister, and J. Zechmeister, Research Methods in Psychology, 10th ed. Dubuque: McGraw-Hill Education, 2014.Google Scholar
- R. Sinha, "Persona development for information-rich domains," in CHI '03 Extended Abstracts on Human Factors in Computing Systems, Ft. Lauderdale, Florida, USA, 2003, pp. 830--831.Google Scholar
- B. Steichen, C. Conati, and G. Carenini, "Inferring Visualization Task Properties, User Performance, and User Cognitive Abilities from Eye Gaze Data," ACM Trans. Interact. Intell. Syst., vol. 4, pp. 1--29, 2014.Google ScholarDigital Library
- M. C. Tang, Y. H. Liu, and W. C. Wu, "A study of the influence of task familiarity on user behaviors and performance with a MeSH term suggestion interface for PubMed bibliographic search," Int J Med Inform, vol. 82, pp. 832--43, Sep 2013.Google ScholarCross Ref
- R. Tang and Y. Song, "Cognitive styles and eye movement patterns: an empirical investigation into user interactions with interface elements and visualisation objects of a scientific information system," Information Research, vol. 23, 2018.Google Scholar
- E. Thivant, "Information Seeking and Use Behaviour of Economists and Business Analysts," Information Research: An International Electronic Journal, vol. 10, 2005.Google Scholar
- P. Thunholm, "Decision-making style: habit, style or both?," Personality and Individual Differences, vol. 36, pp. 931--944, 2004.Google ScholarCross Ref
- D. Toker, C. Conati, and G. Carenini, "User-adaptive Support for Processing Magazine Style Narrative Visualizations: Identifying User Characteristics that Matter," presented at the 23rd International Conference on Intelligent User Interfaces, Tokyo, Japan, 2018.Google Scholar
- A. Tversky and D. Kahneman, "Judgment under uncertainty: Heuristics and biases," Science vol. 185, pp. 1124--1131, 1974.Google ScholarCross Ref
- Y. Ueno, H. Natsukawa, N. Aoyama, and K. Koyamada, "Exploration behavior of group-in-a-box layouts," Visual Informatics, 2019.Google Scholar
- K. Verbert, D. Parra, P. Brusilovsky, and E. Duval, "Visualizing recommendations to support exploration, transparency and controllability," in Proceedings of the 2013 International Conference on Intelligent User Interfaces, 2013, pp. 351--362.Google Scholar
- P. Wittek, Y.-H. Liu, S. Darányi, T. Gedeon, and I. S. Lim, "Risk and ambiguity in information seeking: Eye gaze patterns reveal contextual behavior in dealing with uncertainty," Frontiers in Psychology, vol. 7, p. 1790, 2016.Google ScholarCross Ref
- G.-X. Xie and A. Kronrod, "Is the devil in the details? The signaling effect of numerical precision in environmental advertising claims," Journal of Advertising, vol. 41, pp. 103--117, 2012.Google ScholarCross Ref
- I. Yaniv and D. P. Foster, "Graininess of judgment under uncertainty: An accuracy-informativeness trade-off," Journal of Experimental Psychology: General, vol. 124, pp. 424--432, 1995.Google ScholarCross Ref
- P. Zerbino, D. Aloini, R. Dulmin, and V. Mininno, "Big Data-enabled Customer Relationship Management: A holistic approach," Information Processing & Management, vol. 54, pp. 818--846, 2018.Google ScholarCross Ref
- Y. C. Zhang and N. Schwarz, "The power of precise numbers: A conversational logic analysis," Journal of Experimental Social Psychology, vol. 49, pp. 944--946, 2013.Google ScholarCross Ref
Index Terms
- The effect of numerical and textual information on visual engagement and perceptions of AI-driven persona interfaces
Recommendations
Design Issues in Automatically Generated Persona Profiles: A Qualitative Analysis from 38 Think-Aloud Transcripts
CHIIR '19: Proceedings of the 2019 Conference on Human Information Interaction and RetrievalIncreased access to data and computational techniques enable innovations in the space of automated customer analytics, for example, automatic persona generation. Automatic persona generation is the process of creating data-driven representations from ...
The Choice of a Persona: An Analysis of Why Stakeholders Choose a Given Persona for a Design Task
HCI International 2023 – Late Breaking PapersAbstractAlthough personas have been applied for two decades, not much is known about why a designer chooses a specific persona for a given design task. This question matters because if designers prefer one persona over another, then the needs and ...
The Effect of Experience on Persona Perceptions
CHI EA '20: Extended Abstracts of the 2020 CHI Conference on Human Factors in Computing SystemsUser perceptions of personas affect the adoption of personas for decision-making in real organizations. To investigate how experience affects the way an individual perceives a persona, we conduct an experimental study with individuals less and more ...
Comments