research-article

Some Prior(s) Experience Necessary: Templates for Getting Started With Bayesian Analysis

Authors:

Jessica Hullman,

Paul ResnickAuthors Info & Claims

CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

Paper No.: 479, Pages 1 - 12

https://doi.org/10.1145/3290605.3300709

Published: 02 May 2019 Publication History

Abstract

Bayesian statistical analysis has gained attention in recent years, including in HCI. The Bayesian approach has several advantages over traditional statistics, including producing results with more intuitive interpretations. Despite growing interest, few papers in CHI use Bayesian analysis. Existing tools to learn Bayesian statistics require significant time investment, making it difficult to casually explore Bayesian methods. Here, we present a tool that lowers the barrier to exploration: a set of R code templates that guide Bayesian novices through their first analysis. The templates are tailored to CHI, supporting analyses found to be most common in recent CHI papers. In a user study, we found that the templates were easy to understand and use. However, we found that participants without a statistical background were not confident in their use. Together our contributions provide a concise analysis tool and empirical results for understanding and addressing barriers to using Bayesian analysis in HCI.

Supplementary Material

ZIP File (paper479.zip)

Bayesian templates for beginners IMPORTANT: This version of the templates is FROZEN FOR ARCHIVAL PURPOSES, and was last updated Sept 2018. Please go to https://github.com/cdphelan/bayesian-template/ to access the most updated version.

Download
25.91 MB

ZIP File (paper479pvc.zip)

Preview video captions

Download
1.21 KB

MP4 File (paper479p.mp4)

Preview video

Download
1.76 MB

References

[1]

Monya Baker. 2016. 1,500 scientists lift the lid on reproducibility. Nature News 533, 7604 (2016), 452.

[2]

James O Berger and Donald A Berry. 1988. Statistical analysis and the illusion of objectivity. American Scientist 76, 2 (1988), 159--165.

[3]

Charles C Bonwell and James A Eison. 1991. Active Learning: Creating Excitement in the Classroom. 1991 ASHE-ERIC Higher Education Reports. ERIC.

[4]

Kieth A Carlson and Jennifer R Winquist. 2011. Evaluating an active learning approach to teaching introductory statistics: A classroom workbook approach. Journal of Statistics Education 19, 1 (2011).

[5]

Ben-Zvi Dani and Garfield Joan. 2004. Statistical literacy, reasoning, and thinking: Goals, definitions, and challenges. In The challenge of developing statistical literacy, reasoning and thinking. Springer, 3--15.

[6]

Zoltan Dienes. 2011. Bayesian versus orthodox statistics: Which side are you on? Perspectives on Psychological Science 6, 3 (2011), 274-- 290.

[7]

Alan Dix. 2017. Making Sense of Statistics in HCI: From P to Bayes and Beyond. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '17). ACM, New York, NY, USA, 1236--1239.

Digital Library

[8]

Stéphane Doyen, Olivier Klein, Cora-Lise Pichon, and Axel Cleeremans. 2012. Behavioral priming: it's all in the mind, but whose mind? PloS one 7, 1 (2012), e29081.

[9]

Pierre Dragicevic. 2016. Fair Statistical Communication in HCI. In Modern Statistical Methods for HCI. Springer, 291 -- 330.

[10]

Jim Eison. 2010. Using active learning instructional strategies to create excitement and enhance learning. (2010).

[11]

Michael Fernandes, Logan Walls, Sean Munson, Jessica Hullman, and Matthew Kay. 2018. Uncertainty Displays Using Quantile Dotplots or CDFs Improve Transit Decision-Making. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, New York, NY, USA, Article 144, 12 pages.

Digital Library

[12]

Michael Fernandes, Logan Walls, Sean Munson, Jessica Hullman, and Matthew Kay. 2018. Uncertainty Displays Using Quantile Dotplots or CDFs Improve Transit Decision-Making. In Conference on Human Factors in Computing Systems - CHI '18.

Digital Library

[13]

Scott Freeman, Sarah L. Eddy, Miles McDonough, Michelle K. Smith, Nnadozie Okoroafor, Hannah Jordt, and Mary Pat Wenderoth. 2014. Active learning increases student performance in science, engineering, and mathematics. Proceedings of the National Academy of Sciences 111, 23 (2014), 8410--8415. arXiv:http://www.pnas.org/content/111/23/8410.full.pdf

[14]

Gerd Gigerenzer and Ulrich Hoffrage. 1995. How to improve Bayesian reasoning without instruction: Frequency formats. Psychological Review 102 (1995), 684--704.

[15]

Daniel G Goldstein and David Rothschild. 2014. Lay understanding of probability distributions. Judgment & Decision Making 9, 1 (2014).

[16]

Richard R Hake. 1998. Interactive-engagement versus traditional methods: A six-thousand-student survey of mechanics test data for introductory physics courses. American journal of Physics 66, 1 (1998), 64--74.

[17]

L Hasher and R T Zacks. 1984. Automatic processing of fundamental information: the case of frequency of occurrence. The American psychologist 39, 12 (1984), 1372--1388.

[18]

Rink Hoekstra, Richard D. Morey, Jeffrey N. Rouder, and Eric-Jan Wagenmakers. 2014. Robust misinterpretation of confidence intervals. Psychonomic Bulletin & Review 21, 5 (01 Oct 2014), 1157--1164.

[19]

U. Hoffrage and G. Gigerenzer. 1998. Using natural frequencies to improve diagnostic inferences. Academic Medicine: Journal of the Association of American Medical Colleges 73, 5 (May 1998), 538--540.

[20]

Jessica Hullman, Matthew Kay, Yea-Seul Kim, and Samana Shrestha. 2018. Imagining Replications: Graphical Prediction & Discrete Visualizations Improve Recall & Estimation of Effect Uncertainty. IEEE Trans. Visualization & Comp. Graphics (Proc. InfoVis) (2018). http: //idl.cs.washington.edu/papers/imagining-replications

[21]

Jessica Hullman, Paul Resnick, and Eytan Adar. 2015. Hypothetical outcome plots outperform error bars and violin plots for inferences about reliability of variable ordering. PloS one 10, 11 (2015), e0142444.

[22]

John PA Ioannidis. 2005. Why most published research findings are false. PLoS medicine 2, 8 (2005), e124.

[23]

Yvonne Jansen and Kasper Hornbæk. 2018. How Relevant Are Incidental Power Poses for HCI?. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems (CHI '18). ACM, New York, NY, USA, Article 14, 14 pages.

Digital Library

[24]

Alex Kale, Francis Nguyen, Matthew Kay, and Jessica Hullman. 2018. Hypothetical Outcome Plots Help Untrained Observers Judge Trends in Ambiguous Data. IEEE transactions on visualization and computer graphics (2018).

[25]

Maurits Kaptein and Judy Robertson. 2012. Rethinking Statistical Analysis Methods for CHI. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '12). ACM, New York, NY, USA, 1105--1114.

Digital Library

[26]

Matthew Kay, Tara Kola, Jessica Hullman, and Sean Munson. 2016. When (ish) is my bus? User-centered visualizations of uncertainty in everyday, mobile predictive systems. In Proceedings of the 34th Annual ACM Conference on Human Factors in Computing Systems (CHI '16).

Digital Library

[27]

Matthew Kay, Gregory L. Nelson, and Eric B. Hekler. 2016. ResearcherCentered Design of Statistics: Why Bayesian Statistics Better Fit the Culture and Incentives of HCI. In Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI '16). ACM, New York, NY, USA, 4521--4532.

Digital Library

[28]

Matthew Kay, Shwetak N. Patel, and Julie A. Kientz. 2015. How Good is 85%?: A Survey Tool to Connect Classifier Evaluation to Acceptability of Accuracy. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, New York, NY, USA, 347--356.

Digital Library

[29]

Yea-Seul Kim, Katharina Reinecke, and Jessica Hullman. 2017. Explaining the gap: Visualizing one's predictions improves recall and comprehension of data. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, 1375--1386.

Digital Library

[30]

Yea-Seul Kim, Logan Walls, Peter Krafft, and Jessica Hullman. 2019. In Light of Beliefs: A Bayesian Model of Cognition to Evaluate Everyday Data Interpretation. Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems (2019).

[31]

Kenneth R Koedinger, Jihee Kim, Julianna Zhuxin Jia, Elizabeth A McLaughlin, and Norman L Bier. 2015. Learning is not a spectator sport: Doing is better than watching for learning from a MOOC. In Proceedings of the second (2015) ACM conference on learning@ scale. ACM, 111--120.

Digital Library

[32]

John K Kruschke. 2010. What to believe: Bayesian methods for data analysis. Trends in cognitive sciences 14, 7 (2010), 293--300.

[33]

John K. Kruschke. 2015. Doing Bayesian Data Analysis: A Tutorial with R and BUGS (2nd ed.). Academic Press, Inc., Orlando, FL, USA.

[34]

Jisoo Lee, Erin Walker, Winslow Burleson, Matthew Kay, Matthew Buman, and Eric B. Hekler. 2017. Self-Experimentation for Behavior Change: Design and Formative Evaluation of Two Approaches. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 6837--6849.

Digital Library

[35]

Richard McElreath. 2016. Statistical Rethinking: A Bayesian Course with Examples in R and Stan (1st ed.). CRC Press, Boca Raton, FL, USA.

[36]

Daniel McNeish. 2016. On using Bayesian methods to address small sample problems. Structural Equation Modeling: A Multidisciplinary Journal 23, 5 (2016), 750--773.

[37]

David S Moore. 1997. New pedagogy and new content: The case of statistics. International statistical review 65, 2 (1997), 123--137.

[38]

Carol Moser, Chanda Phelan, Paul Resnick, Sarita Y Schoenebeck, and Katharina Reinecke. 2017. No such thing as too much chocolate: evidence against choice overload in e-commerce. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. ACM, 4358--4369.

Digital Library

[39]

Manuel Nordhoff, Tal August, Nigini A Oliveira, and Katharina Reinecke. 2018. A Case for Design Localization: Diversity of Website Aesthetics in 44 Countries. In Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems. ACM, 337.

Digital Library

[40]

M.W. Oakes. 1986. Statistical inference: a commentary for the social and behavioural sciences. Wiley. https://books.google.com/books? id=OhFHAAAAMAAJ

[41]

Anthony O'Hagan, Caitlin E Buck, Alireza Daneshkhah, J Richard Eiser, Paul H Garthwaite, David J Jenkinson, Jeremy E Oakley, and Tim Rakow. 2006. Uncertain judgements: eliciting experts' probabilities. John Wiley & Sons.

[42]

David R Sokoloff and Ronald K Thornton. 1997. Using interactive lecture demonstrations to create an active learning environment. The Physics Teacher 35, 6 (1997), 340--347.

[43]

Gail M Sullivan and Richard Feinn. 2012. Using effect size - or why the P value is not enough. Journal of graduate medical education 4, 3 (2012), 279--282.

[44]

Fengpeng Yuan, Xianyi Gao, and Janne Lindqvist. 2017. How Busy Are You?: Predicting the Interruptibility Intensity of Mobile Users. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, NY, USA, 5346--5360.

Digital Library

Cited By

Iarygina OHornbæk KMottelson A(2025)Demand characteristics in human–computer experimentsInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2024.103379193:COnline publication date: 1-Jan-2025
https://dl.acm.org/doi/10.1016/j.ijhcs.2024.103379
Taka EStein SWilliamson J(2024)Does Interactive Conditioning Help Users Better Understand the Structure of Probabilistic Models?IEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2022.323196730:7(3256-3267)Online publication date: Jul-2024
https://doi.org/10.1109/TVCG.2022.3231967
Reza MMusabirov ILaundry NLiut MWilliams J(2023)Using A/B Testing as a Pedagogical Tool for Iterative Design in HCI ClassroomsProceedings of the 5th Annual Symposium on HCI Education10.1145/3587399.3587412(43-48)Online publication date: 28-Apr-2023
https://dl.acm.org/doi/10.1145/3587399.3587412
Show More Cited By

Index Terms

Some Prior(s) Experience Necessary: Templates for Getting Started With Bayesian Analysis
1. Human-centered computing
  1. Human computer interaction (HCI)
    1. HCI design and evaluation methods
      1. User studies
    2. HCI theory, concepts and models

Recommendations

Statistics for HCI
CHI EA '22: Extended Abstracts of the 2022 CHI Conference on Human Factors in Computing Systems

Many researchers and practitioners find statistics confusing. This course aims to give attendees an understanding of the meaning of the various statistics they see in papers or need to use in their own work. The course builds on the instructor’s ...
Making Sense of Statistics in HCI: From P to Bayes and Beyond
CHI EA '17: Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems

Many find statistics confusing, and perhaps more so given recent publicity of problems with traditional p-values and alternative statistical techniques including confidence intervals and Bayesian statistics. This course aims to help attendees navigate ...
Statistics for HCI
CHI EA '21: Extended Abstracts of the 2021 CHI Conference on Human Factors in Computing Systems

Many researchers and practitioners find statistics confusing. This course aims to help change that, to give attendees an understanding of the meaning of the various statistics they see in papers or need to use in their own work. The course builds on the ...

Comments

Information & Contributors

Information

Published In

cover image ACM Conferences

CHI '19: Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems

May 2019

9077 pages

ISBN:9781450359702

DOI:10.1145/3290605

General Chairs:
Stephen Brewster
University of Glasgow, Scotland, UK
,
Geraldine Fitzpatrick
TU Wien, Austria
,
Program Chairs:
Anna Cox
University College London, UK
,
Vassilis Kostakos
University of Melbourne, Australia

Copyright © 2019 ACM.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

Sponsors

SIGCHI: ACM Special Interest Group on Computer-Human Interaction

Publisher

Association for Computing Machinery

New York, NY, United States

Publication History

Published: 02 May 2019

Permissions

Request permissions for this article.

Request Permissions

Check for updates

Author Tags

Qualifiers

Research-article

Conference

CHI '19

Sponsor:

SIGCHI

CHI '19: CHI Conference on Human Factors in Computing Systems

May 4 - 9, 2019

Glasgow, Scotland Uk

Acceptance Rates

CHI '19 Paper Acceptance Rate 703 of 2,958 submissions, 24%;

Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

Upcoming Conference

CHI 2025

Sponsor:
sigchi

ACM CHI Conference on Human Factors in Computing Systems

April 26 - May 1, 2025

Yokohama , Japan

Contributors

Other Metrics

View Article Metrics

Bibliometrics & Citations

Bibliometrics

Article Metrics

6
Total Citations
View Citations
550
Total Downloads

Downloads (Last 12 months)62
Downloads (Last 6 weeks)5

Reflects downloads up to 28 Feb 2025

Other Metrics

View Author Metrics

Citations

Cited By

Iarygina OHornbæk KMottelson A(2025)Demand characteristics in human–computer experimentsInternational Journal of Human-Computer Studies10.1016/j.ijhcs.2024.103379193:COnline publication date: 1-Jan-2025
https://dl.acm.org/doi/10.1016/j.ijhcs.2024.103379
Taka EStein SWilliamson J(2024)Does Interactive Conditioning Help Users Better Understand the Structure of Probabilistic Models?IEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2022.323196730:7(3256-3267)Online publication date: Jul-2024
https://doi.org/10.1109/TVCG.2022.3231967
Reza MMusabirov ILaundry NLiut MWilliams J(2023)Using A/B Testing as a Pedagogical Tool for Iterative Design in HCI ClassroomsProceedings of the 5th Annual Symposium on HCI Education10.1145/3587399.3587412(43-48)Online publication date: 28-Apr-2023
https://dl.acm.org/doi/10.1145/3587399.3587412
Masson DMalacria SCasiez GVogel D(2023)Statslator: Interactive Translation of NHST and Estimation Statistics Reporting Styles in Scientific DocumentsProceedings of the 36th Annual ACM Symposium on User Interface Software and Technology10.1145/3586183.3606762(1-14)Online publication date: 29-Oct-2023
https://dl.acm.org/doi/10.1145/3586183.3606762
Liu YKale AAlthoff THeer J(2021)Boba: Authoring and Visualizing Multiverse AnalysesIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2020.302898527:2(1753-1763)Online publication date: Feb-2021
https://doi.org/10.1109/TVCG.2020.3028985
Sarma AKay MBernhaupt RMueller FVerweij DAndres JMcGrenere JCockburn AAvellino IGoguey ABjørn PZhao SSamson BKocielnik R(2020)Prior Setting in Practice: Strategies and Rationales Used in Choosing Prior Distributions for Bayesian AnalysisProceedings of the 2020 CHI Conference on Human Factors in Computing Systems10.1145/3313831.3376377(1-12)Online publication date: 21-Apr-2020
https://dl.acm.org/doi/10.1145/3313831.3376377

View Options

Login options

Check if you have access through your login credentials or your institution to get full access on this article.

Full Access

Get this Publication

View options

PDF

View or Download as a PDF file.

eReader

View online with eReader.

HTML Format

View this article in HTML Format.

Figures

Tables

Media

View Table of Conten