skip to main content
10.1145/3173574.3174079acmconferencesArticle/Chapter ViewAbstractPublication PageschiConference Proceedingsconference-collections
research-article
Public Access

Uncertainty Visualization Influences how Humans Aggregate Discrepant Information

Published: 21 April 2018 Publication History

Abstract

The number of sensors in our surroundings that provide the same information steadily increases. Since sensing is prone to errors, sensors may disagree. For example, a GPS-based tracker on the phone and a sensor on the bike wheel may provide discrepant estimates on traveled distance. This poses a user dilemma, namely how to reconcile the conflicting information into one estimate. We investigated whether visualizing the uncertainty associated with sensor measurements improves the quality of users' inference. We tested four visualizations with increasingly detailed representation of uncertainty. Our study repeatedly presented two sensor measurements with varying degrees of inconsistency to participants who indicated their best guess of the "true" value. We found that uncertainty information improves users' estimates, especially if sensors differ largely in their associated variability. Improvements were larger for information-rich visualizations. Based on our findings, we provide an interactive tool to select the optimal visualization for displaying conflicting information.

Supplementary Material

ZIP File (pn4086-file4.zip)

References

[1]
Anthony D. Andre and Henry A. Cutler. 1998. Displaying Uncertainty in Advanced Navigation Systems. Proceedings of the Human Factors and Ergonomics Society Annual Meeting 42, 1 (1998), 31--35.
[2]
Peter W. Battaglia, Robert A. Jacobs, and Richard N. Aslin. 2003. Bayesian integration of visual and auditory signals for spatial localization. Journal of the Optical Society of America A 20, 7 (Jul 2003), 1391--1397.
[3]
Ken Brodlie, Rodolfo Allendes Osorio, and Adriano Lopes. 2012. A Review of Uncertainty in Data Visualization. Expanding the Frontiers of Visual Analytics and Visualization (2012), 81--109.
[4]
David V. Budescu. 2006. Confidence in aggregation of opinions from multiple sources. In Information Sampling and Adaptive Cognition. 327--352.
[5]
David V. Budescu, Stephen Broomell, and Han-Hui Por. 2009. Improving communication of uncertainty in the reports of the intergovernmental panel on climate change. Psychological science 20, 3 (2009), 299--308.
[6]
Michael Correll and Michael Gleicher. 2014. Error bars considered harmful: Exploring alternate encodings for mean and error. IEEE Transactions on Visualization and Computer Graphics 20, 12 (2014), 2142--2151.
[7]
Helen Couclelis. 2003. The Certainty of Uncertainty: GIS and the Limits of Geographic Knowledge. Transactions in GIS 7, 2 (2003), 165--175.
[8]
Marc O. Ernst. 2010. Decisions Made Better. Science 329, 5995 (2010), 1022--1023.
[9]
Marc O. Ernst and Heinrich H. Bülthoff. 2004. Merging the senses into a robust percept. Trends in Cognitive Sciences 8, 4 (2004), 162 -- 169.
[10]
Nivan Ferreira, Danyel Fisher, and Arnd C. Konig. 2014. Sample-oriented Task-driven Visualizations: Allowing Users to Make Better, More Confident Decisions. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '14). 571--580.
[11]
Nahum Gershon. 1998. Visualization of an imperfect world. IEEE Computer Graphics and Applications 18, 4 (1998), 43--45.
[12]
Gerd Gigerenzer, Ralph Hertwig, Eva Van Den Broek, Barbara Fasolo, and Konstantinos V. Katsikopoulos. 2005. "A 30% chance of rain tomorrow": How does the public understand probabilistic weather forecasts? Risk Analysis 25, 3 (jun 2005), 623--629.
[13]
Miriam Greis, Passant El. Agroudy, Hendrik Schuff, Tonja Machulla, and Albrecht Schmidt. 2016. Decision-Making Under Uncertainty: How the Amount of Presented Uncertainty Influences User Behavior. In Proceedings of the 9th Nordic Conference on Human-Computer Interaction (NordiCHI '16). 52:1--52:4.
[14]
Miriam Greis, Emre Avci, Albrecht Schmidt, and Tonja Machulla. 2017a. Increasing Users' Confidence in Uncertain Data by Aggregating Data from Multiple Sources. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems (CHI '17). ACM, New York, 828--840.
[15]
Miriam Greis, Jessica Hullman, Michael Correll, Matthew Kay, and Orit Shaer. 2017b. Designing for Uncertainty in HCI: When Does Uncertainty Help?. In Proceedings of the 2017 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA '17). 593--600.
[16]
Miriam Greis, Thorsten Ohler, Niels Henze, and Albrecht Schmidt. 2015. Investigating Representation Alternatives for Communicating Uncertainty to Non-experts. In Human-Computer Interaction - INTERACT 2015. Vol. 9299. 256--263.
[17]
Miriam Greis, Hendrik Schuff, Marius Kleiner, Niels Henze, and Albrecht Schmidt. 2017. Input Controls for Entering Uncertain Data. Proceedings of the ACM on Human-Computer Interaction 1, 1 (jun 2017), 1--17.
[18]
Harald Ibrekk and M. Granger Morgan. 1987. Graphical Communication of Uncertain Quantities to Nontechnical People. Risk Analysis 7, 4 (1987), 519--529.
[19]
Robert A. Jacobs. 1999. Optimal integration of texture and motion cues to depth. Vision Research 39, 21 (1999), 3621 -- 3629.
[20]
Susan L. Joslyn and Jared E. LeClerc. 2012. Uncertainty forecasts improve weather-related decisions and attenuate the effects of forecast error. Journal of experimental psychology. Applied 18, 1 (mar 2012), 126--40.
[21]
Malte F. Jung, David Sirkin, Turgut M. Gür, and Martin Steinert. 2015. Displayed Uncertainty Improves Driving Experience and Behavior: The Case of Range Anxiety in an Electric Car. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '15). 2201--2210.
[22]
Matthew Kay, Tara Kola, Jessica R. Hullman, and Sean A. Munson. 2016. When (ish) is My Bus?: User-centered Visualizations of Uncertainty in Everyday, Mobile Predictive Systems. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '16). 5092--5103.
[23]
Matthew Kay, Dan Morris, Mc Schraefel, and Julie A. Kientz. 2013. There's No Such Thing as Gaining a Pound: Reconsidering the Bathroom Scale User Interface. In Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing (UbiComp '13). 401--410.
[24]
Isaac M. Lipkus, Greg Samsa, and Barbara K. Rimer. 2001. General performance on a numeracy scale among highly educated samples. Medical decision making : an international journal of the Society for Medical Decision Making 21, 1 (2001), 37--44.
[25]
Alan M. MacEachren, Anthony Robinson, and Susan Hopper. 2005. Visualizing geospatial information uncertainty: What we know and what we need to know. Cartography and Geographic Information Science 32, 3 (jan 2005), 139--160.
[26]
Rebecca E. Morss, Julie L. Demuth, and Jeffrey K. Lazo. 2008. Communicating Uncertainty in Weather Forecasts: A Survey of the U.S. Public. Weather and Forecasting 23, 5 (oct 2008), 974--991.
[27]
Alex T. Pang, Craig M. Wittenbrink, and Suresh K. Lodha. 1997. Approaches to uncertainty visualization. The Visual Computer 13, 8 (1997), 370--390.
[28]
Kristin Potter, Joe Kniss, Richard Riesenfeld, and Chris R. Johnson. 2010. Visualizing Summary Statistics and Uncertainty. Computer Graphics Forum 29, 3 (June 2010), 823--832.
[29]
Kristin Potter, Paul Rosen, and Chris R. Johnson. 2012. From quantification to visualization: A taxonomy of uncertainty visualization approaches. In IFIP Advances in Information and Communication Technology, Vol. 377 AICT. 226--247.
[30]
Maria H. Ramos, Schalk J. Van Andel, and Florian Pappenberger. 2013. Do probabilistic forecasts lead to better decisions? Hydrology and Earth System Sciences 17, 6 (June 2013), 2219--2232.
[31]
Gordan Ristovski, Tobias Preusser, Horst K. Hahn, and Lars Linsen. 2014. Uncertainty in medical visualization: Towards a taxonomy. Computers and Graphics (Pergamon) 39, 1 (2014), 60--73.
[32]
Mark S. Roulston, Gary E. Bolton, Andrew N. Kleit, and Addison L. Sears-Collins. 2006. A Laboratory Study of the Benefits of Including Uncertainty Information in Weather Forecasts. Weather and Forecasting 21, 1 (2006), 116--122.
[33]
Mark S. Roulston and Todd R. Kaplan. 2009. A laboratory-based study of understanding of uncertainty in 5-day site-specific temperature forecasts. Meteorological Applications 16, 2 (2009), 237--244.
[34]
Enrico Rukzio, John Hamard, Chie Noda, and Alexander De Luca. 2006. Visualization of Uncertainty in Context Aware Mobile Applications. In Proceedings of the 8th Conference on Human-computer Interaction with Mobile Devices and Services (MobileHCI '06). 247--250.
[35]
Orit Shaer, Oded Nov, Lauren Westendorf, and Madeleine Ball. 2017. Communicating Personal Genomic Information to Non-experts: A New Frontier for Human-Computer Interaction. Foundations and Trends in Human-Computer Interaction 11, 1 (2017), 1--62.
[36]
Meredith Skeels, Bongshin Lee, Greg Smith, and George Robertson. 2008. Revealing uncertainty for information visualization. In Proceedings of the working conference on Advanced visual interfaces - AVI '08, Vol. 9. 376.
[37]
Susanne Tak and Alexander Toet. 2014. Color and Uncertainty: It is not always Black and White. Eurographics Conference on Visualization (2014), 55--59.
[38]
J. Taylor. 1997. Introduction to Error Analysis, the Study of Uncertainties in Physical Measurements, 2nd Edition. University Science Books.
[39]
Judi Thomson, Elizabeth Hetzler, Alan MacEachren, Mark Gahegan, and Misha Pavel. 2005. A typology for visualizing uncertainty. In IS&T/SPIE Electronic Imaging, Vol. 5669. 146.
[40]
Amos Tversky and Daniel Kahneman. 1975. Judgment under Uncertainty: Heuristics and Biases. In Utility, Probability, and Human Decision Making. Springer Netherlands, Dordrecht, 141--162.
[41]
Thomas S. Wallsten, David V. Budescu, Amnon Rapoport, Rami Zwick, and Barbara Forsyth. 1986. Measuring the Vague Meanings of Probability Terms. Journal of Experimental Psychology: General 115, 4 (1986), 348--365.
[42]
Jacob O. Wobbrock, Leah Findlater, Darren Gergle, and James J. Higgins. 2011. The Aligned Rank Transform for Nonparametric Factorial Analyses Using Only Anova Procedures. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '11). 143--146.
[43]
Torre Zuk and Sheelagh Carpendale. 2006. Theoretical analysis of uncertainty visualizations. Electronic Imaging 2006 6060, March (jan 2006), 1--14.

Cited By

View all
  • (2024)Practice with uncertainty integration improves performance on a qualitatively different task and with new visualizationsJournal of Visualization and Interaction10.54337/jovi.v1i1.7971Online publication date: 4-Sep-2024
  • (2024)In Dice We Trust: Uncertainty Displays for Maintaining Trust in Election Forecasts Over TimeProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642371(1-24)Online publication date: 11-May-2024
  • (2024)Odds and Insights: Decision Quality in Exploratory Data Analysis Under UncertaintyProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3641995(1-14)Online publication date: 11-May-2024
  • Show More Cited By

Index Terms

  1. Uncertainty Visualization Influences how Humans Aggregate Discrepant Information

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Conferences
    CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing Systems
    April 2018
    8489 pages
    ISBN:9781450356206
    DOI:10.1145/3173574
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Sponsors

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 21 April 2018

    Permissions

    Request permissions for this article.

    Check for updates

    Author Tags

    1. conflicting information
    2. information aggregation
    3. uncertainty
    4. visualization

    Qualifiers

    • Research-article

    Funding Sources

    • German Research Foundation
    • European Research Council
    • NSF

    Conference

    CHI '18
    Sponsor:

    Acceptance Rates

    CHI '18 Paper Acceptance Rate 666 of 2,590 submissions, 26%;
    Overall Acceptance Rate 6,199 of 26,314 submissions, 24%

    Upcoming Conference

    CHI 2025
    ACM CHI Conference on Human Factors in Computing Systems
    April 26 - May 1, 2025
    Yokohama , Japan

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • Downloads (Last 12 months)235
    • Downloads (Last 6 weeks)22
    Reflects downloads up to 27 Jan 2025

    Other Metrics

    Citations

    Cited By

    View all
    • (2024)Practice with uncertainty integration improves performance on a qualitatively different task and with new visualizationsJournal of Visualization and Interaction10.54337/jovi.v1i1.7971Online publication date: 4-Sep-2024
    • (2024)In Dice We Trust: Uncertainty Displays for Maintaining Trust in Election Forecasts Over TimeProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3642371(1-24)Online publication date: 11-May-2024
    • (2024)Odds and Insights: Decision Quality in Exploratory Data Analysis Under UncertaintyProceedings of the 2024 CHI Conference on Human Factors in Computing Systems10.1145/3613904.3641995(1-14)Online publication date: 11-May-2024
    • (2024)From Delays to Densities: Exploring Data Uncertainty through Speech, Text, and VisualizationComputer Graphics Forum10.1111/cgf.1510043:3Online publication date: 10-Jun-2024
    • (2023)Practice Improves Performance of a 2D Uncertainty Integration Task Within and Across VisualizationsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2022.317388929:9(3949-3960)Online publication date: 1-Sep-2023
    • (2023)Communicating Uncertainty and Risk in Air Quality MapsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2022.317144329:9(3746-3757)Online publication date: 1-Sep-2023
    • (2022)“Hartal (Strike) Happens Here Everyday”: Understanding Impact of Disruption on Education in KashmirProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3502126(1-17)Online publication date: 29-Apr-2022
    • (2022)Do You See What You Mean? Using Predictive Visualizations to Reduce Optimism in Duration EstimatesProceedings of the 2022 CHI Conference on Human Factors in Computing Systems10.1145/3491102.3502010(1-19)Online publication date: 29-Apr-2022
    • (2022)Evaluating the Use of Uncertainty Visualisations for Imputations of Data Missing At Random in ScatterplotsIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2022.3209348(1-11)Online publication date: 2022
    • (2022)Examining Effort in 1D Uncertainty Communication Using Individual Differences in Working Memory and NASA-TLXIEEE Transactions on Visualization and Computer Graphics10.1109/TVCG.2021.311480328:1(411-421)Online publication date: 1-Jan-2022
    • Show More Cited By

    View Options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Login options

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media