skip to main content
10.1145/3702038.3702091acmotherconferencesArticle/Chapter ViewAbstractPublication PagesihcConference Proceedingsconference-collections
research-article

Assessing Data Visualization Literacy: Design Implementation and Analysis of a Comprehensive Test

Published: 18 December 2024 Publication History

Abstract

Data Visualization Literacy involves recognizing a given chart, reading it correctly, and extracting information from it. Familiarity with a specific type of visualization does not imply that the person can read or interpret it correctly. Data visualization researchers have attempted to explore and promote solutions to support data visualization comprehension activities. Studies address developing and applying tests to assess literacy, understanding how analysts interpret visualizations and how unfamiliar visualizations are taught, and even identifying the cognitive activities involved in creating visualizations. However, we have not found comprehensive studies relating to these different aspects, which we believe are essential for teaching and learning about data visualizations and data analysis tasks. This paper presents our procedure for designing a new visualization literacy assessment. We devised a test that covered 15 different visualizations and applied it with 68 participants. We analyzed each item, assessing their difficulty level and discrimination. After removing eight items that did not discriminate well, our final test ended with 37 items, all with fair or good discrimination. It means that the test should genuinely represent the test takers’ learning ability, i.e., the items can discriminate well between the high and low-performing groups.

References

[1]
Basak Alper, Nathalie Henry Riche, Fanny Chevalier, Jeremy Boy, and Metin Sezgin. 2017. Visualization Literacy at Elementary School. In Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, Denver, Colorado, USA, 5485–5497.
[2]
Fearn Bishop, Johannes Zagermann, Ulrike Pfeil, Gemma Sanderson, Harald Reiterer, and Uta Hinrichs. 2020. Construct-A-Vis: exploring the free-form visualization processes of children. IEEE Transactions on Visualization and Computer Graphics 26, 1 (Jan 2020), 451–460.
[3]
Michelle A Borkin, Zoya Bylinskii, Nam Wook Kim, Constance May Bainbridge, Chelsea S Yeh, Daniel Borkin, Hanspeter Pfister, and Aude Oliva. 2015. Beyond memorability: Visualization recognition and recall. IEEE transactions on visualization and computer graphics 22, 1 (2015), 519–528.
[4]
Jeremy Boy, Ronald A. Rensink, Enrico Bertini, and Jean-Daniel Fekete. 2014. A Principled Way of Assessing Visualization Literacy. IEEE Transactions on Visualization and Computer Graphics 20, 12 (Dec 2014), 1963–1972.
[5]
Sabrina Bresciani. 2009. The risks of visualization: A classification of disadvantages associated with graphic representations of information. In Identität und Vielfalt der Kommunikations-wissenschaft (2009). UVK Verlagsgesellschaft GmbH, 165–178.
[6]
Katy Börner, Andreas Bueckle, and Michael Ginda. 2019. Data visualization literacy: Definitions, conceptual frameworks, exercises, and assessments. Proceedings of the National Academy of Sciences 116, 6 (2019), 1857–1864.
[7]
Katy Börner, Adam Maltese, Russell Nelson Balliet, and Joe Heimlich. 2016. Investigating aspects of data visualization literacy using 20 information visualizations and 273 science museum visitors. Information Visualization 15, 3 (2016), 198–213.
[8]
Fanny Chevalier, Nathalie Henry Riche, Basak Alper, Catherine Plaisant, Jeremy Boy, and Niklas Elmqvist. 2018. Observations and Reflections on Visualization Literacy in Elementary School. IEEE Computer Graphics and Applications 38, 3 (May 2018), 21–29.
[9]
Lee J. Cronbach. 1951. Coefficient alpha and the internal structure of tests. Psychometrika 16, 3 (Sept. 1951), 297–334.
[10]
Cynthya Letícia Teles de Oliveira, Alan Trindade de Almeida Silva, Jefferson Magalhães de Morais, and Marcelle Pereira Mota. 2020. ChartVision: accessible vertical bar charts. In Proceedings of the 19th Brazilian Symposium on Human Factors in Computing Systems (Diamantina, Brazil) (IHC ’20). Association for Computing Machinery, New York, NY, USA, Article 9, 10 pages.
[11]
E Firat, Alena Denisova, and R Laramee. 2020. Treemap literacy: A classroom-based investigation. In Eurographics Proceedings. The Eurographics Association, 29–38.
[12]
Elif E Firat, Alena Denisova, Max L Wilson, and Robert S Laramee. 2022. P-lite: A study of parallel coordinate plot literacy. Visual Informatics 6, 3 (2022), 81–99.
[13]
Gustavo Romão Gonzales and Flávio Horita. 2020. Supporting visual analytics in decision support system: a systematic mapping study. In Proceedings of the 19th Brazilian Symposium on Human Factors in Computing Systems (Diamantina, Brazil) (IHC ’20). Association for Computing Machinery, New York, NY, USA, Article 34, 10 pages.
[14]
S. Huron, Y. Jansen, and S. Carpendale. 2014. Constructing visual representations: investigating the use of tangible tokens. IEEE Transactions on Visualization and Computer Graphics 20, 12 (Dec. 2014), 2102–2111.
[15]
Daniel A. Keim, Florian Mansmann, Daniela Oelke, and Hartmut Ziegler. 2008. Visual analytics: combining automated discovery with interactive visualizations. In Proceedings of the 11th International Conference on Discovery Science (Budapest, Hungary). Springer-Verlag, 2–14.
[16]
Alicia Key, Bill Howe, Daniel Perry, and Cecilia Aragon. 2012. VizDeck: self-organizing dashboards for visual analytics. In Proceedings of the 2012 ACM SIGMOD International Conference on Management of Data (Scottsdale, Arizona, USA). Association for Computing Machinery, 681–684.
[17]
Andy Kirk. 2016. Data Visualisation: A Handbook for Data Driven Design. Sage Publications Ltd.
[18]
Neesha Kodagoda, B.L. William Wong, Chris Rooney, and Nawaz Khan. 2012. Interactive Visualization for Low Literacy Users: From Lessons Learnt to Design. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, 1159–1168.
[19]
Kenneth R Koedinger, RS Baker, and AT Corbett. 2001. Toward a model of learning data representations. In Proceedings of the Twenty-Third Annual Conference of the Cognitive Science Society. Scholarship University of California, 45–50.
[20]
Andrey Krekhov, Michael Michalski, and Jens Krüger. 2019. Integrating Visualization Literacy into Computer Graphics Education Using the Example of Dear Data. In Eurographics 2019 - Education Papers. The Eurographics Association, 1–8.
[21]
S. Lee, S. Kim, and B. C. Kwon. 2017. VLAT: Development of a visualization literacy assessment test. IEEE Transactions on Visualization and Computer Graphics 23, 1 (Jan. 2017), 551–560.
[22]
Sukwon Lee, Bum Chul Kwon, Jiming Yang, Byung Cheol Lee, and Sung-Hee Kim. 2019. The correlation between users’ cognitive characteristics and visualization literacy. Applied Sciences 9, 3 (2019), 488.
[23]
Franklin M. da C. Lima, Leonardo Cunha de Miranda, and M. Cecília C. Baranauskas. 2022. Visualizing and analyzing the evolution of authorship and co-authorship networks of articles from the Brazilian symposium on human factors in computing systems. In Proceedings of the 21st Brazilian Symposium on Human Factors in Computing Systems (Diamantina, Brazil) (IHC ’22). Association for Computing Machinery, New York, NY, USA, Article 31, 11 pages.
[24]
Maylon Pires Macedo, Ranilson Oscar Araújo Paiva, Isabela Gasparini, and Luciana Aparecida Martinez Zaina. 2020. Vis2Learning: a scenario-based guide of recommendations for building educational data visualizations. In Proceedings of the 19th Brazilian Symposium on Human Factors in Computing Systems (Diamantina, Brazil) (IHC ’20). Association for Computing Machinery, New York, NY, USA, Article 36, 10 pages.
[25]
Adam V. Maltese, Joseph A. Harsh, and Dubravka Svetina. 2015. Data Visualization Literacy: Investigating Data Interpretation Along the Novice—Expert Continuum. Journal of College Science Teaching 45, 1 (2015), 84–90. http://www.jstor.org/stable/43631889
[26]
Tamara Munzner. 2014. Visualization analysis and design. CRC press.
[27]
Saugat Pandey and Alvitta Ottley. 2023. Mini-VLAT: A Short and Effective Measure of Visualization Literacy. Computer Graphics Forum 42, 3 (2023), 1–11. arXiv:https://onlinelibrary.wiley.com/doi/pdf/10.1111/cgf.14809
[28]
Patrícia Martinková and Adéla Drabinová. 2018. ShinyItemAnalysis for teaching psychometrics and to enforce routine analysis of educational tests. The R Journal 10, 2 (2018), 503–515.
[29]
Ariane Moraes Bueno Rodrigues, Gabriel Diniz Junqueira Barbosa, Hélio Côrtes Vieira Lopes, and Simone Diniz Junqueira Barbosa. 2021. What questions reveal about novices’ attempts to make sense of data visualizations: Patterns and misconceptions. Computers & Graphics 94 (2021), 32–42.
[30]
Ariane Moraes Bueno Rodrigues, Gabriel Diniz Junqueira Barbosa, Hélio Côrtes Vieira Lopes, and Simone Diniz Junqueira Barbosa. 2022. Investigating whether people identify how suitable a data visualization is for answering specific analysis questions. In Proceedings of the 21st Brazilian Symposium on Human Factors in Computing Systems (Diamantina, Brazil) (IHC ’22). Association for Computing Machinery, New York, NY, USA, Article 22, 11 pages.
[31]
Puripant Ruchikachorn and Klaus Mueller. 2015. Learning visualizations by analogy: Promoting visual literacy through visualization morphing. IEEE Transactions on Visualization and Computer Graphics 21, 9 (2015), 1028–1044.
[32]
Bahador Saket, Arjun Srinivasan, Eric D. Ragan, and Alex Endert. 2018. Evaluating Interactive Graphical Encodings for Data Visualization. IEEE Transactions on Visualization and Computer Graphics 24, 3 (March 2018), 1316–1330. Conference Name: IEEE Transactions on Visualization and Computer Graphics.
[33]
John R Thompson, Jesse J Martinez, Alper Sarikaya, Edward Cutrell, and Bongshin Lee. 2023. Chart Reader: Accessible Visualization Experiences Designed with Screen Reader Users. In Proceedings of the 2023 CHI Conference on Human Factors in Computing Systems (, Hamburg, Germany,) (CHI ’23). Association for Computing Machinery, New York, NY, USA, Article 802, 18 pages.
[34]
Robert M Thorndike, George K Cunningham, Robert Ladd Thorndike, and Elizabeth P Hagen. 1991. Measurement and evaluation in psychology and education. Macmillan Publishing Co, Inc, 175 5th Avenue New York, NY 10010 United States.
[35]
Edward R Tufte and Peter R Graves-Morris. 1986. The visual display of quantitative information. Information Design Journal 4, 3 (Jan. 1986), 235–236.
[36]
José M. F. Vieira and Luciana A. M. Zaina. 2020. Representation, navigation and exploration: a three layered approach on learning trajectories. In Proceedings of the 19th Brazilian Symposium on Human Factors in Computing Systems (Diamantina, Brazil) (IHC ’20). Association for Computing Machinery, New York, NY, USA, Article 31, 10 pages.
[37]
Zezhong Wang, Lovisa Sundin, Dave Murray-Rust, and Benjamin Bach. 2020. Cheat Sheets for Data Visualization Techniques. In Proceedings of the 2020 CHI Conference on Human Factors in Computing Systems. Association for Computing Machinery, 1–13.
[38]
Colin Ware. 2019. Information visualization: perception for design. Morgan Kaufmann.

Index Terms

  1. Assessing Data Visualization Literacy: Design Implementation and Analysis of a Comprehensive Test

    Recommendations

    Comments

    Information & Contributors

    Information

    Published In

    cover image ACM Other conferences
    IHC '24: Proceedings of the XXIII Brazilian Symposium on Human Factors in Computing Systems
    October 2024
    1070 pages
    ISBN:9798400712241
    DOI:10.1145/3702038
    Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

    Publisher

    Association for Computing Machinery

    New York, NY, United States

    Publication History

    Published: 18 December 2024

    Check for updates

    Author Tags

    1. Visualization Literacy
    2. Assessment Test

    Qualifiers

    • Research-article

    Conference

    IHC 2024

    Acceptance Rates

    Overall Acceptance Rate 331 of 973 submissions, 34%

    Contributors

    Other Metrics

    Bibliometrics & Citations

    Bibliometrics

    Article Metrics

    • 0
      Total Citations
    • 48
      Total Downloads
    • Downloads (Last 12 months)48
    • Downloads (Last 6 weeks)36
    Reflects downloads up to 15 Feb 2025

    Other Metrics

    Citations

    View Options

    Login options

    View options

    PDF

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader

    Full Text

    View this article in Full Text.

    Full Text

    Figures

    Tables

    Media

    Share

    Share

    Share this Publication link

    Share on social media