Abstract
In our information-driven society, there is increasing use of statistical graphics to convey information in a variety of settings, including industry, mass media, government operations, and health care. Current methods for assessing a reader’s ability to comprehend statistical graphics are custom-written, not widely accepted, usable only once, and/or reliant on subjective interpretations and inferences. We have developed a method for generating queries suitable for evaluating graph comprehension capability. Our method is based on the Sentence Verification Technique (SVT), an empirically validated framework for measuring an individual’s comprehension of prose material. Compared to ad hoc methods for testing graph comprehension, our technique is less subjective, requires less manual effort and subject matter expertise, and addresses the essential features of a given graph: values and relationships depicted, frames of reference, and style attributes. The SVT, and therefore our method, combat superficial comprehension by testing what the reader has encoded, as opposed to testing the reader’s ability at visual recall or ability to look up data without reaching real comprehension. We motivate and describe our query generation method and report on a pilot study using queries generated with it.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Roth, W.-M.: Reading graphs: contributions to an integrative concept of literacy. J. Curric. Stud. 34(1), 1–24 (2002)
Galesic, M., Garcia-Retamero, R.: Graph literacy: a cross-cultural comparison. Med. Decis. Mak. 31(3), 444–457 (2011)
Börner, K., Maltese, A., Balliet, R.N., Heimlich, J.: Investigating aspects of data visualization literacy using 20 information visualizations and 273 science museum visitors. Inf. Vis. 15(3), 193–213 (2016)
Kintsch, W.: Comprehension: A Paradigm for Cognition. Cambridge University Press (1998)
Royer, J.M., Cunningham, D.J.: On the theory and measurement of reading comprehension. Technical report no. 91, University of Illinois at Urbana-Champaign (1978)
Wainer, H.: A test of graphicacy in children. Appl. Psychol. Meas. 4(3), 331–340 (1980)
Royer, J.M., Hastings, C.N., Hook, C.: A sentence verification technique for measuring reading comprehension. J. Read. Behav. 11(4), 355–363 (1979)
Bertin, J.: Sémiologie Graphique, 2nd edn., Gauthier-Villars (1973). English translation: Berg, W.J.: Semiology of Graphics. University of Wisconsin Press (1983)
McKenzie, D.L., Padilla, M.J.: The construction and validation of the Test of Graphing in Science (TOGS). J. Res. Sci. Teach. 23(7), 571–579 (1986)
Curcio, F.R.: Comprehension of mathematical relationships expressed in graphs. J. Res. Math. Educ. 18(5), 382–393 (1987)
Svec, M.: Improving graphing interpretation skills and understanding of motion using microcomputer based laboratories. Electron. J. Sci. Educ. 3(4) (1999)
Lai, K., Cabrera, J., Vitale, J.M., Madhok, J., Thinker, R., Linn, M.C.: Measuring graph comprehension, critique, and construction in science. J. Sci. Educ. Technol. 25(4), 665–681 (2016)
Lee, S., Kim, S.-H., Kwon, B.C.: VLAT: development of a visualization literacy assessment test. IEEE Trans. Vis. Comput. Graph. 23(1), 551–560 (2017)
Van Dijk, T.A., Kintsch, W.: Strategies of Discourse Comprehension. Academic Press (1983)
Boy, J., Rensink, R.A., Bertini, E., Fekete, J.-D.: A principled way of assessing visualization literacy. IEEE Trans. Vis. Comput. Graph. 20(12), 1963–1972 (2014)
Baker, F.B.: The Basics of Item Response Theory, 2nd ed. ERIC Clearinghouse on Assessment and Evaluation (2001)
Livingston, M.A., Brock, D., Maney, T., Perzanowski, D.: Extending the sentence verification technique to tables and node-link diagrams. In: Proceedings of Applied Human Factors and Ergonomics (2018)
Kosslyn, S.M.: Graph Design for the Eye and Mind. Oxford University Press (2006)
Shah, P., Freedman, E.G.: Bar and line graph comprehension: an interaction of top-down and bottom-up processes. Top. Cogn. Sci. 3(3), 560–578 (2011)
Judd, T., Ehinger, K., Durand, F., Torralba, A.: Learning to predict where humans look. In: IEEE International Conference on Computer Vision, pp. 2106–2113 (2009)
Harrison, A., Livingston, M.A., Brock, D., Decker, J., Perzanowski, D., Van Dolson, C., Mathews, J. Lulushi, A., Raglin, A.: The analysis and prediction of eye gaze when viewing statistical graphs. In: Proceedings of Augmented Cognition. Neurocognition and Machine Learning. LNCS, vol. 10284, pp. 148–165. Springer (2017)
Matzen, L.E., Haass, M.J., Divis, K.M., Stites, M.C.: Patterns of attention: how data visualizations are read. In: Proceedings of Augmented Cognition. Neurocognition and Machine Learning. LNCS, vol. 10284, pp. 176–191. Springer (2017)
Acknowledgements
The authors wish to thank Mike Royer, Joseph Coyne, Priti Shah, Michael Svec, and the pilot study volunteers. This research was supported by the Naval Research Laboratory Base Program.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 This is a U.S. government work and not under copyright protection in the U.S.; foreign copyright protection may apply
About this paper
Cite this paper
Livingston, M.A. et al. (2020). A Query Generation Technique for Measuring Comprehension of Statistical Graphics. In: Karwowski, W., Ahram, T., Nazir, S. (eds) Advances in Human Factors in Training, Education, and Learning Sciences. AHFE 2019. Advances in Intelligent Systems and Computing, vol 963. Springer, Cham. https://doi.org/10.1007/978-3-030-20135-7_1
Download citation
DOI: https://doi.org/10.1007/978-3-030-20135-7_1
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-20134-0
Online ISBN: 978-3-030-20135-7
eBook Packages: EngineeringEngineering (R0)