Abstract
New visualization systems need to be evaluated ideally with participants representing the target user group doing real tasks. Most evaluations are short, i.e. participants receive an instruction and use the system only once. These short evaluations have many drawbacks, e.g., participants do not get familiar with the system. Long-term case studies that involve the regular use of a system for at least several weeks lead to more reliable assessments of user acceptance. We discuss strategies to plan and conduct long-term case studies and present examples that demonstrate the feasibility of this evaluation strategy.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Bentley, R., Hughes, J.A., Randall, D., Rodden, T., Sawyer, P., Shapiro, D., Sommerville, I.: Ethnographically-informed systems design for air traffic control. In: Proceedings of the 1992 ACM Conference on Computer-Supported Cooperative Work, pp. 123–129 (1992)
Breakwell, G.M., Hammond, S., Fife-Scha, C.: Research Methods in Psychology. Sage Publications, Thousand Oaks (1995)
Carpendale, S.: Evaluating information visualizations. Information Visualization, pp. 19–45. Springer, Berlin (2008)
Crabtree, A., Rodden, T., Tolmie, P., Button, G.: Ethnography considered harmful. In: Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems, pp. 879–888 (2009)
Elmqvist, N., Yi, J.S.: Patterns for visualization evaluation. Inf. Vis. 14(3), 250–269 (2015)
González, V., Kobsa, A.: A workplace study of the adoption of information visualization systems. Proceedings of I-KNOW, vol. 3, pp. 92–102 (2003)
Hughes, J., King, V., Rodden, T., Andersen, H.: The role of ethnography in interactive systems design. Interactions 2(2), 56–65 (1995)
Isenberg, T., Isenberg, P., Chen, J., Sedlmair, M., Möller, T.: A systematic review on the practice of evaluating visualization. IEEE Trans. Vis. Comput. Graph. 19(12), 2818–2827 (2013)
Joshi, A., Papanastassiou, A., Vives, K., Spencer, D., Staib, L., Papademetris, X.: Light-sensitive visualization of multimodal data for neurosurgical applications. In: IEEE International Symposium on Biomedical Imaging (ISBI) (2010)
Joshi, A., Qian, X., Dione, D.P., Bulsara, K.R., Breuer, C.K., Sinusas, A.J., Papademetris, X.: Effective visualization of complex vascular structures using a non-parametric vessel detection method. IEEE Trans. Vis. Comput. Graph. (VIS 2008), 14(6), (2008)
Joshi, A., Scheinost, D., Okuda, H., Murphy, I., Staib, L.H., Papademetris, X.: Unified framework for development, deployment and testing of image analysis algorithms. In: MICCAI Workshop on Systems and Architectures for Computer Assisted Interventions (2009)
Joshi, A., Scheinost, D., Spann, M., Papademetris, X.: Evaluation of multi-viewport based visualization for electrode navigation during stereotactic image guided neurosurgery. In: International Brain Mapping and Interoperative Surgical Planning Society’s 6th World Congress for Brain Mapping and Image Guided Therapy (2009)
Joshi, A., Scheinost, D., Vives, K.P., Spencer, D.D., Staib, L.H., Papademetris, X.: Novel interaction techniques for neurosurgical planning and stereotactic navigation. IEEE Trans. Vis. Comput. Graph. (VIS 2008), 14(6), (2008)
Kang, Y.A., Gorg, C., Stasko, J.: Evaluating visual analytics systems for investigative analysis: deriving design principles from a case study. In: Proceedings of IEEE Visual Analytics Science and Technology, pp. 139–146 (2009)
Kang, Y.A., Stasko, J.: Examining the use of a visual analytics system for sensemaking tasks: case studies with domain experts. IEEE Trans. Vis. Comput. Graph. 18(12), 2869–2878 (2012)
Lam, H., Bertini, E., Isenberg, P., Plaisant, C., Carpendale, S.: Empirical studies in information visualization: seven scenarios. IEEE Trans. Vis. Comput. Graph. 18(9), 1520–1536 (2012)
Ohly, S., Sonnentag, S., Niessen, C., Zapf, D.: Diary studies in organizational research: an introduction and some practical recommendations. J. Pers. Psychol. 9, 79–93 (2010)
Perer, A., Shneiderman, B.: Integrating statistics and visualization: case studies of gaining clarity during exploratory data analysis. In: Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems, pp. 265–274 (2008)
Preim, B., Baer, A., Cunningham, D., Isenberg, T., Ropinski, T.: A survey of perceptually motivated 3d visualization of medical image data. Comput. Graph. Forum 35(3), 501–525 (2016)
Preim, B., Isenberg, P., Ropinski, T.: A critical analysis of the evaluation practice in medical visualization. In: Proceedings of the EG Workshop on Visual Computing in Biology and Medicine (2018)
Saalfeld, P., Luz, M., Berg, P., Preim, B., Saalfeld, S.: Guidelines for quantitative evaluation of medical visualizations on the example of 3d aneurysm surface comparisons. Comput. Graph. Forum 37, (2018)
Saraiya, P., North, C., Lam, V., Duca, K.A.: An insight-based longitudinal study of visual analytics. IEEE Trans. Vis. Comput. Graph. 12(6), 1511–1522 (2006)
Seo, J., Shneiderman, B.: A rank-by-feature framework for unsupervised multidimensional data exploration using low dimensional projections. In: Proceedings of IEEE Symposium on Information Visualization, pp. 65–72 (2004)
Seo, J., Shneiderman, B.: Knowledge discovery in high-dimensional data: case studies and a user survey for the rank-by-feature framework. IEEE Trans. Vis. Comput. Graph. 12(3), 311–322 (2006)
Shneiderman, B., Plaisant, C.: Strategies for evaluating information visualization tools: multi-dimensional in-depth long-term case studies. In: Proceedings of the Workshop on BEyond Time and Errors: Novel Evaluation Methods for Information Visualization (2006)
Stasko, J.T., Görg, C., Liu, Z., Singhal, K.: Jigsaw: supporting investigative analysis through interactive visualization. In: Proceedings of the IEEE Symposium on Visual Analytics Science and Technology, pp. 131–138 (2007)
Valiati, E.R., Freitas, C.M., Pimenta, M.S.: Using multi-dimensional in-depth long-term case studies for information visualization evaluation. In: Proceedings of the Workshop on BEyond Time and Errors: Novel Evaluation Methods for Information Visualization, p. 9 (2008)
Whittaker, S., Sidner, C.L.: Email overload: exploring personal information management of email. In: Proceedings of the ACM SIGCHI Conference on Human Factors in Computing Systems, pp. 276–283 (1996)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Preim, B., Joshi, A. (2020). Evaluation of Visualization Systems with Long-Term Case Studies. In: Chen, M., Hauser, H., Rheingans, P., Scheuermann, G. (eds) Foundations of Data Visualization. Springer, Cham. https://doi.org/10.1007/978-3-030-34444-3_9
Download citation
DOI: https://doi.org/10.1007/978-3-030-34444-3_9
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-34443-6
Online ISBN: 978-3-030-34444-3
eBook Packages: Computer ScienceComputer Science (R0)