- Sponsor:
- sigchi
Information Visualization (InfoVis) has recently gained much relevance for its ability to cope with complex data analysis tasks and communication. Recent public successes like GapMinder, Many Eyes and Tableau demonstrate its impact on real world applications and the growing interest of a large community of casual and professional users.
Despite these success stories, InfoVis is still far from establishing itself as a solid science or design discipline for its lack of methods to evaluate its techniques and tools. Even though established scientific methods like controlled experiments and standard metrics like time and errors continue to be the workhorse of evaluation, there is a growing sense in the community that InfoVis systems need special approaches and metrics.
Proceeding Downloads
Developing qualitative metrics for visual analytic environments
In this paper, we examine reviews for the entries to the 2009 Visual Analytics Science and Technology (VAST) Symposium Challenge. By analyzing these reviews we gained a better understanding of what is important to our reviewers, both visualization ...
Many roads lead to Rome: mapping users' problem solving strategies
Especially in ill-defined problems like complex, real-world tasks more than one way leads to a solution. Until now, the evaluation of information visualizations was often restricted to measuring outcomes only (time and error) or insights into the data ...
Exploring information visualization: describing different interaction patterns
Interactive Information Visualization methods engage users in exploratory behavior. Detailed information about such processes can help developers to improve the design of such methods. The following study which is based on software logging describes ...
Towards information-theoretic visualization evaluation measure: a practical example for Bertin's matrices
This paper presents a discussion about matrix-based representation evaluation measures, including a review of related evaluation measures from different scientific disciplines and a proposal for promising approaches. The paper advocates linking or ...
Learning-based evaluation of visual analytic systems
Evaluation in visualization remains a difficult problem because of the unique constraints and opportunities inherent to visualization use. While many potentially useful methodologies have been proposed, there remain significant gaps in assessing the ...
A descriptive model of visual scanning
When designing a representation, a designer implicitly formulates a sequence of visual tasks required to understand and use the representation effectively. This paper aims to make the sequence of visual tasks explicit, in order to help designers ...
Generating a synthetic video dataset
A synthetic video dataset, scenario, and task were included in the 2009 VAST Challenge, to allow participants an opportunity to demonstrate visual analytic tool use on video data. This is the first time a video challenge had been presented as part of ...
Is your user hunting or gathering insights?: identifying insight drivers across domains
In recent years, using the number of insights to benchmark visual analytics tools has become a prominent method in the Infovis community. The insight methodology has become a frequently used instrument to measure the performance of tools that are ...
Comparing benchmark task and insight evaluation methods on timeseries graph visualizations
A study to compare two different empirical research methods for evaluating visualization tools is described: the traditional benchmark-task method and the insight method. The methods were compared using different criteria such as: the conclusions about ...
Do Mechanical Turks dream of square pie charts?
Online studies are an attractive alternative to the laborintensive lab study, and promise the possibility of reaching a larger variety and number of people than at a typical university. There are also a number of draw-backs, however, that have made ...
Comparing information graphics: a critical look at eye tracking
Effective graphics are essential for understanding complex information and completing tasks. To assess graphic effectiveness, eye tracking methods can help provide a deeper understanding of scanning strategies that underlie more traditional, high-level ...
Evaluating information visualization in large companies: challenges, experiences and recommendations
We examine the process and some implications of evaluating information visualization in a large company setting. While several researchers have addressed the difficulties of evaluating information visualizations with regards to changing data, tasks, and ...