skip to main content
10.1145/2110192acmconferencesBook PagePublication PageschiConference Proceedingsconference-collections
BELIV '10: Proceedings of the 3rd BELIV'10 Workshop: BEyond time and errors: novel evaLuation methods for Information Visualization
ACM2010 Proceeding
Publisher:
  • Association for Computing Machinery
  • New York
  • NY
  • United States
Conference:
CHI '10: CHI Conference on Human Factors in Computing Systems Atlanta Georgia April 10 - 11, 2010
ISBN:
978-1-4503-0007-0
Published:
10 April 2010
Sponsors:
Recommend ACM DL
ALREADY A SUBSCRIBER?SIGN IN
Next Conference
April 26 - May 1, 2025
Yokohama , Japan
Reflects downloads up to 18 Jan 2025Bibliometrics
Skip Abstract Section
Abstract

Information Visualization (InfoVis) has recently gained much relevance for its ability to cope with complex data analysis tasks and communication. Recent public successes like GapMinder, Many Eyes and Tableau demonstrate its impact on real world applications and the growing interest of a large community of casual and professional users.

Despite these success stories, InfoVis is still far from establishing itself as a solid science or design discipline for its lack of methods to evaluate its techniques and tools. Even though established scientific methods like controlled experiments and standard metrics like time and errors continue to be the workhorse of evaluation, there is a growing sense in the community that InfoVis systems need special approaches and metrics.

Skip Table Of Content Section
research-article
Developing qualitative metrics for visual analytic environments

In this paper, we examine reviews for the entries to the 2009 Visual Analytics Science and Technology (VAST) Symposium Challenge. By analyzing these reviews we gained a better understanding of what is important to our reviewers, both visualization ...

research-article
Many roads lead to Rome: mapping users' problem solving strategies

Especially in ill-defined problems like complex, real-world tasks more than one way leads to a solution. Until now, the evaluation of information visualizations was often restricted to measuring outcomes only (time and error) or insights into the data ...

research-article
Exploring information visualization: describing different interaction patterns

Interactive Information Visualization methods engage users in exploratory behavior. Detailed information about such processes can help developers to improve the design of such methods. The following study which is based on software logging describes ...

research-article
Towards information-theoretic visualization evaluation measure: a practical example for Bertin's matrices

This paper presents a discussion about matrix-based representation evaluation measures, including a review of related evaluation measures from different scientific disciplines and a proposal for promising approaches. The paper advocates linking or ...

research-article
Learning-based evaluation of visual analytic systems

Evaluation in visualization remains a difficult problem because of the unique constraints and opportunities inherent to visualization use. While many potentially useful methodologies have been proposed, there remain significant gaps in assessing the ...

research-article
A descriptive model of visual scanning

When designing a representation, a designer implicitly formulates a sequence of visual tasks required to understand and use the representation effectively. This paper aims to make the sequence of visual tasks explicit, in order to help designers ...

research-article
Generating a synthetic video dataset

A synthetic video dataset, scenario, and task were included in the 2009 VAST Challenge, to allow participants an opportunity to demonstrate visual analytic tool use on video data. This is the first time a video challenge had been presented as part of ...

research-article
Is your user hunting or gathering insights?: identifying insight drivers across domains

In recent years, using the number of insights to benchmark visual analytics tools has become a prominent method in the Infovis community. The insight methodology has become a frequently used instrument to measure the performance of tools that are ...

research-article
Comparing benchmark task and insight evaluation methods on timeseries graph visualizations

A study to compare two different empirical research methods for evaluating visualization tools is described: the traditional benchmark-task method and the insight method. The methods were compared using different criteria such as: the conclusions about ...

research-article
Do Mechanical Turks dream of square pie charts?

Online studies are an attractive alternative to the laborintensive lab study, and promise the possibility of reaching a larger variety and number of people than at a typical university. There are also a number of draw-backs, however, that have made ...

research-article
Comparing information graphics: a critical look at eye tracking

Effective graphics are essential for understanding complex information and completing tasks. To assess graphic effectiveness, eye tracking methods can help provide a deeper understanding of scanning strategies that underlie more traditional, high-level ...

research-article
Evaluating information visualization in large companies: challenges, experiences and recommendations

We examine the process and some implications of evaluating information visualization in a large company setting. While several researchers have addressed the difficulties of evaluating information visualizations with regards to changing data, tasks, and ...

Recommendations

Acceptance Rates

BELIV '10 Paper Acceptance Rate 12 of 18 submissions, 67%;
Overall Acceptance Rate 45 of 64 submissions, 70%
YearSubmittedAcceptedRate
BELIV '14302377%
BELIV '10181267%
BELIV '08161063%
Overall644570%