The Risks of Ranking: Revisiting Graphical Perception to Model Individual Differences in Visualization Performance
IEEE Transactions on Visualization and Computer Graphics
Proportions of the population predicted to have each ranking in terms of mean absolute error, listed in order from most to least common ranking. All proportions added together equal 1 (100% of the population) (Fig.9 in paper)
Abstract
Graphical perception studies typically measure visualization encoding effectiveness using the error of an “average observer”, leading to canonical rankings of encodings for numerical attributes: e.g., position > area > angle > volume. Yet different people may vary in their ability to read different visualization types, leading to variance in this ranking across individuals not captured by population-level metrics using “average observer” models. One way we can bridge this gap is by recasting classic visual perception tasks as tools for assessing individual performance, in addition to overall visualization performance. In this paper we replicate and extend Cleveland and McGill’s graphical comparison experiment using Bayesian multilevel regression, using these models to explore individual differences in visualization skill from multiple perspectives. The results from experiments and modeling indicate that some people show patterns of accuracy that credibly deviate from the canonical rankings of visualization effectiveness. We discuss implications of these findings, such as a need for new ways to communicate visualization effectiveness to designers, how patterns in individuals’ responses may show systematic biases and strategies in visualization judgment, and how recasting classic visual perception tasks as tools for assessing individual performance may offer new ways to quantify aspects of visualization literacy. Experiment data, source code, and analysis scripts are available at the following repository: https://osf.io/8ub7t/?view_only=9be4798797404a4397be3c6fc2a68cc0
Citation
BibTeX
@article{perception-individual-differences-2022,
title = {The Risks of Ranking: Revisiting Graphical Perception to Model Individual Differences in Visualization Performance},
author = {Davis, Russell and Pu, Xiaoying and Ding, Yiren and Hall, Brian D. and Bonilla, Karen and Feng, Mi and Kay, Matthew and Harrison, Lane},
year = 2022,
journal = {IEEE Transactions on Visualization and Computer Graphics},
pages = {1--16},
doi = {10.1109/TVCG.2022.3226463}
}