City Research Online

Abstract

We present GraphUnit, a framework and online service that automates the process of designing, running and analyzing results of controlled user studies of graph visualizations by leveraging crowdsourcing and a set of evaluation modules based on a graph task taxonomy. User studies play an important role in visualization research but conducting them requires expertise and is time consuming. GraphUnit simplifies the evaluation process by allowing visualization designers to easily configure user studies for their web-based graph visualizations, deploy them online, use Mechanical Turk to attract participants, collect user responses and store them in a database, and analyze incoming results automatically using appropriate statistical tools and graphs. We demonstrate the effectiveness of GraphUnit by replicating two published evaluation studies on network visualization, and showing that these studies could be configured in less than an hour. Finally, we discuss how GraphUnit can facilitate quick evaluations of alternative graph designs and thus encourage the frequent use of user studies to evaluate design decisions in iterative development processes.

Publication Type:	Article
Additional Information:	This is the peer reviewed version of the following article: Okoe, M. & Jianu, R. (2015). GraphUnit: Evaluating Interactive Graph Visualizations Using Crowdsourcing. Computer Graphics Forum, 34(3), pp. 451-460, which has been published in final form at doi: 10.1111/cgf.12657. This article may be used for non-commercial purposes in accordance with Wiley Terms and Conditions for Self-Archiving.
Publisher Keywords:	Graph evaluation; graph user studies; automating graph evaluation; crowdsourcing graphs
Subjects:	Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Departments:	School of Science & Technology > Computer Science School of Science & Technology > Computer Science > giCentre
SWORD Depositor:	Symplectic Administrator

Export

Downloads