Why-Oriented End-User Debugging of Naïve Bayes Text Classification

Kulesza, T., Stumpf, S., Wong, W-K., Burnett, M., Perona, S., Ko, A. & Oberst, I. (2011). Why-Oriented End-User Debugging of Naïve Bayes Text Classification. Transactions on Interactive Intelligent Systems, 1(1), .2. doi: 10.1145/2030365.2030367

Text - Accepted Version
Download (1MB) | Preview


Machine learning techniques are increasingly used in intelligent assistants, that is, software targeted at and continuously adapting to assist end users with email, shopping, and other tasks. Examples include desktop SPAM filters, recommender systems, and handwriting recognition. Fixing such intelligent assistants when they learn incorrect behavior, however, has received only limited attention. To directly support end-user “debugging” of assistant behaviors learned via statistical machine learning, we present a Why-oriented approach which allows users to ask questions about how the assistant made its predictions, provides answers to these “why” questions, and allows users to interactively change these answers to debug the assistant's current and future predictions. To understand the strengths and weaknesses of this approach, we then conducted an exploratory study to investigate barriers that participants could encounter when debugging an intelligent assistant using our approach, and the information those participants requested to overcome these barriers. To help ensure the inclusiveness of our approach, we also explored how gender differences played a role in understanding barriers and information needs. We then used these results to consider opportunities for Why-oriented approaches to address user barriers and information needs.

Item Type: Article
Additional Information: © Stumpf, S | ACM 2011. This is the author's version of the work. It is posted here for your personal use. Not for redistribution. The definitive Version of Record was published in ACM Transactions on Interactive Intelligent Systems, http://dx.doi.org/10.1145/2030365.2030367
Subjects: Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Divisions: School of Engineering & Mathematical Sciences > Engineering
URI: http://openaccess.city.ac.uk/id/eprint/12414

Actions (login required)

View Item View Item


Downloads per month over past year

View more statistics