City Research Online

Abstract

In survey interviews, “Don’t know” (DK) responses are commonly treated as missing data. One way to reduce the rate of such responses is to probe initial DK answers with a follow-up question designed to encourage respondents to give substantive, non- DK responses. However, such probing can also reduce data quality by introducing additional or differential measurement error. We propose a latent variable model for analyzing the effects of probing on responses to survey questions. The model makes it possible to separate measurement effects of probing from true differences between respondents who do and do not require probing. We analyze new data from an exper- iment which compared responses to two multi-item batteries of questions with and without probing. In this study, probing reduced the rate of DK responses by around a half. However, it also had substantial measurement effects, in that probed answers were often weaker measures of constructs of interest than were unprobed answers. These effects were larger for questions on attitudes than for pseudo-knowledge ques- tions on perceptions of external facts. The results provide evidence against the use of probing of “Don’t know” responses, at least for the kinds of items and respondents considered in this study.

Publication Type:	Article
Additional Information:	This is an Accepted Manuscript of an article published by Taylor & Francis in Journal of the American Statistical Association on 26 June 2017, available online: http://www.tandfonline.com/10.1080/01621459.2017.1323640.
Publisher Keywords:	Interviewing; Latent class model; Latent variable model; Measurement error; Missing data; Questionnaire design
Subjects:	H Social Sciences > HA Statistics H Social Sciences > HM Sociology
Departments:	School of Policy & Global Affairs > Sociology & Criminology
SWORD Depositor:	Symplectic Administrator

Export

Downloads