City Research Online

Abstract

Our knowledge about audience perceptions of manually authored news articles and automated news articles is limited. Although over a dozen studies have been carried out, findings are inconsistent and limited by methodological shortcomings. For example, the experimental stimuli used in some have made isolation of the effects of the actual authorship (automated or manual) difficult. Our study attempts to overcome previous studies’ shortcomings to better evaluate audiences’ relative evaluations of news articles produced with varying degrees of automation—and none. We conducted a 3 (article source: manually written, automated, post-edited) × 12 (story topics) between-subjects online survey experiment using a sample (N = 4,734) representative of UK online news consumers by age and gender. Each of the 36 treatment groups read a data-driven news article that was either: (1) manually written by a journalist, (2) automated using a data-driven template, or (3) automated then subsequently post-edited by a journalist. The articles’ authorship was not declared. To minimise confounding variables, the articles in each of the 12 story sets shared the same data source, story angle, and geographical focus. Respondents’ perceptions were measured using criteria developed in a qualitative group interview study with news consumers. The results show that respondents found manually written articles to be significantly more comprehensible—both overall and in relation to the numbers they contained—than automated and post-edited articles. Authorship did not have any statistically significant effect on overall liking of the articles, or on the positive or negative feelings (valence) articles provoked in respondents, or the strength of those feelings (arousal).

Publication Type:	Conference or Workshop Item (Paper)
Additional Information:	Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s). © 2023 Copyright held by the author(s)
Subjects:	B Philosophy. Psychology. Religion > BF Psychology H Social Sciences > HM Sociology H Social Sciences > HN Social history and conditions. Social problems. Social reform P Language and Literature > PN Literature (General)
Departments:	School of Communication & Creativity > Journalism

Export

Downloads