Building the ‘Truthmeter’: Training algorithms to help journalists assess the credibility of social media sources

Fletcher, R., Schifferes, S. & Thurman, N. (2017). Building the ‘Truthmeter’: Training algorithms to help journalists assess the credibility of social media sources. Convergence: The International Journal of Research into New Media Technologies, doi: 10.1177/1354856517714955

[img]
Preview
Text - Accepted Version
Download (793kB) | Preview

Abstract

Social media is now used as an information source in many different contexts. For professional journalists, the use of social media for news production creates new challenges for the verification process. This article describes the development and evaluation of the ‘Truthmeter’ – a tool that automatically scores the journalistic credibility of social media contributors in order to inform overall credibility assessments. The Truthmeter was evaluated using a threestage process that used both qualitative and quantitative methods, consisting of (1) obtaining a ground truth, (2) building a description of existing practices, and (3) calibration, modification and testing. As a result of the evaluation process, which could be generalized and applied in other contexts, the Truthmeter produced credibility scores that were closely aligned with those of trainee journalists. Substantively, the evaluation also highlighted the importance of ‘relational’ credibility assessments, where credibility may be attributed based on networked connections to other credible contributors.

Item Type: Article
Additional Information: Copyright, Sage 2017.
Uncontrolled Keywords: Algorithms, computational journalism, credibility, fake news, journalistic practice, news, social media, sourcing practices, verification
Subjects: P Language and Literature > PN Literature (General)
Divisions: School of Arts > Department of Journalism
URI: http://openaccess.city.ac.uk/id/eprint/17446

Actions (login required)

View Item View Item

Downloads

Downloads per month over past year

View more statistics