City Research Online

Building the ‘Truthmeter’: Training algorithms to help journalists assess the credibility of social media sources

Fletcher, R., Schifferes, S. & Thurman, N. (2017). Building the ‘Truthmeter’: Training algorithms to help journalists assess the credibility of social media sources. Convergence: The International Journal of Research into New Media Technologies, doi: 10.1177/1354856517714955


Social media is now used as an information source in many different contexts. For professional journalists, the use of social media for news production creates new challenges for the verification process. This article describes the development and evaluation of the ‘Truthmeter’ – a tool that automatically scores the journalistic credibility of social media contributors in order to inform overall credibility assessments. The Truthmeter was evaluated using a threestage process that used both qualitative and quantitative methods, consisting of (1) obtaining a ground truth, (2) building a description of existing practices, and (3) calibration, modification and testing. As a result of the evaluation process, which could be generalized and applied in other contexts, the Truthmeter produced credibility scores that were closely aligned with those of trainee journalists. Substantively, the evaluation also highlighted the importance of ‘relational’ credibility assessments, where credibility may be attributed based on networked connections to other credible contributors.

Publication Type: Article
Additional Information: Copyright, Sage 2017.
Publisher Keywords: Algorithms, computational journalism, credibility, fake news, journalistic practice, news, social media, sourcing practices, verification
Subjects: P Language and Literature > PN Literature (General)
Departments: School of Communication & Creativity > Journalism
Text - Accepted Version
Download (793kB) | Preview



Downloads per month over past year

View more statistics

Actions (login required)

Admin Login Admin Login