City Research Online

Crowd-assessing quality in uncertain data linking datasets

Faria, D., Ferrara, A., Jimenez-Ruiz, E. ORCID: 0000-0002-9083-4599 , Montanelli, S. & Pesquita, C. (2020). Crowd-assessing quality in uncertain data linking datasets. The Knowledge Engineering Review, 35, e33. doi: 10.1017/s0269888920000363


The quality of a dataset used for evaluating data linking methods, techniques, and tools depends on the availability of a set of mappings, called reference alignment, that is known to be correct. In particular, it is crucial that mappings effectively represent relations between pairs of entities that are indeed similar due to the fact that they denote the same object. Since the reliability of mappings is decisive in order to perform a fair evaluation of automatic linking methods and tools, we call this property of mappings as mapping fairness. In this article, we propose a crowd-based approach, called Crowd Quality(CQ), for assessing the quality of data linking datasets by measuring the fairness of the mappings in the reference alignment. Moreover, we present a real experiment, where we evaluate two state-of-the-art data linking tools before and after the refinement of the reference alignment based on the CQ approach, in order to present the benefits deriving from the crowd assessment of mapping fairness.

Publication Type: Article
Additional Information: This article has been published in a revised form in The Knowledge Engineering Review, This version is free to view and download for private research and study only. Not for re-distribution or re-useĀ© The Author(s), 2020. Published by Cambridge University Press
Subjects: Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Departments: School of Science & Technology > Computer Science
[thumbnail of KER2020_Crowd-assessing_Quality_Uncertain_Data_Linking_Datasets.pdf]
Text - Accepted Version
Available under License Creative Commons Attribution Non-commercial No Derivatives.

Download (828kB) | Preview


Add to AnyAdd to TwitterAdd to FacebookAdd to LinkedinAdd to PinterestAdd to Email


Downloads per month over past year

View more statistics

Actions (login required)

Admin Login Admin Login