City Research Online

Crowd-assessing quality in uncertain data linking datasets

Faria, D., Ferrara, A., Jimenez-Ruiz, E. ORCID: 0000-0002-9083-4599, Montanelli, S. and Pesquita, C. (2020). Crowd-assessing quality in uncertain data linking datasets. The Knowledge Engineering Review, 35, e33. doi: 10.1017/s0269888920000363

Abstract

The quality of a dataset used for evaluating data linking methods, techniques, and tools depends on the availability of a set of mappings, called reference alignment, that is known to be correct. In particular, it is crucial that mappings effectively represent relations between pairs of entities that are indeed similar due to the fact that they denote the same object. Since the reliability of mappings is decisive in order to perform a fair evaluation of automatic linking methods and tools, we call this property of mappings as mapping fairness. In this article, we propose a crowd-based approach, called Crowd Quality(CQ), for assessing the quality of data linking datasets by measuring the fairness of the mappings in the reference alignment. Moreover, we present a real experiment, where we evaluate two state-of-the-art data linking tools before and after the refinement of the reference alignment based on the CQ approach, in order to present the benefits deriving from the crowd assessment of mapping fairness.

Publication Type: Article
Additional Information: This article has been published in a revised form in The Knowledge Engineering Review, https://doi.org/10.1017/S0269888920000363. This version is free to view and download for private research and study only. Not for re-distribution or re-use© The Author(s), 2020. Published by Cambridge University Press
Subjects: Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Departments: School of Mathematics, Computer Science & Engineering > Computer Science
Date Deposited: 08 Jul 2020 09:52
URI: https://openaccess.city.ac.uk/id/eprint/24475
[img] Text - Accepted Version
This document is not freely accessible until 2 January 2021 due to copyright restrictions.
Available under License Creative Commons Attribution Non-commercial No Derivatives.

To request a copy, please use the button below.

Request a copy

Export

Downloads

Downloads per month over past year

View more statistics

Actions (login required)

Admin Login Admin Login