City Research Online

Abstract

Purpose – An issue which tends to be ignored in information retrieval is the issue of updating inverted files. This is largely because inverted files were devised to provide fast query service, and much work has been done with the emphasis strongly on queries. In this paper we study the effect of using parallel methods for the update of inverted files in order to reduce costs, by looking at two types of partitioning for inverted files: document identifier and term identifier.

Design/methodology/approach – Raw update service and update with query service are studied with these partitioning schemes using an incremental update strategy. We use standard measures used in parallel computing such as speedup to examine the computing results and also the costs of reorganising indexes while servicing transactions.

Findings – Empirical results show that for both transaction processing and index reorganisation the document identifier method is superior. However, there is evidence that the term identifier partitioning method could be useful in a concurrent transaction processing context.

Practical implications – There is an increasing need to service updates which is now becoming a requirement of inverted files (for dynamic collections such as the Web), demonstrating that a shift in requirements of inverted file maintenance is needed from the past.

Originality/value – The paper is of value to database administrators who manage large-scale and dynamic text collections, and who need to use parallel computing to implement their text retrieval services.

Publication Type:	Article
Additional Information:	This article is (c) Emerald Group Publishing and permission has been granted for this version to appear here http://openaccess.city.ac.uk/. Emerald does not grant permission for this article to be further copied/distributed or hosted elsewhere without the express permission from Emerald Group Publishing Limited.
Publisher Keywords:	Information retrieval, Parallel programming, Query languages, Databases
Subjects:	Z Bibliography. Library Science. Information Resources > Z665 Library Science. Information Science
Departments:	School of Science & Technology > Computer Science > Human Computer Interaction Design
Related URLs:	Author
SWORD Depositor:	Symplectic Administrator

Export

Downloads