City Research Online

Some factors affecting relevance judgements

Pope, J. M. (1976). Some factors affecting relevance judgements. (Unpublished Doctoral thesis, The City University)

Abstract

Relevance judgements were obtained from 31 research workers in Universities and Government Institutions, on titles and abstracts. The titles were obtained using SDI profiles for each user. A questionnaire was given to each user to discover their information gathering habits, attitudes towards information and various demographic details.

Signal detection theory was used to obtain a single measure of a user's sensitivity to relevance and the strictness of his criterion of relevance. Four different detection models were examined. It was found that an exponential model was most suitable for analysing relevance data.

Several variables were tested to see whether they affected relevance judgements. Several statistical tests were used to test the effects of the variables on the conditional probabilities of hits and false alarms and also on the sensitivity and criterion measures. It was found that 15 of the 37 variables examined significantly influenced relevance judgements. These variables related to the user's work and to his use of different information sources,

The distributions of the hit and false alarm probabilities were lognormal in all except one case. The variables examined were highly inter-related. Factor analysis and principal component analysis were used to reduce the complexity of these variables and to discover the underlying factors. Ten factors were extracted from a total of 36 variables.

The research students in the sample formed a distinct group in terms of their information gathering habits and attitudes towards information.

An attempt was made to understand the types of relevance judgements made by users by placing each judgement in one of five categories. This arbitrary classification was helpful in examining the reasons why various relevance
decisions were made.

A model of information seeking and acquisition was proposed which takes into account the storage of information in human memory and the way in which information is organized. This model was used to describe the relationships between information need, question asking and the subsequent relevance judgements.

Publication Type: Thesis (Doctoral)
Subjects: B Philosophy. Psychology. Religion
Z Bibliography. Library Science. Information Resources
Z Bibliography. Library Science. Information Resources > Z665 Library Science. Information Science
Departments: School of Science & Technology > School of Science & Technology Doctoral Theses
Doctoral Theses
[thumbnail of Pope_thesis_1976_Redacted PDF-A_Redacted.pdf]
Preview
Text - Accepted Version
Download (36MB) | Preview

Export

Add to AnyAdd to TwitterAdd to FacebookAdd to LinkedinAdd to PinterestAdd to Email

Downloads

Downloads per month over past year

View more statistics

Actions (login required)

Admin Login Admin Login