City Research Online

Abstract

This paper describes the CitySAT system that we developed for the DBpedia Answer Type (AT) prediction task of the SMART 2021 challenge. The challenge can be interpreted as a multi-class classification task that takes natural language questions and returns pairs of the predicted answer category and types. For training, we merged the SMART 2021 DBpedia dataset with the 2020 dataset given for the previous year's AT task. In this study, three local Machine Learning (ML) models are deployed to cover the three different types of task and question (category prediction, literal type prediction and resource type prediction). The best model obtains a 98.36% accuracy for the category prediction using a Logistic Regression (LR) classifier. Similarly, another LR model results in 97.90% accuracy for the literal type prediction task. Lastly we also built a Multi-Layer Perceptron (MLP) model to deal with several ontology classes (∼760 classes for DBpedia) in the resource type prediction task. The best MLP model achieves 79.34% on the merged training dataset. The final system output obtained a 98.4% accuracy, 84.2% NDCG@5, and 85.4% NDCG@10 on the (official) test dataset.

Publication Type:	Conference or Workshop Item (Paper)
Additional Information:	Copyright ©2021 for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
Publisher Keywords:	Semantic answer type prediction, SMART DBpedia challenge, multi-class classification
Subjects:	Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Departments:	School of Science & Technology > Computer Science

Export

Downloads