City Research Online

Language Model Analysis for Ontology Subsumption Inference

He, Y., Chen, J., Jimenez-Ruiz, E. ORCID: 0000-0002-9083-4599 , Dong, H. & Horrocks, I. (2023). Language Model Analysis for Ontology Subsumption Inference. In: Proceedings of the Annual Meeting of the Association for Computational Linguistics. 61st Annual Meeting of the Association for Computational Linguistics, 9-14 Jul 2023, Toronto, Canada.


Investigating whether pre-trained language models (LMs) can function as knowledge bases (KBs) has raised wide research interests recently. However, existing works focus on simple, triple-based, relational KBs, but omit more sophisticated, logic-based, conceptualised KBs such as OWL ontologies. To investigate an LM's knowledge of ontologies, we propose ONTOLAMA, a set of inference-based probing tasks and datasets from ontology subsumption axioms involving both atomic and complex concepts. We conduct extensive experiments on ontologies of different domains and scales, and our results demonstrate that LMs encode relatively less background knowledge of Subsumption Inference (SI) than traditional Natural Language Inference (NLI) but can improve on SI significantly when a small number of samples are given. We will open-source our code and datasets.

Publication Type: Conference or Workshop Item (Paper)
Additional Information: © 1963–2023 ACL; other materials are copyrighted by their respective copyright holders. Materials prior to 2016 here are licensed under the Creative Commons Attribution-NonCommercial-ShareAlike 3.0 International License. Permission is granted to make copies for the purposes of teaching and research. Materials published in or after 2016 are licensed on a Creative Commons Attribution 4.0 International License.
Subjects: Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Departments: School of Science & Technology > Computer Science
[thumbnail of OntoLAMA_2023.findings-acl.213.pdf]
Text - Published Version
Available under License Creative Commons: Attribution International Public License 4.0.

Download (720kB) | Preview


Add to AnyAdd to TwitterAdd to FacebookAdd to LinkedinAdd to PinterestAdd to Email


Downloads per month over past year

View more statistics

Actions (login required)

Admin Login Admin Login