City Research Online

Abstract

Large language models (LLMs) are viable oracles for use during interactive ontology matching, but their adoption
raises three practical questions: (i) Which model do we select? (ii) How should the prompt be structured? (iii)
What should the prompt contain? This work responds to the first question by benchmarking a large selection of
language models via LogMap’s interactive matching mode on OAEI tasks, recommending models that balance performance, cost, latency, reliability, openness, and provider diversity; we publish the benchmark as a community resource via an online results explorer. For the second and third, we build on prior work, extending existing prompt templates for class, property, and instance matching, investigate zero-shot versus RAG-based few-shot ontology-driven prompting, and introduce collective anchors for few-shot-RAG ontology-driven prompting. We apply these methods to an existing system design, LogMapLLM, and evaluate over five OAEI tracks, spanning a variety of domains. We find that no single configuration consistently wins out, but the oracle’s failure mode helps explain which interventions work: demonstrations recover sensitivity in conservative oracles, whereas mutual subsumption improves specificity in permissive ones. Ultimately, the optimal configuration is model-, prompt-, and domain-specific, motivating ongoing work on automatic selection, where the most preferable model × prompt configuration is recommended given a matching task.

Publication Type:	Conference or Workshop Item (Paper)
Additional Information:	© 2026 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
Publisher Keywords:	ontology matching, large language models, interactive matching, ontology-driven prompts, OAEI
Subjects:	Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Departments:	School of Science & Technology School of Science & Technology > Department of Computer Science
SWORD Depositor:	Symplectic Administrator

Export

Downloads