City Research Online

Abstract

We propose Semantic Averaging with MUltiple Languages (SAMUL) as a method to isolate concept semantics of text by averaging concept activations derived via Sparse Autoencoders (SAEs) for multiple text versions in different languages. We combine autoencoder activations to derive a conceptual average using different approaches. In our experiments with SAMUL, we start with English text from benchmark datasets (Ontology Alignment Evaluation Initiative 2025: conference track, SemEval2014: cross-level semantic similarity, Sentences Involving Compositional Knowledge: relatedness) and automatically translate into French, Chinese, German and Japanese. We obtain SAE feature activations for each class and language version using autoencoders, specifically the open source Gemma Scope 1 and OpenAI’s GPT2 suites of Sparse Autoencoders and average them to SAMUL vectors.

We measure performance in predicting semantic similarity of short texts using predefined ground truth or correlation to the ground truth mapping m between ontology classes in the case of the ontology alignment task. Our results show a consistent improvement over any single translated language, and improvement over English alone in 27 of 32 comparisons. Conceptual averaging therefore yields sparse representations that align more closely with human similarity judgements, suggesting a route to more reliable use of SAE features in semantic tasks such as ontology alignment. Our implementation and experimentation code is openly available.

Publication Type:	Conference or Workshop Item (Paper)
Additional Information:	ACL materials are Copyright © 1963–2026 ACL. This is an open-access accepted manuscript distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms.
Subjects:	P Language and Literature > P Philology. Linguistics Q Science > QA Mathematics > QA75 Electronic computers. Computer science Q Science > QA Mathematics > QA76 Computer software
Departments:	School of Science & Technology School of Science & Technology > Department of Computer Science
SWORD Depositor:	Symplectic Administrator

Export

Downloads