City Research Online

Abstract

Ensuring both syntactic and semantic correctness in Large Language Model (LLM) outputs remains a significant challenge, despite being critical for real-world deployment. In this paper, we introduce SEM-CTRL, a unified approach that allows for enforcing rich context-sensitive constraints, and task and instance specific semantics directly on the LLM decoder. Our approach integrates token-level MCTS which is guided by specific syntactic and semantic constraints. The constraints over desired outputs are expressed using Answer Set Grammars, which is a logic-based formalism that generalizes context sensitive grammars while incorporating background knowledge to represent task-specific semantics. We show that our approach helps guarantee valid completions for any off-the-shelf LLM without the need for fine-tuning. We evaluate SEM-CTRL on a range of tasks, including synthetic grammar synthesis, combinatorial reasoning, JSON parsing, and planning. Our experimental results demonstrate that SEM-CTRL allows even small pre-trained LLMs to efficiently outperform larger variants and state-of-the-art reasoning models (e.g., o4-mini) while simultaneously guaranteeing semantic validity.

Publication Type:	Article
Additional Information:	© The Authors. Published by TMLR. This is an open-access article distributed under the terms of Creative Commons: Attribution License 4.0 (http://creativecommons.org/licenses/by/4.0/).
Subjects:	H Social Sciences > HD Industries. Land use. Labor H Social Sciences > HN Social history and conditions. Social problems. Social reform Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Departments:	School of Science & Technology School of Science & Technology > Department of Computer Science
SWORD Depositor:	Symplectic Administrator

Export

Downloads