City Research Online

Belief Revision based Caption Re-ranker with Visual Semantic Information

Sabir, A., Moreno-Noguer, F., Madhyastha, P. ORCID: 0000-0002-4438-8161 & Padró, L. (2022). Belief Revision based Caption Re-ranker with Visual Semantic Information. In: Proceedings of the 29th International Conference on Computational Linguistics. 29th International Conference on Computational Linguistics, Gyeongju, Republic of Korea, 12-17 Oct 2022.


In this work, we focus on improving the captions generated by image-caption generation systems. We propose a novel re-ranking approach that leverages visual-semantic measures to identify the ideal caption that maximally captures the visual information in the image. Our re-ranker utilizes the Belief Revision framework (Blok et al., 2003) to calibrate the original likelihood of the top-n captions by explicitly exploiting the semantic relatedness between the depicted caption and the visual context. Our experiments demonstrate the utility of our approach, where we observe that our re-ranker can enhance the performance of a typical image-captioning system without the necessity of any additional training or fine-tuning.

Publication Type: Conference or Workshop Item (Paper)
Subjects: H Social Sciences > HM Sociology
Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Departments: School of Science & Technology > Computer Science
[thumbnail of 2022.coling-1.487.pdf]
Text - Published Version
Available under License Creative Commons: Attribution International Public License 4.0.

Download (4MB) | Preview


Add to AnyAdd to TwitterAdd to FacebookAdd to LinkedinAdd to PinterestAdd to Email


Downloads per month over past year

View more statistics

Actions (login required)

Admin Login Admin Login