City Research Online

Formalizing Consistency and Coherence of Representation Learning

Stromfelt, H., Dickens, L., d'Avila Garcez, A. ORCID: 0000-0001-7375-9518 & Russo, A. (2022). Formalizing Consistency and Coherence of Representation Learning. In: Koyejo, S., Mohamed, S., Agarwal, A. , Belgrave, D., Cho, K. & Oh, A. (Eds.), Advances in Neural Information Processing Systems 35: Annual Conference on Neural Information Processing Systems 2022. Neural Information Processing Systems, 28 Nov - 9 Dec 2022, New Orleans, USA.


In the study of reasoning in neural networks, recent efforts have sought to improve consistency and coherence of sequence models, leading to important developments in the area of neuro-symbolic AI. In symbolic AI, the concepts of consistency and coherence can be defined and verified formally, but for neural networks these definitions are lacking. The provision of such formal definitions is crucial to offer a common basis for the quantitative evaluation and systematic comparison of connectionist, neuro-symbolic and transfer learning approaches. In this paper, we introduce formal definitions of consistency and coherence for neural systems. To illustrate the usefulness of our definitions, we propose a new dynamic relation-decoder model built around the principles of consistency and coherence. We compare our results with several existing relation-decoders using a partial transfer learning task based on a novel data set introduced in this paper. Our experiments show that relation-decoders that maintain consistency over unobserved regions of representation space retain coherence across domains, whilst achieving better transfer learning performance.

Publication Type: Conference or Workshop Item (Paper)
Additional Information: Copyright the authors, 2022.
Subjects: H Social Sciences > HN Social history and conditions. Social problems. Social reform
Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Departments: School of Science & Technology
School of Science & Technology > Computer Science
SWORD Depositor:
[thumbnail of NeurIPS_2022.pdf]
Text - Accepted Version
Download (3MB) | Preview


Add to AnyAdd to TwitterAdd to FacebookAdd to LinkedinAdd to PinterestAdd to Email


Downloads per month over past year

View more statistics

Actions (login required)

Admin Login Admin Login