City Research Online

Graph-based Neural Modules to Inspect Attention-based Architectures: A Position Paper

Carvalho, B. W., d’Avila Garcez, A. S. ORCID: 0000-0001-7375-9518 & Lamb, L. C. (2023). Graph-based Neural Modules to Inspect Attention-based Architectures: A Position Paper. In: CEUR Workshop Proceedings. Thinking Fast and Slow and Other Cognitive Theories in AI a AAAI 2022 Fall Symposium, 17-19 Nov 2022, Arlington, Virginia, US.


Encoder-decoder architectures are prominent building blocks of state-of-the-art solutions for tasks across multiple fields where deep learning (DL) or foundation models play a key role. Although there is a growing community working on the provision of interpretation for DL models as well as considerable work in the neuro-symbolic community seeking to integrate symbolic representations and DL, many open questions remain around the need for better tools for visualization of the inner workings of DL architectures. In particular, encoder-decoder models offer an exciting opportunity for visualization and editing by humans of the knowledge implicitly represented in model weights. In this work, we explore ways to create an abstraction for segments of the network as a two-way graph-based representation. Changes to this graph structure should be reflected directly in the underlying tensor representations. Such two-way graph representation enables new neuro-symbolic systems by leveraging the pattern recognition capabilities of the encoder-decoder along with symbolic reasoning carried out on the graphs. The approach is expected to produce new ways of interacting with DL models but also to improve performance as a result of the combination of learning and reasoning capabilities.

Publication Type: Conference or Workshop Item (Paper)
Additional Information: Copyright: 2022 Copyright for this paper by its authors. Use permitted under Creative Commons License Attribution 4.0 International (CC BY 4.0).
Publisher Keywords: Neuro-symbolic models, Deep Learning explainability, Model introspection
Subjects: Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Departments: School of Science & Technology
School of Science & Technology > Computer Science
SWORD Depositor:
[thumbnail of paper4.pdf]
Text - Published Version
Available under License Creative Commons: Attribution International Public License 4.0.

Download (1MB) | Preview


Add to AnyAdd to TwitterAdd to FacebookAdd to LinkedinAdd to PinterestAdd to Email


Downloads per month over past year

View more statistics

Actions (login required)

Admin Login Admin Login