City Research Online

Abstract Rule Based Pattern Learning with Neural Networks

Kopparti, R. M. (2020). Abstract Rule Based Pattern Learning with Neural Networks. Paper presented at the Twenty-Fifth AAAI/SIGAI Doctoral Consortium, 7-12 Feb 2020, New York, USA.

Abstract

The ability to learn abstractions and generalise is seen as the essence of human intelligence.7 Since 1950s, there have been efforts to build systems that learn and think like humans.16 It is observed that humans including infants tend to have good generalisation power when compared to the machine learning models in which hypothesis is usually approximated and may be prone to errors. The examples proposed by Marcus19,18,17 such as the failure to generalise equality, distinguish between even to odd numbers or the recognition of ABA or ABB patterns of syllables have attracted a significant amount of attention in psychology, particularly in the study of human language learning, but they have not been addressed systematically as problems of machine learning and neural networks.

In this article, the problem of learning abstract rules using neural networks is explained and a solution called ‘Relation Based Patterns’ (RBP) which model abstract relationships based on equality is proposed. RBP creates an inductive bias in the neural networks that leads to the learning of generalisable solutions. It is observed that integration of RBP leads to almost perfect generalisation in abstract rule learning tasks with synthetic data and to improvements in neural language modelling on real-world data.

The outline of the article is as follows : introduction to the problem is briefly described followed by a section on what is abstract pattern (rule) learning, the need for inductive bias and various ways of adding inductive bias into neural networks. The RBP method and its integration along with the experiments on the tasks of abstract rule learning, character prediction and melody prediction are summarized followed by conclusions and future work.

Publication Type: Conference or Workshop Item (Paper)
Subjects: Q Science > QA Mathematics > QA75 Electronic computers. Computer science
R Medicine > RC Internal medicine > RC0321 Neuroscience. Biological psychiatry. Neuropsychiatry
Departments: School of Mathematics, Computer Science & Engineering > Computer Science
Date Deposited: 09 Mar 2020 11:15
URI: https://openaccess.city.ac.uk/id/eprint/23874
[img]
Preview
Text - Accepted Version
Download (585kB) | Preview

Export

Downloads

Downloads per month over past year

View more statistics

Actions (login required)

Admin Login Admin Login