Garcez, A., Broda, K. & Gabbay, D. M. (2001). Symbolic knowledge extraction from trained neural networks: A sound approach. Artificial Intelligence, 125(12), pp. 153205. doi: 10.1016/S00043702(00)000771

PDF
Download (2MB)  Preview 
Abstract
Although neural networks have shown very good performance in many application domains, one of their main drawbacks lies in the incapacity to provide an explanation for the underlying reasoning mechanisms.
The “explanation capability” of neural networks can be achieved by the extraction of symbolic knowledge. In this paper, we present a new method of extraction that captures nonmonotonic rules encoded in the network, and prove that such a method is sound.
We start by discussing some of the main problems of knowledge extraction methods. We then discuss how these problems may be ameliorated. To this end, a partial ordering on the set of input vectors of a network is defined, as well as a number of pruning and simplification rules. The pruning rules are then used to reduce the search space of the extraction algorithm during a pedagogical extraction, whereas the simplification rules are used to reduce the size of the extracted set of rules. We show that, in the case of regular networks, the extraction algorithm is sound and complete.
We proceed to extend the extraction algorithm to the class of nonregular networks, the general case. We show that nonregular networks always contain regularities in their subnetworks. As a result, the underlying extraction method for regular networks can be applied, but now in a decompositional fashion. In order to combine the sets of rules extracted from each subnetwork into the final set of rules, we use a method whereby we are able to keep the soundness of the extraction algorithm.
Finally, we present the results of an empirical analysis of the extraction system, using traditional examples and realworld application problems. The results have shown that a very high fidelity between the extracted set of rules and the network can be achieved.
Item Type:  Article 

Uncontrolled Keywords:  neuralsymbolic integration, rule extraction, nonmonotonic reasoning, artificial neural networks 
Subjects:  Q Science > QA Mathematics > QA75 Electronic computers. Computer science 
Divisions:  School of Informatics > Department of Computing 
URI:  http://openaccess.city.ac.uk/id/eprint/293 
Actions (login required)
View Item 
Downloads
Downloads per month over past year