Low-cost representation for restricted Boltzmann machines

Tran, S. & Garcez, A. (2014). Low-cost representation for restricted Boltzmann machines. Lecture Notes in Computer Science, 8834, pp. 69-77. doi: 10.1007/978-3-319-12637-1_9

[img]
Preview
Text - Accepted Version
Download (339kB) | Preview

Abstract

This paper presents a method for extracting a low-cost representation from restricted Boltzmann machines. The new representation can be considered as a compression of the network, requiring much less storage capacity while reasonably preserving the network's performance at feature learning. We show that the compression can be done by converting the weight matrix of real numbers into a matrix of three values {-1, 0, 1} associated with a score vector of real numbers. This set of values is similar enough to Boolean values which help us further translate the representation into logical rules. In the experiments reported in this paper, we evaluate the performance of our compression method on image datasets, obtaining promising results. Experiments on the MNIST handwritten digit classification dataset, for example, have shown that a 95% saving in memory can be achieved with no significant drop in accuracy.

Item Type: Article
Additional Information: The final publication is available at Springer via http://dx.doi.org/10.1007/978-3-319-12637-1_9
Uncontrolled Keywords: Restricted Boltzmann Machines, Low-cost Representation, Knowledge Extraction
Subjects: Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Divisions: School of Informatics > Department of Computing
URI: http://openaccess.city.ac.uk/id/eprint/11840

Actions (login required)

View Item View Item

Downloads

Downloads per month over past year

View more statistics