City Research Online

Low-cost representation for restricted Boltzmann machines

Tran, S. & Garcez, A. (2014). Low-cost representation for restricted Boltzmann machines. Lecture Notes in Computer Science, 8834, pp. 69-77. doi: 10.1007/978-3-319-12637-1_9

Abstract

This paper presents a method for extracting a low-cost representation from restricted Boltzmann machines. The new representation can be considered as a compression of the network, requiring much less storage capacity while reasonably preserving the network's performance at feature learning. We show that the compression can be done by converting the weight matrix of real numbers into a matrix of three values {-1, 0, 1} associated with a score vector of real numbers. This set of values is similar enough to Boolean values which help us further translate the representation into logical rules. In the experiments reported in this paper, we evaluate the performance of our compression method on image datasets, obtaining promising results. Experiments on the MNIST handwritten digit classification dataset, for example, have shown that a 95% saving in memory can be achieved with no significant drop in accuracy.

Publication Type: Article
Additional Information: The final publication is available at Springer via http://dx.doi.org/10.1007/978-3-319-12637-1_9
Publisher Keywords: Restricted Boltzmann Machines, Low-cost Representation, Knowledge Extraction
Subjects: Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Departments: School of Science & Technology > Computer Science
SWORD Depositor:
[thumbnail of iconip_14.pdf]
Preview
Text - Accepted Version
Download (339kB) | Preview

Export

Add to AnyAdd to TwitterAdd to FacebookAdd to LinkedinAdd to PinterestAdd to Email

Downloads

Downloads per month over past year

View more statistics

Actions (login required)

Admin Login Admin Login