Generalising the Discriminative Restricted Boltzmann Machine

Cherla, S., Tran, S.N., Weyde, T. & Garcez, A. (2016). Generalising the Discriminative Restricted Boltzmann Machine.

[img]
Preview
Text - Accepted Version
Download (186kB) | Preview

Abstract

We present a novel theoretical result that generalises the Discriminative Restricted Boltzmann Machine (DRBM). While originally the DRBM was defined assuming the {0, 1}-Bernoulli distribution in each of its hidden units, this result makes it possible to derive cost functions for variants of the DRBM that utilise other distributions, including some that are often encountered in the literature. This is illustrated with the Binomial and {-1, +1}-Bernoulli distributions here. We evaluate these two DRBM variants and compare them with the original one on three benchmark datasets, namely the MNIST and USPS digit classification datasets, and the 20 Newsgroups document classification dataset. Results show that each of the three compared models outperforms the remaining two in one of the three datasets, thus indicating that the proposed theoretical generalisation of the DRBM may be valuable in practice.

Item Type: Article
Additional Information: European Conference on Machine Learning and Principles and Practice of Knowledge Discovery Submitted to ECML 2016 conference track
Subjects: Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Divisions: School of Informatics > Department of Computing
URI: http://openaccess.city.ac.uk/id/eprint/14247

Actions (login required)

View Item View Item

Downloads

Downloads per month over past year

View more statistics