Generalising the Discriminative Restricted Boltzmann Machine

Cherla, S., Tran, S.N., Weyde, T. & Garcez, A. (2016). Generalising the Discriminative Restricted Boltzmann Machine.

Text - Accepted Version
Download (186kB) | Preview


We present a novel theoretical result that generalises the Discriminative Restricted Boltzmann Machine (DRBM). While originally the DRBM was defined assuming the {0, 1}-Bernoulli distribution in each of its hidden units, this result makes it possible to derive cost functions for variants of the DRBM that utilise other distributions, including some that are often encountered in the literature. This is illustrated with the Binomial and {-1, +1}-Bernoulli distributions here. We evaluate these two DRBM variants and compare them with the original one on three benchmark datasets, namely the MNIST and USPS digit classification datasets, and the 20 Newsgroups document classification dataset. Results show that each of the three compared models outperforms the remaining two in one of the three datasets, thus indicating that the proposed theoretical generalisation of the DRBM may be valuable in practice.

Item Type: Article
Additional Information: European Conference on Machine Learning and Principles and Practice of Knowledge Discovery Submitted to ECML 2016 conference track
Subjects: Q Science > QA Mathematics > QA75 Electronic computers. Computer science
Divisions: School of Informatics > Department of Computing

Actions (login required)

View Item View Item


Downloads per month over past year

View more statistics