Discriminative Restricted Boltzmann Machines are Universal Approximators for Discrete Data
نویسنده
چکیده
This report proofs that discriminative Restricted Boltzmann Machines (RBMs) are universal approximators for discrete data by adapting existing universal approximation proofs for generative RBMs. Discriminative Restricted Boltzmann Machines are Universal Approximators for Discrete Data Laurens van der Maaten Pattern Recognition & Bioinformatics Laboratory Delft University of Technology
منابع مشابه
Deep Narrow Boltzmann Machines are Universal Approximators
We show that deep narrow Boltzmann machines are universal approximators of probability distributions on the activities of their visible units, provided they have sufficiently many hidden layers, each containing the same number of units as the visible layer. Besides from this existence statement, we provide upper and lower bounds on the sufficient number of layers and parameters. These bounds sh...
متن کاملRepresentational Power of Restricted Boltzmann Machines and Deep Belief Networks
Deep belief networks (DBN) are generative neural network models with many layers of hidden explanatory factors, recently introduced by Hinton, Osindero, and Teh (2006) along with a greedy layer-wise unsupervised learning algorithm. The building block of a DBN is a probabilistic model called a restricted Boltzmann machine (RBM), used to represent one layer of the model. Restricted Boltzmann mach...
متن کاملSynchronous Boltzmann Machines Can Be Universal Approximators
we prove in this paper that the class of reversible synchronous Boltzmann machines is universal for the representation of arbitrary functions defined on finite sets. This completes a similar result from Sussmann in the sequential case. Keywords-Synchronous random fields, Cellular automata, Gibbs distributions, Boltzmann machines. Neural networks.
متن کاملRefinements of Universal Approximation Results for Deep Belief Networks and Restricted Boltzmann Machines
We improve recently published results about resources of restricted Boltzmann machines (RBM) and deep belief networks (DBN)required to make them universal approximators. We show that any distribution pon the set {0,1}(n) of binary vectors of length n can be arbitrarily well approximated by an RBM with k-1 hidden units, where k is the minimal number of pairs of binary vectors differing in only o...
متن کاملExploiting local structure in Boltzmann machines
Restricted Boltzmann Machines (RBM) are well-studied generative models. For image data, however, standard RBMs are suboptimal, since they do not exploit the local nature of image statistics. We modify RBMs to focus on local structure by restricting visible-hidden interactions. We model longrange dependencies using direct or indirect lateral interaction between hidden variables. While learning i...
متن کامل