Efficient Learning in Sparsely Connected Boltzmann Machines
نویسندگان
چکیده
We present a heuristical procedure for eecient estimation of the partition function in the Boltzmann distribution. The resulting speed-up is of immediate relevance for the speed-up of Boltzmann Machine learning rules, especially for networks with a sparse connectivity.
منابع مشابه
Notes on Boltzmann Machines
I. INTRODUCTION Boltzmann machines are probability distributions on high dimensional binary vectors which are analogous to Gaussian Markov Random Fields in that they are fully determined by first and second order moments. A key difference however is that augmenting Boltzmann machines with hidden variables enlarges the class of distributions that can be modeled, so that in principle it is possib...
متن کاملPhase Transitions in Sparsely Connected BoltzmannMachinesSerge
The papers in this series are intended for internal use and are distributed by the author. Copies may be ordered from the library of Department of Computer Science. Abstract The primary motivation of this work has been the study of phase transitions in the simulated annealing schedules of Boltzmann machines. At rst, we introduce and focus on the harmonium Boltzmann machine, which can be used to...
متن کاملLearning in higher order Boltzmann machines using linear response
We introduce an efficient method for learning and inference in higher order Boltzmann machines. The method is based on mean field theory with the linear response correction. We compute the correlations using the exact and the approximated method for a fully connected third order network of ten neurons. In addition, we compare the results of the exact and approximate learning algorithm. Finally ...
متن کاملSparse Boltzmann Machines with Structure Learning as Applied to Text Analysis
We are interested in exploring the possibility and benefits of structure learning for deep models. As the first step, this paper investigates the matter for Restricted Boltzmann Machines (RBMs). We conduct the study with Replicated Softmax, a variant of RBMs for unsupervised text analysis. We present a method for learning what we call Sparse Boltzmann Machines, where each hidden unit is connect...
متن کاملBoltzmann Machines
A Boltzmann Machine is a network of symmetrically connected, neuronlike units that make stochastic decisions about whether to be on or off. Boltzmann machines have a simple learning algorithm that allows them to discover interesting features in datasets composed of binary vectors. The learning algorithm is very slow in networks with many layers of feature detectors, but it can be made much fast...
متن کامل