Eecient Stochastic Source Coding and an Application to a Bayesian Network Source Model
نویسنده
چکیده
Brendan J. Frey and Geoorey E. Hinton 1997. EEcient stochastic source coding and an application to a Bayesian network source model. The Computer Journal 40, 157{165. In this paper, we introduce a new algorithm called \bits-back coding" that makes stochastic source codes eecient. For a given one-to-many source code, we show that this algorithm can actually be more eecient than the algorithm that always picks the shortest codeword. Optimal eeciency is achieved when codewords are chosen according to the Boltzmann distribution based on the codeword lengths. It turns out that a commonly used technique for determining parameters | maximum likelihood estimation | actually minimizes the bits-back coding cost when codewords are chosen according to the Boltzmann distribution. A tractable approximation to maximum likelihood estimation | the generalized expectation maximization algorithm | minimizes the bits-back coding cost. After presenting a binary Bayesian network model that assigns exponentially many codewords to each symbol, we show how a tractable approximation to the Boltzmann distribution can be used for bits-back coding. We illustrate the performance of bits-back coding using using nonsynthetic data with a binary Bayesian network source model that produces 2 60 possible codewords for each input symbol. The rate for bits-back coding is nearly one half of that obtained by picking the shortest codeword for each symbol.
منابع مشابه
Making Stochastic Source Coding Eecient by Recovering Information
In this paper, we introduce a new algorithm called \bits-back coding" that makes stochastic source codes eecient. For a given one-to-many source code, we show that this algorithm can actually be more eecient than the algorithm that always picks the shortest codeword. Optimal eeciency is achieved when codewords are chosen according to the Boltzmann distribution based on the codeword lengths. Aft...
متن کاملEf"cient Stochastic Source Coding and an Application to a Bayesian Network Source Model
In this paper, we introduce a new algorithm called `bits-back coding' that makes stochastic source codes ef"cient. For a given one-to-many source code, we show that this algorithm can actually be more ef"cient than the algorithm that always picks the shortest codeword. Optimal ef"ciency is achieved when codewords are chosen according to the Boltzmann distribution based on the codeword lengths. ...
متن کاملEfficient Stochastic Source Coding and an Application to a Bayesian Network Source Model
In this paper, we introduce a new algorithm called ‘bits-back coding’ that makes stochastic source codes efficient. For a given one-to-many source code, we show that this algorithm can actually be more efficient than the algorithm that always picks the shortest codeword. Optimal efficiency is achieved when codewords are chosen according to the Boltzmann distribution based on the codeword length...
متن کاملJoint Bayesian Stochastic Inversion of Well Logs and Seismic Data for Volumetric Uncertainty Analysis
Here in, an application of a new seismic inversion algorithm in one of Iran’s oilfields is described. Stochastic (geostatistical) seismic inversion, as a complementary method to deterministic inversion, is perceived as contribution combination of geostatistics and seismic inversion algorithm. This method integrates information from different data sources with different scales, as prior informat...
متن کاملA Model for Tax Evasion Forcasting based on ID3 Algorithm and Bayesian Network
Nowadays, knowledge is a valuable and strategic source as well as an asset for evaluation and forecasting. Presenting these strategies in discovering corporate tax evasion has become an important topic today and various solutions have been proposed. In the past, various approaches to identify tax evasion and the like have been presented, but these methods have not been very accurate and the ove...
متن کامل