Information Geometry of Loopy BP
نویسندگان
چکیده
We have given the information geometrical framework [1] to analyze the BP decoding algorithms of turbo codes and LDPC codes[2, 4]. In this article, we extend our results to the BP algorithm on general graphs. The main results are the characterization of the fixed points and geometrical explanation about the cause of error of the inference. We also gives a new cost function which yields a new algorithm.
منابع مشابه
Information Geometry of Propagation Algorithms and Approximate Inference
Consider the inference problem of undirected graphical models[8, 9]. When the graph is tree, the Belief Propagation (BP) algorithm (J.Pearl[11]) is an efficient algorithm and the exact inference is computed. However, when the graph is “loopy,” and the loops are big, the exact inference becomes intractable. Besides sampling methods, such as MCMC, tractable approximate inference gives us one prac...
متن کاملStochastic Reasoning, Free Energy, and Information Geometry
Belief propagation (BP) is a universal method of stochastic reasoning. It gives exact inference for stochastic models with tree interactions and works surprisingly well even if the models have loopy interactions. Its performance has been analyzed separately in many fields, such as AI, statistical physics, information theory, and information geometry. This article gives a unified framework for u...
متن کاملInformation Geometrical Framework for Analyzing Belief Propagation Decoder
The mystery of belief propagation (BP) decoder, especially of the turbo decoding, is studied from information geometrical viewpoint. The loopy belief network (BN) of turbo codes makes it difficult to obtain the true “belief” by BP, and the characteristics of the algorithm and its equilibrium are not clearly understood. Our study gives an intuitive understanding of the mechanism, and a new frame...
متن کاملGraphs with Cycles: A Summary of Martin Wainwright’s Ph.D. Thesis
The problem around which this paper focuses is that of computing the marginals of a given distribution p(x). This is a common problem that arises in many fields. One of the areas which has drawn much attention recently is that of error correcting codes in information theory. Here, so-called message passing techniques known under the name of belief propagation (BP) are used to quickly decode cod...
متن کاملError Bounds Between Marginal Probabilities and Beliefs of Loopy Belief Propagation Algorithm
Belief propagation (BP) algorithm has been becoming increasingly a popular method for probabilstic inference on general graphical models. When networks have loops, it may not converge and, even if converges, beliefs, Le., the result of the algorithm, may not be equal to exact marginal probabilties. When networks have loops, the algorithm is called Loopy BP (LBP). Tatikonda and Jordan applied Gi...
متن کامل