Pathological Properties of Deep Bayesian Hierarchies
نویسندگان
چکیده
•All have the property that E[θn+1 | θn] = θn. •Called the martingale property •Philosophically desirable because it means that information is preserved as we move down the hierarchy •Theorem (Doob): All non-negative martingale sequences have a limit with probability 1. θn+1 | θn ∼ DP(cθn), θn+1 | θn ∼ BP(cθn), θn+1 | θn ∼ GammaP(cθn), θn+1 | θn ∼ PYP(cθn) Pathological Properties of Deep Bayesian Hierarchies Jacob Steinhardt and Zoubin Ghahramani
منابع مشابه
Bayesian Compositional Hierarchies - A Probabilistic Structure for Scene Interpretation
In high-level vision, it is often useful to organize conceptual models in compositional hierarchies. For example, models of building facades (which are used here as examples) can be described in terms of constituent parts such as balconies or window arrays which in turn may be further decomposed. While compositional hierarchies are widely used in scene interpretation, it is not clear how to mod...
متن کاملTemporal Predictions with Bayesian Compositional Hierarchies
In this note I describe a novel approach to modelling and exploiting probabilistic dependencies in compositional hierarchies for model-based scene interpretation. I present Bayesian Compositional Hierarchies (BCHs) which capture all probabilistic information about the objects of a compositional hierarchy in object-centered aggregate representations. BCHs extend typical Bayesian Network models b...
متن کاملBayesian Network Learning with Abstraction Hierarchies and Context-Specific Independence
Context-specific independence representations, such as treestructured CPTs, reduce the number of parameters in Bayesian networks by capturing local independence relationships. We previously presented Abstraction-Based Search (ABS), a technique for using attribute value hierarchies during Bayesian network learning to remove unimportant dis-ion-Based Search (ABS), a technique for using attribute ...
متن کاملScalable Deep Poisson Factor Analysis for Topic Modeling
A new framework for topic modeling is developed, based on deep graphical models, where interactions between topics are inferred through deep latent binary hierarchies. The proposed multi-layer model employs a deep sigmoid belief network or restricted Boltzmann machine, the bottom binary layer of which selects topics for use in a Poisson factor analysis model. Under this setting, topics live on ...
متن کاملLearning Hierarchical Features from Deep Generative Models
Deep neural networks have been shown to be very successful at learning feature hierarchies in supervised learning tasks. Generative models, on the other hand, have benefited less from hierarchical models with multiple layers of latent variables. In this paper, we prove that hierarchical latent variable models do not take advantage of the hierarchical structure when trained with some existing va...
متن کامل