A tutorial on Bayesian inference for variable dimension models

Variable dimension models are problems where the parameter space is not well defined, therefore the sample space is a infinite collection of unrelated subspaces. If the considered statistical model is not defined in concise way, then the dimensionality of the parameter space can also be part of the model uncertainty. These problems have been studied in the context of Bayesian model comparison and model selection, having several applications in statistical signal processing, image analysis, model-based clustering and financial data anaylisis. A common problem in MCMC methods for the variable dimension case is the lack of a dominating measure for the target distribution. For a dataset d we can consider a finite set of models M = M1,M2, ..,Mk, .. indexed by a parameter k ∈ K. Each model has a parameter vector θk ∈ Θk. If k is the dimensionality of the model, then Bayesian estimate can make use of a prior distribution p(k). However the parameter space θk will be fixed to k, and no dominating measure can be defined for a model jump. Green(1995) proposed a reversible transition kernel for the probability πk(θk). The method recasts the time reversibility property of a Markovian transition kernel X (s), which is a neccesary condition for ensuring convergence to a particular stationary distribution.