Variational inference for continuous sigmoidal Bayesian networks

Latent random variables can be useful for modelling covariance relationships between observed variables. The choice of whether speciic latent variables ought to be continuous or discrete is often an arbitrary one. In a previous paper, I presented a \unit" that could adapt to be continuous or binary, as appropriate for the current problem, and showed how a Markov chain Monte Carlo method could be used for inference and parameter estimation in Bayesian networks of these units. In this paper, I develop a variational inference technique in the hope that it will prove to be more computationally eecient than Monte Carlo methods. After presenting promising inference results on a toy problem, I discuss why the variational technique does not work well for parameter estimation as compared to Monte Carlo.