Hierarchical models of variance sources

In many models, variances are assumed to be constant although this assumption is known to be unrealistic. Joint modelling of means and variances can lead to infinite probability densities which makes it a difficult problem for many learning algorithms. We show that a Bayesian variational technique which is sensitive to probability mass instead of density is able to jointly model both variances and means. We discuss a model structure where a Gaussian variable which we call variance neuron controls the variance of another Gaussian variable. Variance neuron makes it possible to build hierarchical models for both variances and means. We report experiments with artificial data which demonstrate the ability of learning algorithm to find the underlying explanations—variance sources—for the variance in the data. Experiments with MEG data verify that variance sources are present in real-world signals.