Exploration of mean-field approximation for feedforward networks

We present a formulation of mean-field approximation for layered feed-forward stochastic networks. In this formulation, one can obtain not only estimates of averages for state variables of the networks but also those of intra-layer correlations, the latter of which cannot be obtained by the conventional mean-field approximation. Moreover, this formulation provides a framework to treat “conditional” expectations, expectations under the constraint that external information about statistics are fed to some layers of the network, which plays an important role in several applications such as the Helmholtz machine.