Adaptive Tap Equations

We develop a TAP mean eld approach to models with quadratic interactions which does not assume a speciic randomness of the couplings but rather adapts to the concrete data. The method is based on an extra set of mean eld equations for the Onsager correction term to the naive mean eld result. We present applications for the Hoppeld model and for a Bayesian classiier. 1.1 Introduction Mean eld (MF) methods provide eecient approximations which are able to cope with the increasing complexity of modern probabilistic data models. They replace the intractable task of computing high dimensional sums and integrals by the tractable problem of solving a system of nonlinear equations. The TAP (21) MF approach represents a principled way for correcting the deeciencies of simple MF methods which are based on the crude approximation of replacing the intractable distribution by a fac-torized one, thereby neglecting important correlations between variables. In contrast, the TAP method takes into account nontrivial dependencies by estimating the reaction of all other random variables when a single variable is deleted from the system (8). The method has its origin in the statistical physics of amorphous systems, where it was developed by Thouless, Anderson and Palmer (TAP) to treat the Sherrington-Kirkpatrick (SK) model of disordered magnetic materials (19). Under the assumption that the couplings (or interactions) between random variables are themselves drawn at random from certain classes of distributions, the TAP equations provide an exact result in the 'thermodynamic limit' of innnitely many variables. The 'Onsager correction' to the simple or 'naive' MF theory will explicitly depend on the distribution of these couplings. Two models with the same connectivities but diierent distributions for the couplings, like e.g. the SK model and the Hoppeld model (5) have diierent expressions for the TAP corrections (see e.g. (8), chapter XIII). In order to use the TAP method as a good approximation for practical applications to real data, the lack of knowledge of the underlying