Maximizing multi-information

Stochastic interdependence of a probablility distribution on a product space is measured by its Kullback-Leibler distance from the exponential family of product distributions (called multi-information). Here we investigate lowdimensional exponential families that contain the maximizers of stochastic interdependence in their closure. Based on a detailed description of the structure of probablility distributions with globally maximal multi-information we obtain our main result: The exponential family of pure pair-interactions contains all global maximizers of the multiinformation in its closure. Index Terms — Multi-information, exponential family, Kullback-Leibler divergence, pair-interaction, infomax principle, Boltzmann machine, neural networks.