Quantifying cooperation or synergy among random variables in predicting a single target random variable is an important problem in many complex systems. We review three prior information-theoretic measures of synergy and introduce a novel synergy measure defined as the difference between the whole and the union of its parts. We apply all four measures against a suite of binary circuits to demon...

indeed, we demonstrate that the KGV can also be thought of as We introduce a new contrast function, the kemel mutual information (KMIj, to measure the degree of independence of continuous random variables. This contrast function provides an approximate upper bound on the mutual information, as measured near independence, and is based on a kernel density estimate of the mutual information betwee...

Characterization of the correlations of a multiparticle system remains an open problem. We generalize the notion of quantum discord, using multivariate mutual information, to characterize the quantum properties of a multiparticle system. This new measure, called dissension, is a set of numbers – quantumness vector. There are a number of different possible generalizations. We consider two of the...

This paper represents a further steps in combining Information Theory tools with relational learning. We show how mutual information can be used to find relevant relational features.

The mutual information of two random variables ı and  with joint probabilities {πij} is commonly used in learning Bayesian nets as well as in many other fields. The chances πij are usually estimated by the empirical sampling frequency nij/n leading to a point estimate I(nij/n) for the mutual information. To answer questions like “is I(nij/n) consistent with zero?” or “what is the probability t...