We demonstrate an intuitive relation between conditional entropy and conditional expectation that is useful when one want to compare them as measurement tools to evaluate secrecy systems. In particular, we give a Security Property applicable to general vector variables in , using measurement based on vector quadratic distance, and we show that one can derive variables that can be measured with ...

It is well established that the notion of min-entropy fails to satisfy the chain rule of the form H(X,Y ) = H(X|Y )+ H(Y ), known for Shannon Entropy. The lack of a chain rule causes a lot of technical difficulties, particularly in cryptography where the chain rule would be a natural way to analyze how min-entropy is split among smaller blocks. Such problems arise for example when constructing ...

The pure quantum entanglement is generalized to the case of mixed compound states on an operator algebra to include the classical and quantum encodings as particular cases. The true quantum entanglements are characterized by quantum couplings which are described as transpose-CP, but not Completely Positive (CP), trace-normalized linear positive maps of the algebra. The entangled (total) informa...

This paper describes the work on Chinese named entity recognition performed by Yahoo team at the third International Chinese Language Processing Bakeoff. We used two conditional probabilistic models for this task, including conditional random fields (CRFs) and maximum entropy models. In particular, we trained two conditional random field recognizers and one maximum entropy recognizer for identi...

This paper proposes an entropy based Markov chain (EMC) fusion technique and demonstrates its applications in multisensor fusion. Self-entropy and conditional entropy, which measure how uncertain a sensor is about its own observation and joint observations respectively, are adopted. We use Markov chain as an observation combination process because of two major reasons: (a) the consensus output ...

An unsupervised model-based strategy for bias field correction is proposed. We assume that information (in the sense of the information theory) in the corrupted image is greater than that in the uncorrupted one. The method exploits the fact that neighboring voxels are highly correlated to correct the bias field using a linear model.

Exchangeability is a central notion in statistics and probability theory. The assumption that an infinite sequence of data points is exchangeable is at the core of Bayesian statistics. However, finite exchangeability as a statistical property that renders probabilistic inference tractable is less well-understood. We develop a theory of finite exchangeability and its relation to tractable probab...

We propose a new inference rule for estimating causal structure that underlies the observed statistical dependencies among n random variables. Our method is based on comparing the conditional distributions of variables given their direct causes (the so-called “Markov kernels”) for all hypothetical causal directions and choosing the most plausible one. We consider those Markov kernels most plaus...

This paper presents a new period-finding method based on conditional entropy that is both efficient and accurate. We demonstrate its applicability on simulated and real data. We find that it has comparable performance to other information-based techniques with simulated data but is superior with real data, both for finding periods and for just identifying periodic behaviour. In particular, it i...