We outline problems with the interpretation of accuracy in the presence of bias, arguing that the issue is a particularly pressing concern for RTE evaluation. Furthermore, we argue that average precision scores are unsuitable for RTE, and should not be reported. We advocate mutual information as a new evaluation measure that should be reported in addition to accuracy and confidence-weighted score.

In this paper we study the distribution of words across the different parts of a book using tools from information theory. In particular, the mutual information between words in the text and parts of the text is compared with the mutual information of a shuffled version of the book. This analysis allows us to extract not only relevant words of the text but also relationships between the differe...

We give necessary and sufficient conditions for stochastically bounded coexistence in a class of models for two species competing in a randomly varying environment. Coexistence is implied by mutual invasibility, as conjectured by Turelli. In the absence of invasibility, a species converges to extinction with large probability if its initial population is small, and extinction of one species mus...

Given the constant rise in quantity and quality of data obtained from neural systems on many scales ranging from molecular to systems’, information-theoretic analyses became increasingly necessary during the past few decades in the neurosciences. Such analyses can provide deep insights into the functionality of such systems, as well as a rigid mathematical theory and quantitative measures of in...

We generalize the standard competitive rational expectations equilibrium (Hellwig (1980), Verrecchia (1982)) by studying the possibility that informed agents open mutual funds in order to sell their private information. We illustrate how mutual funds endogenously arise in equilibrium and we characterize the fund managers’ optimal investment management fees under imperfect competition. In our mo...

We argue that the estimation of the mutual information between high dimensional continuous random variables is achievable by gradient descent over neural networks. This paper presents a Mutual Information Neural Estimator (MINE) that is linearly scalable in dimensionality as well as in sample size. MINE is backpropable and we prove that it is strongly consistent. We illustrate a handful of appl...

Proliferation of gestural interfaces necessitates the creation of robust gesture recognition systems. A novel technique using Mutual Information to classify gestures in a recognition system is presented. As this technique is based on well-known information theory metrics the underlying operation is not as complex as many other techniques which allows for this technique to be easily implemented....

We consider some formulations of the entropy bounds at the semiclassical level. The entropy S(V ) localized in a region V is divergent in quantum field theory (QFT). Instead of it we focus on the mutual information I(V,W ) = S(V ) + S(W ) − S(V ∪W ) between two different non-intersecting sets V and W . This is a low energy quantity, independent of the regularization scheme. In addition, the mut...

In this paper, a novel pattern classification approach is proposed by regularizing the classifier learning to maximize mutual information between the classification response and the true class label. We argue that, with the learned classifier, the uncertainty of the true class label of a data sample should be reduced by knowing its classification response as much as possible. The reduced uncert...