We investigate methods of estimating residue correlation within protein sequences. We begin by using mutual information (MI) of adjacent residues, and improve our methodology by defining the mutual information vector (MIV) to estimate long range correlations between nonadjacent residues. We also consider correlation based on residue hydropathy rather than protein-specific interactions. Finally,...

Recent works investigated the generalization properties in deep neural networks (DNNs) by studying the Information Bottleneck in DNNs. However, the measurement of the mutual information (MI) is often inaccurate due to the density estimation. To address this issue, we propose to measure the dependency instead of MI between layers in DNNs. Specifically, we propose to use Hilbert-Schmidt Independe...

Mutual information (MI) is currently the most popular match metric in handling the registration problem for multi modality images. However, interpolation artifacts impose deteriorating effects to the accuracy and robustness of MI-based methods. This paper analyzes the generation mechanism of the artifacts inherent in partial volume interpolation (PVI) and shows that the mutual information resul...

Estimating Mutual Information by Local Gaussian Approximation Report Title Estimating mutual information (MI) from samples is a fundamental problem in statistics, machine learning, and data analysis. Recently it was shown that a popular class of non-parametric MI estimators perform very poorly for strongly dependent variables and have sample complexity that scales exponentially with the true MI...

This prototype system demonstrates a novel method of word segmentation based on corpus statistics. Since the central technique we used is unsupervised training based on a large corpus, we refer to this approach as unsupervised word segmentation. The unsupervised approach is general in scope and can be applied to both Mandarin Chinese and Taiwanese. In this prototype, we illustrate its use in wo...

As encoding spatial information into mutual information (MI) can improve the nonrigid registration against bias fields where the conventional MI is challenged, we propose to unify this encoding into the computation of the joint probability distribution function (PDF). The PDF is computed based on local volumes while the global intensity information is also incorporated to maintain the global in...

In this paper we present a novel method for performing image registration of different modalities. Mutual Information (MI) is an established method for performing such registration. However, it is recognised that standard MI is not without some problems, in particular it does not utilise spatial information within the images. Various modifications have been proposed to resolve this, however the...

Mutual information (MI) was introduced for use in multimodal image registration over a decade ago [1,2,3,4]. The MI between two images is based on their marginal and joint/conditional entropies. The most common versions of entropy used to compute MI are the Shannon and differential entropies; however, many other definitions of entropy have been proposed as competitors. In this article, we show ...

We define a metric, mutual information in frequency (MI-in-frequency), to detect and quantify the statistical dependence between different frequency components in the data, referred to as cross-frequency coupling and apply it to electrophysiological recordings from the brain to infer crossfrequency coupling. The current metrics used to quantify the cross-frequency coupling in neuroscience canno...

This paper addresses the problem of correspondence selection in stereo vision using the mutual information (MI) measure. MI is an information theoretic topic that has recently seen a prolific expansion in the computer vision and medical imaging field. Two main issues are considered in this paper. Firstly, a new stereo matching algorithm is presented that uses a histogram-based formulation of mu...