We propose to use precise estimators of mutual information (MI) to find the least dependent components in a linearly mixed signal. On the one hand, this seems to lead to better blind source separation than with any other presently available algorithm. On the other hand, it has the advantage, compared to other implementations of "independent" component analysis (ICA), some of which are based on ...

Feature selection is a critical step for pattern recognition and many other applications. Typically, feature selection strategies can be categorized into wrapper and filter approaches. Filter approach has attracted much attention because of its flexibility and computational efficiency. Previously, we have developed an ICA-MI framework for feature selection, in which the Mutual Information (MI) ...

This study regards discrete mutual information and demonstrates the use of the theory with an example of radiological image registration with in-plane, twodimensional images, using various search strategies. Image registration is the process of finding an optimal geometric transformation between corresponding image data. Although it has applications in many fields, the one that is addressed in ...

There exists strong word association in natural language. Based on mutual information, this paper proposes a new MI-Trigger-based modeling approach to capture the preferred relationships between words over a short or long distance. Both the distance-independent(DI) and distancedependent(DD) MI-Trigger-based models are constructed within a window. It is found that proper MI-Trigger modeling is s...

This paper presents an information theory-driven matching methodology for making causal inference from observational data. The paper adopts a “potential outcomes framework” view on evaluating the strength of cause-effect relationships: the population-wide average effects of binary treatments are estimated by comparing two groups of units – the treated and untreated (control). To reduce the bias...

Most current gene coexpression databases support the analysis for linear correlation of gene pairs, but not nonlinear correlation of them, which hinders precisely evaluating the gene-gene coexpression strengths. Here, we report a new database, MIrExpress, which takes advantage of the information theory, as well as the Pearson linear correlation method, to measure the linear correlation, nonline...

Given the constant rise in quantity and quality of data obtained from neural systems on many scales ranging from molecular to systems’, information-theoretic analyses became increasingly necessary during the past few decades in the neurosciences. Such analyses can provide deep insights into the functionality of such systems, as well as a rigid mathematical theory and quantitative measures of in...

In pattern classification, feature selection is an important factor in the performance of classifiers. In particular, when classifying a large number of features or variables, the accuracy and computational time of the classifier can be improved by using the relevant feature subset to remove the irrelevant, redundant, or noisy data. The proposed method consists of two parts: a wrapper part with...

We present a methodology for incorporating prior knowledge on class probabilities into the registration process. By using knowledge from the imaging modality, pre-segmentations, and/or probabilistic atlases, we construct vectors of class probabilities for each image voxel. By defining new image similarity measures for distribution-valued images, we show how the class probability images can be n...