Lower bounds on mutual information (MI) of long-haul optical fiber systems for hard-decision and soft-decision decoding are studied. Ready-to-use expressions to calculate the MI are presented. Extensive numerical simulations are used to quantify how changes in the optical transmitter, receiver, and channel affect the achievable transmission rates of the system. Special emphasis is put to the us...

We show that the mutual information, in the sense of Kolmogorov complexity, of any pair of strings x and y is equal, up to logarithmic precision, to the length of the longest shared secret key that two parties, one having x and the complexity profile of the pair and the other one having y and the complexity profile of the pair, can establish via a probabilistic protocol with interaction on a pu...

Bertschinger, Rauh, Olbrich, Jost, and Ay (Entropy, 2014) have proposed a definition of a decomposition of the mutual information MI(X : Y, Z) into shared, synergistic, and unique information by way of solving a convex optimization problem. In this paper, we discuss the solution of their Convex Program from theoretical and practical points of view.

This paper introduces a new similarity measure for multimodal image registration task. The measure is based on the generalized survival exponential entropy (GSEE) and mutual information (GSEE-MI). Since GSEE is estimated from the cumulative distribution function instead of the density function, it is observed that the interpolation artifact is reduced. The method has been tested on four real MR...

Mutual information (MI) is a powerful method for detecting relationships between data sets. There are accurate methods for estimating MI that avoid problems with "binning" when both data sets are discrete or when both data sets are continuous. We present an accurate, non-binning MI estimator for the case of one discrete data set and one continuous data set. This case applies when measuring, for...

The entropy and mutual information index are important concepts developed by Shannon in the context of information theory. They have been widely studied in the case of the multivariate normal distribution. We first extend these tools to the full symmetric class of multivariate elliptical distributions and then to the more flexible families of multivariate skew-elliptical distributions. We study...

This report gives an insight about two major aspects of information theory, including data compression and channel coding. It discusses the simulations results obtained using MATLAB from the information theory view point. It also defines these aspects by using the definitions of entropy and mutual information which reflects the optimum data compression could be obtained and the ultimate transmi...

Mutual information is a widely-used metric in medical image registration, which achieves good results; however it can lead to misregistration because of the absence of spatial information. One of the works to overcome this problem is proposed by [10] in which a gradient-based term is combined with mutual information. In this paper, we improve the method in [10] and propose a registration strate...

One of the main notions of information theory is the notion of mutual information in two messages (two random variables in Shannon information theory or two binary strings in algorithmic information theory). The mutual information in x and y measures how much the transmission of x can be simplified if both the sender and the recipient know y in advance. Gács and Körner gave an example where mut...

Registration of two dimensional to three dimensional orthopaedic medical image data has important applications particularly in the area of image guided surgery and sports medicine. Fluoroscopy to computer tomography (CT) registration is an important case, wherein digitally reconstructed radiographs derived from the CT data are registered to the fluoroscopy data. Traditional registration metrics...