We provide a new result that links two crucial entropy notions: Shannon Entropy H1 and collision entropy H2. Our formula gives the worst possible amount of collision entropy in a probability distribution, when its Shannon Entropy is fixed. Our results and techniques used in the proof immediately imply many quantitatively tight separations between Shannon and smooth Renyi entropy, which were pre...

This paper studies the chromosome information of twenty five species, namely, mammals, fishes, birds, insects, nematodes, fungus, and one plant. A quantifying scheme inspired in the state space representation of dynamical systems is formulated. Based on this algorithm, the information of each chromosome is converted into a bidimensional distribution. The plots are then analyzed and characterize...

A new class of information-theoretic divergence measures based on the Shannon entropy is introduced. Unlike the well-known Kullback divergences, the new measures do not require the condition of absolute continuity to be satisfied by the probability distributions involved. More importantly, their close relationship with the variational distance and the probability of misclassification error are ...

The lore is that standard information theory provides an analysis of information quantity, but not of information content. I argue this lore is incorrect, that there is an adequate informational semantics latent in standard theory. The roots of this notion of content can be traced to the secret parallel development of an information theory equivalent to Shannon’s by Turing at Bletchley Park, an...

This thesis focuses on the Shannon capacity of a graph. Suppose we want to send a message across a channel to a receiver. The general question is about the effective size of an alphabet in a model such that the receiver may recover the original message without errors. To answer this question Shannon defined the Shannon capacity of a graph, Θ(G), and stated Shannon’s Theorem. We study an article...

The Shannon upper bound places a limit on the error-free information transmission rate (capacity) of a noisy channel. It has stood for over sixty years, and underlies both theoretical and practical work in the telecommunications industry. This upper bound arises from the Shannon-Hartley law, which has two parameters: the available bandwidth and the signal-to-noise power ratio. However, aside fr...

Shannon entropy H and related measures are increasingly used in molecular ecology and population genetics because (1) unlike measures based on heterozygosity or allele number, these measures weigh alleles in proportion to their population fraction, thus capturing a previously-ignored aspect of allele frequency distributions that may be important in many applications; (2) these measures connect ...

The ionospheric sounding data at two southern hemisphere stations, the Argentine Islands and Port Stanley, are analyzed using a method previously developed by the authors. Negative trends of the critical frequency foF2 are found for both stations. The magnitudes of the trends are close to those at the corresponding (close geomagnetic latitude) stations of the northern hemisphere, as considered ...

In this note we revisit the famous result of Shannon [Sha49] stating that any encryption scheme with perfect security against computationally unbounded attackers must have a secret key as long as the message. This result motivated the introduction of modern encryption schemes, which are secure only against a computationally bounded attacker, and allow some small (negligible) advantage to such a...