Compressed Counting (CC) was recently proposed for approximating the αth frequency moments of data streams, for 0 < α ≤ 2. Under the relaxed strict-Turnstile model, CC dramatically improves the standard algorithm based on symmetric stable random projections, especially as α → 1. A direct application of CC is to estimate the entropy, which is an important summary statistic in Web/network measure...

Thermodynamic entropy, as defined by Clausius, characterizes macroscopic observations of a system based on phenomenological quantities such as temperature and heat. In contrast, information-theoretic entropy, introduced by Shannon, is a measure of uncertainty. In this Letter, we connect these two notions of entropy, using an axiomatic framework for thermodynamics [E. H. Lieb and J. Yngvason Pro...

The paper examines relationships between the Shannon entropy and the `α-norm for n-ary probability vectors, n ≥ 2. More precisely, we investigate the tight bounds of the `α-norm with a fixed Shannon entropy, and vice versa. As applications of the results, we derive the tight bounds between the Shannon entropy and several information measures which are determined by the `α-norm. Moreover, we app...

1.2 REMARK ON INTERPRETATIONS OF THE SHANNON ENTROPY There are standard ways to interpret the Shannon entropy. For instance, the quantity H(p) can be viewed as a measure of the amount of uncertainty in a random experiment described by the probability mass function p, or as a measure of the amount of information one gains by learning the value of such an experiment. Indeed, it is possible to sta...

The paper examines relationships between the conditional Shannon entropy and the expectation of `α-norm for joint probability distributions. More precisely, we investigate the tight bounds of the expectation of `α-norm with a fixed conditional Shannon entropy, and vice versa. As applications of the results, we derive the tight bounds between the conditional Shannon entropy and several informati...

The possibility of reconciliation between canonical probability distributions obtained from the q-maximum entropy principle with predictions from the law of large numbers when empirical samples are held to the same constraints, is investigated into. Canonical probability distributions are constrained by both: (i) the additive duality of generalized statistics and (ii) normal averages expectatio...

Quantifiers were introduced to convert recurrence plots into a statistical analysis of dynamical properties. It is shown that the Shannon entropy, if properly computed, increases as the chaotic regime is developed as expected. Recurrence plots and a new estimator for the Shannon entropy are then used to identify asynchronisms in non-invasive mechanical ventilation. It is thus shown that the pha...

This work is focused on the numerical determination of Shannon probabilistic entropy for MEMS devices exhibiting some uncertainty in their structural response. a universal measure statistical or stochastic disorder static deformation dynamic vibrations engineering systems and available both continuous discrete distributions functions parameters. An interval algorithm using Monte Carlo simulatio...