• The entropy is a state function: S = S(state). Furthermore, it is additive; S12 = S1 + S2 for a system with subsystems 1 and 2. • Entropy is unchanged in a reversible state change. That is, S(f) = S(i) and so ∆S = 0. For a reversible state change, entropy increases: S(f) > S(i), and so ∆S > 0. In all cases, ∆S ≥ 0. • When heat is added to a system, then the entropy changes by an amount propor...

It is an oft-quoted fact that there is much in common between the fields of music and mathematics. This paper seeks to examine that commonality through the use of information theory techniques, specifically, entropy as defined by Shannon. The analysis will be presented through the process of creating an entropy table for a collection of Chopin’s Nocturnes. The method for generating a random str...

The entropy rate of independent and identically distributed events can on average be encoded by H(X) bits per source symbol. However, in reality, series of events (or processes) are often randomly distributed and there can be arbitrary dependence between each event. Such processes with arbitrary dependence between variables are called stochastic processes. This report shows how to calculate the...

We present a constancy rate principle governing language generation. We show that this principle implies that local measures of entropy (ignoring context) should increase with the sentence number. We demonstrate that this is indeed the case by measuring entropy in three different ways. We also show that this effect has both lexical (which words are used) and non-lexical (how the words are used)...

We introduce a method for quantifying the inherent unpredictability of a continuous-valued time series via an extension of the differential Shannon entropy rate. Our extension, the specific entropy rate, quantifies the amount of predictive uncertainty associated with a specific state, rather than averaged over all states. We relate the specific entropy rate to popular ‘complexity’ measures such...

We treat secret key extraction when the eavesdropper has correlated quantum states. We propose quantum privacy amplification theorems different from Renner’s, which are based on quantum conditional Rényi entropy of order 1 + s. Using those theorems, we derive an exponential decreasing rate for leaked information and the asymptotic equivocation rate.

A task is randomly drawn from a finite set of tasks and is described using a fixed number of bits. All the tasks that share its description must be performed. Upper and lower bounds on the minimum ρ-th moment of the number of performed tasks are derived. The case where a sequence of tasks is produced by a source and n tasks are jointly described using nR bits is considered. If R is larger than ...