This paper provides evidence for Genzel and Charniak’s (2002) entropy rate principle, which predicts that the entropy of a sentence increases with its position in the text. We show that this principle holds for individual sentences (not just for averages), but we also find that the entropy rate effect is partly an artifact of sentence length, which also correlates with sentence position. Second...

Based on the studies on information transmission in discrete multivariable linear time invariant (LTI) system disturbed by stationary noise, relations within entropy rate, mutual information rate and H∞ entropy are discussed in both general control problem and classic tracking problem. For the general control systems, equivalent relations between entropy rate and H ∞ entropy are formulated by u...

Mergers are functions that transform k (possibly dependent) random sources into a single random source, in a way that ensures that if one of the input sources has minentropy rate δ then the output has min-entropy rate close to δ. Mergers have proven to be a very useful tool in explicit constructions of extractors and condensers, and are also interesting objects in their own right. In this work ...

The interrelationships of the fundamental biological processes natural selection, mutation, and stochastic drift are quantified by the entropy rate of Moran processes with mutation, measuring the long-run variation of a Markov process. The entropy rate is shown to behave intuitively with respect to evolutionary parameters such as monotonicity with respect to mutation probability (for the neutra...