It is proven that for every random variable with a countably infinite set of outcomes and finite entropy there exists an optimal prefix code which can be constructed from Huffman codes for truncated versions of the random variable, and that the average lengths of any sequence of Huffman codes for the truncated versions converge to that of the optimal code. Also, it is shown that every optimal i...

In its modern formulation, the Maximum Entropy Principle was promoted by E.T. Jaynes, starting in the mid-fifties. The principle dictates that one should look for a distribution, consistent with available information, which maximizes the entropy. However, this principle focuses only on distributions and it appears advantageous to bring information theoretical thinking more prominently into play...

We use the \conservation of entropy" 1] to simplify the formula for the redundancy of a large class of variable-to-variable length codes on discrete, memoryless sources. This result leads to new asymptotic upper bounds on the redundancy of the \Tunstall-Huuman" code and the \Tunstall-Shannon-Fano" code.

This manual describes the Fortran 90 implementation of maximum-entropy basis functions. The main ingredients of the theory are presented, and then the numerical implementation is touched upon. Instructions on the installation and execution of the code, as well as on writing an interface to the library are presented. Each program module and the most important functions in each module are discuss...

We study a quantum phase transition between a phase which is topologically ordered and one which is not. We focus on a spin model, an extension of the toric code, for which we obtain the exact ground state for all values of the coupling constant that takes the system across the phase transition. We compute the entanglement and the topological entropy of the system as a function of this coupling...

We train recurrent neural network on a single, long, complex symbolic sequence with positive entropy. Training process is monitored through information theory based performance measures. We show that although the sequence is unpredictable, the network is able to code the sequence topological and statistical structure in recurrent neurons' activation scenarios. Such scenarios can be compactly re...

In this paper we consider the class of anti-uniform Huffman codes and derive tight lower and upper bounds on the average length, entropy, and redundancy of such codes in terms of the alphabet size of the source. The Fibonacci distributions are introduced which play a fundamental role in AUH codes. It is shown that such distributions maximize the average length and the entropy of the code for a ...

Entropy generation for natural convection in a partitioned cavity, with adiabatic horizontal and isothermally cooled vertical walls, is studied numerically by both a FORTRAN code and the commercially available CFD-ACE software. Effects of the Rayleigh number, the position of the heated partition, and the dimensionless temperature difference on the local and average entropy generation rate are i...