Precise Average Redundancy Of An Idealized Arithmetic Codin

Redundancy is de ned as the excess of the code length over the optimal (ideal) code length. We study the average redundancy of an idealized arithmetic coding (for memoryless sources with unknown distributions) in which the Krichevsky and Tro mov estimator is followed by the Shannon{Fano code. We shall ignore here important practical implementation issues such as nite precisions and nite bu er sizes. In fact, our idealized arithmetic code can be viewed as an adaptive in nite precision implementation of arithmetic encoder that resembles Elias coding. However, we provide very precise results for the average redundancy that takes into account integer{length constraints. These ndings are obtained by analytic methods of analysis of algorithms such as theory of distribution of sequences modulo 1 and Fourier series. These estimates can be used to study the average redundancy of codes for tree sources, and ultimately the context-tree weighting algorithms.