Lecture 7 : Prefix codes , Kraft - McMillian Inequality
نویسنده
چکیده
In the previous lecture, we showed that Shannon constructed a code, which was a one-to-one mapping, that took a stream of data X = (X1, ..., Xn) generated iid from a distribution P (X) over a finite alphabet A = (a1, ..., aA) of size A, and compressed it using ≈ nH(X) bits in total or ≈ H(X) bits per symbol, on average (for sufficiently large n). The code was based considering a special subset of all sequences of size n called the typical set A , which has very few sequences compared to the size, A, of all possible sequences of length n, but contained a large proportion 1− of the probability mass.
منابع مشابه
Permutation codes, source coding and a generalisation of Bollobás-Lubell-Yamamoto-Meshalkin and Kraft inequalities
We develop a general framework to prove Krafttype inequalities for prefix-free permutation codes for source coding with various notions of permutation code and prefix. We also show that the McMillan-type converse theorem in most of these cases does not hold, and give a general form of a counterexample. Our approach is more general and works for other structures besides permutation codes. The cl...
متن کاملKraft - Chaitin Inequality
Kraft’s inequality [9] is essential for the classical theory of noiseless coding [1, 8]. In algorithmic information theory [5, 7, 2] one needs an extension of Kraft’s condition from finite sets to (infinite) recursively enumerable sets. This extension, known as Kraft-Chaitin Theorem, was obtained by Chaitin in his seminal paper [4] (see also, [3, 2], [10]). The aim of this note is to offer a si...
متن کاملComplexity and Randomness
2 Random finite strings 7 2.1 Plain Kolmogorov complexity . . . . . . . . . . . . . . . . . . 7 2.2 Prefix-free complexity . . . . . . . . . . . . . . . . . . . . . . 8 2.3 Kraft inequality, Shannon-Fano code . . . . . . . . . . . . . . 9 2.4 The universal semimeasure m . . . . . . . . . . . . . . . . . . 10 2.5 A priori probability . . . . . . . . . . . . . . . . . . . . . . . . 11 2.6 Coding,...
متن کاملNew Binary Fix - Free Codes with Kraft Sum 3 = 4
Two sufficient conditions are given for the existence of binary fix-free codes. The results move closer to the Ahlswede-Balkenhol-Khachatrian conjecture that Kraft sums of at most 3=4 suffice for the existence of fix-free codes. For each nonnegative integer n let f0; 1gn denote the set of all binary words of length n, and let f0; 1g denote the set of all finite length binary words, including th...
متن کاملOn Maximal Prefix Codes
Kraft’s inequality is a classical theorem in Information Theory which establishes the existence of prefix codes for certain (admissible) length distributions. We prove the following generalisation of Kraft’s theorem: For every admissible infinite length distribution one can construct a maximal prefix codes whose codewords satisfy this length distribution. Prefix codes are widely used in data tr...
متن کامل