Variable-Rate Trellis Source Encoding

The fixed slope lossy algorithm derived from the kthorder adaptive arithmetic codeword length function is extended to the case of finite-state decoders or trellis-structured decoders. It is shown that when this algorithm is used to encode a stationary, ergodic source with a continuous alphabet, the Lagrangian performance (i.e., the resulting compression rate plus times the resulting distortion) converges with probability one to a quantity computable as the infimum of an information-theoretic functional over a set of auxiliary random variables and reproduction levels, where > 0 and is designated to be the slope of the ratedistortion function R(D) of the source at some D; the quantity is close to R(D) + D when the order k used in the arithmetic coding or the number of states in the decoders is large enough. An alternating minimization algorithm for computing the quantity is presented; this algorithm is based on a training sequence and in turn gives rise to a design algorithm for variable-rate trellis source codes. The resulting variable-rate trellis source codes are very efficient in low-rate regions (below 0:8 bits/sample). With k = 8, the mean-squared error encoding performance at the rate 1=2 bits/sample for memoryless Gaussian sources is comparable to that afforded by trellis-coded quantizers; with k = 8 and the number of states in the decoder = 32, the mean-squared error encoding performance at the rate 1=2 bits/sample for memoryless Laplacian sources is about 1 dB better than that afforded by the trellis-coded quantizers with 256 states. With k = 8 and the number of states in the decoder = 256, the meansquared error encoding performance at the rates of a fraction of 1 bit/sample for highly dependent Gauss–Markov sources with correlation coefficient 0:9 is within about 0.6 dB of the distortionrate function. Note that at such low rates, predictive coders usually perform poorly.