1 Deenitions Consider a distribution D on n bits x = x 1 x n. D is k-wise independent ii for all sets of k indices S = fi 1 x ik = a 1 a k ] = 1 2 k : The idea is that if we restrict our attention to any k positions in x, no matter how many times we sample from D, we cannot distinguish D from the uniform distribution over n bits. We can get a Fourier interpretation of k-wise independence by vie...

In this paper we investigate the extent to which Katz backoff language models can be compressed through a combination of parameter quantization (width-wise compression) and parameter pruning (length-wise compression) methods while preserving performance. We compare the compression and performance that is achieved using entropy-based pruning against that achieved using only parameter quantizatio...