A Tree Based Binary Encoding of Text Using LZW Algorithm

The most popular adaptive dictionary coding scheme used for text compression is the LZW algorithm [2]. In the LZW algorithm, a changing dictionary contains common strings that have been encountered so far in t,he text. The dictionary can be represented by a dynamic trie. The input text is examined character by character and the longest substring (called a prefix string) of the text which already exists in the trie, is replaced by a pointer to a node in the trie which represents, the prefix string. Motivation of our research is to explore a variation of the LZW algorithm for variable-length binary encoding of text (we call it LZWA algorithm) and to develop a memory-based VLSI architecture for text compression. We proposed a new methodology to represent the trie in the form of a binary tree (we call it binary trie) to maintain the dictionary used in the LZW scheme. This binary tree maintains all the properties of the trie and can easily be mapped into memory. As a result, the common substrings can be encoded using variable length prefix binary codes. The prefix codes enable us to uniquely decode the text in its original form. Formal definition of the binary trie and the detail schemes for memory mapping of the binary tries during both encoding and decoding operations have been presented in [l]. The proposed algorithms have been implemented and tested with different kinds of texts such as C source file, electronic messages etc. The algorithm outperforms the usual LZW scheme when the size of the text is small (usually less than 5K). Depending upon the characteristics of the text, the improvement of the compression ratio has been achieved around 10.30% compared to the LZW scheme. But its performance degrades for larger size texts. The main reason of this degradation is that most of the branches of the binary trie become highly skewed for less frequently appearing words. As a result, the performance is degraded when the words are encoded using the binary sequences of these highly skewed branches. Generating a height balanced binary trie will prova,bly be a research challenge. One possible remedy is to adaptively cut the highly skewed paths and drop some of the branches of the binary trie which correspond to the less frequently appearing words. There are many applications in the wireless media which deal with small size texts. The results of different variations including the VLSI implementation of the chip will be subject matter of our future papers.