Burrows Wheeler - Alternatives to Move to Front

The Burrows Wheeler Transform, first published in 1994, is a relatively new approach to text compression and has already proven to produce excellent results. However, there has been much research directed towards improving the efficienc y of the Move to Front algorithm with varying degrees of complexity. This paper examines a relatively simple technique using a dual modeling system and achieves very promising results. Because the MTF produces a highly skewed symbol distribution with little contextual information, the data can be more effectively utilized by splitting the alphabet into two models before the final encoding. One method suggested by Fenwick relies on a cache system where the most probable symbols are stored in a prominent foreground model, and the bulk remaining symbols stored in a larger background model. A similar approach has been suggested by Balkenhol but where the background is encoded in the original ASCII alphabet instead of the conventional MTF. Both methods produce superior results to the conventional MTF data stream, producing an overall bits/byte average of 2.345 and 2.361 over the Calgary Corpus respectively. In comparison with Wirth’s results, the overall average of Fenwick’s method reveals an approximate equal if not slightly superior outcome. This is very promising for a relatively simple technique that has much potential for further refinement. Information loss during the Burrows Wheeler Transform is also examined using Deterministic Information Theory measuring techniques. By utilizing the information measurement tools developed by Titchener a clearer understanding of informational changes over BWT has been established. Most importantly, MTF loses information making it amenable for compression, however it also contains little contextual structure and closely resembles the data stream after applying the BWT algorithm. These findings suggest it is viable to focus efforts on developing a better representation of the MTF stream, or even omitting it entirely. Since submitting this paper, further work was conducted using a cache directly on the BWT stream without the MTF recoding, achieving preliminary averages of 2.612bits/byte over the Calgary Corpus, with much promise for further improvements.