An Investigation of Hierarchical Bit Vectors

Hierarchical bit vectors represent sets of integers using a collection of bit vectors. At the top level, a single bit is set iff the set is non-empty. The bits of this next level summarize ranges of the elements. In the case of a binary hierarchical bit vector the two bits of the next level summarize two ranges: the lower half and the upper half of the possible elements. At the lowest level each bit records the membership of a particular integer. Hierarchical bit vectors find application in information retrieval, bioinformatics, nonaveraging sets, and the conversion of NFAs to DFAs. Competing data structures for such applications include simple bit vectors and tree-based structures such as skip-lists. A comparison of hierarchical bit vectors with two other representations (simple bit vectors and binary search trees) is presented. The comparison includes both analytical and empirical analysis of the hierarchical bit vectors. The analytical results show that as the size of the set used increases, hierarchical bit vectors enjoy an advantage over tree-based structures and that as the sets used become sparser, hierarchical bit vectors perform better than standard bit vectors. The empirical results confirm that hierarchical bit vectors sit in between the other two. The empirical investigation also highlights the impact that the processor cache has on the break-even points.