Bit Allocation in Sub-linear Time and the Multiple-Choice Knapsack Problem

We show that the problem of optimal bit allocation among a set of independent discrete quantizers given a budget constraint is equivalent to the multiple choice knapsack problem (MCKP). This result has three implications: first, it provides a trivial proof that the problem of optimal bit allocation is NP-hard and that its related decision problem is NP-complete; second, it unifies research into solving these problems that has to-date been done independently in the data compression community and the operations research community; third, many practical algorithms for approximating the optimal solution to MCKP can be used for bit allocation. We implement the GBFOS, Partition-Search, and Dudzinski-Walukiewicz algorithms and compare their running times for a variety of problem sizes.