Average Case Performance of the Apriori

The Apriori Algorithm examines baskets of items to determine which subsets of the items occur in lots of baskets. Suppose we wish to determine which items sets occur in at least k baskets. The algorithm considers item sets of size l in the order l = 1, 2, : : :. The only way this algorithm can determine that a set occurs at least k times is to count the k occurrences, but it sometimes determines (without counting) that a set occurs less than k times by noticing that some subsets of the l items occur less than k times. For algorithms that require explicit counting to verify the k occurrences, it is useful to seperate the total time into the \success time"; that is used to verify k occurrences, and the \failure time"; that is used to process sets which have less than k occurrences. This paper derives both exact and asymptotic formulas for both success and failure times in the case where the baskets are lled randomly with probability p (each shopper independently buys each item). The Apriori Algorithm considers almost every possible set of l items for those l where k bp l and almost no sets for larger l. For most applications the largest l such that k bp l is not very large. When it is less than one half of the number of items (essentially the only case of interest), the work associated with this largest such l dominates the running time. The probability that a particular set needs processing approaches zero at a rate that is a negative exponential function of the square of the diierence bp l ? k when k is above bp l. When k is large compared to 1, the probability that the set needs processing approaches 1 at a similar negative exponential rate.