A Framework for Estimating Stream Expression Cardinalities

Given m distributed data streams A1, . . . ,Am, we consider the problem of estimating the number of uniqueidentifiers in streams defined by set expressions over A1, . . . ,Am. We identify a broad class of algorithms for solvingthis problem, and show that the estimators output by any algorithm in this class are perfectly unbiased and satisfystrong variance bounds. Our analysis unifies and generalizes a variety of earlier results in the literature. To demonstrateits generality, we describe several novel sampling algorithms in our class, and show that they achieve a novel tradeoffbetween accuracy, space usage, update speed, and applicability.