Two 1%s Don't Make a Whole: Comparing Simultaneous Samples from Twitter's Streaming API

In the present work, we compare samples of tweets from the Twitter Streaming API that were constructed from different connections tracking the same popular keywords at the same time. We find that tweets from the Streaming API are not sampled at random; rather, on average over 96% of the tweets seen in one sample are seen in all others. Somewhat surprisingly, however, tweets found only in a subset of samples do not significantly differ from those found in all samples in terms of user popularity or tweet structure. We conclude they are likely the result of a technical artifact rather than any systematic bias. Practically, our results show that for those wanting more data, an infinite number of Streaming API samples are necessary to collect “most” of the tweets containing a popular keyword. Additionally, it is likely that any findings from one sample from the Streaming API will hold for all samples that could have been taken.