Bipartite Graph Sampling Methods for Sampling Recommendation Data

Sampling is the common practice involved in academic and industry efforts on recommendation algorithm evaluation and selection. Experimental analysis often uses a subset of the entire useritem interaction data available in the operational recommender system, often derived by including all transactions associated with a subset of uniformly randomly selected users. Our paper formally studies the sampling problem for recommendation to understand to what extent population-based algorithm evaluation results correspond with sample-based results using different sampling methods. We use a bipartite graph to represent the key input data of user-item interaction for recommendation algorithms and build on the literature on unipartite graph sampling to develop sampling methods for our context of bipartite graph sampling. We also developed several metrics for assessing the quality of a given sample, including performance recovery and ranking recovery measures for assessing both single-sample and multiple-sample recovery performances. Based on the empirical results from two real-world datasets we provide some general recommendations for sampling for recommendation algorithm evaluation.