PoBery: Possibly-complete Big Data Queries with Probabilistic Data Placement and Scanning

In big data query processing, there is a trade-off between accuracy and efficiency, for example, sampling approaches completeness efficiency. this article, we argue that performance can be significantly improved by slightly losing the possibility of completeness, is, chance complete. To quantify possibility, define new concept, Probability Completeness (hereinafter referred to as PC). For If executed 100 times, PC = 0.95 guarantees are no more than 5 incomplete results among results. Leveraging probabilistic placement scanning, trade off performance. propose PoBery (POssibly-complete Big quERY), method supports neither complete queries nor queries, but possibly-complete queries. The experimental conducted on HiBench prove accelerate while ensuring PC. Specifically, it guaranteed percentage larger given confidence. Through comparison with state-of-the-art key-value stores, show Drill-based performs fast Drill 1.7 ×, 1.1 1.5 × faster average Drill, Impala, Hive, respectively,