Arnold: Declarative Crowd-Machine Data Integration

The availability of rich data from sources such as the World Wide Web, social media, and sensor streams is giving rise to a range of applications that rely on a clean, consistent, and integrated database built over these sources. Human input, or crowd-sourcing, is an effective tool to help produce such high-quality data. It is infeasible, however, to involve humans at every step of the data cleaning process for all data. We have developed a declarative approach to data cleaning and integration that balances when and where to apply crowd-sourcing and machine computation using a new type of data independence that we term Labor Independence. Labor Independence divides the logical operations that should be performed on each record from the physical implementations of those operations. Using this layer of independence, the data cleaning process can choose the physical operator for each logical operation that yields the highest quality for the lowest cost. We introduce Arnold, a data cleaning and integration architecture that utilizes Labor Independence to efficiently clean and integrate large amounts of data.