Parallelizing XML Processing Pipelines via MapReduce

We present approaches for exploiting data parallelism in XML processing pipelines through novel compilation strategies to the MapReduce framework. Pipelines in our approach consist of sequences of processing steps that consume XML-structured data and produce, often through calls to “black-box” functions, modified (i.e., updated) XML structures. Our main contributions are a set of strategies for compiling such XML pipelines into parallel MapReduce networks and a discussion of their advantages and tradeoffs. We present a detailed experimental evaluation of these approaches using the Hadoop MapReduce system as our implementation platform. Our results show that execution times of XML pipelines can be significantly reduced using our compilation strategies. These efficiency gains, together with the benefits of MapReduce (e.g., fault tolerance) make our approach ideal for executing largescale, compute-intensive XML processing pipelines.