Performance Benchmarking of BPEL Engines: A Comparison Framework, Status Quo Evaluation and Challenges

Despite the popularity of BPEL engines to orchestrate complex and executable processes, there are still only few approaches available which help to find the most appropriate engine for individual requirements. One of the more crucial comparison factors for middleware products in industry are the performance characteristics. There exist multiple studies in both industry and academia testing the performance of BPEL engines, which differ in focus and method. We aim to compare the methods used in these approaches and provide guidance for further research in this area. Based on the related work in the field of performance testing, we created a process engine specific comparison framework, which we used to evaluate and classify nine different approaches that were found via a systematical literature survey. With the results of the status quo analysis in mind, we derived directions for further research in this area.