Executing cyclic scientific workflows in the cloud

Abstract We present an algorithm and a software architecture for cloud-based system that executes cyclic scientific workflows whose structure may change during run time. Existing approaches either rely on workflow definitions based directed acyclic graphs (DAGs) or require workarounds to implement structures. In contrast, our supports cycles natively, avoids workarounds, as such reduces the complexity of modelling maintenance. Our traverses transforms them iteratively into linear sequences executable actions. call these process chains. distributes chains multiple compute nodes in cloud oversees their execution. evaluate approach by applying it two practical use cases from domains astronomy engineering. also compare with existing management systems. The evaluation demonstrates is able execute dynamically changing design maintenance complex easier than solutions. It shows can parallel significantly speed up An implementation available Steep Workflow Management System we released under open-source license. resources first case are open source reproduction.