Towards High-Level SLAs with Heterogeneous Workloads: Job Resource Requirements Prediction for Deadline Schedulers

When executing their tasks, Grid and Cloud users want to express their requirements in terms of high-level metrics (e.g. in terms of execution time, not in terms of CPU MHz). Moreover, at the submission time they would like to know if the resource provider will fulfil with their requirements in order to decide if they would rather prefer another provider. On the other hand, the resource provider have to translate these high-level metrics into hardware related metrics, to know if he have enough resources to execute the user’s requests. In this context, we present our prediction system to foresee the amount of CPU required for a job to finish before its deadline. This prediction system uses machine learning techniques to learn about the jobs and online adjust itself. Before all this training is done, the Prediction System uses an analytical model for this purpose. We also contribute with a deadline-based scheduler which uses these predictions to discard jobs that will not meet its deadline in order to maximize the provider’s revenue by means of a dynamic and efficient resource allocation to jobs. We show how our system is able to provide higher revenue to resource providers compared to simple yet well known schedulers like EDF, SJF, etc. Key words— SLA, heterogeneous workloads, resource requirements prediction.