Survey on Load Balancing and Data Skew Mitigation in Mapreduce Applications

Since few years Map Reduce programming model have shown great success in processing huge amount of data. Map Reduce is a framework for data-intensive distributed computing of batch jobs. This data-intensive processing creates skew in Map Reduce framework and degrades performance by great value. This leads to greatly varying execution time for the Map Reduce jobs. Due to this varying execution times results into low resource utilization and high overall execution time since new Map Reduce cycle can start only after all reducers are completed. During this paper, we are going to study various methodologies and techniques used to mitigate data skew and partition skew, and describe various advantages and limitations of these approaches.