Parametric Identification for Comparing Performance Evaluation Techniques in Parallel Systems

Parallel computing has emerged as an environment for computing inherently parallel and computation intensive applications that requires huge processing power. Performance has always been a key factor in determining the success of parallel systems. Evaluating and analyzing the performance of parallel scheduling algorithms (i.e. parallel applications) is an important aspect of parallel computing research. Evaluating and analyzing parallel system is difficult due to the complex interaction between application characteristics and architectural features. Experimental measurement, Theoretical/Analytical modeling and Simulation are the most widely used techniques in the performance evaluation of parallel systems. Experimental measurement uses real or synthetic workloads, usually known as benchmarks, to evaluate and analyze their performance on actual hardware. Theoretical/Analytical models are used to abstract details of a parallel system. Simulation and other performance monitoring/visualization tools are extremely popular because they can capture the dynamic nature of the interaction between applications and architectures. Each of them has several types. For example, Experimental measurement has software, hardware, and hybrid. Theoretical/Analytical modeling has queueing network, Petri net, etc. and simulation has discrete event, trace/execution driven, Monte Carlo. Each of these three techniques has their own pros and cons. The purpose of this paper is firstly to identify parameters for comparing performance evaluation techniques and secondly to present a qualitative comparative survey of these techniques based on the identified parameters. This paper also discusses certain issues related with performance evaluation of parallel systems. These issues include how to select an appropriate metric for evaluating parallel systems.