Rating the process attribute utilizing AHP in SPICE-based process assessments

In SPICE (Software Process Improvement and Capability dEtermination) assessments, the rating of a process attribute is determined by the aggregated achievements of associated MPs (management practices). Previous rating scale usability studies revealed that experienced assessors had difficulty distinguishing the boundaries between adjacent ratings: Fully and Largely, Largely and Partially, and Partially and Not achieved. To cope with these boundary problems, this study presents an AHP (analytic hierarchy process) approach as an objective aggregation method for the achievements of MPs. This research analyzed assessors’ responses to our questions. Results indicated that the AHP approach was a systematic method and helpful in determining assessors’ consensus in boundary problems. Furthermore, respondents answered that they would again use the AHP approach for tackling boundary problems for future assessments. In addition, they felt that the AHP approach could provide the assessors with confidence in the results of an assessment. While assessors responded that the AHP approach was easy to understand, some assessors had difficulty in creating an AHP comparison matrix in detail. Assessors with AHP experience expressed their support through such statements as ‘reduce the assessment effort’ and ‘provide confidence in the assessment results’. These results suggest the AHP approach could be employed as a method in solving the boundary problem in SPICE-based process assessments. Copyright  2001 John Wiley & Sons, Ltd.