Evaluation of an evaluation

Introduction. Evaluation is a systematic way to improve and make more effective actions that involves procedures which are useful, feasible, ethical, and accurate. Common questions in all evaluations are: do all part of program do well and effective? What is the good functioning? Why the program or its parts do not work well? What are the effects and consequences of the program? Is this program more effective than the others? What we learn from this program? What the consumers think about the program? The benefits of evaluation are: assessment of effectiveness, exploring the opportunities, assessing the priorities in allocating the resources, assessing the needs and demands of clients, better decision making and responding to external demands (westerholms, 1999). Unfortunately evaluation is not integrated in all programs. Also any evaluation study is biased to some extent. Decisions that an evaluator makes about what to examine, what methods and instruments to use, who to talk to, even the evaluator’s personal background, professional training and experiences all affect the way the study is conducted and influence the outcome of evaluation. Therefore Meta–evaluation (the evaluation of an evaluation) is important, it adds the credibility of final results and improves an evaluation study before it is irretrievably too late. In this study I evaluated the final report of internal evaluation that was performed by the medical surgical nursing group of faculty of nursing in Kashan medical university. Methods. Medical surgical nursing group of faculty of nursing in kashan medical university conducted an internal evaluation program by the request and supervision of education development center. I evaluated that program and its final report. At first content analysis was performed on the final report of evaluation, then program evaluations, meta-evaluation checklist developed by Daniel stufflebeam used to determine the quality of evaluation study at its utility, feasibility, propriety and accuracy. According to the checklist an evaluation may fall in one of the categories of excellent, very good, good, fair or poor. At the end I used semi-structured interviews with the members of evaluating team for exploring the process and obstacles of the program. Results. The history, context, structure, mission and objectives of the group was not considered neither in the process nor in the final report. The evaluators did not determine the goals and objectives of evaluation they only listed some factors they thought needed to be evaluated. There was any planning for evaluation therefore the process of evaluation was not clear in the final report. No judgments made relating the findings or results and the final report did not contain any discussion and recommendations. According to the program evaluations meta-evaluation checklist, utility, feasibility and propriety of evaluation were fair and its accuracy was poor. Analysis of interviews revealed that the evaluators did not have necessary preparation and they were not ready enough to do this, they did that program only because the EDC hd wanted them to do that. Fear from uncovering weaknesses, lack of mastery in program evaluation in the group and although in EDC, lack of a skilful director, inability of EDC to explain the purposes and benefits of internal evaluation, were some of the most important obstacles of this evaluation program. Conclusion. Although the role and importance of evaluation in improvement of actions and quality of programs are well known, but it is not executed properly in reality. Fear from the results, lack of skill in evaluation, and not integrating the meta-evaluation in evaluation programs are some of the reasons for imperfect and unusable evaluations.