“ Implementation process evaluation ” links “ dissemination ” with “ implementation ”

Bridge-It is an expert system that asks project directors 36 questions to assess readiness and estimate likelihood of implementation success for newly adopted school-centered health and education technologies. Because it is brief, content valid, and internally consistent, Bridge-It promises to be a useful tool. But in two recent evaluation studies—one tracking progress of tobacco prevention and control initiatives (n=93 schools) and another assessing an alcohol safety curriculum (n=50 schools)—approximately half of project directors answered ‘don’t know’ to more than 25% of Bridge-It’s questions. They ‘don’t know,’ for example, how important the new program is to the principal, whether there is opposition from parents, or whether school staff believe the new program is better than what was being done before. This paper describes the model, presents results, traces what happened to the evaluation reports, and reviews options for transforming ‘don’t know’ into useful feedback. Relevance and importance to the field: Hoping to close the huge gap between what is known from research and what is actually practiced in the field, program officers at Federal and State levels increasingly are urging providers to adopt and faithfully implement “evidence-based” programs and technologies for the delivery of health, education, and human services. This programmatic direction places renewed emphasis on the importance of the implementation process evaluation. By assessing readiness and tracking implementation progress, the process evaluation can play an important role in helping to reform systems. But, when results routinely indicate low readiness and no progress, the evaluation is at high risk of being “shelved”—i.e., results are unlikely to gain serious hearing when program decisions are made. This problem has plagued process evaluations based on Bridge-It. By presenting results and tracing what happened in two studies where Bridge-It guided the implementation process evaluation, this paper will help to inform discussion about how evaluation is shaped by the systemic context in which it is embedded. Authors: Cindy Roberts-Gray, Resource Network, Galveston, Texas Phyllis Gingiss, Department of Health and Human Performance, University of Houston Presenter: Cindy Roberts-Gray November 9, 2002 2:00-3:30 pm, Potomac I, Hyatt Regency Crystal City Contact Information: [email protected] Resource Network 3002 Ave O 1⁄2 Galveston, TX 77550 Phone: 409/762-2499 Fax: 409/762-2489 Roberts-Gray, C. & Gingiss, P.M. (2002). “’Don’t know’ responses: Do they inform or undermine the process evaluation.” Houston, TX: University of Houston. Proceedings of the American Evaluation Association Conference, Washington, DC, November 2002. Evaluation can tighten links between research and real world results Researcher roles Developer roles Provider roles Consumer roles