Assess program effectiveness using the CDC framework.
Takeaways:
- Nurses can drive change through program evaluation to improve patient outcomes.
- The Centers for Disease Control and Prevention Program Evaluation Framework helps nurses identify program gaps and strengthen workflows.
Many healthcare organization programs promote a culture of safety and decreasing patient harm. These programs require systematic evaluation to assess their effectiveness and provide valuable data to stakeholders, which they can use to implement improvement initiatives and increase patient safety.
Continuous evaluation can help capture areas in need of immediate improvement, but which may not be feasible. An alternative evaluation approach allows stakeholders to examine who conducts program activities, how they’re conducted, and who benefits. A useful tool for this latter approach is the Centers for Disease Control and Prevention (CDC) framework for program evaluation. Here, I explain how I used this framework at my organization to evaluate an inpatient wound care program. After I shared pressure injury trends and nurse wound care knowledge with stakeholders, they used that data to modify the program’s technology and education activities to decrease pressure injuries.
About the organization
Mercy Medical Center Redding (MMCR), a 267-bed acute care nonprofit organization, implemented a patient safety initiative addendum to its inpatient wound care program. The original program resulted from a political, economic, social, and technological (PEST) analysis that showed a significant increase in hospital-acquired pressure injuries.
The MMCR inpatient wound care program evaluation aimed to determine its effectiveness based on specific outcomes—certification, education, and technology. The data collected during the evaluation prompted the organization to continue the program with modifications. Data included pressure injury numbers before and after implementation of new wound care nurse certification, technology, and education. The evaluation aligned with the organization’s overarching goal to promote a culture of safety, provided data important to the decision-making process, and aided standardization across multiple facilities.
CDC steps and standards
The CDC framework for program evaluation provides an effective road map for nurses because its steps and standards adhere to nursing’s goal of promoting clinical quality and patient safety. In addition, the framework emphasizes practical, continuing evaluation strategies that include all program stakeholders and is easily incorporated into the program’s daily operations. Depending on the organization and the program being evaluated, stakeholders can vary. In my evaluation I partnered with leaders and frontline staff. (See Steps and standards.)
I applied the framework strategically to avoid interrupting wound program activities while still meeting the standards. In addition, I asked questions at the beginning of the evaluation and throughout the implementation phase when assigning value and making judgments. (See Keep questioning.)
The framework’s steps can be customized to meet the needs of a specific program, its stakeholders, and the organization. The steps also can increase multidisciplinary awareness and collaboration. (See Framework steps.)
As I applied the framework, I included stakeholders in the process and updated them with weekly communication. This open communication enhanced stakeholder engagement, collaboration, and transparency. It also increased my credibility. (See CDC framework in action.)
Evaluation report
The evaluation report describes the who, what, how, and why of the program evaluation. It communicates the nurse evaluator’s findings and recommendations to help stakeholders determine program improvements.
The final step in the CDC framework—to ensure use and share lessons—requires evidence dissemination and a description of how the nurse evaluator performed the evaluation. The report format can vary based on stakeholder preferences. For example, I presented the MMCR inpatient wound care program’s evaluation report in an executive summary along with a PowerPoint presentation for a more detailed discussion. My report included the overarching goal, the need, the evaluation purpose, inpatient wound care program background, data collection, data analysis, results, conclusions, and recommendations. (See Key elements of the evaluation report.)
Meta-evaluation
Meta-evaluation reviews the methodology and findings of the evaluation report for accuracy, quality, and consistency. It can be used to improve evaluation processes after design and implementation. A meta-analysis can be conducted internally or externally, and the stakeholders and evaluators can determine the design.
The MMCR stakeholders conducted the meta-analysis to review the quality of the program evaluation standards (utility, feasibility, propriety, and accuracy). They used a Likert scale disseminated via SurveyMonkey to inpatient wound care program stakeholders and documented the evaluation process. Results of the Likert scale analysis determined the evaluation met utility, feasibility, propriety, and accuracy standards as evidenced by the following results: 86.67% very high quality and 13.33% high quality.
Results and recommendations
Nurse evaluators should discuss the motive of a program evaluation with the stakeholders. The subject can be a program, service, process, project, or condition and can focus on different aspects such as inputs, outputs, efficiency, effectiveness, and outcomes. The results can compare outcomes from previous years, compare actual outcomes with intended outcomes, focus on program specifics, promote expansion of the program, or guide stakeholders to identify gaps in education, training, or technology.
The primary motive for the MMCR inpatient wound care program evaluation focused on assessing its effectiveness via specific outcomes. The results showed a 21.5% decrease in pressure injuries associated with certification and a 25.8% decrease associated with technology. The educational activities showed a 6.2% increase in nursing wound care knowledge. My recommendations, based on the evaluation, allowed the stakeholders to research new wound care technology and promote nursing educational opportunities.
Quality tool
The CDC provides many resources to guide nurse evaluators through the framework for program evaluation process. The framework helps nurse evaluators modify organizational programs that affect their daily workflow and patient care, which promotes patient safety and optimal outcomes.
Carrie M. Fisher is a director at Adventist Health in Hidden Valley Lake, California. At the time this article was written, she was a nurse manager at Dignity Health in Redding, California.
References
Boulmetis J, Dutwin P. The ABCs of Evaluation: Timeless Techniques for Program and Project Managers. 3rd ed. San Francisco, CA: Jossey Bass; 2011.
Centers for Disease Control and Prevention. Developing an Effective Evaluation Report: Setting the Course for Effective Program Evaluation. 2013.
Centers for Disease Control and Prevention. Framework for program evaluation in public health. MMWR. 1999;48(RR-11).
Centers for Disease Control and Prevention. Program performance and evaluation office (PPEO). April 9, 2021.
Harris J, Roussel L, Thomas P, Dearman C. Project Planning and Management: A Guide for Nurses and Interprofessional Teams. 2nd ed. Burlington, MA: Jones & Bartlett Learning; 2015.
Stufflebeam D. Meta-evaluation. J Multidiscip Eval. 2010;7(15):99-158.
Key words: Problem identification, Program evaluation
