Electrifying the Adolescent Pregnancy Prevention Program Evaluation
The North Carolina Division of Public Health's (NC DPH's) Adolescent Pregnancy Prevention Program (APPP) funds more than 20 community partners to implement evidence-based teen pregnancy prevention programs. After programming is complete, each site receives an evaluation report summarizing its outcomes for the year. The impetus for the quality improvement (QI) project was that the total turnaround time for the evaluation process was 24 months, with a total of 36 process steps. Local partners were receiving their evaluation reports almost a year after data were submitted, which made it almost impossible for them to use the report for program improvement efforts.
APPP staff partnered with a funded site to complete a Gemba walk and value-stream map. Based on the change ideas and impact matrix, staff completed the following Plan, Do, Study, Act (PDSA) steps:
• Created an APPP manual to clarify the evaluation process and other APPP requirements
• Developed annual APPP orientation for newly funded sites
• Modified the survey submission form for paper/pencil data collection
• Piloted electronic data collection
• Required electronic data collection in new Requests for Applications
• Proposed a new timeline for the evaluation process and evaluation report format
Our proposed future state will
• reduce the number of overall process steps from 36 to 13, a 63.8% improvement; and
• reduce overall process time from 24 months to 18 months, a 25% improvement, enabling local agencies to use the reports for program improvements.
One of the biggest challenges of this project is that staff cannot change programmatic requirements of funded sites midway through their 4-year funding cycle. Given this limitation, APPP will not transition to exclusively electronic data collection until fiscal year 2018 (June 1, 2017–May 31, 2018). Because electronic data collection is the primary mechanism for reducing process steps, the future state will not be fully realized until May 31, 2018.
Loper, A. Public Health Quality Improvement Exchange. Electrifying the Adolescent Pregnancy Prevention Program Evaluation. Fri, 02/27/2015 - 09:31. Available at https://www.phqix.org/content/electrifying-adolescent-pregnancy-prevention-program-evaluation. Accessed January 23, 2021.