Electrifying the Adolescent Pregnancy Prevention Program Evaluation


Impact Statement: 
Review of timely and accurate evaluation data is crucial for health departments to provide effective interventions to improve the health of their communities. The Women’s Health branch of a state health department used several QI methods and tools—including a Kaizen event, multiple PDSA cycles and lean approaches—to reduce waste and decrease the time needed to produce useful evaluation data back to local health departments implementing a teen pregnancy prevention program. This ultimately helps departments use the evaluation data to make their programs more effective.

The North Carolina Division of Public Health's (NC DPH's) Adolescent Pregnancy Prevention Program (APPP) funds more than 20 community partners to implement evidence-based teen pregnancy prevention programs. After programming is complete, each site receives an evaluation report summarizing its outcomes for the year. The impetus for the quality improvement (QI) project was that the total turnaround time for the evaluation process was 24 months, with a total of 36 process steps. Local partners were receiving their evaluation reports almost a year after data were submitted, which made it almost impossible for them to use the report for program improvement efforts.

APPP staff partnered with a funded site to complete a Gemba walk and value-stream map. Based on the change ideas and impact matrix, staff completed the following Plan, Do, Study, Act (PDSA) steps:
• Created an APPP manual to clarify the evaluation process and other APPP requirements
• Developed annual APPP orientation for newly funded sites
• Modified the survey submission form for paper/pencil data collection
• Piloted electronic data collection
• Required electronic data collection in new Requests for Applications
• Proposed a new timeline for the evaluation process and evaluation report format

Our proposed future state will
• reduce the number of overall process steps from 36 to 13, a 63.8% improvement; and
• reduce overall process time from 24 months to 18 months, a 25% improvement, enabling local agencies to use the reports for program improvements.

One of the biggest challenges of this project is that staff cannot change programmatic requirements of funded sites midway through their 4-year funding cycle. Given this limitation, APPP will not transition to exclusively electronic data collection until fiscal year 2018 (June 1, 2017–May 31, 2018). Because electronic data collection is the primary mechanism for reducing process steps, the future state will not be fully realized until May 31, 2018.

Organization that conducted the QI initiative: 

Loper, A. Public Health Quality Improvement Exchange. Electrifying the Adolescent Pregnancy Prevention Program Evaluation. Fri, 02/27/2015 - 09:31. Available at https://www.phqix.org/content/electrifying-adolescent-pregnancy-prevention-program-evaluation. Accessed February 24, 2018.

Submission Status: 
In Progress
0 users have voted.
Link to the resource where this submission is also published: