Increasing ASQ-3 Child Developmental Screenings: Quantity and Validity

Summary

Impact Statement:

Early intervention has been shown to positively impact outcomes across developmental domains. The Fond du Lac County Health Department increased the number of children being screened for developmental delays from 5 to 9 per month, allowing more children to be appropriately connected with services to enhance their early childhood development through early intervention and minimize potential developmental delays.

Summary:

Fond du Lac County Health Department’s (FCHD's) Maternal Child Health (MCH) program participates in the Wisconsin Healthiest Families Initiative (WHFI), in which local health departments work with community partners to build an integrated system that promotes optimal physical, social-emotional, and developmental health of children and their families. Mental health is the focus area in this initiative, and FCHD is a partner in the Fond du Lac County Community Partnership for Young Children (Supporting Positive Relationships so Our children Under 6 can Thrive [SPROUT]), a coalition whose goal is for all Fond du Lac County children to be healthy, nurtured, safe, and successful from birth to school entry.

As a result of this public-private partnership, Fond du Lac is the first county in Wisconsin to offer a community-wide developmental screening site that meets the critical need of improving quality early childhood development. The screening site, unveiled in July 2013, is located at the Fond du Lac County Women, Infants, and Children (WIC) office and offers parents the opportunity to select and complete age-appropriate Ages & Stages Questionnaires™ (ASQ-3). The questionnaires are scored, and parents are notified of results, provided with activities to do with their children, and given recommendations for follow-up. In the process of data entry of screening results, it was discovered that since the initiation of the screening site, nearly 29% of the screens returned and scored were invalid because of incorrect screen selection. As a result, inappropriate referrals were made to Birth to 3, and the validity of the screen results of children who scored in the “above cut-off” range when the wrong screen was used was questioned. This is a problem, as inappropriate referrals made to an agency waste staff time, and can damage the referring agency’s credibility. And, although a child may score “above cut-off” on a screen, if the screen was chosen incorrectly, the child could be experiencing unidentified delays.

What we desired to accomplish with this project was to reduce the percentage of invalid developmental screens completed at the FCHD developmental screening site, thus preventing inappropriate referrals and missed opportunities in identifying children with potential delays, and to increase the number of screens being completed by parents and returned to FCHD for scoring.

Organization that conducted the QI initiative:

FCHD

Citation:

Mittelstadt-Lock, K. Public Health Quality Improvement Exchange. Increasing ASQ-3 Child Developmental Screenings: Quantity and Validity. Tue, 09/26/2017 - 11:40. Available at http://www.phqix.org/content/increasing-asq-3-child-developmental-screenings-quantity-and-validity. Accessed April 18, 2024.

Background and Aim

Aim statement:

By March 14, 2014, reduce the number of invalid ASQ-3 developmental screens for Birth to 60 Months from 29% to 0% and increase the number of screens completed from 5 to 10 per month by April 30, 2014.

Need For The QI Initiative:

The ASQ-3 screening site at FCHD was unveiled in July 2013. It was thought that if staff built the site, parents would complete a screen, their child's developmental needs would be identified, and referrals would be made to appropriate sources. In actuality, the number of screens completed fell far below the anticipated numbers, and some of those completed, 29%, were not completed according to the guidelines of the tool and therefore were deemed invalid.

Area for Improvement:

Improvement in this process will affect the programmatic areas of MCH and WIC, the children who are screened (WIC has an average enrollment of 1,600 children from birth to 5 years), and their parents. The improvement will affect the referral sources outside of FCHD, which include the Fond du Lac County Birth to 3 Program and local school districts. On a systems level, the improvement will indirectly affect the community (with approximately 7,200 children younger than age 5) by working toward ensuring that the goals of the WHFI and SPROUT are attained.

Root Cause:

Data gathered before the Kaizen event indicated that forms were being taken from the display but not returned, and approximately 29% of those that were returned were not valid. It was determined that it is extremely difficult to determine which screen to complete (it is based on the child's age), which results in the wrong screen being completed. Also, surveys of other county staff indicated that it was not clear what the screens at the display were for; evidently, not all parents knew it was something that every parent was encouraged to participate in. It was determined that FCHD staff needed to be clearer about what ages each screen was for and to encourage parents to seek assistance from support staff when choosing a screen. The ideal process is to use the ASQ-3 calculation tool provided on the website, but there is no computer or staff member near the display to help access the tool. Staff also determined the need to reduce the clutter around the display and to provide a title or heading to let parents know the benefits of participating in the screening process.

Planning and Execution Details

QI Implementation:

Before initiating the quality improvement (QI) project, the QI leader attended two Kaizen training sessions provided by Continual Impact LLC. To prepare for the event, some data were gathered regarding how many screens were provided, how many were taken, how many were valid, and how many resulted in a referral made for follow-up. FCHD staff attempted to obtain customer feedback from parents who had completed screens but did not get any of those returned. For the project, a consultant from Continual Impact LLC, Chris Schucker, worked in-house with the QI leader to facilitate a week-long Kaizen event. During the event, the team did process mapping using the value stream map, Gemba visitation, customer feedback interviews, brainstorming, 5-Whys, and prioritization matrices for issues and solutions. Staff continue to use the Continual Improvement System (CIS) to gather input and post progress. Before the event, WIC staff, including interpreters, would verbally introduce parents to the ASQ-3 screening site and would sometimes assist them in choosing a screen for their child. Neither staff nor parents used the calculation tool on the website. Changes made to the display resulted in more screens being taken, and FCHD staff reached the goal of doubling the number returned 1 month after the event. However, the invalid rate was still high at 30%. It was recognized that it is not possible to have a self-serve site without having access to the website with the calculation tool. Staff secured an iPad and have it mounted at the display for parents and staff to use. Staff are implementing additional indicators for the parents to make the imperative time-sensitive nature of the screens clearer. Staff will do this by stamping "Date taken:_____" and "Complete and return in 2 days" on each screen. Through conversations during a CIS meeting, it was also determined that the validity of the screen needs to be evaluated because some screens that were deemed invalid may actually hold valuable information about the child's development.

Focus activities

Maternal and Child Health Services:

Early intervention services for children

QI Methods:

Kaizen

QI Tools:

Brainstorming

Cause and Effect Diagrams

Five whys

Prioritization Matrix

Process maps

Root cause analysis

Surveys

Other QI tools:

CIS

Value stream map

Gemba visitation

QI Tool Documentation:

Prioritization Matrix for Proposed Solutions

Developing Solution for each Issue

Prioritization Matrix for Proposed Solutions

Value Stream Map

Continual Improvement System

Process Mapping

Initiative Dates:

2/1/14 to 3/14/14

Initiative Duration:

Less than 6 months

Methods of evaluation:

Staff will continue to collect data on the number of screens received each month, how many of those are invalid, the number of screens taken from the display each month, and the number of referrals made from the valid screens. The MCH coordinator collects all the data monthly. See the PrISM for details of results.

Other Information:

One subject matter expert (SME) was absent because of illness, and her substitute was able to be in the event only during sporadic intervals. FCHD staff feel that valuable insight into the process was lost by not waiting until that SME could be present. Because most of the QI process is done in the 1-week event, it is crucial to have all team members present. Kaizen methodology and skills go well beyond an event—staff find themselves identifying wastes, finding value-added steps in all their processes, and using "working with others" principles in everything they do now.

Supplemental Materials:

Initiative photo(s)

PrISM

Supplemental Files:

Attachment	Size
ASQ 3 Display_After Pic.docx	884.43 KB
Kaizen Event 006.JPG	2.79 MB
FDL Project Team PrISM _FINAL.pdf	1.41 MB

Results

Measurable QI Outcomes:

We began collecting data in June 2013 when the ASQ-3 screening site was set up. Fourteen screen were received between June 2013 and September 2013, and 4 of those were invalid, a 29% invalid rate. (Invalid means the parents completed the incorrect screen for the age of their child.) The results after the Kaizen project are as follows: Mar-April 2014-10 screens received, 3 invalid, 30% invalid rate; April-May 2014-8 screens received, 3 screens invalid, 37.5% invalid rate; May-June 2014-4 screens received, 1 invalid, 25% invalid rate; Jun-July 2014-7 screens received, 0 invalid, 0% invalid rate; July-Aug 2014-3 screens received, 0 invalid, 0% invalid rate. Two initiatives were instituted during this time as a result of the CIS. On June 16, 2014 books donated books were offered to parents who completed a screen, and on July 28, 2014 an ipad with the ASQ-3 Calculation Tool website was mounted at the site. The goals were to increase the number of screens completed and turned in per month to 10 and have 100% of the screens returned valid. Two months after the Kaizen event and the changes had been made to the display, staff were still seeing high invalid rates of 25%–37%. On the basis of those data (see PrISM for result details), staff determined the need for an iPad, with the ASQ-3 age calculator, at the display with the ASQ-3 screens in order to assist parents and staff in determining the appropriate screen, because the changes that had been made to the display were not effective in increasing the validity rate. Three months after, staff reached the goal of 100% of the screens received being valid. Staff will continue to use the CIS to achieve and maintain the goal of receiving at least 10 screens per month.

Other QI Outcomes:

During the prioritization of solutions, staff decided to implement the solutions that could easily be completed and had low cost. One solution that was not implemented was putting a computer with access to the ASQ-3 website at the display. The data collected after the event indicated that there was still not improvement toward reaching the goal of 100% validity, because parents cannot choose the correct screen 100% of the time, no matter how many directions are given, without the calculation tool. Without going through the process and getting the feedback from others, it would have been difficult to justify the cost of a computer being placed at the site. Staff are trying to make a model site that can be replicated throughout the state and country, so this finding was crucial to know before sharing the process with others. One member of the Kaizen team, the "wild card," after being exposed to the processes of the WIC department via process mapping, determined that she wanted to be a part of that department and decided to change job duties and accept an open position that had been offered.

Lessons Learned, Observations and Insights:

Key successes for this QI project include being able to improve and modernize the developmental screening site within a short time frame. The QI findings were a bit of a disappointment because FCHD staff thought that after implementing all the solutions, they would consistently reach their goals. However, because they did not, they still feel that the QI process was beneficial in determining factors that did affect the process and provided the data to support implementing solutions that were not considered necessary when initially designing this one-of-a-kind display. Difficulties in implementing the improvement include lack of parent participation and staff time to assist parents in using the developmental screening site. Another difficulty was having the public attempt to use the iPad to surf other Internet sites besides the ASQ-3 age calculator. Staff had to work with the IT department to restrict access on the iPad. Through this screening initiative, staff are striving to match the community's resources with those children who need them to address infant and early childhood developmental delays. Staff will continue to increase the number of families reached by continually trying new incentives and continuing to encourage management to allow staff to consistently introduce the parents/guardians to the screening opportunity. Other health departments have expressed interest in duplicating the display to initiate the ASQ-3 screening in their jurisdictions. FCHD staff feel that this QI process has helped ensure that they will give these partners valuable information on how to be successful in that endeavor.

Future Plans:

By installing an iPad at the display site, FCHD staff have accounted for human error in choosing the correct screen. They have also started an incentive program in which the parent receives a free book for completing a developmental screen. However, staff have not reached the goal of receiving 10 developmental screens per month. If another health department were to implement this early development screening initiative, FCHD staff would strongly suggest that their site not be a self-serve display. Personal introductions to the screen's purpose and benefits are evidently necessary to engage parents and give them the information they need to take the time to complete a screen for their child. Staff strove to set the site up as self-serve to conserve staff time, but this approach is still overlooked and not taken advantage of by the public that staff want to reach with this initiative. The CIS will be maintained in order to collect ideas and suggestions and to share data. The MCH department will continue to collect monthly data and share them on the CIS. Ideas to promote the developmental screening site further include Facebook promotion, having staff members close to the screening site for a few hours each week to provide education, encouraging parents to complete screens, and continuing to provide incentive offerings for parents. As of April 2015 we have successfully eliminated receiving invalid screens. We have not maintained the number that was originally set in our goal, however, the state target is for the Birth to 3 program to serve 2.84% of the birth to three population. In 2013 they were exactly at 2.84%. When they submitted their child count information in November of 2014, they were at 106 children compared to 99 children at their October 1 child count in 2013. These numbers were not looked at prior to this CIS. It is a good example of how all data contributing to a QI event must be evaluated prior to the event, and also how important the follow up CIS gatherings are to continually look at the issue. So, our developmental screening initiative is just one more great example of what we are doing as a community to assure that children in need of early intervention ARE being identified and served. If the Birth to 3 Program Director sees a decline below state target they will notify the MCH Coordinator to promote the screening tools.

Training and Preparation

Information about the Community

Population Size:

100,000 to 249,999

Population Characteristics:

Incorporated on April 1, 1839, Fond du Lac County currently has a population of approximately 101,843. As of the 2010 census, 43,910 households and 26,994 families resided in the county. The racial makeup of the county was 94.1% white, 1.3% black or African American, 0.50% Native American, 1.1% Asian, 0.00% Pacific Islander, 1.7% from other races, 1.3% from two or more races, and 4.3% Hispanic or Latino of any race. There were 11,384 households (25.9%) that had children younger than age 18 living with them. Of the households, 53.5% were married couples living together, 7.80% had a female householder with no husband present, 31.00% were nonfamilies, and 25.8% had someone living alone who was older than 65 years. The average household size was 2.41, and the average family size was 2.93. In the county, the population was spread out with 25.5% from 0 to 19 years, 6.2% from 20 to 24, 24.8% from 25 to 44, 28.5% from 45 to 64, and 14.9% who were 65 years or older. The median age was 37 years, an increase of 1.09% from the 2000 census. For 2010, the total property valuation was $6,922,776,000. The total land area is 725 square miles, with 207 center line miles of state highway and 384 miles of county highway. There are approximately 1,640 farms, with nearly 280,000 cropped acres (2007 Census of Agriculture). The average farm size is about 204 acres. Among Wisconsin counties, Fond du Lac ranks fourth for total milk production, eighth for production per cow, and fourth for the number of dairy cows (2010 Wisconsin Agricultural Statistics Service). The county also ranks in the top 10 for grain production of wheat (second), corn (ninth), soybeans (ninth), forage production of corn silage (first), and alfalfa (seventh). Fond du Lac County is also a leading producer of commercial peas and processed sweet corn. With more than $290 million worth of products sold, Fond du Lac County ranks in the top 10 counties in cash receipts from farm markets. With its landscape of rolling countryside, lush forests, and sparkling lake shores, Fond du Lac County offers endless opportunities for the year-round outdoor enthusiast. The city of Fond du Lac is on the shores of Lake Winnebago, which is Wisconsin's largest inland lake at 220 square miles. The county park system includes 11 county parks and a public hunting ground that together total more than 1,200 acres. The park system includes pristine wetlands, native prairie, woodlots, spring brooks, and lake shore. Park visitors may enjoy boating with overnight docking, windsurfing, swimming, camping, fishing, picnicking, hiking, and mountain biking. The county recreation trail system includes six year-round, multipurpose recreation trails, providing more than 50 miles of hiking, running, cross-country skiing, and bicycling opportunities. Additionally, 293 miles of state-funded snowmobile trails open each winter for local and long-distance snowmobiling. Fond du Lac County is located within an hour's drive of the major cities of Madison, Milwaukee, and Green Bay and is less than a 3-hour drive from Chicago. Additional tourist and attraction information is available from the Fond du Lac Area Convention and Visitors Bureau [(800) 937-9123 or www.fdl.com]. Fond du Lac County prides itself on the quality of its educational structure. Wisconsin traditionally ranks at the top of ACT scores for the United States, and several Fond du Lac school districts rank above average within the state. In fact, the Fond du Lac metropolitan statistical area is consistently among the top 25 public school districts in the nation. Higher education opportunities abound in Fond du Lac County, with opportunities at the University of Wisconsin–Fond du Lac, Marian University, Ripon College, and Moraine Park Technical College. These schools of higher education are providing talented human capital for existing and new companies. Historically named one of the top 20 safest metropolitan areas in the nation, Fond du Lac County is home to many top U.S. companies. These include Mercury Marine, MAG, JF Ahern, Brenner Tank, Saputo, the J.M. Smucker Company, and Alliance Laundry Systems. Fond du Lac County offers 10 public and private business and industrial parks, representing a variety of options for new and expanding businesses.

Increasing ASQ-3 Child Developmental Screenings: Quantity and Validity

Summary

Background and Aim

Planning and Execution Details

Results

Training and Preparation

Information about the Community

Information about the Health Department

QI Staff Information

Accreditation Details

Collaborations

Contact Information of the Submitter

QI Spotlight

quality improvement together.

Search form

You are here

Increasing ASQ-3 Child Developmental Screenings: Quantity and Validity

Summary

Background and Aim

Planning and Execution Details

Results

Training and Preparation

Information about the Community

Information about the Health Department

QI Staff Information

Accreditation Details

Collaborations

Contact Information of the Submitter

QI Spotlight