Chapin Hall

Measuring the Impact of Early Home Visiting Programs

Early home visiting programs aim to promote maternal well-being and healthy child development. However, different home visiting models measure different outcomes to determine their effectiveness, making it difficult to compare or evaluate combined impact. To address this gap, the Pew Home Visiting Data for Performance Initiative (Pew Initiative) established a common assessment framework. Chapin Hall conducted a feasibility study to understand the barriers states face to acting on the Pew Initiative’s framework.

What We Did

We pilot tested the Pew Initiative’s nine indicators and 16 descriptive factors in six states/large counties: Connecticut, Iowa, Kansas, Los Angeles County, Massachusetts, and Oklahoma. Each site gathered retrospective participant-level data for home visiting program participants during a 12-month period. We asked the sites to capture as many of the Pew Initiative performance indicators and descriptive factors as possible.

What We Found

Our partner sites were only partly successful in capturing the Pew Initiative performance indicators and descriptive factors:

• One or more of the pilot sites captured the majority of nine home visiting system performance indicators in their retrospective samples. None of the sites collected data for all nine indicators.
• Pilot sites provided more consistent and reliable participant-level data for Pew’s 16 descriptive factors than for the nine Pew performance indicators.
• Sites faced significant barriers in accessing administrative data maintained by other agencies, like healthcare and child welfare.
• All pilot sites struggled to provide accurate descriptions of the home visiting service delivery process, including the number of home visits provided each participant, supervisor-to-home visitor ratio, and average caseload.

There was also some variation in data collection across sites, including differences in how each pilot site defined certain outcome variables.

What It Means

Overall, the Pew Initiative framework will contribute to stronger programs and better coverage of all new parents, help inform the state planning process, and improve public policy. Results of the feasibility study led to revisions to the original performance indicators and descriptive factors. The study also points to a number of best practices for states using the Pew Initiative framework:

• Better tools and processes to share data across state agencies
• Enhanced staff training and ongoing supervision around data collection to maximize data quality and reliability
• Collection of high quality, raw data at the participant level and at several points in time
• Data matching techniques to improve impact monitoring while also minimizing data collection burden for staff of home visiting programs