Chapin Hall

Making Replication Work for Evidence-Based Early Childhood Home Visiting Programs

Proving a program that works is one challenge. Replicating it is another. The inability to scale-up programs, particularly ones that have been found effective through rigorous research, is a major frustration for policy makers. For early childhood home visiting programs, the solution may lie in organizational culture, aligned participant and provider goals, and cooperation within the local provider network.

What we did

Chapin Hall, in partnership with Mathematica Policy Research (MPR) received funding from the U.S. Department of Health and Human Services to evaluate the extent to which 17 organizations in 15 states successfully replicated one of five national early home visiting models – Nurse Family Partnership, Healthy Families America, Parents as Teachers, SafeCare, and Triple P.

We looked at how well each of the sites followed the best practice standards established by the relevant home visiting model. We also collected information about collaboration among local public and private agencies. To measure program implementation fidelity, Chapin Hall developed a fidelity framework that applied across all home visiting models in the sample and captured two important aspects of fidelity – structural fidelity and dynamic fidelity (See Table below). The resulting analysis provided a snapshot of the characteristics of almost 400 home visitors, 5,000 families, and 90,000 home visits.

What we found

The agencies we worked with consistently met many of the fidelity standards. However, they struggled to maintain caseloads and deliver services at the intended intensity. A number of factors seemed to affect implementation:

• Replication fidelity varied more by agency than by home visiting models. This points to the significance of agency and local service network characteristics in making replication work.
• Changes in national and local contexts influenced the order in which infrastructure-building activities were carried out and sustained.
• Local agencies with more positive views of inter-agency cooperation were more able to replicate, scale-up and sustain early home visiting programs.

What it means

Our findings indicate that organizational factors are key. Replicating high quality programs with fidelity requires a supportive environment, where program objectives are reinforced by  organizational culture and a community service network.

Documentation of participant characteristics and service experiences also needs to be built into everyday practice. Providers need to engage participants in setting service goals. This requires a positive provider-participant relationship and a shared commitment to finding the optimal fit between a family’s needs and a program’s potential.

Those designing home visiting models may need to rethink how they define and apply duration and dosage standards, in light of the difficulty programs faced in retaining participants.