What to Make of Family Preservation Services Evaluations
Francine Jacobs
2001
This paper discusses the difficulties that have characterized Family Preservation Services (FPS) evaluations. These difficulties arise out of four challenges: 1) FPS's multiple purposes and the variety in the forms it takes in implementation; 2) the lack of explicit "theories of change" that encompass these many goals and beneficiaries; 3) the challenges of defining FPS and replicating particular models; and 4) the fact that many evaluations have suffered because of lack of access to these data, technical expertise, time, money, and political and administrative support. Reviews of the evaluations have noted their methodological limitations, especially issues of research design and program implementation.
The evaluations have three waves. First, small-scale efforts focused on individual programs, then expanded to several programs in particular states. Second, larger scale, controlled studies that broadened the range of measured outcomes, in response to criticisms of earlier studies. Third, further refinements in evaluation questions, research design, outcome measurement, and analysis. The studies found family preservation services to date do not appear to have a significant impact on placement rates. FPS provides small-and, likely, short-term effects on placement prevention. Both FPS and its evaluations were repeatedly oversold, reflecting a broader context of scant resources. The paper concludes with some recommendations for conducting evaluations of this type of program and with observations on the relationship of evaluation or research to practice and policy.