Evaluating Home Visiting Programs in Pennsylvania: How Are They Faring?

Statement of Problem

In the last two decades, home visitation programs targeting underserved and high-risk families of young children have flourished.  Although the strength of evidence differs by the program model, maternal and child home visitation by trained professionals to the homes of high-risk families has demonstrated beneficial short and long term outcomes related to pregnancy and birth, child health and development, parenting, and improved parental self-sufficiency.  Accompanying a growing evidence-base for these programs has come strong federal support for home visitation.   In March 2010, $1.5 billion was invested in home visitation programs through the Patient Protection and Affordable Care Act (PPACA). This investment supports states’ growth of evidence-based home visiting services along with evaluation of home visiting performance following expansion.   The focus on evaluation is critical, as program impacts may be weakened by implementation challenges across a diversity of clients, staff, health systems, and geographies within and across states nationwide.   

  • Description
  • Next Steps

Evaluating Home Visiting Programs in PA: How are they faring? 

Program effectiveness increased considerably between the initial implementation period (2000-2003) and later stages of program operation (2004-2005).

The Commonwealth of Pennsylvania has been a national leader in supporting rigorous evaluation efforts of state-funded early childhood programs. Building on a previous collaborative relationship with the Commonwealth, PolicyLab is partnering with the Office of Child Development and Early Learning within the PA Department of Public Welfare to evaluate the state’s home visiting system under the PPACA program.  PolicyLab will evaluate program effectiveness across four federally funded home visiting models and identify benchmarks towards an enhanced and sustainable performance monitoring system for home visitation programs in the state.  Maternal and child indicators will be examined both at a community-level for those communities served by home visiting models as well as within priority subgroups of vulnerable families, including young mothers, child welfare involved families, dual language families, and families of children with disabilities.  Additionally, the evaluation will feature a large qualitative component aimed at describing barriers and opportunities for program success as identified by families, home visitors, and program administrators.  The evaluation will be completed in March 2015.

Previously, from 2006-2012, the Department of Public Welfare collaborated with PolicyLab to conduct an evaluation of the Nurse-Family Partnership (NFP) program across the state.  This evaluation employed a novel case-comparison design in order to examine program effect across replication sites for the outcomes of birth spacing, prenatal smoking cessation, and child injury.  The goal of the PolicyLab evaluation was to gain a clearer picture of program success following dissemination across the state. The results of the PolicyLab evaluation indicated that while some sites were achieving positive outcomes, variation existed in performance across sites and outcomes, signaling a need for continued performance monitoring.

Following this evaluation of all NFP clients in PA, a second study was conducted to determine whether the program impacted birth spacing for Latina clients. Given the diversity of the Latino community, the evaluation focused on two subgroups (Mexican, Puerto Rican) as well as two maternal age groups (18 years and younger, over 18 years). The study found that the program had a small but measurable impact, predominantly among adolescents. The data also suggest that the program was particularly effective for women of Mexican heritage, although our results were not conclusive.

The home visiting portfolio has already shaped a number of federal and regional responses to the needs of home visiting programs. The application guidelines for the federal Maternal, Infant, and Early Childhood Home Visiting (MIECHV) home visiting grants to states by the Department of Health & Human Services noted our evaluation as an example of the standards of evaluation the government was seeking. We have also worked closely with our home visiting partners, particularly the Nurse-Family Partnership program, to guide quality improvement within their programs as a response to our data. We hope to generate knowledge for use in guiding quality improvement efforts by discerning program, community, and implementation factors related to successful outcomes.

Suggested Citation

The Children's Hospital of Philadelphia, PolicyLab. Evaluating Home Visiting Programs in Pennsylvania: How Are They Faring? [Online] Available at: http://www.policylab.chop.edu [Accessed: plug in date accessed here].