Evaluation of a Family-Based Prevention Program and Its Online Adaptation

This paper compares outcomes of a family-based prevention program from its original in-person mode to an online mode in response to the COVID-19 pandemic. Celebrating Families!^™ (CF!) is a highly relational and experiential trauma-informed 16-week program designed to improve parenting skills, family functioning, and family relationships for families dealing with a multitude of challenges: multi-generational trauma; physical and mental health challenges; cognitive deficits; compromised safety (child maltreatment and family violence), and substance use problems. Parental substance use problems are highly prevalent in child welfare and have direct effects on child welfare outcomes. Due to the abrupt changes in service delivery brought about by the COVID-19 pandemic, CF! quickly developed an online version of its in-person program. Although telehealth has had a track record in medicine and psychology, the question remained whether social service programs could be adapted to the online environment. The research questions were: a) Does the online CF! program have comparable outcomes as the in-person version? b) Due to the reported health and service disparities for families of color prior to and during the pandemic, are there differential effects of the two treatment conditions by ethnicity?

Method

The study used a quasi-experimental design with a treatment group of online participants and a comparison group of in-person participants. Due to the abrupt need to begin online classes, matching participants in the two treatment conditions (in-person vs. online) was not feasible. The sample consisted of 99 families, 58 in person and 41 online, 65% of whom identified as Hispanic and 27% of whom listed Spanish as the primary language. The pro-social and strengths-based measures, available in both English and Spanish, included a Parenting Scale, Family Strengths and Resiliencies Scale, a Self-Assessment of Parenting Skills, and a set of questions specifically addressing learning from the CF! curriculum. Repeated measures (pre- and posttest) ANOVA analyses tested the hypotheses of within-group improvement over time in both the in-person and online groups, and between-group differences of the in-person and online groups’ scores as well as differences in outcomes by ethnicity and language spoken.

Results

Regardless of primary language or ethnicity, both in-person and online participants had statistically significant improvements in outcomes with small to moderate effect sizes. Mean pre- and posttest scores for Hispanic and Spanish-speaking participants were higher than those of other participants, although the differences were not statistically significant. The program completion rate was higher in the online classes.

Conclusions and Implications

The findings of this preliminary study showed that the online adaptation of a highly relational family-based program performed as well as the in-person program. This is important given the persistence of the pandemic. That the program in both treatment conditions was as effective for Hispanic families may have to do with the unique foci of CF! on individual and family resiliencies as well as the opportunities for cultural adaptation, important factors that make CF! relevant for vulnerable families across cultures.