• By Laura Whybra
  • Posted on Thursday 08th May, 2014

Why it’s hard to evaluate universal parenting programs

strong>Targeted parenting programs consistently prevent and treat child behavior problems – but the results for universal programs are less conclusive. A recent study in Wales of the universally offered Family Links Nurturing Programme shows why it’s hard to pin down effects.Extensive research shows that parenting programs targeting high-risk groups can prevent child behavior problems. In addition to being effective, parenting programs that treat individuals with early signs of behavior problems can also be cost-effective, saving money for health services and social services down the road. While we know that targeted parenting programs work, there is less evidence for the effectiveness of universal approaches. Is this because they don’t work – or because they’re remarkably tough to evaluate? A randomized controlled trial evaluation of one universal, group parenting program – the Family Links Nurturing Programme (FLNP) – was conducted between 2008 and 2011 by a team of researchers from the Universities of Warwick and Swansea.The study found positive effects of FLNP on parenting practices and several secondary measures. But the effects were not statistically significant, meaning that the researchers could not rule out that the positive results were the product of chance. It’s possible that FLNP just doesn’t work as well as targeted parenting programs such as Triple P and Incredible Years. But it’s also likely that larger sample sizes are needed to detect effects in a universal population, where many families are doing fairly well to start with. Evaluating FLNPFLNP is a 10-week manualized program that aims to help parents understand and manage their own feelings and recognize the effect of their childhood experiences on parenting. It aims to reduce problem behavior in children. Using discussion and role plays, parents are taught positive behavior management, communication and relationship strategies, and ways to look after their own emotional needs. Two trained facilitators deliver the program to groups of 6 to 10 parents through weekly 2-hour sessions. The evaluation of FLNP took place in the catchment area of Flying Start Early Years Centers in four sites in South Wales. 286 parents with children aged 2-4 years were randomly assigned either to receive the program or to a comparison group. The study measured negative and supportive parenting based on observations of parent-child interactions, child behavior, and parent and child wellbeing. The study also analyzed the cost-effectiveness of the program. The study found no conclusive evidence that FLNP works. Though there was an improvement in parenting and child behavior, the difference between those who attended FLNP and those who did not was not statistically significant: the difference could have happened by chance. The common challenges of evaluating universal programs So should we rule out FLNP? Not necessarily. Although the evaluation of FLNP failed to find significant positive results, there remains a clear and obvious need to complement targeted parenting programs with universal preventive interventions. The evaluation of universal programs presents its own set of challenges – challenges that are worth understanding in the context of this study.First, programs that are offered universally at a population level often find it hard to get high levels of participation. In this study, more than a third of families who were randomly assigned to the program attended no sessions. Fewer than half completed the course. At the same time, families in the comparison group may seek out another parenting program. Here, almost 20% of the control group families attended a parenting program during the course of the study. When intervention families don’t get the treatment while control families do, it reduces the ability of a study to show the actual impact of a program. In this study of FLNP, however, a comparison between the control group and families treated “per protocol” – those who attended more than six sessions – found that the treated families did better on average, but the difference was not statistically significant.An even more critical issue is that it is often harder to detect average improvements for universal programs, as many participants not have clinically measurable problems in the first place. In targeted parenting programs, by contrast, participants are typically recruited because children are already displaying behavior problems or there is a dysfunctional relationship between the parent and child. For this reason, it is easier to detect improvements when studying targeted programs than universal programs like FLNP. Indeed, with this study, the sample size was based on that used in studies of targeted programs and was therefore probably too small to detect improvements in a universal setting. There is much value in implementing universal parenting programs. First, targeting families on the basis of a small set of risk factors doesn’t always identify the families who would benefit. Second, offering universal parenting programs can reduce the stigma that is sometimes associated with attending targeted programs. Third, ordinary “healthy” families may benefit from universal programs, too, even if these improvements are hard to spot with the sorts of clinical measures that are typically used. We cannot rule out the value of FLNP. Moreover, this study has highlighted some important challenges that arise when evaluating universal programs, which must be considered in future research.***********Reference: Simkiss, D.E., Snooks, H.A., Stallard, N., et al. (2013). Effectiveness and cost-effectiveness of a universal parenting skills programme in deprived communities: multicentre randomised controlled trial. BMJ Open, 3(8). doi:10.1136/bmjopen-2013-002851

Back to Archives