• By Laura Whybra
  • Posted on Wednesday 30th January, 2013

Good delivery delivers the goods #2

strong>Interventions are more effective when they’re carried out as intended. But how can program implementers keep tabs on how faithfully a program is delivered? In a new study, researchers test an easy-to-use measure of adherence to Brief Parent Training (BPT).Parents who said their instructor had stuck closely to BPT’s delivery protocols were also more likely to report better parenting skills after the program. The team of Oslo-based researchers, which included one of BPT’s developers, found that greater adherence by the instructor to the program protocols was related to less use of inconsistent discipline by parents, more use of appropriate discipline, and more use of positive parenting practices. What is adherence? And why does it matter?“Adherence” is the degree to which practitioners follow the protocols of the intervention – whether they go by the book. It’s one aspect of implementation fidelity, a broader concept that also measures whether the intervention has been delivered skillfully. Adherence matters because, over and over, delivering a program as intended has been shown to be key in achieving program impact. Indeed, developing reliable, inexpensive, and easy-to-use measures of adherence is especially important when an intervention is leaving the lab and moving into real-world use. When program developers are no longer involved in implementation, there is a risk that core components may be left out or delivered differently than originally intended. This lack of adherence more often than not results in poorer program outcomes. If delivered badly, the program may have little or no effect – or it may even be worse than doing nothing at all. A good adherence test not only monitors program delivery, but also encourages practitioners to adhere to an intervention’s core components.Measuring adherenceIn response to this need to measure program adherence, researchers at The Norwegian Centre for Child Behavioural Development based at the University of Oslo, Norway, developed an adherence tool for Brief Parent Training (BPT). BPT aims to improve parenting practices in families with children aged 3-12 who have either developed conduct disorder or are displaying its early stages. Parents complete the adherence measure at the end of the intervention. The questionnaire includes 28 statements to assess whether the core topics and components of BPT are being delivered. These questions cover key components such as positive involvement, skills encouragement, discipline, monitoring, and problem solving, and are rated using a five-point scale. The measure includes statements such as “I got practical advice on how to make my child cooperate better” and “We talked about how children learn best through use of praise and encouragement.”Parent reports of adherence are not perfect: parents may over-report the level of adherence when they have positive attitudes towards the practitioner. But unlike more reliable methods such as independent observation, parent reports are inexpensive and easy to use – important considerations in designing a tool that can be widely used.Assessing the impact: better delivery means better parentingTo assess the impact of program adherence on parenting practices and child conduct disorder, researchers used the adherence questionnaires completed by 92 Norwegian families upon completion of the BPT program. They analyzed the relationship between practitioner adherence and several post-intervention aspects of parenting.Results showed that adherence to BPT delivery protocol by practitioners predicted greater reductions in the use of inconsistent discipline as well as greater increases in the use of appropriate discipline and positive parenting. The relationship between adherence and child conduct disorder was less conclusive, although this may be partly due to the relatively small sample size. So what does this mean? Successfully testing an adherence measure such as this has a number of implications. First, it demonstrates a reliable method of measuring practitioner adherence to program protocol in an evidence-based parenting program – enabling meaningful and practical quality assurance of program delivery to take place in the future. Second, it provides even more evidence that greater program adherence results in greater program impact: parenting practices will improve more when practitioners of BPT ensure they are delivering the core components in the right way, according to tried and tested delivery protocol. Although these are positive results for a program adherence tool, further testing is required before its use can become embedded in BPT program delivery. Including evidence from practitioner reports and direct observations of BPT sessions would provide greater levels of detail about the validity of the tool and would also avoid bias in parent-reported results. In addition, it is important to remember that program adherence is only one part of program fidelity. Practitioner competence, not measured by this adherence tool, is another vital aspect of fidelity that has also been shown to impact on program outcomes. Reference: Kjobli, J., Bjorknes, R., & Askeland, E. (2012). Adherence to brief parent training as a predictor of parent and child outcomes in real-world settings. Journal of Children’s Services, 7(3), 165-177.

Back to Archives