Eileen Munro today published the third and final part of her review of child protection for UK Government. She writes about the value and challenges associated with evidence based programmes, and is cautious about their value. The report includes a large quote to some notes that I prepared with my colleagues David Jodrell and Seden Karakurt about evidence based programmes in the context of child protection. I thought it might be useful to blog that submission in full. Here it is.
Much attention has been given in recent years to the place of evidence based programmes in children’s services. These are prevention, early intervention or treatments that are proven at a high standard of evidence to improve children’s health or development.What counts as an ‘evidence based programme?’ For the purposes of this brief overview, the same standards of evidence applied in the Allen Review of Early Intervention have been adopted. These standards, developed by an international team of experts, are not written in stone. But they provide a ready tool to assess the variety of potentially evidence based programmes relevant to the task of protecting children from maltreatment. This process selected programmes that were supported by at least two robust evaluations in which the effects on those receiving the intervention were compared to a control group. One of the evaluations for each programme included was a randomised controlled trial. The search focused on prevention, early intervention and treatment programmes that have the potential to:• reduce the risks of child maltreatment• reduce maltreatment.The list did not extend to programmes that reduced the effects of maltreatment, such as emotional and conduct disorders, or other mental health problems consequent upon maltreatment. (There are many examples of evidence based programmes in this category). The following summary also drew on meta-analyses or systematic reviews that bring together many rigorous evaluations. The paper by Christopher Mikton and Alexander Butchart for the World Health Organization is an exemplar.
The following attached table summarises the types of programmes that exist, and whether there is evidence of impact on risks of maltreatment or the level of abuse.*Intensive parenting programmes focused on child protection reduce maltreatment 1) Community/Public health strategies Two programmes exemplify this category. Much attention has been given to the delivery of all five levels of the Triple-P parenting programme in South Carolina in the United States. This begins with media strategies to change parenting behaviour in the general population but also provides intensive group based training for parents of children who are known to child protection agencies. This strategy has produced significant reductions in abusive behaviour and referrals to child protection. The Safe Environment for Every Kid programme better prepares people living in disadvantaged communities to identify and respond to child maltreatment, and to work more closely with public agencies such as social work. 2) Health visiting programmes The class of evidenced based programme with the greatest impact on child maltreatment involve intensive and enduring work by health visitors with parents whose newborns are at most risk of poor outcomes. Family Nurse Partnership, being delivered in the UK with fidelity to 6,000, soon to rise to 12,000, vulnerable, mostly teenage lone mothers, has the strongest pedigree. Among a long list of benefits to child and mother is a reduction in reported child abuse and neglect of 48% by the time the child is 15 years old. 3) Early Years support There has been relatively little attention given to the contribution played by early years provision, for example Sure Start Children’s Centres, to reducing child maltreatment. Evaluations of an early version of this approach, the Chicago Child Parent Center, report positive findings. For example, the lower incidence of child abuse leading to higher parental involvement in the child’s development explained about a quarter of the variance in reductions high school completion and juvenile arrest attributed to the CCPC programme. 4) Parenting programmes There is good evidence that proven parenting programmes like Incredible Years and Triple-P, both widely applied in the UK, reduces the risks of maltreatment, and some evidence of reductions in actual maltreatment. The Triple-P universal programme, for example, applied in a community of 100,000 people would result in nearly 700 fewer child maltreatment cases, 240 fewer children coming into foster or residential care and 60 fewer injuries. These generic parenting programmes are generally delivered to families with a child experiencing impairments to health or development, a conduct disorder for example. More specialised programmes, such as Parent-Child Interactive Therapy, have been delivered to families known to have abused their children, and with good results. For example, in a US evaluation, more than two years after intervention, less than a fifth of families in the child protection system receiving PCIT had been re-reported for maltreatment compared to nearly half receiving services as normal. 5) Therapeutic modelsA wide range of therapeutic models have been evaluated and shown to have some impact on child abuse outcomes. For example, the Infant-parent Psychotherapy and Psycho-educational Parenting programmes have proven impact on attachment in families where the parent is known to have abused their child. Trauma-focused Cognitive Behaviour Therapy (CBT) has been used with some effect with children who have been badly maltreated.
The same high quality evaluations used to find out whether or not an intervention works can be used to indicate what does not work. That is to say, some programmes, designed with good intentions, result in negative effects, such as more children being abused or reported to child protection agencies.Just as a positive evaluation does not imply a programme must be used, a negative finding does not indicate that a programme must be axed. The findings suggest a direction of travel or encourage further testing and exploration.Generally speaking, there are less than promising results from interventions that focus on keeping together families where there is high risk of child maltreatment. (This is not to say that such efforts should cease, just that impacts on child maltreatment are, at best, likely to be mixed).For example, Homebuilders, an intensive family preservation service delivered in several US states, produced impacts on child maltreatment and placement away from home that were initially promising but soon tailed off.Family Group Conferences also deserve a mention in this category. There have been many studies of family group conferences, but only two that met the exacting standards applied by the Allen Review Team. These evaluations show, at best, mixed results but also evidence of iatrogenic effects on child abuse and maltreatment that correspond with the findings from intensive family preservation. Again, these results do not definitively say that Family Group Conferences are damaging. However, more reflection and evaluation is required.
Any rapid review of evidence based programmes runs the danger of selecting a short list of interventions and giving the impression that they have the potential to eradicate the problem in hand. Evidence based programmes are not a cure-all for child maltreatment or any other aspect of child development. The list of challenges is long, but a few points are sufficient to inject caution. First, because other countries put a higher value on experimental evaluation methods in the context of children’s social needs, many of the proven models come from outside the UK. This does not invalidate them any more than Microsoft computers should be invalidated because they were invented in the US. But it does urge caution and re-testing to ensure that the ideas travel well. (Most partnerships doing this work in the UK are finding that, so far, the programmes do travel well). Second, the relative absence of rigorous evaluation in the UK means that it is simply not known whether home-grown interventions are effective. It may be that UK boasts many more effective responses to child maltreatment than North America, Australasia or Scandinavia. Third, if evidence based programmes are not delivered with fidelity, that is to say if they are not delivered as they were intended to be delivered, with the correct levels of training, coaching and adherence to the manuals, they seldom achieve their intended benefits. Moreover, a proven model can be damaging when delivered badly. Getting the right staff, to deliver the right programmes, to the right people in the right way has proven elusive for some UK agencies. Fourth, there is a difference between proving an intervention in trial conditions and seeing the effects at scale with several tens of thousands of children. Most evidence based programmes have little market penetration. It is for this reason that results from interventions like Family Nurse Partnership that is reaching seven per cent of eligible children within three years of its introduction to the UK are attracting so much attention. Fifth, mainstream systems, such as social care and the police, have little experience of delivering evidence based programmes, most of which depend on short-term, marginal funding. Getting systems ready for evidence based programmes and evidence based programmes ready for systems is fundamental to any progress in this area.
Evidence based programmes and child protection
Evidence based programmes will never be a panacea for problems of child maltreatment. As this brief review demonstrates, there are strategies known to reduce risks of child abuse but implementing those strategies is challenging. Moreover, supposing nothing was known to work, in the context of child abuse it would still be necessary to act. It would, however, seem sensible for managers of resources to have more access to information about what works in the context of child maltreatment, much as argued by the Allen Review in the context of early intervention. This information would never tell managers or practitioners what to do. But it would be an important point of reference during the allocation of scarce resources. Second, more could be done to test promising innovations in the UK to see if they deliver similar results to those rigorously evaluated in the US, Australasia, and Scandinavia. Many of the interventions referenced in the Munro Review, Re-claiming Social Work for example, would be expected to have child protection benefits and deserve to rigorous evaluation. Third, where evidence based programmes are being implemented, there should be the proper attention to fidelity necessary to make them work. With the exception of Chicago Child Parent Centers, where a robust quasi-experimental design gives indications of impact of Children’s Centre type provision on child maltreatment outcomes. Mikton, C. & Butchart, A., ‘Child maltreatment prevention: a systematic review of reviews’, Bulletin of the World Health Organization, 2009, 87, 353-361 See Little, M., Proof Positive, Demos, 2010 for full exposition of this challengeBack to Blogs