• By Kevin Mount
  • Posted on Tuesday 14th October, 2008

Randomized trials, certain benefits

Randomized controlled were introduced in the 1930s, with undergraduate students at Purdue University in the US as the participants. The first trial focusing on children’s well-being, which began in 1936, was intended to find out the effects of social care support on young offenders (it made outcomes worse). Education and youth justice were among the pioneers of RCTs, but they were soon a rarity in both those quarters. In medicine, on the other hand, they became routine.Why so? In their Journal of Children’s Services editorial Carole and David Torgerson point to the frequency of pharmaceutical mistakes as one explanation. Routine oxygen supplementation of premature infants was found to cause blindness. Administering Diethylstilbestrol to prevent miscarriage resulted in an increase in cancers in mothers and children. At least in health we know how long is the list of disasters.In time, at least two robust RCTs became required before a new pharmaceutical product could be licensed. Drug companies invested in the technology and the methods and ideas gradually spilled over into other areas of medicine. As the number of trials increased, systematic reviews emerged as a way of making sense of the evidence.Not all disciplines have followed the medical example. Questions that have a political dimension such as youth justice, the prevention of drug abuse and teenage pregnancy are, according to the Torgersons, the most resistant to agnostic study. But the strongest opposition has been from within certain academic communities. Badly run or contentious trials or results from well run trials that produce, from an academic perspective, the “wrong” result are often paraded as evidence of the dangers.In too many cases the question is not how to evaluate but whether to evaluate at all. The Torgersons, who are the founders of the York Trials Unit, describe policy after policy in the UK in which a simple experiment would have provided good estimates of impact but which have gone entirely unevaluated. To be useful RCTs must be done well. Too many are not. The Torgersons set out a shopping list for the careful buyer. All participants in the trial should be randomized. The sample needs to be sufficiently large to detect effects and to exclude important differences between groups. All participants should receive the correct “dose”. Outcomes should be assessed by people who do not know which of the participants are members of the program or of the control groups.Lack of transparency about these and other attributes has led to the formation of the CONSORT group who are responsible for quality guidelines stipulated by leading medical journals and by the American Psychological Association. So far, social science publications lag behind.The production of good quality trials in the health world is sometimes said to be due to the more straightforward prevailing conditions. Requiring people to pop pills or placebos may seems simple compared to getting parents and children to participate in a trial where, for example, some might get help with reading.Not true on either count, say the Torgersons. There are plenty of poorly designed health trials and the task is no easier. Consent for pharmaceutical trials is harder to come by than for a parenting program. Health practitioners do not routinely follow up patients, unlike teachers their students. Administrative data such as education tests and criminal records that can be used as outcome measures, are more readily available. In health pre-test data is elusive. In other fields it is abundant.In some respects, RCTs are needed more for social interventions because much of what they relate to is mandatory – as in the case of national curricula – or part of a limited menu of offerings – as in the case of programs offered by children’s services agencies. Should novel interventions prove harmful (and there is plenty of evidence to show that many are so badly flawed that it is a danger), then multitudes of often already disadvantaged children will be exposed to them. ReferencesTorgerson, D J and Torgerson, C J (2008) Designing Randomised Trials in Health, Education and the Social Sciences, Basingstoke, Palgrave Macmillan.Torgerson, D J and Torgerson, C J (2008) “Invited editorial: randomised controlled trials in children's services”, Journal of Children's Services 3 (1), 2-8.

Back to Archives