ThinkBase

Evidence, Evidence, Evidence

Sara Fakhro

13 March 2013

Dr Ben Goldacre was recently commissioned by the Secretary of State for Education, Michael Gove, to examine the role of evidence in the education sector. Goldacre’s report, Building Evidence into Education, provides recommendations which include making research into effectiveness a routine part of life in education, empowering teachers to participate in research and disseminating research findings effectively.

Goldacre is a media-friendly academic, who devotes much of his energy to promoting greater scientific accuracy in the press and highlighting bad practice. He advocates the use of randomised trials to establish the success (or failure) of medical interventions and has campaigned for greater use of them in other policy areas, such as education. Health policy is, as far as possible, determined by evidence and if any non-qualified Ministers were to insist on the provision of ineffective treatments for, as an example, a national epidemic, there would be outcry. Why then, questions Goldacre, should education policy not be based on the evidence of ‘what works’?

Qualitative research is currently more prominent in education in the UK than randomised trials, whereas in the US randomised trials are frequently used in education, industry and international development. Although they can provide more accurate measures of success, there are what Goldacre refers to as ‘destructive myths’ surrounding the ethics of randomised trials. Some believe, for example, that when a class is divided to conduct a trial, one group will be at a greater advantage in their education than the other. What must made be clear, according to Goldacre, is that it would not be known before the trial which group, if any, would be at an advantage. Nor would any trial be run alongside a failed intervention.

Goldacre would have done well to include the exemplar of the US Department of Education’s Institute of Education Sciences, which created the What Works Clearinghouse (WWC) in 2002. WWC provides evidence and research for decision makers across the education sector. It includes a database of research and research reviews which users can access to view tested interventions and their outcomes. Interventions according to outcomes desired are also rated by effectiveness according to research findings. These are not the views of policy makers or election-focused politicians, but the findings of teachers and researchers - including both evaluation researchers and those running randomised controlled trials. Schools, school districts and educational programme developers can access these directly to inform any decisions. It has survived the reign of two different political parties and would make any ill-informed opinions on what works in education look foolish. Nobody would (or could) dare remove this body of evidence.

A UK equivalent of the WWC, the Educational Evidence Portal (EEP), also provides research from ‘reputable UK sources’, allowing users access to research reports, articles, data and statistics, practitioner guidance and policy documents, which meet a range of criteria set by the EEP consortium. This, as an example, could be used to provide (and promote) randomised trials in education, publishing the results alongside other relevant qualitative research.

Goldacre seems to be unafraid to criticise anyone and his credibility remains intact because so much of what he says is based on hard evidence. He points out that the evidence is available and that teachers are already on site and able to collect the evidence. Qualitative research has its place informing these trials and understanding why things do or don’t work. Finding out what works is something that should be easy to do and it doesn’t have to drain public resources or lose an election. It would appear that it is not the results of such trials that are in question in the UK, but the process and ethics of conducting them. As such, if the evidence of such trials were presented alongside the appropriate qualitative research, their value could be as substantial as Goldacre suggests and the cultural shift would happen, as it did in medicine, quite naturally.

A limitation of Goldacre’s report is a failure to make clear to Government and Gove (and, indeed, all political parties) that there is no reason or excuse for policy makers to keep imposing interventions which reflect opinion or political leaning. It has one fatal flaw, however: ‘If this all sounds like a lot of work, then it should do: it will take a long time.’ Well, that’s that then. No politician of any denomination is likely to start something he or she is unable to finish and thus let someone else (party colleague or new Government) take the credit. They’d sooner keep paying the price of decisions not grounded in evidence.