ThinkBase

Natural Experiments

Joe Shreeve

19 September 2014

One aspect of the Olympic legacy remains largely undiscussed in the media: the benefit of the Games to the research base. Studies abound - from analyses of the impact on perceptions of disability to surveys investigating the ‘Games Makers’ who volunteered at sporting events (a meta-evaluation is available here). One of the most exciting of these studies, however, is still underway: ENABLE London.

The ENABLE (or ‘Examining Neighbourhoods Activities in Built Living Environments’) study aims to uncover ‘how the environment in which we live affects our health, and how to build housing to reflect this’. This is no mean feat, as is evident from the relative dearth research in the area. The study takes a sample of the population applying to move into the Olympic Village in Stratford - a social and private housing complex with abundant opportunities for exercise - and examines whether the health of these participants is improved by moving.

The ENABLE study relies on a ‘natural experiment’ in that the exposure of participants to different conditions (living or not living in the Olympic Village) is naturally occurring event, beyond the control of experimenters. While other studies have relied on natural experiments to investigate the effect of the built environment on health, ENABLE is unique in the timescale of the intervention.  Previous studies have taken place over a matter of decades, making it difficult to untangle the effect of moving home from general changes in the population. The Olympic Village was created and inhabited within a matter of years, by contrast, making it much more likely that any observed changes will be due to moving per se.

Natural experiments are long established in economics and education. They are at the heart, for instance, of efforts to examine the relationship between class size and educational achievement. The ENABLE study highlights how they can be applied to other areas of public policy where they are relatively uncommon. Devolution has given rise to huge possibilities for studying variation in public policy across the UK, for example; these natural policy experiments are likely to be further expanded due to the additional powers now promised to Scotland.

They need to be employed carefully, however. Earlier in the history of ThinkBase, Sara Fakhro examined the use of Randomised Controlled Trials (RCTs).  Although RCTs are not without their problems, natural experiments should be avoided where RCTs can be used, the latter offering the researcher more control. The main strength of natural experiments is shedding light on research questions would be otherwise be difficult to investigate for practical or ethical reasons.

The Medical Research Council has published guidelines on the subject, which will be useful for researchers from other disciplines.  Central to the recommendations are methods for minimising - or at least making transparent - the effect of confounding factors determining which participants experience a given intervention.

The results of the ENABLE study are likely to become apparent by 2017, and it is hoped that they will inform future housing developments.