Association or causation? A need for transparency of process

October 20, 2016 at 11:46 am | Posted in Feature Articles | Leave a comment

window-w900-marco-zak-flickr

Earlier this month, the Royal Society of Medicine hosted a conference marking 50 years (technically, 51) since the British epidemiologist Sir Austin Bradford Hill published his nine “viewpoints” to be taken into consideration when deciding whether or not an observed association between an environmental agent and a disease outcome is a causal relationship rather than a spurious one.

The viewpoints are very much alive today, often cited and used in the very same format (including the archaic language) as when they first appeared half a century ago. This may or may not be a good thing: either, they are so fundamental that their influence speaks for itself; or they have fossilised scientists’ thinking about strength of evidence when aggregating existing research, by encouraging rote thinking in place of methodological advancement.

One of the problems with the criteria is that, while they show what people are considering when they appraise the strength of a body of evidence, they don’t necessarily encourage consistent or valid interpretation of the evidence, so two different reviewers can look at the same evidence and come to very different conclusions about what it means, without it being clear why this might be the case.

One approach to addressing this problem, conceived as an effort to advance the Bradford Hill viewpoints, is the GRADE methodology for appraising the strength of a body of evidence. GRADE attempts to modernise Bradford Hill’s approach by encouraging systematic appraisal of each viewpoint and guiding the user towards their valid interpretation. It also introduces concepts such as risk of bias which have only been explicitly recognised as being of importance in the last 25 years.

GRADE is not to everyone’s taste but it is finding increasing use in the environmental health literature, including in a newly-published systematic review of whether air pollution is associated with autism spectrum disorder (the study finds “limited evidence” of an association because, while the combined results of the included studies suggest an association, limitations in the evidence base mean that the researchers believe the real effect size could be lower or higher than the observed effect, and possibly zero).

The point is that, while everyone is concerned that when researchers aggregate evidence they may (deliberately or otherwise) be gaming the process to yield desired answers rather than true ones, we need more than Bradford Hill’s viewpoints to ensure this is not happening. When evidence is appraised there is always a process, and if that process is not transparent it cannot be audited; while GRADE may not be the answer, we do need to recognise that the scientific method demands transparency and reproducibility, be it for primary or secondary research.

Leave a Comment »

RSS feed for comments on this post. TrackBack URI

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Blog at WordPress.com.
Entries and comments feeds.

%d bloggers like this: