PEPs - Clinical Quality and Evidence

Module 2 - Evidence

What is evidence?

“At the core of all well-founded belief lies belief that is unfounded.” - Ludwig Wittgenstein

Image by talak on Flickr

“Remember, Marge, when I learned how to drive, I had to forget how to wash up.” - Homer Simpson

Image by besos y flores on Flickr

What is truth? This is something that exercises many professions, including philosophers, murder investigators and journalists. It exercises doctors too: we need to know if it is true that our interventions are helping people.

If all patients with disease X died before drug Y came along, but now some survive on it, or if some patients with X died before drug Y became available, but now none die while taking it, it seems reasonable to conclude that drug Y is helping patients. This is known as the “all or none” principle. But things are rarely that simple. We may be looking at small differences in effects, or ones spread over a long time period, or patients with X may forget to take their tablets, or may be taking other drugs that interact with Y. Or there may be something else entirely that is causing patients with X to die or get better.

What we are trying to establish is causality- do our interventions cause patients to get better?  Or does exposure to A cause disease B? The British statistician, Austin Bradford Hill, proposed nine criteria to determine causality - see below. Once it is broken down to a series of questions, it is possible to see how we can collect evidence to try to prove or disprove causality.

Evidence Supporting Causality

1. Strength: The larger the association, the more likely that it is causal.

2. Consistency: Consistent findings observed by different persons in different places with different samples strengthens the likelihood of an effect.

3. Specificity: The more specific an association between a factor and an effect is, the bigger the probability of a causal relationship.

4. Temporality: The effect has to occur after the cause

5. Biological gradient: Greater exposure should generally lead to greater incidence of the effect. However, in some cases, the mere presence of the factor can trigger the effect.

6. Plausibility: A plausible mechanism between cause and effect is helpful

7. Coherence: Coherence between epidemiological and laboratory findings increases the likelihood of an effect.

8. Experiment: "Occasionally it is possible to appeal to experimental evidence".

9. Analogy: The effect of similar factors may be considered.

List adapted from: Bradford Hill, A. (1965) The Environment and Disease: Association or Causation? Proceedings of the Royal Society of Medicine, 58, 295-300.

The tradition of collecting evidence goes back further than you might think - see list below. However, it was only after the discipline of epidemiology began to emerge at the end of the nineteenth century that doctors started to collect evidence on a systematic basis.

Early examples of Evidence-Based Medicine

  • a reference, in Arabic, to an untreated comparison group when discussing the validity of a treatment for early meningitis in tenth-century Baghdad
  • a comparison, reported in French, of different ways of administering mercury for preventing venereal disease in eighteenth-century Geneva
  • the use of placebos in a trial of homeopathy in St Petersburg in 1832, reported in Russian

Examples taken from: The James Lind Library.

DOEs and POEMs

Evidence can be classified into Disease orientated evidence (DOE), and Patient Orientated Medical Evidence that Matters (POEMS). DOEs are based on proxy indicators that are presumed to be linked to outcomes, whereas POEMs are based on outcomes that matter to patients.

Do POEMs matter? Yes they do. For example, for many years, it was thought that aggressive blood sugar control was the best way to manage diabetes. After all, poor glycaemic control was linked to heart disease, and research showed that many drugs could lower the HbA1c. Then the UK prospective Diabetes Study (UKPDS) showed that tight glycaemic control did not lead to fewer myocardial infarctions or fewer deaths. Since then research has shown that there are worse outcomes with very tight glycaemic control, and we have changed the way we manage our diabetic patients.