Comparing like with like
In this sub-section
- Comparisons are key (this page)
- Treatments with dramatic effects
- Treatments with moderate but important effects
Comparisons are key
Comparisons are key to all fair tests of treatments. Clinicians and patients sometimes compare in their minds the relative merits of two treatments. For example, they may form an impression that they or others are responding differently to a treatment compared with responses to previous treatments. Sometimes the comparisons are made more formally. As early as the ninth-century, the Persian physician al-Razi compared the outcome of patients with meningitis treated with blood-letting with the outcome of those treated without it to see if blood-letting could help.
Treatments are usually tested by comparing groups of patients who have received different treatments. If treatment comparisons are to be fair, the comparisons must ensure that like will be compared with like: that the only systematic difference between the groups of patients is the treatments they have received.
This insight is not new. For example, before beginning his comparison of six treatments for scurvy on board HMS Salisbury in 1747, James Lind (i) took care to select patients who were at a similar stage of this often lethal disease; (ii) ensured that the patients had the same basic diet; and (iii) arranged for them to be accommodated in similar conditions. Lind recognized that factors other than the treatments themselves might influence his patients’ chances of recovery.
One way to make a test unfair would have been to give one of the treatments recommended for scurvy – say, sulphuric acid, which was being recommended by the Royal College of Physicians of London – to patients who were less ill to begin with and in the early stages of the disease, and another treatment – say, citrus fruits, which were being recommended by some sailors – to patients who were already approaching death. This would have made sulphuric acid appear to be better, even though it was actually worse.
Biases such as these can arise unless care is taken to ensure that like is being compared with like in all relevant respects.