Self-directed learning

11 – Getting the right research done is everybody’s business

In this Chapter: Introduction (this page) How can patients and the public help to improve research? Involving patients in research […]

| 0 Comments

3 – More is not necessarily better

In this Chapter: Introduction (this page) Intensive treatments for breast cancer Mutilating surgery Bone marrow transplantation Dare to think about […]

| 0 Comments

10 – Research – good, bad and unnecessary

In this Chapter: Introduction (this page) Good research Stroke Pre-eclampsia in pregnant women HIV infection in children Bad research Psychiatric […]

| 0 Comments

9 – Regulating tests of treatments: help or hindrance?

In this Chapter Introduction (this page) Do regulatory systems for testing treatments get it right? Information and consent What regulatory […]

| 0 Comments

8 – Assessing all the relevant, reliable evidence

In this Chapter: Introduction (this page) Is one study ever enough? Systematic reviews of all the relevant, reliable evidence Reducing […]

| 0 Comments

7 – Taking account of the play of chance

In this Chapter: Introduction (this page) Assessing the role that chance may have played in fair tests What does a […]

| 0 Comments

6 – Fair Tests of Treatments

In this Chapter: Why are fair tests of treatments needed? The beneficial effects of optimism and wishful thinking The need […]

| 0 Comments

5 – Dealing with uncertainty about the effects of treatments

In this Chapter: Introduction (this page) Dramatic treatment effects: rare and readily recognizable Laser treatment of portwine stains Imatinib for […]

| 0 Comments

4 – Earlier is not necessarily better

In this Chapter: Introduction (this page) Lessons from neuroblastoma screening Weighing benefits and harms Phenylketonuria screening: clearly beneficial Abdominal aortic […]

| 0 Comments

2 – Hoped-for effects that don’t materialize

In this Chapter: Introduction (this page) Advice on babies’ sleeping position Drugs to correct heart rhythm abnormalities in patients having […]

| 0 Comments

13 – Research for the right reasons: blueprint for a better future

In this Chapter: Introduction (this page) Ask the right research questions Design and conduct research properly Publish all the results […]

| 0 Comments

12 – So what makes for better healthcare?

In this Chapter: Introduction (this page) What might the ideas in this website look like for you? Shared decision making: […]

| 0 Comments

Linguistic strategies for improving informed consent in clinical trials among low health literacy patients

Evidence-based guidance on how to improve informed consent processes for patients being invited to participate in clinical research.

| 0 Comments | Evaluated

Informed Health Choices Podcasts

Each episode includes a short story with an example of a treatment claim and a simple explanation of a Key Concept used to assess that claim

| 1 Comment | Evaluated

Know Your Chances

This book has been shown in two randomized trials to improve peoples' understanding of risk in the context of health care choices.

| 0 Comments | Evaluated

You Can’t Trust What you read about nutrition

Beware of misleading correlations between foods and chance associations with other factors.

| 0 Comments

Association is not the same as causation. Let’s say that again: association is not the same as causation!

This article explains how to tell when correlation or association has been confused with causation.

| 0 Comments

Using research evidence: a practice guide

NESTA’s guide to using research evidence to inform decisions in policy and practice.

| 0 Comments

Policy: twenty tips for interpreting scientific claims

This list will help non-scientists to interrogate advisers and to grasp the limitations of evidence.

| 0 Comments

Understanding Health Research: evidence-based medicine, practice and policy

Evidence-based medicine, practice and policy are terms used to describe making decisions using scientific evidence.

| 0 Comments

Understanding Health Research, a tool for making sense of health studies: use of statistics

In health research, researchers typically use statistics to determine statistical significance and effect size.

| 0 Comments

Understanding Health Research, a tool for making sense of health studies: Confounders

A confounder (or 'confounding factor') is something, other than the thing being studied, that could be causing the results seen in a study.

| 0 Comments

Understanding Health Research: are some types of evidence better than others?

Understanding Health Research, a tool for making sense of health studies: are some types of evidence better than others?

| 0 Comments

Understanding Health Research: how science media stories work

Understanding Health Research, a tool for making sense of health studies: how science media stories work.

| 0 Comments

Understanding Health Research: Correlation and Causation

A discussion of the difference between correlation and causation.

| 0 Comments

Ice bucket challenge “breakthrough”? Experts pour cold water on superficial reporting

Beware claims of treatment breakthrough. They’re probably not.

| 0 Comments

Do the statistics back up the claim?

‘Ask for Evidence’ introduction to the interpretation and assessment of statistics.

| 0 Comments

Evidence-based medicine

The European Patients’ Academy web-based introductory course on Evidence-Based Medicine.

| 0 Comments

Who funded the study?

‘Ask for Evidence’ warning about the way that vested interests can distort research.

| 0 Comments

Anecdotes, testimonials and personal studies

‘Ask for Evidence’ warning that anecdotes are not a trustworthy basis for inferring treatment effects.

| 0 Comments

Common pitfalls with studies and things to look out for

‘Ask for Evidence’ introduction to the need for critical appraisal of research studies.

| 0 Comments

Randomised controlled trials (RCTs)

‘Ask for Evidence’ introduction to the concept of a randomised comparison.

| 0 Comments

Studies observing people

Ask for Evidence’ explanation of ‘observational’ studies.

| 0 Comments

Animal Studies

‘Ask for Evidence’ information about the relevance and limitations of animal studies for promoting human health.

| 0 Comments

‘In vitro’ (e.g. test tube) studies

‘Ask for Evidence’ explanation of the term ‘in vitro’ research.

| 0 Comments

Randomisation explained in 1 minute

A 1 minute animation produced by Cancer Research UK, explaining the term ‘randomised trial’.

| 0 Comments

Evidence-Based and Shared-Informed Decision-Making According to Homer (Simpson)

With help from Homer Simpson, James McCormack uses a 17-minute slide cast to explain the principles of thoughtful treatment.

| 0 Comments

Evidence for everyday health choices

A 17-min slide cast by Lynda Ware, on the history of EBM, what Cochrane is, and how to understand the real evidence behind the headlines.

| 0 Comments

Sunn Skepsis

Denne portalen er ment å gi deg som pasient råd om kvalitetskriterier for helseinformasjon og tilgang til forskningsbasert informasjon.

| 0 Comments

Dancing statistics: Explaining variance

A 5-minute film demonstrating the statistical concept of variance through dance.

| 0 Comments

Dancing statistics: sampling & standard error

A 5-minute film demonstrating the statistical concept of sampling and standard error through dance.

| 0 Comments

Julia Belluz – Lessons from the trenches of evidence-based health journalism at Vox.com

20-minute talk by Julia Belluz on the need to bring the cultures of health journalism and EBM together.

| 0 Comments

Don’t jump to conclusions, #Ask for Evidence

An introduction to the ‘Ask for Evidence’ initiative launched by ‘Sense about Science’ in 2016.

| 0 Comments

Battling bad science

15-min talk by Ben Goldacre on ways evidence can be distorted.

| 0 Comments

The surrogate battle – is lower always better?

James McCormick recruits a furious Fuhrer to point out that taking drugs to lower surrogate measures of ill health is a confidence trick.

| 0 Comments

Tom Hanks and Type 2 Diabetes

A 50-minute illustrated talk by James McCormack prompted by Tom Hanks’ announcement that he had been diagnosed with Type 2 diabetes.

| 0 Comments

Bohemian Polypharmacy

James McCormack recruits help from Queen to warn of the dangers of ‘Bohemian Polypharmacy’ in music.

| 0 Comments

Choosing Wisely

James McCormack using song and dance to warn about the negative effects of overtreatment.

| 0 Comments

Like a bridge overdiagnosis

James McCormack with another of his brilliant parodies, warning about the dangers of becoming inappropriately labelled as ill.

| 0 Comments

Reporting the findings: Absolute vs relative risk

Absolute Differences between the effects of two treatments matter more to most people than Relative Differences.

| 0 Comments

Explaining the mission of the AllTrials Campaign (TED talk)

Half the clinical trials of medicines we use haven’t been published. Síle Lane shows how the AllTrials Campaign is addressing this scandal.

| 0 Comments

Fish oil in the Observer: the return of a $2bn friend

Ben Goldacre draws attention to people’s wish to believe that a pill can be the solution to a complicated problem.

| 0 Comments

Building evidence into education

Ben Goldacre explains why appropriate infrastructure is need to do clinical trials of sufficient rigour and size to yield reliable results.

| 0 Comments

Anecdotes are great – if they convey data accurately

Ben Goldacre gives examples of how conclusions based on anecdotes and biased research can be damagingly misleading.

| 0 Comments

Studies of studies show that we get things wrong

Ben Goldacre gives examples of how conclusions based on anecdotes and biased research can be damagingly misleading.

| 0 Comments

Dodgy academic PR

Ben Goldacre: 58% of all press releases by academic institutions lacked relevant cautions and caveats about the methods and results reported

| 0 Comments

All bow before the mighty power of the nocebo effect

Ben Goldacre discusses nocebo effects, through which unpleasant symptoms are induced by negative expectations, despite no physical cause.

| 0 Comments

How do you regulate Wu?

Ben Goldacre finds that students of Chinese medicine are taught (on a science degree) that the spleen is “the root of post-heaven essence”.

| 0 Comments

Science is about embracing your knockers

Ben Goldacre: “I don’t trust claims without evidence, especially not unlikely ones about a magic cream that makes your breasts expand.”

| 0 Comments

NMT are suing Dr Wilmshurst. So how trustworthy are this company? Let’s look at their website…

Ben Goldacre celebrates Peter Wilmshurst, the doctor who blew the whistle on research misconduct in a study to which he was a contributor.

| 0 Comments

Over there! An 8 mile high distraction made of posh chocolate!

Ben Goldcare illustrates strategies used by vested interests to discredit research with ‘inconvenient’ results.

| 0 Comments

Brain imaging studies report more positive findings than their numbers can support. This is fishy.

Ben Goldacre explores how twice as many positive findings as could realistically have been expected from the data reported may have occurred

| 0 Comments

What if academics were as dumb as quacks with statistics?

Ben Goldacre introduces a statistical error that appears in about half of all the published papers in academic neuroscience research.

| 0 Comments

The strange case of the magnetic wine

Ben Goldacre shows how claims for the wine-maturing effects of magnets could be assessed with 50 people in an evening.

| 0 Comments

Screen test

Ben Goldacre notes that even if people realize that screening programmes have downsides, people don’t regret being screened.

| 0 Comments

Sampling error, the unspoken issue behind small number changes in the news

Ben Goldacre stresses the importance of taking account of “sampling variability” and confidence intervals.

| 0 Comments

The certainty of chance

Ben Goldacre reminds readers how associations may simply reflect the play of chance, and describes Deming’s illustration of this.

| 0 Comments

Publish or be damned

Ben Goldacre points out the indefensible practice of announcing conclusions from research studies which haven’t been published.

| 0 Comments

How myths are made

Ben Goldacre draws attention to Steven Greenberg’s forensically based illustration of citation biases.

| 0 Comments

Foreign substances in your precious bodily fluids

Ben Goldacre points out that there is no evidence giving strong support either to water fluoridationists or to anti-fluoridationists.

| 0 Comments

Is it okay to ignore results from people you don’t trust?

Ben Goldacre: why it’s important to consider vested interests when judging research, but not to dismiss research by people you don’t like.

| 0 Comments

Cherry picking is bad. At least warn us when you do it.

Ben Goldacre illustrates how biased ‘cherry picking’ and choosing from the relevant evidence can result in unreliable conclusions.

| 0 Comments

Why won’t Professor Susan Greenfield publish this theory in a scientific journal?

Ben Goldacre challenges senior Oxford professor to publish the evidence supporting her claim that computer games cause dementia in children.

| 0 Comments

Weasels Are on the Loose

Weaseling is the use of certain words to weaken a claim, so that the author can say something without actually saying it and avoid criticism

| 0 Comments

Introduction to Evidence-Based Medicine

Bill Caley’s 26 slides with notes used as an ‘Introduction to Evidence-Based Medicine’.

| 0 Comments

The power of the placebo effect

Emma Bryce’s video presents information about placebo effects: treatments not supposed to have an effect but which make people feel better.

| 0 Comments

Not all scientific studies are created equally

David Schwartz dissects two types of studies that scientists use, illuminating why you should always approach claims with a critical eye.

| 1 Comment

Taking account of the play of chance

Differences in outcome events in treatment comparisons may reflect only the play of chance. Increased numbers of events reduces this problem

| 0 Comments

Quantifying uncertainty in treatment comparisons

Small studies in which few outcome events occur are usually not informative and the results are sometimes seriously misleading.

| 0 Comments

Bringing it all together for the benefit of patients and the public

Improving reports of research and up-to-date systematic reviews of reliable studies are essential foundations of effective health care.

| 0 Comments

Tipsheet for reporting on drugs, devices and medical technologies

Questions that will be familiar to reporters covering health and medicine.

| 0 Comments

Tips for understanding Intention-to-Treat analysis

Ignoring non-compliance with assigned treatments leads to biased estimates of treatment effects. ITT analysis reduces these biases.

| 0 Comments

Tips for understanding Absolute vs. Relative Risk

Absolute Differences between the effects of two treatments matter more to most people than Relative Differences.

| 0 Comments

Tips for understanding Non-inferiority Trials

A non-inferiority experiment endeavours to show that a new intervention is ‘not unacceptably worse’ than the comparison intervention.

| 0 Comments

Cyagen is paying for citations

Pharmaceutical company Cyagen offers researchers and other writers $100 or more for citing their products in publications.

| 0 Comments

GenerationR – The importance of involving children and young people in research

3/3, 22-min video at the launch of GenerationR, a network of young people who advise researchers.

| 0 Comments

Generation R – The need to reduce waste in clinical research involving children

1/3, 14-min video at the launch of GenerationR, a network of young people who advise researchers.

| 0 Comments

Making sense of randomized trials

A description of how clinical trials are constructed and analysed to ensure they provide fair comparisons of treatments.

| 0 Comments

Understanding Health Research: Common Sources of Bias

Bias (the conscious or unconscious influencing of a study and its results) can occur in different ways and renders studies less dependable.

| 0 Comments

5 reasons why you might not get the best healthcare

Five reasons why patients may not always get the best care available.

| 0 Comments

Tamiflu: securing access to medical research data

A campaign by researchers has shown that Roche spun the research on Tamiflu to meet their commercial ends.

| 0 Comments

MMR: the facts in the case of Dr Andrew Wakefield

This 15-page cartoon explains the events surrounding the MMR controversy, and provides links to the relevant evidence.

| 5 Comments

Los intervalos de confianza en investigación

¿Para qué sirven los intervalos de confianza en los estudios de investigación?

| 0 Comments

Estadísticamente significativo

¿Qué significa estadísticamente significativo?

| 0 Comments

Toma de Decisiones Compartidas

¿Por qué nosotros, los pacientes, debemos participar en la toma de decisiones médicas importantes?

| 0 Comments

Taller de Tratamientos Mágicos

Taller de Tratamientos Mágicos

| 0 Comments

The need to compare like-with-like in treatment comparisons

Allocation bias results when trials fail to ensure that, apart from the treatments being compared, ‘like will be compared with like'.

| 0 Comments

Why avoiding differences between treatments allocated and treatments received is important

Knowledge of which treatments have been received by which study participants can affect adherence to assigned treatments and result in bias.

| 0 Comments

The need to avoid differences in the way treatment outcomes are assessed

Biased treatment outcome assessment can result if people know which participants have received which treatments.

| 0 Comments

Avoiding biased selection from the available evidence

Systematic reviews are used to identify, evaluate and summarize all the evidence relevant to addressing a particular question.

| 0 Comments

Preparing and maintaining systematic reviews of all the relevant evidence

Unbiased, up-to-date systematic reviews of all the relevant, reliable evidence are needed to inform practice and policy.

| 0 Comments

Dealing with biased reporting of the available evidence

Biased reporting of research occurs when the direction or statistical significance of results influences how research is reported.

| 0 Comments

Using the results of up-to-date systematic reviews of research

Trustworthy evidence from research is necessary, but not sufficient, to improve the quality of health care.

| 0 Comments

Why treatment comparisons must be fair

Fair treatment comparisons avoid biases and reduce the effects of the play of chance.

| 0 Comments

Introduction to JLL Explanatory Essays

Professionals sometimes harm patients by using inadequately evaluated treatments. Research addressing uncertainties can reduce this harm.

| 0 Comments

Avoiding biased treatment comparisons

Biases in tests of treatments are those factors that can lead to conclusions that are systematically different from the truth.

| 0 Comments

Bias introduced after looking at study results

Biases can be introduced when knowledge of the results of studies influences analysis and reporting decisions.

| 0 Comments

Reducing biases in judging unanticipated effects of treatments

As with anticipated effects of treatments, biases and the play of chance must be reduced in assessing suspected unanticipated effects.

| 0 Comments

Recognizing researcher/sponsor biases and fraud

The vested interests of researchers and organizations tend to be reflected in reports of treatment research in which they are involved.

| 0 Comments

Video games and health improvement: a literature review of randomized controlled trials

This is a critical appraisal of a non-systematic review of randomized trials of video games for improving health.

| 0 Comments

Why comparisons must address genuine uncertainties

Too much research is done when there are no genuine uncertainties about treatment effects. This is unethical, unscientific, and wasteful.

| 0 Comments

Surrogate markers may not tell the whole story

A webpage explaining the limitations of using surrogate outcome markers in clinical research.

| 0 Comments

Why treatment comparisons are essential

Formal comparisons are required to assess treatment effects and to take account of the natural course of health problems.

| 0 Comments

Why treatment uncertainties should be addressed

Ignoring uncertainties about the effects of treatments has led to avoidable suffering and deaths.

| 0 Comments

Motivational Deficiency Disorder – a satirical look at disease mongering

Ray Moynihan’s 4-min video on ‘Motivational Deficiency Disorder’, illustrating ‘disease-mongering’.

| 0 Comments

Making Sense of Screening

Screening tests can cause harm. This guide helps you to make sense of claims about screening for health conditions.

| 0 Comments

システマティックレビューとはなにか?

A 3-min video by Jack Nunn and The Cochrane Consumers and Communication group for people unfamiliar with the concept of systematic reviews.

| 0 Comments

Interactive PowerPoint Presentation about Clinical Trials

An interactive Powerpoint presentation for people thinking about participating in a clinical trial or interested in learning about them.

| 0 Comments

Testing Treatments

Testing Treatments is a book to help the public understand why fair tests of treatments are needed, what they are, and how to use them.

| 0 Comments

Annals Graphic Medicine: How screening is portrayed in the media

A cartoon series addressing the theme "Earlier is not necessarily better".

| 0 Comments

Eureka!

Cherry picking the results of people in sub-groups can be misleading.

| 0 Comments

Effectiveness Delusions

Cherry picking the results of people in sub-groups can be misleading.

| 0 Comments

Cecil and those pellets again…

If possible, participants in clinical trials should not know which treatment they are receiving.

| 0 Comments

Catch 22 – clinical trials edition

Fair comparisons of treatments in animals or highly selected groups of people may not be relevant.

| 0 Comments

House Calls Please

Beware reliance on 'experience' without reference to relevant evidence.

| 0 Comments

Composite Outcomes

Fair comparisons of treatments should measure important outcomes and avoid dependence on surrogate outcome measures.

| 0 Comments

Biomarkers unlimited

Fair comparisons of treatments should measure important outcomes and avoid dependence on surrogate outcome measures.

| 0 Comments

Data speaking for Itself

Cartoon about the many different ways of analyzing data

| 0 Comments

Department of Epidemiology

Presenting relative measures of treatment effects without presenting absolute measures can be misleading.

| 0 Comments

Goldilocks

Cartoon and blog about how poorly performed systematic reviews and meta-analyses may misrepresent the truth.

| 0 Comments

House of tottering cards

Poorly performed systematic reviews and meta-analyses may misrepresent the truth.

| 0 Comments

Cherry Picking

Cherry-picking results that only support your own conclusion may mean ignoring important evidence that refutes a treatment claim.

| 0 Comments

Straight ‘A’ student

Cherry picking results can be misleading.

| 0 Comments

Secret life of trials

The results of single comparisons of treatments can be misleading.

| 0 Comments

He said She said

Unsystematic presentation of research results can be misleading.

| 0 Comments

Desert Island Medical Journal

Small studies with few outcome events are usually not informative and can be misleading.

| 0 Comments

Forest Plot Trilogy

Synthesising the results of similar but separate fair comparisons (meta-analysis) may help by yielding statistically more reliable estimates

| 0 Comments

Significant Others

Statistical significance does not always mean meaningful or practical significance.

| 0 Comments

False Precision

The use of p-values to indicate the probability of something occurring by chance may be misleading.

| 0 Comments

Rumor has it

Anecdotes are an unreliable basis for treatment claims.

| 0 Comments

Personal “No Worse”

People with vested interests may use misleading statistics to support claims about the effects of new treatments.

| 0 Comments

Little Ogg

Lack of evidence of effects is not the same as evidence of no effects.

| 0 Comments

Lisa luxuriant hair

If possible, participants in clinical trials should not know which treatment they are receiving.

| 0 Comments

Suzie in despair

Apart from the treatments being compared, the comparison groups need to be similar.

| 0 Comments

World without bias

Overcoming biases is difficult but important. Treatment comparisons must be fair.

| 0 Comments

Evidence, the new Natural

Cherry picking results for reporting can be misleading.

| 0 Comments

Avoid despair about biases

People who choose to ignore biases may do themselves and others harm.

| 0 Comments

Does it work?

People with vested interests may use misleading statistics to support claims about the efects of new treatments.

| 0 Comments

Thousand dollar placebo

People with vested interests may take advantage of peoples' fears or hopes..

| 0 Comments

Promising treatments

'Promising' treatments greatly outnumber actual advances in treatment.

| 0 Comments

Right to remain anxious

Earlier testing is not always better, and can lead to overdiagnosis, overtreatment and anxiety.

| 0 Comments

De-awareness day

Earlier testing is not always better, and can lead to overdiagnosis and overtreatment.

| 0 Comments

Alicia

Earlier testing is not always better, and can lead to overdiagnosis and overtreatment.

| 0 Comments

Peer-Review

Even quality control steps, such as peer-review, can be affected by conflicts of interest.

| 0 Comments

Gertrud

Exaggeration and hopes or fears can lead to unrealistic expectations about treatment effects.

| 0 Comments

Conspiracy

Beware hidden agendas when assessing claims about treatment effects.

| 0 Comments

Fright Night

Even 'experts' can be uninformed.

| 0 Comments

Blackboard Professor

Beware reliance on 'experience' with no reference to relevant evidence.

| 0 Comments

Twilling Sisters

New treatments as well as old treatments can have unwanted effects.

| 0 Comments

Experienced

Beware reliance on 'experience' with no reference to relevant evidence.

| 0 Comments

Soy Lattes

Just because two things are associated, doesn't mean one thing caused the other.

| 0 Comments

Cause and Effect

Just because two things are associated, doesn't mean one thing caused the other.

| 0 Comments

Space-Diving Safety

Small studies with few outcome events are usually not informative and can be misleading.

| 0 Comments

Rhona Feeling Almost Cynical

Anecdotes are an unreliable basis for treatment claims.

| 0 Comments

Significus the Obscure

Cherry picking the results of people in sub-groups can be misleading.

| 0 Comments

Introduction to clinical trials: lay-friendly video

This lay-friendly video introduction to clinical trials was created by the European Communication on Research Awareness Needs Project.

| 3 Comments

Calculating and interpreting absolute and relative change in an unwanted outcome after treatment

Gilbert Welch’s 6-min video explaining how to calculate and interpret absolute and relative change in an unwanted outcome.

| 0 Comments

Mixed Messages about Statistical Significance

A webpage explaining the difference between statistical and practical significance.

| 0 Comments

Testing Treatments Audio Book

The Testing Treatments Audiobook enables visitors to the TTi site to select whichever chapters in the book they would like to listen to.

| 0 Comments

Science fact or fiction? Making sense of cancer stories

A Cancer Research UK blog, explaining how to assess the quality of health claims about cancer.

| 0 Comments

Introduction to Critical Appraisal

30-slide introduction by Jason Curtis, to Critical Appraisal.

| 0 Comments

DISCERN online

A questionnaire providing a valid and reliable way of assessing the quality of written information on treatment choices.

| 0 Comments

7 words (and more) you shouldn’t use in medical news

A webpage explaining that dramatic effects of medical treatments are very rare.

| 0 Comments

Observational Studies – does the language fit the evidence?

A webpage explaining observational studies and their advantage and disadvantages.

| 0 Comments

Means vs. Medians

Keith Bower’s 3-min video explaining how means (averages) and medians can be presented misleadingly.

| 0 Comments

John Ioannidis, the scourge of sloppy science

A 8 min podcast interview with John Ioannidis explaining how research claims can be misleading.

| 0 Comments

Beware conflicts of interest

In this 5-min videoed TED talk, psychologist Dan Ariely explains how conflicts of interest can bias research.

| 0 Comments

Science Weekly Podcast – Ben Goldacre

A 1-hour audio interview with Ben Goldacre discussing misleading claims about research.

| 0 Comments

Randomised Controlled Trials vs. Observational Studies

5-minute video explaining the difference between randomised trials and observational studies.

| 0 Comments

How do you know which healthcare research you can trust?

A detailed guide to study design, with learning objectives, explaining some sources of bias in health studies.

| 0 Comments

Smart Health Choices: making sense of health advice

The Smart Health Choices e-book explains how to make informed health decisions.

| 0 Comments

How to read articles about healthcare

This article 'How to read health news behind the headlines', by Dr Alicia White, explains how to assess health claims in the media.

| 0 Comments

Can measurements show if a treatment works?

An article discussing errors to avoid when testing treatments.

| 0 Comments

More than average confusion about what mean means mean

Cartoon and blog about how averages can hide important variations in effects.

| 0 Comments

The basic principles of Evidence Based Medicine

A webpage explaining the foundations of systematic reviews.

| 0 Comments

Animal and Lab studies

A webpage explaining how results from animal studies may not be transferable to humans.

| 0 Comments

Commercialism

A webpage about commercialism and conflicts of interest in health research.

| 0 Comments

Be careful with composites

Kevin Lomangino's article discusses the limitations of composite outcomes and surrogate markers.

| 0 Comments

Evaluating Online Health Information

A 30 minute e-Learning presentation by Medline Plus explaining how to evaluate health claims found on the internet.

| 0 Comments

Reducing the play of chance using meta-analysis

Combining data from similar studies (meta-analysis) can help to provide statistically more reliable estimates of treatment effects.

| 0 Comments

Absolute versus relative risk – making sense of media stories

This blog by Sarah Williams should help people who want to understand risk in the context of health and medical stories in the press.

| 0 Comments

AllTrials: All Trials Registered | All Results Reported

AllTrials aims to correct the situation in which studies remain unpublished or are published but with selective reporting of outcomes.

| 0 Comments

Routine use of unvalidated therapy is less defensible than careful research to assess the effects of those treatments

It is more difficult to obtain consent to give a treatment in a clinical trial than to give the same treatment for patients in practice.

| 2 Comments

The Gold Standard: What are randomised control trials and why are they important?

A four minute video by the MRC Clinical Trials Unit about the importance of randomised control trials.

| 0 Comments

More is less: an investigation of unnecessary testing

This is a US radio production about unnecessary testing and the associated harm to patients and costs to the health system.

| 0 Comments

Viva la Evidence!

A brilliant song and video by James McCormack explaining the basics of evidence-based medicine.

| 0 Comments

Evidence Based Medicine Matters: Examples of where EBM has benefitted patients

Booklet containing 15 examples submitted by Royal Colleges where Evidence-Based Medicine has benefited clinical practice.

| 0 Comments

Ben Goldacre talks about Bad Pharma on C-SPAN

In this 90 minute videoed lecture, Ben Goldacre talks about key issues raised in his book 'Bad Pharma'.

| 0 Comments

Big data and finding the evidence

“Big data” is large-scale data processing technologies intended to generate insights into performance, behaviour and trends.

| 0 Comments

What does the Cochrane logo tell us?

This video and animated slide presentation prepared by Steven Woloshin shows how the Cochrane logo was developed, and what it tells us.

| 3 Comments

Shared Decision-Making

This resource from the Health Foundation shows how shared decision-making can be made to work in a typical consultation.

| 0 Comments

Ben Goldacre on TED, on drug companies and hidden data.

Ben Goldacre's lecture at TEDglobal, in which he describes how pharmaceutical companies harm patients by failing to report negative outcomes

| 0 Comments

Surgery for the treatment of psychiatric illness: the need to test untested theories

Simon Wessely describes the untested theory of autointoxication, which arose in the 1890s and caused substantial harm to patients.

| 0 Comments

The placebo effect

A video by NHS Choices explaining what the placebo effect is, and describing its role in medical research and the pharmaceutical industry.

| 0 Comments

What did James Lind do in 1747?

A 2 minute Video clip of a BBC documentary recreating James Lind's celebrated experiment to test treatments for scurvy.

| 2 Comments

Manipulating doctors: testimony from an ex-drug rep

In this 10-min video, Gwen Olsen, a former pharmaceutical sales representative, talks about manipulating doctors to sell more drugs.

| 0 Comments

Therapeutic Touch: a schoolgirl shows how to test it

This 5-minute video provides an example of applying scientific method to dodgy treatment claims.

| 3 Comments

Help at last for the Annoyingly Cheerful

This humorous video by the Onion illustrates some of the tactics used to push unnecessary treatments or "sell" sickness.

| 1 Comment

Pre-diabetes screening

Earlier detection is not always better and can lead to overdiagnosis and overtreatment.

| 0 Comments

Connecting researchers with people who want to contribute to research

People in Research connects researchers who want to involve members of the public with members of the public who want to get involved.

| 0 Comments

Why are fair tests of treatments needed?

In this sub-section Nature, the healer (this page) The beneficial effects of optimism and wishful thinking The need to go […]

| 2 Comments

Foreword by Nick Ross

This book is good for our health. It shines light on the mysteries of how life and death decisions are […]

| 0 Comments

Shared decision making: a consultation for a common condition

Doctor: Well, you have moderate osteoarthritis of the knees, which is common as people get older. It’s often referred to […]

| 0 Comments

A new treatment for strawberry birthmarks

Treatments with dramatic effects are occasionally discovered by accident. Take the example of a condition that occurs in infants called […]

| 0 Comments

Mother’s kiss

Low-tech approaches can have dramatic effects too. Young children sometimes place small objects – plastic toys or beads, for example […]

| 0 Comments

Imatinib for chronic myeloid leukaemia

Impressive results have also been seen in patients given imatinib for chronic myeloid leukaemia [4], [5]. Before imatinib was introduced […]

| 0 Comments

Laser treatment of portwine stains

The birthmarks known as portwine stains are caused by permanent and malformed dilated blood vessels in the skin. Commonly occurring […]

| 0 Comments

Breast cancer

The treatment of breast cancer provides another example of professional uncertainty. There is considerable variability in the use of surgery, […]

| 0 Comments

Antibiotics in pre-term labour

Fair tests of treatments with hoped-for beneficial effects, and which are assumed to be harmless, can show that neither is […]

| 0 Comments

Caffeine for breathing problems in premature babies

Large variations in the treatments used for a particular condition provide clear evidence of professional uncertainty about the relative merits […]

| 0 Comments

Genetic tests: sometimes useful, often dodgy

Not so long ago ‘genetic testing’ was more or less confined to generally rare, single-gene disorders – for example, the […]

| 2 Comments

Lung cancer screening: early but not early enough?

Screening may detect disease earlier, but not always early enoughto make a difference (see Figure). Some cancers, for example lung […]

| 0 Comments

Prostate cancer screening: clear harms with uncertain benefits

Prostate cancer is the second most common cancer in men worldwide,14 and broadly falls into two types. Some men have […]

| 0 Comments

Breast cancer screening: well established but remains contentious

Since routine breast screening with mammography is well established in many countries one could well assume that mammographic screening must […]

| 0 Comments

Abdominal aortic aneurysm screening: proceed with care

At the other end of the age spectrum, abdominal aortic aneurysm screening can also be beneficial. The aorta is the […]

| 0 Comments

Phenylketonuria screening: clearly beneficial

Newborn babies are routinely screened for an inherited disease called phenylketonuria (PKU). Babies with PKU are unable to process phenylalanine, […]

| 0 Comments

Foreword by Ben Goldacre

Medicine shouldn’t be about authority, and the most important question anyone can ask on any claim is simple: ‘how do […]

| 3 Comments

Action plan – 10 things you can do

 Identify questions about the effects of treatment that are important to you. Learn to recognize uncertainty; speak up; ask […]

| 3 Comments

Blueprint for a better future

Medical research could be done for the right reasons and could be done and reported well. Taken individually, none of […]

| 0 Comments

Produce unbiased and useful research reports

Even when studies are published, they often omit important elements that enable readers to assess and apply the findings. One […]

| 0 Comments

Publish all the results and make them accessible

Selective reporting of the results of research can lead to serious biases. Some ‘negative’ studies are never published when the […]

| 0 Comments

Design and conduct research properly

Stimulated by surveys revealing the poor quality of many reports of clinical trials, reporting standards have been developed and applied.  […]

| 0 Comments

Ask the right research questions

Sometimes doctors do not know which treatment is likely to be best for their patients because the available options have […]

| 0 Comments

9: How should people avoid being ‘labelled’ with an ‘illness’ and getting unnecessary treatments?

Medicine has made amazing advances: vaccines and antibiotics for preventing and treating infections; joint replacements; cataract surgery; and treatment of […]

| 0 Comments

8: Are there reliable sources of information that can be recommended?

There is no single information source for all diseases and treatments. To apply the principles in this book, readers may […]

| 0 Comments

7: What’s the best way of telling if the evidence (on the web or elsewhere) is reliable?

What should people look out for? Unfortunately there is no completely reliable simple marker for reliable information. If you are […]

| 0 Comments

6: If someone has a condition that is being studied in an ongoing clinical trial, how do they find out about this if their doctor doesn’t know about it?

Fewer than one in 100 people seeing a doctor will be enrolled in a clinical trial. The proportion varies widely […]

| 0 Comments

5: Won’t genetic testing – and ‘personalized medicine’ – mean doctors can work out the specific treatment needed in every individual and make all this unnecessary?

Although the idea of being able to work out the specific treatment needed in every individual is undoubtedly attractive, and […]

| 0 Comments

4: How can someone know that the research evidence applies to them?

All decisions rely on previous experience of some kind – individual or collective. Fair tests of treatments such as randomized […]

| 0 Comments

3: Statistics are confusing – should patients really have to look at the numbers?

The way that numbers are presented can be very daunting – or even downright misleading. But if you really do […]

| 1 Comment

2: Although patients might want to know if a treatment ‘works’, suppose they don’t want all the details?

It is important to strike a balance between information overload and depriving people of enough information to help them make […]

| 0 Comments

1: Isn’t anything worth trying when a patient has a life-threatening condition?

It can be tempting to want to try the latest ‘wonder-drug’, or follow the example of some high-profile celebrity who […]

| 0 Comments

Questions about translating research evidence into practice

In this section: Question 1: Isn’t anything worth trying when a patient has a life-threatening condition? Question 2: Although patients […]

| 1 Comment

What might the ideas in this website look like for you?

Although no two consultations are identical, the guiding principles for how to arrive at the best possible decision, as set […]

| 0 Comments

Mutilating surgery

Until the middle of the 20th century, surgery was the main treatment for breast cancer. This was based on the […]

| 0 Comments

Working collaboratively bodes well for the future

There are numerous ways in which patients and the public can become involved in testing treatments. As we have already […]

| 0 Comments

Bridging the gap between patients and researchers

We drew attention above to problems that can result from patients becoming involved in testing treatments, and ways in which […]

| 0 Comments

Patients’ organizations: independent voices or not?

Another less well known conflict of interest exists in the relationship between patients’ organizations and the pharmaceutical industry. Most patients’ […]

| 0 Comments

How patients can jeopardize fair tests of treatments

In this sub-section: Introduction (this page) Patients’ organizations:  independent voices or not? Bridging the gap between patients and researchers Introduction […]

| 0 Comments

Involving patients in research

How has this involvement of patients in research come about? In a previous section we showed, for example, how the […]

| 2 Comments

How can patients and the public help to improve research?

The formerly closed world of medicine is increasingly opening its doors to admit fresh ideas and former ‘outsiders’, and paternalism […]

| 2 Comments

Who decides what gets studied?

Clearly this situation is unsatisfactory, so how has it come about? One reason is that what gets studied by researchers […]

| 2 Comments

Questions that are important for patients

Researchers in Bristol decided to pose a fundamental question:  ‘To what extent are questions of importance to patients with osteoarthritis […]

| 0 Comments

Distorted research priorities

In this sub-section: Questions that are important for patients Who decides what gets studied? Introduction: For most of the organizations […]

| 0 Comments

Unnecessary research

In this sub-section: Respiratory distress in premature babies Stroke Aprotinin: effect on bleeding during and after surgery Introduction: This sub-section […]

| 0 Comments

Bad research

In this sub-section: Psychiatric disorders Epidural analgesia for women in labour Introduction: This sub-section contains examples of bad research, where […]

| 0 Comments

Good research

In this sub-section Stroke Pre-eclampsia in pregnant women HIV infection in children Introduction: This sub-section contains examples of good research, […]

| 0 Comments

What regulatory systems do not do

Although regulatory systems for research impose onerous requirements on researchers before studies start, there are many things they conspicuously fail […]

| 2 Comments

Information and consent

Requirements relating to provision of information and consent for studies are one of the ways in which the regulatory system […]

| 0 Comments

Do regulatory systems for testing treatments get it right?

Although the level of regulation can be reassuring, current regulatory systems impose very onerous burdens on anyone wishing to study […]

| 0 Comments

Reports of new research should begin and end with systematic reviews

The report of a study [20] to assess the effects of giving steroids to people with acute traumatic brain injury […]

| 0 Comments

Wasted resources in healthcare and research

Failure to do systematic reviews of relevant, reliable research evidence does harm even when it is not harming patients and […]

| 2 Comments

Avoidable harm to people participating in research

Failure to assess all relevant, reliable evidence can also result in avoidable harm to people who participate in research. Researchers […]

| 0 Comments

Avoidable harm to patients

Recommended treatments for heart attacks that had appeared in textbooks published over a period of 30 years were compared with […]

| 0 Comments

What can happen if all the relevant, reliable evidence is not assessed?

In this section Avoidable harm to patients Avoidable harm to people participating in research Wasted resources in healthcare and research […]

| 2 Comments

Recognizing vested interests and spin in systematic reviews

What if the reviewers have other interests that might affect the conduct or interpretation of their review? Perhaps the reviewers […]

| 2 Comments

Reducing the play of chance in systematic reviews

Previously, we explained how the play of chance can be reduced by combining data from similar but separate studies – […]

| 0 Comments

Identifying all the relevant evidence for systematic reviews

Identifying all the relevant evidence for systematic reviews – irrespective of the language or format of the relevant reports – […]

| 0 Comments

Reducing biases in systematic reviews

Just as biases can distort individual tests of treatments and lead to false conclusions, so they can also distort reviews […]

| 0 Comments

Systematic reviews of all the relevant, reliable evidence

Whilst it is easy to state that we should review the results of a particular alongside other relevant, reliable evidence, […]

| 0 Comments

Is one study ever enough?

The simple answer is ‘hardly ever’. Very seldom will one fair treatment comparison yield sufficiently reliable evidence on which to […]

| 1 Comment

Obtaining large enough numbers in fair tests of treatments

Sometimes in tests of treatments it is possible to obtain large enough numbers from research done in one or two […]

| 1 Comment

What does a “significant difference” between treatments mean?

Well, this is a trick question, because ‘significant difference’ can have several meanings. First, it can mean a difference that […]

| 2 Comments

Assessing the role that chance may have played in fair tests

The role of chance can lead us to make two types of mistakes when interpreting the results of fair treatment […]

| 4 Comments

Generating and investigating hunches about unanticipated adverse effects of treatments

Generating hunches about unanticipated effects of treatments Unanticipated effects of treatments, whether bad or good, are often first suspected by […]

| 2 Comments

Fair measurement of treatment outcomes

Although one of the reasons for using sham treatments in treatment comparisons is to help patients and doctors to stick […]

| 2 Comments

Helping people to stick to allocated treatments

Differences between intended and actual treatments during treatment comparisons can happen in other ways that may complicate the interpretation of […]

| 2 Comments

Dealing with departures from allocated treatments

For all the reasons given so far in this section, you will have realized that fair tests of treatments have […]

| 2 Comments

Following up everyone in treatment comparisons

After taking the trouble to assemble comparison groups to ensure that like will be compared with like, it is important […]

| 0 Comments

Ways of using unbiased (random) allocation in treatment comparisons

Random allocation for treatment comparisons can be used in various ways. For example, it can be used to compare different […]

| 0 Comments

Unbiased, prospective allocation to different treatments

In 1854, Thomas Graham Balfour, an army doctor in charge of a military orphanage, showed how treatment groups could be […]

| 0 Comments

Comparing apparently similar groups of patients who happen to have received different treatments in the same time period

Comparing the experiences and outcomes of apparently similar groups of patients who happen to have received different treatments in the […]

| 0 Comments

Comparing patients given treatments today with apparently similar patients given other treatments in the past for the same disease

Researchers sometimes compare patients given treatments today with apparently similar patients given other treatments in the past for the same […]

| 0 Comments

Treatments with moderate but important effects

This section addresses the common situation in which treatments may differ only moderately from each other, but these differences may […]

| 0 Comments

Treatments with dramatic effects

Sometimes patients experience responses to treatments which differ so dramatically from their own past experiences, and from the natural history […]

| 2 Comments

Comparing like with like

In this sub-section Comparisons are key (this page) Treatments with dramatic effects Treatments with moderate but important effects Comparisons are […]

| 2 Comments

So what are fair tests?

Most of us know that it can be a mistake to take a media report of some new medical advance […]

| 0 Comments

The need to go beyond impressions

If patients believe that something helps them, isn’t that enough?  Why is it important to go to the trouble and […]

| 0 Comments

The beneficial effects of optimism and wishful thinking

The psychological reasons for people attributing any improvement in their condition to the treatment they received are now better understood. […]

| 2 Comments

Providing treatment as part of a fair test

So what should happen when there is important about the effects of new or old treatments that have not been […]

| 0 Comments

Addressing uncertainties about the effects of treatments

Where do we go from here? Clinicians need to be able to draw on resources that provide the best current […]

| 0 Comments

When practitioners disagree

In this sub-section Introduction (this page) Caffeine for breathing problems in premature babies Antibiotics in pre-term labour Breast cancer Introduction […]

| 2 Comments

Moderate treatments effects: usual and not so obvious

Most treatments do not have dramatic effects and fair tests are needed to assess them. And sometimes a treatment may […]

| 3 Comments

Dramatic treatment effects: rare and readily recognizable

In this sub-section: Laser treatment of portwine stains Imatinib for chronic myeloid leukaemia Mother’s kiss A new treatment for strawberry […]

| 2 Comments

Is anyone normal?

Whole-body CT scans Among the tests on offer at private clinics are whole-body computed tomography (CT) scans to look at […]

| 0 Comments

What screening aims to achieve and why evidence matters

The examples we have already given show that, before rushing headlong into widespread screening, it is worth pausing a moment […]

| 3 Comments

Weighing benefits and harms

There are many examples of beneficial screening. Perhaps the most widely used in adults is the checking of risk factors […]

| 3 Comments

Lessons from neuroblastoma screening

Experience with screening for neuroblastoma – a rare cancer that mainly occurs in young children – is instructive in several […]

| 2 Comments

Dare to think about doing less

So, more is not always better – and this message remains important. Today, in women with metastatic (widespread) breast cancer, […]

| 1 Comment

Bone marrow transplantation

However, the demise of mutilating surgery did not spell the end of the ‘more is better’ mindset – far from […]

| 0 Comments

Intensive treatments for breast cancer

The therapies advocated for breast cancer – so often in the news – provide some especially valuable lessons about the […]

| 2 Comments

Evening primrose oil for eczema

Even if inadequately assessed treatments do not kill or harm, they can waste money. Eczema is a distressing skin complaint […]

| 0 Comments

Hormone Replacement Therapy (HRT)

In women going through the menopause, hormone replacement therapy (HRT) is very effective in reducing the distressing hot flushes that […]

| 0 Comments

Diethylstilboestrol

At one time, doctors were uncertain whether pregnant women who had previously had miscarriages and stillbirths could be helped by […]

| 0 Comments

Drugs to correct heart rhythm abnormalities in patients having a heart attack

Dr Spock’s advice may have seemed logical, but it was based on untested theory. Other examples of the dangers of […]

| 2 Comments

Advice on babies’ sleeping position

Do not imagine that only can harm – advice can be lethal too. Many people have heard of the American […]

| 0 Comments

Introduction

‘There is no way to know when our observations about complex events in nature are complete. Our knowledge is finite, […]

| 2 Comments

Herceptin

Commercial companies are not alone in trumpeting the advantages of new treatments while down-playing drawbacks. Professional hype and enthusiastic media […]

| 1 Comment

Mechanical heart valves

Drugs are not the only treatments that can have unexpected bad effects: non-drug treatments can pose serious risks too. Mechanical […]

| 0 Comments

Avandia

2010 saw another drug – rosiglitazone, better known by the trade name Avandia – hitting the headlines because of unwanted […]

| 0 Comments

Thalidomide

Thalidomide is an especially chilling example of a new medical treatment that did more harm than good. [1] This sleeping […]

| 0 Comments

Why Testing Treatments was written

The first edition of Testing Treatments, published in 2006, was inspired by a question: ‘How do you ensure that into […]

| 2 Comments