The plant Ginkgo biloba has been used in traditional medicine for thousands of years. Doctor and medical researcher Pierre Le Bars and his colleagues¹ wanted to figure out whether Ginkgo could be safely used to treat symptoms of dementia caused by Alzheimer disease or multiple strokes. Here’s the ​​test they designed to help answer their question:

• Comparing outcomes. Le Bars and his colleagues set up an ​​experiment in which one group of dementia patients received Ginkgo extract (the ​​experimental group), and another (the ​​control group) did not. The researchers then compared dementia symptoms in the two groups over the next 52 weeks of the treatment.

• Controlling variables. To try to rule out other differences that might influence dementia symptoms, only dementia patients without other serious medical problems were asked to participate in the study. The researchers also assessed all participants’ dementia symptoms in the same way and at around the same time, and they ensured that everyone in the experimental group received the same dose of Ginkgo extract. The study aimed to have the only consistent difference between the two groups be whether or not the patients were receiving Ginkgo extract — but in much medical research, this presents a problem. If patients know that they are receiving a special treatment, this knowledge alone can affect their symptoms, behavior, and possibly even the course of a disease. To get around this issue, Le Bars and his colleagues used a ​​placebo — in this case, pills containing inactive ingredients. The control group received the placebo and the experimental group received the Ginkgo extract — so that all participants would have the same experience of receiving a treatment and none would know if they were receiving the Ginkgo.
• Avoiding bias. Bias can sneak into medical studies in many ways. A doctor or medical researcher who knows that a patient is receiving an experimental drug may treat or assess that patient differently without even realizing it. To avoid these biases, Le Bars used a common approach: a ​​double-blind experimental design. This means that the patients don’t know if they are receiving the experimental treatment — here accomplished by providing a placebo to one group of patients — and that the researchers don’t know which patients have received the experimental treatment until after the study is over and the ​​data have been collected. In a double-blind test, both the researchers and the participants are “blind” to the experimental treatment. Disorders like dementia add another challenge because their symptoms are difficult to evaluate ​​objectively. How does one compare patients with symptoms that range from mood swings, to language problems, to memory loss? If researchers simply noted their impressions of symptom severity, it would be easy for bias to slip in. To increase the fairness of the assessment, Le Bars and his colleagues measured patients’ symptoms using several predetermined scales — standardized tests of dementia that rely on tasks like seeing how many words on a list of ten a patient can recall. These standardized ways of measuring symptoms combined with a double-blind study design helped reduce the influence of bias in the experiment.
• Distinguishing chance from real differences. To help ensure that the differences between the experimental and control groups were really due to the Ginkgo treatment and not random factors, the study enrolled a large number of participants (309) and used statistical tests to gauge the importance of the difference between the groups.

At the end of it all, the Le Bars experiment suggested that:

1. Ginkgo extract is safe. Those in the experimental group were no more likely to have adverse side effects or new health problems than those in the control group.
2. Ginkgo extract can help improve dementia symptoms in those with Alzheimer disease or existing dementia. Those in the experimental group showed fewer dementia-related symptoms than those in the control group. Since the only known, consistent difference between the groups was whether or not patients received the Ginkgo, the researchers concluded that the improved symptoms were caused by the extract.

Of course, research on this topic has continued. More recent studies have cast some doubt on these conclusions. And additional research on a slightly different topic — whether taking ginkgo supplements can prevent the onset of dementia or Alzheimer’s — has found no benefit of the extract. How can this be? The outcomes of scientific studies depend critically on the details of the way the tests are designed, but they can also be affected by random factors that cannot be controlled. As we gather additional ​​evidence, science homes in on the most accurate ​​hypotheses and conclusions.

Though scientists formally testing their hypotheses and ​​theories often have complex equipment and technical resources at their disposal, the logic that goes into designing a fair test is the same — whether the test is aimed at cookie making or treating disease. The same considerations even come into play in ​​observational studies that involve no experimental manipulation at all.