Types of Evidence

We use different types of evidence to determine if a treatment is safe and confers benefits to the brain. Clinical studies, which involve people, are more conclusive than preclinical studies conducted in animal models or test tubes. Each type of evidence serves a distinct role and can provide useful information, though no single study—even a large trial—provides enough evidence to make a definitive conclusion.

We look for evidence of potential benefit to brain health as well as evidence that the way the treatment works (i.e., its target) has a plausible rationale. For example, scientists may have found that a certain genetic mutation increases the risk of developing dementia. So, a prevention treatment targeting that mutation would make sense biologically. The types of evidence most commonly evaluated for our ratings are described below:

Different experiments designed to answer the same question will frequently yield different results. A treatment may have been tested in different populations of people, perhaps for different lengths of time or at different doses. Sometimes the results are biased unknowingly by the researchers themselves. Meta-analyses and systematic reviews methodically compile different research studies (e.g., observational or randomized controlled trials) on a given question and combine them into a single analysis that considers and compares the quality of the methods of each study to draw an overall conclusion.

Clinical trials are human research studies designed to test the safety and efficacy of new drugs or interventions to treat medical conditions. In a typical trial, people are randomly selected (i.e., randomized) to receive either the drug or a placebo (i.e., control). In a double-blind trial, both the participants and the researchers are "blind" to which person received the drug until after the results are analyzed. Double-blind randomized controlled trials are the "gold-standard" of biomedical research, but rare for dementia prevention as they are expensive and can be difficult to conduct. Clinical trial results are sometimes in conflict with results from observational studies. There are many reasons for these differences, including that clinical trials are better suited to prove whether a drug causes a given effect.

In observational studies, scientists observe large groups of people to identify traits and choices that correlate with disease versus health. For example, a study might identify 10,000 women between the ages of 55-60 to answer the question, "Does smoking correlate with a higher risk of dementia over 10 years in this group of people?" These studies can provide cost-effective and detailed information but they must be interpreted with caution. Observational studies are often biased, or "confounded", by factors other than those of interest (e.g., people who smoke tend to exercise less, which in itself is a risk factor for dementia). Due to this, observational studies cannot prove whether a treatment causes or protects from disease. (Randomized controlled trials are used to prove causation.) Observational studies can, however, be used to identify patterns across large and diverse groups exposed to a treatment for long durations of time.

Experiments in animals (in vivo) and isolated cells (in vitro) are useful for "proof-of-concept" experiments and to help design future research. However, they do not reliably predict whether a treatment is safe or effective in humans.