Researching the medical literature for studies on alternative therapies can be a frustrating experience for doctors and patients alike. Although there are thousands of studies and reports, not all published studies are equally authoritative. Here's a brief overview of the different types of reports you may come across, and an explanation of why solid evidence on alternative medicine is so hard to come by.

·  What are the different types of studies? Should all evidence be weighed the same?

• Anecdotal evidence. We are all familiar with this. A friend tried a supplement and lost weight; a relative went to a chiropractor and his back problems went away. Also called a personal testimonial, this kind of evidence is highly persuasive, especially when the person is someone we know. But anecdotal evidence is not sufficient for us to draw conclusions about the supplement or therapy in question - it could be that the person healed by chance, by something else they were doing at the time, or by combining a conventional treatment (i.e. something their doctor gave them) and an alternative one. There are too many variables that prevent us from isolating the effect of the alternative treatment alone.

• Observational studies. Observational studies are just that - researchers observe what happens over time to a group of people who are, for example, taking a weight-loss pill or using a homeopathic remedy. They are referred to as the treatment group, because they are receiving some kind of medical treatment or therapy. But over time, some people get better just by chance or because of something else they are doing not related to the treatment, and some people don't. The problem with observational studies is that there is no way to know why people improved. Was it the pill, or was it because the person started exercising more? Observational studies do not take these factors into account.

• Naturalized experiments. Naturalized experiments go one step further. Along with a treatment group, they have a control group. A control group is a group of people with similar health characteristics to the treatment group, except for the fact that they are not taking the pill or remedy. That way, when you compare the two groups over time, you can attribute differences between them to the treatment itself (the pill or therapy), and not to random chance. But even the results of these studies can be flawed. Because people were not randomly assigned to the treatment group or the control group, there may in fact be significant differences between the two groups that account for at least some of the difference over time. For example, the people in the treatment group, who have all chosen to take the supplement or see a reflexologist, might be more concerned about managing their health than the people in the control group, who did nothing. Therefore, it may be true that the people in the treatment group are more motivated to take care of themselves and improve their condition. In that case, it could be the differing levels of people's motivation that causes differences over time, and not the pill itself.

• Randomized controlled clinical trials. These studies are the gold standard of medical research, because they attempt to control for all of the potential biases that frequently skew the results of observational studies and naturalized experiments. `Randomized' means that people are selected at random to be in the treatment or control group. `Controlled' means that the researchers have done as much as possible to ensure that there are no statistically significant differences between the treatment and control group at the outset of the study that could potentially influence the results. If the study is double-blind, that means neither the patient nor the doctor administering the medicine knows if the person is receiving the real pill or a placebo (an inert pill). This prevents the doctor from inadvertently giving facial or body gesture clues to patients about which pill (the real one or the fake one) the person is taking. Research findings from randomized clinical trials are considered to be the most reputable by the medical community.

·  Why is there such a lack of good evidence?

There are many reasons, but they are simple. Consensus in the medical community about the efficacy of an alternative therapy - or a mainstream one, for that matter - is reached only after many large, well-designed, randomized trials find the same, or similar, results. And to do a randomized trial well costs a lot of money - often in the tens of millions of dollars - requires the recruitment of hundreds if not thousands of volunteers, and takes many years to complete. After funding scores of smaller-scale pilot studies on the efficacy of various alternative therapies during the past ten years, the NIH has started funding much larger clinical trials on everything from acupuncture to St. Johns Wort. But the vast majority of them are still underway.

Another reason is self-interest. Many of the studies currently being conducted are funded by the manufacturers of the herbal supplements, and many of the researchers themselves have a stake or financial interest in the drug under review. Even if researchers are not explicitly told to doctor the results, they are often under pressure to demonstrate the drug or herbal supplement yields a positive effect. If the study comes out saying otherwise, it might be suppressed. When considering the validity of a study's results, always ask: who funded this study? Are they neutral, or do they have a stake in the results? Are there potentials for bias, or conflicts of interest? Did the researchers subject the study to the scrutiny of their peers by publishing it in a peer-review journal, or it is just on a supplement maker's website? Answers to these questions will help you determine how to weigh the various, and often contradictory, evidence you will encounter.