On April 21, 2014, the Washington Post ran an article with the headline: “Thunder god vine works as well as a Western drug for rheumatoid arthritis, a study finds.” The Western drug in question is methotrexate.

CreakyJoints posted a link to the article on our Facebook page and within minutes we received a number of responses, many asking “Where can I buy this stuff?” That’s Raining on the Thunder God Vine Paradewhen we decided to sprinkle a little rain on the thunder god vine parade.

We’re as intrigued as anyone about results from the recent study conducted in China that show potential for Tripterygium wilfordii Hook F—more lyrically known as thunder god vine—to treat rheumatoid arthritis. (And, honestly, how cool would it be for a doctor to prescribe something called “thunder god vine”?) But as far as we’re concerned that thunder is still rumbling in the far distance.

Our data scientists took a closer look at the study and answered some questions you might have if you read beyond the headlines:

What does “works as well as” actually mean?

The thunder god vine and the methotrexate yielded similar results. The study did not say thunder god vine was more effective than methotrexate, it said thunder god vine was “not inferior to” methotrexate. Sounds like a tie.

Did another course of treatment work better than thunder god vine?

Yes, the study found that a combination of thunder god vine and methotrexate was more effective than either medication alone. The reason for this, and whether it’s safe to combine the two medications over the long term, requires additional research.

Was the study an objective comparison of methotrexate versus thunder god vine?

Under “Caveats” in the Washington Post article writer Linda Searing notes, “Participants were given doses of methotrexate that are standard for China, where the study was done, but that are lower than doses usually prescribed by U.S. doctors.” From this information one might conclude that methotrexate was at a disadvantage in this study.

How was the performance of the medications evaluated?

A total of 174 people completed the study. Data was collected from surveys that asked participants to assess their own improvement (or lack of it) over the course of the study. “In an ‘open label’ study like this one, where both researcher providing the drugs and test-subjects were aware of which treatment they were receiving, the results may be influenced by the placebo effect,” says CreakyJoints Data Scientist Noam Gerber, MPH. In other words, people who believed the vine would work might have felt more improvement—or at least said they did.

Are there better ways to assess medications?

A double-blind, placebo controlled study provides a more objective assessment. “Double blind” means that neither the study participant nor the researcher knows which patient is receiving which medication. “Placebo controlled” means that some participants receive a “sugar pill” instead of medication. It is, to quote Baylor College of Medicine, the ‘gold standard’ of clinical research studies. This ‘blindness’ ensures that the personal beliefs and expectations of either the researchers or the study subjects do not undermine the objectivity of the results.

Have there been studies of thunder god vine in the United States?

Most studies of thunder god vine have been conducted in China. Of the few studies conducted in the U.S., the one cited most frequently was led by Raphaela Goldbach-Mansky, M.D., at the National Institutes of Health and was published in 2009. It compared sulfasalazine and thunder god vine. The study was limited by its high drop-out rate; only about half of the patients who began treatment completed the full 24-week trial.

What risks have been associated with thunder god vine?

A study on the potential use of thunder god vine to treat certain forms of kidney disease noted that common adverse effects include nausea, vomiting, loss of appetite, decreased white blood cell count, and liver function impairment.

More risks, according to the National Center for Complementary and Alternative Medicine (NCCAM), include: headaches, hair loss, and skin rashes. Prolonged use (over the course of five years or more) can lead to bone density loss.

Hasn’t thunder god vine been found to mess with fertility?

Because thunder god vine is known to affect menstrual cycles, it is recommended for women only if they are post-menopausal or do not wish to become pregnant. Thunder god vine has also been shown to decrease male fertility and alter the composition of sperm. People with a desire to have children were not allowed to participate in this recent study.

What benefits does thunder god vine have over methotrexate?

Other than offering the possibility of an alternative that is “not inferior to” methotrexate, it’s not clear that thunder god vine has superior benefits for arthritis patients.

Aren’t herbal remedies safer than chemical medications?

Just because something comes out of the ground rather than out of a lab doesn’t make it intrinsically safe or healthy. NCCAM says, “Thunder god vine can cause severe side effects and can be poisonous if it is not carefully extracted from the skinned root. Other parts of the plant—including the leaves, flowers, and skin of the root—are highly poisonous and can cause death.”

What if I still want to try it?

“I would strongly discourage individuals with arthritis from making changes in their treatment based on the results of this or any other study without first discussing the implications with their rheumatologist,” noted Ben Nowell, Ph.D., CreakyJoints Director of Patient-Centered Research.

Keep in mind that the effect of thunder god vine’s interaction with other medications has not been thoroughly studied.

Where can I buy this stuff?

NCCAM says, “There are no consistent, high-quality thunder god vine products being manufactured in the United States. Preparations of thunder god vine made outside the United States (for example, in China) can sometimes be obtained, but it is not possible to verify whether they are safe and effective.”

We know that the recent reports have sent plenty of people a’Googling in search of thunder god vine. We urge you to proceed with caution. Educate yourself. Talk with your rheumatologist. Don’t let the sunny headlines cloud your judgment.